podpac.managers.Lambda

class podpac.managers.Lambda(**kwargs: Any)[source]

Bases: Node

A Node wrapper to evaluate source on AWS Lambda function

aws_access_key_id

Access key id from AWS credentials. If session is provided, this attribute will be ignored. Overrides podpac.settings.

Type:

str, optional

aws_region_name

Name of the AWS region. If session is provided, this attribute will be ignored. Overrides podpac.settings.

Type:

str, optional

aws_secret_access_key

Access key value from AWS credentials. If session is provided, this attribute will be ignored. Overrides podpac.settings.

Type:

str

function_name

Name of the lambda function to use or create. Defaults to podpac.settings["FUNCTION_NAME"] or “podpac-lambda-autogen”.

Type:

str, optional

function_timeout

Timeout of the lambda function, in seconds. Defaults to 600.

Type:

int, optional

function_triggers

Methods to trigger this function. May only include [“eval”, “S3”, “APIGateway”]. During the self.build() process, this list will determine which AWS resources are linked to Lambda function. Defaults to [“eval”].

Type:

list of str, optional

function_role_name

Name of the AWS role created for lambda function. Defaults to podpac.settings["FUNCTION_ROLE_NAME"] or “podpac-lambda-autogen”.

Type:

str, optional

function_s3_bucket

S3 bucket name to use with lambda function. Defaults to podpac.settings["S3_BUCKET_NAME"] or “podpac-autogen-<timestamp>” with the timestamp to ensure uniqueness.

Type:

str, optional

eval_settings

Default is podpac.settings. PODPAC settings that will be used to evaluate the Lambda function.

Type:

dict, optional

eval_timeout

Default is None. The amount of time to wait for an eval to return. To get near asynchronous response, set this to a small number.

Type:

float, optional

Other Attributes
----------------
node_attrs

Additional attributes passed on to the Lambda definition of the base node

Type:

dict

download_result

Flag that indicated whether node should wait to download the data.

Type:

Bool

function_api_description

Description for the AWS API Gateway resource

Type:

str, optional

function_api_endpoint

Endpoint path for API Gateway. Defaults to “eval”.

Type:

str, optional

function_api_name

AWS resource name for the API Gateway. Defaults to self.function_name + “-api”.

Type:

str, optional

function_api_stage

Stage name for the API gateway. Defaults to “prod”.

Type:

str, optional

function_api_tags

AWS Tags for API Gateway resource. Defaults to self.function_tags.

Type:

dict, optional

function_api_version

API Gateway version. Defaults to podpac.verions.semver().

Type:

str, optional

function_description

Description for the AWS Lambda function resource

Type:

str, optional

function_env_variables

Environment variables to use within the lambda function.

Type:

dict, optional

function_eval_trigger

Function trigger to use during node eval process. Must be on of “eval” (default), “S3”, or “APIGateway”.

Type:

str, optional

function_handler

Handler method in Lambda function. Defaults to “handler.handler”.

Type:

str, optional

function_memory

Memory allocated for each Lambda function. Defaults to 2048 MB.

Type:

int, optional

function_restrict_pipelines

List of Node hashes (see podpac.Node.hash). Restricts lambda function evaluation to specific Node definitions.

Type:

list, optional

function_role_assume_policy_document

Assume policy document for role created. Defaults to allowing role to assume Lambda function.

Type:

dict, optional.

function_role_description

Description for the AWS role resource

Type:

str, optional

function_role_policy_arns

Managed role policy ARNs to attach to role.

Type:

list of str, optional

function_role_policy_document

Inline role policies to put in role.

Type:

dict, optional

function_role_tags

AWS Tags for role resource. Defaults to self.function_tags.

Type:

dict, optional

function_s3_dependencies_key

S3 path to copy and reference podpac dependencies. Defaults to “podpac_deps_<semver>.zip”.

Type:

str, optional

function_s3_input

Folder in self.function_s3_bucket to store input pipelines when “S3” is included in self.function_triggers. Defaults to “input”.

Type:

str, optional

function_s3_output

Folder in self.function_s3_bucket to watch for output when “S3” is included in self.function_triggers. Defaults to “output”.

Type:

str, optional

function_s3_tags

AWS Tags for S3 bucket resource. Defaults to self.function_tags.

Type:

dict, optional

function_source_bucket

S3 Bucket to use for released podpac distribution during self.build() process. Defaults to “podpac-dist”. This bucket is managed by the PODPAC distribution team.

Type:

str, optional

function_source_dependencies_key

S3 path within self.function_source_bucket to source podpac dependencies archive during self.build() process. Defaults to “<semver>/podpac_deps.zip”.

Type:

str, optional

function_source_dependencies_zip

Override self.function_source_dependencies_key and upload custom source podpac dependencies archive to self.function_s3_bucket during self.build() process.

Type:

str, optional

function_source_dist_key

S3 path within self.function_source_bucket to source podpac dist archive during self.build() process. Defaults to “<semver>/podpac_dist.zip”.

Type:

str, optional

function_source_dist_zip

Override self.function_source_dist_key and create lambda function using custom source podpac dist archive to self.function_s3_bucket during self.build() process.

Type:

str, optional

function_tags

AWS Tags for Lambda function resource. Defaults to podpac.settings["AWS_TAGS"] or {}.

Type:

dict, optional

function_budget_amount

EXPERIMENTAL FEATURE Monthly budget for function and associated AWS resources. When usage reaches 80% of this amount, AWS will notify function_budget_email. Defaults to podpac.settings["AWS_BUDGET_AMOUNT"].

Type:

float, optional

function_budget_email

EXPERIMENTAL FEATURE Email to notify when usage reaches 80% of function_budget_amount. Defaults to podpac.settings["AWS_BUDGET_EMAIL"].

Type:

str, optional

function_budget_name

EXPERIMENTAL FEATURE Name for AWS budget

Type:

str, optional

function_budget_currency

EXPERIMENTAL FEATURE Currency type for the function_budget_amount. Defaults to “USD”. See https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/budgets.html#Budgets.Client.create_budget for currency (or Unit) options.

Type:

str, optional

output_format

Definition for how output is saved after results are computed.

Type:

dict, optional

session

AWS Session to use for this node.

Type:

podpac.managers.aws.Session

source

Node to be evaluated on the Lambda function.

Type:

podpac.Node

source_output_format

Output format for the evaluated results of source

Type:

str

source_output_name

Output name for the evaluated results of source

Type:

str

Alternative Constructors

from_definition(definition)

Create podpac Node from a dictionary definition.

from_json(s)

Create podpac Node from a JSON definition.

Methods

__init__(**kwargs)

Do not overwrite me

add_trigger(statement_id, principle, source_arn)

Add trigger (permission) to lambda function

build()

Build Lambda function and associated resources on AWS to run PODPAC pipelines

create_api()

Create API Gateway API for lambda function

create_bucket()

Create S3 bucket to work with function

create_budget()

EXPERIMENTAL FEATURE Create budget for lambda function based on node hash.

create_function()

Build Lambda function on AWS

create_output_array(coords[, data, attrs, ...])

Initialize an output data array

create_role()

Create IAM role to execute podpac lambda function

delete([confirm])

Remove all cloud resources associated with function

delete_api()

Delete API Gateway for Function

delete_bucket([delete_objects])

Delete bucket associated with this function

delete_budget()

Delete budget associated with function

delete_function()

Remove AWS Lambda function and associated resources on AWS

delete_role()

Remove role from AWS resources

describe()

Show a description of the Lambda Utilities

eval(coordinates[, output, selector])

Evaluate the source node on the AWS Lambda Function at the given coordinates

eval_group(group)

Evaluate the node for each of the coordinates in the group.

find_coordinates()

Get all available coordinates for the Node.

from_name_params(name[, params])

Create podpac Node from a WMS/WCS request.

from_url(url)

Create podpac Node from a WMS/WCS request.

get_api()

Get API Gateway definition for function

get_bounds([crs])

Get the full available coordinate bounds for the Node.

get_bucket()

Get S3 Bucket for function

get_budget()

EXPERIMENTAL FEATURE Get budget definition for function

get_cache(key[, coordinates])

Get cached data for this node.

get_function()

Get function definition from AWS

get_logs([limit, start, end])

Get Cloudwatch logs from lambda function execution

get_role()

Get role definition from AWS

get_ui_spec([help_as_html])

Get spec of node attributes for building a ui

has_cache(key[, coordinates])

Check for cached data for this node.

init()

Overwrite this method if a node needs to do any additional initialization after the standard initialization.

load(path)

Create podpac Node from file.

probe([lat, lon, time, alt, crs])

Evaluates every part of a node / pipeline at a point and records which nodes are actively being used.

put_cache(data, key[, coordinates, expires, ...])

Cache data for this node.

rem_cache(key[, coordinates, mode])

Clear cached data for this node.

remove_trigger(statement_id)

Remove trigger (permission) from lambda function

remove_triggers()

Remove all triggers from function

save(path)

Write node to file.

trait_defaults(*names, **metadata)

Return a trait's default value or a dictionary of them

trait_has_value(name)

Returns True if the specified trait has a value.

trait_is_defined(name)

trait_values(**metadata)

A dict of trait names and their values.

update_function()

Update lambda function with new parameters

validate([raise_exceptions])

Validate cloud resources and interoperability of resources for PODPAC usage

validate_api()

Validate that API will work with function.

validate_bucket()

Validate that bucket will work with function.

validate_budget()

Validate budget definition for function

validate_function()

Validate that function is configured properly

validate_role()

Validate that role will work with function.

Attributes

attrs

List of node attributes

aws_access_key_id

A trait for unicode strings.

aws_region_name

A trait for unicode strings.

aws_secret_access_key

A trait for unicode strings.

base_ref

Default reference/name in node definitions

cache_ctrl

A trait whose value must be an instance of a specified class.

cache_output

A boolean (True, False) trait.

definition

download_result

A boolean (True, False) trait.

dtype

An enum whose value must be in a given sequence.

eval_settings

An instance of a Python dict.

eval_timeout

A float trait.

force_compute

A boolean (True, False) trait.

force_eval

A boolean (True, False) trait.

function_allow_unsafe_eval

A boolean (True, False) trait.

function_api_description

A trait for unicode strings.

function_api_endpoint

A trait for unicode strings.

function_api_name

A trait for unicode strings.

function_api_stage

A trait for unicode strings.

function_api_tags

An instance of a Python dict.

function_api_version

A trait for unicode strings.

function_budget_amount

A float trait.

function_budget_currency

A trait for unicode strings.

function_budget_email

A trait for unicode strings.

function_budget_name

A trait for unicode strings.

function_description

A trait for unicode strings.

function_env_variables

An instance of a Python dict.

function_eval_trigger

An enum whose value must be in a given sequence.

function_handler

A trait for unicode strings.

function_memory

An int trait.

function_name

A trait for unicode strings.

function_restrict_pipelines

An instance of a Python list.

function_role_assume_policy_document

An instance of a Python dict.

function_role_description

A trait for unicode strings.

function_role_name

A trait for unicode strings.

function_role_policy_arns

An instance of a Python list.

function_role_policy_document

An instance of a Python dict.

function_role_tags

An instance of a Python dict.

function_s3_bucket

A trait for unicode strings.

function_s3_dependencies_key

A trait for unicode strings.

function_s3_input

A trait for unicode strings.

function_s3_output

A trait for unicode strings.

function_s3_tags

An instance of a Python dict.

function_source_bucket

A trait for unicode strings.

function_source_dependencies_key

A trait for unicode strings.

function_source_dependencies_zip

A trait for unicode strings.

function_source_dist_key

A trait for unicode strings.

function_source_dist_zip

A trait for unicode strings.

function_tags

An instance of a Python dict.

function_timeout

An int trait.

function_triggers

An instance of a Python list.

hash

json

Definition for this node in JSON format.

json_pretty

Definition for this node in JSON format, with indentation suitable for display.

node_attrs

An instance of a Python dict.

output

A trait for unicode strings.

output_format

An instance of a Python dict.

outputs

pipeline

The pipeline of this manager is the aggregation of the source node definition and the output.

session

A trait whose value must be an instance of a specified class.

source

A trait whose value must be an instance of a specified class.

source_output_format

A trait for unicode strings.

source_output_name

A trait for unicode strings.

style

A trait whose value must be an instance of a specified class.

units

A trait for unicode strings.

Members:

__init__(**kwargs)

Do not overwrite me

add_trigger(statement_id, principle, source_arn)[source]

Add trigger (permission) to lambda function

Parameters:
  • statement_id (str) – Specific identifier for trigger

  • principle (str) – Principle identifier from AWS

  • source_arn (str) – Source ARN for trigger

aws_access_key_id

A trait for unicode strings.

aws_region_name

A trait for unicode strings.

aws_secret_access_key

A trait for unicode strings.

build()[source]

Build Lambda function and associated resources on AWS to run PODPAC pipelines

create_api()[source]

Create API Gateway API for lambda function

create_bucket()[source]

Create S3 bucket to work with function

create_budget()[source]

EXPERIMENTAL FEATURE Create budget for lambda function based on node hash.

create_function()[source]

Build Lambda function on AWS

create_role()[source]

Create IAM role to execute podpac lambda function

delete(confirm=False)[source]

Remove all cloud resources associated with function

Parameters:

confirm (bool, optional) – Must pass in confirm paramter

delete_api()[source]

Delete API Gateway for Function

delete_bucket(delete_objects=False)[source]

Delete bucket associated with this function

Parameters:

delete_objects (bool, optional) – Delete all objects in the bucket while deleting bucket. Defaults to False.

delete_budget()[source]

Delete budget associated with function

delete_function()[source]

Remove AWS Lambda function and associated resources on AWS

delete_role()[source]

Remove role from AWS resources

See self.function_role_name for role_name

describe()[source]

Show a description of the Lambda Utilities

download_result

A boolean (True, False) trait.

eval(coordinates, output=None, selector=None)[source]

Evaluate the source node on the AWS Lambda Function at the given coordinates

eval_settings

An instance of a Python dict.

One or more traits can be passed to the constructor to validate the keys and/or values of the dict. If you need more detailed validation, you may use a custom validator method.

Changed in version 5.0: Added key_trait for validating dict keys.

Changed in version 5.0: Deprecated ambiguous trait, traits args in favor of value_trait, per_key_traits.

eval_timeout

A float trait.

force_compute

A boolean (True, False) trait.

function_allow_unsafe_eval

A boolean (True, False) trait.

function_api_description

A trait for unicode strings.

function_api_endpoint

A trait for unicode strings.

function_api_name

A trait for unicode strings.

function_api_stage

A trait for unicode strings.

function_api_tags

An instance of a Python dict.

One or more traits can be passed to the constructor to validate the keys and/or values of the dict. If you need more detailed validation, you may use a custom validator method.

Changed in version 5.0: Added key_trait for validating dict keys.

Changed in version 5.0: Deprecated ambiguous trait, traits args in favor of value_trait, per_key_traits.

function_api_version

A trait for unicode strings.

function_budget_amount

A float trait.

function_budget_currency

A trait for unicode strings.

function_budget_email

A trait for unicode strings.

function_budget_name

A trait for unicode strings.

function_description

A trait for unicode strings.

function_env_variables

An instance of a Python dict.

One or more traits can be passed to the constructor to validate the keys and/or values of the dict. If you need more detailed validation, you may use a custom validator method.

Changed in version 5.0: Added key_trait for validating dict keys.

Changed in version 5.0: Deprecated ambiguous trait, traits args in favor of value_trait, per_key_traits.

function_eval_trigger

An enum whose value must be in a given sequence.

function_handler

A trait for unicode strings.

function_memory

An int trait.

function_name

A trait for unicode strings.

function_restrict_pipelines

An instance of a Python list.

function_role_assume_policy_document

An instance of a Python dict.

One or more traits can be passed to the constructor to validate the keys and/or values of the dict. If you need more detailed validation, you may use a custom validator method.

Changed in version 5.0: Added key_trait for validating dict keys.

Changed in version 5.0: Deprecated ambiguous trait, traits args in favor of value_trait, per_key_traits.

function_role_description

A trait for unicode strings.

function_role_name

A trait for unicode strings.

function_role_policy_arns

An instance of a Python list.

function_role_policy_document

An instance of a Python dict.

One or more traits can be passed to the constructor to validate the keys and/or values of the dict. If you need more detailed validation, you may use a custom validator method.

Changed in version 5.0: Added key_trait for validating dict keys.

Changed in version 5.0: Deprecated ambiguous trait, traits args in favor of value_trait, per_key_traits.

function_role_tags

An instance of a Python dict.

One or more traits can be passed to the constructor to validate the keys and/or values of the dict. If you need more detailed validation, you may use a custom validator method.

Changed in version 5.0: Added key_trait for validating dict keys.

Changed in version 5.0: Deprecated ambiguous trait, traits args in favor of value_trait, per_key_traits.

function_s3_bucket

A trait for unicode strings.

function_s3_dependencies_key

A trait for unicode strings.

function_s3_input

A trait for unicode strings.

function_s3_output

A trait for unicode strings.

function_s3_tags

An instance of a Python dict.

One or more traits can be passed to the constructor to validate the keys and/or values of the dict. If you need more detailed validation, you may use a custom validator method.

Changed in version 5.0: Added key_trait for validating dict keys.

Changed in version 5.0: Deprecated ambiguous trait, traits args in favor of value_trait, per_key_traits.

function_source_bucket

A trait for unicode strings.

function_source_dependencies_key

A trait for unicode strings.

function_source_dependencies_zip

A trait for unicode strings.

function_source_dist_key

A trait for unicode strings.

function_source_dist_zip

A trait for unicode strings.

function_tags

An instance of a Python dict.

One or more traits can be passed to the constructor to validate the keys and/or values of the dict. If you need more detailed validation, you may use a custom validator method.

Changed in version 5.0: Added key_trait for validating dict keys.

Changed in version 5.0: Deprecated ambiguous trait, traits args in favor of value_trait, per_key_traits.

function_timeout

An int trait.

function_triggers

An instance of a Python list.

get_api()[source]

Get API Gateway definition for function

Returns:

See podpac.managers.aws.get_api()

Return type:

dict

get_bucket()[source]

Get S3 Bucket for function

Returns:

See podpac.managers.aws.get_bucket()

Return type:

dict

get_budget()[source]

EXPERIMENTAL FEATURE Get budget definition for function

Returns:

See podpac.managers.aws.get_budget()

Return type:

dict

get_function()[source]

Get function definition from AWS

Returns:

See podpac.managers.aws.get_function()

Return type:

dict

get_logs(limit=5, start=None, end=None)[source]

Get Cloudwatch logs from lambda function execution

See podpac.managers.aws.get_logs()

Parameters:
  • limit (int, optional) – Limit logs to the most recent N logs

  • start (str, optional) – Datetime string. Must work as input to np.datetime64 (i.e np.datetime64(start)) Defaults to 1 hour prior to end.

  • end (str, optional) – Datetime string. Must work as input to np.datetime64 (i.e np.datetime64(end)) Defaults to now.

Returns:

list of log events

Return type:

list

get_role()[source]

Get role definition from AWS

See self.function_role_name for role_name

Returns:

See podpac.managers.aws.get_role()

Return type:

dict

node_attrs

An instance of a Python dict.

One or more traits can be passed to the constructor to validate the keys and/or values of the dict. If you need more detailed validation, you may use a custom validator method.

Changed in version 5.0: Added key_trait for validating dict keys.

Changed in version 5.0: Deprecated ambiguous trait, traits args in favor of value_trait, per_key_traits.

output_format

An instance of a Python dict.

One or more traits can be passed to the constructor to validate the keys and/or values of the dict. If you need more detailed validation, you may use a custom validator method.

Changed in version 5.0: Added key_trait for validating dict keys.

Changed in version 5.0: Deprecated ambiguous trait, traits args in favor of value_trait, per_key_traits.

property outputs
property pipeline

The pipeline of this manager is the aggregation of the source node definition and the output.

remove_trigger(statement_id)[source]

Remove trigger (permission) from lambda function

Parameters:

statement_id (str) – Specific identifier for trigger

remove_triggers()[source]

Remove all triggers from function

session

A trait whose value must be an instance of a specified class.

The value can also be an instance of a subclass of the specified class.

Subclasses can declare default classes by overriding the klass attribute

source

A trait whose value must be an instance of a specified class.

The value can also be an instance of a subclass of the specified class.

Subclasses can declare default classes by overriding the klass attribute

source_output_format

A trait for unicode strings.

source_output_name

A trait for unicode strings.

update_function()[source]

Update lambda function with new parameters

validate(raise_exceptions=False)[source]

Validate cloud resources and interoperability of resources for PODPAC usage

Parameters:

raise_exceptions (bool, optional) – Raise validation errors when encountered

validate_api()[source]

Validate that API will work with function.

This should only be run after running self.get_api()

validate_bucket()[source]

Validate that bucket will work with function.

This should only be run after running self.get_bucket()

validate_budget()[source]

Validate budget definition for function

Returns:

See podpac.managers.aws.get_budget()

Return type:

dict

validate_function()[source]

Validate that function is configured properly

This should only be run after running self.get_function()

validate_role()[source]

Validate that role will work with function.

This should only be run after running self.get_role()