podpac.managers.Lambda

class podpac.managers.Lambda(**kwargs)[source]

Bases: podpac.core.node.Node

A Node wrapper to evaluate source on AWS Lambda function

aws_access_key_id

Access key id from AWS credentials. If session is provided, this attribute will be ignored. Overrides podpac.settings.

Type

str, optional

aws_region_name

Name of the AWS region. If session is provided, this attribute will be ignored. Overrides podpac.settings.

Type

str, optional

aws_secret_access_key

Access key value from AWS credentials. If session is provided, this attribute will be ignored. Overrides podpac.settings.

Type

str

function_name

Name of the lambda function to use or create. Defaults to podpac.settings["FUNCTION_NAME"] or “podpac-lambda-autogen”.

Type

str, optional

function_timeout

Timeout of the lambda function, in seconds. Defaults to 600.

Type

int, optional

function_triggers

Methods to trigger this function. May only include [“eval”, “S3”, “APIGateway”]. During the self.build() process, this list will determine which AWS resources are linked to Lambda function. Defaults to [“eval”].

Type

list of str, optional

function_role_name

Name of the AWS role created for lambda function. Defaults to podpac.settings["FUNCTION_ROLE_NAME"] or “podpac-lambda-autogen”.

Type

str, optional

function_s3_bucket

S3 bucket name to use with lambda function. Defaults to podpac.settings["S3_BUCKET_NAME"] or “podpac-autogen-<timestamp>” with the timestamp to ensure uniqueness.

Type

str, optional

eval_settings

Default is podpac.settings. PODPAC settings that will be used to evaluate the Lambda function.

Type

dict, optional

eval_timeout

Default is None. The amount of time to wait for an eval to return. To get near asynchronous response, set this to a small number.

Type

float, optional

Other Attributes
----------------
node_attrs

Additional attributes passed on to the Lambda definition of the base node

Type

dict

download_result

Flag that indicated whether node should wait to download the data.

Type

Bool

function_api_description

Description for the AWS API Gateway resource

Type

str, optional

function_api_endpoint

Endpoint path for API Gateway. Defaults to “eval”.

Type

str, optional

function_api_name

AWS resource name for the API Gateway. Defaults to self.function_name + “-api”.

Type

str, optional

function_api_stage

Stage name for the API gateway. Defaults to “prod”.

Type

str, optional

function_api_tags

AWS Tags for API Gateway resource. Defaults to self.function_tags.

Type

dict, optional

function_api_version

API Gateway version. Defaults to podpac.verions.semver().

Type

str, optional

function_description

Description for the AWS Lambda function resource

Type

str, optional

function_env_variables

Environment variables to use within the lambda function.

Type

dict, optional

function_eval_trigger

Function trigger to use during node eval process. Must be on of “eval” (default), “S3”, or “APIGateway”.

Type

str, optional

function_handler

Handler method in Lambda function. Defaults to “handler.handler”.

Type

str, optional

function_memory

Memory allocated for each Lambda function. Defaults to 2048 MB.

Type

int, optional

function_restrict_pipelines

List of Node hashes (see podpac.Node.hash). Restricts lambda function evaluation to specific Node definitions.

Type

list, optional

function_role_assume_policy_document

Assume policy document for role created. Defaults to allowing role to assume Lambda function.

Type

dict, optional.

function_role_description

Description for the AWS role resource

Type

str, optional

function_role_policy_arns

Managed role policy ARNs to attach to role.

Type

list of str, optional

function_role_policy_document

Inline role policies to put in role.

Type

dict, optional

function_role_tags

AWS Tags for role resource. Defaults to self.function_tags.

Type

dict, optional

function_s3_dependencies_key

S3 path to copy and reference podpac dependencies. Defaults to “podpac_deps_<semver>.zip”.

Type

str, optional

function_s3_input

Folder in self.function_s3_bucket to store input pipelines when “S3” is included in self.function_triggers. Defaults to “input”.

Type

str, optional

function_s3_output

Folder in self.function_s3_bucket to watch for output when “S3” is included in self.function_triggers. Defaults to “output”.

Type

str, optional

function_s3_tags

AWS Tags for S3 bucket resource. Defaults to self.function_tags.

Type

dict, optional

function_source_bucket

S3 Bucket to use for released podpac distribution during self.build() process. Defaults to “podpac-dist”. This bucket is managed by the PODPAC distribution team.

Type

str, optional

function_source_dependencies_key

S3 path within self.function_source_bucket to source podpac dependencies archive during self.build() process. Defaults to “<semver>/podpac_deps.zip”.

Type

str, optional

function_source_dependencies_zip

Override self.function_source_dependencies_key and upload custom source podpac dependencies archive to self.function_s3_bucket during self.build() process.

Type

str, optional

function_source_dist_key

S3 path within self.function_source_bucket to source podpac dist archive during self.build() process. Defaults to “<semver>/podpac_dist.zip”.

Type

str, optional

function_source_dist_zip

Override self.function_source_dist_key and create lambda function using custom source podpac dist archive to self.function_s3_bucket during self.build() process.

Type

str, optional

function_tags

AWS Tags for Lambda function resource. Defaults to podpac.settings["AWS_TAGS"] or {}.

Type

dict, optional

function_budget_amount

EXPERIMENTAL FEATURE Monthly budget for function and associated AWS resources. When usage reaches 80% of this amount, AWS will notify function_budget_email. Defaults to podpac.settings["AWS_BUDGET_AMOUNT"].

Type

float, optional

function_budget_email

EXPERIMENTAL FEATURE Email to notify when usage reaches 80% of function_budget_amount. Defaults to podpac.settings["AWS_BUDGET_EMAIL"].

Type

str, optional

function_budget_name

EXPERIMENTAL FEATURE Name for AWS budget

Type

str, optional

function_budget_currency

EXPERIMENTAL FEATURE Currency type for the function_budget_amount. Defaults to “USD”. See https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/budgets.html#Budgets.Client.create_budget for currency (or Unit) options.

Type

str, optional

output_format

Definition for how output is saved after results are computed.

Type

dict, optional

session

AWS Session to use for this node.

Type

podpac.managers.aws.Session

source

Node to be evaluated on the Lambda function.

Type

podpac.Node

source_output_format

Output format for the evaluated results of source

Type

str

source_output_name

Output name for the evaluated results of source

Type

str

Alternative Constructors

from_definition(definition)

Create podpac Node from a dictionary definition.

from_json(s)

Create podpac Node from a JSON definition.

Methods

__init__(**kwargs)

Do not overwrite me

add_trigger(statement_id, principle, source_arn)

Add trigger (permission) to lambda function

build()

Build Lambda function and associated resources on AWS to run PODPAC pipelines

create_api()

Create API Gateway API for lambda function

create_bucket()

Create S3 bucket to work with function

create_budget()

EXPERIMENTAL FEATURE Create budget for lambda function based on node hash.

create_function()

Build Lambda function on AWS

create_output_array(coords[, data])

Initialize an output data array

create_role()

Create IAM role to execute podpac lambda function

delete([confirm])

Remove all cloud resources associated with function

delete_api()

Delete API Gateway for Function

delete_bucket([delete_objects])

Delete bucket associated with this function

delete_budget()

Delete budget associated with function

delete_function()

Remove AWS Lambda function and associated resources on AWS

delete_role()

Remove role from AWS resources

describe()

Show a description of the Lambda Utilities

eval(coordinates[, output])

Evaluate the source node on the AWS Lambda Function at the given coordinates

eval_group(group)

Evaluate the node for each of the coordinates in the group.

find_coordinates()

Get all available coordinates for the Node.

from_url(url)

Create podpac Node from a WMS/WCS request.

get_api()

Get API Gateway definition for function

get_bucket()

Get S3 Bucket for function

get_budget()

EXPERIMENTAL FEATURE Get budget definition for function

get_cache(key[, coordinates])

Get cached data for this node.

get_function()

Get function definition from AWS

get_logs([limit, start, end])

Get Cloudwatch logs from lambda function execution

get_role()

Get role definition from AWS

has_cache(key[, coordinates])

Check for cached data for this node.

init()

Overwrite this method if a node needs to do any additional initialization after the standard initialization.

load(path)

Create podpac Node from file.

put_cache(data, key[, coordinates, overwrite])

Cache data for this node.

rem_cache(key[, coordinates, mode])

Clear cached data for this node.

remove_trigger(statement_id)

Remove trigger (permission) from lambda function

remove_triggers()

Remove all triggers from function

save(path)

Write node to file.

trait_is_defined(name)

update_function()

Update lambda function with new parameters

validate([raise_exceptions])

Validate cloud resources and interoperability of resources for PODPAC usage

validate_api()

Validate that API will work with function.

validate_bucket()

Validate that bucket will work with function.

validate_budget()

Validate budget definition for function

validate_function()

Validate that function is configured properly

validate_role()

Validate that role will work with function.

Attributes

attrs

List of node attributes

aws_access_key_id

A trait for unicode strings.

aws_region_name

A trait for unicode strings.

aws_secret_access_key

A trait for unicode strings.

base_ref

Default reference/name in node definitions

cache_ctrl

A trait whose value must be an instance of a specified class.

cache_output

A boolean (True, False) trait.

definition

download_result

A boolean (True, False) trait.

dtype

A trait which allows any value.

eval_settings

An instance of a Python dict.

eval_timeout

A float trait.

force_compute

A boolean (True, False) trait.

force_eval

A boolean (True, False) trait.

function_allow_unsafe_eval

A boolean (True, False) trait.

function_api_description

A trait for unicode strings.

function_api_endpoint

A trait for unicode strings.

function_api_name

A trait for unicode strings.

function_api_stage

A trait for unicode strings.

function_api_tags

An instance of a Python dict.

function_api_version

A trait for unicode strings.

function_budget_amount

A float trait.

function_budget_currency

A trait for unicode strings.

function_budget_email

A trait for unicode strings.

function_budget_name

A trait for unicode strings.

function_description

A trait for unicode strings.

function_env_variables

An instance of a Python dict.

function_eval_trigger

An enum whose value must be in a given sequence.

function_handler

A trait for unicode strings.

function_memory

An int trait.

function_name

A trait for unicode strings.

function_restrict_pipelines

An instance of a Python list.

function_role_assume_policy_document

An instance of a Python dict.

function_role_description

A trait for unicode strings.

function_role_name

A trait for unicode strings.

function_role_policy_arns

An instance of a Python list.

function_role_policy_document

An instance of a Python dict.

function_role_tags

An instance of a Python dict.

function_s3_bucket

A trait for unicode strings.

function_s3_dependencies_key

A trait for unicode strings.

function_s3_input

A trait for unicode strings.

function_s3_output

A trait for unicode strings.

function_s3_tags

An instance of a Python dict.

function_source_bucket

A trait for unicode strings.

function_source_dependencies_key

A trait for unicode strings.

function_source_dependencies_zip

A trait for unicode strings.

function_source_dist_key

A trait for unicode strings.

function_source_dist_zip

A trait for unicode strings.

function_tags

An instance of a Python dict.

function_timeout

An int trait.

function_triggers

An instance of a Python list.

hash

json

json_pretty

node_attrs

An instance of a Python dict.

output

A trait for unicode strings.

output_format

An instance of a Python dict.

outputs

pipeline

The pipeline of this manager is the aggregation of the source node definition and the output.

session

A trait whose value must be an instance of a specified class.

source

A trait whose value must be an instance of a specified class.

source_output_format

A trait for unicode strings.

source_output_name

A trait for unicode strings.

style

A trait whose value must be an instance of a specified class.

units

A trait for unicode strings.

Members

__init__(**kwargs)

Do not overwrite me

add_trigger(statement_id, principle, source_arn)[source]

Add trigger (permission) to lambda function

Parameters
  • statement_id (str) – Specific identifier for trigger

  • principle (str) – Principle identifier from AWS

  • source_arn (str) – Source ARN for trigger

aws_access_key_id

A trait for unicode strings.

aws_region_name

A trait for unicode strings.

aws_secret_access_key

A trait for unicode strings.

build()[source]

Build Lambda function and associated resources on AWS to run PODPAC pipelines

create_api()[source]

Create API Gateway API for lambda function

create_bucket()[source]

Create S3 bucket to work with function

create_budget()[source]

EXPERIMENTAL FEATURE Create budget for lambda function based on node hash.

create_function()[source]

Build Lambda function on AWS

create_role()[source]

Create IAM role to execute podpac lambda function

delete(confirm=False)[source]

Remove all cloud resources associated with function

Parameters

confirm (bool, optional) – Must pass in confirm paramter

delete_api()[source]

Delete API Gateway for Function

delete_bucket(delete_objects=False)[source]

Delete bucket associated with this function

Parameters

delete_objects (bool, optional) – Delete all objects in the bucket while deleting bucket. Defaults to False.

delete_budget()[source]

Delete budget associated with function

delete_function()[source]

Remove AWS Lambda function and associated resources on AWS

delete_role()[source]

Remove role from AWS resources

See self.function_role_name for role_name

describe()[source]

Show a description of the Lambda Utilities

download_result

A boolean (True, False) trait.

eval(coordinates, output=None)[source]

Evaluate the source node on the AWS Lambda Function at the given coordinates

eval_settings

An instance of a Python dict.

eval_timeout

A float trait.

force_compute

A boolean (True, False) trait.

function_allow_unsafe_eval

A boolean (True, False) trait.

function_api_description

A trait for unicode strings.

function_api_endpoint

A trait for unicode strings.

function_api_name

A trait for unicode strings.

function_api_stage

A trait for unicode strings.

function_api_tags

An instance of a Python dict.

function_api_version

A trait for unicode strings.

function_budget_amount

A float trait.

function_budget_currency

A trait for unicode strings.

function_budget_email

A trait for unicode strings.

function_budget_name

A trait for unicode strings.

function_description

A trait for unicode strings.

function_env_variables

An instance of a Python dict.

function_eval_trigger

An enum whose value must be in a given sequence.

function_handler

A trait for unicode strings.

function_memory

An int trait.

function_name

A trait for unicode strings.

function_restrict_pipelines

An instance of a Python list.

function_role_assume_policy_document

An instance of a Python dict.

function_role_description

A trait for unicode strings.

function_role_name

A trait for unicode strings.

function_role_policy_arns

An instance of a Python list.

function_role_policy_document

An instance of a Python dict.

function_role_tags

An instance of a Python dict.

function_s3_bucket

A trait for unicode strings.

function_s3_dependencies_key

A trait for unicode strings.

function_s3_input

A trait for unicode strings.

function_s3_output

A trait for unicode strings.

function_s3_tags

An instance of a Python dict.

function_source_bucket

A trait for unicode strings.

function_source_dependencies_key

A trait for unicode strings.

function_source_dependencies_zip

A trait for unicode strings.

function_source_dist_key

A trait for unicode strings.

function_source_dist_zip

A trait for unicode strings.

function_tags

An instance of a Python dict.

function_timeout

An int trait.

function_triggers

An instance of a Python list.

get_api()[source]

Get API Gateway definition for function

Returns

See podpac.managers.aws.get_api()

Return type

dict

get_bucket()[source]

Get S3 Bucket for function

Returns

See podpac.managers.aws.get_bucket()

Return type

dict

get_budget()[source]

EXPERIMENTAL FEATURE Get budget definition for function

Returns

See podpac.managers.aws.get_budget()

Return type

dict

get_function()[source]

Get function definition from AWS

Returns

See podpac.managers.aws.get_function()

Return type

dict

get_logs(limit=5, start=None, end=None)[source]

Get Cloudwatch logs from lambda function execution

See podpac.managers.aws.get_logs()

Parameters
  • limit (int, optional) – Limit logs to the most recent N logs

  • start (str, optional) – Datetime string. Must work as input to np.datetime64 (i.e np.datetime64(start)) Defaults to 1 hour prior to end.

  • end (str, optional) – Datetime string. Must work as input to np.datetime64 (i.e np.datetime64(end)) Defaults to now.

Returns

list of log events

Return type

list

get_role()[source]

Get role definition from AWS

See self.function_role_name for role_name

Returns

See podpac.managers.aws.get_role()

Return type

dict

node_attrs

An instance of a Python dict.

output_format

An instance of a Python dict.

property outputs
property pipeline

The pipeline of this manager is the aggregation of the source node definition and the output.

remove_trigger(statement_id)[source]

Remove trigger (permission) from lambda function

Parameters

statement_id (str) – Specific identifier for trigger

remove_triggers()[source]

Remove all triggers from function

session

A trait whose value must be an instance of a specified class.

The value can also be an instance of a subclass of the specified class.

Subclasses can declare default classes by overriding the klass attribute

source

A trait whose value must be an instance of a specified class.

The value can also be an instance of a subclass of the specified class.

Subclasses can declare default classes by overriding the klass attribute

source_output_format

A trait for unicode strings.

source_output_name

A trait for unicode strings.

update_function()[source]

Update lambda function with new parameters

validate(raise_exceptions=False)[source]

Validate cloud resources and interoperability of resources for PODPAC usage

Parameters

raise_exceptions (bool, optional) – Raise validation errors when encountered

validate_api()[source]

Validate that API will work with function.

This should only be run after running self.get_api()

validate_bucket()[source]

Validate that bucket will work with function.

This should only be run after running self.get_bucket()

validate_budget()[source]

Validate budget definition for function

Returns

See podpac.managers.aws.get_budget()

Return type

dict

validate_function()[source]

Validate that function is configured properly

This should only be run after running self.get_function()

validate_role()[source]

Validate that role will work with function.

This should only be run after running self.get_role()