podpac.managers.Lambda¶
-
class
podpac.managers.
Lambda
(**kwargs)[source]¶ Bases:
podpac.core.node.Node
A Node wrapper to evaluate source on AWS Lambda function
-
aws_access_key_id
¶ Access key id from AWS credentials. If
session
is provided, this attribute will be ignored. Overridespodpac.settings
.- Type
str, optional
-
aws_region_name
¶ Name of the AWS region. If
session
is provided, this attribute will be ignored. Overridespodpac.settings
.- Type
str, optional
-
aws_secret_access_key
¶ Access key value from AWS credentials. If
session
is provided, this attribute will be ignored. Overridespodpac.settings
.- Type
str
-
function_name
¶ Name of the lambda function to use or create. Defaults to
podpac.settings["FUNCTION_NAME"]
or “podpac-lambda-autogen”.- Type
str, optional
-
function_timeout
¶ Timeout of the lambda function, in seconds. Defaults to 600.
- Type
int, optional
-
function_triggers
¶ Methods to trigger this function. May only include [“eval”, “S3”, “APIGateway”]. During the
self.build()
process, this list will determine which AWS resources are linked to Lambda function. Defaults to [“eval”].- Type
list of str, optional
-
function_role_name
¶ Name of the AWS role created for lambda function. Defaults to
podpac.settings["FUNCTION_ROLE_NAME"]
or “podpac-lambda-autogen”.- Type
str, optional
-
function_s3_bucket
¶ S3 bucket name to use with lambda function. Defaults to
podpac.settings["S3_BUCKET_NAME"]
or “podpac-autogen-<timestamp>” with the timestamp to ensure uniqueness.- Type
str, optional
-
eval_settings
¶ Default is podpac.settings. PODPAC settings that will be used to evaluate the Lambda function.
- Type
dict, optional
-
eval_timeout
¶ Default is None. The amount of time to wait for an eval to return. To get near asynchronous response, set this to a small number.
- Type
float, optional
-
Other Attributes
-
----------------
-
node_attrs
¶ Additional attributes passed on to the Lambda definition of the base node
- Type
dict
-
download_result
¶ Flag that indicated whether node should wait to download the data.
- Type
Bool
-
function_api_description
¶ Description for the AWS API Gateway resource
- Type
str, optional
-
function_api_endpoint
¶ Endpoint path for API Gateway. Defaults to “eval”.
- Type
str, optional
-
function_api_name
¶ AWS resource name for the API Gateway. Defaults to
self.function_name
+ “-api”.- Type
str, optional
-
function_api_stage
¶ Stage name for the API gateway. Defaults to “prod”.
- Type
str, optional
AWS Tags for API Gateway resource. Defaults to
self.function_tags
.- Type
dict, optional
-
function_api_version
¶ API Gateway version. Defaults to
podpac.verions.semver()
.- Type
str, optional
-
function_description
¶ Description for the AWS Lambda function resource
- Type
str, optional
-
function_env_variables
¶ Environment variables to use within the lambda function.
- Type
dict, optional
-
function_eval_trigger
¶ Function trigger to use during node eval process. Must be on of “eval” (default), “S3”, or “APIGateway”.
- Type
str, optional
-
function_handler
¶ Handler method in Lambda function. Defaults to “handler.handler”.
- Type
str, optional
-
function_memory
¶ Memory allocated for each Lambda function. Defaults to 2048 MB.
- Type
int, optional
-
function_restrict_pipelines
¶ List of Node hashes (see
podpac.Node.hash
). Restricts lambda function evaluation to specific Node definitions.- Type
list, optional
-
function_role_assume_policy_document
¶ Assume policy document for role created. Defaults to allowing role to assume Lambda function.
- Type
dict, optional.
-
function_role_description
¶ Description for the AWS role resource
- Type
str, optional
-
function_role_policy_arns
¶ Managed role policy ARNs to attach to role.
- Type
list of str, optional
-
function_role_policy_document
¶ Inline role policies to put in role.
- Type
dict, optional
AWS Tags for role resource. Defaults to
self.function_tags
.- Type
dict, optional
-
function_s3_dependencies_key
¶ S3 path to copy and reference podpac dependencies. Defaults to “podpac_deps_<semver>.zip”.
- Type
str, optional
-
function_s3_input
¶ Folder in
self.function_s3_bucket
to store input pipelines when “S3” is included inself.function_triggers
. Defaults to “input”.- Type
str, optional
-
function_s3_output
¶ Folder in
self.function_s3_bucket
to watch for output when “S3” is included inself.function_triggers
. Defaults to “output”.- Type
str, optional
AWS Tags for S3 bucket resource. Defaults to
self.function_tags
.- Type
dict, optional
-
function_source_bucket
¶ S3 Bucket to use for released podpac distribution during
self.build()
process. Defaults to “podpac-dist”. This bucket is managed by the PODPAC distribution team.- Type
str, optional
-
function_source_dependencies_key
¶ S3 path within
self.function_source_bucket
to source podpac dependencies archive duringself.build()
process. Defaults to “<semver>/podpac_deps.zip”.- Type
str, optional
-
function_source_dependencies_zip
¶ Override
self.function_source_dependencies_key
and upload custom source podpac dependencies archive toself.function_s3_bucket
duringself.build()
process.- Type
str, optional
-
function_source_dist_key
¶ S3 path within
self.function_source_bucket
to source podpac dist archive duringself.build()
process. Defaults to “<semver>/podpac_dist.zip”.- Type
str, optional
-
function_source_dist_zip
¶ Override
self.function_source_dist_key
and create lambda function using custom source podpac dist archive toself.function_s3_bucket
duringself.build()
process.- Type
str, optional
AWS Tags for Lambda function resource. Defaults to
podpac.settings["AWS_TAGS"]
or {}.- Type
dict, optional
-
function_budget_amount
¶ EXPERIMENTAL FEATURE Monthly budget for function and associated AWS resources. When usage reaches 80% of this amount, AWS will notify
function_budget_email
. Defaults topodpac.settings["AWS_BUDGET_AMOUNT"]
.- Type
float, optional
-
function_budget_email
¶ EXPERIMENTAL FEATURE Email to notify when usage reaches 80% of
function_budget_amount
. Defaults topodpac.settings["AWS_BUDGET_EMAIL"]
.- Type
str, optional
-
function_budget_name
¶ EXPERIMENTAL FEATURE Name for AWS budget
- Type
str, optional
-
function_budget_currency
¶ EXPERIMENTAL FEATURE Currency type for the
function_budget_amount
. Defaults to “USD”. See https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/budgets.html#Budgets.Client.create_budget for currency (or Unit) options.- Type
str, optional
-
output_format
¶ Definition for how output is saved after results are computed.
- Type
dict, optional
-
session
¶ AWS Session to use for this node.
- Type
podpac.managers.aws.Session
-
source
¶ Node to be evaluated on the Lambda function.
- Type
-
source_output_format
¶ Output format for the evaluated results of source
- Type
str
-
source_output_name
¶ Output name for the evaluated results of source
- Type
str
Alternative Constructors
from_definition
(definition)Create podpac Node from a dictionary definition.
from_json
(s)Create podpac Node from a JSON definition.
Methods
__init__
(**kwargs)Do not overwrite me
add_trigger
(statement_id, principle, source_arn)Add trigger (permission) to lambda function
build
()Build Lambda function and associated resources on AWS to run PODPAC pipelines
Create API Gateway API for lambda function
Create S3 bucket to work with function
EXPERIMENTAL FEATURE Create budget for lambda function based on node hash.
Build Lambda function on AWS
create_output_array
(coords[, data, attrs])Initialize an output data array
Create IAM role to execute podpac lambda function
delete
([confirm])Remove all cloud resources associated with function
Delete API Gateway for Function
delete_bucket
([delete_objects])Delete bucket associated with this function
Delete budget associated with function
Remove AWS Lambda function and associated resources on AWS
Remove role from AWS resources
describe
()Show a description of the Lambda Utilities
eval
(coordinates[, output, selector])Evaluate the source node on the AWS Lambda Function at the given coordinates
eval_group
(group)Evaluate the node for each of the coordinates in the group.
find_coordinates
()Get all available coordinates for the Node.
from_url
(url)Create podpac Node from a WMS/WCS request.
get_api
()Get API Gateway definition for function
Get S3 Bucket for function
EXPERIMENTAL FEATURE Get budget definition for function
get_cache
(key[, coordinates])Get cached data for this node.
Get function definition from AWS
get_logs
([limit, start, end])Get Cloudwatch logs from lambda function execution
get_role
()Get role definition from AWS
has_cache
(key[, coordinates])Check for cached data for this node.
init
()Overwrite this method if a node needs to do any additional initialization after the standard initialization.
load
(path)Create podpac Node from file.
put_cache
(data, key[, coordinates, expires, …])Cache data for this node.
rem_cache
(key[, coordinates, mode])Clear cached data for this node.
remove_trigger
(statement_id)Remove trigger (permission) from lambda function
Remove all triggers from function
save
(path)Write node to file.
trait_defaults
(*names, **metadata)Return a trait’s default value or a dictionary of them
trait_has_value
(name)Returns True if the specified trait has a value.
trait_is_defined
(name)trait_values
(**metadata)A
dict
of trait names and their values.Update lambda function with new parameters
validate
([raise_exceptions])Validate cloud resources and interoperability of resources for PODPAC usage
Validate that API will work with function.
Validate that bucket will work with function.
Validate budget definition for function
Validate that function is configured properly
Validate that role will work with function.
Attributes
attrs
List of node attributes
A trait for unicode strings.
A trait for unicode strings.
A trait for unicode strings.
base_ref
Default reference/name in node definitions
cache_ctrl
A trait whose value must be an instance of a specified class.
cache_output
A boolean (True, False) trait.
definition
A boolean (True, False) trait.
dtype
A trait which allows any value.
An instance of a Python dict.
A float trait.
A boolean (True, False) trait.
force_eval
A boolean (True, False) trait.
A boolean (True, False) trait.
A trait for unicode strings.
A trait for unicode strings.
A trait for unicode strings.
A trait for unicode strings.
An instance of a Python dict.
A trait for unicode strings.
A float trait.
A trait for unicode strings.
A trait for unicode strings.
A trait for unicode strings.
A trait for unicode strings.
An instance of a Python dict.
An enum whose value must be in a given sequence.
A trait for unicode strings.
An int trait.
A trait for unicode strings.
An instance of a Python list.
An instance of a Python dict.
A trait for unicode strings.
A trait for unicode strings.
An instance of a Python list.
An instance of a Python dict.
An instance of a Python dict.
A trait for unicode strings.
A trait for unicode strings.
A trait for unicode strings.
A trait for unicode strings.
An instance of a Python dict.
A trait for unicode strings.
A trait for unicode strings.
A trait for unicode strings.
A trait for unicode strings.
A trait for unicode strings.
An instance of a Python dict.
An int trait.
An instance of a Python list.
hash
hash for this node, used in caching and to determine equality.
json
Definition for this node in JSON format.
json_pretty
Definition for this node in JSON format, with indentation suitable for display.
An instance of a Python dict.
output
A trait for unicode strings.
An instance of a Python dict.
The pipeline of this manager is the aggregation of the source node definition and the output.
A trait whose value must be an instance of a specified class.
A trait whose value must be an instance of a specified class.
A trait for unicode strings.
A trait for unicode strings.
style
A trait whose value must be an instance of a specified class.
units
A trait for unicode strings.
- Members
-
__init__
(**kwargs)¶ Do not overwrite me
-
add_trigger
(statement_id, principle, source_arn)[source]¶ Add trigger (permission) to lambda function
- Parameters
statement_id (str) – Specific identifier for trigger
principle (str) – Principle identifier from AWS
source_arn (str) – Source ARN for trigger
-
aws_access_key_id
A trait for unicode strings.
-
aws_region_name
A trait for unicode strings.
-
aws_secret_access_key
A trait for unicode strings.
-
delete
(confirm=False)[source]¶ Remove all cloud resources associated with function
- Parameters
confirm (bool, optional) – Must pass in confirm paramter
-
delete_bucket
(delete_objects=False)[source]¶ Delete bucket associated with this function
- Parameters
delete_objects (bool, optional) – Delete all objects in the bucket while deleting bucket. Defaults to False.
-
download_result
A boolean (True, False) trait.
-
eval
(coordinates, output=None, selector=None)[source]¶ Evaluate the source node on the AWS Lambda Function at the given coordinates
-
eval_settings
An instance of a Python dict.
One or more traits can be passed to the constructor to validate the keys and/or values of the dict. If you need more detailed validation, you may use a custom validator method.
Changed in version 5.0: Added key_trait for validating dict keys.
Changed in version 5.0: Deprecated ambiguous
trait
,traits
args in favor ofvalue_trait
,per_key_traits
.
-
eval_timeout
A float trait.
-
force_compute
¶ A boolean (True, False) trait.
-
function_allow_unsafe_eval
¶ A boolean (True, False) trait.
-
function_api_description
A trait for unicode strings.
-
function_api_endpoint
A trait for unicode strings.
-
function_api_name
A trait for unicode strings.
-
function_api_stage
A trait for unicode strings.
-
function_api_tags
An instance of a Python dict.
One or more traits can be passed to the constructor to validate the keys and/or values of the dict. If you need more detailed validation, you may use a custom validator method.
Changed in version 5.0: Added key_trait for validating dict keys.
Changed in version 5.0: Deprecated ambiguous
trait
,traits
args in favor ofvalue_trait
,per_key_traits
.
-
function_api_version
A trait for unicode strings.
-
function_budget_amount
A float trait.
-
function_budget_currency
A trait for unicode strings.
-
function_budget_email
A trait for unicode strings.
-
function_budget_name
A trait for unicode strings.
-
function_description
A trait for unicode strings.
-
function_env_variables
An instance of a Python dict.
One or more traits can be passed to the constructor to validate the keys and/or values of the dict. If you need more detailed validation, you may use a custom validator method.
Changed in version 5.0: Added key_trait for validating dict keys.
Changed in version 5.0: Deprecated ambiguous
trait
,traits
args in favor ofvalue_trait
,per_key_traits
.
-
function_eval_trigger
An enum whose value must be in a given sequence.
-
function_handler
A trait for unicode strings.
-
function_memory
An int trait.
-
function_name
A trait for unicode strings.
-
function_restrict_pipelines
An instance of a Python list.
-
function_role_assume_policy_document
An instance of a Python dict.
One or more traits can be passed to the constructor to validate the keys and/or values of the dict. If you need more detailed validation, you may use a custom validator method.
Changed in version 5.0: Added key_trait for validating dict keys.
Changed in version 5.0: Deprecated ambiguous
trait
,traits
args in favor ofvalue_trait
,per_key_traits
.
-
function_role_description
A trait for unicode strings.
-
function_role_name
A trait for unicode strings.
-
function_role_policy_arns
An instance of a Python list.
-
function_role_policy_document
An instance of a Python dict.
One or more traits can be passed to the constructor to validate the keys and/or values of the dict. If you need more detailed validation, you may use a custom validator method.
Changed in version 5.0: Added key_trait for validating dict keys.
Changed in version 5.0: Deprecated ambiguous
trait
,traits
args in favor ofvalue_trait
,per_key_traits
.
-
function_role_tags
An instance of a Python dict.
One or more traits can be passed to the constructor to validate the keys and/or values of the dict. If you need more detailed validation, you may use a custom validator method.
Changed in version 5.0: Added key_trait for validating dict keys.
Changed in version 5.0: Deprecated ambiguous
trait
,traits
args in favor ofvalue_trait
,per_key_traits
.
-
function_s3_bucket
A trait for unicode strings.
-
function_s3_dependencies_key
A trait for unicode strings.
-
function_s3_input
A trait for unicode strings.
-
function_s3_output
A trait for unicode strings.
-
function_s3_tags
An instance of a Python dict.
One or more traits can be passed to the constructor to validate the keys and/or values of the dict. If you need more detailed validation, you may use a custom validator method.
Changed in version 5.0: Added key_trait for validating dict keys.
Changed in version 5.0: Deprecated ambiguous
trait
,traits
args in favor ofvalue_trait
,per_key_traits
.
-
function_source_bucket
A trait for unicode strings.
-
function_source_dependencies_key
A trait for unicode strings.
-
function_source_dependencies_zip
A trait for unicode strings.
-
function_source_dist_key
A trait for unicode strings.
-
function_source_dist_zip
A trait for unicode strings.
-
function_tags
An instance of a Python dict.
One or more traits can be passed to the constructor to validate the keys and/or values of the dict. If you need more detailed validation, you may use a custom validator method.
Changed in version 5.0: Added key_trait for validating dict keys.
Changed in version 5.0: Deprecated ambiguous
trait
,traits
args in favor ofvalue_trait
,per_key_traits
.
-
function_timeout
An int trait.
-
function_triggers
An instance of a Python list.
-
get_api
()[source]¶ Get API Gateway definition for function
- Returns
See
podpac.managers.aws.get_api()
- Return type
dict
-
get_bucket
()[source]¶ Get S3 Bucket for function
- Returns
See
podpac.managers.aws.get_bucket()
- Return type
dict
-
get_budget
()[source]¶ EXPERIMENTAL FEATURE Get budget definition for function
- Returns
See
podpac.managers.aws.get_budget()
- Return type
dict
-
get_function
()[source]¶ Get function definition from AWS
- Returns
See
podpac.managers.aws.get_function()
- Return type
dict
-
get_logs
(limit=5, start=None, end=None)[source]¶ Get Cloudwatch logs from lambda function execution
See
podpac.managers.aws.get_logs()
- Parameters
limit (int, optional) – Limit logs to the most recent N logs
start (str, optional) – Datetime string. Must work as input to np.datetime64 (i.e np.datetime64(start)) Defaults to 1 hour prior to
end
.end (str, optional) – Datetime string. Must work as input to np.datetime64 (i.e np.datetime64(end)) Defaults to now.
- Returns
list of log events
- Return type
list
-
get_role
()[source]¶ Get role definition from AWS
See
self.function_role_name
for role_name- Returns
See
podpac.managers.aws.get_role()
- Return type
dict
-
node_attrs
An instance of a Python dict.
One or more traits can be passed to the constructor to validate the keys and/or values of the dict. If you need more detailed validation, you may use a custom validator method.
Changed in version 5.0: Added key_trait for validating dict keys.
Changed in version 5.0: Deprecated ambiguous
trait
,traits
args in favor ofvalue_trait
,per_key_traits
.
-
output_format
An instance of a Python dict.
One or more traits can be passed to the constructor to validate the keys and/or values of the dict. If you need more detailed validation, you may use a custom validator method.
Changed in version 5.0: Added key_trait for validating dict keys.
Changed in version 5.0: Deprecated ambiguous
trait
,traits
args in favor ofvalue_trait
,per_key_traits
.
-
property
outputs
¶
-
property
pipeline
¶ The pipeline of this manager is the aggregation of the source node definition and the output.
-
remove_trigger
(statement_id)[source]¶ Remove trigger (permission) from lambda function
- Parameters
statement_id (str) – Specific identifier for trigger
-
session
A trait whose value must be an instance of a specified class.
The value can also be an instance of a subclass of the specified class.
Subclasses can declare default classes by overriding the klass attribute
-
source
A trait whose value must be an instance of a specified class.
The value can also be an instance of a subclass of the specified class.
Subclasses can declare default classes by overriding the klass attribute
-
source_output_format
A trait for unicode strings.
-
source_output_name
A trait for unicode strings.
-
validate
(raise_exceptions=False)[source]¶ Validate cloud resources and interoperability of resources for PODPAC usage
- Parameters
raise_exceptions (bool, optional) – Raise validation errors when encountered
-
validate_api
()[source]¶ Validate that API will work with function.
This should only be run after running self.get_api()
-
validate_bucket
()[source]¶ Validate that bucket will work with function.
This should only be run after running self.get_bucket()
-
validate_budget
()[source]¶ Validate budget definition for function
- Returns
See
podpac.managers.aws.get_budget()
- Return type
dict
-