podpac.managers.Lambda
- class podpac.managers.Lambda(**kwargs: Any)[source]
Bases:
Node
A Node wrapper to evaluate source on AWS Lambda function
- aws_access_key_id
Access key id from AWS credentials. If
session
is provided, this attribute will be ignored. Overridespodpac.settings
.- Type:
str, optional
- aws_region_name
Name of the AWS region. If
session
is provided, this attribute will be ignored. Overridespodpac.settings
.- Type:
str, optional
- aws_secret_access_key
Access key value from AWS credentials. If
session
is provided, this attribute will be ignored. Overridespodpac.settings
.- Type:
str
- function_name
Name of the lambda function to use or create. Defaults to
podpac.settings["FUNCTION_NAME"]
or “podpac-lambda-autogen”.- Type:
str, optional
- function_timeout
Timeout of the lambda function, in seconds. Defaults to 600.
- Type:
int, optional
- function_triggers
Methods to trigger this function. May only include [“eval”, “S3”, “APIGateway”]. During the
self.build()
process, this list will determine which AWS resources are linked to Lambda function. Defaults to [“eval”].- Type:
list of str, optional
- function_role_name
Name of the AWS role created for lambda function. Defaults to
podpac.settings["FUNCTION_ROLE_NAME"]
or “podpac-lambda-autogen”.- Type:
str, optional
- function_s3_bucket
S3 bucket name to use with lambda function. Defaults to
podpac.settings["S3_BUCKET_NAME"]
or “podpac-autogen-<timestamp>” with the timestamp to ensure uniqueness.- Type:
str, optional
- eval_settings
Default is podpac.settings. PODPAC settings that will be used to evaluate the Lambda function.
- Type:
dict, optional
- eval_timeout
Default is None. The amount of time to wait for an eval to return. To get near asynchronous response, set this to a small number.
- Type:
float, optional
- Other Attributes
- ----------------
- node_attrs
Additional attributes passed on to the Lambda definition of the base node
- Type:
dict
- download_result
Flag that indicated whether node should wait to download the data.
- Type:
Bool
- function_api_description
Description for the AWS API Gateway resource
- Type:
str, optional
- function_api_endpoint
Endpoint path for API Gateway. Defaults to “eval”.
- Type:
str, optional
- function_api_name
AWS resource name for the API Gateway. Defaults to
self.function_name
+ “-api”.- Type:
str, optional
- function_api_stage
Stage name for the API gateway. Defaults to “prod”.
- Type:
str, optional
- function_api_tags
AWS Tags for API Gateway resource. Defaults to
self.function_tags
.- Type:
dict, optional
- function_api_version
API Gateway version. Defaults to
podpac.verions.semver()
.- Type:
str, optional
- function_description
Description for the AWS Lambda function resource
- Type:
str, optional
- function_env_variables
Environment variables to use within the lambda function.
- Type:
dict, optional
- function_eval_trigger
Function trigger to use during node eval process. Must be on of “eval” (default), “S3”, or “APIGateway”.
- Type:
str, optional
- function_handler
Handler method in Lambda function. Defaults to “handler.handler”.
- Type:
str, optional
- function_memory
Memory allocated for each Lambda function. Defaults to 2048 MB.
- Type:
int, optional
- function_restrict_pipelines
List of Node hashes (see
podpac.Node.hash
). Restricts lambda function evaluation to specific Node definitions.- Type:
list, optional
- function_role_assume_policy_document
Assume policy document for role created. Defaults to allowing role to assume Lambda function.
- Type:
dict, optional.
- function_role_description
Description for the AWS role resource
- Type:
str, optional
- function_role_policy_arns
Managed role policy ARNs to attach to role.
- Type:
list of str, optional
- function_role_policy_document
Inline role policies to put in role.
- Type:
dict, optional
- function_role_tags
AWS Tags for role resource. Defaults to
self.function_tags
.- Type:
dict, optional
- function_s3_dependencies_key
S3 path to copy and reference podpac dependencies. Defaults to “podpac_deps_<semver>.zip”.
- Type:
str, optional
- function_s3_input
Folder in
self.function_s3_bucket
to store input pipelines when “S3” is included inself.function_triggers
. Defaults to “input”.- Type:
str, optional
- function_s3_output
Folder in
self.function_s3_bucket
to watch for output when “S3” is included inself.function_triggers
. Defaults to “output”.- Type:
str, optional
- function_s3_tags
AWS Tags for S3 bucket resource. Defaults to
self.function_tags
.- Type:
dict, optional
- function_source_bucket
S3 Bucket to use for released podpac distribution during
self.build()
process. Defaults to “podpac-dist”. This bucket is managed by the PODPAC distribution team.- Type:
str, optional
- function_source_dependencies_key
S3 path within
self.function_source_bucket
to source podpac dependencies archive duringself.build()
process. Defaults to “<semver>/podpac_deps.zip”.- Type:
str, optional
- function_source_dependencies_zip
Override
self.function_source_dependencies_key
and upload custom source podpac dependencies archive toself.function_s3_bucket
duringself.build()
process.- Type:
str, optional
- function_source_dist_key
S3 path within
self.function_source_bucket
to source podpac dist archive duringself.build()
process. Defaults to “<semver>/podpac_dist.zip”.- Type:
str, optional
- function_source_dist_zip
Override
self.function_source_dist_key
and create lambda function using custom source podpac dist archive toself.function_s3_bucket
duringself.build()
process.- Type:
str, optional
- function_tags
AWS Tags for Lambda function resource. Defaults to
podpac.settings["AWS_TAGS"]
or {}.- Type:
dict, optional
- function_budget_amount
EXPERIMENTAL FEATURE Monthly budget for function and associated AWS resources. When usage reaches 80% of this amount, AWS will notify
function_budget_email
. Defaults topodpac.settings["AWS_BUDGET_AMOUNT"]
.- Type:
float, optional
- function_budget_email
EXPERIMENTAL FEATURE Email to notify when usage reaches 80% of
function_budget_amount
. Defaults topodpac.settings["AWS_BUDGET_EMAIL"]
.- Type:
str, optional
- function_budget_name
EXPERIMENTAL FEATURE Name for AWS budget
- Type:
str, optional
- function_budget_currency
EXPERIMENTAL FEATURE Currency type for the
function_budget_amount
. Defaults to “USD”. See https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/budgets.html#Budgets.Client.create_budget for currency (or Unit) options.- Type:
str, optional
- output_format
Definition for how output is saved after results are computed.
- Type:
dict, optional
- session
AWS Session to use for this node.
- Type:
podpac.managers.aws.Session
- source
Node to be evaluated on the Lambda function.
- Type:
- source_output_format
Output format for the evaluated results of source
- Type:
str
- source_output_name
Output name for the evaluated results of source
- Type:
str
Alternative Constructors
from_definition
(definition)Create podpac Node from a dictionary definition.
from_json
(s)Create podpac Node from a JSON definition.
Methods
__init__
(**kwargs)Do not overwrite me
add_trigger
(statement_id, principle, source_arn)Add trigger (permission) to lambda function
build
()Build Lambda function and associated resources on AWS to run PODPAC pipelines
Create API Gateway API for lambda function
Create S3 bucket to work with function
EXPERIMENTAL FEATURE Create budget for lambda function based on node hash.
Build Lambda function on AWS
create_output_array
(coords[, data, attrs, ...])Initialize an output data array
Create IAM role to execute podpac lambda function
delete
([confirm])Remove all cloud resources associated with function
Delete API Gateway for Function
delete_bucket
([delete_objects])Delete bucket associated with this function
Delete budget associated with function
Remove AWS Lambda function and associated resources on AWS
Remove role from AWS resources
describe
()Show a description of the Lambda Utilities
eval
(coordinates[, output, selector])Evaluate the source node on the AWS Lambda Function at the given coordinates
eval_group
(group)Evaluate the node for each of the coordinates in the group.
find_coordinates
()Get all available coordinates for the Node.
from_name_params
(name[, params])Create podpac Node from a WMS/WCS request.
from_url
(url)Create podpac Node from a WMS/WCS request.
get_api
()Get API Gateway definition for function
get_bounds
([crs])Get the full available coordinate bounds for the Node.
Get S3 Bucket for function
EXPERIMENTAL FEATURE Get budget definition for function
get_cache
(key[, coordinates])Get cached data for this node.
Get function definition from AWS
get_logs
([limit, start, end])Get Cloudwatch logs from lambda function execution
get_role
()Get role definition from AWS
get_ui_spec
([help_as_html])Get spec of node attributes for building a ui
has_cache
(key[, coordinates])Check for cached data for this node.
init
()Overwrite this method if a node needs to do any additional initialization after the standard initialization.
load
(path)Create podpac Node from file.
probe
([lat, lon, time, alt, crs])Evaluates every part of a node / pipeline at a point and records which nodes are actively being used.
put_cache
(data, key[, coordinates, expires, ...])Cache data for this node.
rem_cache
(key[, coordinates, mode])Clear cached data for this node.
remove_trigger
(statement_id)Remove trigger (permission) from lambda function
Remove all triggers from function
save
(path)Write node to file.
trait_defaults
(*names, **metadata)Return a trait's default value or a dictionary of them
trait_has_value
(name)Returns True if the specified trait has a value.
trait_is_defined
(name)trait_values
(**metadata)A
dict
of trait names and their values.Update lambda function with new parameters
validate
([raise_exceptions])Validate cloud resources and interoperability of resources for PODPAC usage
Validate that API will work with function.
Validate that bucket will work with function.
Validate budget definition for function
Validate that function is configured properly
Validate that role will work with function.
Attributes
attrs
List of node attributes
A trait for unicode strings.
A trait for unicode strings.
A trait for unicode strings.
base_ref
Default reference/name in node definitions
cache_ctrl
A trait whose value must be an instance of a specified class.
cache_output
A boolean (True, False) trait.
definition
A boolean (True, False) trait.
dtype
An enum whose value must be in a given sequence.
An instance of a Python dict.
A float trait.
A boolean (True, False) trait.
force_eval
A boolean (True, False) trait.
A boolean (True, False) trait.
A trait for unicode strings.
A trait for unicode strings.
A trait for unicode strings.
A trait for unicode strings.
An instance of a Python dict.
A trait for unicode strings.
A float trait.
A trait for unicode strings.
A trait for unicode strings.
A trait for unicode strings.
A trait for unicode strings.
An instance of a Python dict.
An enum whose value must be in a given sequence.
A trait for unicode strings.
An int trait.
A trait for unicode strings.
An instance of a Python list.
An instance of a Python dict.
A trait for unicode strings.
A trait for unicode strings.
An instance of a Python list.
An instance of a Python dict.
An instance of a Python dict.
A trait for unicode strings.
A trait for unicode strings.
A trait for unicode strings.
A trait for unicode strings.
An instance of a Python dict.
A trait for unicode strings.
A trait for unicode strings.
A trait for unicode strings.
A trait for unicode strings.
A trait for unicode strings.
An instance of a Python dict.
An int trait.
An instance of a Python list.
hash
json
Definition for this node in JSON format.
json_pretty
Definition for this node in JSON format, with indentation suitable for display.
An instance of a Python dict.
output
A trait for unicode strings.
An instance of a Python dict.
The pipeline of this manager is the aggregation of the source node definition and the output.
A trait whose value must be an instance of a specified class.
A trait whose value must be an instance of a specified class.
A trait for unicode strings.
A trait for unicode strings.
style
A trait whose value must be an instance of a specified class.
units
A trait for unicode strings.
- Members:
- __init__(**kwargs)
Do not overwrite me
- add_trigger(statement_id, principle, source_arn)[source]
Add trigger (permission) to lambda function
- Parameters:
statement_id (str) – Specific identifier for trigger
principle (str) – Principle identifier from AWS
source_arn (str) – Source ARN for trigger
- aws_access_key_id
A trait for unicode strings.
- aws_region_name
A trait for unicode strings.
- aws_secret_access_key
A trait for unicode strings.
- delete(confirm=False)[source]
Remove all cloud resources associated with function
- Parameters:
confirm (bool, optional) – Must pass in confirm paramter
- delete_bucket(delete_objects=False)[source]
Delete bucket associated with this function
- Parameters:
delete_objects (bool, optional) – Delete all objects in the bucket while deleting bucket. Defaults to False.
- download_result
A boolean (True, False) trait.
- eval(coordinates, output=None, selector=None)[source]
Evaluate the source node on the AWS Lambda Function at the given coordinates
- eval_settings
An instance of a Python dict.
One or more traits can be passed to the constructor to validate the keys and/or values of the dict. If you need more detailed validation, you may use a custom validator method.
Changed in version 5.0: Added key_trait for validating dict keys.
Changed in version 5.0: Deprecated ambiguous
trait
,traits
args in favor ofvalue_trait
,per_key_traits
.
- eval_timeout
A float trait.
- force_compute
A boolean (True, False) trait.
- function_allow_unsafe_eval
A boolean (True, False) trait.
- function_api_description
A trait for unicode strings.
- function_api_endpoint
A trait for unicode strings.
- function_api_name
A trait for unicode strings.
- function_api_stage
A trait for unicode strings.
- function_api_tags
An instance of a Python dict.
One or more traits can be passed to the constructor to validate the keys and/or values of the dict. If you need more detailed validation, you may use a custom validator method.
Changed in version 5.0: Added key_trait for validating dict keys.
Changed in version 5.0: Deprecated ambiguous
trait
,traits
args in favor ofvalue_trait
,per_key_traits
.
- function_api_version
A trait for unicode strings.
- function_budget_amount
A float trait.
- function_budget_currency
A trait for unicode strings.
- function_budget_email
A trait for unicode strings.
- function_budget_name
A trait for unicode strings.
- function_description
A trait for unicode strings.
- function_env_variables
An instance of a Python dict.
One or more traits can be passed to the constructor to validate the keys and/or values of the dict. If you need more detailed validation, you may use a custom validator method.
Changed in version 5.0: Added key_trait for validating dict keys.
Changed in version 5.0: Deprecated ambiguous
trait
,traits
args in favor ofvalue_trait
,per_key_traits
.
- function_eval_trigger
An enum whose value must be in a given sequence.
- function_handler
A trait for unicode strings.
- function_memory
An int trait.
- function_name
A trait for unicode strings.
- function_restrict_pipelines
An instance of a Python list.
- function_role_assume_policy_document
An instance of a Python dict.
One or more traits can be passed to the constructor to validate the keys and/or values of the dict. If you need more detailed validation, you may use a custom validator method.
Changed in version 5.0: Added key_trait for validating dict keys.
Changed in version 5.0: Deprecated ambiguous
trait
,traits
args in favor ofvalue_trait
,per_key_traits
.
- function_role_description
A trait for unicode strings.
- function_role_name
A trait for unicode strings.
- function_role_policy_arns
An instance of a Python list.
- function_role_policy_document
An instance of a Python dict.
One or more traits can be passed to the constructor to validate the keys and/or values of the dict. If you need more detailed validation, you may use a custom validator method.
Changed in version 5.0: Added key_trait for validating dict keys.
Changed in version 5.0: Deprecated ambiguous
trait
,traits
args in favor ofvalue_trait
,per_key_traits
.
- function_role_tags
An instance of a Python dict.
One or more traits can be passed to the constructor to validate the keys and/or values of the dict. If you need more detailed validation, you may use a custom validator method.
Changed in version 5.0: Added key_trait for validating dict keys.
Changed in version 5.0: Deprecated ambiguous
trait
,traits
args in favor ofvalue_trait
,per_key_traits
.
- function_s3_bucket
A trait for unicode strings.
- function_s3_dependencies_key
A trait for unicode strings.
- function_s3_input
A trait for unicode strings.
- function_s3_output
A trait for unicode strings.
- function_s3_tags
An instance of a Python dict.
One or more traits can be passed to the constructor to validate the keys and/or values of the dict. If you need more detailed validation, you may use a custom validator method.
Changed in version 5.0: Added key_trait for validating dict keys.
Changed in version 5.0: Deprecated ambiguous
trait
,traits
args in favor ofvalue_trait
,per_key_traits
.
- function_source_bucket
A trait for unicode strings.
- function_source_dependencies_key
A trait for unicode strings.
- function_source_dependencies_zip
A trait for unicode strings.
- function_source_dist_key
A trait for unicode strings.
- function_source_dist_zip
A trait for unicode strings.
- function_tags
An instance of a Python dict.
One or more traits can be passed to the constructor to validate the keys and/or values of the dict. If you need more detailed validation, you may use a custom validator method.
Changed in version 5.0: Added key_trait for validating dict keys.
Changed in version 5.0: Deprecated ambiguous
trait
,traits
args in favor ofvalue_trait
,per_key_traits
.
- function_timeout
An int trait.
- function_triggers
An instance of a Python list.
- get_api()[source]
Get API Gateway definition for function
- Returns:
See
podpac.managers.aws.get_api()
- Return type:
dict
- get_bucket()[source]
Get S3 Bucket for function
- Returns:
See
podpac.managers.aws.get_bucket()
- Return type:
dict
- get_budget()[source]
EXPERIMENTAL FEATURE Get budget definition for function
- Returns:
See
podpac.managers.aws.get_budget()
- Return type:
dict
- get_function()[source]
Get function definition from AWS
- Returns:
See
podpac.managers.aws.get_function()
- Return type:
dict
- get_logs(limit=5, start=None, end=None)[source]
Get Cloudwatch logs from lambda function execution
See
podpac.managers.aws.get_logs()
- Parameters:
limit (int, optional) – Limit logs to the most recent N logs
start (str, optional) – Datetime string. Must work as input to np.datetime64 (i.e np.datetime64(start)) Defaults to 1 hour prior to
end
.end (str, optional) – Datetime string. Must work as input to np.datetime64 (i.e np.datetime64(end)) Defaults to now.
- Returns:
list of log events
- Return type:
list
- get_role()[source]
Get role definition from AWS
See
self.function_role_name
for role_name- Returns:
See
podpac.managers.aws.get_role()
- Return type:
dict
- node_attrs
An instance of a Python dict.
One or more traits can be passed to the constructor to validate the keys and/or values of the dict. If you need more detailed validation, you may use a custom validator method.
Changed in version 5.0: Added key_trait for validating dict keys.
Changed in version 5.0: Deprecated ambiguous
trait
,traits
args in favor ofvalue_trait
,per_key_traits
.
- output_format
An instance of a Python dict.
One or more traits can be passed to the constructor to validate the keys and/or values of the dict. If you need more detailed validation, you may use a custom validator method.
Changed in version 5.0: Added key_trait for validating dict keys.
Changed in version 5.0: Deprecated ambiguous
trait
,traits
args in favor ofvalue_trait
,per_key_traits
.
- property outputs
- property pipeline
The pipeline of this manager is the aggregation of the source node definition and the output.
- remove_trigger(statement_id)[source]
Remove trigger (permission) from lambda function
- Parameters:
statement_id (str) – Specific identifier for trigger
- session
A trait whose value must be an instance of a specified class.
The value can also be an instance of a subclass of the specified class.
Subclasses can declare default classes by overriding the klass attribute
- source
A trait whose value must be an instance of a specified class.
The value can also be an instance of a subclass of the specified class.
Subclasses can declare default classes by overriding the klass attribute
- source_output_format
A trait for unicode strings.
- source_output_name
A trait for unicode strings.
- validate(raise_exceptions=False)[source]
Validate cloud resources and interoperability of resources for PODPAC usage
- Parameters:
raise_exceptions (bool, optional) – Raise validation errors when encountered
- validate_api()[source]
Validate that API will work with function.
This should only be run after running self.get_api()
- validate_bucket()[source]
Validate that bucket will work with function.
This should only be run after running self.get_bucket()
- validate_budget()[source]
Validate budget definition for function
- Returns:
See
podpac.managers.aws.get_budget()
- Return type:
dict