Wrapping Landsat8, Sentinel2, and MODIS data and improving interpolation.
datalib.satutilswhich wraps Landsat8 and Sentinel2 data
datalib.modis_pdswhich wraps MODIS products [“MCD43A4.006”, “MOD09GA.006”, “MYD09GA.006”, “MOD09GQ.006”, “MYD09GQ.006”]
Added settings[‘AWS_REQUESTER_PAYS’] and
authentication.S3Mixing.aws_requester_paysattribute to support Sentinel2 data
issubsetmethod to Coordinates which allows users to test if a coordinate is a subset of another one
Added environmental variables in Lambda function deployment allowing users to specify the location of additional dependencies (
FUNCTION_DEPENDENCIES_KEY) and settings (
SETTINGS). This was in support the WMS service.
Intake nodes can now filter inputs by additional data columns for .csv files / pandas dataframes by using the pandas
Added documentation on
Compositornodes which indicates the dimensions that sources are expected to have. This fixes a bug where
Nodesthrow and error if Coordinates contain extra dimensions when the
Compositorsources are missing those dimensions.
COSMOSStationswill no longer fail for sites with no data or one data point. These sites are now automatically filtered.
core.data.file_sourceclosing files prematurely due to using context managers
Fixed heterogenous interpolation (where lat/lon uses a different interpolator than time, for example)
datalib.TerrainTilesnow accesses S3 anonymously by default. Interpolation specified at the compositor level are also now passed down to the sources.
core.algorithm.signal.pyand in the process removed
TemporalConvolutions. Users now have to label the dimensions of the kernel – which prevents results from being modified if the eval coordinates are transposed. This was a major bug in the
Convolutionnode, and the new change obviates the need for the removed Nodes, but it may break some pipelines.
Fixing some bugs associated with AWS evaluation and the drought-monitor application
algorithm.TransformTimeUnitsalgorithm to be able to utilize time in terms of
Coordinates.transform_timemethod to transform YYYY-MM-DD time units to something like
authentication.S3Mixinbecause it is not a valid argument
algorithm.Masknode simpler to JSON serialize
interpolatortolerant of string-based timedelta values
Nodeoutputs if it is not present – this caused issues with the
open_consolidatedcall. It was erroneously passing the
datalib.weathercitizento retrieve weathercitizen data
datalib.cosmos_stationsto retrieve soil moisture data from the stationary COSMOS soil moisture network
algorithm.ResampleReduce, which allows users to coarsen a dataset based on a reduce operation (such as mean, max, etc.).
managers.parallelsubmodule that enables parallel computation with PODPAC in a multi-threaded, multi-process, or multi-AWS-Lambda-function way
managers.multi_processsubmodule that enables PODPAC nodes to be run in another process.
compositor.UniformTileMixinto enable compositing of data sources BEFORE harmonization (so that interpolation can happen across data sources with the same coordinate systems)
Coordinatesso that validation of the
crscan be skipped if needed (this improved speed)
managers.aws.Lambda.eval_timeoutattribute so that Python will only wait a certain amount of time before returning from a invoke call.
Added asynchronous evaluation of
managers.aws.LambdaNodes. Note! This does not do error checking (e.g. exceeding AWS allocation f the number of allowed concurrent threads).
alglib.climatology, which allows computation of Beta distribution fits to time series for each day of the year
core.algorithm.stats.DayOfYearWindow, which is similar to
GroupReduce, but also allows a window for the computation and only works for time
Coordinateswill now be automatically simplified when transformed from one
crsto another. Previously, transformations always resulted in
DependentCoordinates. This change adds the
.simplymethod to some
Improved performance of PODPAC AWS Lambda function by only downloading dependencies when the function is not already “hot”. “Hot” functions retain the files in the
/tmpdirectory of the Lambda function.
data.Zarrwill now use
file_mode='r'for more efficient read operations on S3
Added podpac’s version to pipeline definitions
month. It also now returns the time coordinate in one of these units.
Implemented a circular dependency check to avoid infinite recursion and locking up due to cache accessing. This change also defined the
zarr_partformat to work propertly with parallel computations
[algorithm]dependencies as part of the AWS Lambda function build – previously the
numexprPython package was missing
Coordinates. Now specify the
datalib.smap_egimodule to work with
Modified PODPAC’s root setting directory from
Changed default cache behaviour to overwrite by default (prevents certain cache lock situations)
datalib.airmoss– it was no longer working!
Refactored the way PODPAC keeps track of
Nodedefinition. Most all of it is now handled by the base class, previously
Compositorhad to implement specialized functions.
datalibnodes to prefer using the new
cached_propertydecorator instead of
defaultswhich were causing severe circular dependencies
DataSourcenodes that access files on S3 to use a common
Refactored authentication to use more consistent approach across the library
The purpose of this release was to make the software more robust and to improve the Drought Monitor Application.
Algorithm arrays can now be multi-threaded. This allows an algorithm with multiple S3 data sources to fetch the data in parallel before doing the computation, speeding up the process. See #343
Improvements to AWS interface. See #336
Added budgeting / billing capability to manage AWS resources. See #361
Added GeoTIFF export / import capability. Lots of work with geotransforms in the Coordinates object. See #364.
Nodes can now have multiple output channels. This support multispectral or multichannel data. See #348.
When intersecting time coordinate of different precision, no intersection would result. See #344
Arraydatasource serialization 55fcf30
The H5PY, CSV, and Zarr nodes interfaces were unified as such, the following attributes have changed:
datakey –> data_key
latkey –> lat_key
lonkey –> lon_key
altkey –> alt_key
timekey –> time_key
keys –> available_keys
CSV.lat_col –> lat_key
CSV.lon_col –> lon_key
CSV.time_col –> time_key
CSV.alt_col –> alt_key
The purpose of this release was to develop a short course for AMS2020. A major feature of this release is automated creation of the PODPAC Lambda function. As part of this we implemented a few more additional features, and fixed a number of bugs.
Automated Lambda Function creation using PODPAC. See #326, #306
Added a context manager for easy temporary settings. See #329
Added generic algorithm module with
Genericnodes. See #325, #323
Note, this required the new unsafe evaluation setting ce8dd68355c1635214644bb5295325918493da63 bbe251a893212ea42b8c41103330e6dad0e235fe
Added styles to node serialization, enabling customization of WMS layers. See #317
Made mode attr’s read-only by default. See #315
Updated the definition of advanced interpolation for nodes 05163b44ca7ab66729150991bf7a6995382e3c35
Corrected string comparison in AWS Lambda function 7dbaf3f7d79c740cb00a37580fed259d59472e55
_first_init usage should have called super c02dc0387fe3de9223bfac6165d7bc60922982d4
Made style definition consistent 279eab9c40abcd8e8fb49afe366d28058391cfc6
Fixed DroughtCategory algorithm – the upper limit was not correct 9b92bb70b9e9de96d68ed67c4f3b053f71052229
Fixed failure on pre-commit hook installation for dev version of podpac aebe6b0e1db39c802e5613582774594c225c6ff1 #322
Fixed to interpolation #320 f9ad4936a8df0ce9a46f12a7c71b8c50d2ef0701 f9ad4936a8df0ce9a46f12a7c71b8c50d2ef0701 fadf939db1f679594d22690da7a58c6554705440
Fixed error due to missing quality flag in SMAP(EGI) node 26fcb169bd1b137c51d00aa0bc7ddd2f1ae686d8
The Pipeline Node is being deprecated, targeted for 2.0 release. Use Node.from_json instead.
DOC: Add a release procedure (RELEASE.md)
DOC: Add user guide for Settings
DOC: Improve Documentation For Nodes (#57)
DOC: Deploy documentation site to
TEST: Add coordinate testing (#3)
ENH: Cache evaluation results and check the cache before evaluating (#125)
ENH: Add datalib for TerrainTiles
ENH: Add datalib for GFS
ENH: AWS - Update Dockerfile for AWS Lambda to support podpac changes
BUG: Fix unit tests with renamed decorator
BUG: Fix file path sanitization
SMAP_BASE_URLcheck lazy to avoid warning at import
ENH: Remove deprecated caching methods from Node
Initial public release