The default version is very old and hence all the API's does not work
For eg pause_cluster() and resume_cluster() does not work in AWS Glue Python Shell due to this older version
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/redshift.html
Similarly for many other product features.
Additionally, we don't have to Glue internet access by security and hence need a solution based on s3 storage libraries
Which is the best way to upgrade the Python Shell as it seems the lightest weight part of our architecture
Basically, we are using python glue shell as our core workflow engine to asynchronously deisgn our pipeline through boto3 apis
0 comments:
Post a Comment
Thanks