Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error with timescaledb, function not found when deployed to AWS lambda #76

Open
300LiterPropofol opened this issue Nov 3, 2022 · 1 comment

Comments

@300LiterPropofol
Copy link

Hello!

This has been an awesome package and has solved so many issues for me until now.

Currently I am using an AWS lambda which needs psycopg2 as a required package to connect to a timescaledb in the cloud.
I always encounter this error and my code can not work:

[ERROR]	2022-11-03T09:38:04.637Z	d57f065c-6ac4-4c95-8c2e-7e6c6efd89e8	Error during turning states table into hypertable: function create_hypertable(unknown, unknown, if_not_exists => boolean, chunk_time_interval => interval) does not exist
LINE 1: SELECT create_hypertable('states', 'last_updated', if_not_ex...
               ^
HINT:  No function matches the given name and argument types. You might need to add explicit type casts.

[ERROR]	2022-11-03T09:38:04.640Z	d57f065c-6ac4-4c95-8c2e-7e6c6efd89e8	Error during set retention date to hypertable: function add_retention_policy(unknown, if_not_exists => boolean, drop_after => interval) does not exist
LINE 1: SELECT add_retention_policy('states', if_not_exists => TRUE,...
               ^
HINT:  No function matches the given name and argument types. You might need to add explicit type casts.

My code is as below:

convert_hypertable_sql = f"SELECT create_hypertable('states', 'last_updated', if_not_exists => TRUE, chunk_time_interval => INTERVAL '1 day')"
set_rention_sql = F"SELECT add_retention_policy('states', if_not_exists => TRUE, drop_after => INTERVAL '7 days')"
cursor.execute(convert_hypertable_sql)
cursor.execute(set_rention_sql)

I tried the same code when running locally in a terminal and it works without any problem, can create the hypertable and the retention policy in my timescale db in cloud.
I also tried pip3 install psycopg2-binary -t ~/some_local_folder and paste all installed folders to my lambda function directory and deploy to AWS Lambda together, but that didn't work either and gave the same error.

I wonder is this create_hypertable and add_retention_policy SQL something specifically not supported by AWS lambda platform? I couldn't find any related reports online. It will be really appreciated if you can shine some lights on this.
Thank you very much!

@jakebelman
Copy link

I wasn't able to resolve the issues I was having with this, but I was able to set up psycopg2 using klayers.

Klayers is a compilation of python packages built for lambda. They're given as ARNs - to add them, you can scroll to the bottom of your Lambda function in the Lambda console to Layers, click 'Add Layer' in the top-right, select 'Specify an ARN', then paste the ARN for the package you want.

You can find them in this repo: Klayers Repo

Or you can get a simple list by pasting the API link in your browser:
https://api.klayers.cloud//api/v2/{YOUR PYTHON VERSION}/layers/latest/{YOUR AWS REGION}/html
Enter the version of python in the form p0.0 (e.g. p3.8) and the region in the aws shortform (e.g. eu-west-1).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants