-
Notifications
You must be signed in to change notification settings - Fork 19
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
missing liblzma.so
in nodejs environment
#37
Comments
@ferily7 thanks for your issue. What version of nodejs are you using? |
Note, the latest version in us-west-2 are https://github.com/lambgeo/docker-lambda/blob/master/layer.json#L557 |
I'm using |
for Node.js 14 you have to use |
Actually after I changed the version, I get a different error now |
yes I'm seeing this too! I never tried nodejs but seem that there is a difference between the docker image used for the build and the one running the lambda 🤷♂️ The nodejs image from aws seems to have liblzma at the right place tho 🤷♂️
|
Hmm so does that mean that the library is there then? |
the library is in the official AWS Lambda NodeJS 14x image .... but maybe not in the lambda runtime which is really weird! |
Strange... Is there any way I could resolve this issue? |
if you could list all the files found in It's always hard to keep a track of what is shipped in the AWS runtime images 😭 |
How would I find the list of all the files in |
create a lambda function that return the content of |
I'm not too sure how to do that... Is that the only way to get the files in |
Here is how I do in it python import json
from pathlib import Path
def lambda_handler(event, context):
p = Path('/lib64').glob('**/*.so')
files = [str(x) for x in p]
print(files)
return {
'statusCode': 200,
'body': json.dumps(files)
}
as you can see in ☝️ there is not liblzma in the lambda environment 🤷♂️ ahhhhh if I run
it's interesting that this is not in To Do
|
Alright I did it const { exec } = require("child_process");
exports.handler = (event) => {
exec("ldd /opt/bin/ogr2ogr", (error, stdout, stderr) => {
if (error) {
console.log(`error: ${error.message}`);
}
if (stderr) {
console.log(`stderr: ${stderr}`);
}
console.log(`stdout: ${stdout}`);
// TODO implement
const response = {
statusCode: 200,
body: JSON.stringify(stdout),
};
return response;
});
};
Only liblzma is missing 🤷♂️ I think I could add it manually in the built but I just hope that other env don't have other missing libs |
liblzma.so
in nodejs environment
Ah I see so it's missing from the lib then? I'm just wondering if that one is added, will there be any more missing libraries that I will get an error for |
yeah for an unknown reason, liblzma is present in AWS official build images, in python lambda env ... but not in NodeJS lambda env 🤷♂️ It seems to be the only one missing |
What is the next step to get this resolved then? |
I'm still not 💯 how this has to be resolved. we could compile liblzma ourself but it's really weird that it is missing in the nodejs14 lambda runtime image. Might be worth to open a ticket on AWS |
I will see about submitting a ticket to AWS (not sure where for these types of issues). In the meantime, my app ships a layer that just contains the two missing files:
extracted from the official NodeJS Lambda image |
fixed in latest release of AWS Lambda nodejs16 |
🤦 no we still have the issue because the |
Any update on this issue? I have been trying to add |
@JackTiber I didn't had time yet to work on this. you could try doing #37 (comment) The issue is that the build image has liblzma but the lambda end doesn't for node js. We need to put liblzma.so in the package/layer to fix this. |
Thanks @vincentsarago for getting back to me! Yup I actually did that yesterday after finding this thread and it worked with the lambda for |
Hi @JackTiber @danielsharvey @vincentsarago , I have also run into this issue. I am a little stuck with trying to implement the solution metioned in #37 (comment). I am having trouble figuring out how to extract the files from the official aws lambda image for |
Sorry, missed the earlier comments. I have CDK code which performs this - I will extract and post here. |
@arobdog I setup a Lambda layer using a Dockerfile to extract the needed lib files and put them in the expected location. Dockerfile is below for reference.
Then I just add this as another layer to the expected runtime in addition to the OSGeo layer and it works. Hope this helps! |
@JackTiber I managed to get it working. Thanks for your help :) |
I am running into the same issue and even building a layer with I confirmed that both libraries are in
But the
Edit: in the meantime, I switched to python and it works. |
I am using the ARN for a lambda layer (arn:aws:lambda:us-west-2:524387336408:layer:gdal32:3) and adding it to my lambda function, with my lambda function in nodejs. I am trying to use ogr2ogr in the function but I'm getting the following error that is most likely related to the GDAL:
ogr2ogr: error while loading shared libraries: libpcre.so.0: cannot open shared object file: No such file or directory\n
I tried to set GDAL_DATA and PROJ_LIB as environmental variables on my lambda function (I just went to Environment Variables on the UI of the lambda function and added the two as environmental variables), but I am still getting the same error. Am I doing something wrong?
The text was updated successfully, but these errors were encountered: