missing liblzma.so in nodejs environment
#37
Assignees
Labels
bug
Something isn't working
Comments
|
@ferily7 thanks for your issue. What version of nodejs are you using? |
|
Note, the latest version in us-west-2 are https://github.com/lambgeo/docker-lambda/blob/master/layer.json#L557 |
|
I'm using |
|
for Node.js 14 you have to use |
|
Gotcha! I have my environmental variables set up like this in the lambda,
is this the right way to add the environmental variables? |
|
Actually after I changed the version, I get a different error now |
|
yes I'm seeing this too! I never tried nodejs but seem that there is a difference between the docker image used for the build and the one running the lambda 🤷♂️ The nodejs image from aws seems to have liblzma at the right place tho 🤷♂️ |
|
Hmm so does that mean that the library is there then? |
the library is in the official AWS Lambda NodeJS 14x image .... but maybe not in the lambda runtime which is really weird! |
|
Strange... Is there any way I could resolve this issue? |
|
if you could list all the files found in It's always hard to keep a track of what is shipped in the AWS runtime images 😭 |
|
How would I find the list of all the files in |
|
create a lambda function that return the content of |
|
I'm not too sure how to do that... Is that the only way to get the files in |
Here is how I do in it python import json
from pathlib import Path
def lambda_handler(event, context):
p = Path('/lib64').glob('**/*.so')
files = [str(x) for x in p]
print(files)
return {
'statusCode': 200,
'body': json.dumps(files)
}as you can see in ☝️ there is not liblzma in the lambda environment 🤷♂️ ahhhhh if I run
it's interesting that this is not in To Do
|
|
Alright I did it const { exec } = require("child_process");
exports.handler = (event) => {
exec("ldd /opt/bin/ogr2ogr", (error, stdout, stderr) => {
if (error) {
console.log(`error: ${error.message}`);
}
if (stderr) {
console.log(`stderr: ${stderr}`);
}
console.log(`stdout: ${stdout}`);
// TODO implement
const response = {
statusCode: 200,
body: JSON.stringify(stdout),
};
return response;
});
};Only liblzma is missing 🤷♂️ I think I could add it manually in the built but I just hope that other env don't have other missing libs |
vincentsarago
changed the title
liblzma.so in nodejs environment
|
Ah I see so it's missing from the lib then? I'm just wondering if that one is added, will there be any more missing libraries that I will get an error for |
yeah for an unknown reason, liblzma is present in AWS official build images, in python lambda env ... but not in NodeJS lambda env 🤷♂️ It seems to be the only one missing |
|
What is the next step to get this resolved then? |
|
I'm still not 💯 how this has to be resolved. we could compile liblzma ourself but it's really weird that it is missing in the nodejs14 lambda runtime image. Might be worth to open a ticket on AWS |
|
I will see about submitting a ticket to AWS (not sure where for these types of issues). In the meantime, my app ships a layer that just contains the two missing files: extracted from the official NodeJS Lambda image |
|
fixed in latest release of AWS Lambda nodejs16 |
|
🤦 no we still have the issue because the |
|
Any update on this issue? I have been trying to add |
|
@JackTiber I didn't had time yet to work on this. you could try doing #37 (comment) The issue is that the build image has liblzma but the lambda end doesn't for node js. We need to put liblzma.so in the package/layer to fix this. |
|
Thanks @vincentsarago for getting back to me! Yup I actually did that yesterday after finding this thread and it worked with the lambda for |
|
Hi @JackTiber @danielsharvey @vincentsarago , I have also run into this issue. I am a little stuck with trying to implement the solution metioned in #37 (comment). I am having trouble figuring out how to extract the files from the official aws lambda image for |
|
Sorry, missed the earlier comments. I have CDK code which performs this - I will extract and post here. |
|
@arobdog I setup a Lambda layer using a Dockerfile to extract the needed lib files and put them in the expected location. Dockerfile is below for reference. Then I just add this as another layer to the expected runtime in addition to the OSGeo layer and it works. Hope this helps! |
|
@JackTiber I managed to get it working. Thanks for your help :) |
|
I am running into the same issue and even building a layer with I confirmed that both libraries are in But the Edit: in the meantime, I switched to python and it works. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I am using the ARN for a lambda layer (arn:aws:lambda:us-west-2:524387336408:layer:gdal32:3) and adding it to my lambda function, with my lambda function in nodejs. I am trying to use ogr2ogr in the function but I'm getting the following error that is most likely related to the GDAL:
ogr2ogr: error while loading shared libraries: libpcre.so.0: cannot open shared object file: No such file or directory\nI tried to set GDAL_DATA and PROJ_LIB as environmental variables on my lambda function (I just went to Environment Variables on the UI of the lambda function and added the two as environmental variables), but I am still getting the same error. Am I doing something wrong?
The text was updated successfully, but these errors were encountered: