Error during deployment on AWS Lambda

[tfa346]

Hi,

I'm tryin to run this project on an AWS instance thanks to a nodejs layer. When i launch the function, the new Hero() is working well. However, on the hero.goto step, this error is launched:
"/opt/nodejs/node_modules/@ulixee/hero-core/node_modules/better-sqlite3/build/Release/better_sqlite3.node: invalid ELF header"

Is someone has some kind of workaround about that? Thanks

[mehrdad-shokri]

I tried to to make chrome work in lambda environment but it was no success repos like chrome-aws-lambda didn't work for me as well.(it's updated 3 years ago) so decided to host it on ec2 instead

[tfa346]

I tried to to make chrome work in lambda environment but it was no success repos like chrome-aws-lambda didn't work for me as well.(it's updated 3 years ago) so decided to host it on ec2 instead

Ok, my problem is I need to run a couple of hundred robots simultaneously so i cant use ec2 for that. I will dig for another solution, thanks for your answer!

[mehrdad-shokri]

why do you need hundred robots at the same time? if you want to scrape 100 websites at the same time you can have probably 10 instances and they can probably handle 100 sites at the same time in a reasonable time. but if you figure out a way to run this on lambda please let me know. i would like to know how to do it. im looking into this seems a promising project

[blakebyrnes]

Hi @tfa346, sorry you're having issues. If you're still looking into this, how did you package your app to upload? That error sounds a bit like a problem with the target architecture of lambda vs the packaged files that got uploaded. FWIW, I haven't tried to make this work on Lambda yet. I would imagine the Chrome issue is going to be bigger (as @mehrdad-shokri pointed out). From what I know, you have to include all the packaged files, and they have (or used to have) a hard limit on size.