Getting JavaScript memory leak error

[ragermac]

Description

Possible solution

[c0sc0]

can confirm

<--- Last few GCs --->

[12880:0000019DFD18BC20] 3530069 ms: Mark-sweep 2046.1 (2061.7) -> 2044.7 (2060.3) MB, 39.7 / 0.0 ms (+ 0.0 ms in 1 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 40 ms) (average mu = 0.150, current mu = 0.018) finali[12880:0000019DFD18BC20] 3530111 ms: Mark-sweep 2045.7 (2061.3) -> 2045.6 (2062.3) MB, 40.6 / 0.0 ms (average mu = 0.105, current mu = 0.050) allocation failure scavenge might not succeed

<--- JS stacktrace --->

==== JS stack trace =========================================

0: ExitFrame [pc: 00007FF61097DD6C]

Security context: 0x009739000921
1: getData [00000165A8242B99] [H:\git\nvidia-snatcher\node_modules\ws\lib\receiver.js:~337] [pc=000001D507DEE92D](this=0x015c1a97f951 ,0x015c1a97fa91 <JSBoundFunction (BoundTargetFunction 000003E02EB93DD1)>)
2: socketOnData [0000005FF7AC0B29] [H:\git\nvidia-snatcher\node_modules\ws\lib\websocket.js:~871] [pc=000001D...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
1: 00007FF60FDE0C3F napi_wrap+126287
2: 00007FF60FD7DF76 v8::base::CPU::has_sse+34950
3: 00007FF60FD7EC46 v8::base::CPU::has_sse+38230
4: 00007FF6105A803E v8::Isolate::ReportExternalAllocationLimitReached+94
5: 00007FF61058F791 v8::SharedArrayBuffer::Externalize+785
6: 00007FF6104567FC v8::internal::Heap::EphemeronKeyWriteBarrierFromCode+1468
7: 00007FF610461B40 v8::internal::Heap::ProtectUnprotectedMemoryChunks+1312
8: 00007FF61045E634 v8::internal::Heap::PageFlagsAreConsistent+3188
9: 00007FF610453D43 v8::internal::Heap::CollectGarbage+1283
10: 00007FF6104524D4 v8::internal::Heap::AllocateExternalBackingStore+1396
11: 00007FF61047B888 v8::internal::Factory::NewRawOneByteString+216
12: 00007FF61047EA32 v8::internal::Factory::NewStringFromUtf8+130
13: 00007FF6105A494A v8::String::NewFromUtf8+298
14: 00007FF60FCE978F node::tracing::TraceEventHelper::SetAgent+40943
15: 00007FF60FD95F95 v8::internal::Malloced::operator delete+2021
16: 00007FF61097DD6C v8::internal::SetupIsolateDelegate::SetupHeap+44124
17: 000001D507DEE92D
npm ERR! code ELIFECYCLE
npm ERR! errno 134

[jef]

What commit are you guys on? This problem I believe was solved, but may have been wiped.

[c0sc0]

What commit are you guys on? This problem I believe was solved, but may have been wiped.

Wow, could've sworn I was up to date but I'm 36 commits behind 🤡

[jef]

Gonna close for now unless this continues to be a problem on latest main.

Thanks!

[amarshall]

I still get OOM on fd294d2; I’m using low bandwidth mode (mentioning since previously ad blocker seems to have been an issue). Can open a new issue if you’d like. I work around it by configuring the container to restart on failure, but obviously that’s not ideal.

[jef]

I'm also getting OOM from time to time. Have to look more into it. Probably some pages not closing overtime.

[FrankFioretti]

I, too, am on the latest commit, am using low bandwidth mode, and got the exact error above.

[jef]

Yeah, definitely an issue. Probably something not getting closed. I'll look into it more.

[EchoJoker]

Been running into this issue recently. Happens around 20-ish hours after I start the script. I attached are the logs and my current configuration.

[jef]

@EchoJoker I updated your comment and remove the log, as it contained personal information. Thanks for the log though and will look into it!

[EchoJoker]

@jef It did? I thought I modified all of that out. Eh oh well, hopefully, it helps.

[LuPro]

I'm also getting some fairly consistent memory leaks - it did crash once or twice a while ago but I unfortunately forgot to store the log.
I'm running streetmerchant in a ubuntu server container in a proxmox hypervisor and gave it 2GB of RAM, right when starting the program I'm using around ~400MB RAM but over time it fills my 2GB very consistently - it can run for about ~9 hours before getting close to the limit:

That's RAM overview over one hour - the high part in the beginning was while it was running for a while, then killed it (drop) then started it again (jump to ~400M) and then slowly rising

here the overview over the last ~day

again, this is a clean ubuntu container, there is absolutely nothing else running on it besides street merchant and it's freshly installed 3 days ago (though it is ubuntu server 18.04lts, have been too lazy to download the 20.04lts)

for now I killed the program regularly and restarted it to "reset" ram usage, but I can leave it running to get another crash out of it to see if I can get some other log data for you all

Edit: It crashed at ~1.7GB ram usage (for the entire system), got a very similar error message as posted above by tcosculluela
Edit 2: Now 9 hours later it ran out of RAM again (so about 9 hours for filling ~1.5GB of ram)

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days

[jef]

Is this still an issue?

[LuPro]

I haven't used the bot in a while (since I got the part I needed already) but updated everything and spinned it up again - it's now been running for ~30 minutes and it seems the issue is resolved, at least for me - or at the very least memory is leaking way slower - I feel like there's an increase of maybe 10-30MB in RAM usage since start, but that could just as well be normal variation in operation - I'll update this comment once it has run for longer and I can say something with more confidence.

Edit: 2h in memory still seems to be stable.

Edit 2:

There still seems to be a slow rise in memory use (Started at 490MB, now 19h later is at 640MB), so some sort of leak still seems to persist, but it's way better than before.

[jef]

Thanks for looking into that. What program are you using by the way?

Perhaps I can do a trace of objects that are getting kept in store.

Thanks!

[LuPro]

The memory usage graph comes from the hypervisor (Proxmox Linux) that the Ubuntu Server VM with streetmerchant is running on - I'm running streetmerchant without docker, so just the npm run start, so for any kind of stack traces any kind of tool that runs in ubuntu should be available if you want me to look into something. I'll try for now to let the current instance fill up RAM and check what errors it throws once that happens (that will probably take a while since it's filling so slowly now)

[LuPro]

It seems after about 2 weeks of running, something odd has happened with the process - haven't checked in recently but it seems it's stuck at fully using 1 CPU core with almost no change in RAM usage (and it's not polling any stores anymore, so no network usage):


Gonna see if I can find the program's output, but since it's running mostly unsupervised on a server and I'm not the biggest server maintenance pro I'm not entirely sure where to look - Will try to look into things soon, though the next few days are pretty busy for me.

[jef]

This is interesting. If you find something, I'd love to know! Thanks for keeping this updated.

[LuPro]

Come to think of it, I did have an internet outage for a while a few days ago, it could coincide with streetmerchant getting stuck in Limbo - I'll keep you updated when I find time to investigate.

[jef]

No rush, I don't see this happening too often.

[LuPro]

Looked into it a bit, unfortunately it seems I can't get the terminal output back and it seems that whatever crashed/softlocked the program also stopped it from writing a log file (the newest log file is from 2021-01-06, which was when I failed at starting streetmerchant for further investigation)

Can't get it to run now on the newest commit - something about downlevelIteration compiler flag not being set, managed to fix that with adding that in the tsconfig.json (new issue? could be just on my end, just had to update npm and some stuff, could be something broke), however there's another issue where it can't read a file (tsconfig-google.json) and since my time is still kinda limited I can't check that out further. Will keep you posted if something develops