Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DeleteAll fails at small scale #7

Open
randomuserid opened this issue Jul 20, 2024 · 0 comments
Open

DeleteAll fails at small scale #7

randomuserid opened this issue Jul 20, 2024 · 0 comments

Comments

@randomuserid
Copy link
Member

randomuserid commented Jul 20, 2024

Around 6k alerts and around 300 entities, or possibly less, the utils.DeleteAll(session) procedure fails with the oom errors below. I have experimented with increasing memory allocations to no avail. I think the best option may be to use apoc to delete in batches like this;

CALL apoc.periodic.iterate(
  'MATCH (n) RETURN n',
  'DETACH DELETE n',
  {batchSize:1000}
)

This is working for me on a local Windows instance. IDK how to install it on our Linux instance, it needs to go in a plugins folder? https://neo4j.com/labs/apoc/5/installation/ - I am heads down atm testing the entity fields and adding metadata

 run main.go
Driver =  &{0xc000132090 0xc00011ea10 0xc000132120 {0 0} 0xc000022250 driver 1 0x106b040}
Session =  &{0xc00011ea10 0 []  0xc000132120 0xc000022250 <nil> false <nil> 0xb88a20 0xbcf860 sess 1 0x106b040 1000000000}
Processing JSON data 100% [##################################################] (2459/2459) Data imported successfully.

go.exe" run main.go
Driver =  &{0xc000132090 0xc00011ea10 0xc000132120 {0 0} 0xc000022250 driver 1 0xacb040}
Session =  &{0xc00011ea10 0 []  0xc000132120 0xc000022250 <nil> false <nil> 0x5e8a20 0x62f860 sess 1 0xacb040 1000000000}
2024/07/20 14:58:39 Failed to run queryServer error: [Neo.TransientError.General.MemoryPoolOutOfMemoryError] The allocation of an extra 2.0 MiB would use more than the limit 716.8 MiB. Currently using 716.0 MiB. dbms.memory.transaction.total.max threshold reached
exit status 1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant