New mapped_private mode: avoid crash by flushing dirty pages when memory pressure gets high. #1725
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves #1721.
The Linux kernel assigns and maintains an Out of Memory (OOM) score on a per-process basis, The valid range is from 0 (never kill) to 1000 (always kill), the lower the value is, the lower is the probability the process will be killed.
We track this score for Leap, and when the score gets high, flush the dirty pages of our private mapping to disk, lowering the Leap process OOM score, and thus preventing the system from killing our process.
Currently, we trigger the flush() for an
oom
value of980
, which is pretty hard to reach on a system which has more memory than the state database working set (about 20GB for EOS). I tried launching a bunch of programs gobbling memory whilenodeos
was running, but these programs were killed by the OS whilenodeos
continued working happily, even with a loweroom_score
threshold (I tried900
and800
).This PR makes leap
mapped_private
mode function in a somewhat optimal mode, using as much memory as possible for its chainbase db in order to preventing constant writebacks to disk, while still doing occasional writebacks (to avoid crashing) on systems with limited RAM.