Implement std::pmr based memory tracking. #4683
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This pull request has been linked to Shortcut Story #36075: Consolidation: allow experiments for writer profiling.. |
davisp
force-pushed
the
pd/sc-36075/implement-pmr-based-memory-tracker
branch
5 times, most recently
from
7b4785e
to
f5a0378
Compare
davisp
force-pushed
the
pd/sc-36075/implement-pmr-based-memory-tracker
branch
2 times, most recently
from
7228426
to
08c12d4
Compare
abigalekim
force-pushed
the
pd/sc-36075/implement-pmr-based-memory-tracker
branch
from
7676d6e
to
2b124b5
Compare
KiterLuc
reviewed
Feb 8, 2024
KiterLuc
changed the title
KiterLuc
reviewed
Feb 9, 2024
| } | ||
| } | ||
|
|
||
| // Thread start logic mirrored from the ThreadPool. |
There was a problem hiding this comment.
This code is now "copy pasted" 5 times... We should implement a common method for this. Note that I'm opened for a follow up story that we do next iteration.
tiledb/common/memory_tracker.cc
Outdated
| out.open(filename_.value(), std::ios::app); | ||
|
|
||
| if (!out) { | ||
| LOG_ERROR("Error opening MemoryTrackerReporter file: " + filename_.value()); |
There was a problem hiding this comment.
Same comment here as above for throwing.
There was a problem hiding this comment.
This one I'm slightly less inclined to turn into an exception. The first reason is that throwing an exception here just translates into a std::terminate since it's not handled by the thread. Secondly, blowing up because of things changing out from under us at runtime could be a weird behavior.
However, I'll freely admit to writing this in haste with just a goal of getting something working. I think a better approach would be to handle any errors that arise more gracefully and then handle resuming log writes when/if the file becomes writable again. Which is to say, keeping these as log messages but trying to resume writing when we're able is likely better than crashing the entire process.
There was a problem hiding this comment.
Let’s file a follow up story to clean this up if needed…
There was a problem hiding this comment.
I kept the log messages but added the logic to continually attempt re-opening the log instead of just silently closing the reporter on first error.
This change is just to save on typing since we'll be referring to memory types quite often it just means we can type `MemoryType::RTREE` instead of `MemoryTracker::MemoryType::RTREE`.
This change helps avoid life time issues of raw pointers to MemoryTrackers since we now require a MemoryTracker for the entire lifetime of the tdb::pmr::vector class.
This change adds a new vector type for use when tracking memory usage within TileDB. This new class is a very thin sub-class of the std::pmr::vector with all of the constructors and assignment operators updated to require an allocator argument rather than making them optional. This will allow us to progressively instrument the TileDB codebase without making some common mistakes of forgetting to pass the required memory tracking references when adding them.
For now we'll juse use the default memory resource while we make sure that translations aren't going to be insanely difficult with the new memory resource constructor argument.
This commit changes a single member variable on the FragmentMetadata class to use the new tdb::pmr::vector type and includes all of the related fixes for updating the new memory tracking requirements.
These particular member variables were chosen because these are the only variables tracked by the existing MemoryTracker implementation. Once I implement the counting and switch the implementations it will break a decent number of unit tests because the new PMR implementations track all allocations, not just a calculation of what we logically need. In other words, we're not currently keeping an accurate count of memory, and getting more accurate counts breaks the existing tests.
This splits up `MemoryType::MIN_MAX_SUM_NULL_COUNT` into four separate types for better specificity.
This adds a new class that inherits from our `tdb::pmr::memory_resource` definition. Instances of this class can be passed to `tdb::pmr::vector` constructos to enable memory tracking at runtime.
This is to distinguish between different instances of MemoryTracker in a given context. For instance, during consolidation we'll end up with five. One for the read array, read query, write array, write query, and one for the consolidator itself.
This allows us to track memory used by instances of Query which include the bulk of the data in RAM in the form of filtered and unfiltered data in tiles.
This adds a new MemoryTrackerManager to the ContextResources. All MemoryTrackers should be created through this API. Eventually I'll probably make MemoryTracker constructors private so that we force this condition. However, that would currently make the test changes fairly verbose so I'm avoiding it for now to reduce the size of this already decently large diff.
This generates a JSON representation of all the currently active MemoryTracker instances.
This adds a new MemoryTrackerReporter. If configured, this reporter starts a background thread that writes a JSON string to the configured file once a second. To configure the filename, just pass a config value for `sm.memory.tracker.reporter.filename` when creating the context. This configuration key is purposefully undocumented so that we aren't creating a public API contract as the contents of the JSON blobs will change without warning.
Avoid using iterators while modifying a vector that's being iterated. I must have read the docs incorrectly on which pointers are invalidated when calling erase while traversing the iterator. Regardless, using traditional while loop with indices makes the logic a bit more clean anyway, so it's an all round win.
A common first issue when adding new memory tracking was to create new instances of MemoryTracker which should happen very rarely in test code. If we create new instances outside of a MemoryTrackerManager, these instances will not be included in the memory reporting. The correct solution should almost always be to use an existing memory tracker. Some common trackers to use are on both the Array and Query classes. In cases where neither of those is appropriate, it may be time to start considering a new instance of a memory tracker by calling ContextResources::create_memory_tracker(). Cases where we may need to create new instances include the usage of instrumented classes outside of an Array or Query. I.e., when creating a new Array, there's no Array instance which means we'll have to create a new ARRAY_CREATE (or some other name) MemoryTracker::Type.
This allows the memory tracking helpers to be used in unit tests.
Spotted by Luc in code review.
davisp
force-pushed
the
pd/sc-36075/implement-pmr-based-memory-tracker
branch
from
3d380f0
to
d6c74f5
Compare
davisp
force-pushed
the
pd/sc-36075/implement-pmr-based-memory-tracker
branch
from
d6c74f5
to
0249685
Compare
KiterLuc
approved these changes
Feb 12, 2024
5 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This implements memory allocation tracking by using the
std::pmrinterface introduced in C++17.TYPE: FEATURE
DESC: Add std::pmr based memory allocation tracking.