Use diskcache for caching ProtocolDAGResults in the Alchemiscale client
#271
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* The default Disk used by diskcache uses pickle when storing python objects. Instead, we are now storing byte arrays. Depending on the size of the byte array, this is either stored in the SQLite3 DB or or as a separate file if it's too large (>32 kb by default). * A test has been added that checks the hits and misses when pulling PDRs using the get_transformation_results method. The in-memory LRU cache is cleared manually for accurate stats.
New objects supported: * Transformations * AlchemicalNetworks * ChemicalSystems * Generally anything that can be a KeyedChain
* With known cached results, corrupt the values and make sure the user is warned that there was a problem with deserialization and that a new result will be downloaded. * Lowered the cache size limit for tests to avoid running out of space
* Removed unsused imports
The AlchemiscaleBaseClient now determines the cache directory when one is not specified directly (i.e. a None is provided to the AlchemiscaleBaseClient constructor). When a path to this directory is provided, it must be a string or pathlib.Path object. The logic for this operation lies in the `AlchemiscaleBaseClient._determine_cache_dir` method, which can raise a TypeError on invalid input. The `cache_size_limit` is now verified within the constructor to be >= 0. If it is not, then a ValueError is raised. New tests have been added for the above changes: * Negative cache_size_limit: checks for constructor-raised ValueError with a meaningful message. * cache_directory is None: checks output of the underlying _determine_cache_dir method with and without the XDG_CACHE_HOME environment variable. If we test it with the client constructor, the directory is made automatically, which we don't want in the tests as it may touch real data. * cache_directory is not None, str, or Path: Check that the constructor raises a TypeError with a meaningful message.
ianmkenney
changed the title
diskcache for caching ProtocolDAGResults in the Alchemiscale clientdiskcache for caching ProtocolDAGResults in the Alchemiscale client
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Addresses, in part, #58. Using the diskcache library, we can cache
ProtocolDAGResults client-side and reduce the number of API calls for calculating free energy differences.