Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve mapped and head modes. #21

Merged
merged 33 commits into from
Oct 5, 2023
Merged

Improve mapped and head modes. #21

merged 33 commits into from
Oct 5, 2023

Commits on Sep 18, 2023

  1. Incremental read/write for heap mode to reduce memory contention 

    Instead of two large file mappings, use a series of smapp mappings for the copy.
    @greg7mdp
    greg7mdp committed Sep 18, 2023
    Configuration menu
    Copy the full SHA
    e5cf68b View commit details
    Browse the repository at this point in the history

Commits on Sep 25, 2023

  1. Finish implementing the readonly mapped mode.

    @greg7mdp
    greg7mdp committed Sep 25, 2023
    Configuration menu
    Copy the full SHA
    8213c33 View commit details
    Browse the repository at this point in the history

  2. In mapped mode, save only modified pages at exit.

    @greg7mdp
    greg7mdp committed Sep 25, 2023
    Configuration menu
    Copy the full SHA
    c6735eb View commit details
    Browse the repository at this point in the history

  3. Update cicd to use debian:bullseye instead of debian:buster.

    @greg7mdp
    greg7mdp committed Sep 25, 2023
    Configuration menu
    Copy the full SHA
    93cd1d3 View commit details
    Browse the repository at this point in the history

  4. Avoid multiple calls to msync

    @greg7mdp
    greg7mdp committed Sep 25, 2023
    Configuration menu
    Copy the full SHA
    19ee2e0 View commit details
    Browse the repository at this point in the history

Commits on Sep 26, 2023

  1. Use boost interprocess mmap APIs

    @greg7mdp
    greg7mdp committed Sep 26, 2023
    Configuration menu
    Copy the full SHA
    235d956 View commit details
    Browse the repository at this point in the history

  2. Reuse _file_mapping instead of creating a new bip::file_mapping

    @greg7mdp
    greg7mdp committed Sep 26, 2023
    Configuration menu
    Copy the full SHA
    66d3326 View commit details
    Browse the repository at this point in the history

  3. Fix my previous change for flushing the region to disk.

    @greg7mdp
    greg7mdp committed Sep 26, 2023
    Configuration menu
    Copy the full SHA
    832805a View commit details
    Browse the repository at this point in the history

Commits on Sep 27, 2023

  1. Add instance tracker so that we can flush all dbs to disk before cl… 

    …earing the Soft-Dirty bits.
    @greg7mdp
    greg7mdp committed Sep 27, 2023
    Configuration menu
    Copy the full SHA
    4cde714 View commit details
    Browse the repository at this point in the history

  2. Cleanup error cases.

    @greg7mdp
    greg7mdp committed Sep 27, 2023
    Configuration menu
    Copy the full SHA
    4bc07e4 View commit details
    Browse the repository at this point in the history

  3. code cleanup and renaming some members.

    @greg7mdp
    greg7mdp committed Sep 27, 2023
    Configuration menu
    Copy the full SHA
    619ba1f View commit details
    Browse the repository at this point in the history

Commits on Sep 28, 2023

  1. Add missing Boost random dependency (needed in Leap).

    @greg7mdp
    greg7mdp committed Sep 28, 2023
    Configuration menu
    Copy the full SHA
    4dcbb00 View commit details
    Browse the repository at this point in the history

  2. Update boost version

    @greg7mdp
    greg7mdp committed Sep 28, 2023
    Configuration menu
    Copy the full SHA
    219e89b View commit details
    Browse the repository at this point in the history

  3. Remove benchmark from default build.

    @greg7mdp
    greg7mdp committed Sep 28, 2023
    Configuration menu
    Copy the full SHA
    6b4eda4 View commit details
    Browse the repository at this point in the history

  4. Reduce overlap of memory mappings existence.

    @greg7mdp
    greg7mdp committed Sep 28, 2023
    Configuration menu
    Copy the full SHA
    6e1aa5a View commit details
    Browse the repository at this point in the history

  5. Add description for clear_refs_failed error

    @greg7mdp
    greg7mdp committed Sep 28, 2023
    Configuration menu
    Copy the full SHA
    d6c1dcc View commit details
    Browse the repository at this point in the history

Commits on Sep 29, 2023

  1. Merge branch 'main' of github.com:AntelopeIO/chainbase into mapped_an… 

    …d_heap_improvement
    @greg7mdp
    greg7mdp committed Sep 29, 2023
    Configuration menu
    Copy the full SHA
    4a8070e View commit details
    Browse the repository at this point in the history

  2. Remove unused code.

    @greg7mdp
    greg7mdp committed Sep 29, 2023
    Configuration menu
    Copy the full SHA
    c3352cc View commit details
    Browse the repository at this point in the history

Commits on Sep 30, 2023

  1. Add extra test mode mapped_shared.

    @greg7mdp
    greg7mdp committed Sep 30, 2023
    Configuration menu
    Copy the full SHA
    d275422 View commit details
    Browse the repository at this point in the history

  2. Make sure we don't try to use the pagemap feature on platforms wher… 

    …e it is not available.
    @greg7mdp
    greg7mdp committed Sep 30, 2023
    Configuration menu
    Copy the full SHA
    65eefd4 View commit details
    Browse the repository at this point in the history

Commits on Oct 2, 2023

  1. Remove leftover comment not necessary anymore.

    @greg7mdp
    greg7mdp committed Oct 2, 2023
    Configuration menu
    Copy the full SHA
    abc648c View commit details
    Browse the repository at this point in the history

  2. Address PR comments.

    @greg7mdp
    greg7mdp committed Oct 2, 2023
    Configuration menu
    Copy the full SHA
    da2910c View commit details
    Browse the repository at this point in the history

  3. Add another commment.

    @greg7mdp
    greg7mdp committed Oct 2, 2023
    Configuration menu
    Copy the full SHA
    7ae2b7c View commit details
    Browse the repository at this point in the history

  4. Check for db file on tempfs and refuse to start unless in `mapped_sha… 

    …red` mode.
    @greg7mdp
    greg7mdp committed Oct 2, 2023
    Configuration menu
    Copy the full SHA
    e7a9b5a View commit details
    Browse the repository at this point in the history

  5. Add API to flush RW db and convert to RO mapping after snapshot.

    @greg7mdp
    greg7mdp committed Oct 2, 2023
    Configuration menu
    Copy the full SHA
    6cce710 View commit details
    Browse the repository at this point in the history

Commits on Oct 3, 2023

  1. Fix divide by zero in heap mode.

    @greg7mdp
    greg7mdp committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    4ced7af View commit details
    Browse the repository at this point in the history

  2. Remove some unneeded includes.

    @greg7mdp
    greg7mdp committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    44c9a20 View commit details
    Browse the repository at this point in the history

  3. mapped mode: add code to write some pages to disk when available RA… 

    …M becomes scarce.
    @greg7mdp
    greg7mdp committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    4b7cf64 View commit details
    Browse the repository at this point in the history

  4. Address PR comments.

    @greg7mdp
    greg7mdp committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    4ab8944 View commit details
    Browse the repository at this point in the history

  5. Make new node the non-default one (mapped_private)

    @greg7mdp
    greg7mdp committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    173287c View commit details
    Browse the repository at this point in the history

  6. Disable check_memory_and_flush_if_needed() which was not working co… 

    …rrectly as Matt pointed out.
    @greg7mdp
    greg7mdp committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    7ff3038 View commit details
    Browse the repository at this point in the history

Commits on Oct 4, 2023

  1. Address PR comment

    @greg7mdp
    greg7mdp committed Oct 4, 2023
    Configuration menu
    Copy the full SHA
    d928ec5 View commit details
    Browse the repository at this point in the history

  2. Remove unneeded std::cerr message as per PR comment.

    @greg7mdp
    greg7mdp committed Oct 4, 2023
    Configuration menu
    Copy the full SHA
    7817736 View commit details
    Browse the repository at this point in the history