Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bottomless: less bugs more robustness #1685

Draft
wants to merge 15 commits into
base: main
Choose a base branch
from
Draft

bottomless: less bugs more robustness #1685

wants to merge 15 commits into from

Commits on Aug 21, 2024

  1. fix wal restore process for situation where S3 has more than 1 page o… 

    …f data (1000+)
    @sivukhin
    sivukhin committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    f88e699 View commit details
    Browse the repository at this point in the history

  2. validate that WAL was transferred completely to the DB after restore 

    - now we are relying on the fact that last SQLite connection will perform checkpoint
  this is fragile because if DB + WAL malformed somehow - SQLite will exit silently

- one way to resolve this issue is to trigger wal_checkpoint(TRUNCATE)
  manually, but this potentially can interfere with bottomless somehow

- so, more robust way to resolve this issue were implemented: we just
  check that WAL was transfered and -wal + -shm files were deleted. If
  no - we abort restore process
    @sivukhin
    sivukhin committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    f41bc58 View commit details
    Browse the repository at this point in the history

  3. add logs in bottomless wrapper

    @sivukhin
    sivukhin committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    d49154b View commit details
    Browse the repository at this point in the history

  4. do not restore backup if we detected issues with it

    @sivukhin
    sivukhin committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    c399ea5 View commit details
    Browse the repository at this point in the history

  5. add simple test to check that DB will not be restored from broken bot… 

    …tomless backup
    @sivukhin
    sivukhin committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    3445e9a View commit details
    Browse the repository at this point in the history

  6. fix test

    @sivukhin
    sivukhin committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    77bf765 View commit details
    Browse the repository at this point in the history

  7. make bottomless more tolerant to errors

    @sivukhin
    sivukhin committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    04c21df View commit details
    Browse the repository at this point in the history

  8. change test

    @sivukhin
    sivukhin committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    42689df View commit details
    Browse the repository at this point in the history

  9. cargo fmt

    @sivukhin
    sivukhin committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    fbfa808 View commit details
    Browse the repository at this point in the history

  10. minor cosmetic changes

    @sivukhin
    sivukhin committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    b63857d View commit details
    Browse the repository at this point in the history

  11. add utils.rs

    @sivukhin
    sivukhin committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    58bd196 View commit details
    Browse the repository at this point in the history

  12. implement optional integrity check and add one more test

    @sivukhin
    sivukhin committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    ca675a7 View commit details
    Browse the repository at this point in the history

  13. fix test

    @sivukhin
    sivukhin committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    f67abef View commit details
    Browse the repository at this point in the history

  14. cargo fmt

    @sivukhin
    sivukhin committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    2feba76 View commit details
    Browse the repository at this point in the history

  15. fix test

    @sivukhin
    sivukhin committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    56eb171 View commit details
    Browse the repository at this point in the history