Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat[MQB]: Do not record last confirm #442

Merged
merged 17 commits into from
Oct 24, 2024

Conversation

bbpetukhov
Copy link
Contributor

@bbpetukhov bbpetukhov commented Oct 7, 2024

Describe your changes
The goal of this optimization is to avoid storing the last Confirm record in the journal file, because the Delete record follows right after this Confirm record, thus the last Confirm record is redundant.

Also since we'll have to store one record less, I decreased from 3 to 2 the default static size of records array.

Previously there were no unit tests for FileBackedStorage. Added 22 unit tests for FileBackedStorage by analogy to InMemoryStorage, since both classes inherit from ReplicatedStorage and their usage scenarios are similar.

Testing performed

  • Manual testing via bmqtool and bmq storage tool to make sure that the last Confirm record is not written. Tested both Priority and Fanout mode (w/ 3 appIds).
  • CI Workflow passed successfully.

Additional context

Priority mode:

Screenshot 2024-09-16 124215
Screenshot 2024-09-16 124237

Fanout mode:

Screenshot 2024-10-08 212644
Screenshot 2024-10-08 212626

@bbpetukhov bbpetukhov force-pushed the do-not-record-last-confirm branch from 6d8e88a to dccafc0 Compare October 7, 2024 10:43
@bbpetukhov bbpetukhov force-pushed the do-not-record-last-confirm branch from 1f0b061 to 14c461b Compare October 8, 2024 18:31
@bbpetukhov bbpetukhov marked this pull request as ready for review October 8, 2024 18:31
@bbpetukhov bbpetukhov requested a review from a team as a code owner October 8, 2024 18:31
bbpetukhov and others added 2 commits October 9, 2024 11:19
Copy link
Collaborator

@678098 678098 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

src/groups/mqb/mqbs/mqbs_filebackedstorage.t.cpp Outdated Show resolved Hide resolved
src/groups/mqb/mqbs/mqbs_filebackedstorage.t.cpp Outdated Show resolved Hide resolved
src/groups/mqb/mqbs/mqbs_filebackedstorage.t.cpp Outdated Show resolved Hide resolved
src/groups/mqb/mqbs/mqbs_filebackedstorage.t.cpp Outdated Show resolved Hide resolved
src/groups/mqb/mqbs/mqbs_filebackedstorage.t.cpp Outdated Show resolved Hide resolved
src/groups/mqb/mqbs/mqbs_filebackedstorage.t.cpp Outdated Show resolved Hide resolved
src/groups/mqb/mqbs/mqbs_filebackedstorage.t.cpp Outdated Show resolved Hide resolved
src/groups/mqb/mqbs/mqbs_filebackedstorage.t.cpp Outdated Show resolved Hide resolved
src/groups/mqb/mqbs/mqbs_filebackedstorage.t.cpp Outdated Show resolved Hide resolved
@bbpetukhov bbpetukhov force-pushed the do-not-record-last-confirm branch from c3e9108 to 92d8f6c Compare October 14, 2024 10:56
@bbpetukhov bbpetukhov requested a review from 678098 October 14, 2024 12:10
Copy link
Collaborator

@678098 678098 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few small comments and good to merge

src/groups/mqb/mqbs/mqbs_filebackedstorage.cpp Outdated Show resolved Hide resolved
src/groups/mqb/mqbs/mqbs_filebackedstorage.h Outdated Show resolved Hide resolved
src/groups/mqb/mqbs/mqbs_filebackedstorage.t.cpp Outdated Show resolved Hide resolved
src/groups/mqb/mqbs/mqbs_filebackedstorage.t.cpp Outdated Show resolved Hide resolved
bbpetukhov and others added 4 commits October 16, 2024 08:09
…ucture in InMemoryStorage unit tests

Signed-off-by: Dmitrii Petukhov <[email protected]>
Update comment on number of records

Co-authored-by: Evgeny Malygin <[email protected]>
Signed-off-by: Dmitrii Petukhov <[email protected]>
Fix typo in comment

Co-authored-by: Evgeny Malygin <[email protected]>
Signed-off-by: Dmitrii Petukhov <[email protected]>
@bbpetukhov bbpetukhov force-pushed the do-not-record-last-confirm branch from 93bc887 to ca16e5f Compare October 21, 2024 15:48
@bbpetukhov bbpetukhov force-pushed the do-not-record-last-confirm branch from 9b64aef to 4cf7d27 Compare October 24, 2024 15:01
@678098 678098 merged commit a57b45c into bloomberg:main Oct 24, 2024
30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants