builder: introduce chunkdict save subcommand #1305
Conversation
cslinwang
requested review from
liubogithub,
changweige and
adamqqqplay
and removed request for
a team
|
@cslinwang , a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/75718 |
Codecov Report
@@ Coverage Diff @@
## master #1305 +/- ##
==========================================
+ Coverage 46.31% 46.39% +0.07%
==========================================
Files 122 123 +1
Lines 38077 38557 +480
Branches 38077 38557 +480
==========================================
+ Hits 17635 17888 +253
- Misses 19518 19698 +180
- Partials 924 971 +47
|
|
@cslinwang , the title has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/75719 |
|
@cslinwang , the title has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/75720 |
|
@cslinwang , The CI test is completed, please check result:
Sorry, your test job failed. Please get the details in the link. |
||||||||||||||||||||||||
|
@cslinwang , The CI test is completed, please check result:
Sorry, your test job failed. Please get the details in the link. |
||||||
|
@cslinwang , The CI test is completed, please check result:
Sorry, your test job failed. Please get the details in the link. |
||||||
|
/retest |
|
@Desiki-high , the test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/75722 |
|
@Desiki-high , The CI test is completed, please check result:
Congratulations, your test job passed! |
||||||||||||||||||||||||
There was a problem hiding this comment.
Thanks for your contribution. This feature looks interesting, but maybe it could be more closely integrated with previous work #956 ?
I've made some suggestions to your code. Also, some unit tests need to be added and the docs/nydus-image.md document updated. Thanks in advance.
imeoer
changed the title
|
@imeoer , the title has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/76031 |
|
@cslinwang , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/89153 |
cslinwang
force-pushed
the
linwang-dev
branch
2 times, most recently
from
06d7c9a
to
16a1609
Compare
|
@cslinwang , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/89157 |
|
@cslinwang , The CI test is completed, please check result:
Congratulations, your test job passed! |
||||||||||||||||||||||||
|
@cslinwang , The CI test is completed, please check result:
Congratulations, your test job passed! |
||||||||||||||||||||||||
|
@cslinwang , The CI test is completed, please check result:
Congratulations, your test job passed! |
||||||||||||||||||||||||
|
@cslinwang , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/89524 |
|
@cslinwang , The CI test is completed, please check result:
Congratulations, your test job passed! |
||||||||||||||||||||||||
Updates 'Cargo.toml' and 'Cargo.lock' files to include the 'rusqlite' package. Enabling interaction with SQLite databases in the project. Signed-off-by: Lin Wang <[email protected]>
Add functionality to store chunk and blob metadata from nydus source images. Use the 'nydus-image chunkdict save' command with the '--bootstrap' option followed by the path to the nydus bootstrap file (e.g., '~/output/nydus_bootstrap') Signed-off-by: Lin Wang <[email protected]>
|
@cslinwang , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/89672 |
|
@cslinwang , The CI test is completed, please check result:
Congratulations, your test job passed! |
||||||||||||||||||||||||
Relevant Issue (if applicable)
N/A
Details
These commits introduce the following changes:
Background: In high-density deployment scenarios, the issue of redundant storage and distribution of chunks in Nydus images poses significant challenges. This problem leads to inefficient utilization of storage space and excessive consumption of bandwidth resources. To address this problem, it is crucial to design an effective chunk deduplication mechanism. This entails conducting in-depth research on chunk deduplication algorithms and developing solutions that enable deduplication across different business contexts, within the same business, across different layers, and within the same layer. By implementing comprehensive chunk deduplication strategies, we can significantly optimize storage utilization and enhance bandwidth efficiency in high-density deployment environments.
Commit: "cargo: Add rusqlite package to dependencies"
Commit: "nydus-image: Store chunk and blob metadata"
Types of changes
What types of changes do your commits introduce? Put an
xin all the boxes that apply:Checklist
Go over all the following points, and put an
xin all the boxes that apply: