Skip to content

Commit

Permalink
nydus-image: support chunk level deduplication between local images
Browse files Browse the repository at this point in the history
Add the dedup option to nydus-image to support block level deduplication
between local images. A local database cas.db was introduced, which records
all chunks saved locally. Then We can redirect the chunks to those that already
exist locally to avoid duplicate downloads.

There is still some work needs to be done, including:
1. Insert the on-demand downloaded chunk into the database to ensure that
the database stores locally existing chunks.
2. Snapshotter's GC mechanism with the reference relationship of bootstrap
after deduplication.

Signed-off-by: Huang Jianan <[email protected]>
Signed-off-by: xwb1136021767 <[email protected]>
  • Loading branch information
xwb1136021767 authored and xwb1136021767 committed Aug 11, 2023
1 parent f3cdd07 commit 10b2aa6
Show file tree
Hide file tree
Showing 31 changed files with 1,621 additions and 51 deletions.
147 changes: 127 additions & 20 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit 10b2aa6

Please sign in to comment.