Option to create cumulative snapshots

[rawtaz]

Output of restic version

restic 0.16.4 (v0.16.4-0-g3786536dc) compiled with go1.22.3 on darwin/arm64

What should restic do differently? Which functionality do you think we should add?

If restic had an option to include files referenced in the previous snapshot (for the same backup set) as well as the ones in the current backup run, it could be used as more of an archiving tool than a backup tool, but with all the awesome features restic currently has such as deduplication, compression, superb integrity checks, robustness and so on.

That is, if snapshot N-1 contains a folder /foo in which there were some files which have later been deleted, and some files have been added to the folder as well, then backing up /foo a second time would create snapshot N which contains/references both the still existing, the new, and previously deleted files.

What are you trying to do? What problem would this solve?

See discussion in https://forum.restic.net/t/need-help-with-restic-backup-configuration/7554 .

Did restic help you today? Did it make you happy in any way?

Yep!

[aneesh-n]

This is an internal PR created for a similar requirement - zmanda#13
It was mainly for the ability to use restic with file watchers for Continuous backups.
This PR avoids having to scan the whole backup folder again, while allowing specify what new files need to be added or removed w.r.t the previous/specified snapshot (when --parent is specified).

"Add new flags --include/-i and --remove for backup command to support delta backups. --include is used when specific backup paths need to be scanned without scanning the whole backup set. --remove is used for removing backed up paths from the backup set. This is useful in combination with file watchers watching for changes to files/folders and calling restic with a batch of changes to create a new snapshot without having to go through the entire tree of the backup set."

Example usage -
restic backup <path> -i <path>/newfile.sh -i <path>/newfile2.sh -i <path>/newdir --remove <path>/removedfile1.sh --remove <path>/removeddir.sh

Any thoughts on this approach?

[rawtaz]

You are describing a different use case. This issue is only about telling restic that the next snapshot should contain what's in the backup set at this point in time, as well as what the previous snapshot for the same backup set contains.

[konidev20]

Hey @rawtaz ,

From what I understood of your use case, you want to maintain the tree of the previous snapshot (--parent snapshot), and add only new files in the repository. You don't want to remove the deleted files?

The --include flag will accept a list of files. Internally, it will preserve the parent snapshot tree and will only append to the tree with new files mentioned in the --include flag. Now, if restic internally is able to figure out the new files, would the same approach help your use case as well right?

[rawtaz]

From what I understood of your use case, you want to maintain the tree of the previous snapshot (--parent snapshot), and add only new files in the repository. You don't want to remove the deleted files?

It is not me who has this use case :) See the referenced forum thread for the original use case description. But yes, the deleted files should not be removed. The point of this issue is basically to have an easy way to make restic archive your files cumulatively.

---

Again, the use case or feature you are talking about is different from the one in this issue.

The cumulative snapshot feature suggested in this issue is just that - adding to what was in the previous snapshot, what has been added or changed since then. That's all. And this without having to add additional list of files, since we're still talking about the same backup set.

[MichaelEischer]

Also related: #4469 and #3405

[xingxiuyi]

Great feature