Proper way to do of-site backups with history (X time worth of snapshots saved at target)

[MrRinkana]

Hello! I have successfully set up sanoid such that it takes monthly, weekly, daily and hourly snapshots of my server and prunes them to my liking. That was really easy and I'm very thankful for this great tool!

Now I want to backup to a remote server, where a history should also be kept (not just the latest state), lets say three months of snapshots.

I am not sure what the intended or best way to do this is since (according to my understanding, please correct me if I'm wrong):

1. sanoid will not prune @syncoid_ snapshots, so --keep-sync-snap cannot be used as the target would just fill up with snapshots that are never pruned, not to mention they are kept at the source too. Is there a way around this (don't keep multiple on source, prune on target)?

2. One could use sync snapshots, and then let sanoid at the target create @autosnap_ snaphots and manage those. However then those would all be newer than the @syncoid_ snapshot such that next time a sync is initiated the target will need to rollback to the previous snapshot destroying the @autosnap_:s generated on the target.. Maybe --no-rollback is for this use-case but wouldn't everything break if the targets filesystem would for some reason get modified locally in between syncs i.e. creating a branch (can you sync without the dataset being mounted)? It feels weird having other snapshots than those sent as base for the next..

3. One could use --no-sync-snap but then some problems arise:
   3.1. A case when no common dataset exists if the sync gets delayed or takes a long time such that the source deletes the previously synchronised snapshot. Is this best solved by --create-bookmark or --use-hold? It seems bookmarks are not deleted so I would get an absurd amount of bookmarks quite fast considering multiple datasets and frequent sync..
   3.2 Without --include-snapshots=REGEX (which is not in distros for a while) it's a bit uncertain which type (weekly, hourly, frequent etc.) of snapshot that gets sent (since it will be the latest at the time syncoid runs) which makes the target sanoid rules complicated, or?

Sorry for having such many questions in the same post, but they all have to do with understanding how to best get to my goal of having my backup target keeping a history of all the snapshots it got sent during the last X time.

How does one do this the best/correct way?

The use for this..

.. is to protect against unnoticed accidental deletions (the backup-target will keep a longer history than the server, since it has more storage and not all datasets) or in the unlikely case of ransomware/accidental file deletion preceding (and getting synced to the backup target) a catastrophic failure (such as a fire) that destroys the source server file integrity.

[jimsalterjrs]

Syncoid replicates ALL snapshots (which are newer than the latest common snapshot) to the target. All you need to do is run sanoid on the target, using either the hotspare template (if replicating hourly) or the backup template (if replicating daily), so that the sanoid instance on the target prunes old snapshots but does **not** attempt to take new snapshots locally.

[MrRinkana]

Ah, I realise I forgot to specify I wanted to use --no-stream since there's a large amount of temporary files getting created and deleted during the day which would get synced since the source takes a bunch of frequent and hourly snapshots.. In my case the frequent and hourly snapshots can be inflated by upwards 200GiB on a busy day, which is a lot when the target is on a slow link or slow storage..

But maybe still using -I send streams is the answer if combined with a regex filter of the snapshots (--exclude-snaps) to exclude the hourly and frequent snapshots..? The problem with there would be that that will take ages before it would make it into debian as its still in master only

[MrRinkana]

After thinking about it a bit more I have concluded that using -I (and pruning with sanoid on target) as @jimsalterjrs said (by not specifying --no-stream) but using --exclude-snaps=.*(?:frequently|hourly) sufficiently solves the issue of temporary file inflation and is the best way to get the desired snapshot history on the target.

Requires current master, or probably sanoid 2.3+