Remove files from suprsync queue?

[dpdutcher]

How do I remove files from the queue that suprsync is trying to move, but no longer exist or are unsuable for some reason?

On my lab smurf server, my ocs-timestream-sync instance of the suprsync agent is crashing with the message

symlink has no referent: "/tmp/tmpya23by3g/16686/crate1slot2/1668634764_000.g3"
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1207) [sender=3.1.3]
2023-10-12T16:28:14+0000 run:0 CRASH: [Failure instance: Traceback: <class 'subprocess.CalledProcessError'>: Command '['rsync', '-Lrt', '--rsh', 'ssh -i /home/cryo/.ssh/id_rsa', '/tmp/tmpya23by3g/', '[email protected]:/data/timestreams']' returned non-zero exit status 23.

Symlinks and the /tmp directory aside, the file 16686/crate1slot2/1668634764_000.g3 is from just about a year ago and is no longer on the server. How can I get suprsync to skip this file?

[jlashner]

For any of the files in the suprsync db, you can set ignore=True and suprsync will skip over it. The easiest way to do it is probably to open sqlite database in the terminal and run something like:

UPDATE supersync_v0
SET ignore=1
WHERE
    remote_path='16686/crate1slot2/1668634764_000.g3'

If socs is installed on the smurf-server, you could also set ignore=True through sqlalchemy

[dpdutcher]

I must be on an older version, because that column doesn't exist for me:

sqlite> .schema supersync_v0
CREATE TABLE supersync_v0 (
	id INTEGER NOT NULL, 
	local_path VARCHAR NOT NULL, 
	local_md5sum VARCHAR NOT NULL, 
	archive_name VARCHAR NOT NULL, 
	remote_path VARCHAR NOT NULL, 
	timestamp FLOAT NOT NULL, 
	remote_md5sum VARCHAR, 
	copied FLOAT, 
	removed FLOAT, 
	failed_copy_attempts INTEGER, 
	deletable BOOLEAN, 
	PRIMARY KEY (id)

Can I instead set failed_copy_attempts for that entry to be larger than my max-copy-attempts value (10)?

[dpdutcher]

Apparently not, that doesn't seem to work. Still get the same error.

[jlashner]

Yeah you gotta update your suprsync and follow instructions here to add the ignore column:

https://github.com/simonsobs/daq-discussions/discussions/27

[dpdutcher]

This also worked (after installing sqlite on my smurf server and using it to open /data/so/dbs/suprsync.db):

sqlite> DELETE FROM supersync_v0 WHERE local_path='/data/timestreams/16686/crate1slot2/1668634764_000.g3';