Bandwidth isn’t great. It has taken months and will take another month or two for the initial sync to finish.
Doing a physical transfer, copying files to a disk in location A, and location B (150km) is actually quite practical. Some may recognize Backblaze Fireball, or Restore by Mail
The sync is partial: one folder has 1.3TiB/1.9TiB transferred.
It’s inefficient to copy over everything in loc A to an external (well internal) drive. It would take noticeably more time and drive space. All data would have to be re-indexed, hashed at loc B as well.
Could I tell Syncthing at loc A to:
Copy everything (optionally select folders) unsynced on loc B to directory /run/media/…
Inform loc B that we did a physical transfer. loc B will not change the current unsynced, but will sync any new changes.
The files, that were copied over will not be synced over the network, but any new changes since the transaction of copying loc A to an external drive will be synced over the network.
At loc B:
Allow to dismiss the sync lock (in case the external drive never arrives)
Inform loc B, that an offline pigeon drive is available at directory /run/media/…
The directory already contains information on what and how it has. loc B will copy as much as it can use from the drive.
The sync is partial: one folder has 1.3TiB/1.9TiB transferred.
It’s inefficient to copy over everything in loc A to an external (well internal) drive. It would take noticeably more time and drive space. All data would have to be re-indexed, hashed at loc B as well.
Besides wasting time, and drive space (weight and money), you also have to tell Syncthing to not sync the current unchanged, but sync new changes.
Hacking something together with rclone union (listing loc B, and writing to ext drive) could work with rsync (or well, rclone then).
Though, this is not user friendly, can’t be relied on (in case of a time limit, you have to have it setup and tested earlier).
I’m mostly asking if this is:
worth the codebase maintenance
wanted by some other folk
a major change, loads of code, implementation time vs bodgeing stuff together
Additionally, from my experience, doing any move outside syncthing is 50/50 whether syncthing accepts it, or nukes current data,
and starts the sync from scratch (either nuking loc b data, what is fine; but usually loc A data, which means (residential, so it’s worse) it’s going to upload).
My protocol has been:
open loc a dashboard, check if it is synced
change loc b to receive-only
move the pigeoned files to loc b’s folder
have it wanting re-sync (not file permissions, doesn’t seem to be modtime either)
well let’s just assume it worked this time
wait until scanning-hashing completes (no changes allowed as it’s receive-only¹)
If any locally changed files are present (could be none, or thousands²)
Try to resolve them, or revert fully.
Revert loc b to normal sync.
Merge any local changes (¹) made meanwhile.
¹ any changes would be lost in the potential ², it’s more work to reverse-engineer what changes you actually made, than keeping a separate off-syncthing folder meanwhile.
The entire part about moving data manually to another location and adding it to a folder, thus not needing to sync actual data, already works. If I am understanding correctly, you are asking for two new behaviours:
Be able to transfer not only data, but also Syncthing metadata offline. Such that the receiving device doesn’t need to scan and hash the data. That sounds like a variant of https://github.com/syncthing/syncthing/issues/4488
Be able to somehow indicate on the sending side, that some sub-paths will be transferred out-of-band. I.e. the receiving side should not try to sync those paths but patiently wait for them to be added manually, locally. That’s a lot of changes (including protocol), for a very niche use-case - that’s not likely to happen. You can ignore them on the sending side or do the transfer in a separate folder.