For a status indicator, I would like to know when the local device has shared all it’s changes with at least one other node in the network. From my understanding, this would mean that all the current file blocks in the global model, that are on the local device, are also at at least one other device in the cluster.
This is something that has come up in various threads before, but hasn’t lead to a fully applicable solution yet;
- Upload status API events?
- How to ensure changes get propagated before shutdown?
- Is there a finishing flag or auto close
- Check whether file has been 'uploaded'?
- Proper way for "Sync complete" notification and actual idle detection
- How to check for sync completion with the api?
In my specific setup, device A (laptop) and B (desktop) aren’t always online at the same time, but device C (server) is. We can also assume that if a device is online, it can always connect to all other online devices in the network, and we don’t have to account for any other devices, that might not be visible to all others.
Looking at the device completion api (/rest/db/completion), we can query how in-sync remote devices are with the global model, but we get no insight in where they could get the data they are missing.
Scenario 1:
- Device A changes the folder (10% difference)
- Completion:
- A: 100%
- B: 90%
- C: 90%
As A is the only device at 100%, it knows it has data that still has to be pulled by B & C
Though because B & C are pulling different blocks from A, they can also start to pull blocks from eachother, so there is a point in time where they are <100% complete, but have enough pieces together to finish synchronizing without A.
In this case, I would be fine with the indicator ignoring that possiblity, and as a safeguard, only showing it’s finished when at least one of the others reaches 100%.
We can simplify this to showing 100% - max(B, C) as the percentage of data that still needs to be synced from A.
Scenario 2:
- Device B changes the folder (10% difference)
- Completion:
- A: 90%
- B: 100%
- C: 90%
A and C will then start pulling those changes and move towards 100%, but during this, because A can see B is at 100%, A can know it there’s no need for it to remain online Thus the indicator can show A has some changes to pull in, but has no unique data to provide to the cluster.
Scenario 3:
- Device A changes the folder (10% diff)
- Device B changes the folder (10% diff)
- Completion:
- A: 90%
- B: 90%
- C: 80%
This is where it starts to become a lot more complicated
I guess by assuming that whatever percentage A is missing, is also what C is missing from B, so 80% + (100% - 90%) is what we still need to send to C?
These approximations might sorta work but I feel like there will be weird (edge)cases where this really misrepresents the situation.
From my understanding of the protocol, this is something we could calculate more accurately by cross-checking our index, the global model, and index info sent from remote devices.
Taking a list of all the blocks in our local index, removing all of the blocks that are also advertised by any of the other peers. I guess this is already stored in the LevelDB tables to some degree, but that doesn’t really support (read-only) access while Syncthing is active. The (Debug) API also doesn’t seem to provide information that’s detailed enough to track this. Am I missing something? I went through the documentation and the forum fairly thoroughly.