Let’s sketch out something that could be sent between devices to inform others about what they’re doing. The purpose is to be able to show more accurate descriptions in the GUI about what other devices are doing. For other consumers it would be exposed as events. The trade off is that we still want to maintain device privacy to some extent, so should consider what to send on a need-to-know basis. Here’s one suggestion to start with, in simplified pseudo-protobuf syntax…
We send one message, always containing the full, latest update. (No need to piece together a picture from multiple update messages, such as one per folder etc.)
This is a new message type, so ignored by old devices who don’t understand it. Backwards compatible.
message DeviceStatus {
// Total input and output rate for the device, in kilobits per
// second. Would replace the rates we show per device today, or
// complement them perhaps...
int32 in_kbps
int32 out_kbps
// For each folder shared with the peer device
repeated FolderStatus folders
}
message FolderStatus {
string id
FolderState state
float32 completion // 0.0 - 1.0
}
enum FolderState {
// We are happy with the current state of the folder, nothing
// to do.
IDLE
// We are bringing the folder up to date.
SYNCING
// We would like to be SYNCING, but there is a problem (out of
// space, folder marker missing, etc.)
STOPPED
// The folder is paused by the user. Maybe? Or maybe such
// folders are not present at all, because they were marked as
// paused in the ClusterConfig and there is no need to repeat
// it... This could potentially become the new mechanism for
// announcing that a folder is paused, but that might be out of
// scope for the first implementation.
PAUSED
// The folder is being scanned. But - this might make the
// protocol very chatty and is not super interesting to the
// other side, so perhaps should not be announced?
// Or we could give guidance that we don't send updates more
// than x times per time unit, to dampen this...
SCANNING
// A generic "I'm busy doing something not directly related to
// syncing" state. Today this might mean "sorting indexes" and
// things like that, other implementations might have other
// reasons.
PROCESSING
}
A device that is up to date will advertise IDLE and completion=1.0 for all its folders.
When there is a problem that prevents getting fully up to date (permission issue on a file or similar) we’ll see it cycling between SYNCING, completion<1.0 and IDLE, completion<1.0.
When the folder is totally screwed we’ll see STOPPED and maybe completion=0.0 or whatever completion percentage we had in the index before? The value is going to be out of date or bogus anyway as the device apparently can’t fully see the actual contents of the folder.
The percentage we show in the UI is 0-100, so using a float and again wondering if 0.999(9) actually means 1 sounds like a trap?
The rates we show per device are relative to how much network that device is using from us, so setting that to that devices overall rate sounds wrong.
Otherwise this looks sane, we might want to break down processing into the two larger steps of sorting and sending. Also, we send indexes while scanning, so I suspect there would be a lot of jitter there?
I don’t think that needs to be a trap. If it’s not 1.0 exactly then it’s something less - but we could certainly make this an integer representing 1/10000 units or something, and require that the sending device round downwards - 9999 means 99.99% and not 100%.
I think we probably should, yes. There should certainly be some sort of indication that the device does not have all the data we expect it to, but the main indication should probably be that the device has all the data it thinks it needs, and is not doing anything at the moment to change that situation.
I brought the rates into it because there’s been confusion before about what they mean. My “or complement?” comment above meant that we could do something like show “Downloading at 1234 kbps (345 kbps from us)” or similar.
About jitter, yes, that could happen. In practice it would need to be rate limited somehow, and the other side is anyway not interested in any state that doesn’t last for at least a couple of seconds or so. Hence why I think we may not want to announce “scanning” at all, and in the normal case would probably not send “processing” during normal operations. We would send that if that state lasts longer than x seconds, or something.
Then again, if we have that logic we might as well have the “scanning” state. We just wouldn’t send the update if the scan was quick enough.
Ok sounds good, my only worry in that case is float precision causing havoc, as we don’t use floats elsewhere. I’d rather have 0-10000 or even 100000, oppose to floats, as I suspect (don’t know much about this) there are still cases there your 1.0 ends up a 0.99999
Maybe we don’t even need a percentage. We’ll still need to calculate what they are missing locally, and will do so regardless of what percentage they announce, so all we really need is a boolean indicating if they think they are done or not?
I disagree. If third parties start using these events to show info to the user, it would be good to make sure that everyone is using the same calculation for percentages: this means that they’re consistent across different consumers, and also that they all change if the algorithm for calculating them changes.
This requires including the percentages in the messages.