Deleted items are no longer kept forever in the database, instead they are forgotten after six months.
My apologies for bothering the community, but could somebody explain what this actually means in practice? If I have two devices A and B that have the same state and then they disconnect, and I delete a file on A, and they connect after eight months, what happens?
I decided to try to up this interval just to be safe, but I couldnāt find it in āAdvancedā > āOptionsā or the regular āSettingsā. Where would I find it in the UI? Iām guessing that Iām just being blind and I canāt find it, my apologies.
Thank you for the clarification! Is there any hope for it to be added to the UI? Modifying the start scripts on all my machines could be a little cumbersome. (I guess I have about six months to figure out how to set it.)
Not really, it needs to be the same for the entire cluster to make sense, so making it an easily changeable value per device is likely to result in more confusion.
Of course, your setup is your setup, but Iād venture that for syncing things once every eight months, Syncthing might not be the perfect tool. (āContinuousā and āreal-timeā are in our tagline since forever ;))
(Also, for guaranteed immediate consistency of deletes, you do not have six months from today. You have six months from when the devices were last in contact, effectively.)
Iāve been using it this way since forever, so it seems to be the perfect tool for me modifying the startup scripts is more difficult since for some setups theyāre provided by the distribution. So if Iām meant to set this option everywhere, I think that would be a good point for a UI option. My apologies if Iām misunderstanding something, however.
Unfortunately, at this point, this is your only option. I donāt like this behaviour either, so I just modify the source code and set it not to delete anything regardless of how long time has passed.
In our case, weāve got a setup of maybe a dozen devices or so, and while most of them run 24/7 and are always connected, there is one that is turned on maybe once or twice a year. It doesnāt share many folders either, however what it does sync is very important, so we cannot risk the folder contents getting messed up when the device is finally turned on after being offline for many monthsā¦
Iād also add that even though weāre talking about ādeletionsā, for Syncthing, if you move or rename files or folders, the previous locations are also treated as ādeletionsā, so I think in some cases the impact may be even more serious.
I think this is really the only negative about this release. I do think itās gonna cause a lot of support issues. It would be interesting to collect data on this kind of thing.
You could put a ālast connection ageā in the Mon.syncthing.net dataset and see how many instances have such outdated connections. I suspect quite a bit.
I am not a Go coder myself but I would still think this 6 month limit could be dynamically adjusted based on the oldest connection (and successful metadata sync) date among the list of devices. Perhaps the limit is 6 months or if a device hasnāt been seen for 8 months, then it becomes 8 months⦠And when that device finally reconnects, then it becomes 6 months again and 2 months of deletes can be purged.
Anyway. For now Iām not upgrading because it will be more difficult to manage my cluster of 15 machines (some of which have been offline a while but maybe not 6 months.)
Would you guys be more chill if the cutoff were 12 months instead of six then? Or is this a case of āmy workflow requires devices to be offline for arbitrary amounts of time!ā?
That sounds nice but is difficult to implement, because all devices would have to agree on the same limit, and not all devices see each other, nor all the time (so this āoldest connection ageā would differ between devices).
Itās major version increment after all, and Iām surprised thereās so little breaking changes. Maybe recommend on main page universal transfer tools like rclone/rsync? Might as well suggest using backup tools for the job (you might know some, Jakob Borg)
I figured each device would need to agree only with the other devices they can see.
Anyway. Itās a real use case. Itās apparently more difficult to manage than I thought because now I have to change all the instances to the same limit. Even if they donāt talk to each other. And I donāt have direct access to all the machines.
so I basically am prohibited from upgrading unless Iām okay to have to manage a cleanup of my data when an old machine comes back online.
So hereās the thing, there are two individually reasonable requirements here that are unfortunately in conflict:
A device should be able to be offline for an arbitrary amount of time without losing any sync changes
The Syncthing database should have a finite size for a finite set of files
Previously we optimized for requirement one and allowed an infinitely large database. You can still get that behavior by setting the retention period to zero (forever). Syncthing not optimizes for the latter and sets the interval at six months. Iām not very particular about that interval, I think it could be 12 months instead by default. But, I do think there should be a cutoff at a reasonable interval, and those who want to do something less reasonable can configure accordingly.
If you canāt bother to set a command line switch or environment variable, sure.
In that case nothing we do will help, so it is indeed your only recourse to never upgrade.
Edit: If I sound a bit exasperated itās because when these things boil down to āI canāt change anything, you must put in the engineering effort to make sure the defaults are perfectly suitable for my use caseā I do, in fact, get a bit exasperated.
I think it is more than reasonable to expect users to make certain changes in their config on a major release like this (other command line flags changed as well). Just stay on 1.x if you want everything to stay exactly the same, or change the interval to something absurdly large on 2.x, your choice.
(@calmh is there a particular reason though why this setting is node-wide instead of folder-specific? I assume all nodes sharing a folder will have to agree on the interval, but it should be fine to have different intervals for folders shared by the same devices, and also you might want to pick different intervals for different folders depending on your use case. I.e. if I have a folder that has a lot of deletions I might want a smaller interval, and a larger for another. Note I do not have any specific use case myself for this, I am fine with the current default)
I get it. And I also donāt mean to sound ungrateful. Iām not at all. This is just a problem for me and I wish I could upgrade. I guess I have to see if thereās some manageable/controlled approach I can take to upgrade one machine at a time and if the machines I donāt control donāt upgrade then I guess Iām still okay.