Feature:
Supporting storage limit folders. Set a storage limit in the config, and upon reaching the limit, clear some of the oldest files. Like LRU Cache. This is a common use case where backups of a chat application are taken, and the names would be different to be supported by the current file versioning system.
Or, if any other idea is more suitable, I would be more than happy to help.
A little bit about me:
I am a software engineer and am currently working on Go in my tech stack. I have been using Syncthing for a long time, and I want to start contributing. I felt this issue would be suitable for me to get started since the good-first-issues are very old.
If any of the contributors can help me out, it would be awesome!
I think syncthing deciding what to delete is a bad thing. It would be pretty easy to write some kind of cleanup script in your language of choice to delete old files and plot it at your shared folder. Syncthing would synchronize the deletes from that script.
I could see the “quota” idea as useful wherein if the total size of the folder is exceeded then the syncing is paused, as long as it’s done so if the folder size on the remote side is reduced again the syncing can resume.
Sounds like a good idea. Currently, the Trash Can File Versioner cleans out based on the number of days, per the versioning.params.cleanoutDays setting.
It sounds like you are suggesting we add another parameter, say versioning.params.cleanoutDiskFree, that would delete trashcan’d files starting with the oldest file first, when the free space on the folder’s volume reached that limit, and specified as either a percentage, or a number, in the same way the folder’s minDiskFree option defines the minimum disk free for a folder.
I think that would be pretty easy to implement. The only question I can think of is: if a user specified both a cleanoutDays and a cleanoutDiskFree, would both be active, or would one take precedence over the other? My guess is they would both want to be active.
I agree the size limitation for the trash can is a good idea as is written in the github post.
But I think he’s talking about a size limitation for an active shared folder, not the versioning folder. (Inspired by, but different than the GitHub post)
I really don’t see the authors supporting any kind of idea where syncthing is deciding which files in an active folder get deleted and then those changes get propagated to all the machines?
Anyway I don’t speak for the authors. They can reply directly.
I completely agree, Syncthing should not be deleting files on its own. That’s an external script’s job. Perhaps Syed can provide clarity on if they are suggesting that, or if they want to delete files in the Trash Can (.stversions) by oldest date first, as I described.
My intention was to have an upper limit on the storage being used. Perhaps as Mike suggested, instead of deleting active files, the syncing can be paused with some logic behind it.
And also, another feature would be having the upper limit on the trash can too.
If any of the above two(or both) ideas looks good, I would like to help in contributing.
Quotas are not a new concept so could be useful both for versioning and also for active directories. I think it’s a good idea if there’s some pause logic for the Active Directory. (No deleting)
I’m okay with a maximum trash can size with some kind of deletion logic. Would also be useful for all types of versioning but gets a little complicated for different versioning types. For staggered versioning do you always delete the file in the version folder longest? Or do you try to preserve at least one version of each file and delete secondary versions first?
Anyway good ideas. Just don’t delete files in the active folder directly.
Re your second bullet point, I think a folder’s minDiskFree setting will automatically pause the folder when the free space reaches the set limit. Is that what you are looking for?
Re your first bullet point, does my response to your initial post, describing adding a versioning.params.cleanoutDiskFree setting, meet your requirements?
minDiskFree is suitable for my second bullet point.
But for the first bullet point, I am wondering if versioning.params.cleanoutDiskFree will be a good choice. Because it sets a limit with respect to the free size of the disk, which means it also depends on files external to the active directory.
For example, if a user is copying an external file to the disk, will the deletion get triggered because of that for every block of file transferred, and won’t that cause a drop in performance?
I was thinking a limit on the trash can/active directory size would be more suitable due to the above described issue.