Lets say I have a two device setup, with one folder master and one other device that should just receive updates.
Alice [master] -> Bob
I have a couple of directories ignored as they contain stuff I don’t want to transfer. Now I mess up my ignores for a moment and these files “escape” from Alice onto Bob. I quickly re-ignore the files on Alice and restart, but some have already been transferred and some were in progress and are half done. On Alice, it now looks like this:
Those 60 gigs of out of sync files are the things that have escaped to Bob. They are out of sync because Bob announces that he has them, and we do not (as far as we know, because they are ignored).
Problem
Clicking “Override Changes” at this point does nothing. Nothing at all. Why not? Because all we do internally with ignored files is to set the “invalid” bit, which says "do not attempt to download this file from me"1.
Ignoring files mean we’re agnostic about their existence - they may exist or they may not, and we don’t enforce it either way. But this means Alice can never be an “in sync” master again. What I would like to happen is for the files to be deleted on Bob.
Questions
Should ignoring files (that exist at the time of the ignore patterns taking effect) cause them to be deleted on other devices? Generally speaking I’d say “no”.
Should the “Override Changes” button on a master device mark existing, ignored files as deleted? For my setup now “yes” would be convenient, but it would mean we do two different things with ignored files depending on the situation.
Or is this is all as it should be, and what I need to do is remove the ignore patterns, move the files away so they are genuinely “deleted”, then ignore them and move them back on Alice… That’s a bit of a process, just to get rid of files that should no longer be synced to the non-master devices…
1) We also don’t send index updates for ignored files, so that would have to change too.
For folder master I would say: delete!
Unless Bob has his own different ignores, “override changes” should result in Bob having exactly the files Alice is sharing.
For normal shares (non master) I think the current behavior is the correct one.
Those 60 gigs of out of sync files are the things that have escaped to
Bob. They are out of sync because Bob announces that he has them, and we
do not (as far as we know, because they are ignored).
Question… why does Bob even announce that he has them when they were never completed? Why not have Bob (all nodes really) put them in a different state when downloading and only changing that state to completed once done? That way different behaviors can be applied to not yet completed files if needed, yes (like in this case, or if all sending machines go offline during transfer, recieving machine goes offline during, etc…)?
My understanding is that the 60GiB were transfered completely and other files were not completely transfered.
Syncthing uses temp files, while as transfer is ongoing. Only if the transfer is complete, the file is renamed to the original filename and added to the index. Temp files will be deleted (by default) after 24 hours.
I was not aware that temp file would be deleted after 24 hours.
I had a situation recently where this was not the case and getting rid of zombie files (after several nodes went offline perm, fdisk/reformat) required me to delete the share and re-share it and resync it too all the others from one I chose as the good copy.
I never use master dir’s (if that helps with info) since I usually make edits to some files and folders from multiple computers (my desktop at home, my laptop on the go, my android phone). Heck that’s really why I use a sync program, it lets any change made from any client to propagate to the others regardless of which one. Otherwise if you’ve always got a master and the rest slaves… isn’t that really much closer in behavior to a backup program instead?
What about situation when folder is not master.
I am syncing PC and Laptop and synology as backup. And I am not seting any “master folder”.
I have projects in my sync folder and using this ignore pattern
*~*
**/*results*/**
So I am not interesting in syncing results because that folder can be very big {GB} and those data can be reproduce.
Sometime it happen:
I am renaming directory {project} which contain directory results. Other devices get out of sync and I have to manually solve that situation. Problem is ignore directory because his parent has rename but syncing is not able to do anything with ignore directory. Same problem is if I am moving project directory in some another place in sync directory. Or when I am deleting project directory on other devices I have to delete ignore directory manually.
Solution: syncthing should check what kind of “change” is request to do with ignore files. Easy solution is if syncing want to delete directory, then every ignore files/directories in that directory should be delete.
Wouldn’t it be convenient for the syncers to know each others ignore list?
You already send the whole sync status to every remote anyhow. Ever syncer can than calculate themselves if the are realy out of sync or just a difference in ignore rules.
I wouldn’t delete the ignored files automaticaly for sure. Image you sync your entire photo/home video collection from your laptop to your NAS and at a certain point in time, your laptop disks gets full. To make room you decide that all the photos from 5 years old and older are not needed on your laptop anymore (by then you should have sorted them out right? ), so you put the in the ignore list on the laptop and deletes them on your laptop.
On a later time I might want to get year x back on my laptop, so I unignore it and it will sync back to my laptop.
I would agree with wweich and Kluppy since isn’t that what would commonly be expected from a folder named “Master”?
I am only using masters very selectively and sparsely since I usually want two-way syncs. But sometimes a Master is just very handy. And in those rare cases I explicitly want it to be the folder with which I control what other machines might potentially receive.
Idef1x has a good argument not to delete the remote files:
… but in my eyes that not the idea of Syncthing. That would be - sort of - using Syncthing as a backup tool. Which we probably all do at one time or the other - but thats not the point of Syncthing. Syncthing should sync. Maybe it should ignore files now and then, okay, but if not explicitly told so, it should make sure both sides are the same.
So yes, a Master should be able to politely ask the recipients to delete files that it does not publish.
), so you put the in the ignore list on the laptop and deletes them on your laptop.
On a later time I might want to get year x back on my laptop, so I unignore it and it will sync back to my laptop.