FR: Supervised synching


Is it possible to have some kind of synced files controls? For instance some kind of conflict overview before doing it all automatic?

This could be a special mode for the repo, when checked the user has to intervene to resolve the conflicts. There could be a ui that shows what is going to be written, deleted etc. But this is not just a list of files, as long as the list is up the synching wont go forward. The list ui will provide ability to manipulate the sync list. Like every file will have “accept, ignore etc” The sync could start after the user resolves conflicts or ignores certain files for that repo just for that sync.

No it’s not possible, and I think it’s out of the scope of the project.

Here is the main issue. It is hard to know when one has a conflict or unless the user constantly digs through the folders to see the conflicts (or run cron jobs that checks of conflict files and email them to user which is more technical than it needs to be) and I think that this area is the weakest part of Syncthing. Seeing the list of conflicts in the ui can help. For example this is even a worse issue for remote nodes where only way to access is ssh or such. Lets say I have like 10 different nodes that are spread around the world? How can I track the conflicts in a sane way? This can be a full time job daily. However if there is som e indication of file conflict on the local and the remotes, that can really help the user.

I think a bit of UI which lists all of the current conflicts and allows the user to resolve each isn’t too far-fetched: I’m currently implementing this exact functionality for SyncTrayzor. Stopping syncing until a conflict has been resolved is a different matter, though.

So you’d think so, but at the point we resolve the conflict, we loose all the metadata about it, so there isn’t much we could show apart from these are the filenames which have sync-conflict words in them, which can already be done by this super advanced OS feature called file search.

Also, I don’t think Dropbox et al gives an UI for conflicts. Dropbox just shows a balloon, which can be easily missed.

Sync conflicts are a natural part of life with bidirectional syncing, unless some discipline is being enforced to avoid it. I don’t see it as something that requires too much handling. If it is a problem it needs to be solved some other way; by using a central repository of come kind, having users talk to each other about changes they are going to do, etc.

Sure, but there’s a PR work-in-progress to improve that anyway.[quote=“AudriusButkevicius, post:5, topic:6404”] which can already be done by this super advanced OS feature called file search. [/quote]

Sure, and syncing files can already be done using this super-advanced tech called FTP. The tooling exists to make the user’s life easier. As a user, I want to be able to click a button and have that rename the appropriate conflicted file, while deleting the rest. That’s why I’m writing that for SyncTrayzor.

I’m not trying to say “you should implement this feature”, I’m just saying that, as a feature suggestion, it isn’t too far-fetched.

Agree, we could have that, but it’s more than an afternoon of someones time to make this happen. I guess you can put another feature ticket in the pile, though don’t expect it will happen.

Again, I never asked for it to happen. I have no expectation of it happening, which is why I’m putting it into SyncTrayzor. I just said that that particular part of the initial post has potential merit.

That is the thing though. You expect users to constantly search for conflicts all day which is really not realistic whatsoever. Even if they find the conflicst, the conflict was already resolved by an algorithm for them. Generally algorithm will do a good job but cant not be guaranteed.

I just did a conflict search on my Repos for you to entertain. I have like 864 conflicts already in 13 repositories. And I am the only user of these repos. So it is not like I am writing the same files multiple times on multiple nodes. Even so how do I resolve all these past conflicts? I do not even remember what has been done to all these files. So what do I do? I have to ignore them for now until I see something messed up. Why? Because resolving conflicts that were done long time ago will take a long time, quite counterintuitive.

Like canton7, I am not demanding anything. I am just bringing up an issue that effects the users.

No, I expect users not to produce conflicts. If you don’t want conflicts, set max-conflicts to 0, and there won’t be anything to resolve.

Well then you are not seeing my point. I am saying that I did not produce conflicts. The conflicts were created by Syncthing somehow. Even if the user creates these things, the user might not be awere of that they cause a conflict. That is when Syncthing comes in and tells the user that what they did has caused a conflict. The software can inform the user, nothing wrong with it.

Ok, with that I agree. Exposing the fact is fairly important.

Providing a way to resolve it is fairly hard, as we loose all the metadata related to the file, so there is not much we can say, apart from the fact that it happened, providing a delete/replace buttons.

That is great. Even showing a list of conflicts or some warning can be great then the user can act asap if he/she wants to do antyhing about them

You said that “I expect users not to produce conflicts”. But that sounds like you think that the users always manualy craft and modify their files which might be true for some files but not true for all occasions. Many files in a repo could be auto generated by backup softwares, or created as settings etc.

For instance most of the conclicts in my repos are results of the backups. The backups were done on Androidx86 or Android Arm. So I would never know this until I find and grep my folders. I hope it makes sense.

Let me try to summarize a bit here?

  • “Conflicts were created by Syncthing” - if that happens, it’s a bug and we should fix it. There is some issue like that in the tracker that I still need to look at, so it’s certainly not impossible. That said, I haven’t seen it myself or a reproducible case for it…

  • “Auto generated files such as settings” - yeah, this happens. On Mac, the .DS_Store file is a really f*cking annoying thing that always causes this as it changes if you do anything to a folder window in Finder… I think the correct solution here is usually to ignore them. If backups cause conflicts, there’s something odd about the backup system or there is a bug (see above)?

  • “Users need to hunt for conflict files” - in general I think they don’t, as they’re mostly interested in the content that won? And if it is for example a document (as opposed to an obscure settings file), the conflict copy will be right next to the original in their folder so that’s fairly visible.

I’m not opposed to showing a list of recent conflicts or something like that. However I’m not sure of the value to the users of something like “These files recently had conflicts: <864 instances of .DS_Store>”… There’s nothing actionable for the user other than deleting and then ignoring the files.

If there’s something that causes continuous conflicts (and it’s not just a bug on our side), then that needs fixing or any bidirectional syncing system is going to struggle…

I don’t think anyone has suggested it yet, but just to preventively nip it in the bud I can say I’m fundamentally opposed to any system that requires user intervention to resolve conflicts and continue. It should be automatic and as painless as possible, although possibly with ways to affect the rules and see what has happened after the fact.

… if they always open documents by browsing to them in the file browser, yes. Here’s a quick list of situations I can think of where this won’t be the case:

  1. User opens a Word document by using Word’s “Recently Opened” list (or any other editor).
  2. User opens e.g. a Visual Studio solution. One of the source files has conflicted. They never know, because they never look at that file using the file browser, as VS never tells them.
  3. Same situation but with a music library. They open mp3s using a media player, but the media player doesn’t tell them when new files are found.
  4. Indeed any sort of multi-file project. Or a portable application. Or a .app folder on OS X. Or…
  5. User opens a file using a shortcut.
  6. There is a large file which conflicted. The user cares about this, because they want to delete the conflicted versions to save space, but very rarely actually browse to the file in a file browser.

In all of these cases, yes, the user could manually search for conflict files using their super-advanced file browser’s search feature. But they have to remember to do this, and experience shows that most users won’t, even if they are aware that they should. Alerting the user of the fact that they have a conflict is useful.

I agree with @canton7 on general situations. The problem is that if this conflict thing never known or shown most average users will never know about this until they try to open a file that was conflicted and they will get confused since they were never aware of such “feature” and this might reflect badly on Syncthing. You know people easily get frustrated and angry especially when the software makes them work extra to get the things done that were supposed to be done automatically for them.

It sounds like there is a general concensus on making the user aware of such situations which is great.

It would be nice if one never gets a conflict, yeah I would love that but personally the current conflict handling is counterintuitive from the user point of view (at least from my own personal usage). As I mentioned before the main issue is that I never know if there is a conflict or not when the conflict happens. I tend to find these days or months later.

Backup software should not cause the conflict. I am thinking that it is a filesystem related issue. I seem to get these more when I backup on Arm Android and they are sync to x86 Android, or the otherway around. However I am not sure how it happens exactly. If a back up is written now by one device, the other device should just update with the newly written backup files. I am really not sure how a differently dated backup process can cause a conflict.

On the other hand I had other type of conflicts that had nothing to do with backups. But I tend to think that most conflicst arises on files from Androids.