Implementing case insensitivity

Ok right. New ambiguity with that schema - case conflict on two devices: Device A creates foo, device B Foo and these are totally different files (both case sensitive). Device C in insensitive and pulls both of them, and will just regard it as an update. And I wouldn’t be suprised if there were quite few more such problematic border cases. It just feels more complicated and thus more likely to create problems than some kind of inter-device solution/cluster wide setting. However it’s entirely possible that this is just bias, because I had that option in my head a lot longer and I still struggle wrapping my head around the conflict bit one.

Yeah… You don’t even need the case insensitive device in the middle. Given that they are all case insensitive by default the issue in this case will be triggered directly between two case sensitive devices as well.

Although, if the two files have different contents and and created simultaneously it will be a sync conflict so we’ll end up with foo and Foo.sync-conflict-blah. At that point the user can go “WTF?” and rename the conflict file to Foo, and the case conflict bit stuff kicks in. This should be the same with the case insensitive box in the middle, except the option to do the rename doesn’t exist at that point.

So I think it would be handled correctly-ish, even if the initial sync conflict will be surprising to a user expecting case sensitive defaults.

That means in this stage multiple files with the same name except case aren’t handled anymore on case-sensitive systems at all, right? Meaning device A scans foo and Foo, sets the case bit, sends to B which then errors on pulling - even though both are case-sensitive. That wouldn’t be nice in my opinion.

However even with defaulting to what the FS can do, my scenario should be fixed by your sync-conflict argument: The insensitive device C will wrongly consider Foo and foo to be the same file, but as they are conflicts, it will create a conflict copy. And that scenario (user having both case-sensitive and -insensitive devices connected and actively using filenames with case differences only) is unlikely enough to just accept these sync-conflicts - as you say, it probably creates a WTF moment, but data is safe.

So currently to me it seems like case-conflict bit, lower-case keys with list of folder infos in db and defaulting to the case-sensitivity of the FS seems like the least invasive change, both to the user-experience and implementation.

No, if the folder filesystem is case sensitive there is no reason to error out here. I imagine the behavior will differ based whether we detect a case insensitive filesystem or not at startup, by probing.

Then I am all good, because that’s what I meant by saying “defaulting to the case-sensitivity of the FS” (not db being case sensitive) :slight_smile:

1 Like

Just from my point of view, I will never sync something to case-insensitive systems, thus any work I have to do to make Syncthing behave sane (as all of my systems) is absolutely unnecessary and incredibly time consuming if it’s per-folder. Any case insensitivity related conflicts should be something syncthing warns about (possibly adds a file extension that the file would conflict on this system), it should never touch the case of my files when there’s nothing case-insensitive in my Syncthing mesh.

1 Like

I have not contributed any intelligent thoughts, but it turned out, you all did. Thank you.

I think, by reading the initial post and its processing, my usecases are mentioned and handled well.

I especially support

and

Hello:

This case-sensitivity issue is causing problems with our users, so I’m keen to do anything to help progress this solution that my limited abilities might allow…

Just re-reading through the thread, just one thought so far occurs to me:

Would this still work sensibly where a parent folder of a file is renamed in case-only? e.g. where this:

folder1/folder2/folder3/foo.txt
folder1/folder2/folder3/foo2.txt

…gets renamed to:

folder1/Folder2/folder3/foo.txt
folder1/Folder2/folder3/foo2.txt

Assuming renames occur file-by-file, wouldn’t there be a point where folder1/folder2 AND folder1/Folder2 need to co-exist?

There were several annoying problems that blocked me when I lasted tried to implement this…

1 Like

Did anything shareable come out of your last try? Or some info on the annoying problems? I probably can’t work on it in the very near future, but just in case I ever get the inclination and time it might help save some time by knowing some obstacles up front.

I’ll look into it beginning of next week, I don’t fully remember what I stumbled on

Actually while “real case insensitive” operation would definitely be ideal, I came up with an idea for a clutch against it, which might be more realistic to happen in a shorter time-frame:
The PR about evaluating symlinks on windows used GetFinalPathNameByHandleW, which also returns the “real case”. It should be possible that on pulling when the file on disk is not what we expect in db, we get the “real name” and if there is a case only difference either merge the filenames if contents are the same, or create sync-conflicts (or potentially a new “case” class of conflicts). That’s still a nuisance to the user, but at least a recoverable one.
Apart from all the real-world issues that will come up when actually implementing this, it is a windows only clutch (if I remember correctly mac can also have case issues?) and if it works, it will lessen the incentive to come up with a real solution :slight_smile:

Does this make any sense to you?

2 Likes

Also, if we did deletes before creations for case only stuff we could recover for this more gracefully.

I’d love to have the option to resolve case conflicts with a conflict file rather than restoring from backup :smiley: I made an error a couple of years ago by renaming my boss’ (father in law…) folders to all title case in his main work folder and then having them seemingly randomly disappear on sync due to case issues between MacOS, Android, Linux, and Windows.

1 Like

Same problem here. Just wanted to remind that there is a bounty on this topic:

Would be great to find other backers :slight_smile:

1 Like

Yikes, seeing that contribution I have to express my intense displeasure at the idea again. Can I add a negative bounty to something?

It’s a horrible default, it’s not correct to assume people definitely use case-insensitive OSs, that they have files that could cause issues or both. This issue is further exasperated by the fact that AudriusButkevicius said “Yet I am not sure if I would bother with the extra flag.”, it’s asinine, what’s next? Syncthing will chmod 777 and give read-write on Windows to Everyone on every file because Android doesn’t do permissions nicely?

If some OSs work at a reduced feature set doesn’t mean nearly half of the userbase should suffer due to it. Directly related to this lowest common denominator issue is that Syncthing doesn’t follow symlinks, intentionally (instead of following and syncing the file behind it :roll_eyes:), it wouldn’t hurt at all to let Linux users have their symlinks but no, because “compatibility”. Majority of Syncthing users use OSs that are case-sensitive and/or support symbolic links by-default (check out https://data.syncthing.net/), so pretty much more than half of the users get thrown under the bus and have to suffer from primarily Windows’ idiosyncrasies.

Dude, chill. If anything, Syncthing currently throws the case insensitive half of the users under the bus as they currently suffer data loss and mysterious issues.

Following symlinks has many problems that I’ve run into when trying to implement it, none of them related to Windows.

4 Likes

I like this. Let ST make files like FOO.case-conflict… for the user to resolve case conflicts manually.

Hi Jacob, I’ve seen that a bounty is associated with this feature, that is currently at 160 USD; I would like to up that bounty so this feature would get a high priority - what kind of amount would you be thinking of? I’m unfamiliar with the bounty program, so my apologies if I’m misunderstanding how this works or is intended.

I honestly don’t think the bounty makes much of a difference at the current levels or thereabout. It’s a tricky problem that is already quite high priority. Me and other have tried tackling it and will do so further in the future, I’m sure.