This is a discussion thread for the issue/proposal outlined in this wiki page: https://github.com/syncthing/syncthing/wiki/Filesystem-Case-Sensitivity#detect-real-case-proposal
3 Likes
Hi,
I love that this is getting some love again - all credit to the brains that are working hard to solve this problem.
I don’t have any huge insights to offer - but I just want to make sure that the proposal will also cover case-sensitive renames of folders - e.g. /a/foo.txt and /A/foo.txt.
So far I’ve only seen discussions of file name conflicts - but folder ones happen too.
Thanks!
3 Likes
Files and folders (and symlinks for the matter) are all the same when it comes to their name. I try to use “item” when I don’t mean files specifically, but clearly am not consistent 
1 Like
Regarding the detection of the underlying filesystem’s case-sensitivity: it will be a mess. At the very least, it will have to be done for every subdirectory, because every subdirectory may turn out to be a mounted filesystem with different case-capabilities.
Hi:
Cool - I suspected you might have considered it already… 
For months, my poor little lizard brain has been trying to get itself round what happens when you have folder a containing foo1.txt and foo2.txt, and the folder gets renamed to A…
In the current workings (if I understand correctly), there’s a chance that these renames might get processed as adds then deletes - but that means you’ll have folders a and A temporarily trying to reside in the same place…
But I’m sure this is just tiny scratching of the surface of a whole bunch of cases that have to be considered!
Thanks all for giving this some attention. You’ll save me hours of work of trying to undo other peoples’ mess.
Is it that simple though?
If I have
Add foo/
Add foo/A
Add foo/B
and now, because of batching, I get a partial rename
Remove foo/A
Remove foo/
Add Foo/
Add Foo/A
what do you do with foo/B?
Also, what happens to the directory where B resided?
Or do we just rename everything that was under foo to Foo and rescan somehow?
Potentially end up with conflicts of new unexpected B appearing?
1 Like
Already today we wont remove foo as foo/B still exists - dirs with non-deleted children are a sync error. And then with the new case-awareness, we will also throw errors for (and not create) Foo and Foo/B. Once the batch removing foo/B arrives, everything will resolve.
For the record, the current plan is not to attempt to detect whether the filesystem is case sensitive or not. The plan is to correctly detect and handle case-only renames regardless of the case sensitivity of the filesystem, and to detect case conflicts on sync operations. Where a “case conflict” is that we’re going to do something to A but instead find an a that gets returned when looking for A – something that only happens on case insensitive fs:s. This we might do by directory listings or system-specific magic syscalls where available, as appropriate.
I was referring to section 1 of the “Open Questions” in the linked wiki article. Either I misunderstood it or @imsodin is indeed talking about sensitivity autodetection there.
The wiki article lists possible options for what I consider(ed) open questions, and it still does list my proposal of probing the fs to detect case-insensitivity and Audrius’ proposal to extend the FAT detection on android next to your proposal to just always check for case.
Right, I hadn’t actually fully read it, instead basing stuff on the plan in my head, sorry 
I don’t think we should attempt any kind of filesystem detection, for among other things the reasons mentioned by nekr0z. We should simply operate on the assumption that the filesystem is a bastard and be surprised and delighted if it isn’t. I also don’t think we need to fire off a case conflict when dealing with a and A in the same place. We only do that when we detect (during our usual stat-before-commit check) that the stat for A actually returned a.
There will be tricky corner cases (particularly merging two Windowses where both have pre-existing data in a new folder that differ just in case). Regardless I think we should try very hard to make it a general solution that can work on Windows, Linux, Windows with case sensitivity enabled, Linux on case-insensitive FS, etc without any user input or “detection” other than what we can tell about the specific file we are working with at the moment.
2 Likes
Detecting or not detecting is deep in bikeshedding territory at this point (entirely optional optimisation), nevertheless I can’t hold myself back: An if _, err := fs.Lstat(strings.ToUpper(fcfg.MarkerName)); fs.IsNotExist(err) { *filesystem is not a bastard* } doesn’t seem very messy to me
As to cornercases: I am currently quite confident that the simplest approach (throw sync errors at the user for every case problem) solves all of them. Small disclaimer: Confidence is usually way too high at the beginning of projects.
3 Likes
We’re not creating a marker in every subdirectory now, are we?
But I agree, let’s try to implement that universal listdir-based method first, otherwise it’s all moot.
2 Likes
Problem is a folder might consist of any number of different file systems mounted in various places. Conditions at the folder root may not be representative of any other directory.
4 Likes
Sure, yet I think we should auto-detect but allow users to toggle, for cases like that.
1 Like
Great that this issue moves forward!
In the current proposal there were two possibilities mentioned upon failure: halt/error or create a conflict. As someone who has been hit by this before when a user renamed “documents” to “Documents” a sync error seems much saner than creating a conflicted (duplicated?) directory tree, an operation that can easily fail with large directories.
Sure, yet I think we should auto-detect but allow users to toggle, for cases like that.
Absolutely yes please! This is clearly a case where auto-detection would get it right in 99% of cases and the advanced users like nekr0z that dare to mount filesystems with different case-sensitivity under the same folder should change the defaults to their liking.
The idea is that the mechanism shouldn’t need auto detection or configuration, because the same behavior (from Syncthing’s side) should result in correct results on both sensitive and insensitive file systems. The reason to have a configuration anyway is that the new, safer, behavior would result in more file system operations. Having a switch to say “there is no need to be safe, I promise everything is case sensitive everywhere” might be useful in some scenarios. Might.
3 Likes
My 1.5 cents, please be lenient as I’m not aware of all difficulties you devs have to face with this issue:
When a case sensitive device case-only-renames an item (let say item to Item) I figure the receiving case insensitive device detects this as is and could deal it a special way, e.g. creating an hidden temporary item called something like .syncthing-caseSensitivity-Item, let it sync, delete item but does not update DB, and only then move .syncthing-caseSensitivity-Item to Item and terminates the process the same way as usual.
Sorry if you already thought it this way.
I know this is a little old now, but I am not sure I understand what the final solution was. I am hoping that someone can update this with what it does or provide a link to that information.
I read somewhere that essentially says syncthing now simulates a case sensitive file system when interacting with case insensitive ones, but that is a simple explanation and sounds like the opposite of the desired behaviour.
Note that NTFS partitions can be created in Linux as well, which Linux treats as case insensitive. In addition, Windows now has the ability to mark a specific NTFS folder as case sensitive, but this does not apply to its subfolders.
I look forward to reading the response. Thank you.
This should answer your question: Understanding Synchronization — Syncthing v1 documentation
tl;dr: You shouldn’t need to care about case sensitivity anymore - if there’s problems/clashes, there will be an error telling you about it.