First of all, thanks to the development team for this great piece of software. It has replaced ownCloud for syncing data between my different computers, and works overall much better.
The one problem I haven’t been able to solve yet is that files with non-ASCII file names don’t get synced. That seems to be a widespread issue, but my case looks different from the other ones I see discussed in this forum or in various GitHub issues. The error messages I get look like this:
INFO: File “datei-mit-äöüß-im-namen.txt” path has UTF8 encoding conflict with another file; ignoring.
That particular file is one I freshly created for this test, using Emacs under MacOS. It is the only file in its folder, so I don’t see which other file it could be in conflict with.
The folder containing this file was created with the default options, in particular with autoNormalize=true, so this shouldn’t be a normalization issue.
There is one particularity in my setup: my Syncthing-managed folders are kept on a ZFS filesystem in order to make them accessible from Linux as well. I am not aware of any UTF8-related issues with ZFS, but I am not much of an expert on that topic either.
If a file name isn’t normalized, ZFS won’t normalize it for you, and will report its name as non-normalized
However if there is a non-normalized file name, and you ask ZFS whether the normalized version of that file name exists, it will answer “yes”. That is, ZFS does normalization when finding a file with a given name.
Looking at the code, this will probably trigger the behaviour you’re seeing, but I don’t know ZFS well enough to say whether this is actually how it behaves…
I checked the ZFS documentation and the specific configuration of my ZFS pool. ZFS never normalizes filenames before storing them. Normalization happens “as part of any comparison process” - the documentation isn’t more explicit as to what a “comparison process” is exactly.
My pool is configured to allow only valid UTF8 filenames, and to use a normalization algorithm that ZFS calls “formD” for comparisons.
Is there anything I can configure in Syncthing to disable the check that fails? It looks as if ZFS already guarantees the absence of UTF8-related name conflicts on my system.
Looking at the code, I think the only thing you can do is to normalize the filename yourself, using something like convmv --notest -f utf-8 -t utf-8 --nfc filename (untested).
I suspect this will be picked up and fixed fairly quickly…
There may also be some fun around the fact that you’re using normalization form D, and Syncthing’s assuming you’re using normalization form C (as that’s the standard for non-mac systems). I don’t know what the consequences of this would be, whether you’d need --nfd instead of --nfc above, etc.
No more error message. On every scan of the folder, I see the message
Normalized UTF8 encoding of file name “datei-mit-äöüß-im-namen.txt”.
The file gets transferred correctly to a Linux system (ext4 filesystem), including the filename. In other words, everything looks as if my problem is solved, although I will do some more experiments to be sure.
No more error message or warning. All files get correctly copied to a Linux machine. Files with UTF8 filenames generated on the Linux machine get copied to the Mac correctly as well. In short: no more problem
BTW, I don’t mind testing builds at all. I can’t remember any other software with such easy deployment. Download, unpack, run.
I’ll give this some more extensive testing over the weekend!
