German Umlaute like äöü seems to make trouble

I copied initial a directory from linux to mac with rsync -avh as root. So we can assume that both directories 100% same. I than added a folder with same FolderId on both devices and wait till they are completely scanned. Then I shared them which each other. I choose on both devices to ignore permission, because not all files are owned by the user who run syncthing, but the user who run syncthing can read all files via group permission. On the linux side everything looks fine, but on Mac I got “Out of Sync”. 588 Items are out of sync. The error message was “file does not exist”. On both platforms the encoding is de_DE.UTF8. For a small test everything seems to work. For 333 files I found a .syncthing..tmp file.

How can I fix it?

1 Like

Can you post the full error from logs please (also visible when hovering the error in UI, but best to get logs directly).

I tried it for a subfolder and recognized, that on a subfolder some files are not listed in the auditlog at mac-os. Here is the list of files

Belege/WileyfoxKonfliktlösungen – PayPal.pdf
Belege/WileyfoxKonfliktlösungen – PayPal2.pdf

This is weird indeed. However I’d say it’s not related to umlaute, as they shouldn’t be a problem, and there’s also an item in the list above without umlaute. The errors with “creating parent dir:” come from this operation:

	if err := f.mtimefs.MkdirAll(parent, 0755); err != nil {

A “file does not exist” error on a MkdirAll call is entirely unexpected (after all, the call should create dirs at a path where nothing exists yet). Not sure what to make of this.

For the files that you found temporary files, I’d expect different errors. Can you check if there’s any other errors than the !file does not exist" ones here - likely towards the beginning of the errors.

While both systems use UTF8 they are still different:

Syncthing uses the native normalization form on each system. This should just work out of the box, and does for me at least over Mac+Linux+FreeBSD. I have no idea what’s happening to the OP…

1 Like

I’m unsure if it’s clever to analyze the big folder. As you could see even a smaller subset doesn’t work, because syncthing doesn’t recognize the files.

The Simplest example I have: Create a file with echo "German Umlaute" > äöü.cox.txt in Default-folder. The file never listed in the auditlog and wasn’t sync between both systems (never created at linux) - found no error about it.

@bt90 Thanks for the hint with the different UTF-8 encodings. I use rsync for my backups, so iconv would be there a good option.

Is it possible to run a separate instance just for debugging purposes?

I don’t understand what you do when you say “I tried it”. Please be specific about what you do. While it’s obvious to you, we can’t look over your shoulder and have to work with what you write here :wink:

What exactly are you doing here: Do you create that file on mac directly in the root of the Syncthing folder? Then do you do a full scan, and it doesn’t get picked up? If that’s the case, that’s a different issue and requires different diagnostics.

However I really would like to follow up on my earlier question:

Can you please check?

Yes it is, just point -home to a different directory. However you can also just create a new folder in the existing instance for testing.

And I still believe we are chasing a red herring here with umlaute. As Jakob wrote, those usually just work fine.

Also, I’m assuming you’re not doing any weird stuff. If your filesystem on Mac is in fact an external NTFS disk, or a NAS mount, etc, then you have other things to troubleshoot.

Sorry for the missing information. It’s the local drive from mac book air m1-chip. The only “wired” thing I did, was to create a group w and make it the main group for g and n. This is similar to the setup at ubuntu 16.04, where I use a local hard disk too. Further unusual setup: I make a rsync copy first.

I will try to make it reproducible on a subset of my files and make it with different home. So I could easily send you the complete log.

I think directories with 7500 files are difficult to debug. That’s why I started to copy subfolder to special directories and sync them to get more “Feeling” what’s going on. One observation was that some files wasn’t mentioned in auditlog (see list above).

About the simplest example: I make echo "German Umlaute" > ~/Sync/äöü.cox.txt on the apple device and echo "German Umlaute" > ~/Sync/äöü.polo.txt on the linux device. Then sync both. The äöü.polo.txt was synced the ~/Sync/äöü.cox.txt wasn’t.

I would suggest that I make tests with smaller subsets, where I can provide the complete logs. Should I create separate threads for it or discuss it here. Hopefully at the end of the day all issues are gone.

@imsodin I believe you that you make a hard work, to handle encodings. It’s always a pain.

1 Like

When you do that, is the file scanned, i.e. after scanning does the file count of the locate state increase by one? If not, then before reproducing please enable walkfs,scanner,model debug facilities, do it again and then post those logs.

syncthing.log (53.8 KB) console-out.log (30.6 KB) audit-20210514-134505.log (10.4 KB)

After export STTRACE=walkfs,scanner,model

start with /Applications/ -no-browser -no-restart -logfile=default -audit -home=/Users/niels/test > console-out.log File created in testSyncFolder with echo "German Umlaute 2" > äöüß.cox.v2.txt

Folder was created on both devices with same id. I hope I didn’t missed anything.

Thanks, that’s exactly what I asked for :slight_smile:
Unfortunately, it just made the mystery more mysterious: The filesystem sees and walks the files in question, no errors, but neither the walker nor the model report anything about it (expected would be an error or a to hash: message):

[FLLLN] 2021/05/14 13:44:35.698625 walk.go:112: DEBUG: Walk [] Matcher/[(?i)*~ (?i)**/*~ (?i)*~/** (?i)**/*~/** (?i).ds_store (?i)**/.ds_store (?i).ds_store/** (?i)**/.ds_store/** (?i).xvpics (?i)**/.xvpics (?i).xvpics/** (?i)**/.xvpics/** (?i).backup_files (?i)**/.backup_files (?i).backup_files/** (?i)**/.backup_files/**]@0x140005b2900

[FLLLN] 2021/05/14 13:44:35.701819 logfs.go:55: DEBUG: casefs.go:406 basic /Users/niels/testSyncFolder DirNames . [äöüß.cox.txt .stignore .stfolder äöüß.polo.txt] <nil>

[FLLLN] 2021/05/14 13:44:35.702043 logfs.go:61: DEBUG: walkfs.go:118 basic /Users/niels/testSyncFolder Lstat .stfolder {0x14000659ad0} <nil>
[FLLLN] 2021/05/14 13:44:35.702048 walkfs.go:84: DEBUG: walk: path=.stfolder
[FLLLN] 2021/05/14 13:44:35.702053 walk.go:270: DEBUG: ignored (internal): .stfolder
[FLLLN] 2021/05/14 13:44:35.702072 logfs.go:61: DEBUG: walkfs.go:118 basic /Users/niels/testSyncFolder Lstat äöüß.polo.txt {0x14000659ee0} <nil>
[FLLLN] 2021/05/14 13:44:35.702074 walkfs.go:84: DEBUG: walk: path=äöüß.polo.txt

[FLLLN] 2021/05/14 13:44:35.702820 folder.go:491: DEBUG: sendreceive/testSyncFolder@0x140010ca000 finished scanning, detected 0 changes

Would it be possible for you to do the same thing again with a debug build? I created a PR with more debug logging (lib/scanner: Extend and improve debug logging by imsodin · Pull Request #7675 · syncthing/syncthing · GitHub) and you could download the build from our CI here: (you can login as guest).

I think that just means it was successfully scanned in a previous pass?

It’s a clean start: There’s db migrations starting from 0 at the top of the logs.

1 Like

console-out.log (39.4 KB) syncthing.log (92.7 KB) audit-20210514-174048.log (10.5 KB)

I didn’t start with a complete empty setup, but created a third file with echo "German Umlaute 3" > äöüß.cox.v3.txt. The counter is still at 1 for the file I created at the linux side.

Good luke by catching this ghost.

Thanks. The hint to a/the bug is here:

[FLLLN] 2021/05/14 17:41:38.430411 walk.go:569: DEBUG: walker/testSyncFolder@0x140002ba1e0: normalizing path: dropping not-exist-error for äöüß.cox.v3.txt: file does not exist

We shouldn’t drop those errors. I extended the PR to fix that: lib/scanner: Extend and improve debug logging by imsodin · Pull Request #7675 · syncthing/syncthing · GitHub

The error is occurring when trying to rename the file to normalize it:

		if err = w.Filesystem.Rename(path, normPath); err != nil {
			return "", err

I still don’t understand how it is possible to get a not-exist error here, after the previous lstats didn’t return an error.

Why you rename the file? Why not only use the normalized name only for compare the list of files, but use local always the system-default? I was wondering about the pseudo-changes at linux too.

However, I make a further test with the amd64-version to check if it is an M1 specific error which solved by Rosetta. But it seems to be the same. Here the logs: syncthing.log (74.9 KB)

Just addition information. I created with TextEdit a file äöüß.cox.u1.rtf this file is found. syncthing.log (91.1 KB)

Here the differences between the two file

 find rtestSyncFolder -maxdepth 1 -type f -exec sh -c 'printf "%-10s %s\n" "$1" "$(printf "$1" | xxd -pu )"' None {} \; 
rtestSyncFolder/.stignore 727465737453796e63466f6c6465722f2e737469676e6f7265
rtestSyncFolder/äöüß.cox.u1.rtf 727465737453796e63466f6c6465722f61cc886fcc8875cc88c39f2e636f
rtestSyncFolder/äöüß.cox.v1.txt 727465737453796e63466f6c6465722fc3a4c3b6c3bcc39f2e636f782e76

which should show the filenames HEX-Coded. Unsure how reliable it is.

Here a simplier example which demonstrate the problem.

find . -maxdepth 1 -type d -exec sh -c 'printf "%-10s %s\n" "$1" "$(printf "$1" | xxd -pu )"' None {} \;
.          2e
./gäöüß 2e2f6761cc886fcc8875cc88c39f
./cäöüß 2e2f63c3a4c3b6c3bcc39f

the directory startet with g was created with finder, the other with mkdir. Seems for me that first one is utf-8-mac, second one utf-8.