I copied initial a directory from linux to mac with rsync -avh as root. So we can assume that both directories 100% same. I than added a folder with same FolderId on both devices and wait till they are completely scanned. Then I shared them which each other. I choose on both devices to ignore permission, because not all files are owned by the user who run syncthing, but the user who run syncthing can read all files via group permission.
On the linux side everything looks fine, but on Mac I got “Out of Sync”. 588 Items are out of sync. The error message was “file does not exist”.
On both platforms the encoding is de_DE.UTF8. For a small test everything seems to work. For 333 files I found a .syncthing..tmp file.
This is weird indeed. However I’d say it’s not related to umlaute, as they shouldn’t be a problem, and there’s also an item in the list above without umlaute. The errors with “creating parent dir:” come from this operation:
if err := f.mtimefs.MkdirAll(parent, 0755); err != nil {
A “file does not exist” error on a MkdirAll call is entirely unexpected (after all, the call should create dirs at a path where nothing exists yet). Not sure what to make of this.
For the files that you found temporary files, I’d expect different errors. Can you check if there’s any other errors than the !file does not exist" ones here - likely towards the beginning of the errors.
Syncthing uses the native normalization form on each system. This should just work out of the box, and does for me at least over Mac+Linux+FreeBSD. I have no idea what’s happening to the OP…
I’m unsure if it’s clever to analyze the big folder. As you could see even a smaller subset doesn’t work, because syncthing doesn’t recognize the files.
The Simplest example I have: Create a file with
echo "German Umlaute" > äöü.cox.txt in Default-folder. The file never listed in the auditlog and wasn’t sync between both systems (never created at linux) - found no error about it.
@bt90 Thanks for the hint with the different UTF-8 encodings. I use rsync for my backups, so iconv would be there a good option.
Is it possible to run a separate instance just for debugging purposes?
I don’t understand what you do when you say “I tried it”. Please be specific about what you do. While it’s obvious to you, we can’t look over your shoulder and have to work with what you write here
What exactly are you doing here: Do you create that file on mac directly in the root of the Syncthing folder? Then do you do a full scan, and it doesn’t get picked up? If that’s the case, that’s a different issue and requires different diagnostics.
However I really would like to follow up on my earlier question:
Can you please check?
Yes it is, just point -home to a different directory. However you can also just create a new folder in the existing instance for testing.
And I still believe we are chasing a red herring here with umlaute. As Jakob wrote, those usually just work fine.
Also, I’m assuming you’re not doing any weird stuff. If your filesystem on Mac is in fact an external NTFS disk, or a NAS mount, etc, then you have other things to troubleshoot.
Sorry for the missing information.
It’s the local drive from mac book air m1-chip. The only “wired” thing I did, was to create a group w and make it the main group for g and n. This is similar to the setup at ubuntu 16.04, where I use a local hard disk too. Further unusual setup: I make a rsync copy first.
I will try to make it reproducible on a subset of my files and make it with different home. So I could easily send you the complete log.
I think directories with 7500 files are difficult to debug. That’s why I started to copy subfolder to special directories and sync them to get more “Feeling” what’s going on. One observation was that some files wasn’t mentioned in auditlog (see list above).
About the simplest example: I make echo "German Umlaute" > ~/Sync/äöü.cox.txt on the apple device and echo "German Umlaute" > ~/Sync/äöü.polo.txt on the linux device. Then sync both. The äöü.polo.txt was synced the ~/Sync/äöü.cox.txt wasn’t.
I would suggest that I make tests with smaller subsets, where I can provide the complete logs. Should I create separate threads for it or discuss it here. Hopefully at the end of the day all issues are gone.
@imsodin I believe you that you make a hard work, to handle encodings. It’s always a pain.
When you do that, is the file scanned, i.e. after scanning does the file count of the locate state increase by one? If not, then before reproducing please enable walkfs,scanner,model debug facilities, do it again and then post those logs.
Thanks, that’s exactly what I asked for
Unfortunately, it just made the mystery more mysterious: The filesystem sees and walks the files in question, no errors, but neither the walker nor the model report anything about it (expected would be an error or a to hash: message):
I didn’t start with a complete empty setup, but created a third file with echo "German Umlaute 3" > äöüß.cox.v3.txt. The counter is still at 1 for the file I created at the linux side.
Why you rename the file? Why not only use the normalized name only for compare the list of files, but use local always the system-default?
I was wondering about the pseudo-changes at linux too.
However, I make a further test with the amd64-version to check if it is an M1 specific error which solved by Rosetta. But it seems to be the same. Here the logs: syncthing.log (74.9 KB)
Just addition information. I created with TextEdit a file äöüß.cox.u1.rtf this file is found. syncthing.log (91.1 KB)
