Raspberry PI: Some Receive Only files always get marked as “Local Additions”

[kay_D]

Hi,

My aim is to sync my mp3 collection from my Linux PC to some OSMC Raspberries. Because I had storage problems (files getting corrupt due to SD or USB-Stick wear) and do not want to get corrupted files synced back to my PC, I set “send only” on the PC and “receive only” on the Raspberries. In general, all works fine.

However, some files are marked as “locally changed” right after writing them. First guess: It could be the file attributes (e.g. different user, etc.), so I checked (on both sides) the “ignore permission”. This did not change anything.

Now I observed, that the reported files all have a timestamp mismatch of 1 second. I presume this is due to the FAT filesystem on the USB stick I store the files on.

I have read [SOLVED] Receive Only folders always get marked as “Local Additions” in Android and I am under the impression that the “1 second timestamp off” fix is compiled into the binary and therefore OS dependent. Instead, as my Raspberries are also Linux, the behaviour should be filesystem dependent, or - better - be adjustable, i.e. the “ignore permissions” (which in its description specifically mentiones FAT file systems) should also fix the “1 second timestamp off” problem.

Any thoughts on this?

[AudriusButkevicius]

The only os we can distinguish is “linux” “windows” “android” etc. There is no OS for rasberry pi’s. You most likely experience this because the filesystem in question is fat32.

Best is to try a different filesystem that supports higher precision tiemstamps.

The mtime allowance used on android is already adjustable, just from the advanced settings with very little documentation/explanations.

[kay_D]

Hi Audrius, thanks for the quick answer. Surely I agree that Raspberry is a linux flavour OS, and should also be treated like it. The problem I have is that it is inherent not to the OS, but to the mounted filesystem. Also on my Linux-PC, when I use a USB stick, the same problems would arise. That’s why I would assume that the “ignore permissions” checkmark that “enables” (i.e. workaround) “FAT, exFAT, Synology, Android” should also fix the timestamp imprecision issue (it is present elsewhere in the code already). Would this be a valid “bug report” or better enhancement issue?

About your suggestion to use another filesystem, well that’s difficult on raspberries, as SD cards (where the OS is located) are frequently failing (due to amount of log stuff written), and USB sticks are generally FAT. (I have about 100 GB of media to sync, so SD cards and USB sticks are on the expensive side )

Kind regards, Kay

[AudriusButkevicius]

Not sure, in a normal bidirectional scenario this is not a problem, as the tiems would be synced back. Also, on a device with a real clock source I don’t think this would happen.

We should probably expand on thsi in the documentation, but I don’t think making the option more easily accessible is the right thing, as it can cause harm if misunderstood and misconfigured.

Ignoring permissions has very limited blast radius.

[odin (Herbert Snorrason)]

@kay_D

That’s why I would assume that the “ignore permissions” checkmark that “enables” (i.e. workaround) “FAT, exFAT, Synology, Android” should also fix the timestamp imprecision issue (it is present elsewhere in the code already).

Keep in mind, however, that this is not an issue with exFAT, which does have timestamps with higher resolution. Moreover, I think you’re missing an important point, specifically:

@AudriusButkevicius

The mtime allowance used on android is already adjustable, just from the advanced settings with very little documentation/explanations.

On the presumption that this refers to the window you get by clicking on ‘Advanced’ in the ‘Actions’ menu in the GUI, I went and looked. Sure enough, one of the folder options given is “Mod Time Window (seconds)”. This would appear to be the exact knob you’re looking for.

That said, the documentation (look for modTimeWindowS) says “Used in Android only.”

[odin (Herbert Snorrason)]

@AudriusButkevicius

Also, on a device with a real clock source I don’t think this would happen.

The problem is that FAT only has 2-second resolution, regardless of the system clock’s accuracy.

[AudriusButkevicius]

Sure, and we already have measures in place that deal with that without the need of that knob.

[odin (Herbert Snorrason)]

I’m just at a loss to see why it would matter whether or not the system has an RTC.

[AudriusButkevicius]

So I am not sure if it’s relates to RTC, but it seems that fat32 sometimes provides up to one second precision, but it’s all cached in ram, and after reboot becomes 2s precision, moving the time on most files.

This is what happens on android.

[rustycanb (Russell)]

@kay_D Why not use Linux filesystems on your PI? I’ve had a garden variety Sandisk Ultra 64GB with boot(FAT) / OS(Ext4) / data(XFS) partitions operating for months 24/7 with no failure on my RPI3. Setup to boot from USB is very easy. ST works perfectly as the hub for my two laptops and two Android 11 devices.

[tomasz86]

@kay_D

the behaviour should be filesystem dependent, or - better - be adjustable

If you want to get rid of the errors, simply set modTimeWindowS to 2 using the Advanced Configuration for each folder that is located on the FAT partition, and then revert local additions.

@odin

That said, the documentation (look for modTimeWindowS) says “Used in Android only.”

It should say “By default, used in Android only”.

For reference:

• https://github.com/syncthing/syncthing/issues/7067
• https://github.com/syncthing/docs/pull/584

[kay_D]

@odin wrote:

I’m just at a loss to see why it would matter whether or not the system has an RTC. Definitely not a problem of a RTC, the timestamps in question are all the mp3 modification files (some are from 1999 ).

@rustycanb wrote:

use Linux filesystems on your PI

osmc@osmc:~$ mount | grep sda1
/dev/sda1 on /media/VERBATIM type vfat (rw,nosuid,nodev,relatime,uid=1000,gid=1000,fmask=0111,dmask=0000,allow_utime=0022,codepage=437,iocharset=ascii,shortname=mixed,showexec,utf8,errors=remount-ro,uhelper=udisks)

While it would be technically possible to re-format this USB stick, I am a bit hesitant to do so for the various others I have, since I also use them occasionally elsewhere. (Also, it was not exactly a cheap one.)

@tomasz86 wrote:

simply set modTimeWindowS to 2 using the Advanced Configuration

This sounds like exactly what I am looking for. Advanced Configuration looks like this:

Where can I set modTimeWindowS?

Thanks so much

[kluppy]

It will be in the advanced settings of Syncthing not the folder’s advanced settings. You can find it in the drop down menu on the top right of Syncthing’s home screen.

[kay_D]

Thanks. Sorry I overlooked this

At first glance, it looks like this setting resolves my problem! Thanks to all who helped.

Not sure if this is a common problem for all people using USB sticks. If it is, I would suggest to have this “2 second fix” handled together with the “Ignore Permissions” checkmark - or at lease read the “Mod Time Window” as a suggestion.

Also, I would like to edit the Topic Title of this thread (I seem to cannot), to be more specific to “all OS, and FAT”, as well as to set the topic as resolved.

[imsodin (Simon)]

The thing is, 2-second precision should not be a problem without using mod. time window:

@imsodin

There is already a mechanism that deals with any filesystem that does not have nano-second precision: The mtimeFS. Meaning that covers the 2s precision of FAT. That mechanism requires though, that the filesystem is consistent: If it told Syncthing at some point that the mod. time of a file is 01:30:03, when asked again it should say 01:30:03 as well - not 01:30:02. And that’s what a “normal” FAT filesystem does. Android on the other hand puts some virtual filesystem shenanigans on top of FAT, and apparently that changes reported mod. times upon restart (well remount probably). And that’s the problem and that’s why the mod. time window workaround is necessary despite having the mtimeFS.

One possibility is that the following problem is not exclusive to android:

@AudriusButkevicius

So I am not sure if it’s relates to RTC, but it seems that fat32 sometimes provides up to one second precision, but it’s all cached in ram, and after reboot becomes 2s precision, moving the time on most files.

@kay_D Are you sure the mounts are “normal” FAT? No overlayfs or the like on top of it?