After the recent daylight saving time change to standard time, ST for Android reindexed the entire contents of the SD card (450 GB, 210,000 files, 30,000 directories) for about 3 hours because there was a one-hour shift for all files.
I tried setting the time zone shift to daylight saving time on my phone manually, it reindexes whole SD card now again, but the file times are already correct, compared to the backup on one ST client (it is set to “Send only” to protect the data).
I was quite scared when I first detected that it wanted to rescan and resync the whole SD card.
I added the SD card to ST sharing in the summer and I sync it manually about once a month, sequentially to two different locations (Windows + Raspberry Pi client), so I can detect that something has been added/lost or changed (using Total Commander - Synchronize dirs over LAN/SMB), otherwise there is normally a “Paused” in ST on all clients.
Is there any way to correct this ST behaviour on Android?
What is causing this? Android API date/time management for SD cards?
Is this happening the same way on internal storage, I just haven’t noticed it yet?
I know that Android/Linux uses UTC, but that’s all I understand.
Thank you for your attention and ideas on how to fix this would be greatly appreciated.
There is special handling of modification time in Syncthing only on Android due to the OS’s restrictions that prevent apps from modifying mtime directly, however someone more knowledgeable would need to explain how it relates and affects timezone/daylight saving changes.
I can only say that you’re lucky that your device still runs Android 9 and thus managed to index the files in just 3 hours, because otherwise indexing this many files could easily take days on Android 11 and newer.
Heh. This is the gift of FAT filesystems storing timestamps in local time. Local time changes, timestamps change. Timestamps changed, we gotta check for changed contents. Does this seem incredibly stupid? Yep, sure does. Why are we using FAT filesystems in 2023? Who knows…
Thanks, I’ve already found out something about this mtime limitation on Android in the meantime, annoying.
I’ve also already found out the difference in SD card access on Android 9 and 11+ (scoped storage + FUSE), also annoying. On my next phone I’ll have Android 13, so I’m curious about the performance, and hopefully root will save it eventually.
Google must have reasons to keep it on Android, maybe purely for backwards compatibility? Also, I think all external USB sticks/disks and SD cards come pre-formatted to exFAT, so FAT filesystems aren’t going anywhere soon .
It’s not only about SD cards, it applies to both internal and external storage. Rooting doesn’t really change anything. There are some commands that you can run through ADB to enable legacy storage access, however they also break so many apps that I’d say they are basically useless.
Thank you all for your replies. So is it legitimate to escalate this as a bug or as a request for improvement? How?
I’m a newbie here in the forum and although I’ve tried searching for various keywords here and on the internet, I haven’t found a similar problem and/or bug and solution for ST yet.
I think quite a lot of people on Android sync from the SD card as well.
Likely nothing can be done, as this is literally how the filesystem works. This shouldn’t be a big deal under normal circumstances though, as timezone/daylight changes don’t happen very often (unless you travel all the time between different time zones?).
Most flagship devices don’t support SD cards anymore, however this problem affects both internal and external (SD card) storage, as both of them use the same VFAT as their filesystem. In other words, basically all Android users of Syncthing are affected.
Workaround? I think BT could try detecting a change in the received date/time from the Andoid API on its checked file somewhere in the .stfolder, confirm it with a set of random other files, notify the user, internally use and update the correction constant X hours to recalculate the date/time and not perform unnecessary full rescan and resync against other working clients.
I would really appreciate if this was at least some developer option in the settings.
How do other synchronization tools solve this problem with FAT32 and Android? Dropbox? One Drive? Nextcloud? Google Drive? Resilio Sync? ownCloud? Pydio? Seafile? SparkleShare? Box Sync? Cloudike? CloudMe? Egnyte? Gladinet? GoDrive? GoodSync? MediaFire? Mega? SecureSafe? SpiderOak? ShareFile? SugarSync? Syncplicity? Tonido? Tresorit?
Which one should I switch to?
I haven’t used all of the ones you listed above, but most of them aren’t directly comparable to Syncthing.
Resilio Sync is the closest to Syncthing and it’s also impacted by time differences due to limitations of FAT.
The mobile apps for Google Drive, One Drive, Dropbox, Box Sync, pCloud, Seafile and others like it don’t sync a local folder between another device or cloud storage (not bidirectional sync tools like Syncthing). They’re closer to a file manager/browser with the ability to quietly auto-download files on-demand. You can see this by trying to add a new file, which requires selecting a file for upload.
Because of the way they work, file timestamps are in the cloud storage and visible locally independent of the underlying filesystem. So they basically don’t care what filesystems Android and iOS use. (In the cloud, the servers are most likely Linux or other Unix-like OS with timestamps in UTC.)
Just for the record, there is indeed a full rescan, however if the file contents haven’t changed, there won’t be any “full resync”. Syncthing will just update the timestamps on other devices without transferring any actual file contents.
Thanks for shedding light on the situation, I used to use the predecessor of Resilio Sync, but then I switched to ST, and it has worked quite satisfactorily up until now, but this bug is killing me.
I looked at it, in principle it might be enough in the current setup where I manually sync once a month, but I plan to do the sync on a new and more powerful phone once a day and fully automatically, and there the risk of not catching a changed file is extreme and I won’t go for it.
That it’s just rescanning I understand, but even then there will be time updates on all clients and they indicate hundreds of GB differences throughout the long hours of scanning, which is very stressful when you don’t know what’s really going on.
For now, I’ll use the workaround of manually setting the offset time on the phone and then unpause the sync sharing from the SD card. Once the sharing sync is complete, I’ll pause again and return the time on the phone to normal.
If I can contribute in any way to modifying the ST Android code to implement my time shift detection and full scan elimination described above, I’d be happy to get involved.
I’ve posted my ideas in the Github PR… Anyone here that is experienced enough with Android to be able to reproduce the problem, could go for adb logcat analysis and may be willing to work together with me testing the code by putting it in a fork beta?
I’m currently trying to reproduce the problem with the android emulator and DST change on Android 14 using Syncthing 1.27.1(rc1). But my files are not rescanned after a DST change in date&time settings. Can anyone here please provide exact instructions on how to reproduce the problem?
.