Just did a quick test:
Rsync syncthing git repo to new syncthing shared repo: 2331 files and 432 dirs (including the .git/* stuff).
Shared with android phone and waited until synced (no other cluster member sharing this other than Linux host and Android phone).
Rebooted phone.
1225 “Locally Changed items” in hierarchy.
Will send logs later.
===Rich
I had a gut feeling this might be the case (fs cache having better precision than actual fs) when I read the initial issue but didn’t dare suggest something so idiotic 
It won’t help, as you already do that by pressing the button, but it comes back because the accuracy of the timestamps is inconsistent over time.
I could imagine an “ignore mtime” option or interval. If set to for example 2s we use that much leniency when doing the “is this fileinfo equal” comparison in the scanner, or we truncate to that interval when doing the chtime in mtimefs. Of course, the mtimefs was designed to avoid having to do precisely that…
Figure 1: The new folder settings panel
Rsync has --mtime-tolerance or something along those lines, so it’s not totally unseen, but to be honest, fuck android, people should go out in the streets with pitchforks, oppose to everyone being forced to put bandaids.
Funny enough, Google itself official dropped support for SD cards, so perhaps the riots should be against the vendors that unleash this non-sense to the wild.
Hello,
It seems that the SDCARD only preserves only even integer seconds, not odd ones. Any odd integer second is preserved during a session, but not across reboots, so there is some volatile caching of metadata. At least on my Android system.
I did this (forgive me for not having done it before):
Create a shared dir with the syncthing source (plus ./git/* stuff)
Syncthing share with Android
Start syncthing on phone - wait until all up to date on both sides.
Reboot phone.
Install Busybox. Busybox find does not seem to support -printf options.
Create tiny shell ‘find-stat.sh’ script on Android syncthing-repo:
STAT=$(stat -c "ctime: %z mtime: %y" $1)
SHA1=$(sha1sum $1)
echo -ne "$(echo "$SHA1 $STAT" |tr -d '\n' )\n"
find . -type f -exec sh find-stat.sh {} \; > first.txt
first.txt # After reboot but before reverting.
Revert files.
Run:
find . -type f -exec sh find-stat.sh {} \; > second.txt
second.txt #after reverting.
Reboot phone and start syncthing
Run:
find . -type f -exec sh find-stat.sh {} \; > third.txt
third.txt #after reboot but before reverting.
Revert files.
Run:
find . -type f -exec sh find-stat.sh {} \; > fourth.txt
fourth.txt # after reversion.
Diff second.txt and fourth.txt #Ctime diffs but no mtime diffs.
Diff second.txt and third.txt mtime diffs only
An example:
From second.txt:
2ec3c7a83daa7bef0eccf1cb8108058a3368c1ea ./lib/fs/fakefs.go ctime: 2019-06-24 19:34:34.000000000 mtime: 2019-05-31 13:24:33.000000000
80f3091a614e5d7a2b5ebd9c79837d0fe1ed6f50 ./lib/fs/basicfs_test.go ctime: 2019-06-24 19:34:34.000000000 mtime: 2019-04-11 11:20:21.000000000
11f41fb9a098b4f8db3865ee3e81b21954cee840 ./lib/fs/basicfs_watch_eventtypes_inotify.go ctime: 2019-06-24 19:34:34.000000000 mtime: 2018-05-03 10:16:25.000000000
8cc8e4ad846ddaef09de8ba32304f8f65a64f440 ./lib/fs/basicfs_fileinfo_windows.go ctime: 2019-06-24 19:34:34.000000000 mtime: 2019-02-25 15:34:33.000000000
29f4bb75d21be3cb4b2f3d82b736fe3ac8247d29 ./lib/fs/basicfs_watch_errors_others.go ctime: 2019-06-24 19:34:34.000000000 mtime: 2017-10-31 14:26:57.000000000
f164fa42565d2500e7933f0262cb505aa76844e9 ./lib/fs/basicfs_fileinfo_unix.go ctime: 2019-06-24 19:34:34.000000000 mtime: 2019-02-25 15:34:33.000000000
feeb19f66d443877f4c4f94594702b59d0714a97 ./lib/fs/filesystem.go ctime: 2019-06-24 19:34:34.000000000 mtime: 2019-05-31 13:24:33.000000000
From third.txt:
2ec3c7a83daa7bef0eccf1cb8108058a3368c1ea ./lib/fs/fakefs.go ctime: 2019-06-24 19:34:34.000000000 mtime: 2019-05-31 13:24:32.000000000
80f3091a614e5d7a2b5ebd9c79837d0fe1ed6f50 ./lib/fs/basicfs_test.go ctime: 2019-06-24 19:34:34.000000000 mtime: 2019-04-11 11:20:20.000000000
11f41fb9a098b4f8db3865ee3e81b21954cee840 ./lib/fs/basicfs_watch_eventtypes_inotify.go ctime: 2019-06-24 19:34:34.000000000 mtime: 2018-05-03 10:16:24.000000000
8cc8e4ad846ddaef09de8ba32304f8f65a64f440 ./lib/fs/basicfs_fileinfo_windows.go ctime: 2019-06-24 19:34:34.000000000 mtime: 2019-02-25 15:34:32.000000000
29f4bb75d21be3cb4b2f3d82b736fe3ac8247d29 ./lib/fs/basicfs_watch_errors_others.go ctime: 2019-06-24 19:34:34.000000000 mtime: 2017-10-31 14:26:56.000000000
f164fa42565d2500e7933f0262cb505aa76844e9 ./lib/fs/basicfs_fileinfo_unix.go ctime: 2019-06-24 19:34:34.000000000 mtime: 2019-02-25 15:34:32.000000000
feeb19f66d443877f4c4f94594702b59d0714a97 ./lib/fs/filesystem.go ctime: 2019-06-24 19:34:34.000000000 mtime: 2019-05-31 13:24:32.000000000
A mtime configurable window, similar to the rsync --modify-window option
would be very handy.
Once again, sorry for not having done this previously.
Attached archive with all four txt files.
===Rich
sha1ctimemtime.tar.xz (94.9 KB)
Yeah, I’m not sure what to do with this. The sheer stupidity of the android handling of this suggests we should just stay away.
Hello,
Please don’t:
fuck android, people should go out in the streets with pitchforks, oppose to everyone being forced to put bandaids.
It would be a shame to drop any part of Android, not least for the time many have spent adding it.
The shitty, second hand Android phone with the cracked screen and dodgy battery may be the best computing device an economically challenged guy/girl can ever aspire to own in a less developed country. The functionality you offer for these people is just as useful to them, if not more, than that which you offer more advanced users in developed countries.
I cannot bear iOS systems; I often feel like waving your pitchfork at Apple for their lack of file system support. But I’d prefer to see their devices included in your fantastic application, rather than excluded, to the detriment of many. It makes collaboration so much easier for everyone.
Just having the meta data from your database and the blobs would be still be useful on iOS without general file system support.
For iOS users, could you not just deliver up data to a browser with enough info for a browser, or other app to try, to open it in the same way xdg-mime/xdg-open works under Unix? Just saying open this blob in a web browser with these ‘open’ characteristics would be remarkably useful on portable systems. And they could still be part of the cluster, regardless of whether or not they could directly access files.
When file system incompatibilities block file transfers (such as NFC/NFD Unicode normalisation for Macs or case sensitivity for Macs and Windows), would it not be better, from an ease of use point of view, to transfer the blob, point out the problem in the UI, and, perhaps munge the filename in some way, giving the user the data and some resolution? This file name munging is already done with conflicts and versioning.
Please don’t reduce the amount of systems syncthing works on, no matter how brain dead those systems might be. Its cross platform plurality is what makes it useful to so many across the globe.
Forgive the ramble, and thank you, catfriend1 and Jakob for
staying with this for so long. I’m very grateful and don’t wish
to sound like a whining SJW (or not too much). 
===Rich
Slightly OT: googled around and found that the MTP implementation on Android is also buggy related to mtime , just like to Share the article with you https://freefilesync.org/forum/viewtopic.php?t=5896
How to preserve time stamps? - Total Commander
Tracker:
PR work from the past:
Maybe I’m asking for the non-implementable but would it be possible to add “ignore mtime” xml node to the folder node so the wrapper who knows when a folder lies on the external sdcard fat-formatted can set this in config.xml to help the user avoiding this common Problem? I don’t know much core code so can’t say if it is a good demand or not. Just feel this is somehow necessary as I also know a lot of user reports complaining about data loss after they got in sync again after Android’s timestamp mess took place. Especially in sendreceive setups. SendOnly could be “beared” by hitting override again and again over time.
We use mtime to determine if the file was modified, if we just ignore it, files can look like they are never modified, hence never synced.
The MTP thread talks about the issue but not about how it’s solved, so not super useful.
Ok I see… Any other ideas what could help here assuming Google doesnt do anything about it? Maybe checking file size as a second criteria.
That already happens, we check mtime and size, and hash if either differs. I think the solution is clear and ugly (apart from fixing (or pitch-forking) android - any volunteers ): Add an option for a 2s mtime window. Or potentially don’t add an option but hard-code a 2s window on the android build tag.
I would not be ok with 2s across the board on android. I think adding a generic “within 2s” is not good enough either, as you still want to preserve the timestamps via mtimefs as that is why it’s there, so you need some sort of:
onDisk = stat(path)
Virtual, real = mtimefs.load(path)
Tolerance = config.Folder.MtimeTolerance
if onDisk.Difference(real) < Tolerance {
Return virtual
}
Return onDisk
And then default tolerance could be if goos=android && root.startswith(/storage/0/) or something along those lines.
That’s what I meant by a 2s window.
On my device /storage/0 is the internal storage (no idea whether it has this defect) and /storage/*someSDCardIdentifier* and I think there’s also systems where storage is in /media. That’s why I thought it’s futile to try to detect when it is needed and just enable it across the board, but sure we could try. And I am not aware of a use case outside of android, that’s why I thought it might not be necessary to add a new config option and just add the clutch to android.
Well perhaps this mtime cache nonsense is on fat on linux in general. I have no idea.
I think it should be enabled by the wrapper for specific folders for the reason the wrapper can ask if there is a sdcard underlaying and go cannot. The option can be hidden or only be available for android builds that would be fine to me. Would love to help from the wrapper side.
Why did you have to mention that explicitly - until now I successfully suppressed that thought and blissfully assumed that this must be an android (only) fuckup xD
It is Android only, found a Link yesterday pointing out it’s on googles bugtracker since 4.4.x (yeah thats long). Hm checked again seems also to depend on mount options in unix.
Another thought: Wait for an incoming file to be written on android fat, store its perceived mtime in a second column of the db aside the expected mtime. If the perceived mtime did not change during next scan, report the expected mtime to the sync cluster devices so they know android still has the right version of the file. E.g. this helps in cases where I download a video with newpipe and the mtime constantly goes up until the file finished writing and then settles to a constant value letting the puller on the other side fetch it successfully (it couldnt complete pulling while the file is in download progress)
Good to know, thanks. Could you share that link for posterity?
And that wouldn’t work, as the cache is only dropped and thus the “real mtime” reported after a phone restart, not on next scan (I am assuming Richards steps above as the canonical description of the problem Ignore Permissions / Different mtimes on Android Sdcard).
The download of a big file problem is a different issue, I’d argue that’s the fault of whatever you use to copy that file. It could use a temp file during transfer or only update mtime once it’s finished. Or you could increase Syncthing’s change watcher delay. Anyway lets keep this discussion focused on the mtime caching problem, if necessary this can discussed in a separate thread.