Regularly creates sync conflicts for untouched files

(Trent M) #1

I opened an issue on the bug tracker because I thought I found an honest to goodness bug but I got closed and bounced over here. My b.

So, I use todo.txt. I keep it as a subfolder of my Documents, and my Documents is synced between my two computers, my two servers, and my phone. I used to have simpletask, but I stopped using it in favor of a web UI on my server.

In the middle of every night, and often during the day when I’m actively productive, sync conflicts will appear for todo.txt and done.txt. But they only appear when my phone has syncthing enabled, so I must presume that syncthing-android is the cause of this. Those files don’t get touched on my phone at all! Or rather… I’d hope they don’t! I don’t know why my phone continuously asserts that its untouched files are the newest versions in contradiction to the other hosts which really do have the newest versions, creating those conflicts.

todo.txt is written in bash, which I believe means that instead of actually modifying an existing file, it will just make a new one with the same name and replace the old one. So doing tasks, cleaning my done.txt and moving it to the archive, I think technically results in the creation of new files. I wonder if this is part of what makes syncthing-android wig out so hard?

I’m not sure what I would do to diagnose this and make sure that todo.txt and done.txt aren’t actually being continuously touched by my phone somehow. Recommendations on that?

(Audrius Butkevicius) #2

Do you have ignorePermissions everywhere? Android storage is usually formatted using vFat or something like that that does not support permissions, so I suspect every time you sync a file to android, it comes back with different permissions, causing these conflicts.

(Trent M) #3

I have it set on the Documents repo on the phone, but by “everywhere” do you mean that the other hosts should have it set too?

(Audrius Butkevicius) #4

Usually no. Did you check if the content of the files is different?

(Trent M) #5

The content of the files is different; the sync-conflict files always have old content. So if I add a task to my todo.txt and a conflict appears, todo.sync-conflict*.txt has the old content of the file.

(Audrius Butkevicius) #6

Perhaps you have some file indexer on the phone that happens to open and close the file modifying it’s timestamp, hence causing a version increase.

(Trent M) #7

I’ve considered that possibility but I honestly have no idea how to investigate that kind of thing. As far as I know, the file manager I use, Amaze, doesn’t index things. And I made sure simpletask was uninstalled.

What would you recommend I do to find out what’s touching those files?

(Audrius Butkevicius) #8

Not sure, this is a question beyond syncthing.

(Audrius Butkevicius) #9

I’d try to figure out if it happens immediately after sync, or some time later, and what is different between the old old version and new old version (coming from the phone).

#10

Hi! A while ago I was trying syncing wy work files between Home and Work PCs using local wi-fi hotspots and my smartphone. I noticed that syncthing Android could not modify file date/time properly and the file on the smartphone was listed with a timestamp of the syncing activity. I searched this forum and found a discussion on this problem but there was no solution as far as I remember. I ended up no longer using my phone for any sort of two way sync to prevent the mess with timestamps.

I am not sure but I think this might be your case as well. This is the supposed scenario:

  1. You modify your todo.txt on your PC at timestamp t1.
  2. Cellphone syncs but the file gets the timestamp t2>t1. The PC is unaware of that for a while and has the same file with timestamp t1.
  3. You modify your txt once again at timestamp t3>t2.
  4. Your devices sync and find out that your file was changed on both sides because they both have timestamps later than t1. Since t3>t2, you get the t1-modified file on your PC as a sync conflict with timestamp t2. And on the phone it will get either t2 or t3.

Could you check whether this is your case?

(Trent M) #11

That theory seems sound to me. My phone’s regular file manager doesn’t let me see second precision, however, ConnectBot’s local terminal did let me get into my Documents directory inside the syncthing data directory, and stat its copy of todo.txt. One interesting thing of note is that my phone does not have sub-second precision at all!

On my work computer, todo.txt was changed at 14:34:12.506667755 -0400. On my phone, though, todo.txt was changed at 14:34:16.000000000.

Since I noticed the phone was presently ahead, I went ahead and added an arbitrary task to todo.txt. A conflict file was not created by this or the previous add, but on the phone, the change time remains in the future.

(Audrius Butkevicius) #12

We work around the timestamp issue in android by having a virtual mapping of mtime on disk when we created the file to mtime of what we advertise to others.

#13

If there is a solution it’s good news indeed. But could you please explain to a Windows user what mtime is or provide any links?

(Trent M) #14

He’s not describing a solution. He’s just describing why the problem isn’t what we were starting to think it was.

2 Likes
#15

I can confirm this sadly existing behaviour.

I wonder why syncthing uses the “Change” and not the “Modify” date.

Slave vfat Android 9 (OmniROM)

stat xxx.jpg

File: `xxx.jpg’

Size: 601064 Blocks: 1216 IO Blocks: 512 regular file

Device: b301h/45825d Inode: 108391 Links: 1

Access: (770/-rwxrwx—) Uid: ( 1023/media_rw) Gid: ( 1023/media_rw)

Access: 2015-07-14 00:00:00.000000000

Modify: 2015-07-14 22:39:42.000000000

Change: 2019-05-07 10:38:22.000000000

Master ext4 Linux Mint 19.1 (Ubuntu 18.04; 4.18.0-18-generic)

stat xxx.jpg

File: xxx.jpg

Size: 601064 Blocks: 1176 IO Blocks: 4096 regular file

Device: 804h/2052d Inode: 2886257 Links: 1

Access: (0644/-rw-r–r--) Uid: ( 1000/ martin) Gid: ( 1000/ martin)

Access : 2019-05-08 10:21:26.441523414 +0200

Modify: 2015-07-14 22:39:43.818750000 +0200

Change : 2019-03-07 10:43:58.304522864 +0100

Birth : -

There are 3 kind of “timestamps”:

Access - the last time the file was read

Modify - the last time the file was modified (content has been modified)

Change - the last time meta data of the file was changed (e.g. permissions)

(Audrius Butkevicius) #16

I don’t see how change vs modified is relevant in this discussion or problem.

We check for modified + permission changes, essentially reimplementing what changed means but in a cross platform way, because changed is not available on Windows and a few other platforms.

#17

Sorry, I maybe wrote in the wrong thread due to missing knowledge. :slight_smile:

I opened now my own question thread about my specific problem with “Out of Sync” errors on master.