Changes only propagating one direction

Using ignore patterns to only sync (bidirectional) one subdirectory of docs/ from server to my phone. Changes propagate from server to phone but not from phone to server. Setting up identical ignore patterns on a separate computer (not phone) correctly does the bidirectional sync. All folders are set as Send & Receive. Permissions are set to Ignore on Android, as per normal.

Version: 1.4.0 on server and phone (although this has been going on for several versions, I can’t pinpoint with which version it stopped working).

Directory tree on server:

docs/
  documentation/
  gdrive/
  family/
    wife/
    scott/
      src/
      wiki/
        Home.md
        ....    

Folder directory on phone:

/storage/emulated/0/Documents/docs

No ignore patterns set on server. Ignore patterns on phone:

/family/scott/wiki/.git
!/family/scott/wiki
/family/scott
!/family/scott/
/family
!/family/
*

Is there something different about ignore patterns now on Android or is this a bug? Which STTRACE options will give you the correct logs needed?

Thanks!

What happens to changes in the other direction?

  1. New file created on phone: no log entries, not synced to server
  2. New file created on server: file sync’d to phone
  3. Existing file on both, deleted on phone or server, correctly deleted on both
  4. Existing file on both, edited on server: sync’d to phone correctly
  5. Existing file on both, edited on phone: not sync’d to server, no log entries.
  6. Now if that file is subsequently edited on the server, a sync-conflict file is correctly created on both devices.

To summarize: new files created on phone are not sync’d, nor are changes made on the phone to existing files. All other changes/deletes/conflicts are handled correctly.

Thanks!

Try opening the web ui and pressing the manual scan button after creating the file. I suspect change watching is not working well on android.

1 Like

Sorry, forgot to say that I had done that already. Looking at the Web GUI on the phone, started with Local State showing 3413 files. Added file test.md then hit rescan. Still showed 3413 files.

Edit: I have other folders on this phone without ignore patterns that sync fine in both directions without having to rescan.

Well…I deleted the folder from Syncthing and set it up again. Now it works great again. Hrmm. Sorry I didn’t think to try that first! Wonder what the glitch was though…

Well, I spoke too soon. It worked correctly for a short time then stopped working again exactly as I described above:

To summarize: new files created on phone are not sync’d, nor are changes made on the phone to existing files. All other changes/deletes/conflicts are handled correctly.

After it stopped working, I again deleted the folder from Syncthing Android and from the phone, recreated the folder in Syncthing, and then allowed it sync from the server. And again, for about 10-15 minutes it all worked correctly again…and then it didn’t :thinking:

Any ideas (including which STTRACE options might provide some insight)? Thanks!

Does it work if you press the scan button manually?

I suggest you try to enable scanner/model and see if there are any entries about the files you expect to sync.

Does it work if you press the scan button manually?

As before, no, the scan button doesn’t help. Other folders on the phone also sync just fine without using it (both ways).

I’ll post some logs from scanner/model. Thanks

Copied file on phone into /storage/emulated/0/docs/family/scott/wiki/diary/2020/IMG_20200329_182226.jpg

Before copy: Syncthing Android reports global state 38,122 files, local state 3418 files.

After copy: Same number of files. Logs with scanner and model enabled: https://firecat53.com/privatebin/?05c81a661b25d925#6pch9FVwXKvRzkYZRFeFQaonpteYVTcMzekKJozJeo21

No trace of the copied file being scanned that I can see.

Edit: logs after editing a file on the phone (wiki/diary/2020/2020-03-31.md): https://firecat53.com/privatebin/?b53a1b1d75270ed5#AtGJ1vvD54rdh4HAQFLfTco9fMFJVC4V8Afok6Tzyg1f

Editing same file on server (2020-03-31.md) after editing it on the phone to generate a conflict file: https://firecat53.com/privatebin/?be7c6fe98e6be3f0#937uwL8aWiWRLppHveYrxomxSLcEKNs2yiS3apy9dHKu

Note that the conflict file is correctly generated and synced to both server and phone.

Did you click the scan button after copying the file? If the watcher does not work you won’t see much in the logs, in which case its not very useful to us.

Of course… Wouldn’t make sense to do it before the copy :wink: That’s why this is so odd. It’s like Syncthing is blind to certain changes happening on the phone (but not all, as I described above). In any case, the watcher normally sees changes without using the scan button for the other folders on the phone.

Edit: I was able to reproduce this by creating a simple directory with subdirectory underneath and ignores setup to only sync the subdirectory to the phone. Again, it worked for a short time, but after coming back to it with no changes after several hours, it was “stuck”.

So the logs were captured during when scan button was pressed?

The logs state that family is ignored, so your patterns are wrong. I think nowadays you can just un-ignore the specific directory you want and ignore the rest, no need for the 3 entries per directory (and you forgot to unignore !/family/scott)

Seems that the simplified ignore statement works fine, thanks. However, things still get “stuck” after a short time.

Below I provided more logs for both when it’s working correctly and then from later when it stops working.

Ignore patterns:

!/subdir/subsubdir
*

~/test/subdir/subsubdir/new.txt

Logs from editing new.txt (edit happened at 09:10:42…ran vi new.txt && date in termux) on phone while it’s working: https://firecat53.com/privatebin/?2fdae32ed865b410#6fePbe2M1CGZS7aQC1J6ch6H5ZdR1dGS4HMasUykv42G I did not need to push scan and it synced the change correctly.

Later, changed new.txt again at 11:02:52. Syncthing doesn’t see the change. Scan button not pushed. https://firecat53.com/privatebin/?15abe414f28c915e#8Qr5TYn9jPbhToAN4NPYo5k6aYZgruPXCkJ7mGPVPZ5c

Now, with that edit still in place and not synced, copied a file adminzen.png into subsubdir/ at 11:07:08. File still not seen by Syncthing. https://firecat53.com/privatebin/?06c2e66d5aa20382#5ZQNbPGitsPvt4hYrZ2KywsVE4miZ374E6br7boiauRn

With both these changes still not synced, pushed rescan as close to 11:11 as possible: https://firecat53.com/privatebin/?0cfe8559b7caa9e5#23dC3muhTPaxjbAvZqCM2o7zL61oAJ7eDPWzAoAsemZG

Changes still not synced after the rescan.

Hopefully that helps!

Can you open a github issue?

It seems the scan succeeds when scanning the file specifically, but fails when scanning the directory as a whole.

Relevant parts that should land in the github issue are:

04-01 09:10:52.869 I/SyncthingNativeCode(10855): [BQMAU] DEBUG: Walk [subdir/subsubdir/new.txt] Matcher/[!/subdir/subsubdir !/subdir/subsubdir/** * **/* */** **/*/**]@0x4001165b00
04-01 09:10:52.869 I/SyncthingNativeCode(10855): [BQMAU] DEBUG: rescan: File{Name:"subdir/subsubdir/new.txt", Sequence:13, Permissions:0660, ModTime:2020-04-01 16:06:52.363230458 +0000 UTC, Version:{[{ATUYUND 5} {BQMAUHB 5}]}, Length:49, Deleted:false, Invalid:false, LocalFlags:0x0, NoPermissions:false, BlockSize:131072, Blocks:[Block{0/49/2755465447/79d04840fe79e58c912456068542762229aa32ac36168b4078ee070cf2573a4b}]} 1585757442 -rw-rw----
04-01 09:10:52.870 I/SyncthingNativeCode(10855): [BQMAU] DEBUG: to hash: subdir/subsubdir/new.txt File{Name:"subdir/subsubdir/new.txt", Sequence:0, Permissions:0660, ModTime:2020-04-01 16:10:42.865204718 +0000 UTC, Version:{[{ATUYUND 5} {BQMAUHB 5}]}, Length:63, Deleted:false, Invalid:false, LocalFlags:0x0, NoPermissions:false, BlockSize:131072, Blocks:[]}
04-01 09:10:52.870 I/SyncthingNativeCode(10855): [BQMAU] DEBUG: real to hash: subdir/subsubdir/new.txt
04-01 09:10:52.870 I/SyncthingNativeCode(10855): [BQMAU] DEBUG: Walk progress done 3427d-oqk5y [subdir/subsubdir/new.txt] Matcher/[!/subdir/subsubdir !/subdir/subsubdir/** * **/* */** **/*/**]@0x4001165b00
04-01 11:11:00.750 I/SyncthingNativeCode(10855): [BQMAU] DEBUG: Walk [] Matcher/[!/subdir/subsubdir !/subdir/subsubdir/** * **/* */** **/*/**]@0x40002b81b0
04-01 11:11:00.751 I/SyncthingNativeCode(10855): [BQMAU] DEBUG: ignored (internal): .stfolder
04-01 11:11:00.751 I/SyncthingNativeCode(10855): [BQMAU] DEBUG: ignored (internal): .stignore
04-01 11:11:00.751 I/SyncthingNativeCode(10855): [BQMAU] DEBUG: ignored (patterns): subdir
04-01 11:11:00.752 I/SyncthingNativeCode(10855): [BQMAU] DEBUG: Walk progress done 3427d-oqk5y [] Matcher/[!/subdir/subsubdir !/subdir/subsubdir/** * **/* */** **/*/**]@0x40002b81b0

Which is a bug… It seems the super smart “unignoring” is not working. I suspect

!/subdir/subsubdir/**
!/subdir/subsubdir
/subdir/**
!/subdir
/**

as patterns might work?

3 Likes

Ah, brilliant! New ignore patterns work perfectly to get around the bug. I’ll post that on Github shortly. Thanks so much for the help!!

Edit: should the issue be posted under syncthing or syncthing-android? The problem doesn’t seem to happen without the phone involved.

It’s a syncthing bug.

1 Like

Github issue submitted. Thanks again for the help.

2 Likes