Ignore Permissions / Different mtimes on Android Sdcard

Support
24

Here’s another just now from an gocryptfs encrypted file system.

[XHTJH] DEBUG: mtimefs.go:71 basic /storage/B977-5AE1/Android/data/com.github.catfriend1.syncthingandroid.debug/st/crypted-shared Lstat C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ {0x4000fd18c0} <nil>
[XHTJH] DEBUG: load: loaded 2019-05-17 19:05:41 +0000 UTC / 2019-05-17 19:05:41.214081426 +0000 UTC for C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ
[XHTJH] DEBUG: mtimefs.go:71 basic /storage/B977-5AE1/Android/data/com.github.catfriend1.syncthingandroid.debug/st/crypted-shared Lstat C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ {0x40004ae300} <nil>
[XHTJH] DEBUG: load: loaded 2019-05-17 19:05:41 +0000 UTC / 2019-05-17 19:05:41.214081426 +0000 UTC for C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ
[XHTJH] DEBUG: receiveonly/cpdpj-ytgwa@0x4000185080 taking shortcut on C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ
[XHTJH] DEBUG: mtimefs.go:56 basic /storage/B977-5AE1/Android/data/com.github.catfriend1.syncthingandroid.debug/st/crypted-shared Chtimes C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ 2019-05-17 19:05:41.214081426 +0000 UTC 2019-05-17 19:05:41.214081426 +0000 UTC <nil>
[XHTJH] DEBUG: mtimefs.go:60 basic /storage/B977-5AE1/Android/data/com.github.catfriend1.syncthingandroid.debug/st/crypted-shared Lstat C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ {0x400035a840} <nil>
[XHTJH] DEBUG: save: saving 2019-05-17 19:05:41 +0000 UTC / 2019-05-17 19:05:41.214081426 +0000 UTC for C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ

===R

25

Looks like it’s doing the right thing.

I guess a longer log with the file arriving, being scanned (enabling scanner together with fs), becoming out of sync, and reverting would be more interesting ideally you should filter this down to a single file.

26

There’s quite a lot of reverted files. Can I use the 'taking shortcut' from 'model' to find a file of interest?

===R

27

Here’s a go enabling fs, model and scanner. It’s 13K so uploaded, too.

===Rst_paste3.txt (12.8 KB)

[XHTJH] DEBUG: mtimefs.go:71 basic /storage/B977-5AE1/Android/data/com.github.catfriend1.syncthingandroid.debug/st/crypted-shared Lstat C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ {0x40015972c0} <nil>
[XHTJH] DEBUG: load: loaded 2019-05-17 19:05:41 +0000 UTC / 2019-05-17 19:05:41.214081426 +0000 UTC for C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ
[XHTJH] DEBUG: rescan: File{Name:"C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ", Sequence:1774, Permissions:0750, ModTime:2019-05-17 19:05:41.214081426 +0000 UTC, Version:{[{NPAZACX 1} {WKOOOGB 1} {WR6WMT7 1} {2VVXIMR 1} {3ABJ3OJ 2}]}, Length:13131181, Deleted:false, Invalid:false, LocalFlags:0x0, NoPermissions:false, BlockSize:131072, Blocks:[Block{0/131072/3920915798/6fe58fd9cbb7e19de29295c710453d7a98dc4d0ac6492dedf3cd98e4429f0a2b} Block{131072/131072/55154110/00f5df6fb2877b4e252f1333391e3c7bf00600bb46373ef38aea72b3645f0a79} Block{262144/131072/3648095085/71ef26f4aa762df443caf458a0b9c6691f364771647f8da6d0ba84636c0f32ff} Block{393216/131072/2718853866/83e21bada7683f291b66f7b2c685ed5d8b84a4ed23c9d55818b49c5c17675304} Block{524288/131072/474282646/b68b717ee2470cb428f530942cdc10a60350afc9c9b750ddf36a011668a462e5} Block{655360/131072/598679759/4c2e648de7924587de108e8822f9a0e9ac4dd4453e64f39d66272cdaeaacb8a9} Block{786432/131072/2571126889/69923b3c6286c2a51e7e56fd4bfeee09b585ec6904cc427a70bf6791970019f3} Block{917504/131072/1360846349/cee2e0396488dae860d9967ebe8e31301ee1f219da0437ea2e5cc0d29aeb1cbc} Block{1048576/131072/1097097279/8714012e31004e05cea1eb111ba7a0fe721dd7cfa178f0453450a9dd89453c78} Block{1179648/131072/2630940416/4d7d9070a0428159dda9b5efb3041df7f3854cbbd5ccc5bd38d904b5c80edfa9} Block{1310720/131072/3795281701/dc7bfbefbe69aeb5b0434d80571c4ca6d12a0662988990de0f2223e1b070aba9} Block{1441792/131072/2874134457/12db166d20dff0936c3ced79e1d42a4e48e1d6ced2efa819d010b64943b0f3df} Block{1572864/131072/4163101715/8b13b372772ebd51c8b94767b2d50d0b12c200774ee29e05763f9c208ae57e0a} Block{1703936/131072/3098123671/29a8340e66685199a99feb9412e1b59e688d5aea4743675286bdbfe7565ec16f} Block{1835008/131072/1480136479/46c93340ad4592be1849a85b5ff6000a34df7d1464b62557049cc7423bf4fc76} Block{1966080/131072/4193584507/799d47e9be08090670e52358ca618915df452fec7d4943f2a72ede18ceb44366} Block{2097152/131072/762823171/83f2c5a4e4b0ede786b8c074ad5b868c23f7dcc21841659aae244fc28b67d399} Block{2228224/131072/920170385/0ef8cd900a25b8dfe5746b7bda2e6d4b63b1fd7fc90305c64d27dc93f7588def} Block{2359296/131072/2701320898/1480ba6c53dbdd7773524ba4dba03246d1e696af203dd545977910d77d570b85} Block{2490368/131072/1757613839/ed7c94c936e62e6b55e1cc68ae2e6dc11599614b6ee12eecaec650b7582e948d} Block{2621440/131072/3924627115/ec43801dc0d43add833f6f13809651e9227ec273564987f30d3629b5a6652763} Block{2752512/131072/4074562965/ad4f09cc67209bb3cf0cc1961c7865c99a8a2b44c815ea5f6c95f807f8d7fd7f} Block{2883584/131072/1900407944/1b975646bd3e73d3a7cf2dfd36c648cf8326cf26eb7856339490ed2faef3221f} Block{3014656/131072/1541517119/56caa128e30c960c4b01592c37c58143e2583eedd3def06d0a30132951fd6559} Block{3145728/131072/117022612/7318fb12c8a16527f24b3d8e333f18d82af9696402c59981c58cbb995f47fa83} Block{3276800/131072/1175726016/a31fc279eb830149d3256a3dd450df9b267415fb83313701e6589d2d695ae508} Block{3407872/131072/1133050906/dd0cbab332fb8bd3e13eb28817e7c0abe10f900d7ff030a58bc3a292809136c4} Block{3538944/131072/3260438469/08a6aea480ba04a15bc11b206329797846516a05d2a925f6cba5f78beb20b3a7} Block{3670016/131072/1319164946/5034165a57c3136dcd674d52f937ebaff3b748dea5dec606e0248864acf18074} Block{3801088/131072/76514149/4132602d997bb2f53390f41788d1297a9ec30c8e889a86c8cef383c3a8c2f1ae} Block{3932160/131072/3420012210/92054f9f666d28a2bfb15f7e189d33c9e2d13a26fa699e51dd7460011f673c81} Block{4063232/131072/1557871090/9ee6eda3fcb84c6ed214c5cec52463c4002b08fddcc14b78e8cfbe4bd738e12f} Block{4194304/131072/2742610123/06a1dd7219b12f999755a1c026de2ae8e3540c2d701adfd6a0f829c4402f75a8} Block{4325376/131072/3620206951/f1f2f2ba378f04c8397ab8821a90d131d77a325cb6a1416bbcc823f6a7af5e3d} Block{4456448/131072/2800568359/ca3a0e0bf56e94a565fff3e0d7dbceea04576bec56664f2345189da5588bbfda} Block{4587520/131072/1919051117/32c4a6dba0f81cfe2b663f3749dba4be3ea80c3be74bc1e0a9a7b646e1739683} Block{4718592/131072/4190749975/b4fcb993b6efc336b96a145aeae0c95933d0fa021290ba266ab0db4771c71937} Block{4849664/131072/3750754480/db696c8406060f007cd9765b35536ef969c0257fd270f992ac63144d05c2a81b} Block{4980736/131072/3236002196/b820057d93d0e77b6141cf1799a3eca7a801cd159a9fec31ff9b5d0cc8dcf28c} Block{5111808/131072/2511127078/49a967fa586278df8ec0f98d44af3e25da0d03c0159f1f0af94aab6caa32571c} Block{5242880/131072/3672539103/d33b4618b14b719aac3e5c5a5e2829fb7c50fcd5ef2752f2bd0afa22dcd4ee78} Block{5373952/131072/3165812632/53d1070fef1ef2f997614026301031032dfd9b527e0ddcc46312f546e466a258} Block{5505024/131072/1330770794/03854aaab818e9d8deb116bf40b1c4207625
28

Look at the attached file, not the inline paste, that didn’t go correctly.

===R

29

There is no part of the file arriving, just scanning straight away.

30

Actually, never mind, I think I see the issue.

MtimeFS handles Lstat, and I think Walk does not use that.

31
1 Like
32

Thanks for all your efforts on this.

===R

2 Likes
33

Hi,

Just updated to Catfriend1’s latest release incorporating changes as in b7c70a981734:

lib/fs: Enhance mtimefs, use everywhere (fixes #5777) (#5776)

The problem still seems to exist. :frowning:

===R

34

Same logs would help, specially around scanning and finding differences.

36

The same scanner logs would do, potentially with the diff cherry-picked. The diff was never applied upstream.

38

Hi,

Here’s a log for one file:

[XHTJH] DEBUG: load: loaded 2019-05-17 19:05:41 +0000 UTC / 2019-05-17 19:05:41.214081426 +0000 UTC for C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ
[XHTJH] DEBUG: mtimefs.go:88 basic /storage/B977-5AE1/Android/data/com.github.catfriend1.syncthingandroid.debug/st/crypted-shared Lstat C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ {0x40028c8d80} <nil>
[XHTJH] DEBUG: load: loaded 2019-05-17 19:05:41 +0000 UTC / 2019-05-17 19:05:41.214081426 +0000 UTC for C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ
[XHTJH] DEBUG: mtimefs.go:88 basic /storage/B977-5AE1/Android/data/com.github.catfriend1.syncthingandroid.debug/st/crypted-shared Lstat C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ {0x4001541d40} <nil>
[XHTJH] DEBUG: load: loaded 2019-05-17 19:05:41 +0000 UTC / 2019-05-17 19:05:41.214081426 +0000 UTC for C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ
[XHTJH] DEBUG: receiveonly/cpdpj-ytgwa@0x40001f3080 taking shortcut on C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ
[XHTJH] DEBUG: mtimefs.go:56 basic /storage/B977-5AE1/Android/data/com.github.catfriend1.syncthingandroid.debug/st/crypted-shared Chtimes C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ 2019-05-17 19:05:41.214081426 +0000 UTC 2019-05-17 19:05:41.214081426 +0000 UTC <nil>
[XHTJH] DEBUG: mtimefs.go:60 basic /storage/B977-5AE1/Android/data/com.github.catfriend1.syncthingandroid.debug/st/crypted-shared Lstat C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ {0x40000f2540} <nil>
[XHTJH] DEBUG: save: saving 2019-05-17 19:05:41 +0000 UTC / 2019-05-17 19:05:41.214081426 +0000 UTC for C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ

That’s with the calmh patch applied as above on git show bf744ded311:./lib/fs/mtimefs.go and using catfriend1’s f01f8d26ba6 with cherry-picked additions to his app/build.gradle.

A stat of the same file on the linux side:

File: ./C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ
  Size: 13131181        Blocks: 25656      IO Block: 4096   regular file
Device: fd00h/64768d    Inode: 32682803    Links: 1
Access: (0750/-rwxr-x---)  Uid: ( 1000/     rah)   Gid: (  100/   users)
Access: 2019-06-12 19:37:56.868586136 +0100
Modify: 2019-05-17 20:05:41.214081426 +0100
Change: 2019-06-05 17:14:47.956074496 +0100
 Birth: 2019-05-23 05:09:12.380379157 +0100

Sorry for the delay on this.

===Rich

1 Like
39

This does not have the relevant logs from the scan

40

Hello,

Another try.

This is what I’m doing to create this:

Enable these STRACE options: fs, model, scanner

Find a file with the ‘taking shortcut’ tag.

Grep log for for all instances of that file name.

Upload the results of the grep.

Is that correct?

Not sure why the block info wasn’t included before.

===Rich

[XHTJH] DEBUG: load: loaded 2019-05-17 19:05:41 +0000 UTC / 2019-05-17 19:05:41.214081426 +0000 UTC for C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ
[XHTJH] DEBUG: mtimefs.go:88 basic /storage/B977-5AE1/Android/data/com.github.catfriend1.syncthingandroid.debug/st/crypted-shared Lstat C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ {0x400110ed80} <nil>
[XHTJH] DEBUG: load: loaded 2019-05-17 19:05:41 +0000 UTC / 2019-05-17 19:05:41.214081426 +0000 UTC for C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ
[XHTJH] DEBUG: rescan: File{Name:"C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ", Sequence:4710, Permissions:0750, ModTime:2019-05-17 19:05:41.214081426 +0000 UTC, Version:{[{NPAZACX 1} {WKOOOGB 1} {WR6WMT7 1} {2VVXIMR 1} {3ABJ3OJ 2}]}, Length:13131181, Deleted:false, Invalid:false, LocalFlags:0x0, NoPermissions:false, BlockSize:131072, Blocks:[Block{0/131072/3920915798/6fe58fd9cbb7e19de29295c710453d7a98dc4d0ac6492dedf3cd98e4429f0a2b} Block{131072/131072/55154110/00f5df6fb2877b4e252f1333391e3c7bf00600bb46373ef38aea72b3645f0a79} Block{262144/131072/3648095085/71ef26f4aa762df443caf458a0b9c6691f364771647f8da6d0ba84636c0f32ff} Block{393216/131072/2718853866/83e21bada7683f291b66f7b2c685ed5d8b84a4ed23c9d55818b49c5c17675304} Block{524288/131072/474282646/b68b717ee2470cb428f530942cdc10a60350afc9c9b750ddf36a011668a462e5} Block{655360/131072/598679759/4c2e648de7924587de108e8822f9a0e9ac4dd4453e64f39d66272cdaeaacb8a9} Block{786432/131072/2571126889/69923b3c6286c2a51e7e56fd4bfeee09b585ec6904cc427a70bf6791970019f3} Block{917504/131072/1360846349/cee2e0396488dae860d9967ebe8e31301ee1f219da0437ea2e5cc0d29aeb1cbc} Block{1048576/131072/1097097279/8714012e31004e05cea1eb111ba7a0fe721dd7cfa178f0453450a9dd89453c78} Block{1179648/131072/2630940416/4d7d9070a0428159dda9b5efb3041df7f3854cbbd5ccc5bd38d904b5c80edfa9} Block{1310720/131072/3795281701/dc7bfbefbe69aeb5b0434d80571c4ca6d12a0662988990de0f2223e1b070aba9} Block{1441792/131072/2874134457/12db166d20dff0936c3ced79e1d42a4e48e1d6ced2efa819d010b64943b0f3df} Block{1572864/131072/4163101715/8b13b372772ebd51c8b94767b2d50d0b12c200774ee29e05763f9c208ae57e0a} Block{1703936/131072/3098123671/29a8340e66685199a99feb9412e1b59e688d5aea4743675286bdbfe7565ec16f} Block{1835008/131072/1480136479/46c93340ad4592be1849a85b5ff6000a34df7d1464b62557049cc7423bf4fc76} Block{1966080/131072/4193584507/799d47e9be08090670e52358ca618915df452fec7d4943f2a72ede18ceb44366} Block{2097152/131072/762823171/83f2c5a4e4b0ede786b8c074ad5b868c23f7dcc21841659aae244fc28b67d399} Block{2228224/131072/920170385/0ef8cd900a25b8dfe5746b7bda2e6d4b63b1fd7fc90305c64d27dc93f7588def} Block{2359296/131072/2701320898/1480ba6c53dbdd7773524ba4dba03246d1e696af203dd545977910d77d570b85} Block{2490368/131072/1757613839/ed7c94c936e62e6b55e1cc68ae2e6dc11599614b6ee12eecaec650b7582e948d} Block{2621440/131072/3924627115/ec43801dc0d43add833f6f13809651e9227ec273564987f30d3629b5a6652763} Block{2752512/131072/4074562965/ad4f09cc67209bb3cf0cc1961c7865c99a8a2b44c815ea5f6c95f807f8d7fd7f} Block{2883584/131072/1900407944/1b975646bd3e73d3a7cf2dfd36c648cf8326cf26eb7856339490ed2faef3221f} Block{3014656/131072/1541517119/56caa128e30c960c4b01592c37c58143e2583eedd3def06d0a30132951fd6559} Block{3145728/131072/117022612/7318fb12c8a16527f24b3d8e333f18d82af9696402c59981c58cbb995f47fa83} Block{3276800/131072/1175726016/a31fc279eb830149d3256a3dd450df9b267415fb83313701e6589d2d695ae508} Block{3407872/131072/1133050906/dd0cbab332fb8bd3e13eb28817e7c0abe10f900d7ff030a58bc3a292809136c4} Block{3538944/131072/3260438469/08a6aea480ba04a15bc11b206329797846516a05d2a925f6cba5f78beb20b3a7} Block{3670016/131072/1319164946/5034165a57c3136dcd674d52f937ebaff3b748dea5dec606e0248864acf18074} Block{3801088/131072/76514149/4132602d997bb2f53390f41788d1297a9ec30c8e889a86c8cef383c3a8c2f1ae} Block{3932160/131072/3420012210/92054f9f666d28a2bfb15f7e189d33c9e2d13a26fa699e51dd7460011f673c81} Block{4063232/131072/1557871090/9ee6eda3fcb84c6ed214c5cec52463c4002b08fddcc14b78e8cfbe4bd738e12f} Block{4194304/131072/2742610123/06a1dd7219b12f999755a1c026de2ae8e3540c2d701adfd6a0f829c4402f75a8} Block{4325376/131072/3620206951/f1f2f2ba378f04c8397ab8821a90d131d77a325cb6a1416bbcc823f6a7af5e3d} Block{4456448/131072/2800568359/ca3a0e0bf56e94a565fff3e0d7dbceea04576bec56664f2345189da5588bbfda} Block{4587520/131072/1919051117/32c4a6dba0f81cfe2b663f3749dba4be3ea80c3be74bc1e0a9a7b646e1739683} Block{4718592/131072/4190749975/b4fcb993b6efc336b96a145aeae0c95933d0fa021290ba266ab0db4771c71937} Block{4849664/131072/3750754480/db696c8406060f007cd9765b35536ef969c0257fd270f992ac63144d05c2a81b} Block{4980736/131072/3236002196/b820057d93d0e77b6141cf1799a3eca7a801cd159a9fec31ff9b5d0cc8dcf28c} Block{5111808/131072/2511127078/49a967fa586278df8ec0f98d44af3e25da0d03c0159f1f0af94aab6caa32571c} Block{5242880/131072/3672539103/d33b4618b14b719aac3e5c5a5e2829fb7c50fcd5ef2752f2bd0afa22dcd4ee78} Block{5373952/131072/3165812632/53d1070fef1ef2f997614026301031032dfd9b527e0ddcc46312f546e466a258} Block{5505024/131072/1330770794/03854aaab818e9d8deb116bf40b1c4207625838a04918528cb4c6dc9cb9b14be} Block{5636096/131072/2717823223/d9cb1242a445d5a856ed24306dcd60605407d033cab5e5e7e933cf2889d71a3d} Block{5767168/131072/1448588123/8b942b45d7efc40d19b3d0c3dfd770c524309a47fd43448a830c13bf45db36b9} Block{5898240/131072/639293231/2340910f587f3cdbb8bd9644c23e9cc60a764322b27d8f68231ce2a5b04a04b2} Block{6029312/131072/882341413/ac38182b6b1119d65685ee565faf957e2c22146a5f67b2852fad24b3daeed45a} Block{6160384/131072/3989870265/445aa0cd1762f8ed70eac21bc056395082ce1d7f337aac42f655faf754780923} Block{6291456/131072/1794659851/02e695764edb42cd2dc6b645ebf12038761b102584c33fb91ac66bd89320abe9} Block{6422528/131072/4191604317/da1936bddacb7a946c3a174d49b05bbb83f912514bd7fa0b8b3cdb528fdfcee0} Block{6553600/131072/2422637526/fb1a7b0ce4c25a7dc166953d5de8cd7b687c08a2f2d1cad82d75fd76d237db13} Block{6684672/131072/4084023314/d21ab1a416336879649a6729b98d00c226e933082490648ce4a086d73d6cd935} Block{6815744/131072/2072124299/fb0e5e50f5ab08f9c8ae03c22dc7b509ed9b53c5b143d14b1fc392a7527341a2} Block{6946816/131072/2788540011/5be110f3789eeb5386decea3dcb81bb4ec1d09d9edfa80838e14ded4d2a93a6d} Block{7077888/131072/3954338253/600bf7c244c2b92facb06b28e7678b58ae40765cda354642ae6275ac064ca2c4} Block{7208960/131072/4018793635/ea637ac886a460e569279592f2ce30ec33941f55b2e2a6521eb468f0778f692c} Block{7340032/131072/3970165180/9e8c9234516acbc3f1a6994bd0a3433454a0d5146249191da4e168e622c484f9} Block{7471104/131072/3593432921/83e07045ecfdae536a83fe99918485d16655ee79018151eb0d170739441a3fa6} Block{7602176/131072/2867836455/a0f185543cd37f069a5cc9dfe46ed55f3823428416a7ca511001f17d779c89a3} Block{7733248/131072/635614932/08bf88fcd5696c1f0f6f412a78755fb3e532cd6f5458fec5ab4537b3c0b64074} Block{7864320/131072/1574060121/6faaaccb6b6da3bb7aa7c4084a34542f6eaecf74fb5cf74572b4514cfa7fc9a0} Block{7995392/131072/3626648309/4545919ef91df09229c65de855c71863ee2b902f2e325d51ff1c9013470659cd} Block{8126464/131072/1472003033/8c3746f038560fa41efce27aeae7ba1c36036fb68c521782e2bb78bc0c975c7d} Block{8257536/131072/1504635999/1e5c860d14394f7530d9795b342b85b70ce43a9ccd91f2ada52e91f1df2908bc} Block{8388608/131072/533375159/0c4a86fcbaab2c595a4bf3ed1afc640ffc761e1ea9e133780281ea11cf5b80c2} Block{8519680/131072/3347519004/403e08b96b585454376e4469544f27d3456fd9e5f398940da6702f95bb9e8c2a} Block{8650752/131072/2997135881/70229873e24164de290c80a4338df840b21955e60b41adec568f3245c1f3ec61} Block{8781824/131072/1748792787/c45e45a248aab45c93bbb29dbae772388d9f5ac18ae753fc7314c120183952fc} Block{8912896/131072/3956334757/71ec995cb2178eeaaf46837f668381105e887d83ce30aae1abc442113e2e92cc} Block{9043968/131072/236001229/01c1bf3154ccc5f9ffe4f4b76c744e8c400774a75c146ba12314b8d742fe3cbd} Block{9175040/131072/2368292183/0036be6e2139062eea22d3c5411862ded1e85aad5bb99221bfede5e1a935e5ab} Block{9306112/131072/3722719221/6e657a12494b0c6b90c32ead3272a6e538e601ba2402d6feb5c686cb27045581} Block{9437184/131072/1063381054/971a84ef437edeba6224682543a9b37866c710dba7a74cecb0903ac268b9ae33} Block{9568256/131072/580901269/05ae059020e7b0585c2db4426ed990333de640cbd7d70f4afd87afc196840a04} Block{9699328/131072/1864060902/0942db87344d080184c59185a54f7ecc301f320a92b0ea085ef8a8317633276f} Block{9830400/131072/45137027/01cd73297a2db04f8c73b8c9c7461bddb19e2ad259057fa18aa6d15d984da64d} Block{9961472/131072/3089977986/f78921031023d3142f41a0dfef2340a890aec04fde9a785e8ab0e344d030c10b} Block{10092544/131072/869740514/2c45149780f7a7633b58edf9eff0d5423b7fd8d85d0b148d8f464bcbb9ed478a} Block{10223616/131072/3044230131/0eee8b31934ddc29b25bbdd41405e2c3019eed1d5a3e85d1cb8858150537189d} Block{10354688/131072/3490441550/071ee25cc53c3be5fe67488fb23445a50342bb4fa198f937ef97aaf0242a2a54} Block{10485760/131072/762785220/b6de3436bf89fd1df632e55522489b6779efa5366ea7410435f3bd230080cada} Block{10616832/131072/1971940660/1223b3a0a24293939bed0d3fab6ed9692dd5f65c6bd00bf2a772f72b5dcb2d18} Block{10747904/131072/1115357622/94431ee6aeafdf295763f2734f438d34028203ffbb47ef867b26c6d54c135b1f} Block{10878976/131072/1579633066/95bc0d2e8ed6d6130c22480661a75d804a4ebb87fb25d0017270d9ff1d92dc68} Block{11010048/131072/755776366/531ecaf25b3278fbb2bfd1f9816f86084eb57e3744ea8b2d05a68edceacdfb2e} Block{11141120/131072/1138395592/67275159afcb7766f28a58ef5f47e7cf335ff75e266b4f3c2f3f12dd257b3fb1} Block{11272192/131072/2209426993/5c644b42be5fbbe2f8a64866ab0de83cce1bd8bc74f660b12510283d3badc7b9} Block{11403264/131072/3708593717/9762f0ded90ad45fb82c47cb1ccc18a2d91bcc7f3f706873974ed24c6e56bd4c} Block{11534336/131072/3677161337/81787842102c83ab8eecb93d5255815887fea3ba2a91b105a7df9182d0f1f5a4} Block{11665408/131072/68726615/0bf4629e97e718d646931b1e10f4fe9b371531d4949c864e167b65ce6ac729ad} Block{11796480/131072/305372172/3e1b48bf1b3a3981095999c6dae1c47ef7bf5b5ac2e5155c514ca5a5970b65a2} Block{11927552/131072/513188141/30f3394902b0466899ef0f7dbb76bd1118e9fb2c4cb746c192136d1c53081375} Block{12058624/131072/923046231/41ac14287834af4335144411022440fe476e3b219d967f5848234981c539ddd5} Block{12189696/131072/992172003/b14cc0ccbb41b2bccb7ae6e0cfa845f47a5f648554d991c0723ef4cf704315c3} Block{12320768/131072/2306033136/3b038ddc89555d4c8a9440a6b17ef0f249284c636f239af5c09ead57580fe1ed} Block{12451840/131072/352299164/51458c252bffd2c87b556303d04db945dc52b1074587b58611ca85fe6f73e60f} Block{12582912/131072/3606533247/9745a6a999978ee2cdeab08ea398f8a0ac4aca323d06e96a7c47bb9ea376f4d8} Block{12713984/131072/3874703508/78e75ea6b93a4c64c94e6886c27b3354514ee3752af262d7d4ec4efc47d641b9} Block{12845056/131072/4233496100/a648bd0af29234b0aba4800bb0f8c16e5f4eb587b2f975bac89cff127a25d50e} Block{12976128/131072/1971536543/886a5b6cc70d07f714ec8b50b2ae060dcdc0230dd2a1cc66220744a2770d6814} Block{13107200/23981/2265688633/304d56d848718dc5a898212755b08c53adc091437aad249f57f1c64d101df7fd}]} 1558119940 -rwxr-x---
[XHTJH] DEBUG: to hash: C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ File{Name:"C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ", Sequence:0, Permissions:0750, ModTime:2019-05-17 19:05:40 +0000 UTC, Version:{[{NPAZACX 1} {WKOOOGB 1} {WR6WMT7 1} {XHTJHWI 1} {2VVXIMR 1} {3ABJ3OJ 2}]}, Length:13131181, Deleted:false, Invalid:false, LocalFlags:0x8, NoPermissions:true, BlockSize:131072, Blocks:[]}
[XHTJH] DEBUG: real to hash: C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ
[XHTJH] DEBUG: mtimefs.go:129 basic /storage/B977-5AE1/Android/data/com.github.catfriend1.syncthingandroid.debug/st/crypted-shared Open C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ {0x40007cd568 C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ} <nil>
[XHTJH] DEBUG: load: loaded 2019-05-17 19:05:41 +0000 UTC / 2019-05-17 19:05:41.214081426 +0000 UTC for C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ
[XHTJH] DEBUG: load: loaded 2019-05-17 19:05:41 +0000 UTC / 2019-05-17 19:05:41.214081426 +0000 UTC for C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ
[XHTJH] DEBUG: mtimefs.go:88 basic /storage/B977-5AE1/Android/data/com.github.catfriend1.syncthingandroid.debug/st/crypted-shared Lstat C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ {0x4000cda600} <nil>
[XHTJH] DEBUG: load: loaded 2019-05-17 19:05:41 +0000 UTC / 2019-05-17 19:05:41.214081426 +0000 UTC for C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ
[XHTJH] DEBUG: receiveonly/cpdpj-ytgwa@0x400010ab00 taking shortcut on C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ
[XHTJH] DEBUG: mtimefs.go:56 basic /storage/B977-5AE1/Android/data/com.github.catfriend1.syncthingandroid.debug/st/crypted-shared Chtimes C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ 2019-05-17 19:05:41.214081426 +0000 UTC 2019-05-17 19:05:41.214081426 +0000 UTC <nil>
[XHTJH] DEBUG: mtimefs.go:60 basic /storage/B977-5AE1/Android/data/com.github.catfriend1.syncthingandroid.debug/st/crypted-shared Lstat C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ {0x4000f58540} <nil>
[XHTJH] DEBUG: save: saving 2019-05-17 19:05:41 +0000 UTC / 2019-05-17 19:05:41.214081426 +0000 UTC for C2j8NNXQ3wHPht9gP7SrHQ/3vOpfRpZp4BwAe8p-ZlJSg/3HuQOxuQc8-RIlDo40s1mA/0IbYz4jP0WVtZaYV4XaNyQ

st_paste_20jun19.txt (13.3 KB)

1 Like
Android Device Always Has Local Changes
41

Are you 110% positive this includes the commit with the fix?

I can still see the same issue.

42

Hello,

Your fix was here, right(?):

commit b7c70a981734803f5dbf45acd6bd2a643c1d75cf
Author: Audrius Butkevicius <audrius.butkevicius@gmail.com>
Date:   Tue Jun 11 07:27:13 2019 +0100

    lib/fs: Enhance mtimefs, use everywhere (fixes #5777) (#5776)

From the log I grepped:

[XHTJH] INFO: syncthing v1.2.0-preview.2-1-g93e428a8 "Fermium Flea" (go1.12.6 android-arm64) rah@xxx.xxx.xxx.xxx 2019-06-19 17:06:08 UTC

Note the: -g93e428a8

From my android build tree:

syncthing-android-rah/syncthing/src/github.com/syncthing/syncthing

git log b7c70a981734803f5dbf45acd6bd2a643c1d75c^^..HEAD |grep ^commit

commit 93e428a8d424f8253c6c0b33a77d3820bc0b7926
commit bf744ded311cdeac6234bce95a8a4871baaefcc0
commit 0d8616689069a689fbc8b6eb0726d6dde85627f0
commit cea59624170c42ac49f0341f5722759d5cfb1e62
commit 02752af86233849746c638f3e8cbbd78cce46878
commit 6b1d7ac727c523d11096e99ed3faaadd8ad095d1
commit abd363e8bb8d2abe34601f3657a9badbbc0b8b27
commit bff1a5f5e446924b875e23232f9d4d178b253c09
commit 38302270d40aefb2394fd7bd6ad31a30d69b7006
commit 1b4fe39a89cf238c66eb1b89650ea1d09ef52aab
commit 6b74cdc6138a0e07f1954eb03ad479438082a8b1
commit 13a746e0fb2e58736a054e5712ff65d8b99f1213
commit 21f50e2f8fb57f30ffc313a115350f0ef9abdd06
commit b7c70a981734803f5dbf45acd6bd2a643c1d75cf
commit 42ce6be9b90981917af85274315a8da07924429c

The commit 93e428a8d424f8253c6c0b33a77d3820bc0b7926 is just me adding Jakob’s patch back in:

git log -p -1

commit 93e428a8d424f8253c6c0b33a77d3820bc0b7926
Author: Richard Hoyle <rah@xxx.xxx.xxx.xx>
Date:   Wed Jun 19 18:06:08 2019 +0100

    Add Debug Statements to mtimefs.go
    
    Add Jakob Borg's (calmh's) debug statements
    as per:
    
    https://forum.syncthing.net/t/ignore-permissions-different-mtimes-on-android-sdcard/13342/17
    
    modified:   lib/fs/mtimefs.go

diff --git a/lib/fs/mtimefs.go b/lib/fs/mtimefs.go
index 199fb8ac..c086f9e4 100644
--- a/lib/fs/mtimefs.go
+++ b/lib/fs/mtimefs.go
@@ -59,6 +59,7 @@ func (f *MtimeFS) Chtimes(name string, atime, mtime time.Time) error {
  // because it might be "does not exist" or similar.
  info, err := f.Filesystem.Lstat(name)
  if err != nil {
+  l.Debugln("Chtimes: Lstat:", err)
   return err
  }
 
@@ -86,6 +87,7 @@ func (f *MtimeFS) Stat(name string) (FileInfo, error) {
 func (f *MtimeFS) Lstat(name string) (FileInfo, error) {
  info, err := f.Filesystem.Lstat(name)
  if err != nil {
+  l.Debugln("Lstat:", err)
   return nil, err
  }
 
@@ -150,6 +152,7 @@ func (f *MtimeFS) save(name string, real, virtual time.Time) {
  if real.Equal(virtual) {
   // If the virtual time and the real on disk time are equal we don't
   // need to store anything.
+  l.Debugf("save: clearing saved for %s, time is %v", name, real)
   f.db.Delete(name)
   return
  }
@@ -159,6 +162,7 @@ func (f *MtimeFS) save(name string, real, virtual time.Time) {
   virtual: virtual,
  }
  bs, _ := mtime.Marshal() // Can't fail
+ l.Debugf("save: saving %v / %v for %s", real, virtual, name)
  f.db.PutBytes(name, bs)
 }
 
@@ -169,14 +173,16 @@ func (f *MtimeFS) load(name string) (real, virtual time.Time) {
 
  data, exists := f.db.Bytes(name)
  if !exists {
+  l.Debugln("load: nothing for", name)
   return
  }
 
  var mtime dbMtime
  if err := mtime.Unmarshal(data); err != nil {
+  l.Debugln("load: unmarshal error for", name)
   return
  }
-
+ l.Debugf("load: loaded %v / %v for %s", mtime.real, mtime.virtual, name)
  return mtime.real, mtime.virtual
 }

Since my commit, 93e428a8d, is included as evinced from the git log, with the output statements and the git describe string as per catfriend1’s code, I guess I’d go as far as 95% certain! I’d definitely bet you 50 GBP that it’s there given (and that’s where my doubt lies, now) that your fix was:

b7c70a981734803f5dbf45acd6bd2a643c1d75cf

Hope that’s the commit! :slight_smile:

===R

43

Right. I need to look into this more and write a better test case I guess.

45

@catfriend1 As you know, my Android build foo is not exactly stellar. The whole clean build thing was clearly an issue so, initially, I zapped the whole tree, and started afresh.

I’ve now got down to naively doing this (which almost certainly wastes time and has redundant steps):

#Zap the lot
rm -rf app/src/main/jniLibs
find syncthing/pkg -type d -exec chmod u+w {} \;
rm -rf syncthing/pkg

#Try to build the go stuff a la catfriend1
cd syncthing
python build-syncthing.py
cd ..

#Clean again
rm -rf app/src/main/jniLibs
#Clean as per gradle
./gradlew cleanNative

#Build
./gradlew buildNative
./gradlew lint assembleDebug

#How did I do
find . -iname \*libsyncthing.so\* -ls
find . -iname \*.apk -ls

I’m reasonably sure that I have good grounds for believing AB’s code is incorporated into the build, especially because the tree I started with was brand spanking new two days ago, and the file I patched already had AB’s changes in it.

I’m very grateful for the time and effort you’ve spent helping me understand the ins and outs of the Android/Gradle build process and your helpful explanations regarding signing debug apps (via PMs).

===R

1 Like
46

Can you try this branch and report back with the logs? There is a suspect as to what can cause this, but the logs will be a lot more verbose this time.