Filesystem Watcher Errors - Android

Hi, I use Syncthing to sync certain folders from my devices to my home server, a kind of backup. Unfortunately, the synchronization of the screenshot folder from my android phone doesn’t work and I get the following error message:

The full path should be /storage/emulated/0/Pictures/Screenshots

I also tried lstat manually via adb:

redfin:/ $  lstat /storage/emulated
/system/bin/sh: lstat: inaccessible or not found

But listing the screenshot-directory on the other hand works:

redfin:/ $ ls -la /storage/emulated/0/Pictures/Screenshots/                                           
total 7655
...

The synchronization of the camera folder /storage/emulated/0/DCIM/Camera however is working perfectly fine.

I can’t explain this error and I also could not find anything helpful on the web. Any ideas?

I had some similar issues in the past. Could you try to change the whole path from

/storage/emulated/0/DCIM/Camera

to

/sdcard/DCIM/Camera

and then try again? In order to change the path, you can remove and re-add the folder (or just use the Advanced Configuration if you know what you are doing).

That’s interesting, I didn’t know about this directory before. But my camera-folder is synchronizing fine. I’ve got only problems with the Screenshot directory.

Edit: I just realized that the error is still there (even with pointing the screenshot sync to /sdcard/DCIM/Camera) Probably it’s a problem with the Application. I’ll reinstall it and tell you the outcome!

Reinstalling the program didn’t help, the error message still occurs. But it’s also pretty weird that screenshots witch I take are now synced to the server but not the other way around though on both, server and phone the “Folder Type” is set to “Send & Receive”.

Edit: The phone is a Pixel 5 with an Android AOSP called GrapheneOS. It’s very security hardened and privacy focused which is why I first thought it might have something to do with it.

But why does this problem occure only with /storage/emulated/0/Pictures/Screenshots and not with all the other Syncs under /storage/emulated/

Edit 2:

I just noticed, that this means lstat isn’t existing on my phone in general. Is this a bug of the syncthing app?

1 Like

I don’t think so. I also get the not found error when trying to call lstat like that, but the file watcher works fine regardless.

This may be some kind of a permission issue. The only thing that I can think about right now is that the /sdcard/Camera/DCIM folder seems to exist in Android by default from the very first boot, while the /sdcard/Pictures/Screenshots folder gets created only when you actually take the first screenshot.

Some screenshots of the Syncthing Web GUI from both sides would probably help. You should also check the log files, especially on Android, to see if there are any errors recorded there. See https://forum.syncthing.net/t/only-non-pdf-files-are-getting-deleted/16408/16 for how to get the logs on Android.

Thank you for the help tomasz86!

I get it working without errors by deleting the F-Droid Syncthing package and installing it from the Play Store. To be honest I don’t want to play around with it too much until I mess things up again ^^

For those who encounter the same problem:

My Camera pictures are syncing from /storage/emulated/0/DCIM/Camera

And my Screenshots syncing from /storage/emulated/0/Pictures/Screenshot/

Seems, that it was a problem with the F-Droid package …

1 Like

Interesting. Are you using the official app or the Fork? As far as I know, there is no difference when it comes to the official one, regardless of the source.

I was using the official app Syncthing | F-Droid - Free and Open Source Android App Repository

I thought so too, which is why I was surprised. Well I’ll keep an eye on it in the future and see if it occurs again.

1 Like

Hi,

I’m currently working on Syncthing-Fork v1.14.x and after compiling SyncthingNative v1.14.0 and putting it into the wrapper, starting the whole thing on my existing Android 11 emulator instance, the same error immediately popped up.

image

If I run the v1.13.1.1 (containing SyncthingNative v1.13.1 compiled from source), the error is not coming up. I suspect some code change causes this to happen, but maybe it was just an android restriction SyncthingNative did not “see” before and now it does some more extensive checks of the environment / paths?

Files can be scanned by SyncthingNative. I’ve put a test picture in the camera folder and had to manually hit “Rescan”.

1 Like

Now compiling the wrapper APK including SyncthingNative v1.15.0-rc.3 for testing purposes and the filewatcher errors are still there.

image

Interesting, I don’t have the filewatcher error on Android 10. I guess it’s the Android 11 restrictions to blame here, as they limit listing parent paths like /storage/ and /storage/emulated/. The first list-able path is /storage/emulated/[uid]/

image

image

Did you recompile now with v1.13.1 to check it isn’t anything else that changed in between?

There’s no changes relating to the filesystem watcher in between v1.13.1 and v1.14.0.

Seems likely to be due to Android 11: In that discussion about the external SD being accessible, info was posted that suggest that now all “native filesystem calls” are routed through androids storage access framework. And that hardly implements inotify. If that’s, I guess the only option is to disable filesystem watching. As far as I recall the app had it’s own watching using android apis and then syncthing’s rest api before filesystem notifications were introduced in Syncthing itself - maybe that can be resurrected.

1 Like

@imsodin : I’ve now tried fresh setups of all Syncthing(Native) versions beginning from v1.8.x until nowadays on Android 11 emulator and all show the filewatcher error. So it definitely IS Android 11 related. (Did not notice it before, because my physical device is Android 10, sorry for the noise.)

This would be a good idea for Android 11+ users.

Do we have alternatives? Would it be possible to do a “test Syncthing code” which avoid accessing /storage and /storage/emulated but directly try to setup watching from /storage/emulated/0 (going deeper on) or is that technically impossible ?

You can probably run with enough tracing to see exactly where the error originates and comment that out. My guess would be that it’s the part where we resolve the correct casing of the folder root.

1 Like

I’m getting the Log:

2021-03-13 14:28:52 casefs.go:241 basic /storage/emulated/0/DCIM Stat . {0x6272bc20} <nil>
2021-03-13 14:28:52 casefs.go:321 basic /storage/emulated/0/DCIM Watch . Matcher/[]@0x62658190 true lstat /storage/emulated: no such file or directory
2021-03-13 14:28:52 Error while trying to start filesystem watcher for folder "Android Camera" (sdk_gphone_x86_arm_f78t-photos), trying again in 1m0s: lstat /storage/emulated: no such file or directory
1 Like

Thanks. I did remove that part:

diff --git a/lib/fs/casefs.go b/lib/fs/casefs.go
index 6a27be845..8d47ddfd7 100644
--- a/lib/fs/casefs.go
+++ b/lib/fs/casefs.go
@@ -315,9 +315,6 @@ func (f *caseFilesystem) Walk(root string, walkFn WalkFunc) error {
 }

 func (f *caseFilesystem) Watch(path string, ignore Matcher, ctx context.Context, ignorePerms bool) (<-chan Event, <-chan error, error) {
-       if err := f.checkCase(path); err != nil {
-               return nil, nil, err
-       }
        return f.Filesystem.Watch(path, ignore, ctx, ignorePerms)
 }

It did not solve the issue. Log:

2021-03-13 15:50:01.828 [MN7RW] DEBUG: sendonly/sdk_gphone_x86_arm_f78t-photos@0x6081c000 scanning
2021-03-13 15:50:01.835 [MN7RW] DEBUG: casefs.go:241 basic /storage/emulated/0/DCIM Stat . {0x608f2120} <nil>
2021-03-13 15:50:01.838 [MN7RW] DEBUG: casefs.go:318 basic /storage/emulated/0/DCIM Watch . Matcher/[]@0x600c2780 true lstat /storage/emulated: no such file or directory
2021-03-13 15:50:01.841 [MN7RW] INFO: Error while trying to start filesystem watcher for folder "Android Camera" (sdk_gphone_x86_arm_f78t-photos), trying again in 1m0s: lstat /storage/emulated: no such file or directory
2021-03-13 15:50:01.842 [MN7RW] DEBUG: casefs.go:241 basic /storage/emulated/0/DCIM Stat .stfolder {0x608f21b0} <nil>

I’ve (blindly) tried to remove that part as well:

and then get:

2021-03-13 15:59:43.110 17911-17979/com.github.catfriend1.syncthingandroid.debug I/SyncthingNativeCode: [MN7RW] DEBUG: folder.go:944 basic /storage/emulated/0/DCIM Watch . Matcher/[]@0x652a4140 true permission denied
2021-03-13 15:59:43.111 17911-17979/com.github.catfriend1.syncthingandroid.debug I/SyncthingNativeCode: [MN7RW] INFO: Error while trying to start filesystem watcher for folder "Android Camera" (sdk_gphone_x86_arm_f78t-photos), trying again in 1m0s: permission denied

Notify (underlying library) also does checks on all path components (resolving symlinks).

I don’t know what to do about android. On the one hand it is used by many (including me), on the other hand it is terrifying: It’s clear now that under certain circumstances android reports that paths are missing, when in fact it’s a permission issue. That will destroy user data (the thing they are so concerned about when it comes to privacy…). Probably one can handle it by having code in the wrapper, that checks that folder roots are accessible. I would hope it behaves sanely if that is the case. @Catfriend1 Do you already do something like that?

1 Like

@imsodin I do not check from the wrapper if folders are there and accessible as I would then have to go through SAF to do that with the (Android deprecated) absolute paths. Or we would have more of the ugly shell command wrapping which I’ve already used in some places where I’d like to have the same point of view as if I were a native executable but from the wrapper.

I’ve looked for the Android side watcher code : syncthing-android/FolderObserver.java at 285797f89b4cf1ae5fd4c1d121b1567d6399f40d · syncthing/syncthing-android · GitHub

It depended on new File() objects. This is what Google also restricted more and more ; it could maybe work together well if we use all files access.

What I dislike in that code is that it was the “battery eater” in the past. Example: back in 2017, I’ve setup a sync folder which points to my internal storage root. I then ignored my messengers database dir as it constantly changed the files and the Syncthing app dir as it contained the logs. With that, my phone got hot and burned the battery through fast because the watch mechanism picked up a messenger db change (as the android code does not respect .stignore), fired the scan via restapi and logged this, then it watched the syncthing log change, fired a scan, and so on. That was btw one main reason to join the Android dev in 2018 because I wanted to get Syncthings internal watcher in which is technically very good.

1 Like