Android 11 "All files access" for the Syncthing App

Hi,

I’m currently playing with Google’s new feature “all files access”. Going through Android settings and searching for “special app access”, then tapping on “all files access” and granting my test build this permission unfortunately does not allow SyncthingNative to write any single file to storage. Screenshots can be found in the references github issue.

I’ve also enabled STTRACE=fs model and was surprised SyncthingNative did not report anything like “failed to write file blabla, path is read-only”. It just shows 0/0 files synced and out of sync + revert. If I revert, the sendOnly side (PC) does show the override changes button. If I hit it on PC, the Android shows the revert button and so on.

I’m opening this topic to help preparing the compatibility with Android 11 enforced Scoped Storage feature and maintain file-write access by something still required to figure out.

If anyone reads this and has useful input or information to help me, please post. We still have a lot of time until Android 11 will hit - and hopefully - Google will make the all “files access” better during next betas.

Ref.:

Screenshots from Android 11 AVD:

image image

1 Like

The folders are send only so Syncthing isn’t going to try to write anything regardless of what you click.

1 Like

Also are you sure this means actual direct file access as opposed to through SAF as ususal? Could you point me at some documentation then please.

Oh my bad, I did my patch of code so fast that it starts up on 11 I didn’t notice my sendreceive and later receiveonly folder setting got overwritten by some other piece of code. The problem here is, our current SAF implementation fails on 11 because getVolumePath is no longer available. I’ll try again later with disabled “sendonly auto decision code” and report back.

Btw documentation is rare but here: https://developer.android.com/preview/privacy/storage#all-files-access

Still not sure about if all files access also includes our native .

1 Like

Wow, thanks a lot for that link. This is a 180deg turn on google side, lets hope it stays that way (I even think this would include external SD storage o.O):

Apps can access these files using either the MediaStore API or raw file paths. If your app uses the Storage Access Framework, you cannot use it to access the additional files and directories that the All Files Access permission makes available.

It looked too promising to be true. Changed the folder type to sendReceive on the Android side but didn’t get a single file synced over again.

image

I’m getting a lot of permission denied for every file and dir now in the SyncthingNative log.

2020-03-16 19:15:36.128 11608-12030/com.github.catfriend1.syncthingandroid.debug I/SyncthingNativeCode: [LDE4A] DEBUG: sendreceive/kjyfi-p6svl@0x661e0000 resurrecting parent directory of Tntwin/Tntwin.ini
2020-03-16 19:15:36.128 11608-12030/com.github.catfriend1.syncthingandroid.debug I/SyncthingNativeCode: [LDE4A] DEBUG: folder_sendrecv.go:673 basic /storage/1AE6-2B0F/synctest/android-11 MkdirAll Tntwin -rwxr-xr-x mkdir /storage/1AE6-2B0F/synctest/android-11/Tntwin: permission denied
2020-03-16 19:15:36.128 11608-12030/com.github.catfriend1.syncthingandroid.debug I/SyncthingNativeCode: [LDE4A] DEBUG: Repeat error on puller (folder "z_synctest_android-11" (kjyfi-p6svl), item "Tntwin/Tntwin.ini"): resurrecting parent dir: mkdir /storage/1AE6-2B0F/synctest/android-11/Tntwin: permission denied

The wrapper also cannot write to the folder.

2020-03-16 19:19:32.334 11608-11608/com.github.catfriend1.syncthingandroid.debug D/Util: runShellCommand: echo "" > "/storage/1AE6-2B0F/synctest/android-11/.stwritetest"
2020-03-16 19:19:32.354 11608-11608/com.github.catfriend1.syncthingandroid.debug I/Util: Failed to write test file '/storage/1AE6-2B0F/synctest/android-11/.stwritetest', Permission denied

Not sure if this is of any help, but I found this information in the article “Small steps and a big leap - First pre-release of Android 11 and new features for pixel phones released” on page 42 of issue 07/2020 of the German computer magazine c’t:

“The Scoped Storage introduced with Android 10 has been improved in terms of security and privacy. The sandbox principle of the internal memory is extended to the shared memory or SD card: apps can no longer access any data there either. The API innovations suggest that for extended access, an app must first request the necessary authorization from the user.”

Translated from German by deepl.com

We know that, that’s the problem making Syncthing on android much less useful. However the above (Android 11 "All files access" for the Syncthing App) sounds like android 11 will bring a new special permission to access files outside of scope storage directly, which would be a life-safer.

To get syncthing android working on my sd card, I had to do the SDCard\Android\media\com.nutomic.syncthing\ (Folder1/folder2/folder3) route.

Yes , that always worked before android 11.

Just to be clear, does this mean that it will no longer work after android 11?

Well when scoped storage was announced we expected it to no longer work on 10. Then Google made a last-minute decision to keep the opt-out of this feature alive in release builds of android 10 which saved us. Now they announced 11 with that feature enforced. It’s up to them if Syncthing will work or not on 11. Things today not working may change in future 11 preview’s.

Maybe not for external SD, but the following article is also very clear that the MANAGE_EXTERNAL_STORAGE permission will allow direct access to files: https://www.androidpolice.com/2020/02/19/scoped-storage-in-android-11-will-have-exemptions-for-older-apis-and-core-apps-like-file-managers/ Did you try enabling scopes storage and then access files on “internal storage”?

1 Like

Not yet, the manifest has still the opt-out by requirelegacystorage in place. Will try it when time :-). I hope if it works there is some compatibility because requiring requirelegacystorage in the manifest for 10 and not for 11 sounds ugly. My aim is to avoid having two apps targeting specific android versions.

1 Like

I am just interested in testing, it’s already nice that the deadline is extended to 21 instead of end of this year.

1 Like

Enabled Scoped Storage in the app manifest and followed Google’s advice to enable it via developer options, too. All files access has also been enabled and I’ve built the app upon R-rc.2 SDK and NDK r21.

Internal sdcard: I can create a new folder with Android UI when a new ST folder is added.

image

External sdcard: I CANNOT create a new folder with Android UI when a new ST folder is added. Icon “+” is greyed out.

image

External SDcard, formatted as portable: I chose the folder root to assign to the new ST folder as I couldn’t create a subfolder through Android UI.

Surprisingly, the app’s write test via shell cmd succeeded.

2020-03-22 14:17:57.084 9502-9502/com.github.catfriend1.syncthingandroid.debug W/FileUtils: getVolumePath failed for volumeId='141C-1116'
2020-03-22 14:17:57.085 9502-9502/com.github.catfriend1.syncthingandroid.debug V/FolderActivity: onActivityResult/CHOOSE_FOLDER_REQUEST: Got directory path '/storage/141C-1116'
2020-03-22 14:17:57.088 9502-9502/com.github.catfriend1.syncthingandroid.debug D/Util: runShellCommand: echo "" > "/storage/141C-1116/.stwritetest"
2020-03-22 14:17:57.150 9502-9502/com.github.catfriend1.syncthingandroid.debug I/Util: Successfully wrote test file '/storage/141C-1116/.stwritetest'
2020-03-22 14:17:57.168 9502-9502/com.github.catfriend1.syncthingandroid.debug D/Util: runShellCommand: rm "/storage/141C-1116/.stwritetest"
2020-03-22 14:17:57.353 9502-9569/com.github.catfriend1.syncthingandroid.debug D/EGL_emulation: eglMakeCurrent: 0xf77a7c30: ver 3 0 (tinfo 0xf781eb50)
2020-03-22 14:17:57.456 9502-9502/com.github.catfriend1.syncthingandroid.debug V/NotificationHandler: Cancelling notification with id 717
2020-03-22 14:17:59.363 9502-9569/com.github.catfriend1.syncthingandroid.debug D/EGL_emulation: eglMakeCurrent: 0xf77a7c30: ver 3 0 (tinfo 0xf781eb50)
2020-03-22 14:17:59.387 9502-9569/com.github.catfriend1.syncthingandroid.debug D/EGL_emulation: eglMakeCurrent: 0xf77a7c30: ver 3 0 (tinfo 0xf781eb50)
2020-03-22 14:17:59.419 9502-9569/com.github.catfriend1.syncthingandroid.debug D/EGL_emulation: eglMakeCurrent: 0xf77a7c30: ver 3 0 (tinfo 0xf781eb50)
2020-03-22 14:17:59.436 9502-9514/com.github.catfriend1.syncthingandroid.debug I/ngandroid.debu: Background concurrent copying GC freed 38835(2499KB) AllocSpace objects, 47(964KB) LOS objects, 49% free, 5022KB/10044KB, paused 1.497ms total 237.060ms
2020-03-22 14:17:59.439 9502-9569/com.github.catfriend1.syncthingandroid.debug D/EGL_emulation: eglMakeCurrent: 0xf77a7c30: ver 3 0 (tinfo 0xf781eb50)
2020-03-22 14:17:59.462 9502-9569/com.github.catfriend1.syncthingandroid.debug D/EGL_emulation: eglMakeCurrent: 0xf77a7c30: ver 3 0 (tinfo 0xf781eb50)
2020-03-22 14:17:59.492 9502-9569/com.github.catfriend1.syncthingandroid.debug D/EGL_emulation: eglMakeCurrent: 0xf77a7c30: ver 3 0 (tinfo 0xf781eb50)
2020-03-22 14:17:59.530 9502-9516/com.github.catfriend1.syncthingandroid.debug D/StrictMode: StrictMode policy violation: android.os.strictmode.LeakedClosableViolation: A resource was acquired at attached stack trace but never released. See java.io.Closeable for information on avoiding resource leaks.

I can choose Send&Receive on the wrapper UI now.

I’m really surprised I can WRITE to an EXTERNAL sdcard now.

image

If it stays like this, we just need to figure out how to make the wrapper right. It’s now upside down world because SyncthingNative can write to SDCARD and the wrapper can NOT.

1 Like

Ok, there must be something wrong with the preview R-rc2 image of Android. When I use Android’s builtin file manager (without the ST app) and try to create a new folder on the external sdcard, it’s also greyed out.

image

Looks good to me! Syncing with WRITE to the internal storage “/storage/emulated/0/Test” also works fine.

image

1 Like

@Catfriend1 Thanks for taking all that testing effort and the results look really promising.

1 Like