I’ve read through #29 and most of the links in there. From what I gather, syncthing shouldn’t work with an SD card. But I started up an Android R AVD emulator with an SD card, and syncthing seems to be writing to the SD card ok. I set it up as a separate storage device, not combined with internal memory. I can see the files from both the Amaze File Manager and Files by Google apps.
Am I missing something?
As I said, emulators are meant for development, so they might not have the same sandboxes as real phones, as it would be hard to do development on them.
@anderspitman did you target Android R when compiling?
As of the current plan by google, it’s a yes, see Android 11 "All files access" for the Syncthing App Catfriend1 pointed out that google often changes even fundamental stuff in the current phase, so I guess we know for sure in June when the first beta and with it “final API” is out (https://developer.android.com/preview/overview#timeline). It’s still not going to be super simple to make the app work on both Android 11 and <=10, so don’t expect any immediate changes.
I didn’t compile for my tests; just installed ST from the Play store. So I’m pretty certain it’s targeting a lower SDK.
FWIW, I also used Amaze File Explorer to test SD writability on APIs 27,28, and 29. None of them allow writing. I’m pretty sure Amaze does not use SAF. It pops up a UI to attempt to instruct the user on granting permissions, but the instructions seem out of date and didn’t work for me.
Es Explorer did automatically prompt for allowing access, and was writable, so I’m guessing it supports SAF.
@imsodin how do developers feel generally about “All files access”. It initially looked like a decent solution to me, but I’m pretty concerned about the play store whitelisting, and requirement for the user to be navigated to system settings (will that work across vendors?). I’m not an Android developer. Do they typical enforce things like that just by keeping it off the Play store, or can they refuse to sign it alltogether preventing side-loading?
It does, and that means you are most likely seeing a emulator deficiency, not a real change as suggested by Audrius above.
Not an Android dev either - I just do “release management” of syncthing-android. From how I understand it there will be an intent (or whatever that thing is called) that the app can send and then the user is sent to the right location in the system settings. Most likely vendors will be able to break that, but that’s nothing new. As to playstore whitelisting: Afaik they can only restrict what goes into the playstore, but given the tight integration of that into android and inability of most users to use alternative ways to install apps, that’s pretty authoritative already (personally I wouldn’t mind a shift to F-Droid and the like, but I am too realistic to hope for that).
Man, things are still looking pretty dire. They’re explicitly trying to prevent the use of SAF for broad filesystem access, and sounds like it won’t be easy at all to get approval for All files access: https://youtu.be/UnJ3amzJM94?t=1193
The linked video starts out with SAF, which is explicitly not what the new all file access/manage external storage (did they rename it?) permission is about: Accessing files natively, not through SAF: https://developer.android.com/preview/privacy/storage#all-files-access
I am not going to watch a more than half-year old video for it to maybe give some tangible information, if there is any, please enlighten me about it (and ideally link a text source) - thanks.
The video link should start at 19:50, and you only need to watch for about 3 minutes to see everything I’m talking about.
What gives you the impression that the intent of “All access files” is for native apps? It explicitly says in the section you linked that the intent is to provide exceptions for apps that do things like backup or file management.
EDIT: They address file path access for native apps earlier in the video: https://youtu.be/UnJ3amzJM94?t=876. It’s rather ambiguous, but seems like they’re saying it may only work for files provided through the MediaStore API, with all the limitations that go along with it, including non-media files must be put in the Downloads directory, and accessing non-media files created by other apps requires SAF. In any case they don’t mention “All files access” at all in that context.
[quote=“anderspitman, post:10, topic:14983”] What gives you the impression that the intent of “All access files” is for native apps? […]
Native probably isn’t a good choice of words: What I mean is code/libraries using direct filesystem interaction, not the Android apis. From said link (“raw file paths” being the most relevant bit):
Apps can access these files using either the
MediaStoreAPI or raw file paths. If your app uses the Storage Access Framework, you cannot use it to access the additional files and directories that the all files access permission makes available.
Well, file management is what Syncthing does, right?
That was interesting indeed, especially I didn’t yet get that file path access will be translated to android apis internally - that’s likely going to be fun 
Actually I believe we were using the term “native” in similar ways, so I think we’re on the same page there.
I upgraded my Asus Zenfone 6 to Android 11 and Syncthing is able to read and write to folders on the SD card.
I now have other problems. From time to time, Syncthing “hangs” when scanning/starting which then ends up with Android asking me if I want to force close the app. Two “send only” folders that I share with my phone now get out of sync every time Syncthing on the phone performs a scan. In the past 2 days, I had 2 folders renamed (added .0 to the folder name) on other send/receive ST folders.
Since I don’t have enough storage to have these folders on the internal storage, I can’t confirm if the issues are caused by me using an SD card or if it’s something else.
You are probably using the syncthing-fork app from F-Droid, right? In that case it is expected to work. Android 11 added this new, reasonable permission to “Allow management of all files”. That doesn’t mean really all files, but it finally does include the SD card. However that option is disabled for apps on google play for now, thus only available through F-Droid and when targeting Android 11 - syncthing-android targets Android 10 to escape the wide-ranging restrictions without this permission.
Yeah that maybe was more confusing than clearing things up - android permissions have that effect.
I’m using the official(?) app, version 1.13.0: https://play.google.com/store/apps/details?id=com.nutomic.syncthingandroid
(I’ve used Syncthing-Fork before but moved back to Syncthing for stability.)
Since I’m rooted, I uninstalled Magisk and reinstalled Syncthing (without restoring any backup) just to be sure. I can create a folder on the SD card, share it with my laptop, and add/update/delete files on both ends. No issues syncing the changes.
I don’t know if ASUS did something to allow this or if it’s something else, but it’s working. On Android 10, I had run Syncthing as root so it could write to the SD card.
And yes, I’m on Android 11:
One option is always that the vendor (Asus) did something non-standard. If that’s not the case, then the reason might be that targeting android 10 and enabling legacy storage is equivalent to enabling all storage access on android 11. Thus everyone on android 11 should have access to the SD card currently. That would be nice. Anyone that also has android 11, preferably a different device, please test whether it works for yo too.
Okay so I did some tests with my device.
Factual:
When adding a new folder, the system file browser appears. On the file browser I can select a folder on the SD Card. As soon as I do this, the following popup appears:
(Translated: “Allow syncthing to access files in Test?” “Syncthing can then access current and future content stored in Test.”)
The Test-folder is physically located on the SD Card.
If I click Allow/Zulassen, Syncthing gets read and write access to this folder on the SD Card. It sounds like syncthing does not have full access to the entire SD card, but I can choose folders on the SD Card where Syncthing then has access too (subfolders are included too, I just verified this). Sounds like a really nice solution for me.
The “All files access permission” seems to not exist, which is probably expected since the app doesn’t target Android 11. The special permissions dialog for all files access doesn’t list syncthing:
This is the system file dialog for “Special Permissions → Allow management of all files”. Syncthing isn’t listed as an option and thus cannot be turned on or off.
But it works anyway, because the system files dialog asks whether to grant access on a per-folder basis. It feels like Samsung has solved this really nicely, I hope it remains this way (full access would be fine too). I have no idea if this a new thing with Android 11, as I haven’t done any testing while still on Android 10. I knew that it didn’t work when still on Android 9.
Edit: I just found that there is an “all files access” permission, located unter the normal app settings. Syncthing has permissions here:
“Allow management of all files” is set to allow, which was apparently done automatically upon migrating to Android 11. At least I don’t remember changing anything here.
TL;DR: Syncthing & SD Card “works for me”.
Thanks for testing. It’s not just my Asus then.
Are your scans slower than before? And if you have any, does your “receive only” folders go out of sync because of “changes” Syncthing keeps detecting?
It seems that the changes Google made to Android 11/scoped storage made read/writes way slower… https://www.reddit.com/r/androiddev/comments/kpn68k/android_11_very_slow_file_access_performance/
Good info following that link, thanks: Apparently all filesystem access to “external storage” (as in /storage, not SD card) now goes through the SAF, i.e. including native access. That explains why it works on Android. And also why it’s slow.
I usually don’t have syncthing running all the time on the phone, I only start it when I need it (once every few weeks), let it sync, then stop it again. As I said, internal storage is large enough for me (the phone has 128 Gigs and I’m basically just syncing a password manager + some important work related files + a handful of photos), so I do not have any folders on the SD card - I only added one for testing purposes.
Scanning does take a few seconds, probably more than before, but still under 10 seconds for initial scans, < 3 seconds for subsequent scans, largest folder is 4.7 Gigs. As for the local changes problem, I do not have this issue, but I didn’t do any extensive testing - my normal setup doesn’t include a receive-only folder, I’ve everything on send-receive, but the phone has never touched any file as far as I can tell. A very quick test showed no anomalies regarding receive-only.
EDIT: I just flipped some folders on the phone between send-receive and receive-only and one of the folders indeed showed some locally changed items after swicthing to receive-only, but they disappeared automatically after a few seconds.
