Android frozen syncs

That’s an error message bug (the prefix “mkdir:” and printing it all when it doesn’t affect anything), fixed now, thanks.

Nevertheless, ignoring permissions only means we ignore them as a factor in the sync. We still do chmods when we create stuff and to make sure directories are writeable before we create files in them, etc. This is as it should be; even FAT filesystems, where we might want to “ignore permissions”, still take their read only bits seriously and we need to override that if we are to write to files.

Syncing files, especially to crappy filesystems, turns out to be a convoluted business. :wink:

@calmh

I am still not convinced that the failing syncs are truly the result of failied time stamps attempts. First of all ST syncs some files so that tells me that it is somewhat doing what it needs to do. Secondly it is having issues with certain files, because when I look at the logs I only see time stamp issues with certain files. And the last issue is that the files that it is having chtime issues have not even changed in both directions.

it feels like ST on Android always is confused about somefiles and that state of confusion causes further issues like frozen syncs, failing to update repos etc.

So far I have yet to sync my repo fully between a desktop and a laptop after 3 weeks of trying to get it synced. Please bear in mind that I deleted recreated the repo on android, I installed the android version countless times and I probably wasted over 15 hours on this easily (testing, bug repors, logging etc) which is fine to me. I just wish that at the end of 3 weeks I could have successfully synced my repos.

For example I have a single txt file in that repo that I change it purposefully to see what happens. It sometimes updates sometimes not for days.

I am wondering if this is an actual issue with the Android binary, maybe some threading issues, maybe ram usage. I cant tell but what I can tell is that ST for Android is not reliable whatsoever. It could be reliable probably if the repo is made of couple files. But in my case I tested with mix repos with txt, zips, pdfs, mp3s etc (around 3gb). I still have issues with it.

The thing is that the desktop versions of ST has no issues between other desktop clients. So I am just thinking that this is more of a ST binary/libraries on Android having issues. I hope that some experienced Android dev could look into binary compilation process.

thanks

It would be nice if someone tried to run at least the integration tests on Android. From my point of view, it’s an untested platform, other than by users and going on the reports here it’s not working awesomely.

Let’s not forget the 800 pound gorilla in the room … that is, the inherent limitations of most Android phones.

Unless you are rooted AND using pre-Kitkat Android, you are pretty much limited to using the device’s internal memory for your repos. Forget about your new 64Gb external SD card, Syncthing will not have read and write access to it.

Given these limitations, I decided that my phone would be part of a cluster sharing a few light files that I need while “on the go” - and separate from other heavy duty data clusters that link our servers and PC’s. The Syncthing node on my phone is limited to a few small Truecrypt containers with critical current work files, a few photos, and my keepass database. The whole thing is approx. 300 Mb.

I don’t expect more - given the storage limitations of internal memory.

Is the Android version of Syncthing buggy? Yes, although most of my issues surround visual bugs and a disconnect between the native android UI and the Syncthing UI.

Other than that, it works reliably - as long as I don’t push it to sync large files.

@NickPyz

It is not just the large files and repos. Android ST does not even properly delete folders and files deleted from the repo. It does not update even deleted empty folders. Again the desktop version is almost flawless, although I have not done super stress tests with them.

You say it is reliable as long as you keep it small. But what is small? Where do you draw the line for it? As long as there are no clear definitions and limitations on the Android version I say it is unreliable because the ST behavior on Android is not well defined for cases.

Another issue that hits me hard is that the ST binary sometimes never dies off even after quiting the application. In fact sometimes I kill the process from app manager and it will still stay persistent in the memory , using alot of cpu. I sw that multiple people reported such beavior even after crashes. How is that even possible? That by itself tells me that the binary is not stable and well integrated.

@kahun

I understand you wanted a quantifiable answer, and mine was general. Fair enough. @calmh suggested something called integration testing - which is beyond my knowledge. And you suggested code / compilation review by an Android expert. I agree - why not? Hopefully, we will find one as the user base expands.

@Nutomic has invested a lot of time and effort to take the Andoid version to where it is - I am hoping he will jump in with his thoughts.

Syncthing is intended to connect devices. The problem is the variety of devices, and the OS’es that run them are numerous. How does this project maximize device reach, AND ensure reliability and quality on every device type - given that we have calmh and a very small group of developers? Some development is transparent on Github, other pieces such as some Linux distro and NAS packages are developed and hosted elsewhere.

I don’t know the answer to ensure that all the pieces integrate perfectly.

@NickPyz

Nutomic is doing a great job. We all appreciate anyone who puts time into any project that will benefit people. However in this case I feel like the situation is a bit convoluted because we cant even define the actual areas of the problems. Is it the platform? Is it the libraries? Is it the binary? Is it the wrapper app? Is it the file system? There are many questions need to be handled.

From now on I am only going to use the cli version, I feel like that is going to be only way to see the real behavior. I just want to be able to provide healthier feedback to developers.

Ok I have been testing with the Cli version and some of the problems just went away

Here is my latest comment

I definitely recommend the cli version if you want reliability until @Nutomic can figure out the real issues with the Gui version.

So far the Android binary works very well, I redid all the test that I tried to do with the gui version and this one passed all without any hiccups.

@kahun My Android phone is company issued, and I promised to not root it. CLI access is out of the picture for me.

Your initial comments are promising. I might install a 'droid emulator on my PC and test this too, time permitting. Keep us all posted.

@NickPyz

Sorry to hear. Maybe someone can wrap it up quickly for user space execution without bells and whistles. I wonder if one can install it right on the sd card instead of “data”. I think that is the main reason for root requirement.

As I mentioned, it is working almost flawlessly, instant updates, no missing syncs, no frozen syncs, no crazy cpu use and no zombie process.

Btw root user is going to create new keys and new id.

Just want to add here too that while it’s great that it’s working better for @kahun, I would be surprised if there was something about the Android GUI+packaging that screwed up the syncthing “core” to the point that it causes these issues. It could perhaps be as simple as you running a slightly newer syncthing now, or the configuration being a little bit different?

@calmh

I doubt so. All the problems went away. I cant seem to explain why beside the differences between the gui and cli version. Maybe the gui version is having hard time with user space permissions and other processes. Someone with Android dev experiences can answer probably, but that is all I can provide

Are you running the CLI version as root? I just started testing a little bit with Android, and I can’t even get syncthing to start as a CLI program when not running as root. Presumably the relevant permissions aren’t given unless the program is started in the correct way with some manifest etc.

I get the feeling that when running as root we’re bypassing a lot of the special Android stuff and it behaves more like a straight Linux dist.

@calmh

Yes I am running it as root. You can gain root by typing su. But I am on a rooted device. I am not sure how that works out on an emulator.

I have not had a single glitch with syncing since I started using cli version between 2 android and one desktop. Before that it was all hell for me (and I was not the only one who had issues ) . I think that like you mention, some setup is not done correctly with the gui version.

I can safely say that (so far), the Android binaries are good. I had cases of frozen communication between devices, like thye stopped seeing eachother as if they lost contact. I had to restart my devices. That happened only once. And 1 or 2 times I had to restart the binaries manually. Still these did not seemed sync related issues rather network stack issues. However I cant pinpoint the problem atm.

FYI, I’ve almost got a little kivy-powered syncthing wrapper (no root required!) working so we can test this theory some more. It just starts the binary…that’s it. Everything else is done from the web browser UI. The learning curve for Android dev…even with Kivy…is steep at best :confused:

Scott

I just installed whatever Android it is you get from the android-x86.org site, and “SSHDroid” that gives you a root prompt via SSH. Via Nutomics app, syncthing seems to work, and run as root it seems to work, but if I su - 10072 (the uid that the syncthing-android app got allocated) and then try to run the binary, it’s unable to open a listening port. So I’m assuming that’s because the process doesn’t get the capability to open ports unless started using the standard framework.

Anyway, my point was that I could see things working better when run as root because we apparently take a bunch of the non-standard (from a Linux POV) security features out of the game that way.

Rough wrapper for syncthing with logging enabled so you can view the output with logcat. See the README in the repo for more info. The APK is in bin/.

https://github.com/firecat53/syncthing_android_wrapper/

Let me know if it’s useful (or even works, for that matter)

Scott

@firecat4153

Thanks for the progress. Do you have some screenshots of what we are going to see?

I added one to the git repo: https://github.com/firecat53/syncthing_android_wrapper/blob/master/screenshot.png

Not very interesting :slight_smile:

I also remembered I need to add a switch to turn on the STTRACE=all for debugging -facepalm-

Scott

cool I will try.

Did you have time to try on rather complex repositories?