I couldn’t get the stalling to occur with F-Droid v1.30.0.2. Repeated these steps twice:
PC: v2.0.0-rc.23, Windows (64-bit Intel/AMD)
Android:
- Uninstall existing SyncThing
- Install v1.30.0.2 from F-Droid
- Enable Web UI remote access
- Add shared folder, let it scan (127GB)
- Add PC using QR code
PC:
- Accept prompts for new shared folder
- Navigate to PHONEIP:PORT
- Enable "debugging" under GUI
- Navigate to PHONEIP:PORT/rest/debug/cpuprof
- .pprof automatically downloads
Confused, I try to replicate it using a previously downloaded SLOW apk - v1.28.0.0
Android:
- Uninstall v1.30.0.2
- Install v1.28.0.0 previously downloaded apk
- Add shared folder, let it scan (128GB)
- Add PC using QR code
PC:
- Accept prompts for new device and shared folder
It doesn’t stall. WTF???
The only weird issue was I noticed v1.28 had folder created + PC added even though “keep data” was not ticked when I uninstalled. I tried twice and folder + PC was pre-populated. I had to go to app options > clear data+cache to get a truly ‘fresh’ install.
I’ll clear data+cache, uninstall v1.28.0.0 and install v1.28.01.00 and see if it stalls.
Either your phone is reaaally fast or something in this test case is fishy. That would translate to an average throughput of around 500MB/s (read: megabyte per second). Translated to the more common unit for network related stuff, that would be nearly 4Gb/s.
I was also surprised by this. For the ‘fast’ test cases, including the ones I did last night, it appears to be that fast. I’ll double check the folder size logs.
I even set a different destination directory, thinking Syncthing will ‘restore’ files that are marked for deletion by Windows and no longer visible but not actually overwritten yet.
@Catfriend1@bt90 Please let me know if either of you have a precise test method you’d like me to try.
Your “problem” is most likely that Syncthing is smart enough to reuse already existing blocks on the receiving side. If your test dataset is a copy of files which are also present in other folders of the node, Syncthing will simply copy the similiar parts locally.
Either use a fresh dataset for these kind of tests or use a temporary Syncthing installation on your PC.
That might be what is happening. What a great feature. I’ve just been sharing my Android:/DCIM/Camera/ folder from my phone and pointing it to a new PC:/Syncthing-Test/DCIM/ folder.
The existing PC:/Syncthing/DCIM/Camera folder has always been there during the tests though, so shouldn’t it have always been grabbing it locally and fast regardless of the Android APK that I was testing?
Please set up from scratch without importing your settings.
This uses SyncthingNative 2.0.3 with the following changes:
Revert ac8b3342ac9d9fcd2df8655a62bfa2ffb505ab42
Revert cbe1220680fc3d05688aae77e9a50581e669d1be (as this depends on the other reverted commit)
This version fixes the stalling for me. ac8b3342ac9d9fcd2df8655a62bfa2ffb505ab42 is the commit that git bisect said is the first slow commit.
I used this command to generate a (random) dataset for triggering this bug:
for i in $(seq 1 3400); do dd if=/dev/urandom of=/sdcard/dummy/$i bs=10M count=1; done
The dataset I got from that was particularly good for this purpose because it stalled at the beginning in problematic versions. I can distribute the random dataset I got, if that matters to anybody.
The title of that PR isn’t that great: Cache wasn’t removed, it’s just a refactor still caching the cache. However I do remember there was brokeness in the main branch (not stable releases) at some point, so if you are bisecting commits you might run into that.
I’m a new user to syncthing, but have faced the slow android sync on one of my devices for a folder of: 666 files 82 folders ~99.1 GiB.
I sorted it with 1.9 version apk, but I’m willing to nuke it and do some more testing as I have a few such devices, my other main one that receives this whole folder did not have this problem.
1.9 has a fair few inferior bits to it.
Want me to do any testing?
edit
Deleted my data folder on the recipient and I’ve fired up the debug apk above out of curiosity. It’s going fairly slow vs v1.9 - which maintained a steady 60-80 MB/s and got it all done in 20- 30 mins.
This one is spending a lot of time at 0.0MB/s and jumping up to 80MB/s occasionally for a little burst, then back down again. I’m 15 mins in and have got 11GB done (average of 12 MB/s). It is more stable than the previous latest version I used though, that kept stopping completely if I left the device alone.
will report back with more later. If you want any logs or anything let me know (maybe with some instructions if it’s more involved than the basic logs!)
I can honestly say for me this version is no better than the latest fork. The connection issue was my device sleeping rather than anything else related to the slowdown. I’m 1.5hrs in and it’s done 40GB. going to stop it soon.
I renamed the topic, because it seems to be on the Android end, and it looks like it would have an effect no matter the OS on the receiving end, and to emphasize the direction of sync.