Android frozen syncs

I could probably try…what do you mean by complex? Hopefully not huge :slight_smile: I’ve got about 7GB free on my phone.

maybe 3gb mixed files?

PM or email me and we can try this if you still want to.

If you want to capture logcat output from non-rooted phone, I updated my little wrapper to allow STTRACE=all for capturing if you want to try it out https://github.com/firecat53/syncthing_android_wrapper/releases.

Here we go

Now I ran libsyncthing.so from an adb shell hoping that killing the adb shell also kills the ST binary if ever it gets stuck. So it got stuck and I killed and disconnected the adb shell. But that did not kill ST binary. How is that possible though? I literally kill the parent process and libsyncthing.so can just go on its zombie life as if it was never part of a parent process. Same thing with running it from a terminal.

But that is not the case always, generally it quits gracefully unless it is in that crazy “Zombie” status.

Sorry guys I am just speculating because i really do not have any concrete of this particular problem.

How is the binary compiled? Which sdk version is it used with?

My other tablet is a Kitkat 4.4.4 (CM), and that one does not freeze so I do not seem to see a zombied out crazy ST process. The only difference between this tablet and the otherone seems to be that this one has SeLinux enforced and the other one does not.

I will remove everything and try this again

@firecat4153

I want to try it, let me straighten this out first. I will pm you. Thanks for your effort.

@calmh

Do you think that the web server might be causing the process issues Android? I have a new insight, well I might say a new observation.

It seems like the process turns into zombie when I refresh (not restart) the web page, but not always. If I do that after sometimes the process started, it is more likely to turn into this zombie thing.

I hope it makes sense.

After running syncthing as root on android, sync doesn’t freeze any more at 99% or similar with 0.4.13. Do you have suggestions how to start/stop it nicely via (bash) scripts (initiated from Tasker)? Is the correct way how to stop it via: am kill com.nutomic.syncthingandroid ?

1 Like

Can’t really say. :confused: I’ve never heard of this happening on “regular” Linux, and I’m not experienced enough with Android to give a qualified guess.

@calmh I am pretty sure that is the issue. If you refresh the browser , it seems to be turning the process into some kind of dangling state. I am thinking that is the cause because I actually managed to get that happen with chroot Debian on Android. It happened as if the browser refresh killed the original process and started a new process which was not part of the original process tree. And then it was in that weird unkillable state.

Maybe it is the way compiled for Arm is the issue ? Debug flags, optimizations, arm versions?

@peter_sc

I just use the root terminal really.

What does the console log say?

there is no console says really, no crash, but when I run the binary I run is as attached (ie without &, just “syncthing” ). What I see is that somehow the first process is killed and restarted as it run as “syncthing &”), not sure what that is called.

Btw here is a crash from a tablet that was left overnight. it might be relevant

bash syncthing
[5HY7A] 05:19:27 INFO: syncthing v0.9.10 (go1.3 linux-arm android) vagrant@testing32 2014-08-31 19:52:29 UTC
[5HY7A] 05:19:27 INFO: My ID: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
[5HY7A] 05:19:29 INFO: Starting web GUI on https://127.0.0.1:8080/
[5HY7A] 05:19:39 INFO: Performing initial repository scan
[5HY7A] 05:19:50 INFO: Created UPnP port mapping - external port 43368
[5HY7A] 05:19:50 INFO: Starting local discovery announcements
[5HY7A] 05:19:50 INFO: Starting global discovery announcements
[5HY7A] 05:19:50 OK: Ready to synchronize default (read-write)
[5HY7A] 05:19:50 OK: Ready to synchronize xxxxx (read-write)
[5HY7A] 05:19:50 INFO: Node xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx is "xxx+_root" at [dynamic]
[5HY7A] 05:19:50 INFO: Node xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx is "localhost" at [dynamic]
[5HY7A] 05:19:50 INFO: Node xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx is "xxxx" at [dynamic]
[5HY7A] 05:19:50 INFO: Node xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx is "xxxx" at [dynamic]
[5HY7A] 05:19:54 INFO: Established secure connection to xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx at 192.168.4.101:56296-192.168.4.166:22000
[5HY7A] 05:19:54 INFO: Node xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx client is "syncthing v0.9.10"
[5HY7A] 05:19:54 INFO: Nodexxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx hostname is "localhost"
[5HY7A] 05:19:55 INFO: Established secure connection toxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx at 192.168.4.101:40936-xxxxx:41304
[5HY7A] 05:19:55 INFO: Node xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxclient is "syncthing v0.9.10"
[5HY7A] 05:19:55 INFO: Node xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx hostname is "xxxxx"
[5HY7A] 05:33:24 INFO: Connection to xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx closed: ping timeout
[5HY7A] 05:40:56 INFO: Connection to xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx closed: write tcp xxxxx:41304: connection reset by peer
[5HY7A] 05:43:57 INFO: Established secure connection to xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx at 192.168.4.101:40962-xxxx:41304
[5HY7A] 05:43:57 INFO: Node xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxclient is "syncthing v0.9.10"
[5HY7A] 05:43:57 INFO: Node xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx hostname is "xxxxx"
[5HY7A] 05:49:51 INFO: Renewed UPnP port mapping - external port 43368
[5HY7A] 06:09:20 INFO: Connection to xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx closed: EOF
[5HY7A] 06:27:00 INFO: Paused state detected, possibly woke up from standby.
[5HY7A] 06:27:00 INFO: Restarting
[5HY7A] 06:27:40 OK: Exiting
localhost ~ # [5HY7A] 06:47:00 INFO: syncthing v0.9.10 (go1.3 linux-arm android) vagrant@testing32 2014-08-31 19:52:29 UTC
[5HY7A] 06:47:00 INFO: My ID: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
[5HY7A] 06:47:00 INFO: Waiting for parent to exit...
[5HY7A] 06:47:00 INFO: Continuing
panic: runtime error: slice bounds out of range

goroutine 16 [running]:
runtime.panic(0x40e308, 0x744eef)
        /home/vagrant/build/com.nutomic.syncthingandroid/build/go/src/pkg/runtime/panic.c:279 +0x1a0
os.(*File).write(0x1090e6a0, 0x10936a00, 0x4c2, 0x13a9, 0xffffffff, 0x0, 0x0)
        /home/vagrant/build/com.nutomic.syncthingandroid/build/go/src/pkg/os/file_unix.go:218 +0xe4
os.(*File).Write(0x1090e6a0, 0x10936a00, 0x4c2, 0x13a9, 0x0, 0x0, 0x0)
        /home/vagrant/build/com.nutomic.syncthingandroid/build/go/src/pkg/os/file.go:139 +0x88
github.com/syndtr/goleveldb/leveldb/table.(*Writer).writeBlock(0x11824000, 0x1182402c, 0x2, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
        /home/vagrant/build/com.nutomic.syncthingandroid/ext/syncthing/src/github.com/syncthing/syncthing/Godeps/_workspace/src/github.com/syndtr/goleveldb/leveldb/table/writer.go:190 +0x41c
github.com/syndtr/goleveldb/leveldb/table.(*Writer).finishBlock(0x11824000, 0x0, 0x0)        /home/vagrant/build/com.nutomic.syncthingandroid/ext/syncthing/src/github.com/syncthing/syncthing/Godeps/_workspace/src/github.com/syndtr/goleveldb/leveldb/table/writer.go:225 +0x98
github.com/syndtr/goleveldb/leveldb/table.(*Writer).Append(0x11824000, 0x10c05a82, 0x8b, 0x3e457e, 0x10c05b0d, 0x30, 0x3e44f3, 0x0, 0x0)
        /home/vagrant/build/com.nutomic.syncthingandroid/ext/syncthing/src/github.com/syncthing/syncthing/Godeps/_workspace/src/github.com/syndtr/goleveldb/leveldb/table/writer.go:258 +0x300
github.com/syndtr/goleveldb/leveldb.(*tWriter).append(0x109961b0, 0x10c05a82, 0x8b, 0x3e457e, 0x10c05b0d, 0x30, 0x3e44f3, 0x0, 0x0)
        /home/vagrant/build/com.nutomic.syncthingandroid/ext/syncthing/src/github.com/syncthing/syncthing/Godeps/_workspace/src/github.com/syndtr/goleveldb/leveldb/table.go:445 +0x2c8
github.com/syndtr/goleveldb/leveldb.(*tOps).createFrom(0x10b909e0, 0xb6f13b60, 0x10996030, 0x0, 0x0, 0x0, 0x0)
        /home/vagrant/build/com.nutomic.syncthingandroid/ext/syncthing/src/github.com/syncthing/syncthing/Godeps/_workspace/src/github.com/syndtr/goleveldb/leveldb/table.go:311 +0x1b4
github.com/syndtr/goleveldb/leveldb.(*cMem).flush(0x10b8e360, 0x10b82380, 0x0, 0x0, 0x0)
        /home/vagrant/build/com.nutomic.syncthingandroid/ext/syncthing/src/github.com/syncthing/syncthing/Godeps/_workspace/src/github.com/syndtr/goleveldb/leveldb/db_compaction.go:80 +0x154
github.com/syndtr/goleveldb/leveldb.(*DB).recoverJournal(0x1092c380, 0x0, 0x0)
        /home/vagrant/build/com.nutomic.syncthingandroid/ext/syncthing/src/github.com/syncthing/syncthing/Godeps/_workspace/src/github.com/syndtr/goleveldb/leveldb/db.go:532 +0x674
github.com/syndtr/goleveldb/leveldb.openDB(0x10a95080, 0x0, 0x0, 0x0)
        /home/vagrant/build/com.nutomic.syncthingandroid/ext/syncthing/src/github.com/syncthing/syncthing/Godeps/_workspace/src/github.com/syndtr/goleveldb/leveldb/db.go:104 +0x3d4
github.com/syndtr/goleveldb/leveldb.Open(0xb6f0f058, 0x10b8c8c0, 0x0, 0x0, 0x0, 0x0)
        /home/vagrant/build/com.nutomic.syncthingandroid/ext/syncthing/src/github.com/syncthing/syncthing/Godeps/_workspace/src/github.com/syndtr/goleveldb/leveldb/db.go:169 +0x1b4
github.com/syndtr/goleveldb/leveldb.OpenFile(0x10b908e0, 0x1f, 0x0, 0x0, 0x0, 0x0)
        /home/vagrant/build/com.nutomic.syncthingandroid/ext/syncthing/src/github.com/syncthing/syncthing/Godeps/_workspace/src/github.com/syndtr/goleveldb/leveldb/db.go:191 +0x98
main.main()
        /home/vagrant/build/com.nutomic.syncthingandroid/ext/syncthing/src/github.com/syncthing/syncthing/cmd/syncthing/main.go:199 +0x1db0

goroutine 19 [finalizer wait, 23 minutes]:
runtime.park(0x407ac, 0x748f94, 0x747b49)
        /home/vagrant/build/com.nutomic.syncthingandroid/build/go/src/pkg/runtime/proc.c:1369 +0x5c
runtime.parkunlock(0x748f94, 0x747b49)
        /home/vagrant/build/com.nutomic.syncthingandroid/build/go/src/pkg/runtime/proc.c:1385 +0x40
runfinq()
        /home/vagrant/build/com.nutomic.syncthingandroid/build/go/src/pkg/runtime/mgc0.c:2644 +0xa0
runtime.goexit()
        /home/vagrant/build/com.nutomic.syncthingandroid/build/go/src/pkg/runtime/proc.c:1445

goroutine 21 [select]:
github.com/syncthing/syncthing/events.(*Subscription).Poll(0x10900fa0, 0xf8475800, 0xd, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        /home/vagrant/build/com.nutomic.syncthingandroid/ext/syncthing/src/github.com/syncthing/syncthing/events/events.go:162 +0x300
github.com/syncthing/syncthing/events.(*BufferedSubscription).pollingLoop(0x1091a8a0)        /home/vagrant/build/com.nutomic.syncthingandroid/ext/syncthing/src/github.com/syncthing/syncthing/events/events.go:194 +0x40
created by github.com/syncthing/syncthing/events.NewBufferedSubscription
        /home/vagrant/build/com.nutomic.syncthingandroid/ext/syncthing/src/github.com/syncthing/syncthing/events/events.go:188 +0x178

goroutine 22 [runnable]:
main.trackCPUUsage()
        /home/vagrant/build/com.nutomic.syncthingandroid/ext/syncthing/src/github.com/syncthing/syncthing/cmd/syncthing/gui_unix.go:22 +0x108
created by main.init·2
        /home/vagrant/build/com.nutomic.syncthingandroid/ext/syncthing/src/github.com/syncthing/syncthing/cmd/syncthing/gui_unix.go:15 +0x3c

goroutine 23 [chan receive, 3 minutes]:
main.saveConfigLoop(0x1091b380, 0x24)
        /home/vagrant/build/com.nutomic.syncthingandroid/ext/syncthing/src/github.com/syncthing/syncthing/cmd/syncthing/main.go:776 +0x50
created by main.main
        /home/vagrant/build/com.nutomic.syncthingandroid/ext/syncthing/src/github.com/syncthing/syncthing/cmd/syncthing/main.go:199 +0x1738

goroutine 24 [runnable]:
time.Sleep(0x3b9aca00, 0x0)
        /home/vagrant/build/com.nutomic.syncthingandroid/build/go/src/pkg/runtime/time.goc:39 +0x3c
github.com/syndtr/goleveldb/leveldb/util.(*BufferPool).drain(0x10a22ff0)
        /home/vagrant/build/com.nutomic.syncthingandroid/ext/syncthing/src/github.com/syncthing/syncthing/Godeps/_workspace/src/github.com/syndtr/goleveldb/leveldb/util/buffer_pool_legacy.go:136 +0x34
created by github.com/syndtr/goleveldb/leveldb/util.NewBufferPool
        /home/vagrant/build/com.nutomic.syncthingandroid/ext/syncthing/src/github.com/syncthing/syncthing/Godeps/_workspace/src/github.com/syndtr/goleveldb/leveldb/util/buffer_pool_legacy.go:160 +0x1a8
localhost ~ # ps

Oh yes, that’s a perfectly fine crash log Would you mind creating a github issue just for the sake of it?

I do not have gh account. I will try to join later and add it.

Could this be related to the issue where 0 byte files would cause sync to stop. I haven’t tested on Android for a while (will do as soon as I get a moment). Not sure if that bug was identified/addressed, but, if not, it would be great to at least get a test up that checks for behaviour of 0 byte files on sync.

(When I last tested, I could reliably cause sync to hang by including a 0 byte file and get it to restart by removing it.)

(And apologies in advance if this has been addressed and if I missed it.)

@firecat4153 I just saw your wrapper project now. I think it would make sense to include the STTRACE functionality directly into syncthing-android, so it’s easier to use (eg as a setting where you can choose what to trace).

I agree! Wasn’t trying to start a new project…just heard that you were really busy so I was trying to help people get the logs that @calmh needed for troubleshooting :slight_smile:

Although if someone wants to port to iOS, Kivy might be an easy way to cross-platform develop without having to reinvent the wheel for each one!

Scott

It would actually be pretty simple to add it to syncthing-android, I just haven’t done it because it wasn’t requested yet iirc.

If you (or someone else) wants to implement it, that would be great (and I could also give some help if needed). Otherwise, I might add it at some point after 0.5.0.