The panics are because (our usage of) the database doesn’t tolerate random write errors when it’s writing to disk. This is expected if the database-holding disk is flaky.
1 Like
Thank you very much for the explanation. Those panics occurred when the /sdcard partition was formatted to exFAT. I have now changed it to the regular FAT, and the files seem to be syncing correctly, albeit super slow, but I can live with that, as long as there no errors 
.
You should also try another sd card.
Syncthing is very good at “detecting” faulty hardware.
2 Likes
This is not a true sdcard though, just the Android’s /sdcard partition located on the internal memory, so there is that. I do hope that it is still fine nevertheless.
I am sorry for reviving this topic (again), but I have been getting these database-related panics recently. When this happens, Syncthing does restart by itself and begins syncing again, so fortunately no manual intervention is required, but still…
Is there anything useful in this panic log?
Panic at 2020-05-16T14:07:17Z
panic: read /sdcard/.config/syncthing/index-v0.14.0.db/000321.ldb: interrupted system call
goroutine 23525 [running]:
github.com/syncthing/syncthing/lib/db.(*Snapshot).Get(0x5450b4c0, 0xffffffff, 0xffffffff, 0xffffffff, 0xffffffff, 0xffffffff, 0xffffffff, 0xffffffff, 0xffffffff, 0x53d5d1a0, ...)
syncthing/lib/db/set.go:248 +0x1c4
github.com/syncthing/syncthing/lib/model.cFiler.CurrentFile(0x5450b4c0, 0x53d5d1a0, 0x2a, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
syncthing/lib/model/folder.go:931 +0x60
github.com/syncthing/syncthing/lib/scanner.(*walker).walkRegular(0x54d22420, 0x410cf1d0, 0x52908e80, 0x53d5d1a0, 0x2a, 0x410d2c80, 0x51f31290, 0x54236900, 0x0, 0x9)
syncthing/lib/scanner/walk.go:334 +0x50
github.com/syncthing/syncthing/lib/scanner.(*walker).handleItem(0x54d22420, 0x410cf1d0, 0x52908e80, 0x53d5d1a0, 0x2a, 0x54236900, 0x54236940, 0x0, 0x0, 0x2a, ...)
syncthing/lib/scanner/walk.go:327 +0x25c
github.com/syncthing/syncthing/lib/scanner.(*walker).walkAndHashFiles.func1(0x53d5d1a0, 0x2a, 0x410d2c80, 0x51f31250, 0x0, 0x0, 0x0, 0x0)
syncthing/lib/scanner/walk.go:268 +0x980
github.com/syncthing/syncthing/lib/fs.(*MtimeFS).Walk.func1(0x53d5d1a0, 0x2a, 0x410d2c80, 0x51f31250, 0x0, 0x0, 0x0, 0x405d75e8)
syncthing/lib/fs/mtimefs.go:124 +0x60
github.com/syncthing/syncthing/lib/fs.(*walkFilesystem).walk(0x52892190, 0x53d5d1a0, 0x2a, 0x410d2c80, 0x51f31250, 0x54e978b0, 0x0, 0x0)
syncthing/lib/fs/walkfs.go:46 +0x78
github.com/syncthing/syncthing/lib/fs.(*walkFilesystem).walk(0x52892190, 0x54ca48c8, 0x6, 0x410d2c80, 0x549db6d0, 0x54e978b0, 0x0, 0x0)
syncthing/lib/fs/walkfs.go:71 +0x320
github.com/syncthing/syncthing/lib/fs.(*walkFilesystem).walk(0x52892190, 0x409a9edd, 0x1, 0x410d2c80, 0x549db130, 0x54e978b0, 0x0, 0x54e978b0)
syncthing/lib/fs/walkfs.go:71 +0x320
github.com/syncthing/syncthing/lib/fs.(*walkFilesystem).Walk(0x52892190, 0x409a9edd, 0x1, 0x54e978b0, 0x778, 0x415c23c8)
syncthing/lib/fs/walkfs.go:93 +0xb4
github.com/syncthing/syncthing/lib/fs.(*MtimeFS).Walk(0x5450b500, 0x409a9edd, 0x1, 0x523dec30, 0x54236940, 0x523dec30)
syncthing/lib/fs/mtimefs.go:110 +0x7c
github.com/syncthing/syncthing/lib/scanner.(*walker).walk.func1(0x54d22420, 0x410cf1d0, 0x52908e80, 0x54236900, 0x54236940)
syncthing/lib/scanner/walk.go:114 +0x1f4
created by github.com/syncthing/syncthing/lib/scanner.(*walker).walk
syncthing/lib/scanner/walk.go:111 +0x168
panic-20200516-140717.log (81.4 KB)
This is Syncthing 1.4.2 under Android 4.4, and this one device I cannot upgrade to newer Android, unfortunately.
Unfortunately not - if syncthing can’t read the db, it can’t do its job. And I doubt we can do anything about the system interrupting us while reading the db.
I believe that something is simply broken in the OS. Granted, these are custom ROMs, but I have tried several different ones, and the interrupted system call thing is the case in all of them. The last official version of Android for this device is Android 4.1, and there everything seems to work fine, at least as far as filesystem operations go. The problem is that I need to run 1 application, which requires Android 4.4 and later, so I cannot stay on the old Android because of it.
The panics are not that often though. As long as there is not much filesystem activity, Syncthing seems to stay up and running for long time, e.g. 1d 7h 40m right now. Also, as long as Syncthing manages to restart itself after these panics, and there are no file corruptions (and there has been none so far), I will likely just keep using it this way.
This has been a change in the go runtime. Previously it used to hide and retry a bunch of these interrupted calls, it no longer does. You can try to compile yoour own version with an old version of go, but we’re definately not going to use an outdated version of go just to support outdated phone versions.
Really? Which version of Go do you mean specifically? This is Syncthing 1.4.2 compiled with Go 1.13.10.
At the moment, most of the errors seem to be ignored, i.e. they are reported in the log, but the synchronisation just goes on, e.g.
2020-05-17 10:47:04 Scanner (folder xxxxx-xxxxx, item "xxxxx"): scan: readdirent: interrupted system call
2020-05-17 10:47:15 Database indirection GC failed: write /sdcard/.config/syncthing/index-v0.14.0.db/000376.ldb: interrupted system call
Only this one seems to trigger a panic.
panic: read /sdcard/.config/syncthing/index-v0.14.0.db/000321.ldb: interrupted system call
https://golang.org/doc/go1.14#runtime
A consequence of the implementation of preemption is that on Unix systems, including Linux and macOS systems, programs built with Go 1.14 will receive more signals than programs built with earlier releases. This means that programs that use packages like syscall or golang.org/x/sys/unix will see more slow system calls fail with EINTR errors. Those programs will have to handle those errors in some way, most likely looping to try the system call again. For more information about this see man 7 signal for Linux systems or similar documentation for other systems.
1 Like
This topic was automatically closed after 37 days. New replies are no longer allowed.