BTW, for perspective, my Syncthing deployment is a PastPerfect dataset that is 10 TiB across more than a million files, the indexes are all larger than 10 GiB, and most of the TrueNAS nodes have 8 GiB of physical memory.
Per comments at Github, Syncthing 2.0 could be a bumpy upgrade, but let me nevertheless express gratitude that the Syncthing team makes good software. Years-ago I ran trials with competing products, and only Syncthing could handle big data across relatively slow networks and small machines.
On TrueNAS 25.04+ the middleware/docker silently aborts syncthing migration operation.
See my other post from before I found this thread, or this github issue with a very hacky workaround. Deleting old database works as well, but I wanted to prevent that.
As for logs from syncthing: the migration is reported as running (ref logs here), but no new messages keep being logged, which in turn means the migration is not running at all.
If Syncthing replied on /rest/noauth/health while migration is in progress, it might mitigate this issue, but I have not tested it. The internal test command is plain: wget --quiet --spider http://127.0.0.1:8384/rest/noauth/health.
@calmh maybe this^ would help.
Nothing in the TrueNAS webui or logs indicates that the docker container is misbehaving or that the middleware manager is killing processes. I would expect a red blinky in the Alerts list
Even if TrueNAS kills the syncthing parent, its children persist and are actively doing I/O.
Nothing in the syncthing log plainly indicates that it was interrupted.
Syncthing runs as two processes, one monitor and one worker child. I donāt know truenas so Iām just speculating, but if it kills the monitor process abruptly (like sigkill instead of sigterm) while also not killing the entire process group, that would leave the worker child running as an orphan while logs go to dev null. Not a great thing to do really.
Iāve had the same error and the migration was making no progress in nearly 24 hours. I decided to stop it, rename the old and the new db to .bak and then start syncthing and let it index again.
A pain but at least it will be on SQLlite then and hopefully no more db changes going forward.
Not sure what you mean by ārename the old and the new db to .bak and then start syncthing and let it index again.ā, i looked in the .STfolder , but was empty, could you possibly elaborate for a newby?
If i just delete the .stfolders will syncthing just rebuild?
I used SSH to open Midnight Commander and navigate to the config folder of my syncthing instance.
in there there was say the index-v14 db and the new index-v2 database.
used F6 to add .bak after the end of both folders.
I then started syncthing again and itās currently still indexing but using the SQLlite db now, once indexed I will just delete both of the .bak db I renamed.