Your case was different; you had hundreds of millions of blocks and massive files, and it would legit take a long time on a fast system. I’m not sure if there something to improve; I mean obviously there would be, engineering wise, but not something short term. The things I quoted was a fair number of files but a handful of blocks (I’m guessing a lot of deleted files and/or directories, so no blocks), which would normally migrate quickly.
1 Like
I’ve had few tens of minutes long migration of 3GiB database on my TrueNAS 25.04.2.1 deployment. It took around ~25 minutes to complete.
There were 2 issues:
- TrueNAS apparently kills syncthing during migration
- Syncthing cannot recover from unfinished migration when interrupted. This does not seem deterministic.
I’ve filed downstream issue with TrueNAS for healthcheck/monitoring of the container which does not provide sufficient time for the migration to finish.
2 Likes
Currently at…
files=61000 blocks=30028318 duration=23h24m12s filesrate=0.7240119024475011
This is just the 4th folder (28TB indexed), took another 10 hours to convert the first three.
464545280 Aug 14 20:48 folder.0001-cxtfdwnt.db
2499584000 Aug 14 22:46 folder.0002-6yenjrsk.db
5970890752 Aug 15 05:48 folder.0003-aqebwydr.db
6867693568 Aug 16 13:38 folder.0004-ce9z7tuv.db
32768 Aug 15 05:48 main.db
Not sure if the migration will finish sooner, if ever, than deleting and recreating the database. That usually takes a week for me. Every other month the database gets corrupted. I hope this format will be more resilient.
Quick side question: how much disk space does the new database format need compared to old one? Thanks!
This is just a single point of data, but it looks like mine was right around 78% larger?
root@truenas[/mnt/fast/custom-apps/syncthing/config]# du index-v0.14.0.db-migrated
23382699 index-v0.14.0.db-migrated
root@truenas[/mnt/fast/custom-apps/syncthing/config]# du index-v2
41679183 index-v2
root@truenas[/mnt/fast/custom-apps/syncthing/config]#
We had some discussion about the database size (and how to possibly reduce it) during the initial testing (see Syncthing on SQLite -- help test! - #75 by tomasz86 and later posts). I’m not sure any type of automatic cleanup has actually been implemented since then though.
I still get huge space savings with NTFS compression on my Windows systems (e.g. 5.48 GB uncompressed vs 2.66 GB compressed on my main PC right now).
Hi I updated to 2.0.1 and I have problem with migration freezing. I am running trunas scale 24.10. and syncthing 2.0.1 is now two days in deployment phase. When I check logs, it shows messages about migrating folders but after 20 minutes or so, it stopped migrating, no new logs at all. When I stop and restart deployment is shows new messages about migration, again for about 20 minutes and than it stops again. I am fine with migration taking some time, but why it is freezing constantly ?
My database was 8GB, now after migration 42GB! wtf?
Is this right after the migration? Have you tried waiting a bit for the database to settle down? Normally, it should at the maximum be about twice as large as the old one…
It looks like my container stops migrating database after 15minutes ( almost exactly 15 ) every time. Is there something that could be causing this stoping? Like some healthcheck, timeout counter or something like that ?
Here is my Container LOG:
ContainerLOG15minFreeze.txt (17.0 KB)
I’m sure Syncthing v2.0 has some secret greatness, but the upgrade has brought down Sync between three TrueNAS 25.04 servers in different locations. After it has churned for over 4 days, I’m preparing to start over with a fresh database.
My question; if I have a folder with 1000’s files and I configure it to sync and the files are exactly the same on the three servers (because they were previously replaced with the earlier version), is there a procedure for Syncthing v2.0 to just index instead of copying the files for no apparent reason?
Thanks for any advice.
Ahother run, same behavior, I even increased RAM that is assigned for this app from 1GB to 4GB.
Another_run.txt (17.2 KB)
I’ve merged your topic into the existing one, as this is the same problem about very slow migration, with the common thread being that most of the affected systems run TrueNAS.
Not sure the “best” way but I would probably set them all to “receive only” and then pause all the connected devices but not the folders. Let the folders all scan completely.
Once they all scan I would unpause the devices and let them connect. If everything is “in sync” then you can change them back to send/receive. If there is a mismatch then decide what you want to do. Maybe one of the 3 devices becomes the reference. Set that machine to send only, and push the changes to the other machines. Then once everything is in sync you can set all back to send/receive.
Maybe there’s a better way but this is probably what I would do.
1 Like
I think all of you, who have experienced Syncthing seemingly freezing while performing the migration, may want to check if you actually have enough free space on the disk for the new database. We’ve had a report on GitHub just today, where the log was stuck at “still migrating folder”, while in reality, the device ran out of space and was unable to continue the migration process (see https://github.com/syncthing/syncthing/issues/10253#issuecomment-3194465384 for details).
Thanks for the suggestions.
My servers have +40TB of free space. Syncthing freezing is not being caused by storage constraints. I think this is being caused when the number of files is high.
However, I asked for advice about stopping Syncthing from re-replicating all the files after the upgrade. I have the main server configured as read only and send only. The two replication servers are receive only. After the Syncthing upgrade to 2.0, it had been trying to copy all files again. BUT, I selected 'Ignore Permissions’ on all the repliacted folders and it stopped trying to replicate the files again.
So, apparently something in the upgrade seemed to flag a change to the files that was being picked up as a permissions change. Once I ignored this, the servers reindexed and I’m back to a reasonable state.
I’m going to update the destination to 2.0 and hopefully nothing new will create problems…
Thanks all.
Let me report the successful update of all servers to Syncthing v2.0.1 on TrueNAS 25.04.2.1.
This was a pretty painful process, but it was a significant improvement to ‘ignore permissions’ on all shared datasets. Because it is a one-way replication from a read-only source, I don’t see a downside of this setting and it definitely resolved unnecessarily re-replicating all files.
Thanks.
2 Likes
Hi all.
Not sure if this is the right place to post, but I have a similarly long migration process.
I’ve compiled Syncthing 2.0.x for my old Drobo 5N2. After upgrading my app, the migration began, and I could see the progress on the temporary GUI:
*** Database migration in progress ***
2025-09-02 18:22:55 INF syncthing v2.0.5 "Hafnium Hornet" (go1.25.0 linux-arm) builder@github.syncthing.net 2025-09-02 19:03:22 UTC (log.pkg=main)
2025-09-02 18:22:56 INF Migrating old-style database to SQLite; this may take a while... (log.pkg=syncthing)
2025-09-02 18:23:01 INF Starting temporary GUI/API during migration (address=0.0.0.0:8384 log.pkg=main)
After about 40 hours, the migration seems to have completed. However, my instance seems to be stuck at “INF Cleaned out database file for old, dropped folder”:
2025-09-04 10:56:37 INF Migrated folder (folder=<redacted> files=2573 blocks=2129234 duration=1h11m56s filesrate=0.5961062708107198 log.pkg=syncthing)
2025-09-04 10:56:37 INF Migrating virtual mtimes... (log.pkg=syncthing)
2025-09-04 10:56:42 INF Migration complete (files=534901 blocks=5723 duration=40h33m46s log.pkg=syncthing)
2025-09-04 10:56:43 INF Cleaned out database file for old, dropped folder (path=folder.0001-<redacted>.db log.pkg=db/sqlite)
I have enough free space on the NAS.
Is this normal? Is there some other process going on that I can’t see? What should happen after the DB migration?
Thanks for any help you can provide!
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.