reinstalled the OS, best way to migrate Syncthing?

Tom · February 8, 2018, 5:11pm

My setup hes been syncing between 2 hosts (WAN) since the beginnings of time…well, syncthing time anyway. The synced folders have grown to 5TB across > 7000 files. This data is living outside of the OS drive.

I have now moved one of the hosts over to a new OS drive, and from Win8 to Win10. The “old” Win8 drive is still available, as a failsafe/backup. The index is on yet another drive. The data between both hosts hasn’t changed.

Which steps would be the best way to migrate Syncthing to the fresh OS, so it can start talking to its remote brother again ?

imsodin · February 8, 2018, 5:34pm

If the data is unchanged and at the same path, you can just copy over the entire config directory without any changes.

Tom · February 9, 2018, 11:11am

Thanks @imsodin , data is indeed unchanged & at same path.

Something did not go as expected…

On 1 side of the sync, 2376 items across 3 Folders are stuck “out of sync” this number isn’t changing. This is the host that has remained untouched.

On the other host, which got a fresh OS on the C drive, 1 item remains stuck “out of sync”.

This seems eerily similar to Out of sync items are incorrect and not syncing , but setting “ignore permissions” hasn’t helped. (I tried the option on the host with just the 1 out-of-sync.)

Help ?

imsodin · February 9, 2018, 11:50am

Do you get a failed items row on that folder? If so check what the error is. Also you can go to Actions->Logs to get information on the out of sync items.

Most likely issue are permissions, but the above should clear that up.

Tom · February 9, 2018, 11:56am

no failed items, not in log either (running with sttrace=model) on both hosts.

imsodin · February 9, 2018, 11:59am

If items are out of sync, there must be lines Puller (folder ..., file ...): *operation*: *error-msg* in your log, otherwise something is very weird. Can you double check and post screenshots of the web UI if there is nothing.

Tom · February 9, 2018, 12:11pm

this is the log: https://pastebin.com/YsNUjLtx , starting from me pausing & resuming the folder that is out of sync, and the subsequent reconnect to other host.

And here’s a screeny:

imsodin · February 9, 2018, 12:30pm

This line shows that nothing is out of sync in your folder:

[A7JAA] 2018/02/09 13:01:54.920131 rwfolder.go:273: DEBUG: sendReceiveFolder/music@0xc04262c480 changed 0

The screenshots is from a remote device that is syncing, not a folder that is out of sync. The are connected, but distinctly different: Are any folders out of sync or only devices?

Tom · February 9, 2018, 12:34pm

ahh, thank you for pointing this out, I had never stopped to contemplate the difference! Now i know.

So, all folders show “Up to Date” on both hosts. But still the devices show out of sync items …? Now what ?

imsodin · February 9, 2018, 12:41pm

There is two messages, a good one and a bad:
Good: All your shares are up to date, your data is in sync.
Bad: There is something bogus with the remote completion indication and I would like to debug that with your help

Please output the result of a call to the /rest/db/file endpoint for the out of sync item shown in your screenshot on both devices. If you don’t know how to do that, please ask and I will search for the last support thread that this was needed (I am unfortunately not familiar with windows).

Tom · February 9, 2018, 1:46pm

never done that before, and we’re never too old to learn.

This is the output on from the host that is “missing” the item

{"availability":null,"global":{"deleted":false,"invalid":false,"modified":"2010-01-17T13:42:35+01:00","modifiedBy":"A7JAAVJ","name":"Dalida\\Les Années Orlando [Disc 1]\\04 - Les Hommes de Ma Vie.flac","noPermissions":false,"numBlocks":219,"permissions":"0644","sequence":15778,"size":28626414,"type":0,"version":["A7JAAVJ:1"]},"local":{"deleted":false,"invalid":false,"modified":"2010-01-17T13:42:35+01:00","modifiedBy":"A7JAAVJ","name":"Dalida\\Les Années Orlando [Disc 1]\\04 - Les Hommes de Ma Vie.flac","noPermissions":false,"numBlocks":219,"permissions":"0644","sequence":15778,"size":28626414,"type":0,"version":["A7JAAVJ:1"]}}

and this is what the other side has to say about that file

{"availability":[{"id":"A7JAAVJ-CVXYAH5-XJDMR76-ESC7HH4-LGQU6TH-KF45JZK-LE5AVCW-ZWVDUAB","fromTemporary":false}],"global":{"deleted":false,"invalid":false,"modified":"2010-01-17T13:42:35+01:00","modifiedBy":"A7JAAVJ","name":"Dalida\\Les Années Orlando [Disc 1]\\04 - Les Hommes de Ma Vie.flac","noPermissions":false,"numBlocks":219,"permissions":"0644","sequence":15778,"size":28626414,"type":0,"version":["A7JAAVJ:1"]},"local":{"deleted":false,"invalid":false,"modified":"2010-01-17T13:42:35+01:00","modifiedBy":"","name":"Dalida\\Les Années Orlando [Disc 1]\\04 - Les Hommes de Ma Vie.flac","noPermissions":false,"numBlocks":219,"permissions":"0644","sequence":16561,"size":28626414,"type":0,"version":["A7JAAVJ:1"]}}

imsodin · February 9, 2018, 1:53pm

I think I can stop asking for that info: They agree on everything, just that the “missing” host didn’t get the info that the other one has the file as well. I am pretty sure running syncthing once with -reset-deltas will resolve your problem. There was another case recently with a similar issue and I still have no clue how this might happen. If this is indeed a one of thing, than I guess it’s ok, but if it happens again, please let us know.

Tom · February 9, 2018, 2:02pm

this is on the host that thinks it needs to sync an item ?

My startup cmd would then become syncthing.exe -home D:\Syncthing -reset-deltas , just for one run and afterwards return back to normal?

And when that is done, do the same on the host that thinks it needs to sync 2300 items ?

imsodin · February 9, 2018, 2:03pm

Doesn’t matter which side and yes, just for one run.

Tom · February 9, 2018, 2:04pm

did you read my edit ? edit : nvm, run it once on any of the hosts.

Tom · February 11, 2018, 10:01am

this isn’t going well…

Ever since my previous post about 48 hours ago, Syncthing has been thrashing its home folder with .db files in tmp-index-sorter folders. See resmon.exe screenshot.

On top of that, there are now thousands of “Out of sync items” on both hosts, and the numbers aren’t shrinking. See webif screenshot.

What should i do ?

calmh · February 11, 2018, 11:19am

I don’t know but it doesn’t look healthy. I’d check what it says in the log, as it seems to have exchanged almost no data since connecting based on your screenshot.

Tom · February 11, 2018, 11:24am

i know

What i’ve done now is to pause all Folders, on both hosts. I’m unpausing them one by one, from smallest to largest. Waiting until things settle down before goint to next one.

Thanks for your reply @calmh , i think i will keep an eye on it for a few more hours & update this thread then.

Just 1 thing, is “model” the best sttrace to use, to keep an eye on this ?

calmh · February 11, 2018, 11:43am

I don’t recommend any special trace options here. Something odd must have happened to either the database or the files in the migration or things would just have continued nicely where you left off. Given that, -reset-deltas needs to send the entire index, which can be a significant amount of sorting and stuff but hardly 48 hours worth. Well, maybe, if the 8000 files we see above are just 3% of the actual total…

Tom · February 11, 2018, 11:48am

isn’t the 3% relative to the data size, instead of relative to the number of files ?