Issue with v0.14.47 with Large Folder Sync

Setup. 15 folder shares with roughly 1GB -> 10GB all syncing correctly. The largest folder is 13TB roughly 60,000 files and since the update it is stuck at 0% syncing.

Two are running dockers on unraid, the 3rd syncthing is running on a OSX 64bit binary. We are all on the same version, 0.14.47.

We have had this running for over a year, and this is the first time running into this issue.

In the logs I can see alot of traffic going on, regarding blocks and it traversing through the folders off and on.

What should I look at, or what logs can I provide?

Thanks

It’s been out for a week, are you sure it’s .47 and not .48-rc?

The dockers we use only update on Friday nights and lag behind a little. I just confirmed everyone is on 0.14.47. Any ideas what to look at next?

Well you should look at the logs and see if there are any errors there. Also, posting screenshots should help.

Out of the logs, which debugging facilities should I be looking at so I can post them here.

Check which folders the out of sync items are coming from by clicking on the out of sync text and check the logs on the other side.

The out of sync folders are the ones marked Up To Date on both sides. Both devices says they are syncing, but its never changing from 0%. This is after 3 days of letting it run since the other paused folders work, we thought it might be exchanging metatadata in the background and the ui wasnt telling us.

How do you know that? What does the screenshot on the other side show.

I was referring to the folders marked “Up to Date” on my side. The only change between each side is the update.

Here is a screen shot from the other side.

My question was how do you know which folders are out of sync, because out of sync items might list items for different folders.

Anyways, this side looks like it’s scanning (or atleast attempting). You should probably enable scanner logging facility and see what’s in the logs.

Hey Audrius,

Long time ((syncthing) user). First time (caller).

I’ll represent the Mac side of this equation. For clarification, the Mac is actually not running either of the file systems that I have read about you guys testing (HFS+ and APFS). It is running on OS X 10.11 and the storage array is on openZFS.

Let’s say that we have Folder ‘A’ syncing between device ‘1’, ‘2’, & ‘3’. Is there a limitation of file watchers being enabled within that set of devices? Device 1 runs file watcher on Folder A. Device 2 says that’s nice, I will also enable file watcher. Would that cause any issues? I can see that enabling and disabling file watcher changes something in the file system locally.

It seems that something caused the entire folder to be re-synced across all devices. We paused machine 3 to get 1 and 2 back in sync. Once that was accomplished, we unpaused device 3. We did notice while catching device 3 up, if device 1 dropped new files, device 2 saw that they were out of sync items, but did not start pulling those items. Currently waiting for device 3 to catch up to device 1 and 2, then suspect that the out of sync items will sync to device 2 and 3. Is this expected behavior? Could device 2 not go ahead and sync the items?

File watcher is enabled locally, for a selected local folder. It has no relation to devices at all.

Enabling or disabling watcher does not change anything on the filesysyem, unless your filesystem does something when it’s asked to watch for changes.

Device 2 should start downloading the files straight away, as there isn’t cluster wide “I am now downloading files” lock. Any peer can download and “provide” files at the same time. As I said above, you should look at the logs, I suspect it’s failing.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.