Syncing conflict + "isn't making sync progress"

Support
1

I see a few discussion topics but I don’t see one that applies to me. First thing first: Syncthing continues to be rock solid heavy-duty darn near flawless for my use.

I don’t what has changed except one machine is now on a different network;instead of being stand-alone remote, it is now a part of my home LAN.

I have a shared folder across many Syncthing machines and syncing has been perfect. There is no case where two or more machines are trying to access and/or modify the same file at the same time. When this suspect machine (lets call it S02) is on I get syncing conflicts, and it reports the typical “Failed to sync”, “…isn’t making sync progress - retrying in 1m0s.” messages in the log. It does also report in the log "INFO: Puller (folder “/xxxxx…item “yyyyyyy.jpg”): syncing: delete file: file modified but not rescanned; will try again later” message. This only occurs when S02 is running and syncing. As soon as I shutdown S02 all other machines operate perfectly fine and the conflicting ceases to exist.

I also noticed with S02 is on, if I delete files on another other machine (S01, S03, C01, etc) the files come back, and if I delete them 5 times, they come back 5 times. If I delete them on S02, they come back, but it appears only once more, if I delete the files on S02 a second time then they are deleted (on all machines) for good.

When you first boot S02 up and run Syncthing the scanning of the folders seems pretty slow, with some folders in a “Waiting to Sync” state; significantly slowed than compared with the behaviour of my other machines… I checked the machine logs to look for any hardware / software errors and see none, and the machine appears to run perfectly fine otherwise. I even put the disks into another machine and get same error free results but Syncthing still behaves (and logs) the same.

So I am convinced there is something awry with S02. So, what else can I do to troubleshoot? Is there a private folder to drop logs in? Or should I do a --reset-database and be done with it? If I do reset the database do I have to do anything else like re-add all the folders? Or does all the configuration stay as is, and Syncthing just does its re-scanning to re-build a fresh new database?

FYI: S01, S02, SO3 = FreeBSD headless servers. Linux-C01 = Linux Ubuntu laptop WIN-C01 = Windows 10 laptop Two other devices = Android

I’ll try to check back later today, but it just might be a couple days.

Thanks again, FreeBSD_Lover

2

Well I ran syncthing --reset-database. Syncthing now loads up much faster, meaning the bootup scan of each folder on the problematic machine (S02) is very quick, very comparable to the other machines.

However my primary problem remains:

root@s002:/path/goes/here # ls -lhG *conflict*
-rw-r--r--  1 syncthing  mygroup    61K Jan 31 17:36 device_01i.sync-conflict-20230131-174206-SYNC-ID.jpg
-rw-r--r--  1 syncthing  mygroup    75K Jan 31 17:41 device_01i.sync-conflict-20230131-174733-SYNC-ID.jpg
-rw-r--r--  1 syncthing  mygroup   279K Jan 31 17:35 device_02i.sync-conflict-20230131-174206-SYNC-ID.jpg
-rw-r--r--  1 syncthing  mygroup   189K Jan 31 17:40 device_02i.sync-conflict-20230131-174621-SYNC-ID.jpg
-rw-r--r--  1 syncthing  mygroup   130K Jan 31 17:38 device_03i.sync-conflict-20230131-174206-SYNC-ID.jpg
-rw-r--r--  1 syncthing  mygroup   128K Jan 31 17:41 device_03i.sync-conflict-20230131-174733-SYNC-ID.jpg

…plus the log messages continue.

I should add is what happens is S01 creates a file, and then on a regular basis it updates the file. No other Syncthing machine does anything with the file except sync a copy. Only S01 creates, modifies, and deletes the file. When the file is created all appears good. But when the file is modified, approx every 5 minutes, that is when the conflict file is created, every 5 minutes when the file is updated. From what I can tell the update and any future deletion successfully syncs across the devices.

I failed to mention that the S02 machine was running version 1.15.1, was shutdown for many months, and then after a recent boot up, and appearing to sync fine, it was upgraded from v1.15.1 to v1.22.0. The logs show right after that upgrade when we see the conflict files and logs message about failing to sync and not making progress.

I should add that Syncthing has permission to the drwxrwxr-x folder, and is not complaining of permissions issues. There have been no changes made here and the logs show the issue occurring after the upgrade.

Looking for comments now for sure.

Thanks again, FreeBSD_Lover

3

Well I just deleted one of the synced folders in Syncthing, put it back, and no change.

Really, looking for comments now for sure.

4

Could you post screenshots of Syncthing’s web GUI from S01 and S02? (Please be sure to expand the folder and device panels so that the details are visible.)

Syncthing’s run-time log from S02 would also be helpful.

5

What specifically are you looking for? These are identical across all machines: Status, Folder ID, Folder Path, Global State, Local State, Shared With, and Latest Change.

I’ve been doing various debugs and watching (tail -f) the logs in real time and nothing has caught my eye. What specifically should I look for in log/debugs? Sorry but don’t really want to post key info out in public.

6

Actual folder ID/name and device ID/name isn’t important as long as it’s still easy to understand the interconnections between devices and folders.

In Syncthing’s web GUI, under Actions → Logs there’s a tab labeled “Debugging Facilities”. Check out the list of debugging options to increase the amount of info in the run-time log.

It’s fine to redact info you don’t feel comfortable with sharing. Just try not to blot out so much that it’s difficult, or impossible, for others to help.

How is the system clock on S02 relative to the other hosts? Filesystem type on the volume with the conflicting files?

7

Yeah I have been turning on extra debug, but nothing seems glaringly wrong. If I knew what key debug items to turn on (versus all of them) I could do that and post here, albeit it a bit sanitized.

Servers S01, S02, and S03 are FreeBSD machines with UFS filesytem, and all three are running NTP. I see no error messages in the logs about time but will try to double check that. These machines running Syncthing have been bullet proof flawless for years. The only possibility I can think of is what I already mentioned previously:

S02 has been turned off for at least 45 minutes and not a single conflict anymore. But as soon as it powers up and starts syncing (which will be tomorrow now), it will create a new sync conflict file every time an existing file is updated; yet those existing files do appear to successfully sync across all machines, including client and Android machines.

Thanks again, FreeBSD Lover

8

There are known bugs regarding needless conflict file creation around the v1.22 versions. Mostly surfaced on Android, but maybe the UFS has similar characteristics triggering the bug. Try updating to the latest release, v1.23 right now.

9

Thanks.

Verified the time, machines are within 1 second of each other thanks to NTP. I will try to see what could be different with S02, because other is also FreeBSD UFS and is fine. Maybe when I installed the OS I did something slightly different.

I’ll install v1.23 soon enough.

Thanks again!

10

Sorry for the reply, I’ve been busy and I wanted to be sure of the results. I upgraded S02 to v1.23.1 and the sync conflict issue went away. I do have another issue (that I reported above, but thought it was related), but it’s so infrequent I need more time to try to nail down what is going on, and I need to search the forum for possible related discussion. In a nutshell sometimes when I delete a file, it comes back. I have to delete it a 2nd time. If this is a known issue point me to the existing discussion and off I go.

Thanks again everyone for the great support, FreeBSD_Lover

11

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.