Why so many locally modified files?

Hi, I’m quite new to syncthing, I recently moved over from using rsync to keep my files between systems in sync.

So my environment has 3 systems running syncthing:

Machine A is my master machine and the only place files should be modified, added to or deleted from. This is an Unraid NAS running the latest versions of Unraid and syncthing. This is strictly a sender for 14 shared folders.

Machine B is strictly a receive only machine and acts as a backup/copy of everything on A. This is another Unraid NAS running the latest versions of Unraid and syncthing. This is strictly a receiver

Machine C is a media server running Plex on a Synology NAS DS1019+. It’s running the latest versions of DSM, Plex and syncthing

All the machines are on the same lan segment and have static addresses

So to my problem/question. On machine C everything was synced and working quite happily if some what slowly, but for some reason it would only connect to machine A with a connection type of TCP WAN, where machine A was connected to C with a connection type of TCP LAN, so I started fiddling around to try and get it to be TCP LAN, thinking that this might improve the speed of the connection. Well the up shot of the fiddling I managed to break the connection to the GUI, their was something listening on port 8384, but the connection was constantly be rejected, tried stopping and starting syncthing, no change, rebooted the machine still no connection. The actual syncing was going fine as I could see additions to my plex server appearing the plex GUI. So after a couple of days with no GUI access I tried ssh’ing into the box to see if I could see an issue in the config files, but I couldn’t, mainly cause I had no idea what I was looking for, so I decided to uninstall syncthing and reinstall. This all happened smoothly and I was able to reconnect to Machine A and reset up all the shares and I even managed to get he conectionto be TCP LAM. The issue I’m having is a the scanning is happening painfully slowly. Of the 14 shares only 3 are in the scanning phase and then after 30 hours have completed 18% 4% and 8% respectively, all the other 11 are in a waiting to scan state. To take the most extreme case one of the folder shares is a pictures folder and on machine A the local and global stats are 621986 files, 2169 directories and 362GiB, on machine C the Global state is 291625 files, 2169 directories, and 150 GIB, and the local state is 258026, files 2169 directories and 136 GiB, along with 121471 items, size 73.2 GiB out of sync, and locally changed items 66465 items 58.6 GiB in size. Prior to reinstalling syncthing both systems were in lockstep showing the same files, directories and size stats with no local changes. So 1. Why is machine C not seeing all the files from machine A, if I add the local files and the out of sync I get a figure 3800022 which is some what short of the 621086 files that it was reporting and 2, why does it believe that so many of the files have been changed locally. This folder gets maybe 100 additions a day, but has had none since i reinstalled syncthing as I didn’t want to confuse it to much. I did the reinstall around 6pm on Sunday and now 30 hours later it has managed to scan 8% of the folder, at this speed it’s going to take 15 days to complete. But I am worried about all the locally modified files, on my other folders the had the locally modified the big red reset button did absolutely nothing. I tried deleting the files on the receiving machine that it felt were locally modified and allow syncthing to bring them back across, but they still showed up as being locally modified. The only way I could get rid of the big red button was to remove everything from the receiving side remove the folder share from syncthing on both sides and reconfigure from scratch then once everything was transfer’d over all was OK no big red button, I’m hoping the I don’t have to do that as all the 3 shares that are currently scanning on macing C are showing large numbers of locally modified files and god know what it will look like once it gets round to scanning the other 11 folders. I would truly appreciate any advise as to how I could speed up this re-syncing and how to handle the locally modified files with out deleting everything and starting again, that’s not really a viable solution as i have a number of external friends and family that use the plex instance to watch tv and movies

Thanks for getting this far and for any possible assistance

It’s a bit difficult filtering out the important bits, but as a hunch based on what you typed (at least what I filtered out of it);

Are you sure you are running only one instance of Syncthing on device A (I think, could also be C)?

If you simply reinstall Syncthing, one wouldn’t need to set up the shares again since the same config would be used. Then 8384 was already in use, but clearly not by the instance you wanted to talk to. Locally modified files, could surely be explained by another instance touching them. And the different connection type between two devices is a bit curious - although some delay is absolutely possible here.

1 Like

Pictures are worth a thousand words.

OK, sorry my description of the problem was so convoluted and rambling, here’s some pictures

In the pictures I have blanked out the IP addresses and the machine and folder identifiers, it shouldn’t be necessary as the connection is actually working, but if you think it’s relevant I can always post the un-redacted images.

So image 1 below shows the configuration of the Source machine A in the top panel and the recipient machine C in the bottom panel, this is from machine A

This next image 2 shows the configuration of the folder from Machine A’s point of view

Folder View Machine A - Source

In this next image 3, it should be showing the machine C configuration in the top panel and the machine A’s config from C’s point of view in the lower panel

This last Image is showing the folder config on machine C

Folder Viiew Machine C - Recipient

Now, before the reinstallation of syncthing on machine C, it was showing the same numbers in the folder view as machine A for both global and local states, after the reinstallation for some reason syncthing, has determined that, currently - going up all the time, their are 212636 items that have been changed locally. I don’t know why there should be so many local changes as it appears to have no problem with the other 417013 items in the currently in local state.

I’m thinking a possible course of action is to remove this folder from machine C, remove the folder and use rsync to recreate and copy over all the files and then setup syncthing to keep everything in sync from that point

That’s precisely it. Reinstalling presumably meant removing the existing config and database. The files on disk are thus, from new Syncthing’s perspective, pre-existing on disk and not downloaded from the cluster. That means locally changed. However, if they are actually identical this should resolve once everything has converged.

In general, I think the best approach would be to let it finish scanning and stop fiddling around while that is still ongoing. You might get a surprise by everything just jumping to green lights with enough patience.

This amount of data can definitely take a long while to scan on not-so-beefy machines low on CPU power or memory (such as most NAS). You can see the raw hashing performance as one of the earliest log messages, but it’s still contending for disk I/O and memory. Running out of RAM and swapping to disk adds even more I/O load. If these are rotational disks, no SSDs, scanning more things in parallel will slow down considerably, so try pausing all folders but one. If your Syncthing database happens to be on the same (rotating) disk, performance is expected to be abysmal, as a lot of write activity happens there during scanning.

1 Like

Thank you Jakob, I’ll try and exercise some restraint and leave it all well alone and see where everything finally shakes out. As i write this the local state is now correct but the number of locally changed items appears to have plateued, I’ll check and see if it starts going down the next time a scan of the folder happens

Screenshot 2023-03-22 at 1.22.00 AM

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.