Years with Syncthing and never fully synced

I have used Syncthing for years and it has never really given me a “Warm Fuzzy”. It always seems to say files are up-to-date, but when you do a file properties of the folders, they always show VERY differently.

Maybe it had do so with my large file repository with over 400Gb and files ranging from with VMs that are several GBs to many small source files! Total files is around 850K (before Ignore)

I figured I would try to forums as a last effort before I throw in the towel.

I have tried everything I know and ever did the delta rebuild feature but still this our server.

Now, I need to migrate a machine and need a solution or move to another solution!

Can anyone offer suggestions to stop the bleeding please?

Thanks!

!

You should post full screenshots of UI, as it’s not clear what the state is from these partials.

1 Like

Ok, sorry about that, I had sensitive identifying data in the names and was trying to be security proactive :wink:

Let me create and get refreshed on ticket!

Tks! :slight_smile:

Ok, here is complete screens! My guess is you might say it is still syncing but it never finishes!

Try shutting down both instances of syncthing, running syncthing -reset-deltas, and restarting syncthing.

Do you have only one device on each side? Stuff is blanked out so it’s not clear if there are more devices or more than one folder involved.

Yes, I was trying to not overwhelm with to many issues, but I left the comma unblocked so you could see multiple shares.

  • My local PC syncs with the main device above.
  • Main syncs with the Remote.
  • Local PC does not sync directly with Remote.

Syncthing3

My device has about a 1M files but it which in theory are supposed to sync with main device? However, it does not seem to be synced correctly either. Some is because of some ignores in there for large code repositories, but I just removed and -reset-deltas on My PC, which I had not done yet.

I have already done -reset-deltas on the other 2 last week and on Remote again tonight while doing this thread.

Couple of things.

  1. Remote can be purged and let it resync. It is replacing Main!
  2. My Local PC which has 1M files, I increased scan time from 3600 to 10000 in the thinking it might not be completing scan with large number of files on a SATA hard drive.

Are the ignore patterns the same on all machines?

I guess a screenshot from the third side would help.

I guess you could remove the folder on all devices (at the same time, so they all drop the index), and readd it (with ignore patterns from the start), and reshare them. If the index is old, it’s possible it’s caused by bug in an earlier version.

No patterns were s bit different so maybe they were acting as choke points, but I have removed ALL ignore patterns as of now and letting all level the waters up again and take a look afterwards.

Regarding your comment to “Remove on all devices”, my questions is what is going to happen at that point?

Does it just choose the most recent file and it gets replicated to the others.

I’m a little scared to do that because some of these have versions of work in-progress files that we have versions in various places.

While we are on the subject, at some point I might want to simple version everything since the nature of these files are valuable and many of the files that were ignored were SVN repository files so I would like to start having a version for regular binary files as well, but SVN is not a good method to store those.

I assumed Syncthing can simple add a version modifier to a copy of the file?

Not sure what you are referring to.

Removing and readding all folders would only be advised once they are in sync, or if one of them is a master folder, or you believe that the content of the files is the same.

Syncing SVN repositories is a terrible idea as it’s most likely will just leade to corruption, as SVN is not a single file but a collection of files, which when merged from two different machines are likely result in a corrupt repo.

Yes, which is why I originally had the SVN ignored, not to mention they we a copy of the main repository so just a copy already.

The global state being off with no activity should never happen. Can you restart Syncthing with the STRECHECKDBEVERY env var set to 0 to recalculate those numbers. (Unless it is doing something, as in the remote out of sync items and/or the total amount of up/downloaded data change).

Ignore patterns, even if different on device should not cause these problems.

As already said, SVN or any version control repos shouldn’t be synced. Also access.log caught my eye: Syncing continuously changing files doesn’t work, as by the time the remote request data the local file already changed, so it needs to be rescanned and the circle starts again.