Hi I have 2 syncthing boxes that share a particular folder. On source side it has 85k folders while on the encrypted receiving side it has 70k folders.
Total size roughly matches up but the source complains that it has 15k folders out of sync that weigh 128kb each and around 1.85mb in total but it doesn’t sync them. The receiving side shows as Up to Date for this folder.
Source:
Receiving side:
What can I do to fix this so all folders are synced?
What does Syncthing say when you click on the “15,121 items, ~1.85 MiB” link?
It shows a list of 128b folders. I’ve checked a few of them and they do have normal files inside them so they should sync normally, they are not empty (well at least the ones on the first page, I’m not going to manually check 15k folders.
A little too much was redacted in the images to be certain which box is which, but the next step would be to hover a mouse pointer over one of the out of sync files in the pop-up to see what Syncthing says about the file.
I had to redact those because they contain company name.
There are 4 screenshots in the first post, 2 for source and 2 for receiving side. First one in both after the colon is a Folder box on the left side in syncthing, while second one is the device side on the right side in syncthing.
I hovered over the files and nothing pops up.
Bump.
I get a lot of these too. Check for folder renames as I tend to see them more what a folder gets renamed at the send only end. Essentially St knows a folder was created but sometimes doesn’t see it’s been renamed so it’s orphaned in the db. When it gets really bad I will delete the database on the SO side and let it rescan everything. It clears out a lot of clutter
Wouldn’t nuking the database make everything be resynced? The destination side is encrypted.
I would rather avoid resyncing 6tb of half a million files.
I doubt these are all folder renames, this is basically a week old sync folder and this company doesn’t change that many files daily. Certainly not to rename 15k files and folders in a week.
Clearing the database shouldn’t affect any previously downloaded / synced files. It’s a quick and dirty way of updating / refreshing the database. If your not sure, rename the db folder. close St, rename db, open St and a new one is recreated. if something is not right, close St and revert the renamed one back.
If it’s not a folder being renamed, maybe check what temp files are being created and add them as an exception
So this is completely wrong. I just tried it and it absolutely does resync everything. I’ve changed none of the files and yet it started downloading over 140gb just after starting with a new database. Now here is hoping that restoring the database doesn’t destroy things further.
Edit: Yeah it’s absolute mess. Thanks for nothing. I will be opening a bug report on Syncthing github after this mess somewhat stops doing whatever it is doing.
What you are seeing is comparisons. If files appear on both sides, clearing the database does not invoke a redownload / resync. Let the new database do it’s thing and wait.
I have a remote St install on an 8Tb drive with around 400 Gb free space. If I blow that db away, it will not redownload 7.5Tb of data. It will take a few days to catalogue the drive and compare it to the sending end.
It absolutely did start resyncing files. I saw 140gb of traffic going between routers and I can see it in Syncthing stats. 140gb of data is not “comparisions”
Your decision, but if you were to wait, it will be fine, as the db fills up, the 140Gb will come down without redownloading the files. If it is downloading, then there’s likely to be something else that’s causing the St on the NAS to think there’s a mismatch, modified files / different time stamps?
Again, this is not 140gb of data to Sync, it’s already synced data.
this is how syncthing shows data “to be synced” after I restored the database back, it went back from 12tb of data to be synced down to just 2.83tb and it’s dropping.
After restoring the database and during that almost 10tb of comparisions only 2.6gb of data was actually sent between devices.
Maybe syncthing behaves like you say it does on flat normal file sync but this is ENCRYPTED sync which i’ve stated before.
If you want to be extra sure that no data is transferred while the new database is rebuilding, there is a simple solution. Just uncheck all the options for sharing with other devices in the folder’s configuration. Then let it finish scanning and rebuilding its database. When you re-enable sharing, the data exchanged will be the block hashes from both devices, which can amount to some gigs with such large folders. Once both sides have compared those hashes, that’s when they start pulling missing data from the other. If the data is already identical, nothing will be transferred.
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.
