Files don't sync until first scan complete?

Support
1

I’ve got a folder that I’m syncing and I’m using an ignore filter (because the full shared folder is many terabytes).

!Archive 2000\**

*

And it’s syncing all of the subfolders but 0 files. I’ve tried

!Archive2000**

!Archive2000\**.*

And every imaginable combination but it only ever syncs folders and no files. Is that just standard behavior until the first scan on the host side is complete?

2

I think you should either wait for a bit more time or test with a smaller folder first. For the record, you could use just

!Archive 2000
*

to achieve the same effect.

1 Like
3

I’ll try again in a week when the initial scan is complete :smiley:

image
image686×285 17 KB

4

With a folder of this size, I think you may be experiencing the same performance issues as reported and discussed in https://forum.syncthing.net/t/is-syncthing-v2-with-very-large-folders-possible/26082. This is because even though you’re ignoring most of it, the ignored files are still recorded and present in the database, meaning that Syncthing may end up spending most of the time on working on it instead of the actual synchronisation.

If you intend to share only a small part of the very large folder, I’d suggest either splitting it into multiple smaller folders, or simply adding the subfolder to Syncthing separately and sharing it instead.

5

After reviewing that thread, it looks like the GC bug hit me. Scanning crashed overnight but stopping/starting resumed it. (I saw the same GC Timed Out 5minute error that they ran into.)

But my setup says “Up to Date” not “Preparing to Sync” so I don’t think it’s actually relevant. I also have some equally or larger shares that were humming along fine before, so I suspect I just need to wait it out to see if it eventually starts transferring files once the scan is complete.

I’m not running on NVMe but I am running on 28 drives + ZFS + 256GB of ARC + 8TB of nvme L2ARC + 3x NVDIMM pMEM LOG devices so it is unlikely that it’s even touching the spinning disks.

Edit: Yeah, I just was able to bring up my index folder and my largest WAL file is only 36.1M on an 11.5G db. So, I don’t think the cleanup passes are breaking anything for me.