To much scan... kills

General
1

Hi, I started to test a mini share between 1 nas (arm), 2 x linux pc and 1 win. The win and one linux ran a portable version on an usb 16Gb key, sharing a folder on the key itself. The master is the nas, read only, no delete. But I played to try and discover. The volume of data was about 2Gb, in 13 to 17 files. The scan freq left as default was 60 sec. But it took more than 1’30" sec to scan, so it nearly scans all the time. As I wish to share some 80Gb of thousand files, I don’t think syncthing could do the job if the nas scans all day long. Running 24/7 will kill the 4 bay disks to early. I think there is a real need of scan optimization, not rescanning things that didn’t change according to same size, same attributs, same time stamps; or not rescanning if not asked (event / button trigered). So was there something wrong ? I have had an issue twice of sync stuck close to 100% because of hidden files not treated, (reported in another post). So was it a side effect of this problem or is it normal ? I 'm not sure because I went to sleep :wink: Thanks anyway.

2

That is obviously already the case, as you mentioned it would be ridiculous to hash all files every 60s for a multitude of reason. Probably on your nas the initial scan hasn’t finished yet (in which case rescan intervals don’t matter, it will just scan until finished and then “rescan”, which means just checking metadata, no hashing if nothing changed).

2 Likes
3

Thanks for your answer ! I played with a stable directory adding and removind a single 700Mb file. The problem is that it keeps scanning the incoming file, reducing the ressource used to copy which increases back the scan time… As workaround, I 've found I can either pause the directory shared until copy is over or set scan freq to 0 which stops it (instead of continuus scan as I first thought, but negative value forbidden, I tried 0). Is there a parameter to set avoid scanning of incoming files ? I 've seen syncthing can “see” the difference between an incoming and a full finished file because the number of files in the directory changes only when the file is fully finished. There is the “disable temp indexes” box which I didn’t try because I wasn’t sure it is meant therefore and because while adding the file, it doesn’t use a temp file; it uses instead the final name and you see the size growing… Thanks :slight_smile:

4

Well, now I’ve tried and it doesn’t change a thing in this case.

5

Possible workaround: upload/copy in with a .tmp extension, then rename when complete. Ignore .tmp files in Syncthing. Or increase the rescan interval.

1 Like
6

I 've tried to forbid everything except .MyFilesExtension : (?i)!*.MyFilesExtension ; ** It’s solves the problem of tmp,temp, but not when I add files using the local OS which creates growing files with allready their final names. Until now, either I copy them to another folder then move them in, either I set scan freq to 0 while copying. Thanks.