Doing first full sync of my home folder. Earlier I had done a few folders inside my home folder and that went by fast. However, because I have files too, I changed to do the entire home folder and then use ignore patterns to get files and folders I need.
Already went over the suggestions in the documentation, but in general the speed of this sync is looking suspiciously slow. Specially since the syncs without ignore patterns were fine. Are there any known issues with slow downs when using ignore patterns? I have 15 lines in the ignore pattern.
Checked CPU and memory on both machines and no issues.
[edit] Just saw this on the terminal I ran syncthing from
[KVVP3] 13:07:24 INFO: Joined relay relay://162.202.207.121:22067
[KVVP3] 13:34:10 INFO: Folder “francisco-home” (qc9mr-r3gzh) isn’t making progress. Pausing puller for 1m0s.
[KVVP3] 13:34:15 INFO: Folder “francisco-home” (qc9mr-r3gzh) isn’t making progress. Pausing puller for 1m0s.
[KVVP3] 13:34:41 INFO: Folder “francisco-home” (qc9mr-r3gzh) isn’t making progress. Pausing puller for 1m0s.
[KVVP3] 13:34:46 INFO: Folder “francisco-home” (qc9mr-r3gzh) isn’t making progress. Pausing puller for 1m0s.
There is large overhead per file, so small files are generally much slower to sync.
If you have errors in the console, there should be a message with the issue somewhere around there, or you might have “Failed items” in the UI, which when clicked explains the reasons why it failed to sync.
[KVVP3] 14:08:38 INFO: Joined relay relay://107.150.6.122:8080
[KVVP3] 14:08:38 INFO: Puller (folder “francisco-home” (qc9mr-r3gzh), file “dev/django/uvenv/lib/python3.6/site-packages/pytz/zoneinfo/Pacific/Palau”): finisher: pull: peers who had this file went away, or the file has changed while syncing. will retry later
There is nothing going on in the source machine. So can’t be about files getting updated.
If safe, thinking of pausing the sync and using rsync for first pass.
I wouldn’t expect wonders from rsync, if any speedup at all. The actual transfer is usually not the bottleneck on initial sync, looking up, indexing and comparing all files is. The peers wo had this file went away error seems weird. Then again, it’s initial sync with lots of files, that is just slow - let it stew and once finished it will be a lot faster. And maybe v0.14.49 will make that part faster too (maybe not so much though).
Has there been any thought/consideration on having a fast mode which does lesser checks? Basically check only name, size and date. Something one would use primarily only on initial syncs of new large folders. In essence for cases where the target machine(s) won’t even have the files.
Checking only name, size and date makes sense only when scanning. When downloading you have to insert these values into the database, which is much slower than downloading the file.
We can’t speed up the inserts, and we can’t just assume that X gave us the right content, we still have to compare it.
I suggest you go read the code, understand how it works, before making suggestions.
Actually, my answer might have been completely irrelevant: If you have data on only one device, only that device does the indexing and then sends everything over - no comparing/resolving of file infos generated on both devices at the same time happening.
I am not sure how that sounded on your head as you were writing it, but allow me to let you know that on the receiving end it sounds condescending at best.
Simon, I want to thank you for all your feedback and answers today. However after seeing the title "maintainer" in someone who writes the type of replies I just got, I think I have seen all I need to make me decide against using this software. Sure there are lots of great points to Syncthing, but none will ever be enough to make me want to deal with such behavior.
Don’t take this personally, but as every open source project we have an influx of suggestions, starting from adding rewriting everything in Java because they can’t read go, ending with adding a built in music player.
I think people should understood the problem at hand and the complexity of the implementation before their suggestions have merit, and explaining why people’s ideas and suggestions will not work requires energy.
I did exlain why your suggestion will not work, yet to refrain from further “why’s” I suggested you do your homework.
