Why is data being sent during rescan?

My bad. My apologies. :grinning: But potential bugs should be taken more seriously imo.

So far, all Iā€™ve seen is a Syncthing developer saying ā€œIā€™d expect that behaviour. Thatā€™s fineā€.

Bear in mind that Syncthing is a free open-source project. The developers work on it in their free time: lunch breaks, evenings, weekends, even Christmas Day if this thread is any indication. Thereā€™s an issue list on GitHub with 149 open issues on it. The developers have to prioritise: fix the bugs which affects lots of people, or affect people badly, as well as adding new features which benefit the community.

Behaviour which causes 1MB to be written to disk every scan (even if it is in error) has a very minor severity, and will only affect the very small (nonexistent?) proportion of people doing one scan every second or so. Iā€™m not surprised itā€™s not being treated as serious :smile:

EDIT: Letā€™s not forget the fact that youā€™re coming across as quite confrontational, so someoneā€™s unlikely to look into it as a personal favour.

It was enough to make my HD make interesting sounds through the night :grinning:

I do appreciate ST / open source. Hence the reason iā€™m here trying to improve the software by reporting a possible issue. Yes, perhaps itā€™s very low on the priority listā€¦

Have you tried using syncthing-inotify? That way youā€™d only need to rescan as often as you are paranoid about FreeBSDā€™s filesystem notification system dropping events. The rescans themselves are wasteful considering the availability of this solution, even before considering the disk I/O for the database.

Can you try adding a significantly higher amount of data to a Syncthing folder and see how that affects the disk I/O?

Also, if you consider this amount of disk I/O excessive, maybe you could try keeping the Syncthing database in RAM, or on a partition with an fstab sync interval (the ā€œcommitā€ value) longer than the default. A very long commit value is basically the same thing as keeping the partition in RAM.

This approach will have some disadvantages, but you will write to disk much less often. The main disadvantage that comes to mind when keeping the database in RAM will be a total loss of state each time Syncthing is stopped. You will lose the ā€œlogical historyā€ provided by the version vector system and your conflicts will be messier. If you simply increase the ā€œcommitā€ value, you will lose whatever state was un-synced when Syncthing stopped, for the time since last sync.

Due to that disadvantage, I donā€™t recommend you take that approach for critical data or data that changes in different places often, or for clusters whose nodes go up and down a lot (laptops and mobile devices).

I also wouldnā€™t recommend these approaches for a lot of data ā€” it would be very expensive to rescan each time Syncthing starts, but for such a small amount (69 megabytes), the rescan will finish quickly.

Of course, my final recommendation is to just let the machines work for you and not worry about it so much :smile:

Also, I noticed the same disk I/O as you. I believe it is expected, although a Syncthing developer should weigh in on that (I believe Audrius already has). But the database does not grow in size, so I donā€™t mind.

I think next time you should spend more time defining your problem better, as it seems its more about the HDD waking up, rather than the fact that we are writing 1MB to the disk every rescan interval (which is perfectly reasonable and makes you look very strange when you have so much dissatisfaction about it)

I donā€™t think you can avoid waking up the disk if you ask us to read off the disk (scan) every second, so there is no bug here, there are only wrong expectations by the user.

Also whether we write 0MB, 1MB or 10MB is also meaningless as that operation completes almost immediatelty and causes no noticable strain on the disk.

There is a bug report already about moving the marker check further down the pull loop which should prevent hdd wakes in some cases, but I donā€™t think we can avoid waking up the HDD during rescans, unless you use inotify as someone already suggested.

Wow, long thread, not much hard info. I guess you didnā€™t try the script I posted above? I donā€™t actually think we should do any disk writes when just scanning with no devices connected (unless there are changes discovered). But there are lots of other scenarios that would cause writes. And we might obviously wake up the disks to perform the scan.

I tried for myself, and there are no writes on my system by the syncthing process when running scans on four folders totaling 140k files (as forced by the ā€œRescan allā€ button). There are some occasional database writes once a minute, when other devices are online, as we record the fact that they are still online.

Hmm, and I take it youā€™re running v0.12.3 -64bit on BSD.

@lfam, do you see writes on every scan or occasionally as @calmh mentions?

No, this was 0.12.9 on Solaris. Why are you running 0.12.3? Donā€™t do that, itā€™s old. But nevertheless, the behavior (code) is the same for the relevant parts.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.