Why is data being sent during rescan?

I’m running ST (v0.12.3 - 64Bit) on my FreeBSD server (see upload below for screenshot of setup).

I’m monitoring a 10s running mean of KB/s of data being sent and received. I have paused the other devices and I have not added any folders/files to the folder being watched.

When I change the rescan interval from 10s to 0s, the “write bandwidth” drops from ~100Kb/s to 0Kb/s:

When I change the rescan interval to 60s, this appears every 60 seconds, so it is clearly the scan that is sending something. Why does this happen? What is being sent and to what? Both other devices are paused, everything is up to date and no files have been added/changed for days.



Did you test it with the WebUI open or closed? If it was open, than the traffic was probably the scanning status send to the GUI.

You’re talking about network sends, but monitoring disk writes? Does your pool or filesystem have noatime set (otherwise you’ll get writes for every read)?

There’s a neat dtrace script to show zfs read/writes taking more than x ms, which should work on FreeBSD too I think:

Running that with 0 as threshold should show you all writes in real time, so you can determine what’s going on.

Yes without the GUI. It’s still writing something to disk at every scan.

I’ve been monitoring network traffic with vnstat, and the total amounted to ~12MiB in ~4hours. Does the “K” in my bash screenshot above mean KB/s (this output was achieved with the “zpool iostat -v data_pool 10” command)? If that is the case and it is also network traffic then “vnstat” would have returned 12MiB after 3 minutes, not 4 hours… So i’m guessing it’s not network traffic but rather something being written to disc at every ST folder scan. I’m just not sure where and what it is.

It’s maintaining a database, things such as when is the last time we’ve seen a device, etc, whats the last file we’ve synced etc. Why are you concerned it’s writing to disk?

Do you know where it’s writing? I find it odd that it’s writing almost 1MB every scan when the directory content is unchanged. 1MB is not negligible. But if ST is meant to be doing that, then so be it.

Sorry, but why is 1MB not negligible?

I assume it’s 1MB in metadata? Not actual content being synced. That seems high to me but you’d know best…

No, it’s 1MB written to disk, which is genuinely negligible.

Okay. Next is to find out where it’s being written to and then i’m satisfied. Thanks for the help.

I’d say that 1MB per scan is quite significant. Because, if the scan interval is 1s, or even 10s, this would imply a constant load on the filesystem (and disk), which generally isn’t healthy.

Well I don’t think it’s constant, or relative to the scan interval. I don’t think that maintaining a database is unhealthy in the general case, 1MB/s once every 60s is negligible.

What are you basing your claims on?

Sorry what claims are you referring to? The claim, that a constant load on any filesystem should be avoided if possible?

Perhaps I misunderstood something. From this thread I understood that the same database is updated at each scan. If, in the most aggressive of cases, this happens every 1 second, and it writes 1mb at each second, then this is certainly significant. Where have I gone wrong?

you say ‘I don’t think’ twice, followed by some statements. Let’s avoid speculation. :smiley:

I think anyone doing a scan every second probably has more resource-usage issues to worry about than 1MB of disk activity.

If you choose to scan every second, I don’t think it’s a miracle that we write 1MB every second. By the end of the day you asked for it.

Perhaps but not necessarily. With Syncthing perhaps, but some similar tools are based on inotify to detect differences, which allows for almost arbitrarily small time increments at no overhead.

molofishy how large is the content of your syncthing folders?

Yeah, and they’ve all got backoffs which result in far less than one update every second.

Only one folder is synced and it’s currently 69MB