Concurrency

Yes, for my situation I would basically like to have maxConcurrentScans and maxConcurrentSyncs instead of having the two merged into one maxFolderConcurrency.

In my case, syncing at low speeds, e.g. 25-100 KB/s, does not thrash the drives at all, but as such it still prevents the other folders from even starting scanning/syncing.

Scanning, on the other hand, always thrashes the drives, so being able to limit it separately with maxConcurrentScans before 1.4.0 was very convenient.

1 Like

I am very sorry for bumping the thread up, but is there any chance that something will be done with this issue?

Right now, the default maxFolderConcurrency is straight unusable with my configuration. I need to specifically set it to -1 (or other very high number), because otherwise a few of my folders, constantly syncing at slow speeds, completely block scanning and synchronisation of the other folders. On single- or dual-core machines, the other folders basically never sync.

I personally believe that scanning and syncing are two completely different things. Why? Scanning depends on the local hardware performance, while syncing is both about the hardware and the network involved.

maxFolderConcurrency lumps scanning (hardware) and syncing (hardware and network) together into one setting, but does not consider network performance at all.

Edit: I apologise again if I sound whiny, but with my growing number of folders, I do need to limit scanning in some way, just not to peg the hardware, but without limiting syncing at the same time (which is going on at slow speeds with no impact on the hardware). If this is to remain impossible as of v.1.4.0 and later, then I will likely need to modify the source code either to completely revert the relevant commit, or to modify it to control scanning and syncing separately.

I think the best would be to do what you suggest,

then file this as a pull request. That way everyone benefits.

Perhaps the most general model is a top level setting like Concurrency limit: [some number] (i.e., what we have today) and then folder level settings (i.e., per each folder) Concurrency limit applies to: [x] scanning, [x] syncing. By default those would be checked, and the behavior would be identical to current. Uncheck either to modify for any given folder. In your case, I guess uncheck scanning everywhere. For others, maybe uncheck everything on a specific folder that is on its own hardware, leave the others at default, etc.

2 Likes

Thank you for sharing your thoughts. I will see what I can do with my limited skills and then write back with the results.

I doubt that I will be able to implement the general model though. I should probably be able to make some changes to keep the two as independent settings. However, there is a problem of naming. maxFolderConcurrency is a broad concept, while the old maxConcurrentScans (and a possible anological maxConcurrentSyncs) clearly indicates its limited role.

If I try not to revert the commit and simply go on with modifications on top of the current state, then I have little idea on how the settings should be named at the end. If it was for my personal use only, I would simply rename maxFolderConcurrency to maxConcurrentSyncs limiting it to control only syncing, and then bring back the old maxConcurrentScans (perhaps limited to CPU core count by default) to control scanning. This will likely cause problems with migration from the current config though.