It is my understanding that Syncthing uses batching when transferring files.
Is it possible to limit the batch size on a Syncthing transfer?
From the Syncthing config “order” Documentation:
“Note that the scanned files are sent in batches and the sorting is applied only to the already discovered files.”
The problem I am trying to solve:
I need a true LIFO transfer of files.
Example from my testing:
If I drop 100 files into Syncthing they will be LIFO, but it will be batched together and then new files will need to wait for that original 100 file batch to complete transferring before the new files are batched and begin transferring.
My thought is, if I were to limit the batch size to a small number, I could achieve a true LIFO transfer every time.
If the intention is that any new change should jump to the head of the queue while there are already changes being processed… I don’t think this is currently possible.
In general the queue control is best effort at best, as there is no “I am now done sending changes”, as its a continuous process, so the side that is downloading at some point has to give up waiting and start working on what it has been informed about so far. At the moment this happens as soon as it has received the info about the first batch of changes. At this points it builds the queue from whatever it knows its outstanding, and doesn’t look at any new arrivals until it finished going through that queue.
Otherwise, if the other side is scanning and sending us batches of millions small files, we’d either have to wait an undefined about of time before we start doing any work, or have to sort millions of entries after every file we’ve downloaded, to ensure that latest/smallest/whatever invariant you pick is still maintained, which simply does not scale beyond a certain number of files. In might net even be feasible/undesirable (in terms of memory use) to store info about the millions of files in memory to sort them as per invariant.