Feature Request: Progress on large individual files in overview

" … blocks are settled after the scan …"

So the reason that I am seeing no improvement whatsoever after activating big blocks on all nodes, is that I have to delete all scan indexes and re-scan all files?

Assuming that is correct, maybe this information should be added to the documentation.

Also - do I need to delete indexes (database) on all synced nodes or just the sender node?

Thanks, Andrej

Or “touch” a file so that it is rescanned due to changed modification time. The large blocks setting determines what happens when the file is hashed, so you just need to trigger a rehash.

That said, the “min blocks” thing defaults to 10. The block size for a 3.2 GB file should be 2 MiB. So there should be an update sent every 20 MiB or so.


FWIW, this works for me. Running the sending side with -verbose I see

[I6KAH] 08:03:05 VERBOSE: RemoteDownloadProgress ... "state":map[string]int{"testfile":9}}}
[I6KAH] 08:03:06 VERBOSE: Completion for folder "default" on device MRIW7OK-NETT3M4-N6SBWME-N25O76W-YJKVXPH-FUMQJ3S-P57B74J-GBITBAC is 0.8549742109219327%

and

[I6KAH] 08:03:45 VERBOSE: RemoteDownloadProgress... "state":map[string]int{"testfile":10}}}
[I6KAH] 08:03:46 VERBOSE: Completion for folder "default" on device MRIW7OK-NETT3M4-N6SBWME-N25O76W-YJKVXPH-FUMQJ3S-P57B74J-GBITBAC is 0.9499713454688141%

That state thing is the number of blocks downloaded for the file in question. These are large blocks, and the updates are sent for each block despite the above because some time limit is exceeded.

The remote completion is also getting incremented (on the sending side):

(1% at this point; I had restarted to enable -verbose so it had downloaded ~40 MiB with the 24 MiB being in just this session.)

However this isn’t taken into account on the receiving side; it is 0% until the file is completed and inserted into the index. Is it this we are talking about?

1 Like

I don’t appear to be getting a progress update on either side.

Can you post your configs please?

I’m guessing the sending side might be send-only?

Yes, one side is send only.

receive_config.xml (4.0 KB) send_config.xml (4.1 KB)

Right. So incoming progress updates are dropped for send only folders. It’s been a long time and the PR that added this went on for a year and through multiple code review systems nonetheless… I think the reasoning is that we track these blocks (in RAM, at cost) in order to be able to download them. Send only folders do not download, so there is no need to do the bookkeeping. Hence, the side effect that is more granular progress status doesn’t happen.

This may be a feature or a bug, I’m not sure.

It’s clearly surprising, and surprising is bad so I’m leaning towards bug.

1 Like

Ok, that makes sense. I have quite a few folders that are send only, but only a couple with large files (and recently large blocks), so I hadn’t noticed it before. The majority of send only folders that I have update presumably because they increment the percentage after each small file.

Good news is the feature works as intended, even with large blocks. But maybe we should enable it for all folder types regardless. There is still an off switch in disableTempIndexes for those who need the performance…

1 Like

Sounds like a good idea. Could the option be presented in the UI when selecting folder type? The user could choose when the folder is created.

I don’t think this is something we should bother the user with at folder creation. It should just work, and temp indexes can be disabled in the rare cases where it matters enough for someone to look for that solution…

Fair enough. :slight_smile: