In the Block Exchange Protocol it states:
File data is described and transferred in units of blocks, each being 128 KiB (131072 bytes) in size.
Was there a reason for this choice in block size?
I understand that the block size directly impacts the index size by the number of blocks needed to define a file (larger the block size the lower the index size and vice versa), but was more curious if someone had done some benchmarks indicating this size as a good balance of block size to index size.
It felt like a reasonable trade off between data to shuffle and metadata to track.
It felt like a reasonable trade off between data to shuffle and metadata to track.
That’s fine. I’m sure there’s some optimal number out there. But “optimal” is probably subjective based on the network scenario (i.e. LAN, mobile etc) and quantity of files to track. I may do some number crunching and let you know what I find.
What about the choice of Kibibyte instead of Kilobyte?
Tradition? I’m not quite sure what you’re asking. As I think you’re implying, there’s usually no relevant difference between a 128000 bytes block and a 131072 bytes block - sometimes the latter may be more effective though, by being a multiple of the file system block size.
Potentially we may introduce a variable, larger, block size to handle large files more efficiently.