Block size default

Hi all,

I use Syncthing mainly for the transfer of huge MXF files as well as tons of small audio MXFs, so both extremes of filesize, and was looking at increasing the minimum block size to 2MB to hopefully improve performance on our low level hardware.

I’ve looked at some other posts including the following from calmh

“One of the reasons I would not recommend it is that Syncthing with a different blocksize is a fundamentally different software compared to standard Syncthing. It’s not protocol compatible, the database format is different, and we can’t support it.”

Totally understand this, but my main question is: Can a normal build of Syncthing connect and co-exist with one of a different blocksize or will this point-blank not work? If it does work… can the normal build be replaced with the bigblock build and still work with the former’s database? I assume that any new hashed files from that point on will be in min 2MB blocks and anything already indexed will stay as 128Kib min block?

Thanks! Jon

[Edited for clarity.]

What are you trying to achieve exactly? Variable block sizes have been implemented years ago: useLargeBlocks — Syncthing documentation

Block sizes are now choosen based on filesize, so if you have large files larger blocks should be selected automatically.

Are you trying to achieve something different?

1 Like

Apologies, I realise I just wrote that on autopilot…

Data stored is a mixture of Video MXFs and Audio MXFs (tons of them)… A lot of the smaller files are being chunked into 128KiB blocks, so I wanted to up that to something higher to lessen the DB load (low powered devices with DB on same RAID as data).

1 Like

That is not something that is adjustable and is purely a product of file size.

This also could not really be adjustable as it has to match on all devices to make sense, and also in theory needs to be applied retroactively to avoid issues that we had as we moved from fixed to variable size.

1 Like

Understood, thanks for the reply.

So I could up the minblock size but only if it’s from scratch on new devices running the same modified version? I think I’ll benchmark the difference to see if its worth it, I might just be asking too much of a low spec nas!

Best, Jon

2 Likes

Pretty much, you’d have to build your own, and probably rescan everywhere.

I am fairly certain it will have no effect.

1 Like

Just to be extra clear, your large files will not be indexed with 128 KiB blocks, they’ll likely use 16 MiB blocks. You can increase the minimum block size by hacking and recompiling, but it will have no effect on your large files, and the effect on small files won’t be significant.

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.