Hashing use SHA1 or 256 and Intel CPU instructions

I’m not sure which hashing algorithm sync thing currently uses, but it would be possible to improve the performance substantially on modern CPU’s by making use of special instruction sets.

Assuming that SHA1 or SHA256 are used or can be used for the verification of the blocks of files that are being synched then this could be accelerated by making use of the special instruction set created for this: https://en.wikipedia.org/wiki/Intel_SHA_extensions

The instruction set is supported by the Intel sandy bridge (2011) and later CPU’s. I’m not sure if there are any AMD CPU’s that support it as this time, but I believe support for it can be added to them since the instruction set is based on the SSE instructions.

We tried this, it’s a bit faster indeed, but so far only exists as a patched Go standard library:

A bigger win would be to change to a cheaper hash function entirely: