Please test in-filesystem copies

The 1.8.0 release candidate has a new feature where it can copy data by asking the file system to do it directly, instead of first reading and then writing the data itself. Per the feature description:

This can be used to optimise copies on network filesystems, improve speed of large copies or clone the data using copy-on-write functionality if the underlying filesystem supports it.

The details of what is supported on which OS and filesystem are somewhat hairy. However, if you’re running Linux with BTRFS or ZFS or (maybe?) ZFS, or ZFS on FreeBSD, or Windows with ReFS, CsvFS or a modern CIFS file share, you might see improved performance and/or reduced space usage.

However this is not well tested on all possible systems, so we’d greatly appreciate some help in seeing what works and what doesn’t.

:warning: :warning: :warning:

This is not without risk. Please do testing on a less critical system, and only with a good backup of your data available. Minimizing that risk for regular users is why we’re asking intrepid ones to do testing. :slight_smile:

:warning: :warning: :warning:

Enabling the feature requires setting the appropriate value for the advanced folder option copyRangeMethod. The documentation is here:

https://docs.syncthing.net/advanced/folder-copyrangemethod.html

2 Likes

This looks very exciting - woop!

Is there a particular way that you would like success/failure to be determined?

Yehaaa thanks, I’ll shoot my data in :+1:

If it doesn’t work, it will error out most likely.

I guess the way to test this is:

  1. Observe no errors
  2. Not lose data
  3. Maybe verify that some blocks were reused between files by checking “total size of data” vs “total size on disk” equivalents.

For network shares I guess verify the speed of 1GB large file before and after or something.

And as far as I understand, savings will happen in two cases:

  • When using versioning, the copy might be copy-on-write so used size is smaller than the whole file
  • When syncing a file and creating a temporary copy of it, blocks will be reused from the existing file

In both cases the amount of read/write I/O should also be less so things might be faster in general.

Or reused from existing file, namely you have file A and make a copy of it as file B, it should reference blocks from file A and not take up any more space.

1 Like