syncthing duplicating vdo storage volume appears to be delayed by other processes targeting volume

[averyfreeman]

Hi, just wondering if you have done any testing with the storage driver dm-vdo that’s included as an in-tree module as of kernel version 6.9? Here’s the release announcement: Linux_6.9 - Linux Kernel Newbies

Here’s some pretty comprehensive info about the technology - it pre-dates the announcement as it’s been an add-on for RedHat since ~2018, but it should still apply: Chapter 1. Introduction to VDO on LVM | Red Hat Product Documentation

Note: There are outdated usage instructions floating around. The API for managing vdo volumes is provided through the lvm2 package now, specifically the lvcreate binary as command-line arguments. I think RH made that design decision to ease adoption, and because vdo volumes are technologically related to lvm thin volumes (plus, cleaner to fold management into a command already in use to promote adoption, than to add a bunch of new commands people not paying attention will ever know about).

The big lead-in is to say, it’s new, it’s unique, it’s interesting, and since it saves space for people without much of a penalty, it’s likely to be something you guys encounter. I’m running it on Fedora 40 right now for my entire /home volume, and it’s been going well so far.

I haven’t run Syncthing in several years, so I’m getting used to it all over again, but so far I’ve noticed my computer using vdo will prevent syncthing from moving past the scanning or preparing to sync modes indefinitely until storage-related process like fstrim (aka "discard command") or the vdo compression finish running. Is this typical behavior of volumes besides vdo, as well? I guess it would make sense if the files or blocks of the volume are actually still changing.

It seems like my other 2 computers I’m syncing with took maybe 20-30 min tops to move past the scanning and preparing to sync stages, and began sending files, while the vdo computer has been there for a few hours now.

I need to run some logs so I can actually send you something substantive besides noticing which word is listed in the web GUI, but anyway, that’s what I’ve got so far… will report back when I know more - ttys

[calmh (Jakob Borg)]

I don’t know anything about vdo but, yes, Syncthing will not successfully sync a file while it’s also changing. Mounted file-backed volumes will typically suffer from that.

[averyfreeman]

Thanks for the heads-up. Speaking of file-backed volumes, what are some examples of those?

[calmh (Jakob Borg)]

Most virtual machines and encrypted volume things

[averyfreeman]

Very helpful, thanks - surprisingly, I’ve never heard them described that way.

On that subject, I’ve been creating logical volumes for each one of my VMs, so instead of a file, each one has an lv (they’re self-contained multi-partitioned drives, formatted ‘raw’). Does that mean I should have less of a penalty for backing up my VMs with syncthing than typical .qcow2 VMs? (just curious)

I can’t take responsibility for thinking of the configuration, I just noticed that Proxmox uses this setup, so I copied them because I thought it was cool.