How to sync files that are open with read-only access?

[oskar]

@gadget

WSL 2 can definitely run Syncthing for Linux, but then Samba, NFS, or some other network service must be used to provide access to the storage volumes inside the VM (ext4 filesystem by default). At that point, it’d be like maintaining a mini Linux server on each Windows workstation.

Probably be easier to just switch to Linux.

Would love to be able to run only linux. Unfortunately have a few programs that require windows

WSL 2 could be a viable workaround then probably. But would have a performance and maintenance penalty. Not worth it in my case.

And definitely don’t have the skills/time to fork syncthing and implement a in-place file update on windows (that would be required to make it multi-user environment compatible).

[oskar]

@gadget

It turns out to be more difficult than I’d originally thought. In Windows on a NTFS volume, if a file is open read-only but allows shared writing by another process, data can be inserted and replaced. However, the file can only remain the same size or grow, it cannot shrink.

Over multiple tests the file got corrupted while attempting to delete data. After the file handle was released a review of the file contents didn’t match what had been shown in the application window during the write process (the disturbing part was there was no warning from Windows).

So while it’s possible to replace the contents of an open file, the final results can be indeterminate if the updated file is smaller than the file it’s replacing.

Oh wow, I totally missed this part of your reply!

That is some super fast testing! Hats off!

Slight problem if file size can’t be reduced! 50% chance the newer file will be smaller than the existing one (ok maybe 25% in reality). So can then not be updated in place. Thumbs down for windows on this aspect. I would assume this corruption is due to the file management system caching file data in ram, and there being a bug or functionality restraint in how that is implemented between processes.

I can only imagine the complexity involved in getting around that issue/bug in windows! Clearly not the job of a “external” program to manage anyway.

Is there some method to do a bug report to microsoft?

[gadget]

@oskar

Slight problem if file size can’t be reduced! 50% chance the newer file will be smaller than the existing one (ok maybe 25% in reality). So can then not be updated in place. Thumbs down for windows on this aspect. I would assume this corruption is due to the file management system caching file data in ram, and there being a bug or functionality restraint in how that is implemented between processes.

In-memory caching, temporary files and perhaps other things make successfully overwriting an open file hit or miss on Windows.

I can only imagine the complexity involved in getting around that issue/bug in windows! Clearly not the job of a “external” program to manage anyway.

Is there some method to do a bug report to microsoft?

Seems like it’s more of a deliberate design decision rather than a bug.

I was curious if a copy-on-write filesystem would work, so after a bit of searching I stumbled across WinBtrfs, a native btrfs driver for Windows. It solves multiple issues but is still constrained by Windows not allowing renaming of files with open file handles.

[oskar]

@gadget

I was curious if a copy-on-write filesystem would work, so after a bit of searching I stumbled across WinBtrfs, a native btrfs driver for Windows. It solves multiple issues but is still constrained by Windows not allowing renaming of files with open file handles.

Good find! Seems like it’s too early for that to be a daily driver yet. With defrag etc on the todo list. But very promising!

If it doesn’t suffer from data corruption during writing in place, it might be a long term solution to be able to have syncthing multi-process/collaboration compatible on windows sometime in the future!

[calmh (Jakob Borg)]

In-place modification of files by Syncthing is likely never going to happen, regardless of the filesystem on Windows, so I wouldn’t hold my breath.