Is it possible for Syncthing not to lock files in Windows when hashing them?

imsodin · July 22, 2021, 1:50pm

Relevant proposals, both declined:

github.com/golang/go

proposal: os: Create/Open/OpenFile() set FILE_SHARE_DELETE on windows

opened 07:17PM - 16 May 19 UTC

closed 05:09PM - 09 Oct 19 UTC

networkimprov

FrozenDueToAge OS-Windows Proposal Proposal-FinalCommentPeriod release-blocker

On Linux & MacOS we can write this; on Windows it fails with a "sharing violatio…n": ``` path := "delete-after-open" fd, err := os.OpenFile(path, os.O_RDWR|os.O_CREATE, 0600) if err != nil { ... } err = os.Remove(path) // or os.Rename(path, path+"2") if err != nil { ... } fd.Close() ``` If you develop on Windows and deploy to Linux etc, and your code relies on this _undocumented_ GOOS=windows behavior of os.Rename() & .Remove(), it is broken and perhaps vulnerable. Note that package "os" has _fifteen mentions_ of other Windows-specific behavior. To fix this, syscall.Open() at https://golang.org/src/syscall/syscall_windows.go#L272 needs `sharemode |= FILE_SHARE_DELETE` **Microsoft** recommends this be made the default: https://github.com/golang/go/issues/32088#issuecomment-502850674 **Rust** made it a default: https://doc.rust-lang.org/std/os/windows/fs/trait.OpenOptionsExt.html **Mingw-w64** made it a default seven years ago: https://sourceforge.net/p/mingw-w64/code/HEAD/tree/stable/v3.x/mingw-w64-headers/include/ntdef.h#l858 **Erlang** made it a default over six years ago: erlang/otp@0e02f48 **Python** couldn't adopt it due to limitations of MSVC runtime: https://bugs.python.org/issue15244 ~Therefore syscall.Open() should *use file_share_delete by default*, and syscall should provide both: a) a global switch to disable it (for any existing apps that rely on its absence), and b) a flag for use with os.OpenFile() to disable it on a specific file handle.~ Update after https://github.com/golang/go/issues/32088#issuecomment-532784947 by @rsc: a) os.Create/Open/OpenFile() should *always enable file_share_delete* on Windows, b) syscall.Open() on Windows should accept a flag which enables file_share_delete, and c) syscall on Windows should export a constant for the new flag. The docs for os.Remove() should also note that to reuse a filename after deleting an open file on Windows, one must do: `os.Rename(path, unique_path); os.Remove(unique_path)`. Win API docs https://docs.microsoft.com/en-us/windows/desktop/api/fileapi/nf-fileapi-deletefilea https://docs.microsoft.com/en-us/windows/desktop/api/fileapi/nf-fileapi-createfilea If there is a reason not to do this, it should be documented in os.Remove() & .Rename(). cc @alexbrainman @gopherbot add OS-Windows

github.com/golang/go

proposal: syscall: define Windows O_ALLOW_DELETE for use in os.OpenFile

opened 08:26PM - 03 Oct 19 UTC

closed 06:08PM - 15 Jan 20 UTC

rsc

FrozenDueToAge Proposal

The discussion on #32088 has so far been unable to reach consensus on setting FI…LE\_SHARE\_DELETE unconditionally in os.OpenFile. Right now it looks likely to be declined, possibly revisited in a few years. In the interim, it is more difficult on Windows than on Unix to pass an extra flag to open a file with package syscall. syscall.Open takes fake O\_\* flags, not actual Windows permission bits. It wraps syscall.CreateFile, but it does a lot of argument preparation before making that call: // package syscall func Open(path string, mode int, perm uint32) (fd Handle, err error) { if len(path) == 0 { return InvalidHandle, ERROR_FILE_NOT_FOUND } pathp, err := UTF16PtrFromString(path) if err != nil { return InvalidHandle, err } var access uint32 switch mode & (O_RDONLY | O_WRONLY | O_RDWR) { case O_RDONLY: access = GENERIC_READ case O_WRONLY: access = GENERIC_WRITE case O_RDWR: access = GENERIC_READ | GENERIC_WRITE } if mode&O_CREAT != 0 { access |= GENERIC_WRITE } if mode&O_APPEND != 0 { access &^= GENERIC_WRITE access |= FILE_APPEND_DATA } sharemode := uint32(FILE_SHARE_READ | FILE_SHARE_WRITE) var sa *SecurityAttributes if mode&O_CLOEXEC == 0 { sa = makeInheritSa() } var createmode uint32 switch { case mode&(O_CREAT|O_EXCL) == (O_CREAT | O_EXCL): createmode = CREATE_NEW case mode&(O_CREAT|O_TRUNC) == (O_CREAT | O_TRUNC): createmode = CREATE_ALWAYS case mode&O_CREAT == O_CREAT: createmode = OPEN_ALWAYS case mode&O_TRUNC == O_TRUNC: createmode = TRUNCATE_EXISTING default: createmode = OPEN_EXISTING } h, e := CreateFile(pathp, access, sharemode, sa, createmode, FILE_ATTRIBUTE_NORMAL, 0) return h, e } A direct call to syscall.CreateFile would have to duplicate all that code. A direct call to syscall.Open would still lose out on the adjustments made inside os.Open, most importantly the call to fixLongPath: // package os func openFile(name string, flag int, perm FileMode) (file *File, err error) { r, e := syscall.Open(fixLongPath(name), flag|syscall.O_CLOEXEC, syscallMode(perm)) if e != nil { return nil, e } return newFile(r, name, "file"), nil } If package syscall on Windows defined a bit O\_ALLOW\_DELETE (maybe 0x100000), then syscall.Open could convert that bit into FILE\_SHARE\_DELETE. Then it would be easy for calls to either syscall.Open or os.OpenFile to cause the underlying call to use FILE\_SHARE\_DELETE if they really needed it. This proposal assumes #32088 is declined. It should be considered withdrawn if #32088 is accepted.