Reserved filenames/symlinks/case-sensitive syncing on Windows

Here’s a proof of concept you all might enjoy seeing. I used this folder dialog:

(built from merge/9539-9623-9266-tbd1-and-8002), to sync all files in https://github.com/rasa/git-os-test from a Linux system, to a Windows 11 system (except for ‘a-backslash\’).

Here’s a listing of the synced directory in a Git-Bash shell on Windows:

The four feature branches that comprise this work are:

  1. feature/9539-encoderfs
  2. feature/9623-allow-reserved
  3. feature/tbd1-casesense
  4. feat/8002-enable-symlinks

The fifth branch, feature/9266-integrationtests, tests two nodes running various configurations of encoders. The test run is documented at: https://github.com/rasa/syncthing/wiki/go-run-build.go-integration-on-merge-9539-9623-and-9266-branch

If any Windows users are intrepid enough, there are unsigned binaries at the bottom of https://github.com/rasa/syncthing/actions/runs/23691665992

I have used encoder-enabled builds on my personal cluster for a couple of years now, and haven’t seen any issues. Let me know if anyone is keen enough to try out these builds too.

If you encounter any issues, please submit them at https://github.com/rasa/syncthing/issues.

If you do post on the forum as well, please consider replying to this topic, instead of creating a new post.

1 Like

Looks interesting :wink:.

I’d consider adding all that as a separate section under the “Advanced” tab though (especially since some of these are very specialised and shouldn’t really be used except for very specific needs :sweat_smile:).

1 Like

The encoder is used on one side only yes?

How are the files with illegal characters encoded? (Share an example?)

If I rename a source file that has been encoded on the destination I presume the destination filename becomes unencoded assuming the offending characters are removed.

what happens if I rename the encoded filename on the destination side?

I have a few directories where this feature would be helpful and I would probably use it.

The encoding method is documented at

https://github.com/syncthing/syncthing/issues/9539

Encoding is invisible to the other nodes. The only issues are with using encoded files in Windows Explorer and on the command line.

OK, I will move the settings to the Advanced tab.

I tried adding

ng-if="version.os === 'windows'"

to only show the tab on Windows clients, but it doesn’t appear to work. Any ideas?

in theory, FAT filesystems can be used in linux and os x, right? :slight_smile:

@mraneri to answer your questions specifically:

  1. Yes the encoder is only on one side. The other nodes don’t seeing anything different on their end. In fact, they don’t even know a node (or more properly: folder) is an encoding node.
  2. The FAT encoder uses the same encoding scheme used by WSL, Git-Bash, etc. So the filename a-pipe| is stored on disk as a-pipe\xf07c. See issue #9539 for details.
  3. Renames/deletes work as expected as encoded filenames are decoded before being transmitted on the wire.

As I mentioned, users may have issues with handling encoded filenames in Windows Explorer and on the command line. But Windows 11 appears much more forgiving than Windows 10 was in handling these encoded filenames.

Yes, the FAT encoder is for any FAT-style partition, no matter the OS, so NTFS, exFAT, FAT32, and ReFS formatted partitions would all benefit from this encoder.

Per @tomasz86 ‘s suggestion, I will move the Filename Encoder setting to the Advanced tab. The other settings, I would still like to hide for non-Windows users, but I’m not sure how.

For another proof of concept, I added an “Rclone” encoder. The character mapping is defined in https://rclone.org/local/#restricted-characters

Here’s a screenshot in Windows Explorer:

2 Likes