Option to follow directory junctions / symbolic links?

A pull request created. We can bring the discussion about the code there. I need your assistance there, as it seems to me that the failed tests are not related with my code changes.

1 Like

Wow cool, I’ll have a look later because I’m curious how you did it :-). Ref. https://github.com/syncthing/syncthing/pull/6606

FYI: The functionality is implemented and merged into the master branch.

2 Likes

Thank you soooo much @calmh @xarx @AudriusButkevicius @imsodin and all others. I’m so happy it finally came true after four years of waiting and maintaining my own fork for symkink support :+1: . Great thing! :slight_smile: :rocket:

Just to avoid any confusion: The functionality added by @xarx is not following symlinks, but treating directory junctions like directories and detecting path loops.

2 Likes

With all do respect to @xarx’s contribution (which was very well executed, no qualms there), why I didn’t want this feature in the first place: This creates problems: Windows 10 User Folder Sync, v1.7.0 Attempting to open folders that do not exist and probably more.

Thus my proposal: Make following junctions an advanced config option, disable it by default and discourage using it (like ignore delete thing, which doesn’t prevent people from using it, but at least allows us to give limited support for it). Any disagreements on this @calmh and @AudriusButkevicius?

Sounds good.

Probably the few who depend on it hang on the forum and can be told of the new option, it might otherwise be a rude surprise when data behind junctions is deleted on other devices after upgrade.

I think I argued against an off switch previously, thinking one could be introduced later if it was required. I guess it is required. That should then switch off the recursion detection as well?

1 Like

Maybe leave the setting as-is for existing configs, and only apply disabled-by-default to new configs? That would minimize the risk of breaking existing setups. That would of course mean that all current setups would have it on, but I guess most people affected have already worked around any issues (with ignores).

1 Like

We have seen funny filesystems blow up with recursion errors too, so I guess it should be an off switch for both. There are also cases where even on normal filesystems we might trip over the recursion check, because for example ReFS uses 128 bits for identifying filesystem tree entries, yet OS apis only return the last 64bits…

Also, I am not sure if we want this to be an off switch, I think we want this to be an on switch instead?

Yeah no, that’s too scary.

Nice, so now we have to either keep it default enabled, letting it cause problems in the future too. Or the change is going to be more complex than the original feature. Something like exposing junctions on fs.FileInfo and scanner invalidating files behind junctions instead of marking them as deleted. Default enabled it is (unless someone is motivated to do something nice): lib/config, lib/fs: Make junction behaviour configurable (ref #6606) by imsodin · Pull Request #6907 · syncthing/syncthing · GitHub

A switch sounds good to me. For me, the junctions are a great improvement on Windows. But yes, I think the “normal user” is not able to setup ignores when he first tries out Syncthing coming from somewhere else and still needs to discover and learn.

In my opinion “default ignores” would be a nicer approach because we know that it’s mainly %userprofile%\KnownSystemJunctionFolder causing the cosmetic problem. If we could detect “user creates a new folder with %userprofile% as root” we could then ship those ignores.

Update: Not a good idea as Microsoft made those folder names localized :frowning:

So instead of filtering for the folder path on creation and auto-turning the junction follow off a default to off would be better. It could also be called “junction strategy: ignore” on the UI (like Duplicati has it).

I can’t tell my following this thread, is this feature request dead?

It’s implemented ages ago.

Is it a setting? Because I can’t find one and it’s not working for me. I am a returning Syncthing user, I used it years ago and trying to see it will work for me to replace Resilio.

Add an option to follow symlinks instead of syncing them “as is” · Issue #1776 · syncthing/syncthing (github.com)

This has been cooking for a long time and each attempted implementation has run into unanticipated difficulties. Given that and the theoretical difficulties of ensuring correctness in the presence of (followed) symlinks I’ve decided that following symlinks will not be implemented. I’m also locking this issue because I don’t want followup arguing about in the bug tracker. Anyone is welcome to argue about it on the forum, but unless the argument is backed up by a foolproof implementation it’s unlikely to have a positive outcome. Sorry, peeps.

I just realized, that is your reply @calmh. Is this thread not the same thing?

There’s a new advanced folder setting “Junctions as dirs”. The feature and it’s issues are discussed (in part) in this thread. The feature treats Windows directory junctions like a “normal” directory.

Oh, I was looking in the wrong place, I found it.

I enabled it for a test folder with a Junction Point in it, but that is not working.

Enabled for directory Temp. Added Junction Point to Temp2.

Not syncing:

image

Are you using the newest version of Syncthing? In one of the previous ones, there was a bug that required you to restart the program in order to “see” the junction points.

I just downloaded it: v1.18.0