Symlinks act differently when synced

Support
1

Hi,

I’ve encountered a strange error which I am not sure how to describe properly. I’ll try my best.

I have 3 MacOS machines and an Odroid. Odroid one is always on, so it often acts as a server. Not all machines share everything but they all have a “Shared Apps” folder in common. That “Shared Apps” folder contains non-appstore-apps that I have to update manually, and if possible, I prefer to avoid doing that on every machine separately. This way when an app is updated on a machine, it gets updated on every other one too. Neat…

The problem I’ve encountered is about symlinks, so I’ve read several forum threads, bug reports and the bounty message here but I’m not sure my problem is discussed in any of them.

Here is the situation:

I have an app called MKVToolNix. I’ve noticed it doesn’t launch on synced machines and only the original copy continues to work. So I started to invesigate. I found that in application package several libraries (.dylib) are just symlinks to their respective more up-to-date versions.

Ex: These three "libQt5Concurrent.dylib, libQt5Concurrent.5.dylib, libQt5Concurrent.5.7.dylib" files are symlinks to “libQt5Concurrent.5.7.1.dylib” file.

… and that these symlink do not work properly on synced machines. They do not point to the file they suppose to.

I’ve found a little software called “symlinks” -link: https://github.com/brandt/symlinks-

Description: Symlinks scans directories for symbolic links, identifying dangling, relative, absolute, messy, and other_fs links. It can also change absolute links to relative within a given filesystem.

I have scanned both working and synced(non-working) application folders for symlinks and found the following:

Working apps symlinks: relative: /pathToFolder…/MKVToolNix-10.0.0.app/Contents/MacOS/libs/libQt5Concurrent.5.7.dylib -> libQt5Concurrent.5.7.1.dylib relative: /pathToFolder…/MKVToolNix-10.0.0.app/Contents/MacOS/libs/libQt5Concurrent.5.dylib -> libQt5Concurrent.5.7.1.dylib relative: /pathToFolder…/MKVToolNix-10.0.0.app/Contents/MacOS/libs/libQt5Concurrent.dylib -> libQt5Concurrent.5.7.1.dylib

Synced apps symlinks: dangling: /pathToFolder…/MKVToolNix-10.0.0.app/Contents/MacOS/libs/libQt5Concurrent.5.7.dylib -> urrent.5.7.1.dylib dangling: /pathToFolder…/MKVToolNix-10.0.0.app/Contents/MacOS/libs/libQt5Concurrent.5.dylib -> urrent.5.7.1.dylib dangling: /pathToFolder…/MKVToolNix-10.0.0.app/Contents/MacOS/libs/libQt5Concurrent.dylib -> urrent.5.7.1.dylib

Notice “relative” is changed to “dangling”. And link paths is getting truncated “urrent.5.7.1.dylib” instead of “libQt5Concurrent.5.7.1.dylib”.

So I think symlinks are treated as symlinks and they are not followed while being synced -which is also suggested by other threads and bug reports- but they are losing some bits of information in the process.

I am not sure if this a known issue, bug, feature or a file system related error etc., so any insight would be much appreciated.

2

Definitely looks like a bug. I can’t reproduce it on Mac. Can you reproduce it, by creating a new symlink on one of your Macs and see what ends up synced? Can you reproduce it without the Odroid in the loop?

3

Ok, I’ve tried the following:

  1. Turned off Odroid-Ubuntu machine completely. With only 2 machines running, I’ve placed a working copy of MKVToolNix app in the shared folder, on the desktop. It synced and worked correctly on the laptop.

Removed all MKVToolNix’, then

  1. I’ve paused the syncthing connection between the desktop and the laptop, then started up the Ubuntu machine. Both laptop and the desktop connected to Ubuntu. Let’s call the ubuntu machine server. I’ve copied the working MKVToolNix app in the shared folder on the desktop. It synced to the server and then to the laptop. The copy of the app on laptop suffered from mentioned condition, didn’t work.

I’ve cleaned everything up once again

a. I’ve continued with the same setup. The laptop and the desktop are not connected to each other while both are connected to the server. From the desktop, I’ve copied (using Finder) a working MKVToolNix app directly to the server’s shared folder via SMB. It synced to all machines, but on both machines MKVToolNix app failed. b. I’ve tried the same thing but I’ve rsync’ed the MKVToolNix app to the server instead of using Finder. It synced to both machines, but the apps failed.

  1. Thinking this issue maybe caused by different filesystems between the machines, I’ve created a SMB share on desktop and mounted it on the server. I’ve placed the server’s syncthing shared on this network share so that all files are on a HFS drive. I’ve copied a working copy of MKVToolNix in the SMB Shared folder. It is picked up by the syncthing on the server and synced to both the laptop and the desktop. App failed to run on both.

  2. I thought, maybe, it wasn’t because of a ext4/hfs incompatibility, but the Ubuntu is having a trouble reading MacOS symlinks. This one is a little more complicated. I hope I’ll manage to put it in words correctly…
    I’ve created a VMware virtual machine on the desktop machine. Installed Ubuntu Mate 16.04, updated/upgraded etc. Installed open-wm-tools and syncthing. Created a VMware Shared folder. Enabled it. On the host (desktop machine), VMware shared folder was on the main system drive (HFS). On the guest (Virtual Ubuntu Mate) it is mounted at boot by fstab, and it is set up as the syncthing shared folder. I’ve copied a working MKVToolNix app to this VMware Shared folder, on the host (desktop). So for the desktop it is actually a local file. It got picked up by syncthing on the Virtual Ubuntu and synced to both the laptop and the desktop. At this moment we have 2 copies of the MKVToolNix app on the desktop. One is the local copy in VMware Shared Folder, the other is synced with syncthing. Synced apps failed to run on both machines. But when I tried to run the copy which resides in the VMware Shared folder, the app continued to work properly. So to check if Ubuntu is reading MacOS symlinks correctly, I compiled the symlinks software (mentioned in the first post) on the Virtual Ubuntu Machine and checked the MKVToolNix app. It showed correct and complete information.

relative: /home/ubuntu-vm/vmhgfs/syncthing/MKVToolNix-10.0.0.app/Contents/MacOS/libs/libQt5Concurrent.5.7.dylib -> libQt5Concurrent.5.7.1.dylib

  1. So, Ubuntu was able to read MacOS symlinks properly and syncthing doesn’t have a difficulty reading and syncing them on MacOS… I’ve tried the following to pin point where things go wrong. I’ve booted up the server machine again and compiled symlinks software on it too. I’ve turned off the Virtual Ubuntu Machine. Syncthing’s on the laptop and the desktop were both connected to the server, but not to each other. I’ve copied a working copy of the MKVToolNix app to the syncthing shared folder on the desktop. It got synced to the server and then to the laptop. The app on the laptop failed to launch again. I ran the symlinks software on the server and checked the synced MKVToolNix app. It showed correct and complete information. Which I actually didn’t expect, to be honest. I expected the synced symlinks on the server machine to be corrupted. I assumed that Syncthing on Ubuntu was having trouble writing MacOS symlink information locally.

So, presumably, symlinks are transferred correctly from MacOS to Ubuntu but they get corrupted while they are synced from Ubuntu to MacOS.

Hope these help…

4

Maybe. I can’t reproduce any issues in my Mac<->FreeBSD<->Ubuntu setup, in either direction, though. There could be an issue with the Ubuntu-writing-to-VMWare-shared-HFS+ as that is a very unusual setup indeed.

5

Hmmm. That’s strange. It is very possible to VMware HGFS driver causing this, it is a known problematic filesystem driver, but then wouldn’t it suppose to work on Mac<->odroid Ubuntu too?

Can you try downloading MKVToolNix app and placing it in the syncthing shared folder on your Ubuntu machine please? Will it sync to and work properly on your Mac?

DMG container link: https://www.fosshub.com/MKVToolNix.html/MKVToolNix-10.0.0.dmg

I sincerely appreciate the quick replies by the way.

6

I don’t think there’s anything especially magic with the app, I created symlinks the same as your ones above.

7

I don’t know the internals of symlinks but due to the fact that the symlinks software groups them as dangling, relative, absolute, messy, and other_fs, I thought the symlinks in the package and the ones you create with “ln -s” might be different.

But I’ll take your word for it…

8

Those are just comments and opinions on the target of the symlink.

9

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.