Editing file via SFTP (or sshfs) causes file to fail to sync between computers. Am I missing something?

Support
1

Scenario: I have 2 computers (let’s call them A and B) syncing a multi-level folder with at least a thousand files. There’s no special push-only/receive-only rules on this sync directory – it should be bidirectional. I also have a laptop (C) that, due to drive size restrictions, mounts a few directories via Samba to computer B. Additionally, in cases where a folder isn’t registered with Samba, sftp or sshfs are used (both are interchangable).

So C has multiple folders mounted from B, most of which are also syncthing folders that are intended to be synced to another desktop in the house. When C modifies a file that exists in a samba share directory on B, the syncing works appropriately and it all acts as you’d expect. However, when a folder is mounted onto C from B via sshfs or sftp, only file creation events seem to register. Here’s an example:

  • C creates a file in B’s synced folder via sftp called test.txt.
  • A gets an update from B, test.txt shows up. All good :white_check_mark:
  • C then modifies test.txt, and makes the file contain the string This is a test. I make these changes via gedit or (insert GUI application here).
  • A’s file remains unchanged, a sync failure is inevitable if this file ever changes. :x:

Expected Behavior: I would expect that this file would change akin to how ssh’ing into the server and modifying the file with vim would work. Despite the fact that C is effectively out of syncthing’s ecosystem, modifying a file on C that’s mounted as a user from B should effectively be the same as modifying the file on the computer B directly.

So the question is whether or not my expected behavior is “correct”. Is there some reason why this scenario should be causing inevitable sync errors? It’s extremely frustrating in scenarios where a “dumb” client only needs to borrow files from a computer with shared ownership for temporary reading a writing (think a Steam Deck or laptop where file “ownership” is not strong, it’s simply borrowing a file from a computer or laptop that has proper ownership.) Is this a bug? Or just a flaw due to how sftp/sshfs works?

2

Is the change picked up by a full rescan of the folder?

3

In addition to what @bt90 asked, there’s another thing to check…

For debugging purposes, instead of gedit or another GUI app, use the terminal version of Vim (not the GUI) – or even better, the command-line.

Let’s assume that the sshfs mount point on C is /home/eoneill/B).

First, repeat the file creation test with just an empty file to see if Syncthing picks it up from B and syncs it to A as expected…

touch /home/eoneill/B/test.txt

… then if that passes, append a line of text to the empty file:

echo 'This is a test.' >> /home/eoneill/B/test.txt

See if the change also syncs from B to A. I suspect that either both tests pass or both fail.

With sshfs, there’s an extra layer of abstraction compared to ssh’ing to B and directly editing test.txt with Vim:

C -> sshfs+FUSE -> SFTP -> sshd -> B

When C creates/modifies test.txt, the changed file is transparently copied from C to B via SFTP.

Because the file is technically temporarily saved on C first before it’s transferred to B, it means that the umask on the C side applies (if possible, it’s preserved on the destination side).

Also, although the umask in a terminal is usually the same as for a WYSIWYG editor in the desktop environment, it’s not guaranteed.

It’s possible that the umask sets the permissions on test.txt such that Syncthing on device B cannot access it. Is the user:group combination used for the sshfs connection to device B the same as who Syncthing is running as on device B?

4

I tried this just now. Yes, a forced full rescan will pick up the change.

Both tests pass in this case.

So if it’s umask problems, I take it that turning on ignore permissions will help?

The server (computer B) side of things here are a bit complicated, in an attempt for me to deal with multiple users in a household needing to sync to single “server”.

Basically I’ve made a syncthing user (and group, by extension) named syncthing. Two users, who both need to use syncthing, are registered with the server to manage files and both belong to the group syncthing. This also requires special flags so that the users can read/write, and that ownership is automatically assigned to the user when putting them into their respective folders (using setfacl for special needs.) I don’t love this situation, but I don’t know a better way to deal with multi-user needs on the “server” end (I use server in quotes, because obviously it’s just another client in a p2p network.)

5

That’s good. Helps rules out a file permissions issue.

Since a full rescan in Syncthing successfully synced the test file from B to A, it’s safe to also rule out a permissions issue while using gedit or other WYSIWYG editor.

Unfortunately no. The ignore permissions setting in Syncthing helps when files are synced between different filesystems that might not support the same type of permissions system, and it also helps when Syncthing is running as a user that’s not that exactly the same as on another device, but it won’t have any impact on the sshfs connection.

Let’s assume that that on C, the umask for user “eoneill” is 0077. Creating a new file results in the following permissions:

-rw------- 1 eoneill eoneill 0 Jan 01 00:00 test.txt

So read/write only for the owner (plus root).

If the test file above is then copied/moved to a sshfs mount between C and B, it might end up looking like this on B:

-rw------- 1 eoneill eoneill 0 Jan 01 00:00 test.txt

But that’s only if the user login for sshfs happens to be “eoneill”. If instead it’s a different user named “smith”:

-rw------- 1 smith smith 0 Jan 01 00:00 test.txt

Having different users on C and B aren’t a problem with sshfs. But if Syncthing on B is running as user “syncthing”, then it won’t be able to access test.txt because it’s only readable/writable by user “smith”.

6

I posted my previous reply while you were updating your reply, so it answers some of my questions. :grinning:

A multi-user setup as you’ve described above requires extra care to get all of the various parts working smoothly.

Do the two users have direct access to B, or indirectly via SMB/sshfs or any other type of network share? If B is just acting as the “server” to sync files between A and C, then there are simpler ways to set things up for a multi-user environment.

There’s still the question of why your test.txt example didn’t get picked up by the filesystem watcher but was successfully synced via a full rescan. You didn’t mention “Linux”, but the pathname syntax, “gedit”, “setfacl” all say it’s Linux on at least B and C.

What I would do next is use inotifywait and inotifywatch to debug the test.txt issue (if the two commands aren’t available, install the inotify-tools package).

  1. On B, with an exiting test.txt file, in a terminal, run the command inotifywatch test.txt.
  2. On C, set up the sshfs mount, then open and add some text to test.txt and save it.
  3. Back on B, hit [Ctrl + C] to tell inotifywatch to stop and print out the fsnotify stats it collected about test.txt.

For example, here’s a sample sequence of steps where an empty text file is created → the string “hello” is appended → the file is deleted along with the output from inotifywatch:

$ touch test.txt

$ inotifywatch test.txt
Establishing watches...
Finished establishing watches, now collecting statistics.

$ echo hello > test.txt

$ rm -f test.txt

$ inotifywatch test.txt
Establishing watches...
Finished establishing watches, now collecting statistics.
^Ctotal  modify  attrib  close_write  open  delete_self  filename
7      2       1       1            1     1            test.txt

Note how inotifywatch reported that there were 2 modify events (the file contents + meta-data), 1 attribute event (meta-data update), 1 close after write event, 1 open event, and one delete event.

All of the events except the file open would’ve resulted in Syncthing initiating a full rescan (after a 10-second countdown).

Syncthing is relying on the Linux kernel’s inotify subsystem for notifications that something has changed. If for some reason no filesystem events are detected, the scheduled full rescan is required to pick up changes.

7

Thanks for all the help. I’ll have some more time this weekend to work on this I think, so I’ll get back to you if I can give this a thorough testing.

8

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.