Two masters never in sync

Support
1

Yes, the title sounds like the problem should be obvious, but there’s more going on here.

Device A has folders X and Y. Both have the same path, but Y is a folder master.

Device B also has folders X and Y. Same config as above; same path, and Y is a master.

All shares have “ignore permissions” turned on.

Both devices share both folders with each other. (This may seem like an unusual config, but there is a purpose. The gist of it is that there are other devices not listed here that are also sharing folder Y, and device B is in a location with a faster upload pipe, but isn’t always powered on. So we want to sync quickly from B when we can.)

Expectation: The X share keeps the two folders in sync. The Y folders notice the underlying filesystem changes and come to agreement on their identical state.

Reality: The X share keeps the two folders in sync. The Y folders are hopelessly confused and neither of them agree about the folder state. Since they are both masters, this disagreement is permanent. Pressing the override button on one causes every file to be listed out of sync on the other.

Here is a /rest/db/file request from the perspective of the out-of-sync master:

{
  "availability": [
    "AMSHJOQ-WMLJLRU-UZXN5PD-7XLVCQV-7D7E5CJ-UDLKX7Q-QTSWGJB-JHYGXAS"
  ],
  "global": {
    "flags": "0100666",
    "localVersion": 28220,
    "modified": "2012-04-28T13:35:28-04:00",
    "name": "00 Alien Brambles.mp3",
    "numBlocks": 32,
    "size": 4097674,
    "version": [
      "226434223863681720:2",
      "4082906498302680946:2"
    ]
  },
  "local": {
    "flags": "0100666",
    "localVersion": 23923,
    "modified": "2012-04-28T13:35:28-04:00",
    "name": "00 Alien Brambles.mp3",
    "numBlocks": 32,
    "size": 4097674,
    "version": [
      "226434223863681720:1",
      "4082906498302680946:2"
    ]
  }
}

Everything seems to be in agreement except for localVersion and I’m not sure what significance this has.

2

Same here, I also couldn’t figure it out :confused: - two user share a computer but are offline most of the time. To make sure that everything is synced as often as possible I set up two Syncthing instances that both share the same folder as folder master. The receiving computer never was in sync with a nice game of “override ping pong” :tennis: - exactly like @cdhowie described. As a workaround I used nssm to setup a Sycnthing service

3

This is a completely untested setup (i.e. we don’t test to make sure this kind of thing works, even a little). Perhaps if you post the full configs somewhere (to make sure we don’t miss anything from just the verbal description) we can add a test case for it.

4

Is there a workaround? The goal is to have two nodes that will sync with each other, but will not accept changes from a third node. This is the simplest config I could come up with to allow that.

As long as one of the two masters is out of sync, it will not send files to the third node. In other words, it’s not just a UI/display annoyance. The desired behavior simply doesn’t seem possible with my config.

5

I have already changed the config back since it wasn’t accomplishing anything in the broken state, but I’ll spell it out as precisely as I can here.

  • Device A
    • Folder X
      • Path: /foo/bar
      • Ignore permissions: Yes
      • Folder master: No
    • Folder Y
      • Path: /foo/bar
      • Ignore permissions: Yes
      • Folder master: Yes
    • Has device B, sharing X and Y

Device B is exactly the same (except that it has device A instead).

Set up initially without Y, so that X (read-write) is the only folder shared. Put some stuff in it and let it sync to the other device. Then set up folder Y on both nodes and restart them. They won’t ever come into sync.

6

Why both? Wouldn’t X be enough, and both just confusing?

Sharing the same directory in two folders between the same device seems bad. They’ll step on each others temp files and interpret changes synced in one folder as local changes by the other…

7

The final config is as follows, leaving out the X and Y folder config details, and just who is sharing what with whom. Three devices, all connected to each other.

  • Device A
    • Shares X and Y with B
    • Shares Y with C
  • Device B
    • Shares X and Y with A
    • Shares Y with C
  • Device C
    • Shares Y with A and B

Devices A and B do not technically have to share Y with each other, but it doesn’t matter. They transitively share it through device C, who will do the standard synchronization logic on the indices from A and B and propagate them. A and B will fall out of sync with each other whether or not they share Y directly with each other.

The only difference if A and B don’t share Y directly is that the “Y cluster” changes from mesh model to spoke model.

Therefore the desired behavior of “A and B both help to seed the data to C” doesn’t happen. The device that thinks its view is out of date won’t help to seed C. (Or, more correctly, C won’t ask that device for files because it doesn’t think it has the most up-to-date copies.)

8

One possible solution, though a huge undertaking, is to extend the model to include the notion of a “share” and separate that from a “folder.”

What I mean is that a “folder” is a local filesystem resource with some knobs, and a “share” is an association between a folder and a device. I assume that we have this already in some form since folder-to-device is a many-to-many relationship. (“Share” is the bridge entity in the model that connects the two.)

Perhaps then we would just need to move some knobs from “folder” to “share.” Since a folder just defines a local filesystem resource, the “master” setting might be moved to the “share” object. This would allow one to share the same physical folder to one device read-write while sharing it to another as read-only or master.

This could simplify this use case and would allow Syncthing to only need to scan the folder once.

For example:

  • Folder ID: Should actually belong to shares and not folders; why can’t I share the same physical folder with two different devices under two different names?
  • Folder path: Belongs on folder.
  • Rescan interval: Belongs on folder.
  • Minimum free disk space: Belongs on folder.
  • Folder master: Belongs on share, perhaps with an override on the folder to act as a failsafe in case you really really want to make sure that if you accidentally turn off master on a share, the contents are still protected.
  • Ignore permissions: I could see arguments for putting it either place. Putting it on share is more flexible, but putting it on folder will cover 95% (made-up statistic) of the use cases, will be less confusing, and will require less tinkering when setting stuff up.
  • Pull order: Belongs on share.
  • File versioning: Belongs on folder.

So what I’m suggesting is a division between “stuff about the physical disk resource” and “stuff about how I talk to peer nodes about this.” Note that the index/state would need to be associated with the share and not the folder – because we accept index changes on a master folder, but we just don’t do anything locally with it. If we have a folder that’s shared once read-only and once read-write, the read-only index shouldn’t clobber the read-write index. (Or maybe we just need two indexes. Hmm. This is where it starts to get hairy. I’ll stop editing my post now and let you digest this idea before I spray more thoughts at you.)

9

I tried this configuration because it’s easier to set up for the average user and more secure than running everything as a service. It was a test, I’m not sure if this use case is relevant for every day use

Computer A (both folder master, ignore permissions)

  • User/Syncthing #1: share folder X
  • User/Syncthing #2: share folder X

Computer B (ignore permissions)

  • Syncthing simply receives folder X

If Syncthing #1 clicks override, Syncthing #2 is out of sync and vice-versa

10

I assume #1 and #2 don’t share it with each other? In which case when #1 overrides, #2 should rescan and agree on the state, atleast I think so.

11

No, they don’t share with each other.

12

If Computer B is “receiving” folder X form both A1 and A2 then it doesn’t matter, A1 and A2 might as well be sharing with each other directly. As I mentioned above, the only difference is whether the network is mesh or spoke; if A1 and A2 share with a common node then they are effectively sharing with each other and they will think they are out of sync even when configured to look at the same directory on disk.

13

Slight tl;dr here for me so this may only be pseudo-relevant, just throwing this out there for consideration:

  • X and Y are two folders with the same path.
  • Two devices A and B share X and Y with each other.
  • X is set as “master” on both sides, because you don’t want other clients modifiying A or B:s contents.

(I think that’s the setup you’ve described, as far as the two masters are concerned.)

Consider a file update.

  • A detects an update to X/foo and announces it.
  • B rejects it because it is also a folder master of X. B is out of sync with A now.
  • A detects the same change to Y/foo and announces it.
  • B syncs that change.
  • B detects that change as a local change in X/foo and announces it.
  • A rejects that because it’s a folder master of X. A is out of sync with B now.
  • There is a conflict between X/foo on A and X/foo on B now, although only on metadata, and neither of them is going to resolve it as they are both masters.

That’s suboptimal. You can try to work around this by not sharing X between A and B, only Y where they are not masters. However, then any change by the other to Y/foo will be detected as a local change to X/foo and you’ll again have a conflict. That conflict will be resolved by the clients out there (because they get two conflicting updates from each of the masters), but the masters won’t accept the resolution back.

In short, I don’t think your scenario is cleanly implementable with the current definition of “master”.

14

Right. That’s part of the reason for my suggestion that some of these settings be associated with the share of a specific folder to a specific device, rather than being on the share itself. For example, if “master” is a setting associated with a share and not a folder, then we only need one folder on A and B, they share with each other, but also with C – however, their shares with C are “master” while the shares with each other are not.

This does complicate indexes though, since we need one index between A and B, and another index to “accept” changes from C but not push them to the filesystem (current master behavior). Changes to the read-write index inform the read-only index, but not the other way. Only the read-write index gets to make disk changes.

For reference, the behavior I’m looking for would be similar to a btsync setup where two nodes have the read-write key and one has the read-only key. I don’t see a clear way to mirror this kind of config with Syncthing today.

15