Awsome but cloning is dangerous

User Stories
1

Hi there. I started using syncthing a few months ago, thank you for a great piece of software.

My use-case is to run a bridged VPN (openvpn) between a few sites and have nodes on each site that replicate users and common data directories. For now the architecture is a star but we could go p2p in the future.

The satellite sites are running all (?) the required services on a Pi 3 dropped on some local network. The user owning the common data is the same on all nodes. Some users have local accounts on more than 1 node.

Re: syncthing, I came to set it up without any master folders nor .stignores. This is slightly risky and a bit sub-optimal (at the main site and contrary to all satellites storage is nfs-mounted; client machines are Apple macs dropping hidden files everywhere) but conflicts are almost non-existent in this configuration. I will mention here that I love the “limitBandwidthInLAN” option.

Problem, I started cloning my Pi 3 software image on new instances. And I was severely bitten when putting a new node online without having generated a new machine ID. Basically, most files disappeared from the shared folders, but I was lucky enough to have one node offline and could recover. I will display my stupidity and inconsequence here, but this near-disaster happened to me … twice. The 1st time, I had no idea what happened, restored data and left it at that. The second time I saw a hostname being recognized as another one, and from there I went on to investigate the Machine ID concept.

To save the hide of dummies like me I would suggest adding a big warning about the risk of cloning machines/accounts. Or perhaps tie/verify Machine ID against host characteristics, like a MAC address or hostname?

Anyways. Thanks a lot to all the devs and contributors who made syncthing what it is. I can’t wait to see what’s next.

1 Like
2

While cloning the device ID is indeed very wrong, it’s not clear to me why it would have this effect. Might you have accidentally cloned the index data base as well? That could have this effect (erasing files) when you bring up a new device, if the index says it’s supposed to have all files but it doesn’t.

3

Well I’m sorry I can’t tell what happened. But it happens fast and it comes from all sides :slight_smile:

(I could show the file hierarchy I use on all nodes in /opt, containing homes and data directories, but unfortunately I can’t seem to be able to manage ASCII art in my posts.)

Normally on a new node:

  • most homes are cloned, some are original to the node.
  • data directories are empty, but during my tests I could have used an instance with a few files (maybe the db includes them.)

Assuming empty directories, is it recommended to nuke the db file as well when re-creating the keys?

4

Yes. Not only recommended, it’s essential.

5

Ok. Is it clear enough in the documentation then?

I think I’ve read the documentation, which is “something” already. But obviously I missed some important messages!

Anyways. All is fine now. Thanks for your kind support and a great project.

6

Oh it’s probably not at all clear in the documentation. I don’t think we deal at all with any kind of mass installation tips there. An article on the subject would be most welcome.

2 Likes
7

I think you should gear for mass installation scenarios. This is great stuff, you know :wink:

I don’t feel really knowledgeable enough to write anything in the doc or the wiki (yet). But I can translate EN to FR if needed.

EDIT. Clippy told me how to do ASCII art. Here is our file hierarchy

/opt/foo.org/
├── org (*)
│   ├── storage
│   └── spool
└── usr
    ├── timecapsule
    ├── home
    │   ├── foo.org
    │   ├── localuser1
    │   └── localuser2
    └── shared (*)
        ├── Some-dir
        └── Another-dir

User foo.org is nologin, he only has .config in his home. He owns the data in dirs marked with (*).

Localusers have .config, a Sync dir and data in their homes. It is possible that localuser1 may exist on more than one node (people working at more than 1 site.)

There is one syncthing instance for each user (including foo.org)

Everything is under /opt/foo.org, the idea was to ease cloning/deployment.

8

I wish we could get a blog post from whoever is running a cluster with 123 nodes, as shown here:

https://data.syncthing.net/

9

Hmm interesting. But I dont understand the meaning of columns in that “Usage Metrics” table. What does it mean 5/50/95/100% ?

10

5th, 50th, 95th, and 100th percentiles.

EDIT: I also put a better description together here.

1 Like
11

… and since that Wikipedia page makes my eyes glaze over (and I understand percentiles), let me summarize it as simply that 50% of the devices report a value that is equal to or lower than the value in the 50% column, etc. So they can be though of as “almost noone”, “the median”, “almost everyone” and “max”, respectively.

2 Likes
12

Hello first off I would like to say excellent project I have been using since the beginning and at this point couldn’t work without it. I came across a similar scenario as the above today as I have a common windows VirtualBox image that is configured with syncthing and today I realized that if two copies of this image are brought up at the same time they will sync as if they are the same machine. With no indication anywhere that there is another node in the cluster. This along with some other reconfigurations I have done lately lead me to believe that a machine could be cloned simply by manually configure a new pc with the correct machine id which is exchanged with any shared folder. I am by far a security novice and could be way off, I don’t have the capabilities to fully test all avenues of this but could help if there is something I am obviously missing here.

13

The id is not just a blob of text, the id is backed by a cryptographic certificate, without which you cannot prove you are that id.