Out of Sync folder, find out which client caused it

Support
1

Hi all,

I have a 3 spoke setup: 1 master server = send only 2 slave servers = send&receive each slave server has it’s own set of clients = send&receive

The reason for this setup is:

  • master server downloads files/subfolders and is responsible as the “master” of the data.
  • 2 slave servers are there to handle all traffic with clients and between themselves
  • clients are syncing with eachother as they help locally speed up distribution as long as they are on the same subnet.

Once the Master server shows “out of sync” how can I figure out which client is causing it?

2

You might have two problems.

If you have users connected to the two “slave” servers, and those two sync with each other, that’s probably where your problem is.

Your master server will never receive the edited files from the two receive only servers.

If a user modifies a file such as birdlist.docx on server A and it’s synced to the other slave server (B), there is a risk of another user on that server also modifying and saving the very same file. If two people edit/save the same file before one side can sync to the other, you might very well have a problem.

How likely is it in your environment that two people will want to work on the same file?

And, the master server will never receive newer versions of files.

If you have enough storage space on the master server to store 2 copies of your important files you can have one be the original files that never change and one set that synchronizes in both directions with the two other servers. You still can’t have multiple people editing the same file at the same time on different servers.

3

So in this situation we’re talking about firmware packages for mobile devices. The files are not changed by clients, possibly the ‘smash’ software could download a package I missed on the master. I decide that manually and change the folder to send&receive temporary, or override.

By dividing the firmware in subfolders by device type and models we limit disk usage of the clients. There are a few nitwits who are smashing other types than synced. This setup itself is obvious a bit ‘dangerous’ as client A could manually delete the files and it will delete everywhere (except from master). But this responsibility is put on the operations themselves.

But it’s annoying that people start smashing a iPhone while the station was setup for Android in the first place. Then all android stations get iphone firmware… And that’s what I’m trying to figure out, I just caught one by live monitoring which client was synchronising after I did override on the server, but it’s a tedious job :smiley:

My apologies, for not searching fully through the forum, I just stubbed on Out Of Sync, but Who? which is a post of 2016, where a API endpoint has been mentioned. Maybe there is a better way these days after 7 years? :smiley:

4

OK, so it seems I misinterpreted your issue.

Is “smashing” a software app you’re using? Or are you referring to users messing up the file structure?

You have me a bit confused now.

5

smash app is to erase and install firmware via the computer and multiple devices at the same time.

6

Ahh, OK

Well, I will have to think a bit on a solution.

In the mean time you might find this utility helpful for monitoring the status all all of your servers from a single terminal window.

You can see up-to-date stats on all 3 servers from any one of them. You can SSH terminal into one of the servers from your own desktop PC.

This link is a pointer in syncthing docs to the app and where to get it.

It offers:

Syncthing Cli - a simple command line tool for getting status and performing basic operations from the shell / terminal without need of a web browser.

STC

I’m running it on Windows, but it works on Linux too. You need to specify one flag and that’s the homedir to the config.xml file on your server.

stc-386-windows.exe -homedir C:/Users/leo/AppData/Local/Syncthing

$HOME/.config/syncthing on Linux

7

You’d be better off setting the slave servers to Receive Only, since the expectation is that files are never modified there. (Yes, they still participate in sharing file content with others as usual.)

8

Sry not an answer just an additional question: If I understand @martinleben’s answer correctly the “clients” are also syncthing clients? Which means the master shares a folder two those to slave servers, and those share them to multiple clients (and among each other)?

And the slaves would still share a new file - coming from master - to the clients even if they (slaves) were set to receive only?

9

If I understand your question the answer would be yes.

If the master server is send only and the other two servers are receive only they will always match and you won’t have any files out of sync with each other like you’re having now.

If the two slaves servers are syncing bidirectionally with each other than you run the risk of having one server change a file send it to the other slave and then it doesn’t match the master server anymore.

I’m not percent sure what the master server will do when it determines that one of the slave machines is not in sync with it anymore. My guess without experimenting would be that it should overwrite the slave and put it back in sync with the master but I’m not 100% certain about this.

Anyone else want to chime in and provide some expertise beyond what I know? I have never personally configured or used a master slave server configuration.

I think most people are accustomed to referring to Syncthing servers as servers. Not master and slave or server and client. If a machine is running Syncthing it’s a server as far as I know.

On your master server you created a folder and put a bunch of firmware and it and sorted by subdirectories and then you shared that folder with the other two servers that you are referring to as slaves. That folder has a unique ID number associated with it that matches a folder ID on each of the other two servers. That is how Syncthing knows that two folders are paired on two different servers.

10

On each participating node you have a Syncthing instance. There is no such thing as server/client/master/slave or any other type of naming indicating role, because each Syncthing instance is completely on its own and they don’t have any hierarchical relationship, unless you give them that. By specifying folder type you have kind of done that, that is why I used your way of naming them.

Yes, your clients will participate in the file sharing as usual, even if they have folder type Receive Only. The folder rype is NOT a limiter of the “file distribution” as such. Receive Only means that they will receive changes by others and apply them to the local file system. They will not transmit changes detected on their local file system to others.

Yes.

1 Like
11

Yeah, I only used the terms that were in the original post. I recently asked about this on reddit, and someone answered with a “no”. But maybe I misunderstood the situations…

https://www.reddit.com/r/Syncthing/comments/17fmhzq/does_a_receive_only_folder_still_propagate_changes/

12

So to explain a bit more about the “master, slave, client” setup I made.

  • Master: send-only: is the main server that continuously checks for new files on the internet, and makes sure it has the latest ones in its share folder. It’s also on a different subnet than clients, and uses a different internet connection. This is automated and the “only true source of data” to me.
  • Slave: send&receive : Is on both subnets, the one where the ‘Master’ is in, and the clients. They handle their own set of clients, but all have same files, based on the shared folder. These servers are further untouched, no manual interactions nothing, except to monitor sync status with clients.
  • Clients: send&receive :‘clients’ subnet only, receive files.

Obviously there is no such thing within Syncthing, but I divided the roles for: bandwidth limitation, and because there are some pretty much clients I decided to make 2 “slave” servers, that handle their own set of clients as they are divided over multiple buildings, lets say building a/b and building c/d.

But it is possible that on rare occasions that the client did download a different file from the internet as the Master simply did not detect it or it’s a special device type. So yes, I want the clients to sync with eachother and there for also the ‘slave’ servers. Except for the ‘Master’ just to be sure I still have the files ready for sync if some “not so smart” human decided to delete files from the folder.

I’m syncing mobile device firmwares, the clients are restoring devices. Some devices do Android phones, some do Tablets, some do both. Same counts for iPhone and/or iPads. And a few clients will do any device and type. The ‘smash-tool’ software uses a caching folder. When it does not have the firmware files for the detected device it will download from the internet.

Now there are more “not so smart” people around and they just do whatever device type they want… and I want to find out which client does that because it’s giving other clients additional useless storage consumption. However the clients are marked with labels saying which vendor and device type are being restored there.

If I somehow can ‘catch’ the ones that are restoring other devices than ‘designed’ for. I can inform the team lead about it so they can explain them what the labels are for :wink:

Clients are on send&receive because I noticed with receive only, they will go out-of-sync in above situation, and then I’ll have to override the changes on the client-side every time and I’ll have more to monitor. And the clients will not synchronise with eachother if one has this occasion where the master didn’t had the firmware file.

Slaves are set on send&receive as well, for the same reason as clients, and will also update the other clients on other a/b <> c/d

Master is on send only, just to make sure the clients will not delete files from the master, and I can decide for myself if I override all changes, or temporary set it to send&receive so the master also has those files. Also I trigger an override via the API when the master detected new firmware files and started downloading them.

This setup is done because there is no such thing as ‘master’/‘slave’/‘client’ on SyncThing and needed a way around and yet keep it manageable.

Yesterday by coincidence I found one client who was syncing and looking into it, it was indeed doing iPads instead of android tablets. I called the supervisor and asked and he confirmed they shifted that machine for more workload. Therefore I changed the shared folder and it’s good again. So I’m looking for a good simple way to find out Who(what client) caused the additional sync in case they are syncing the wrong files for the designated shared folder.

13

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.