Insights Module for Syncthing


(Eser Gozcu) #1

Couple of weeks ago I opened a ticket for having some statistics/historical data on Syncthing UI.

Meanwhile, me and @bahadir60 started investigating the options and came up with a small proof of concept. We thought it would be best to discuss this on the forum instead at the ticket since it would attract more attention. And more attention generally leads to a better discussion :slight_smile:

So, we do have an insights module/feature where you can see most of the data available on the existing GUI in a historical way. It is real time, you do not need to refresh the page to get updates and it gives a quick view of what is happening at your Syncthing Instance.(download rate, upload rate, shared folders per remote device, global and local number of files/folders. per shared folder

We haven’t done any benchmarking yet on the resource usage, and more statistics can be added if need be. This is to give an idea to everyone about the possibilities. And if this gets enough interest, we will be more than happy to spend some considerable amount of time to make it happen.

What do you think guys?


(Nas) #2

I’m all for it. Great! More, more!


(Audrius Butkevicius) #3

How does this work. I guess it accumulates stats while the GUI is open, so a refresh nukes everything?


(uok) #4

My opinion that this should not be implemented has not changed.
Monitoring should be done externally with a separate software that links to Syncthing API.


(Eser Gozcu) #5

@AudriusButkevicius It is highcharts there and no it will not be nuked if you refresh the page :slight_smile: It does keep track of last 5 min on linecharts and there is a datapoint at every 10 seconds. highcharts supports reading data from csv,json,and db where in this case it is json.


(Eser Gozcu) #6

For those who cannot see the video, here is an external link: https://www.youtube.com/watch?v=iM4kd8Gzet8&feature=youtu.be


(Eser Gozcu) #7

@uok this by no means is a monitoring system. Neither alerting nor sending an email/triggering some actions is there. This is an information board. It also does not correlate the data with logs or whatsoever. This is for those who are interested in having a quick view at the instance activity. Instead clicking every shared folder and see some info there, you can see them all on a quick glance.


(Jakob Borg) #8

I’m not necessarily opposed to including this, but it depends significantly on how intrusive it is. It would be nice if this took some sort of approach to #5175 Record more performance metrics that means our internal metrics stuff gets cleaned up and extended. As is, I have no idea as there is no code on display.


(Audrius Butkevicius) #9

Given you call it a “module”, would this be part of opensource syncthing or is this a closed source plugin of some sorts?


(Eser Gozcu) #10

Aim is to make it part of opensource Syncthing and it works out of box, no additional plugin or anything to install. We understand that this might not make it to Syncthing, which is fine and of course understandable. Then me and @bahadir60 can think about other options. like doing thisas a seperate software for those who are interested in having a dashboard app.


(Audrius Butkevicius) #11

Do you have a branchfor people to look at? I cam’t read what metrics are there on my phone so I’d be more interested in the code.


(Eser Gozcu) #12

Sure, we can put it somewhere soon :slight_smile:


(Audrius Butkevicius) #14

There is no diff to read as everything is a single commit. The beef is burried somewhere between 2k lines of other controller code. Others I guess can diff it with whats in master now to try and understand it, I currently have only mobile access.


(Simon) #15

Actually even those with “proper pc access” can’t really diff conveniently, because it’s not a fork, but a new repo that has the synthing code in the subdir syncthing. Just fork the syncthing repo and do your changes on top of it please.


(Audrius Butkevicius) #16

You can diff the files between vanilla and this


(Jakob Borg) #17

Yeah that’s a cruel and unusual way to do it. Why not have it be a fork of Syncthing with the changes in a normal branch?


#18

Sorry for that I have also Syncthing fork and don’t wanna mess it up. Changing are available in here: Github


(Audrius Butkevicius) #19

So I had a look at the code, and it seems it stores stuff in local storage, so it still relies on you having the tab open somewhere to have history, which is quite poor experience. Ideally we’d collect arbitary metrics in process, and expose these as a rest api, and potentially Prometheus endpoint.


(Eser Gozcu) #20

It should not rely on having a tab open somewhere, me and Bahadir explicitly discussed that before the proposal. However if thats the case, then it needs to be changed. And being able to access the metrics via REST surely sounds good. I will test this later tmrw and write back more. Thanks @AudriusButkevicius much appreciated


(Audrius Butkevicius) #21

Well if you look at the code, it’s only html and js, so unless js no longer needs a browser, I have no idea how it works.