Report free space on remote devices

Feature
8

We’ve gone against these sort of features before, even for locally checking cpu/ram usage etc, as that is not what the application is meant for.

True. Syncthing does not sync RAM contents, nor it does create CPU tasks remotely (as primary objective). So it would make little sense to monitor RAM or CPU.

Syncthing is used to sync data to storage devices, so when talking about simple indication of free space it is completely different story than CPU or RAM. I understand that you have to protect syncthing from scope creep, but this really feels well within the original scope.

At least as long as we consider that using syncthing on consumer devices at home is legitimate use case. If you say that syncthing is exclusively designed for big datacenters, then i would agree that you should go with separate industrial-grade monitoring system instead. But to be honest i think that syncthing has a lot to offer to everyday end users.

2 Likes
9

Not actually related, see two posts below. This seems related to folder storage quotas. I don’t think that’s entirely out of scope, and I can see a good proposal/implementation be integrated. There’s an open enhancement request already: https://github.com/syncthing/syncthing/issues/4244

1 Like
10

Not sure I agree that “how much free space do I have on a remote device” is related to storage quotas.

2 Likes
11

Actually, you are right. I was thinking about the underlying use-case and skipped on step in the thought process: Here the use-case managing media on a device (mobile) with limited storage. Now I thought you could just set the quota to that limit, then you see that on the remote too. That’s my mistake: If it were that easy, you could also just look at the directory size on the remote and add just the right amount of data. The quota would have to be relative to the mobile’s free space, so back to square one.

1 Like
12

I understood that syncthing uses some kind of filesystem abstraction layer. Is this layer already aware of the amount of available free space?

1 Like
13

I’m not sure what “aware” means specifically, but there is a way to check for free space. Syncthing does this when syncing files to try to make sure they fit instead of filling the disk.

1 Like
14

Displaying free space is also a bit more complex than it looks like at first glance. A single folder could contain multiple mountpoints.

2 Likes
15

That sounds promissing. I was afraid that one of the biggest challenges will be to implement free space checking in some platform-independent way. But now it seems the code is already in place and all we need to do is to send this information to other nodes.

Displaying free space is also a bit more complex than it looks like at first glance. A single folder could contain multiple mountpoints.

I don’t think we need to worry about this. This is unlikely on consumer devices which are target of this feature. if you need some more complex setup, you can deploy complex monitoring as well. i would just check space on root of shared folder. If you need to monitor complex mountpoint setup, but don’t wanna setup separate monitoring, you can always share each mount individualy.

1 Like
16

I would also support for this feature, although it is clearly not as simple as just showing the remaining space.

I am using Syncthing to sync multiple local folders to multiple remote devices, each of them syncthing on several of these folders, which means shared space for several synced folders on several devices. Here is an example: image

Assuming the space left is shared by the synced devices, each of them has its own. So it makes no sense to say that it remains xxx MB. Either we display the remaining space on each of them, or we assume there is a reduction logics to apply (take the min, max, avg, reference device, min over mobile devices, etc.). If we go the reduction route, one of them is to take the minimal space left. Only in this case, we can display something like “xxx MB remaining on yyy” with yyy being the synced device showing the least space left. Moreover, since it is a remote information, it is subject to obsolescence, which means that all we can do is showing the remaining space that was shared the last time we synced the device. In other words “xxx MB remaining on yyy on the last sync (zzz)” with zzz being the date of the update. Not only it makes the info heavy, but it can be fairly old if the device synces rarely (I have some devices like that, which sync manually, like every several months).

And that is just to display the minimal remaining space among your devices, assuming this is exactly what you want. But in fact, it can quickly come to needing a bit more here, then a bit more there, just to have something yet a bit more helpful.

Briefly, this kind of feature is a domain on its own. I would not expect Synthing to support that directly. Simple application of the single responsibility principle of SOLID softwares (yes you can apply them to a whole application too). But I agree this is useful and to be expected at some point, like many other things. Consequently, I would expect Syncthing to be extensible. Simple application of the Open-closed principle, a SOLID aspect too. Basically, other people should be able to produce plugins to cope with specific needs of specific contexts. Then the users can install the plugins they want for their own context.

A plugin support should allow several things, among which:

  • adapt the information sent to other devices, like the remaining space
  • know whether the plugin is effective on the remote device, to know whether such adaptation is applicable without disturbing the remote Syncthing (a bit like the Accept HTTP header)
  • adapt the processing of the receiving device to deal with this adapted information
  • adapt the interface to display the processed information correctly

Yet again, this is a domain on its own. However, the Syncthing team only needs to cope with the effort required by this one. Then any feature asked by the community to deal with specific needs can be answered with a genuine, positive answer: search for the plugin or create it.

I don’t know if there is such a (planned?) plugin support in Syncthing. But that would unlock a lot of things. And if there is current features that the team don’t want to support anymore, they can refactor them as plugins and provide them to the community as-is, before telling that they won’t maintain it anymore, so people actually needing it will have to take the lead. That would be a fair distribution of the work

1 Like
17

I have no idea how syncthing protocol looks like, but i can imagine that syncthing would send some kind of JSON formated string to its peers (on connection and later on each resync). That JSON might contain basic stuff like free storage space for each folder shared with given peer. Being JSON it would allow us to add few similar informations in future if needed without breaking compatibility, if that JSON variable is missing, it would simply mean that the peer does not support space reporting. On the flip side there would be no reason not to send free space to peers who do not know how to interpret that. They will simply ignore that JSON variable (simply by not parsing it which they don’t do anyway).

I agree this JSON should not be too big, just the most essential stuff like free space.

1 Like
18

In syncthing GUI when you click “edit” in the device list, you see all directories shared with that device. There can be free space reported for that individual device.

In list of folders i would the suggest to show minimal value from all devices. Also we can report that directory is 123% full, which would mean that at least one of the devices is not able to sync, because global state is 23% over its capacity.

1 Like
19

I would use this feature too (I sync stuff to my phone, which has more limited storage than my PC), so a +1 from me. But I can see that it’s more complex, and not what the application is really ‘for’ (whatever that means). A halfway house for me would be more obvious reporting when a remote device is out of space.

20

A halfway house for me would be more obvious reporting when a remote device is out of space.

Syncthing actualy already reports when sync fails for this reason.

21

I suppose this could be solved by a cron job on the client, piping “df” to a file in the synced folder…

And then improved from there, independently from Syncthing. I know everyone in the thread probably thought of this, I am just pointing out that sometimes you can add features to a system as pluggable additions, instead of expanding the core product.

I think Syncthing is at a level of maturity when we should be very careful with adding features/complexity. I think of Jira and Trello, both at one time favourites of mine, but now impossible to introduce to a non tech-savvy user.

22

I am well aware of this “solution”. Might work for syncthing running on linux server and being managed by linux admin. But try setting that up on several android devices (some of them being managed by average consumers/users). That is just terrible.

I want syncthing to be solution that i can use to share files with my family, not just with my tech savvy co-workers.

On the other hand, these basic users will not care that there is no support for nested mountpoints and similar complex configurations they will never have on their devices.

23

I agree with the general statement, but:

Try getting free space information from Go on Android :smiley:

24

Earlier in this thread i had the same concern, but it was confirmed that syncthing already knows how to do that:

25

Yeah, we do something. But it might fail (because not implemented), in which case we do nothing. Best case the library does return something on android, maybe even a relevant number for /data (where the DB is), but I’d be surprised if it works.

26

We may discuss all you want about the potential technical solutions and issues, but it looks to me that there is a more pressing matter: is Syncthing expected to provide this feature? On that matter, only the Syncthing team can answer.

If the answer is no, then the next question is: how to extend Syncthing to provide additional features that the team won’t have to spend effort maintaining?

I really think that these questions should be answered first. Because if we have to rely on a plugin, then we can split the effort:

  • have a thread about supporting plugins (if there is no such support yet)
  • redirect this thread towards how to implement a plugin for sharing space information

Does it sound reasonable?

1 Like
27

That sounds very reasonable. There’s been some discussion about whether Syncthing even should send this information - one might not want remotes to know about local system details. Then again, when you are sharing data with said device, sharing a limit on how much data one can share seems like much less sensitive information.

Anyway, there’s the REST api. You cat get a list of remote devices and free disk space (not sure about the latter, better do that in the plugin), then you can do whatever you want with that, to share it with other devices and reconfigure things. So a plugin/estension/companion/… is possible.