Should SyncthingFUSE be merged into Syncthing?

General
1

Continuing the discussion from SyncthingFUSE first release candidate:

I have previously considered adding the streaming of SyncthingFUSE into Syncthing, especially for the use case of small frequently used files always in full sync and large low-frequency files (e.g. photos) in streaming.

I predict users would benefit from a unified experience, better UX, and less bit rot of SyncthingFUSE.

(As an aside, you don’t need to choose between Syncthing and SyncthingFUSE. Both are able to co-exist on the same system, giving you some streaming and some sync folders.)

There are some challenges.

I’m not sure how to not make FUSE a hard dependency for Linux and OS X. We’d need to experiment a bit with running Syncthing with streaming disabled and FUSE libraries missing.

The model that Syncthing and SyncthingFUSE use are quite different, from what I’ve seen so far. Building the streaming feature into Syncthing will require much more understanding of the Syncthing model and pullers. I am not yet motivated to bite that off.

The Syncthing core developers will also have to keep working around the SyncthingFUSE code if it gets merged in. This might be easy, but will probably be annoying. I’d only want to do that if enough people find the feature important enough. I have been curious how many people are actually using SyncthingFUSE but not curious enough to setup any data collection. If we did want more data, would we want to try to use the existing https://data.syncthing.net, or find somewhere else to do the reporting?

I’m leaning mostly toward not merging the streaming feature into Syncthing, but am open to continued discussion. I will also provide assistance to any souls wishing to start work on it themselves.

Building an HTTP API endpoint for pulling block data is interesting. Configuration might get rather strange: Here’s some info about this folder, but don’t try to sync it. I’d want it to be GET /rest/db/block, not file.

I very much want the streaming model for some folders on my (Android) phone. (Obviously, not yet enough to do anything about it yet, either, though.)

1 Like
2

I suggest starting at this end as well. Possibly as a new folder type “cached-only” or something. An API for requesting data for a block, a range of a file, or a complete file would be doable and could form the base for future experiments.

(Possible bonus points for adopting an existing API of some sort. Maybe a read only subset of the S3 API or similar.)

3

It would allow us to implement syncthing-android#205, and probably also syncthing-android#81.

But I’m not sure what GET /rest/db/block would give us. Does that mean we’d have to reconstruct files from blocks on the client side? Sounds like a really leaky abstraction.

4

Yes and no. I don’t think the API should be block focussed, as that’s an internal implementation detail. But it should probably be file range focussed, in that you can ask for either an entire file (we’ll stream it as best we can) or byte range 1231230000-1231260000 of a file and we’ll resolve that to the necessary blocks.

3 Likes
5

Isn’t the most interesting feature that one can do server-like things (WebDAV) without having to care about firewalls, DNS or even the location of the device? You’d just move the server like everything else without having to set up anything again. Off-site backup? Just use Synching and another device that can be located anywhere else and freely moved as well.

Could money speed up the realization of this feature?

6

It’s completely not obvious to me what problem you are trying to solve.

7

Simple. Syncthing doesn’t need IP addresses or advanced configuration: it just works, over LAN and over the Internet. Every machine is identified by an ID. Just give your ID to your friends, share a folder and watch: UPnP will do if you don’t want to port forward or you don’t know how.

  1. SyncFUSE in general? Syncthingish WebDAV.
  2. What I just wrote? Movable WebDAV server, because 1.
8

Again, that’s not a problem statement. What problem does syncthing or syncthing-fuse not solve in your case? Webdav server with no data? You can probably find a portable webdav server and export syncthing fuse via it.

9

You mean a problem that can’t be solved by using both Syncthing and SyncthingFUSE? That would be the problem of having to set up both. And probably using more ressources.

I don’t understand what exporting SyncthingFUSE via a portable WebDAV server means :rolling_eyes: