Splitting Syncthing into a library and a file sharing tool


Looking at the protocol description and the software in general I see that the protocol is a kind of generic thing for creating distributed systems that can share different things (while organized in a file-like structure these do not necessarily be files). The application itself is specific to file sharing. What about splitting the protocol implementation into a separate library?

There are already implementations in different languages (java, swift, I am working on a PHP implementation). The lib could be used for multiple different implementations of distributed applications while the application stays specific to file sharing.

I have no idea what @aral is creating with ind.ie and I’m afraid we do not get much details about this until Nov. 8 2014, but as it is based on this tool I assume it most likely does not use the frontend and file sharing part but only the protocol to sync between instances.

This may also be a solution to the naming thing, letting Syncthing be the app and pulse be the library?

Let me know what you think about this.

kind regards, Carsten


The current implementation language (Go) is awesome for many things, but unfortunately very ill suited for creating a loadable library. I think a C or Java implementation would be a better base to build on. (No idea about Swift really but I suspect it may have similar issues to Go, seeing as it has garbage collection and thus a runtime of its own.)

Well, my main point was not that it should be a loadable library, even splitting out the Go implementation as a separate package would allow using it in other Go projects. The main idea here is to have a reference implementation of the protocol in a separate package that defines the scope of this package regarding included features and the protocol specs.

This would allow porting it to many different languages so that many programmers can build distributed systems on it. As a separate library it can get broader usage not limited to file sharing.

Well, in that sense this is already done; the package is github.com/syncthing/syncthing/internal/protocol. It’s under the internal namespace (because we’re the only consumer), but there’s nothing to stop anyone from pulling it out.

1 Like

Cool, did not notice that. Its a bit hard to figure out how everything works together. I have never worked with Go before and there are near to no comments in your code :smiley:

Guilty as charged… :confused: That’s old code by now, written in proof of concept frenzy (although it works quite well). Many newer contributions are clearer.

1 Like

This is a really interesting topic. I think this idea should be reconsidered in the near future.

The SyncthingFUSE app uses many packages of the main syncthing repo and works like a standalone application, with its own data and configuration, that basically uses the syncthing protocol to talk to other people running syncthing.

It’s a use case for the idea of more explicitly separating code for the app and code for the app internals, if writing this kind of program is something we would like to encourage.

It’s actually fairly split nowadays and getting more so. Although a number of the packages depend on each other.

1 Like