I had to laugh a little while reading this, searching for nextcloud and syncthing integration. I’m not 100% sure how right Jos Poortvliet is about scalability. Sure syncthing consumes more RAM/CPU because the design is different (block indexes) compared to a “simple” webdav share.
The statement “it seems a non-enterprise, home user tech” with the sentence before “it is not an PHP app” smells like Eating your own dog food. Enterprise solutions are always a mix of a bunch of software packages.
The scalability answer isn’t very convincing - sure, syncthing works with 2000 devices. Enough for, say, 1000 users. That is a medium size customer of ours - bigger ones have 100.000 users, say 150.000 devices. Some customers have millions of users.
Then about using the syncthing protocol in our client instead of webDAV.
We picked webDAV because it is a standard that is widely supported and works through any firewall. Syncthing uses its own, special protocol - which you can’t just mount in mac or windows, nor use from third party apps, and if it doesn’t use port 443 it won’t work through many firewalls.
Also, syncthing isn’t a PHP app so it can’t be installed easily on Nextcloud or integrated, especially in a large scale environment.
Sorry, it seems a nice non-enterprise, home user tech. Not really suitable for Nextcloud which has to work in a serious setup.
Look, if it is delta-sync and lan-sync you want, those are perfectly possible in Nextcloud. Just not done. Find a C++ developer and bug him/her to contribute these features to Nextcloud. Heck, deltasync is already mostly done and its developer wants to put it in Nextcloud - LANSync can be done, too.
I think the two are more or less not compatible and two different models, hence it’s unfair to compare them in terms of scalability and what not. Sure, the biggest one that exists is 2k devices and what not, but that’s just one node in the network. We have no idea what’s the size of the biggest network.
Does Nextcloud support multiple servers, even geographically distributed?
One of the best features of Syncthing, at least for my use case, is that it can provide as much redundancy as you want, out of the box.
If I remembered correctly owncloud/nextcloud was initiated with the goal of a “dropbox clone”.
It is not difficult to construct a dropbox clone based on syncthing. With syncthing’s decentraliztion, the performance should be much better.
I might be wrong, but if we force one dedicated device to serve as a central server, other devices will just sync to/from this server, I guess we have something to rely on?
Perhaps Syncthing wouldn’t be considered initially by those who are running the show but I would imagine that for the guys on the ground (the average worker), being able to set up simple synchronized folders for working groups (perhaps even across organizations) quickly and painlessly would be so valuable, especially if they don’t have to call and wait for IT support just to get it done.
Idk, maybe Nextcloud can do this too but it doesn’t seem natural due to it’s centralized nature.
Actually i kinda get what theyre saying, its partly because theres a mismatch between e.g. owncloud and syncthing use cases. Syncthing isnt exactly like owncloud or dropbox as a node in syncthing is more of like a owncloud / dropbox storage server than a frontend for accessing the storage itself (the frontend being the local filesystem).
I’ve been incubating the idea of a “web node” that allows a syncthing node to present a web interface for its storage in a convenient way (e.g. sharing a password protected folder with a user) but encrypts the storage to allow it to be hosted on commodity VPS servers. Some kind of centralized / public interface that allows it to be accessed by users who havent “installed” it is to my mind what users of dropbox/owncloud or similar cloud services want which isnt how syncthing works. In most other respects syncthing is superior.
I actually use syncthing together with nextcloud. The syncing of the nextcloud client/server is way to slow, so i use syncthing for my files and nextcloud as frontend for these files. Syncthing is also handy that it’s multipoint, so when I don’t have internet access my Android phone still can sync with my laptop
Same here. I also use Syncthing and Nextcloud. But not (yet) integrated with each other. Syncthing has its strengths/focus in Syncing. Nextclouds doesnt offer Delta Sync, LAN Sync etc.
Nextcloud does indeed support multiple Servers, even geographically “distributed”.
But NOT for one single folder. Its for an massive amount of Users.
There are two possibilities:
Normal way: Multiple (PHP) Frontends, Clustered Database + an Loadbalancer on front
High Scaleout: Its a product called “global scale”, its like an software loadbalancer/directory server which redirects users to their real “home” server/cluster(normal way)). This might scale to infinite amount of Users
But replication of Users/Folders is not part of Nextcloud. Its mostly done by big NFS shares/DRBD/FilesystemReplicationToolXYZ
So its not really comparable.
But Nextcloud runs on any cheap Webspace. (Mass Adoptable)
It doesnt have some kind of Daemon, which is by design. (Just a Cronjob for Background Tasks).
But this also limits its functionality in certain areas.
Nextcloud
PRO
More Features (Apps for allmost everything in the WebGUI)
Better Mobile Clients (Native; Battery Friendly; Doesnt Download everything directly)
Offers E2E Encryption (Clients) (Just for important Folders; Their are not accessable via the WebGUI anymore. Just a dumb but E2E Encrypted Sync)
Runs on cheap PHP/MariaDB Webspace. Hosted Solutions for some Euros per month
CONTRA
Federated, not Decentralized
Slow, Dumb Sync
Idea: How to make Syncthing and Nextcloud more Plug&Play?
In Nextcloud, there is a fully integrated way to have “External Storage”. This can be Google Drive, Dropbox, Webdav, S3, SFTP etc. You can even allow Users that they are able to “mount” their existing Google Drives in Nextcloud by themself.
So if someone want to use Syncthing with Nextcloud, the easiest way to get it running should be like this:
Nextcloud Admin installs the “Syncthing” App/Plugin from the App Store
User(or admin) adds an External Storage from type “Syncthing” - And points it to a central Server with the Syncthing Daemon running (that might not be the same server; like in my case)
But that means, Syncthing needs to have some kind of embedded Webdav etc.
Of course, installing Apache+mod_dav besides Syncthing would also do it. But thats more for Advanced Users. (Or there is a prepared an Docker Image )
But an own Nextcloud Plugin with access to the API would allow more features than just webdav.
Like:
Syncthing Folder/Device Management: Folders can be added & shared to Devices, directly from within Nextcloud
Dynamic Folders: If the user creates an “Subfolder” in the Syncthing Nextcloud Folder. The app would automatically add a new Folder in Syncthing
etc.
That would combine both Worlds & it would be more “Enterprise Ready”
I have a prototype setup like @fti7 describes, it only has external storage “mounted” with syncthing folder into nextcloud (this is a filesystem readonly bindmount).
Syncthing could have an extra protocol to have a sidedoor (e.g HTTP) to fetch directory contents and file content. Which makes integration with non-syncthing BEPv1 protocol talking services much easier. Indeed it could also be webdav which is fairly easy in go to get it working. But can also be a big security concern. See https://gist.github.com/xor-gate/101b5a817191e8a9f26f39eb4ada40fa
I’m already using syncthing + nextcloud together in my setup it’s actually something that I prefer to use, so that I can have an independent node that I can use to access the data, without being a syncthing node.
The reason for that is using nextcloud as a UI (app is decent, web is fine and universal) to access to the data I’m syncing between my machines with synchting.
So basically, I’ve a syncthing snap installed in my main node server that is synchronized. Then the nextcloud snap which mounts the syncthing folders root (I had to use bindFS to make sure the permissions / owners match), this way I can upload things via WEB UI or app and get them in all my nodes, or at the same time access to them when I’m away.
I also had to use a inotify based script (inotify-exec.py + syncthing-nextcloud-inotify-bridge.sh) to monitor changes done via syncthing in order to update nextcloud of the changes promptly, however this might be done better by just monitoring syncthing via web API, but I just went for the faster development path
This is something that could be in some sort be optimized and made possible even at official level.
It now works with Local External Storage (folders can be shared among users). The scanner also handles multiple folders and multiple users. Plus I’ve added an example docker deployment.
)