Who is up for UDP 10-20%speed boost and better link traversal?

(James Gardiner) #1

I have been looking at building a set of servers around the net for file respotory and distribution… First thing to come to mind was bit torrent sync… But not wanting to rely on proprietary implementations, I saw syncthing and thought hallaluya…

But then read more then thought Doh… Aspira and other file transfer companies say they allow turbo speed file transfers… 10-20% faster then typical http etc ie tcp based file transfers… They do this by using UDP as the main protocol…

This allows far better throughput as there is no handshaking and the transfer can run at the full line synchronous rate. no chatty protocols in the way etc… This is where the 10-20% speed (over tcp) kick comes from…

It also is for more robust and can overcome ISP tricks to throttle certain TCP connections. This is especially important for country to country sync where a lot of packet shaping can be in use…

The limitation of syncthing for TCP and a single connection per instance really puts syncthing on the back foot in terms of all these well understood Internet-pitfalls…

I was a little surprised that a sync application in this age did not start with an implementation to address this out of the gate…

Oh well… But we can always improve… I would like to encourage more work on this area and also mention I am willing to do testing… I am very busy with my own projects but could probably help out with testing the bitrate on such changes in the protocol used by Syncthing.

Great work guys, James

(Jakob Borg) #2

I suspect that in a large majority of cases we are limited more by actual line rate, crypto (i.e. CPU usage) and disk performance than by TCP. At least I haven’t run into that being the bottle neck so far. Also ultimate speed has never been Syncthing’s core focus - that’s security and reliability.

But yeah, it would be nice, if nothing else because we’d gain better NAT busting. But from my side there’s a number of more important things higher up on the todo list.

(Also, raw UDP transfers in Go are out of the box significantly slower than over TCP, in terms of CPU usage, on fast networks. The reason is that each sent or received packet becomes a syscall and thus a kernel/user space context switch. For TCP this multiplexing is done in kernel or even on the NIC. This means TCP is likely to win on fast networks, and UDP may be a win on slow or congested networks. It’s a trade off.)

(Gutleib) #3

Excuse me if this is necroposting, but UDP gives a lot more than 10-20% in low latency networks. Such as networks over radio or sattelite link. Or sync overseas.

So that’s an important feature for some users.

(Stefan Tatschner) #4

(totoba) #5

There is a tool called UdpCast. I use it to send my disk image back ups over the net or network, it is significantly (much much much faster) faster than say using netcat tcp. You can try it yourself.

Does Syncthing use Udp for locally discovered network nodes at all?

(Audrius Butkevicius) #6

It uses udp for discovery, but thats it. No data is transferred over udp.

(Jakob Borg) #7