Question about multiple connections coming in 1.25

mraneri · September 24, 2023, 11:33pm

I have a NAS with 4 gigabit NICs. We have the 4 gigabit NICs connected to a 10GB switch. Currently we haven’t enabled LACP link aggregation because SMB3 wants to see multiple independent NICS or the multiple connections required to spread the traffic across the NICs aren’t established and all the SMB data goes over one NIC at 1GBit instead of spreading across the 4 NICS.

This is really a failure in thinking of the SMB3. No idea why SMB3 doesn’t support multiple connections over a single logical device. Not like LACP is new or obscure.

Anyway, my NAS has 4 different local IP Addresses (One for each NIC, 192.168.1.5, .6, .7, and .8). Will it be possible in v1.25 to have syncthing establish 4 connections with local devices, one via each IP address in order to utilize 4 NICs and possibly achieve 4Gbit/s transfer rate? Or Syncthing will only ever listen on one address?

calmh · September 25, 2023, 6:26am

I don’t think this will work, at least not reliably.

Syncthing will listen on all adapters by default, and you can add four manual addresses for the other device, but that’s where my mental modelling of what’s going on gets blurry. How does the system decide which interface to route traffic out? What will be the local address of the socket? Does it matter for the routing?

Maybe with enough connections established in both directions between different addresses you might get some spread but I don’t really see how to make it properly deterministic.

bt90 · September 25, 2023, 11:03am

Multipath TCP might be an option, but we don’t support it yet:

github.com/syncthing/syncthing

MPTCP support

opened 12:29AM - 31 Mar 23 UTC

voidpointertonull

enhancement

Initial multipath TCP support was merged into golang recently which should make …its usage quite convenient: https://github.com/golang/go/issues/56539 Given that MPTCP is essentially an extension of TCP, basic support should only need opting into the extension at the time of socket creation. The kernel definitely supports graceful fallback to TCP in case the other side doesn't signal MPTCP support or a bogus middlebox breaks support, and apparently the golang implementation also adds fallback, even though the test case also explicitly checks for success: https://github.com/golang/go/commit/383a4e78861218bd861632d6cdcac6c8af6a5470 Support should be straight-forward (once the time is right with the appropriate golang release) with the mentioned fallbacks. A likely good practice for adoption would be adding a config option making the feature opt-in first, then given no issues and the community supporting the idea, opt-out could become the default in the future. Security considerations: Eventually it might be a good idea to support the concept of one host connecting with multiple addresses at the same time. Given the typical usage of Syncthing and its security model, it already seems to be largely independent of relying on hosts using only specific IP addresses, and by starting with opt-in support, security considerations of likely mostly enterprise users would be revealed eventually. Also consider that having multiple addresses is not unique to MPTCP as Syncthing itself is already aware of the concept to some degree, and there are also plans for multipath support in QUIC just on a different layer: https://datatracker.ietf.org/doc/draft-ietf-quic-multipath/ https://github.com/quic-go/quic-go/issues/3343