I had my server sync’ing to my father-in-law’s server for quite some time with syncthing, and it worked great. We upgraded to 14.40, and it broke. I commented on the issue here, and I was hoping it would resolve itself with future updates. I’m on 14.43 now, and now that I can grab logs easily from the UI, I figured I’d upload them here to see if they can help.
Machine2 is the machine in my house. It’s a FreeNAS server that also syncs with a FreeNAS server at my parents’ house.
Machine1 is the machine at my in-laws’ house. It’s an Ubuntu desktop machine, used as a server, with a USB hard drive attached to it. We primarily run Plex with it.
Let me know if there is any additional detail I can include. I hope this issue can be resolved. Thanks so much!
At my father-in-law’s place, he doesn’t have access to the router. But why would this have worked just fine until version .40 and then stopped working? Are there logs or other information I can dig up so we can fix the actual issue?
You only need to set tcp ports up on one side for it to work. This is definitely the best way to go, as it will give you the most stable and fast connection. The reason it worked before is most likely that you connected by relay. Since .40 kcp is enabled by default which takes precedence over relays. Looking at the logs you provided it is able to establish a kcp connection, but then it fails after ~1min on one device and the other device reports tons of connected to already connected devices. Does that already ring a bell @AudriusButkevicius?
In any case it would be good (or at least not hurt ) if you could collect debug logs from both sides before solving it via tcp port mappings. Just enable the connection facility in the tab next to the logs in the web UI (and probably pause and resume the remote device such that the connection is reset).
I checked a couple extra boxes in the logging tab, and then I restarted both nodes and grabbed the logs. Anything useful or interesting in there that might help in fixing the 14.40 introduced bug?
Send rate limit is 25 KiB/s, receive rate limit is 800 KiB/s
This is probably your problem, the rate is probably too low to even work, especially when dealing with initial data exchanges.
Also, I think you have something misconfigured, it seems the connections are happening to the GUI port:
Established secure connection to 3WAMNRI-AHOWLOL-5DR3MMZ-YSL7HX3-QC5MKTR-EHDTCCH-FKTVKG6-JNFHUQM at 192.168.2.200:31032-72.203.97.200:8043 (tcp-client) (TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305)
Thanks for the feedback. I quadrupled the rate limit to 100kbps. In terms of having something misconfigured, I’m pretty sure I went with the defaults on most things, with the exception of a rate limit. What would I tweak?
Another thought… Why would the initial 25kbps limit be an issue? This Syncthing node syncs with TWO other machines. I’m having difficulty with one, post 14.40 (worked great beforehand), but the other one had no issue previously and still does not.
The issue is the fact that it uses KCP as the transport. Not tcp and not a relay. I suspect any kind of rate limiting might cause it issues. You can try removing limits all together and see if it works. Your other option would be to make sure you have TCP port mappings setup properly, so that the devices could connect using tcp.
Potentially dumb question… Why did everything work great up until 14.40? Isn’t the fact that everything worked, with no router configuration, just great, and now it doesn’t, indicative of some kind of regression? I’ve posted all the logs I know how to, but I’m happy to post more. I’ve seen others have similar issues in another thread, and I’d love to do my part in helping to solve this issue. Thanks!
Update: Your speculation that this had to do w/ a router setting was incorrect. I wiped out the settings folder on one of the remote machines, and now it’s syncing. It appears that even though 90% of the data was already in sync, something is causing it to try and re-sync everything… It looks like something in the syncthing upgrade process broke it a while back. So yeah it’s working now - just starting over-ish from a sync perspective.
After two days of sync’ing, with a connection constant the entire time, we’re now back where we were before with the broken pipe errors. Considering that we had over 48 hours of perfect connectivity, I don’t think it’ s router config issue. Anything I can do to help solve this? Again, this never happened pre 14.40.
) if you could collect debug logs from both sides before solving it via tcp port mappings. Just enable the connection facility in the tab next to the logs in the web UI (and probably pause and resume the remote device such that the connection is reset).