Failed Service Nat.Service

General
1

I have syncthing setup between my windows 10 pc and my unraid server (6.8.3). Everything is going well after I disabled relay, however I keep seeing this over and over in the syncthing console for windows (SyncTrayzor).

[AKBFZ] 11:10:19 INFO: Exiting backoff state.
[AKBFZ] 11:10:19 INFO: connections.Service: Failed service 'nat.Service@0xc000f280e0' (1.000000 failures of 5.000000), restarting: true, error: "{nat.Service@0xc000f280e0 nat.Service@0xc000f280e0} returned unexpectedly", stacktrace: [unknown stack trace]
[AKBFZ] 11:10:19 INFO: connections.Service: Failed service 'nat.Service@0xc000f280e0' (2.000000 failures of 5.000000), restarting: true, error: "{nat.Service@0xc000f280e0 nat.Service@0xc000f280e0} returned unexpectedly", stacktrace: [unknown stack trace]
[AKBFZ] 11:10:19 INFO: connections.Service: Failed service 'nat.Service@0xc000f280e0' (3.000000 failures of 5.000000), restarting: true, error: "{nat.Service@0xc000f280e0 nat.Service@0xc000f280e0} returned unexpectedly", stacktrace: [unknown stack trace]
[AKBFZ] 11:10:19 INFO: connections.Service: Failed service 'nat.Service@0xc000f280e0' (4.000000 failures of 5.000000), restarting: true, error: "{nat.Service@0xc000f280e0 nat.Service@0xc000f280e0} returned unexpectedly", stacktrace: [unknown stack trace]
[AKBFZ] 11:10:19 INFO: connections.Service: Failed service 'nat.Service@0xc000f280e0' (5.000000 failures of 5.000000), restarting: true, error: "{nat.Service@0xc000f280e0 nat.Service@0xc000f280e0} returned unexpectedly", stacktrace: [unknown stack trace]
[AKBFZ] 11:10:19 INFO: Entering the backoff state.
[AKBFZ] 11:10:19 INFO: connections.Service: Failed service 'nat.Service@0xc000f280e0' (6.000000 failures of 5.000000), restarting: false, error: "{nat.Service@0xc000f280e0 nat.Service@0xc000f280e0} returned unexpectedly", stacktrace: [unknown stack trace]

Any ideas as to how I can correct this? Thanks for any help!

2

Thanks for reporting, which Syncthing version are you using?

3

Sorry, I forgot to add that.

SyncTrayzor is v1.1.24 and Syncthing is v1.3.4 Windows (64bit) / v1.3.4 Linux (64bit) on UNRAID

4

Are there any logs before that?

5

its just those same lines repeating over and over. If there was something before it, this has eaten it up. This is only on the windows side of the connection.

The logs on the UNRAID side look pretty good except for this which also repeats over and over:

> Connected to myself (#######-#######-#######-#######-#######-#######-#######-#######) at 172.17.0.2:56936-at 172.17.0.2:56936-172.17.0.1:22000/tcp-client/TLS1.3-TLS_AES_128_GCM_SHA256 - should not happen
6

You should restart it and check the logs before it goes into this state.

7

Alternatively, if it doesn’t, hop into Actions -> Logs, Debugging Facilities tab and check “nat”, then let it roll for a few minutes and capture the output from the Log tab. Most likely there’s funkiness going on between Syncthing and your NAT router.

8

ok ill do that. it working but man at the amount of spamming im getting from both sides of these connections

EDIT: adding NAT debugging did not produce anything new, however restarting both sides did. The errors are gone?

[start] 18:26:24 INFO: syncthing v1.3.4 "Fermium Flea" (go1.13.7 windows-amd64) teamcity@build.syncthing.net 2020-01-14 07:01:03 UTC
[start] 18:26:24 INFO: Using large-database tuning
[AKBFZ] 18:26:24 INFO: My ID: XXXXXXX
[AKBFZ] 18:26:25 INFO: Single thread SHA256 performance is 1894 MB/s using minio/sha256-simd (440 MB/s using crypto/sha256).
[AKBFZ] 18:26:25 INFO: Hashing performance is 1079.80 MB/s
[AKBFZ] 18:26:25 INFO: Overall send rate is unlimited, receive rate is unlimited
[AKBFZ] 18:26:25 INFO: Using discovery server https://discovery.syncthing.net/v2/?noannounce&id=XXXXXXX
[AKBFZ] 18:26:25 INFO: Using discovery server https://discovery-v4.syncthing.net/v2/?nolookup&id=XXXXXXX
[AKBFZ] 18:26:25 INFO: Using discovery server https://discovery-v6.syncthing.net/v2/?nolookup&id=XXXXXXX
[AKBFZ] 18:26:25 INFO: TCP listener (192.168.1.20:8384) starting
[AKBFZ] 18:26:25 INFO: Ready to synchronize "Default Folder" (default) (sendreceive)
[AKBFZ] 18:26:25 INFO: Ready to synchronize "Folder" (whfqz-p3xsu) (sendonly)
[AKBFZ] 18:26:25 INFO: Completed initial scan of sendreceive folder "Default Folder" (default)
[AKBFZ] 18:26:25 INFO: GUI and API listening on 127.0.0.1:8384
[AKBFZ] 18:26:25 INFO: Access the GUI via the following URL: http://localhost:8384/
[AKBFZ] 18:26:25 INFO: My name is "Win10"
[AKBFZ] 18:26:25 INFO: Device UG7JA42 is "UNRAID" at [tcp://192.168.1.50:8384]
[AKBFZ] 18:26:30 INFO: Completed initial scan of sendonly folder "Folder" (whfqz-p3xsu)
[AKBFZ] 18:26:35 INFO: Detected 0 NAT services
[AKBFZ] 18:27:10 INFO: Established secure connection to XXXXXXX at 192.168.1.20:8384-192.168.1.50:39336/tcp-server/TLS1.3-TLS_AES_128_GCM_SHA256
[AKBFZ] 18:27:10 INFO: Device XXXXXXX client is "syncthing v1.3.4" named "UNRAID" at 192.168.1.20:8384-192.168.1.50:39336/tcp-server/TLS1.3-TLS_AES_128_GCM_SHA256

Thats the windows side above:

2020-03-12 18:27:10 Established secure connection to XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX at 172.17.0.2:39336-192.168.1.20:8384/tcp-client/TLS1.3-TLS_AES_128_GCM_SHA256
2020-03-12 18:27:10 Device XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX client is "syncthing v1.3.4" named "WinPC" at 172.17.0.2:39336-192.168.1.20:8384/tcp-client/TLS1.3-TLS_AES_128_GCM_SHA256

The UNRAID side. Nothing else has posted for either for over 10 minutes or so… I have no idea.

Thank you for your help!!!

The only other issue i have is speed. I am getting around 10MB/s on average and read that I could increase this by increasing pullers? Is there anywhere I could read to figure out how to do this? I have terrabytes I need to initially sync so this could take a while at 10MB/s I have relay turned off.

9

Make sure you are connected directly and not via a relay.

Puller settings dont do much anymore, there are concurrency settings and number of bytes in flight setting that you could adjust (but it needs to be adjusted on both sides). Yet this could lead to disk thrashing etc, and worse performance. You should check if you have other bottlenecks, CPU, memory, disk io etc.

10

I believe it could be the origination disk. I seem to recall it having very slow xfer speeds. When I xfer files from other disks it is 10-20x faster. This drive I am backing up is probably about to die.

If this is a problem with other drives I will address this again at that point. I am gonna call this one done for now. Thanks for all the help!