How to prevent devices from announcing themselves to stdiscosrv with random ports in addition to port 22000, while nat and lan discovery are already disabled?

$ curl -X GET --insecure "https://rsync:8443/?device=N74QTL7-GVVDDQK-OPTDSXQ-DEGHCK7-ZB7VPOS-ZIKWZJ5-5WKHG3L-2SGOVAK"
{"seen":"2024-09-18T13:59:29.367939221+03:00","addresses":["tcp4://10.x.x.x:22000","tcp4://10.x.x.x:42676","tcp4://10.x.x.x:42680","tcp4://10.x.x.x:42684","tcp4://10.x.x.x:42688","tcp4://10.x.x.x:42694","tcp4://10.x.x.x:42700","tcp4://10.x.x.x:42706","tcp4://10.x.x.x:42714","tcp4://10.x.x.x:42720","tcp4://10.x.x.x:42722","tcp4://10.x.x.x:42732","tcp4://10.x.x.x:42738","tcp4://10.x.x.x:42744","tcp4://10.x.x.x:42750","tcp4://10.x.x.x:42756","tcp4://10.x.x.x:42762","tcp4://10.x.x.x:42768","tcp4://10.x.x.x:42774","tcp4://10.x.x.x:42786","tcp4://10.x.x.x:42792","tcp4://10.x.x.x:42802"]}

announceLANAddresses is false

natEnabled is false

the app does not bind to announced ports but lookups from discovery server is causing unnecesary traffic by failed connection attempts[*]

the only open ports are the web gui and 22000/tcp

[*] [ONX5K] 2024/09/18 14:09:31.346886 service.go:1172: DEBUG: dialing N74QTL7-GVVDDQK-OPTDSXQ-DEGHCK7-ZB7VPOS-ZIKWZJ5-5WKHG3L-2SGOVAK tcp4://10.x.x.x:42680 error: dial tcp4 10.x.x.x:42680: connect: connection refused

$ diff --suppress-common-lines -y .local/state/syncthing/config.xml config.xml-default | grep -E -v 'user|password|apikey'
    <folder id="default" label="Default Folder" path="/home/s |     <folder id="default" label="Default Folder" path="/home/s
        <device id="ONX5KQ3-ST4PELN-EOLR4GG-KUDGPX3-2NCAABW-H <
            <encryptionPassword></encryptionPassword>         <
        </device>                                             <
        <maxConflicts>0</maxConflicts>                        |         <maxConflicts>10</maxConflicts>
        <blockPullOrder>random</blockPullOrder>               |         <blockPullOrder>standard</blockPullOrder>
    <device id="ONX5KQ3-ST4PELN-EOLR4GG-KUDGPX3-2NCAABW-HZQKY <
        <address>dynamic</address>                            <
        <paused>false</paused>                                <
        <autoAcceptFolders>true</autoAcceptFolders>           <
        <maxSendKbps>0</maxSendKbps>                          <
        <maxRecvKbps>3</maxRecvKbps>                          <
        <maxRequestKiB>0</maxRequestKiB>                      <
        <untrusted>false</untrusted>                          <
        <remoteGUIPort>0</remoteGUIPort>                      <
        <numConnections>0</numConnections>                    <
    </device>                                                 <
    <device id="SXUMNJK-GUG632P-RONH6WD-WI6C6GT-QSEL5VZ-CEYHY <
        <address>dynamic</address>                            <
        <paused>true</paused>                                 <
        <autoAcceptFolders>false</autoAcceptFolders>          <
        <maxSendKbps>0</maxSendKbps>                          <
        <maxRecvKbps>0</maxRecvKbps>                          <
        <maxRequestKiB>0</maxRequestKiB>                      <
        <untrusted>true</untrusted>                           <
        <remoteGUIPort>0</remoteGUIPort>                      <
        <numConnections>0</numConnections>                    <
    </device>                                                 <
    <device id="36J46B5-NVJFLWX-N3A3JAE-ZBFM5V6-XKBINMT-JX6KZ <
        <address>dynamic</address>                            <
        <paused>true</paused>                                 <
        <autoAcceptFolders>false</autoAcceptFolders>          <
        <maxSendKbps>0</maxSendKbps>                          <
        <maxRecvKbps>0</maxRecvKbps>                          <
        <maxRequestKiB>0</maxRequestKiB>                      <
        <untrusted>true</untrusted>                           <
        <remoteGUIPort>0</remoteGUIPort>                      <
        <numConnections>0</numConnections>                    <
    </device>                                                 <
        <address>0.0.0.0:8384</address>                       |         <address>127.0.0.1:8384</address>
        <listenAddress>tcp4://:22000</listenAddress>          |         <listenAddress>default</listenAddress>
        <globalAnnounceServer>https://rsync:8443/v2/?id=QKNYI |         <globalAnnounceServer>default</globalAnnounceServer>
        <localAnnounceEnabled>false</localAnnounceEnabled>    |         <localAnnounceEnabled>true</localAnnounceEnabled>
        <maxRecvKbps>3</maxRecvKbps>                          |         <maxRecvKbps>0</maxRecvKbps>
        <startBrowser>false</startBrowser>                    |         <startBrowser>true</startBrowser>
        <natEnabled>false</natEnabled>                        |         <natEnabled>true</natEnabled>
        <urAccepted>-1</urAccepted>                           |         <urAccepted>0</urAccepted>
        <urURL>https://localhost</urURL>                      |         <urURL>https://data.syncthing.net/newdata</urURL>
        <autoUpgradeIntervalH>0</autoUpgradeIntervalH>        |         <autoUpgradeIntervalH>12</autoUpgradeIntervalH>
        <keepTemporariesH>48</keepTemporariesH>               |         <keepTemporariesH>24</keepTemporariesH>
        <releasesURL>https://localhost</releasesURL>          |         <releasesURL>https://upgrades.syncthing.net/meta.json
                                                              >         <unackedNotificationID>authenticationUserAndPassword<
        <crashReportingURL>https://localhost</crashReportingU |         <crashReportingURL>https://crash.syncthing.net/newcra
        <crashReportingEnabled>false</crashReportingEnabled>  |         <crashReportingEnabled>true</crashReportingEnabled>
        <stunServer>rsync:3478</stunServer>                   |         <stunServer>default</stunServer>
        <announceLANAddresses>false</announceLANAddresses>    |         <announceLANAddresses>true</announceLANAddresses>
        <connectionPriorityTcpLan>20</connectionPriorityTcpLa |         <connectionPriorityTcpLan>10</connectionPriorityTcpLa
        <connectionPriorityQuicLan>10</connectionPriorityQuic |         <connectionPriorityQuicLan>20</connectionPriorityQuic
        <connectionPriorityTcpWan>40</connectionPriorityTcpWa |         <connectionPriorityTcpWan>30</connectionPriorityTcpWa
        <connectionPriorityQuicWan>30</connectionPriorityQuic |         <connectionPriorityQuicWan>40</connectionPriorityQuic
            <paused>true</paused>                             |             <paused>false</paused>
            <untrusted>true</untrusted>                       |             <untrusted>false</untrusted>

Is your discovery server running behind a reverse proxy? Are there any NATs involved in this setup?

no, and no

Is the affected device running on Windows by any chance?

I think this is caused by TCP port reusing not being supported in the local interface, which causes syncthing to use a normal ephemeral port during announce. (Normally, syncthing would try to announce to the discovery server via the sync port, 22200). The discovery server records all ports used by the announcing device, which then causes the list to be filled with ephemeral ports.

The TCP port reusing may break for various reasons, but it tends to happen more on Windows machines.

If you want more data on this, I suggest enabling debug logging (dialer and discovery).

no, they’re linux. one some older fedora 22, one an oracle linux 7. see more details and logs on these threads

1 Like

Yeah, I think one of the announced addresses contains a 0 port for whatever reason, which is then replaced by the actual remote port in stdiscosrv, which doesn’t work properly if the dialer uses ephemeral ports.

Debug logging will probably show what’s going on.

I am already running in debug logging but see nothing about extra ports being announced. right now I’m struggling to setup a reverse proxy in front of stdiscosrv to be able to capture plain http traffic between the reverse and discovery server. unfortunately stdiscosrv tells nothing relevant in -debug mode

yes, wireshark confirms announcing port 0

JavaScript Object Notation: application/json
    Object
        Member Key: "addresses"
            Array
                String value: tcp4://0.0.0.0:0
                String value: tcp4://:22000

client config is in first post from here Cannot get QUIC working at all with current modifications that I also set

announceLANAddresses is false

natEnabled is false

and changed back from non-working quic to tcp4

found the debug line where it announces port 0 despite explicit tcp4 hostname:port pair defined in listenAddress

Sep 18 16:32:53 t00901 syncthing[17372]: [N74QT] DEBUG: global@https://rsync:8443/v2/ Announcement: {[tcp4://0.0.0.0:0 tcp4://t00901:22000]}

and debug logs again to save you digging the other thread

[user@T00901 ~]$ export STTRACE=connections,dialer,discover,nat,upnp,protocol,net,beacon,relay
[user@T00901 ~]$ export QUIC_GO_LOG_LEVEL=DEBUG
[user@T00901 ~]$ /usr/local/bin/syncthing serve --no-upgrade --no-browser --no-restart
2024/09/18 16:38:20.440367 control_unix.go:35: DEBUG: SO_REUSEPORT supported
2024/09/18 16:38:20.464097 control_unix.go:35: DEBUG: SO_REUSEPORT supported
[start] 2024/09/18 16:38:20.473947 main.go:535: INFO: syncthing v1.27.11 "Gold Grasshopper" (go1.22.6 linux-amd64) builder@github.syncthing.net 2024-08-28 06:32:03 UTC
[start] 2024/09/18 16:38:20.508937 main.go:761: INFO: No automatic upgrades; STNOUPGRADE environment variable defined.
[N74QT] 2024/09/18 16:38:20.510140 syncthing.go:155: INFO: My ID: N74QTL7-GVVDDQK-OPTDSXQ-DEGHCK7-ZB7VPOS-ZIKWZJ5-5WKHG3L-2SGOVAK
[monitor] 2024/09/18 16:38:20.940967 internal.go:44: DEBUG: Dialer logging disabled, as no proxy was detected
[N74QT] 2024/09/18 16:38:20.965567 internal.go:44: DEBUG: Dialer logging disabled, as no proxy was detected
[N74QT] 2024/09/18 16:38:21.012429 syncthing.go:182: INFO: Hashing performance is 61.40 MB/s
[N74QT] 2024/09/18 16:38:21.013031 limiter.go:109: INFO: Device ONX5KQ3-ST4PELN-EOLR4GG-KUDGPX3-2NCAABW-HZQKYU3-SQB75N2-YGLH7QZ send rate is unlimited, receive rate limit is 3 KiB/s
[N74QT] 2024/09/18 16:38:21.013095 limiter.go:162: INFO: Overall send rate is unlimited, receive rate limit is 3 KiB/s
[N74QT] 2024/09/18 16:38:21.013113 limiter.go:168: INFO: Rate limits do not apply to LAN connections
[N74QT] 2024/09/18 16:38:21.013428 service.go:818: DEBUG: Starting listener tcp4://t00901:22000
[N74QT] 2024/09/18 16:38:21.014081 manager.go:92: INFO: Using discovery mechanism: global discovery server https://rsync:8443/v2/?id=QKNYIZC-RIY7HKO-GGL6TQA-F7IK6EO-JFRIMGU-NXW7TKS-54MUSD2-3IE43QZ
[N74QT] 2024/09/18 16:38:21.014200 model.go:415: INFO: Ready to synchronize "Default Folder" (default) (receiveonly)
[N74QT] 2024/09/18 16:38:21.015008 tcp_listen.go:77: INFO: TCP listener (10.x.x.x:22000) starting
[N74QT] 2024/09/18 16:38:21.015229 service.go:477: DEBUG: Connection loop
[N74QT] 2024/09/18 16:38:21.015290 service.go:479: DEBUG: Connection loop in initial rampup
[N74QT] 2024/09/18 16:38:21.016361 folder.go:925: INFO: Completed initial scan of receiveonly folder "Default Folder" (default)
[N74QT] 2024/09/18 16:38:21.016968 api.go:406: INFO: GUI and API listening on [::]:8384
[N74QT] 2024/09/18 16:38:21.017030 api.go:407: INFO: Access the GUI via the following URL: http://127.0.0.1:8384/
[N74QT] 2024/09/18 16:38:21.017130 syncthing.go:312: INFO: My name is "T00901"
[N74QT] 2024/09/18 16:38:21.017237 syncthing.go:315: INFO: Device ONX5KQ3-ST4PELN-EOLR4GG-KUDGPX3-2NCAABW-HZQKYU3-SQB75N2-YGLH7QZ is "rsync-stage" at [dynamic]
[N74QT] 2024/09/18 16:38:21.017282 syncthing.go:315: INFO: Device SXUMNJK-GUG632P-RONH6WD-WI6C6GT-QSEL5VZ-CEYHYTM-X6LF2K3-OOB6VQJ is "T00901" at [dynamic]
[N74QT] 2024/09/18 16:38:21.017319 syncthing.go:315: INFO: Device 36J46B5-NVJFLWX-N3A3JAE-ZBFM5V6-XKBINMT-JX6KZN4-MQXE5SA-UJIM6QX is "T00901" at [dynamic]
[N74QT] 2024/09/18 16:38:21.483847 public.go:74: DEBUG: Dialing direct result tcp rsync:8443: &{{0xc00032e200}} <nil>
[N74QT] 2024/09/18 16:38:22.263831 global.go:186: DEBUG: globalClient.Lookup https://rsync:8443/v2/?device=ONX5KQ3-ST4PELN-EOLR4GG-KUDGPX3-2NCAABW-HZQKYU3-SQB75N2-YGLH7QZ 404 Not Found
[N74QT] 2024/09/18 16:38:22.264328 manager.go:164: DEBUG: lookup results for ONX5KQ3-ST4PELN-EOLR4GG-KUDGPX3-2NCAABW-HZQKYU3-SQB75N2-YGLH7QZ
[N74QT] 2024/09/18 16:38:22.264387 manager.go:165: DEBUG:   addresses:  []
[N74QT] 2024/09/18 16:38:22.264437 service.go:657: DEBUG: Resolved device ONX5KQ3 addresses: []
[N74QT] 2024/09/18 16:38:22.264499 service.go:507: DEBUG: Next connection loop in 5s
[N74QT] 2024/09/18 16:38:23.018495 global.go:273: DEBUG: global@https://rsync:8443/v2/ Announcement: {[tcp4://0.0.0.0:0 tcp4://t00901:22000]}
[N74QT] 2024/09/18 16:38:23.054146 public.go:74: DEBUG: Dialing direct result tcp rsync:8443: &{{0xc0005a2980}} <nil>
[N74QT] 2024/09/18 16:38:23.263935 global.go:282: DEBUG: global@https://rsync:8443/v2/ announce POST: 204 No Content
[N74QT] 2024/09/18 16:38:23.264137 global.go:309: DEBUG: global@https://rsync:8443/v2/ announce Reannounce-After: 3373 <nil>
^C[monitor] 2024/09/18 16:38:25.210892 monitor.go:161: INFO: Signal 2 received; exiting
[N74QT] 2024/09/18 16:38:26.021916 tcp_listen.go:120: INFO: TCP listener (10.x.x.x:22000) shutting down
[N74QT] 2024/09/18 16:38:26.022683 syncthing.go:351: INFO: Exiting

We always add a :0 for good measure.

1 Like

can it be disabled? this will multiply by hundreds of devices resulting in repeated loops of tcp resets, and that would be anything but “good measure” for my network

You can’t. It’s 2 packets, not sure why you are so concerned.

it’s 2 packets multiplied by hundreds or more devices repeated all over. I need to reduce traffic to minimum because it’s a high latency low bandwith infrastructure

plus, I don’t think it’s normal for clients to search for each other on invalid ports in an infinite loop. at least open/bind to that port you are announcing.

Feels like you need specialised software if you care about that. Syncthing chats quite a lot willy nilly, and priority is connectivity rather than minimal bandwidth.

still remains a valid point that clients are instructed to connect to other clients to ports on which they don’t listen. that’s got nothing to do with being chatty, but sounds rather a design bug to me.

it’s like meeting a person and telling them call me on this number, and oh, look me up in the directory, but hey, you might find an invalid number of me there.

Clients don’t know what ports the nat mapped for them, hence the notion of :0, which gets replaced by the outgoing connection port by the discovery server.

So, as already mentioned port 0 gets replaced by the dialing (source) port of the announce, which should be a valid port that syncthing also listens on (like 22000). This is done via TCP port reuse, so that the outgoing port is the same as the incoming port (i.e. you’ve both a TCP client and a TCP server socket on the same port). This causes the annoucement to be sent from port 22000, which your NAT might then remap to whatever. The discovery server records the NAT-mapped port so that hole punching knows where to knock.

In your case this goes wrong (which can happen due to unsupported operating systems, network interface or routing specifics), in that case syncthing falls back to letting the operating system choose an outgoing port. This port is obviously not setup for incoming connections, so it causes the behaviour you’re seeing. This is not the intended behaviour of the announcement, but rather the result of TCP port reusing not working properly for some reason.

There’s debug logging in syncthing which logs TCP port reuse problems, so data from that could help.

there is no nat involved. the nat is disabled in configuration. there is direct routing via isolated network. there is no relay enabled. there is no local announcement enabled. there is no firewall.

it’s just plain simple host A to host B communication via standard router.

I fail to see any logical reason to still announce :0 in this case.