Not sure precisely, but I think it may be 512MB. top
says:
Mem: 300540K used, 105676K free, 16284K shrd, 140156K buff, 88444K cached
Here’s roughly when the crashes started:
[PVCOA] 10:52:03 INFO: c.S.listenerSupervisor: Entering the backoff state.
[PVCOA] 10:52:03 INFO: c.S.listenerSupervisor: Failed service 'dynamic+https://relays.syncthing.net/endpoint' (2.963105 failures of 2.000000), restarting: false, error: %!s(<nil>)
[PVCOA] 11:02:15 INFO: c.S.listenerSupervisor: Exiting backoff state.
[PVCOA] 11:02:15 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) starting
[PVCOA] 11:02:16 INFO: Listen (BEP/relay): Get "https://relays.syncthing.net/endpoint": x509: certificate signed by unknown authority
[PVCOA] 11:02:16 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) shutting down
[PVCOA] 11:02:16 INFO: c.S.listenerSupervisor: Failed service 'dynamic+https://relays.syncthing.net/endpoint' (1.000000 failures of 2.000000), restarting: true, error: %!s(<nil>)
[PVCOA] 11:02:16 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) starting
[PVCOA] 11:02:17 INFO: Listen (BEP/relay): Get "https://relays.syncthing.net/endpoint": x509: certificate signed by unknown authority
[PVCOA] 11:02:17 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) shutting down
[PVCOA] 11:02:17 INFO: c.S.listenerSupervisor: Failed service 'dynamic+https://relays.syncthing.net/endpoint' (1.987327 failures of 2.000000), restarting: true, error: %!s(<nil>)
[PVCOA] 11:02:17 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) starting
[PVCOA] 11:02:17 INFO: Listen (BEP/relay): Get "https://relays.syncthing.net/endpoint": x509: certificate signed by unknown authority
[PVCOA] 11:02:17 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) shutting down
[PVCOA] 11:02:17 INFO: c.S.listenerSupervisor: Entering the backoff state.
[PVCOA] 11:02:17 INFO: c.S.listenerSupervisor: Failed service 'dynamic+https://relays.syncthing.net/endpoint' (2.955315 failures of 2.000000), restarting: false, error: %!s(<nil>)
[monitor] 11:04:28 INFO: Syncthing exited: signal: killed
[start] 11:04:31 INFO: syncthing v1.12.1 "Fermium Flea" (go1.15.6 linux-arm) teamcity@build.syncthing.net 2020-12-06 12:46:27 UTC
[PVCOA] 11:04:40 INFO: My ID: PVCOALW-VXLFTYA-4HPJX3M-JUYVHDB-MNSKMX5-Y74JAO4-EL7XE27-CDEPJAQ
[PVCOA] 11:04:41 INFO: Single thread SHA256 performance is 4.2 MB/s using minio/sha256-simd (4.2 MB/s using crypto/sha256).
[PVCOA] 11:04:43 INFO: Hashing performance is 3.74 MB/s
[PVCOA] 11:04:43 INFO: Overall send rate is unlimited, receive rate is unlimited
[PVCOA] 11:04:43 INFO: Using discovery mechanism: global discovery server https://discovery.syncthing.net/v2/?noannounce&id=LYXKCHX-VI3NYZR-ALCJBHF-WMZYSPK-QG6QJA3-MPFYMSO-U56GTUK-NA2MIAW
[PVCOA] 11:04:43 INFO: Using discovery mechanism: global discovery server https://discovery-v4.syncthing.net/v2/?nolookup&id=LYXKCHX-VI3NYZR-ALCJBHF-WMZYSPK-QG6QJA3-MPFYMSO-U56GTUK-NA2MIAW
[PVCOA] 11:04:43 INFO: Using discovery mechanism: global discovery server https://discovery-v6.syncthing.net/v2/?nolookup&id=LYXKCHX-VI3NYZR-ALCJBHF-WMZYSPK-QG6QJA3-MPFYMSO-U56GTUK-NA2MIAW
[PVCOA] 11:04:43 INFO: Stored folder metadata for "kerq9-hpdtz" is inconsistent; recalculating
[PVCOA] 11:04:43 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) starting
[PVCOA] 11:04:43 INFO: Using discovery mechanism: IPv4 local broadcast discovery on port 21027
2021/01/18 11:04:43 connection doesn't allow setting of receive buffer size. See https://github.com/lucas-clemente/quic-go/wiki/UDP-Receive-Buffer-Size for details.
[PVCOA] 11:04:43 INFO: Using discovery mechanism: IPv6 local multicast discovery on address [ff12::8384]:21027
[PVCOA] 11:04:43 INFO: QUIC listener ([::]:22000) starting
[PVCOA] 11:04:43 INFO: TCP listener ([::]:22000) starting
[PVCOA] 11:04:43 INFO: quic://0.0.0.0:22000 detected NAT type: Full cone NAT
[PVCOA] 11:04:43 INFO: quic://0.0.0.0:22000 resolved external address quic://95.232.25.97:64594 (via stun.syncthing.net:3478)
[PVCOA] 11:04:44 INFO: Listen (BEP/relay): Get "https://relays.syncthing.net/endpoint": x509: certificate signed by unknown authority
[PVCOA] 11:04:44 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) shutting down
[PVCOA] 11:04:44 INFO: c.S.listenerSupervisor: Failed service 'dynamic+https://relays.syncthing.net/endpoint' (1.000000 failures of 2.000000), restarting: true, error: %!s(<nil>)
[PVCOA] 11:04:44 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) starting
[PVCOA] 11:04:44 INFO: Listen (BEP/relay): Get "https://relays.syncthing.net/endpoint": x509: certificate signed by unknown authority
[PVCOA] 11:04:44 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) shutting down
[PVCOA] 11:04:44 INFO: c.S.listenerSupervisor: Failed service 'dynamic+https://relays.syncthing.net/endpoint' (1.983902 failures of 2.000000), restarting: true, error: %!s(<nil>)
[PVCOA] 11:04:44 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) starting
[PVCOA] 11:04:45 INFO: Listen (BEP/relay): Get "https://relays.syncthing.net/endpoint": x509: certificate signed by unknown authority
[PVCOA] 11:04:45 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) shutting down
[PVCOA] 11:04:45 INFO: c.S.listenerSupervisor: Entering the backoff state.
[PVCOA] 11:04:45 INFO: c.S.listenerSupervisor: Failed service 'dynamic+https://relays.syncthing.net/endpoint' (2.952322 failures of 2.000000), restarting: false, error: %!s(<nil>)
[PVCOA] 11:04:47 INFO: GUI and API listening on [::]:8384
[PVCOA] 11:04:47 INFO: Access the GUI via the following URL: http://127.0.0.1:8384/
[PVCOA] 11:04:47 INFO: My name is "temp"
[PVCOA] 11:04:47 INFO: Device B3TV5JK-7WSO5HH-5VD4XMG-NOV6TK5-MVT2Z57-5LPWSDM-H45OI67-3ALLEQV is "SyncThingSrv" at [dynamic]
[PVCOA] 11:04:47 WARNING: Syncthing should not run as a privileged or system user. Please consider using a normal user account.
[PVCOA] 11:04:53 INFO: Detected 0 NAT services
[PVCOA] 11:05:10 INFO: Established secure connection to B3TV5JK-7WSO5HH-5VD4XMG-NOV6TK5-MVT2Z57-5LPWSDM-H45OI67-3ALLEQV at 192.168.1.158:22000-192.168.1.170:22000/tcp-server/TLS1.3-TLS_CHACHA20_POLY1305_SHA256
[PVCOA] 11:05:10 INFO: Device B3TV5JK-7WSO5HH-5VD4XMG-NOV6TK5-MVT2Z57-5LPWSDM-H45OI67-3ALLEQV client is "syncthing v1.12.1" named "SyncThingSrv" at 192.168.1.158:22000-192.168.1.170:22000/tcp-server/TLS1.3-TLS_CHACHA20_POLY1305_SHA256
[PVCOA] 11:11:07 INFO: Device B3TV5JK-7WSO5HH-5VD4XMG-NOV6TK5-MVT2Z57-5LPWSDM-H45OI67-3ALLEQV folder "Main Library" (kerq9-hpdtz) has a new index ID (0x989F5108B952D2B2)
[PVCOA] 11:12:42 INFO: Ready to synchronize "Main Library" (kerq9-hpdtz) (receiveonly)
[PVCOA] 11:19:21 INFO: c.S.listenerSupervisor: Exiting backoff state.
[PVCOA] 11:19:22 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) starting
[PVCOA] 11:19:26 INFO: Listen (BEP/relay): Get "https://relays.syncthing.net/endpoint": x509: certificate signed by unknown authority
[PVCOA] 11:19:26 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) shutting down
[PVCOA] 11:19:26 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) starting
[PVCOA] 11:19:26 INFO: c.S.listenerSupervisor: Failed service 'dynamic+https://relays.syncthing.net/endpoint' (1.000000 failures of 2.000000), restarting: true, error: %!s(<nil>)
[PVCOA] 11:19:27 INFO: Listen (BEP/relay): Get "https://relays.syncthing.net/endpoint": x509: certificate signed by unknown authority
[PVCOA] 11:19:27 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) shutting down
[PVCOA] 11:19:27 INFO: c.S.listenerSupervisor: Failed service 'dynamic+https://relays.syncthing.net/endpoint' (1.972294 failures of 2.000000), restarting: true, error: %!s(<nil>)
[PVCOA] 11:19:27 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) starting
[PVCOA] 11:19:28 INFO: Listen (BEP/relay): Get "https://relays.syncthing.net/endpoint": x509: certificate signed by unknown authority
[PVCOA] 11:19:28 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) shutting down
[PVCOA] 11:19:28 INFO: c.S.listenerSupervisor: Entering the backoff state.
[PVCOA] 11:19:28 INFO: c.S.listenerSupervisor: Failed service 'dynamic+https://relays.syncthing.net/endpoint' (2.917575 failures of 2.000000), restarting: false, error: %!s(<nil>)
[monitor] 11:23:13 INFO: Syncthing exited: signal: killed
[start] 11:23:15 INFO: syncthing v1.12.1 "Fermium Flea" (go1.15.6 linux-arm) teamcity@build.syncthing.net 2020-12-06 12:46:27 UTC
[PVCOA] 11:23:19 INFO: My ID: PVCOALW-VXLFTYA-4HPJX3M-JUYVHDB-MNSKMX5-Y74JAO4-EL7XE27-CDEPJAQ
[PVCOA] 11:23:20 INFO: Single thread SHA256 performance is 4.4 MB/s using crypto/sha256 (4.2 MB/s using minio/sha256-simd).
[PVCOA] 11:23:22 INFO: Hashing performance is 3.73 MB/s
[PVCOA] 11:23:22 INFO: Overall send rate is unlimited, receive rate is unlimited
[PVCOA] 11:23:22 INFO: Using discovery mechanism: global discovery server https://discovery.syncthing.net/v2/?noannounce&id=LYXKCHX-VI3NYZR-ALCJBHF-WMZYSPK-QG6QJA3-MPFYMSO-U56GTUK-NA2MIAW
[PVCOA] 11:23:22 INFO: Using discovery mechanism: global discovery server https://discovery-v4.syncthing.net/v2/?nolookup&id=LYXKCHX-VI3NYZR-ALCJBHF-WMZYSPK-QG6QJA3-MPFYMSO-U56GTUK-NA2MIAW
[PVCOA] 11:23:22 INFO: Using discovery mechanism: global discovery server https://discovery-v6.syncthing.net/v2/?nolookup&id=LYXKCHX-VI3NYZR-ALCJBHF-WMZYSPK-QG6QJA3-MPFYMSO-U56GTUK-NA2MIAW
2021/01/18 11:23:23 connection doesn't allow setting of receive buffer size. See https://github.com/lucas-clemente/quic-go/wiki/UDP-Receive-Buffer-Size for details.
[PVCOA] 11:23:23 INFO: QUIC listener ([::]:22000) starting
[PVCOA] 11:23:23 INFO: TCP listener ([::]:22000) starting
[PVCOA] 11:23:23 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) starting
[PVCOA] 11:23:23 INFO: Using discovery mechanism: IPv4 local broadcast discovery on port 21027
[PVCOA] 11:23:23 INFO: Using discovery mechanism: IPv6 local multicast discovery on address [ff12::8384]:21027
[PVCOA] 11:23:23 INFO: Ready to synchronize "Main Library" (kerq9-hpdtz) (receiveonly)
[PVCOA] 11:23:24 INFO: Listen (BEP/relay): Get "https://relays.syncthing.net/endpoint": x509: certificate signed by unknown authority
[PVCOA] 11:23:24 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) shutting down
[PVCOA] 11:23:24 INFO: c.S.listenerSupervisor: Failed service 'dynamic+https://relays.syncthing.net/endpoint' (1.000000 failures of 2.000000), restarting: true, error: %!s(<nil>)
[PVCOA] 11:23:24 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) starting
[PVCOA] 11:23:25 INFO: Listen (BEP/relay): Get "https://relays.syncthing.net/endpoint": x509: certificate signed by unknown authority
[PVCOA] 11:23:25 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) shutting down
[PVCOA] 11:23:25 INFO: c.S.listenerSupervisor: Failed service 'dynamic+https://relays.syncthing.net/endpoint' (1.972814 failures of 2.000000), restarting: true, error: %!s(<nil>)
[PVCOA] 11:23:25 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) starting
[PVCOA] 11:23:26 INFO: Listen (BEP/relay): Get "https://relays.syncthing.net/endpoint": x509: certificate signed by unknown authority
[PVCOA] 11:23:26 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) shutting down
[PVCOA] 11:23:26 INFO: c.S.listenerSupervisor: Entering the backoff state.
[PVCOA] 11:23:26 INFO: c.S.listenerSupervisor: Failed service 'dynamic+https://relays.syncthing.net/endpoint' (2.930912 failures of 2.000000), restarting: false, error: %!s(<nil>)
[PVCOA] 11:23:29 INFO: GUI and API listening on [::]:8384
[PVCOA] 11:23:29 INFO: Access the GUI via the following URL: http://127.0.0.1:8384/
[PVCOA] 11:23:29 INFO: My name is "temp"
[PVCOA] 11:23:29 INFO: Device B3TV5JK-7WSO5HH-5VD4XMG-NOV6TK5-MVT2Z57-5LPWSDM-H45OI67-3ALLEQV is "SyncThingSrv" at [dynamic]
[PVCOA] 11:23:29 WARNING: Syncthing should not run as a privileged or system user. Please consider using a normal user account.
[PVCOA] 11:23:33 INFO: Detected 0 NAT services
[PVCOA] 11:24:58 INFO: Established secure connection to B3TV5JK-7WSO5HH-5VD4XMG-NOV6TK5-MVT2Z57-5LPWSDM-H45OI67-3ALLEQV at 192.168.1.158:22000-192.168.1.170:22000/tcp-server/TLS1.3-TLS_CHACHA20_POLY1305_SHA256
[PVCOA] 11:24:58 INFO: Device B3TV5JK-7WSO5HH-5VD4XMG-NOV6TK5-MVT2Z57-5LPWSDM-H45OI67-3ALLEQV client is "syncthing v1.12.1" named "SyncThingSrv" at 192.168.1.158:22000-192.168.1.170:22000/tcp-server/TLS1.3-TLS_CHACHA20_POLY1305_SHA256
[PVCOA] 11:28:33 INFO: quic://0.0.0.0:22000 detected NAT type: Full cone NAT
[PVCOA] 11:28:33 INFO: quic://0.0.0.0:22000 resolved external address quic://95.232.25.97:64594 (via stun.syncthing.net:3478)
[PVCOA] 11:38:06 INFO: c.S.listenerSupervisor: Exiting backoff state.
[PVCOA] 11:38:07 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) starting
[PVCOA] 11:38:08 INFO: Listen (BEP/relay): Get "https://relays.syncthing.net/endpoint": x509: certificate signed by unknown authority
[PVCOA] 11:38:08 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) shutting down
[PVCOA] 11:38:08 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) starting
[PVCOA] 11:38:08 INFO: c.S.listenerSupervisor: Failed service 'dynamic+https://relays.syncthing.net/endpoint' (1.000000 failures of 2.000000), restarting: true, error: %!s(<nil>)
[PVCOA] 11:38:09 INFO: Listen (BEP/relay): Get "https://relays.syncthing.net/endpoint": x509: certificate signed by unknown authority
[PVCOA] 11:38:09 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) shutting down
[PVCOA] 11:38:09 INFO: c.S.listenerSupervisor: Failed service 'dynamic+https://relays.syncthing.net/endpoint' (1.985104 failures of 2.000000), restarting: true, error: %!s(<nil>)
[PVCOA] 11:38:09 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) starting
[PVCOA] 11:38:09 INFO: Listen (BEP/relay): Get "https://relays.syncthing.net/endpoint": x509: certificate signed by unknown authority
[PVCOA] 11:38:09 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) shutting down
[PVCOA] 11:38:09 INFO: c.S.listenerSupervisor: Entering the backoff state.
[PVCOA] 11:38:09 INFO: c.S.listenerSupervisor: Failed service 'dynamic+https://relays.syncthing.net/endpoint' (2.961271 failures of 2.000000), restarting: false, error: %!s(<nil>)
[PVCOA] 11:43:16 INFO: Completed initial scan of receiveonly folder "Main Library" (kerq9-hpdtz)
[PVCOA] 11:51:00 INFO: c.S.listenerSupervisor: Exiting backoff state.
[PVCOA] 11:51:00 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) starting
[PVCOA] 11:51:02 INFO: Listen (BEP/relay): Get "https://relays.syncthing.net/endpoint": x509: certificate signed by unknown authority
[PVCOA] 11:51:02 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) shutting down
[PVCOA] 11:51:02 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) starting
[PVCOA] 11:51:02 INFO: c.S.listenerSupervisor: Failed service 'dynamic+https://relays.syncthing.net/endpoint' (1.000000 failures of 2.000000), restarting: true, error: %!s(<nil>)
[PVCOA] 11:51:02 INFO: Listen (BEP/relay): Get "https://relays.syncthing.net/endpoint": x509: certificate signed by unknown authority
[PVCOA] 11:51:02 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) shutting down
[PVCOA] 11:51:02 INFO: c.S.listenerSupervisor: Failed service 'dynamic+https://relays.syncthing.net/endpoint' (1.987572 failures of 2.000000), restarting: true, error: %!s(<nil>)
[PVCOA] 11:51:02 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) starting
[PVCOA] 11:51:03 INFO: Listen (BEP/relay): Get "https://relays.syncthing.net/endpoint": x509: certificate signed by unknown authority
[PVCOA] 11:51:03 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) shutting down
[PVCOA] 11:51:03 INFO: c.S.listenerSupervisor: Entering the backoff state.
[PVCOA] 11:51:03 INFO: c.S.listenerSupervisor: Failed service 'dynamic+https://relays.syncthing.net/endpoint' (2.960581 failures of 2.000000), restarting: false, error: %!s(<nil>)
[PVCOA] 12:05:30 INFO: c.S.listenerSupervisor: Exiting backoff state.
[PVCOA] 12:05:31 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) starting
[PVCOA] 12:05:32 INFO: Listen (BEP/relay): Get "https://relays.syncthing.net/endpoint": x509: certificate signed by unknown authority
[PVCOA] 12:05:32 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) shutting down
[PVCOA] 12:05:32 INFO: c.S.listenerSupervisor: Failed service 'dynamic+https://relays.syncthing.net/endpoint' (1.000000 failures of 2.000000), restarting: true, error: %!s(<nil>)
[PVCOA] 12:05:32 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) starting
[PVCOA] 12:05:33 INFO: Listen (BEP/relay): Get "https://relays.syncthing.net/endpoint": x509: certificate signed by unknown authority
[PVCOA] 12:05:33 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) shutting down
[PVCOA] 12:05:33 INFO: c.S.listenerSupervisor: Failed service 'dynamic+https://relays.syncthing.net/endpoint' (1.980551 failures of 2.000000), restarting: true, error: %!s(<nil>)
[PVCOA] 12:05:33 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) starting
[PVCOA] 12:05:34 INFO: Listen (BEP/relay): Get "https://relays.syncthing.net/endpoint": x509: certificate signed by unknown authority
[PVCOA] 12:05:34 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) shutting down
[PVCOA] 12:05:34 INFO: c.S.listenerSupervisor: Entering the backoff state.
[PVCOA] 12:05:34 INFO: c.S.listenerSupervisor: Failed service 'dynamic+https://relays.syncthing.net/endpoint' (2.947724 failures of 2.000000), restarting: false, error: %!s(<nil>)
[monitor] 12:14:12 INFO: Syncthing exited: signal: killed
After this, crashes became more and more frequent, every 2 minutes or so, but the log is similar.
Anyway, now I disabled basically everything except “local discovery”.
At the moment the local state is ~90GB and there are still ~60GB to go, but I’m not sure it will survive an “unpause”.
Mem: 401748K used, 4468K free, 19724K shrd, 24376K buff, 284312K cached
CPU: 5% usr 14% sys 60% nic 0% idle 0% io 0% irq 19% sirq
Load average: 2.44 2.39 2.06 3/115 4719
PID PPID USER STAT VSZ %VSZ %CPU COMMAND
1785 1779 root SN 795m 200% 82% ./syncthing -no-browser -paused
1779 426 root T 793m 200% 0% ./syncthing -no-browser -paused
Would for example wipe the db help somehow?