syncthing 0.14.2 crashes after minutes while talking with peer

Updated one device offline to 0.14.2, took 24 hours to reindex. Now another device is online and building his index. Since then syncthing crashes and reboots. I think this is related to v0.14.2 crashes a few minutes after start #3448

Panic : Panic at 2016-07-27T16:30:20+02:00 panic: writing to temporary database: write /tmp/syncthing-db.245441880/000100.ldb: no space left on device

goroutine 170 [running]:
panic(0x5e5490, 0x12113740)
    /usr/local/go/src/runtime/panic.go:481 +0x330
github.com/syncthing/syncthing/lib/model.(*onDiskIndexSorter).Append(0x111dd3a0, 0x112ccf80, 0x32, 0x0, 0x1303f17, 0x0, 0x1ff, 0x55be0ee4, 0x0, 0x10000, ...)
    /home/buildslave/workspace/syncthing-release/src/github.com/syncthing/syncthing/lib/model/sorter.go:166 +0x3b8
github.com/syncthing/syncthing/lib/model.(*autoSwitchingIndexSorter).Append(0x12d16188, 0x112ccf80, 0x32, 0x0, 0x1303f17, 0x0, 0x1ff, 0x55be0ee4, 0x0, 0x10000, ...)
    /home/buildslave/workspace/syncthing-release/src/github.com/syncthing/syncthing/lib/model/sorter.go:65 +0x21c
github.com/syncthing/syncthing/lib/model.sendIndexTo.func1(0x3119a9d8, 0x11b80690, 0xc084b8)
    /home/buildslave/workspace/syncthing-release/src/github.com/syncthing/syncthing/lib/model/model.go:1273 +0x1ec
github.com/syncthing/syncthing/lib/db.nativeFileIterator.func1(0x3119a9d8, 0x11b80640, 0x2000)
    /home/buildslave/workspace/syncthing-release/src/github.com/syncthing/syncthing/lib/db/set.go:323 +0x1fc
github.com/syncthing/syncthing/lib/db.(*Instance).withHave(0x1121dce0, 0x10fa7d70, 0xb, 0xb, 0x10fa7de4, 0x20, 0x20, 0x0, 0x0, 0x0, ...)
    /home/buildslave/workspace/syncthing-release/src/github.com/syncthing/syncthing/lib/db/leveldb_dbinstance.go:260 +0x54c
github.com/syncthing/syncthing/lib/db.(*FileSet).WithHave(0x1115eae0, 0xffffffff, 0xffffffff, 0xffffffff, 0xffffffff, 0xffffffff, 0xffffffff, 0xffffffff, 0xffffffff, 0x10e0b1a0)
    /home/buildslave/workspace/syncthing-release/src/github.com/syncthing/syncthing/lib/db/set.go:194 +0x22c
github.com/syncthing/syncthing/lib/model.sendIndexTo(0x0, 0x0, 0x30f79100, 0x12ce44c0, 0x12f96784, 0xb, 0x1115eae0, 0x12069d70, 0x0, 0x0, ...)
    /home/buildslave/workspace/syncthing-release/src/github.com/syncthing/syncthing/lib/model/model.go:1275 +0x3bc
github.com/syncthing/syncthing/lib/model.sendIndexes(0x30f79100, 0x12ce44c0, 0x12f96784, 0xb, 0x1115eae0, 0x12069d70, 0x0, 0x0)
    /home/buildslave/workspace/syncthing-release/src/github.com/syncthing/syncthing/lib/model/model.go:1214 +0x49c
created by github.com/syncthing/syncthing/lib/model.(*Model).ClusterConfig
    /home/buildslave/workspace/syncthing-release/src/github.com/syncthing/syncthing/lib/model/model.go:733 +0x1244

Corresponding (redacted) log :

sh-4.3$ /usr/local/syncthing/bin/syncthing -logflags="11" -verbose -audit --home /usr/local/syncthing/var/
[monitor] 2016/07/27 16:27:42 cmd/syncthing/monitor.go:94: INFO: Starting syncthing
[start] 2016/07/27 16:27:42 cmd/syncthing/main.go:914: INFO: Audit log in /usr/local/syncthing/var//audit-20160727-162742.log
[SRQKF] 2016/07/27 16:27:42 cmd/syncthing/main.go:569: INFO: syncthing v0.14.2 "Dysprosium Dragonfly" (go1.6.3 linux-arm) unknown-user@lx-buildslave.syncthing.net 2016-07-26 11:55:25 UTC
[SRQKF] 2016/07/27 16:27:42 cmd/syncthing/main.go:570: INFO: My ID: MY_ID
[SRQKF] 2016/07/27 16:27:43 cmd/syncthing/main.go:851: INFO: Single thread hash performance is ~7.4 MB/s
[SRQKF] 2016/07/27 16:27:43 cmd/syncthing/verboseservice.go:48: VERBOSE: Starting up (/usr/local/syncthing/var/)
[SRQKF] 2016/07/27 16:29:28 lib/model/model.go:237: INFO: Ready to synchronize SharedFolder (readwrite)
[SRQKF] 2016/07/27 16:29:28 lib/model/model.go:237: INFO: Ready to synchronize SharedFolderID (readwrite)
[SRQKF] 2016/07/27 16:29:28 cmd/syncthing/main.go:705: INFO: Using discovery server https://discovery-v4-1.syncthing.net/v2/?id=SR7AARM-TCBUZ5O-VFAXY4D-CECGSDE-3Q6IZ4G-XG7AH75-OBIXJQV-QJ6NLQA
[SRQKF] 2016/07/27 16:29:28 cmd/syncthing/main.go:705: INFO: Using discovery server https://discovery-v4-2.syncthing.net/v2/?id=DVU36WY-H3LVZHW-E6LLFRE-YAFN5EL-HILWRYP-OC2M47J-Z4PE62Y-ADIBDQC
[SRQKF] 2016/07/27 16:29:28 cmd/syncthing/main.go:705: INFO: Using discovery server https://discovery-v4-3.syncthing.net/v2/?id=VK6HNJ3-VVMM66S-HRVWSCR-IXEHL2H-U4AQ4MW-UCPQBWX-J2L2UBK-NVZRDQZ
[SRQKF] 2016/07/27 16:29:28 cmd/syncthing/main.go:705: INFO: Using discovery server https://discovery-v6-1.syncthing.net/v2/?id=SR7AARM-TCBUZ5O-VFAXY4D-CECGSDE-3Q6IZ4G-XG7AH75-OBIXJQV-QJ6NLQA
[SRQKF] 2016/07/27 16:29:28 cmd/syncthing/main.go:705: INFO: Using discovery server https://discovery-v6-2.syncthing.net/v2/?id=DVU36WY-H3LVZHW-E6LLFRE-YAFN5EL-HILWRYP-OC2M47J-Z4PE62Y-ADIBDQC
[SRQKF] 2016/07/27 16:29:28 cmd/syncthing/main.go:705: INFO: Using discovery server https://discovery-v6-3.syncthing.net/v2/?id=VK6HNJ3-VVMM66S-HRVWSCR-IXEHL2H-U4AQ4MW-UCPQBWX-J2L2UBK-NVZRDQZ
[SRQKF] 2016/07/27 16:29:28 lib/connections/tcp_listen.go:69: INFO: TCP listener ([::]:22000) starting
[SRQKF] 2016/07/27 16:29:28 cmd/syncthing/verboseservice.go:48: VERBOSE: Folder "SharedFolderID" is now scanning
[SRQKF] 2016/07/27 16:29:28 cmd/syncthing/verboseservice.go:48: VERBOSE: Folder "SharedFolder" is now scanning
[SRQKF] 2016/07/27 16:29:28 lib/model/rwfolder.go:299: INFO: Completed initial scan (rw) of folder SharedFolderID
[SRQKF] 2016/07/27 16:29:28 cmd/syncthing/verboseservice.go:48: VERBOSE: Folder "SharedFolderID" is now idle
[SRQKF] 2016/07/27 16:29:29 cmd/syncthing/main.go:750: INFO: Device OTHER_DEVICE_ID_1 is "OTHER_DEVICE_1" at [dynamic]
[SRQKF] 2016/07/27 16:29:29 cmd/syncthing/main.go:750: INFO: Device OTHER_DEVICE_ID_2 is "OTHER_DEVICE_2" at [dynamic]
[SRQKF] 2016/07/27 16:29:29 cmd/syncthing/main.go:750: INFO: Device OTHER_DEVICE_ID_3 is "OTHER_DEVICE_3" at [dynamic]
[SRQKF] 2016/07/27 16:29:29 cmd/syncthing/main.go:750: INFO: Device MY_ID is "MY_DEVICE" at [dynamic]
[SRQKF] 2016/07/27 16:29:29 cmd/syncthing/main.go:750: INFO: Device OTHER_DEVICE_ID_4 is "OTHER_DEVICE_4" at [dynamic]
[SRQKF] 2016/07/27 16:29:29 cmd/syncthing/usage_report.go:264: INFO: Starting usage reporting
[SRQKF] 2016/07/27 16:29:29 cmd/syncthing/verboseservice.go:48: VERBOSE: Startup complete
[SRQKF] 2016/07/27 16:29:29 cmd/syncthing/gui.go:322: INFO: GUI and API listening on [::]:7070
[SRQKF] 2016/07/27 16:29:29 cmd/syncthing/gui.go:323: INFO: Access the GUI via the following URL: http://127.0.0.1:7070/
[SRQKF] 2016/07/27 16:29:30 lib/discover/local.go:159: WARNING: Incompatible (v0.13) local discovery packet from 192.168.1.101:34129 - upgrade that device to connect
[SRQKF] 2016/07/27 16:29:30 lib/discover/local.go:159: WARNING: Incompatible (v0.13) local discovery packet from [SOME_ADDRESS]:48207 - upgrade that device to connect
[SRQKF] 2016/07/27 16:29:31 lib/connections/service.go:619: WARNING: Connecting to "OTHER_DEVICE_3" (syncthing v0.13.10, OTHER_DEVICE_ID_3): the remote device speaks an older version of the protocol (v0.13) not compatible with this version
[SRQKF] 2016/07/27 16:29:33 lib/connections/service.go:252: INFO: Established secure connection to OTHER_DEVICE_ID_1 at 192.168.1.100:22000-IP_DEVICE_1:50281 (TCP (Server))
[SRQKF] 2016/07/27 16:29:33 lib/model/model.go:1105: INFO: Device OTHER_DEVICE_ID_1 client is "syncthing v0.14.2" named "OTHER_DEVICE_NAME_1"
[SRQKF] 2016/07/27 16:29:33 cmd/syncthing/verboseservice.go:48: VERBOSE: Connected to device OTHER_DEVICE_ID_1 at IP_DEVICE_1:50281 (type TCP (Server))
[SRQKF] 2016/07/27 16:29:34 lib/model/model.go:697: INFO: Device OTHER_DEVICE_ID_1 folder "SharedFolder" is delta index compatible (mlv=0)
[SRQKF] 2016/07/27 16:29:36 lib/connections/service.go:619: WARNING: Connecting to "OTHER_DEVICE_2" (syncthing v0.13.10, OTHER_DEVICE_ID_2): the remote device speaks an older version of the protocol (v0.13) not compatible with this version
[SRQKF] 2016/07/27 16:29:37 lib/connections/service.go:205: INFO: Connected to already connected device (OTHER_DEVICE_ID_1)
[SRQKF] 2016/07/27 16:29:38 cmd/syncthing/verboseservice.go:48: VERBOSE: Device OTHER_DEVICE_ID_1 sent an index update for "SharedFolder" with 24 items
[SRQKF] 2016/07/27 16:29:38 cmd/syncthing/verboseservice.go:48: VERBOSE: Device OTHER_DEVICE_ID_1 sent an index update for "SharedFolder" with 36 items
[SRQKF] 2016/07/27 16:29:39 cmd/syncthing/verboseservice.go:48: VERBOSE: Device OTHER_DEVICE_ID_1 sent an index update for "SharedFolder" with 69 items
[SRQKF] 2016/07/27 16:29:40 cmd/syncthing/verboseservice.go:48: VERBOSE: Device OTHER_DEVICE_ID_1 sent an index update for "SharedFolder" with 31 items
[SRQKF] 2016/07/27 16:29:41 cmd/syncthing/verboseservice.go:48: VERBOSE: Device OTHER_DEVICE_ID_1 sent an index update for "SharedFolder" with 31 items
[SRQKF] 2016/07/27 16:29:41 cmd/syncthing/verboseservice.go:48: VERBOSE: Device OTHER_DEVICE_ID_1 sent an index update for "SharedFolder" with 12 items
[SRQKF] 2016/07/27 16:29:42 lib/relay/client/static.go:92: INFO: Joined relay relay://138.201.156.90:22067
[SRQKF] 2016/07/27 16:29:48 cmd/syncthing/verboseservice.go:48: VERBOSE: Listen address dynamic+https://relays.syncthing.net/endpoint resolution has changed: lan addresses: [relay://138.201.156.90:22067/?id=XYWRY6J-I4PVS76-OYBJEK5-MMWAFOQ-4DFHD45-6WVDXVN-N63DPXD-CAZ56AD&pingInterval=1m0s&networkTimeout=2m0s&sessionLimitBps=0&globalLimitBps=300000&statusAddr=:22070&providedBy=aucubin] wan addresses: [relay://138.201.156.90:22067/?id=XYWRY6J-I4PVS76-OYBJEK5-MMWAFOQ-4DFHD45-6WVDXVN-N63DPXD-CAZ56AD&pingInterval=1m0s&networkTimeout=2m0s&sessionLimitBps=0&globalLimitBps=300000&statusAddr=:22070&providedBy=aucubin]
[SRQKF] 2016/07/27 16:29:50 cmd/syncthing/verboseservice.go:48: VERBOSE: Device OTHER_DEVICE_ID_1 sent an index update for "SharedFolder" with 18 items
[SRQKF] 2016/07/27 16:29:51 lib/nat/structs.go:32: INFO: New NAT port mapping: external TCP address MY_IP:60258 to local address 0.0.0.0:22000.
[SRQKF] 2016/07/27 16:29:51 lib/nat/service.go:61: INFO: Detected 2 NAT devices
[SRQKF] 2016/07/27 16:29:51 cmd/syncthing/verboseservice.go:48: VERBOSE: Listen address tcp://0.0.0.0:22000 resolution has changed: lan addresses: [tcp://0.0.0.0:22000] wan addresses: [tcp://0.0.0.0:22000 tcp://0.0.0.0:60258 tcp://0.0.0.0:60258]
[SRQKF] 2016/07/27 16:30:15 cmd/syncthing/verboseservice.go:48: VERBOSE: Device OTHER_DEVICE_ID_1 sent an index update for "SharedFolder" with 13 items
panic: writing to temporary database: write /tmp/syncthing-db.245441880/000100.ldb: no space left on device
[monitor] 2016/07/27 16:30:20 cmd/syncthing/monitor.go:188: WARNING: Panic detected, writing to "/usr/local/syncthing/var//panic-20160727-163020.log"
[monitor] 2016/07/27 16:30:20 cmd/syncthing/monitor.go:189: WARNING: Please check for existing issues with similar panic message at https://github.com/syncthing/syncthing/issues/
[monitor] 2016/07/27 16:30:20 cmd/syncthing/monitor.go:190: WARNING: If no issue with similar panic message exists, please create a new issue with the panic log attached
[monitor] 2016/07/27 16:30:20 cmd/syncthing/monitor.go:163: INFO: Syncthing exited: exit status 2

Can provide more context, logs or traces, here or in issue.

exactly the same issue as here: Syncthing on Qnap Crashes - No GUI.

solution: Increase the size of /tmp or set the environment variable TMPDIR

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.