Syncthing on OSX stalls

Support
1

Hi there! I’ve just started using Syncthing about a week ago. Since two days ago, every time I come in the office the next morning, I find that Syncthing on our “main server” has stalled. The GUI loads up slow all the shares show Unknown and all the other machines are disconnected.

Just some background on my setup.

  • Syncthing version on 0.14.7
  • Syncing about 5 machines
  • Roughly 110k files and 1TB data
  • “Server” is a Hackintosh. All files stored on software mirror RAID.
  • 4 machines on OSX 10.10
  • 1 machine (the “server”) on OSX 10.6
  • Syncing over LAN as well as Internet
  • Port forwarding done

I’d love to provide more information but I have no idea where to look or where to pull the information out from. Any help would be appreciated!

1 Like
2

This sounds like it’s deadlocking for some reason. Easiest way to debug it is to run it manually and capture the output, as I’m not sure how you’re starting it today.

  1. Stop Syncthing from running using your current method

  2. Open a new terminal, run syncthing -no-restart. Leave it like that.

  3. When this happens again, open another terminal and run pkill -QUIT syncthing. It will exit with lots of trace data.

Paste that data somewhere, here will do in a pinch. :slight_smile:

1 Like
3

I think I’m seeing the same issue. I’m running on Ubuntu 14.04 systems. I’ll try what you mentioned here to generate the trace data.

4

You can also set the environment variable STDEADLOCKTIMEOUT=600 and otherwise run it as usual. It should then panic when detecting a deadlock (after ten minutes), producing a nice panic file with all the relevant data in the configuration directory.

5

So, it finally stalled, but when I run pkill I get a -bash: pkill: command not found - it turns out 10.7 doesn’t have pkill yet.

I did a regular kill but it did not exit with any trace data. I started syncthing again just to let stuff sync, so it’ll be awhile before I get another chance to kill it.

Any alternative ideas to pkill would be welcome!

I managed to figure out how to set the STDEADLOCKTIMEOUT=600. I’ll upload the panic file when I have it. :slight_smile:

6

you need to kill -QUIT not just kill, that should produce the same effect.

1 Like
7

It finally died sometime today. I suspect when one of the remote points came online. These were the last few lines. Are these the lines you guys need to troubleshoot or is there a panic file somewhere that I need to upload?

panic: deadlock detected

goroutine 42 [running]:
panic(0x5798e0, 0x420de8320)
	/usr/local/go/src/runtime/panic.go:500 +0x1a1
github.com/syncthing/syncthing/lib/model.deadlockDetect.func1(0x8bb2c97000, 0x8420f00208, 0x420fce200)
	/Users/jb/jenkins/workspace/syncthing-release-mac/src/github.com/syncthing/syncthing/lib/model/util.go:32 +0x148
created by github.com/syncthing/syncthing/lib/model.deadlockDetect
	/Users/jb/jenkins/workspace/syncthing-release-mac/src/github.com/syncthing/syncthing/lib/model/util.go:35 +0x53
8

Check if there is a panic file in syncthing’s home dir with more content or more content in the crash in general.

9

I’ve looked at the following locations, but I don’t see anything panic file or any file that looks like a log of a panic.

  • ~/Library/Application Support/Syncthing
  • /Applications/Syncthing

I should mention that this was the app running with STDEADLOCKTIMEOUT=600. I don’t know if that affects the existence of a panic log.

10

Check syncthing -help and syncthing -paths. Also make sure syncthing is not running via service manager.

11

That output, but we need all of it. There should be lots. :slight_smile:

1 Like
12

You do know go 1.7 defaults dumping only the routine panicing right?

1 Like
13

No, but I do now. You’d think that might have been mentioned somewhere in the release notes… If you knew, why didn’t you fix it already? This renders our panic traces useless. fixing…

1 Like
14

So I’ve read it, thought it will be good for most cases, apart from the deadlock detection, and never got to fixing that case… Or raising a ticket…

2 Likes
15

Just got the panic log.

http://pastebin.com/3VBddRWg

1 Like
16

Can you raise a bug with this.

17

Which category does that belong to? Development?

18

Bugs in github do not have categories I think.

19

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.