"fatal error: thread exhaustion" on existing device when trying to sync new device

This is the very first time I’ve seen this kind of panic. Basically, I reinstalled the OS on one device, and then set Syncthing up and started to pull files from an existing device. However, while pushing the files to the newly configured device, the existing device keeps panicking, always with the same “fatal error: thread exhaustion” error.

I’m attaching three log files below. They are huge when unzipped. This is happening with Syncthing v1.28.0 (self-compiled with a few tweaks, mostly GUI-related though) under Windows 11. Edit: The same panic is present in v1.28.1.

I’ve only noticed that until the crash happens, Syncthing’s RAM usage keeps growing and growing, eventually taking around 3-4 GB. However, the device itself has 32 GB of RAM, and there is still a lot of it available during the crash.

Have you got any idea what may be going on here?

Looks like a lot of concurrent requests are happening.

I’ve narrowed the problem down to trying to sync two folders with a larger number of files (one with 64k files / 3.5k folders, and one with 77k files / 15k folders). As soon as I try to sync those two, Syncthing’s RAM usage starts increasing to abnormal levels, eventually resulting in the same panic.

I’ve gone through the recent commits, but I haven’t noticed anything that could potentially cause such a problem, so I’ve decided to recompile my build with Go v1.22 instead of Go v1.23.

When using Go v1.22, the problem is completely gone. In fact, even the RAM usage stays at a lower level at about 1.5 GB instead of keeping to grow beyond 3 GB, as is the case when using Go v1.23. All folders have also managed to finish syncing with no further issues.

1 Like

Interesting

I’ve found yet another panic log but from a completely different device. This one happened two weeks later than the other crashes.

I’m attaching the log below, however it’s the exact same error as reported previously.

panic-20241225-212255.zip (3.8 MB)

Just for the record, the hardware is completely different - the first system was Intel, this one is AMD. Both run Windows 11.

One thing in common is that the crash also took place when a new device connected and synced (in reality just exchanged indexes) for the very first time, and the version of Syncthing running there was also compiled with Go v1.23. The crash doesn’t occur when using Go v1.22.