Yet another Syncthing Tray

It is not expected that it uses so much memory. It is possible that Syncthing Tray or one of its dependencies leak resources. One could use valgrind to investigate this potential problem.

Note that memory freed by an application is not necessarily immediately reflected by the OS memory statistics. I once had an application with a huge in-memory db which could regenerate all the data and then throw away the old data. Throwing away the old data was not fully reflected in the OS memory statistics. However, even after weeks it never exceeded a certain memory usage after all.

I’ve checked today with valgrind under GNU/Linux and there were in fact 3 places in my code where it would leak memory. I have fixed those. However, these places can not be responsible for the growth in memory usage you’re observing. I’ve also seen some potential leaks in code of dependencies but I also doubt they would actually accumulate. I’m not sure what I can do to improve the situation.

Is there anything that I could do in Windows to help find the possible culprit? The valgrind software unfortunately seems to be available for Unix-like systems only.

Just for the record, this is what RAM usage looks like after a reboot and a few hours of work.

image

The previous screenshot was taken after several days of work though, so something must have happened in the meantime, but I cannot think about anything special right now that could have caused the possible memory leak.

The Qt Creator documentation links Windows users to Heob. Otherwise you’ll need to google yourself. It is possible that you one also need to use a debug build and sources for useful output.

Recently, I have had a problem with two devices stuck in a “Synchronizing” state.

However, at the same time, they are marked as “Up to Date” in the actual Web GUI.

It has been about a week since this first happened and they have been stuck like that since then. Restarting the PC, Syncthing, or Syncthing Tray does not change anything.

Do you have any idea what the problem may be about?

Syncthing Tray is v1.1.3 (Qt 6).

Since it differs from the official web UI and also persists after restarting Syncthing Tray it is likely a bug on my side. Without being able to reproduce it myself it is hard to tell what the problem is, though. If it would occur on my setup, I’d just use GDB to find out what makes Syncthing Tray behave like that.

It would be cool if I had implemented a feature to capture the Syncthing’s config/status/events. Then you could send me the files and I could load them locally in a mocked build. Unfortunately not done yet.

I can only say that now I have also tried to completely reset the Syncthing Tray settings and re-configure the connection from scratch. However, the problem is still here. The two devices keep being marked as “synchronizing”.

There seems to exist a GDB release for Windows. If I try running it with Syncthing Tray here, will the debug logs be of any use?

No, I’d use GDB for an interactive debugging session and it would require a debug build.

Maybe it would help if you would capture the traffic between Syncthing and Syncthing Tray with Wireshark. Of course you can’t use TLS for that and you need to make sure to redact all sensitive information before sharing it.

Today, I have had Syncthing Tray crash on one of my Windows computers.

image

The Event Viewer recorded only the following.

The program syncthingtray.exe version 1.1.3.0 stopped interacting with Windows and was closed. To see if more information about the problem is available, check the problem history in the Security and Maintenance control panel.
 Process ID: 1244
 Start Time: 01d7211ed6e0c5d1
 Termination Time: 133
 Application Path: syncthingtray.exe
 Report Id: a718d7bc-940f-11eb-89bd-40167e136d92
 Faulting package full name: 
 Faulting package-relative application ID: 

This happened when the machine was locked and idle, so there was nothing special going on there, except for Syncthing running. Unfortunately, there doesn’t seem to be anything else recorded anywhere in any Windows log files, etc. Syncthing Tray itself does not seem to be recording any logs, or does it?

No, it doesn’t record any logs. If it was Linux I’d inspect the coredump via GDB although that’s also not that helpful unless a debug build was used. Unfortunately I doubt I can do anything at this point.

Note that it is a known bug that tests currently don’t pass. It is not a regression of the new version but I have to adapt my code to Syncthing v1.15.1.

Maybe Syncthing devs know more about the change within Syncthing itself:

[12:13] Were there any changes about querying the completion of paused devs/dirs? I’m asking because my test setup for Syncthing Tray does that and I now apparently get a 404 error where I previously got 200: syncthingtray test 2 fails · Issue #102 · Martchus/PKGBUILDs · GitHub

[12:14] Not sure whether I’ve got any useful data before, though. Maybe I should just ignore the 404 error in case of paused dirs/devs or not do this request in the first place.

Seems it changed in: lib: Return error from db.FileSet.Snapshot (fixes #7419, ref #5907) (… · syncthing/syncthing@310fba4 · GitHub

It no longer returns empty completion, but returns a 404 now.

Thanks for the hint. This seems to be the culprit. Then I’ll just refrain from querying this for paused directories and devices.

To follow up on my previous idea: It looks like Syncthing itself allows to dump events via the -audit CLI option. I haven’t tried it myself but this kind of log would be useful. Maybe I can make Syncthing Tray consume such a log instead of polling for real events to ease testing/reproducing. Of course I also needed the config and answers to other REST-API calls.

I have just had the same crash today, on the very same machine. This is strange, because I had had no such crashes in the past, and the device configuration itself has not really changed since then. Syncthing Tray at the moment is v1.1.5 (Qt 6).

The crash seems to be happening with no particular reason after several days of working fine. This time it was 4 days. Also, it happens when the machine is idle, and nothing special goes on then, except for Syncthing working in the background.

Just in case, if there existed some kind of a debug build that could help diagnose this kind of problems, then I would be glad to test it :wink:.

Under GNU/Linux (with systemd) I’d just use coredumpctl to start the debugger GDB for investigating crashed applications. For better results I’d create a build with debug symbols (but maybe still with optimizations because the problem might not be reproducible anymore when turning them off).

I’m sure Windows offers something similar to investigate a crashing application. I haven’t used Windows for development for ages but I remember that it even offered one to start Visual Studio’s debugger when an application crashed. It should also be possible to conduct a debug build under Windows using GCC/mingw-w64 (e.g. via MSYS2) or Visual Studio’s C++ compiler/stl¹.

¹I’ve never been using Visual Studio to compile Syncthing Tray because as an open source project Syncthing Tray’s focus is on using open source technology. Hence you might run into configuration, compilation and runtime errors.

I have a problem with Syncthing Tray not trying to connect to Syncthing. I have both set to start on boot in Windows using the Task Scheduler. This likely means that when Syncthing Tray starts for the first time, Syncthing is not yet ready, so not being able to connect then is understandable.

However, I have also set the reconnect interval to 60000 ms, so I would expect Syncthing Tray to try to connect again later. However, it doesn’t seem to be case. Basically, Syncthing Tray seems to never re-try to connect. The only way to work around this appears to be to press “connect” manually in the GUI. Then it connects almost immediately.

The re-connect interval is a connection-specific setting so be sure that it is set for the relevant connection. Also make sure that the internal launcher’s state is not used (the check boxes in the launcher setting are unchecked). (Generally the same counts for the systemd integration.)

I have gone through the settings one more time and realised that I had forgotten to tick the “Connect automatically on startup” box on this machine :sweat_smile::sweat_smile:.

Apologies for wasting your time :pensive:.

Oh, right. In my explanation I’ve also forgotten to mention this one. No problem you’ve asked :slight_smile: