Hi,
I am running syncthing on centos8 using the latest release of v1.27.3.
When I start the syncthing it randomly freezes even though it’s not scaning or no directories added for syncing.
The freeze can be seen on the video below.
It freezes the ssh session as well as the syncthing GUI.
It can be seen in video that it just prints the output as it suppose to. It does look like an io hang.
I cross-checked the hangs with iotop:
With syncthing (freeze):
I don’t know if it’s a coincidence but with syncthing, mdmon can be seen doing a write operation.
The whole system is unresponsive, syncthing gui connection also gets disrupted. My gut feeling is that there is some blocking io job happening but dunno
As a user-level process, it’s unlikely that Syncthing itself could directly cause a Linux system to hang. It’s often the case that it’s due to a hardware issue.
If the problem is easy to reproduce, after the system hangs and has been rebooted, look in the systemd journal for any clues via the following command:
Looking at the journalctl, I only see my logins and sudo usage,
-- Subject: Session 30 has been terminated
-- Defined-By: systemd
-- Support: https://support.oracle.com
-- Documentation: https://www.freedesktop.org/wiki/Software/systemd/multiseat
--
-- A session with the ID 30 has been terminated.
Mar 05 08:33:15 mycomputer sshd[41732]: Accepted publickey for bar from 10.10.10.250 port 60818 ssh2: RSA SHA256:0ztp470oOPcskllq/2tajRrHoZkZkYK7Bvhtr8v6sUc
Mar 05 08:33:15 mycomputer systemd-logind[2398]: New session 31 of user bar.
-- Subject: A new session 31 has been created for user bar
-- Defined-By: systemd
-- Support: https://support.oracle.com
-- Documentation: https://www.freedesktop.org/wiki/Software/systemd/multiseat
--
-- A new session with the ID 31 has been created for the user bar.
--
-- The leading process of the session is 41732.
Mar 05 08:33:15 mycomputer systemd[1]: Started Session 31 of user bar.
-- Subject: Unit session-31.scope has finished start-up
-- Defined-By: systemd
-- Support: https://support.oracle.com
--
-- Unit session-31.scope has finished starting up.
--
-- The start-up result is done.
Mar 05 08:33:15 mycomputer sshd[41732]: pam_unix(sshd:session): session opened for user bar by (uid=0)
Mar 05 08:33:42 mycomputer sudo[41825]: bar : TTY=pts/0 ; PWD=/home/bar ; USER=root ; COMMAND=/bin/journalctl -xa
Mar 05 08:33:42 mycomputer sudo[41825]: pam_unix(sudo:session): session opened for user root by bar(uid=0)
Mar 05 08:33:42 mycomputer sudo[41825]: pam_unix(sudo:session): session closed for user root
Mar 05 08:33:54 mycomputer sudo[41828]: bar : TTY=pts/0 ; PWD=/home/bar ; USER=root ; COMMAND=/bin/journalctl -xa
Mar 05 08:33:54 mycomputer sudo[41828]: pam_unix(sudo:session): session opened for user root by bar(uid=0)
Was the output redacted? After a recent reboot, there should normally be quite a few Linux kernel log entries.
Assuming no redactions and systemd’s journal isn’t being cleared shortly after boot time, the mystery system hangs might be caused by filesystem corruption. Either way, it’d be worth running fsck to rule it out.
Indeed, it was redacted. I thought it was like dmesg, so I just tailed it.
I rebooted twice one with sycnthing service enabled and one with sycnthing service disabled and gathered the both journalctl -xa outputs read through them in nvim -d and saw no difference.
Solution
I was digging around the syncthing configuration; rm -rf’ed the .local/state/syncthing directory. Have the sycthing recreate the configuration and freezes were gone.
