Syncthing stalled due to full disk; however the disk is not full, just confusingly mounted

I’m running Syncthing on a Raspberry Pi 4. It’s syncing files to an external SSD (1TB). There’s currently 53GB of data synced to the drive. However, Syncthing has stalled:

 2022-02-21 12:18:51: Error on folder "Music Downloads" (2cdrv-nsxe9): insufficient space on disk for database (/home/pi/.config/syncthing/index-v0.14.0.db): 1.0 % < 1 %

 2022-02-21 12:19:13: Error on folder "Music" (p4i5j-wuagt): insufficient space on disk for database (/home/pi/.config/syncthing/index-v0.14.0.db): 1.0 % < 1 %

2022-02-21 13:01:04: Error on folder "Music for pi jellyfin server" (hfeqp-kxgcx): insufficient space on disk for database (/home/pi/.config/syncthing/index-v0.14.0.db): 0.5 % < 1 %

I believe this is because the Pi isn’t seeing the SSD as a mounted drive:

$ df
Filesystem     1K-blocks     Used Available Use% Mounted on
/dev/root       61249276 58462148    263976 100% /
devtmpfs         1709568        0   1709568   0% /dev
tmpfs            1841664        0   1841664   0% /dev/shm
tmpfs             736668     4704    731964   1% /run
tmpfs               5120        4      5116   1% /run/lock
/dev/mmcblk0p1    258095    49436    208660  20% /boot
tmpfs             368332        0    368332   0% /run/user/1000
$ blkid
/dev/mmcblk0p1: LABEL_FATBOOT="boot" LABEL="boot" UUID="5484-8F96" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="d20c66d5-01"
/dev/mmcblk0p2: LABEL="rootfs" UUID="64bc182a-ca9d-4aa1-8936-d2919863c22a" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="d20c66d5-02"

But instead as storage integrated directly to the Pi under /media/[drive name]:

$ df
Filesystem     1K-blocks     Used Available Use% Mounted on
/dev/root       61249276 58445108    281016 100% /
devtmpfs         1709568        0   1709568   0% /dev
tmpfs            1841664        0   1841664   0% /dev/shm
tmpfs             736668     1952    734716   1% /run
tmpfs               5120        4      5116   1% /run/lock
/dev/mmcblk0p1    258095    49436    208660  20% /boot
tmpfs             368332        0    368332   0% /run/user/1000/

What can I do to get the Pi to see the external drive as such, and stop falsely reporting /dev/root as full? This needs to take into account that the Pi is also running Jellyfin and Samba shares, which has required a little trickery with permissions (and I’m not sure I fully understand how I fixed that).

I don’t see any external storage in your df output.

I don’t think it matters how much space is left on the external SDD, because the errors are about the database, which is normally located in paths described under https://docs.syncthing.net/users/config.html.

This would mean /home/pi/.config/syncthing in your case. Which storage is that located in exactly?

2 Likes

Yeah, that was what confused me, as the Pi doesn’t boot when the SSD isn’t connected. Instead of thinking “the drive isn’t mounted” I thought “the drive is being mounted but shown only as it’s mount point, not as the drive itself”.

Just started everything over, df listed the drive as mounted, set up Sync thing from scratch. Stalled with an input/output error; checking again showed that Syncthing was still writing to the SD card. There’s a bigger problem here, either with the fstab or at the hardware level, I think

As Tomasz86 says, the error is that there isn’t a safe amount of free space (according to Syncthing defaults) in /home, which looks like it’s / on your setup.

I’m 99% sure this was a hardware error. There was definitely free space on the SSD, I checked it on another device. The problem was that the SSD wasn’t staying mounted; but Syncthing kept writing to its mount point, but as a path on the SD card, resulting in the SD card filling up with the new data and the data remaining in a previous state on the SSD.

The SSD was in a USB3 to SATA caddy. And the cadddy was the point of failure. I wiped the SSD clean and tried to copy a bunch of data over to it, but after about 10 minutes of copying got a read-only error.

Syncthing is saying that the place where the database is stored has less than 1% free, which is the default threshold where we consider it dangerously full. (There’s some rounding going on; the “1.0% < 1%” is something like “0.999% < 1%”).

Corroborating:

There’s 263976 / 61249276 = 0.43% free at this point in time. Presumably some more space was used after Syncthing stopped syncing.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.