False detection of read-only filesystem

Support
#1

I have a system that has a very slow RAID that is also generally not very fast, and sometimes the filesystem would take quite some time to respond. Syncthing is a little bit too eager to declare the filesystem read-only (when it fails to chmod a file in what Syncthing thinks is a reasonable amount of time) and show the folder as out of sync. Even worse, Syncthing never retries these cases, not even upon rescans, so restarting the whole Syncthing seems to be the only way to resume normal operation.

A bug? An unfortunate combination of conditions? Some misconfiguration?

#2

We don’t do any form of read only fs detection, and all failing sync operations should be retried indefinitely (though possibly with increasing delays in between). Explain what you’re actually seeing?

#3

Folder goes “Out of sync”, clicking on “Failed items” has items that list “read-only filesystem” as the reason of failure. Hovering over “read-only filesystem” produces a tooltip that I can’t at the moment quote exactly, but something along the lines of “read only filesystem, chmod /some/file failed”.

#4

Your filesystem has been remounted read only by the OS, probably due to an I/O error. No amount of retries from Syncthing will change this.

#5

I wouldn’t open this thread if that was the case. Of course I checked that the filesystem in question is RW.

Also, restarting Syncthing resumes normal operation.

#6

Nonetheless, we have no logic that does what you describe. “Read only fs”, is the error from the OS.

#7

So you’re saying that the OS said the FS was RO at some point, and then resumed normal operation? Not entirely impossible, I need to dig into this more.

But in any case, Syncthing never retries these until I restart it. That normal?

#8

Yes. Lack of retries is unexpected, I think I/o errors should be retried forever in intervals but after a while be done as seldom as once an hour.

#9

Definitely went at least 6 hours without retries in the latest observed episode on my system.

#10

Got hit by this again. The screenshot is made at about 07:50 local time, so the last scan is less than 30 minutes old.

image

And it’s a read-only filesystem according to syncthing:

image
image910×404 30.3 KB

except it’s not:

image
image713×39 7.15 KB

Judging by the system’s logs, /dev/md0 has been read-only for several minutes during the boot process (something to do with the fact that the system is quite underpowered for that software RAID to work fast), but has been RW for the last 11 hours at least.

Restarting Syncthing from the web interface made it resume normal operations on this folder.

#11

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.