Potentially Corrupting Exfat Filesystem?

lucidBrot · July 5, 2021, 5:45pm

I encountered an issue with my external SD-card’s filesystem being somewhat corrupted and I’m wondering if syncthing could be the culprit.

Not sure wat kind of information would be useful to determine this, but I have opened an issue on the syncthing-android fork that I’ve been using (idk what is better or worse, I just happened to get this one) and they say it’s not a problem with their app, but with the underlying syncthing usage.

github.com/Catfriend1/syncthing-android

Potentially corrupting filesystem

opened 02:14PM - 04 Jul 21 UTC

closed 02:47PM - 04 Jul 21 UTC

lucidBrot

discussion

### Description of the issue I'm not certain that syncthing is the reason for… my issues, but it seems likely to me, hence this report. I currently do not need further assistance in fixing. When I reboot my phone (LineageOS 17) it seems to be checking the external SD card for corruptions right after I unlock it the first time. It says "Phone is starting" and then does any of these: * Finishes starting and everything works perfectly well * Somehow crashes and goes into a bootloop (probably not your problem) * Finishes starting and says "Issue with SD Card" ([this notification](https://android.stackexchange.com/q/233783/96334)) and does not automatically mount it. I've encountered this last case three or four times in the last year and the fixes have been so far: * Just rebooting until the message goes away * Running `fsck.exfat -y /dev/block/mmcblk0p1` * If that also did not work, running `chkdisk H: /F` on a windows machine with the sd card mounted on `H:`. Here's why I think syncthing has to do with this: * I am using syncthing to sync to places on the external sdcard * The output of `fsck.exfat` said there was a `.gif` file in my synced folder that was empty but "marked as contiguous" * The output of `chkdisk` contained several errors that `fsck` did not find, stating that there was more than one entry for some of the files. They were all in the same subfolder, but only half of the files in that subfolder had this issue. ### Reproduction Steps I don't think this is reliably reproducible. ### Version Information - App Version: (what is this?) - Syncthing-Fork Wrapper Version: v1.16.0.3 - Syncthing Version: v1.16.0 - Android Version: Android LineageOS 17 (`lineage_beryllium-userdebug 10 QQ3A.200805.001 c65720e3ac`) - Device manufacturer: Xiaomi- Device model: Pocophone F1 ### Device platform info ```sh [ro.product.board]: [sdm845] [ro.product.brand]: [Xiaomi] [ro.product.cpu.abi]: [arm64-v8a] [ro.product.cpu.abilist]: [arm64-v8a,armeabi-v7a,armeabi] [ro.product.cpu.abilist32]: [armeabi-v7a,armeabi] [ro.product.cpu.abilist64]: [arm64-v8a] [ro.product.device]: [beryllium] [ro.product.first_api_level]: [27] [ro.product.locale]: [en-US] [ro.product.manufacturer]: [Xiaomi] [ro.product.model]: [POCO F1] [ro.product.name]: [beryllium] [ro.product.odm.brand]: [Xiaomi] [ro.product.odm.device]: [beryllium] [ro.product.odm.manufacturer]: [Xiaomi] [ro.product.odm.model]: [POCO F1] [ro.product.odm.name]: [beryllium] [ro.product.system.brand]: [Xiaomi] [ro.product.system.device]: [beryllium] [ro.product.system.manufacturer]: [Xiaomi] [ro.product.system.model]: [POCO F1] [ro.product.system.name]: [beryllium] [ro.product.vendor.brand]: [Xiaomi] [ro.product.vendor.device]: [beryllium] [ro.product.vendor.manufacturer]: [Xiaomi] [ro.product.vendor.model]: [POCO F1] [ro.product.vendor.name]: [beryllium] ``` ### CHKDISK output ``` C:\WINDOWS\system32>chkdsk H: /F Der Typ des Dateisystems ist exFAT. Volumeseriennummer : 6F63-1277 Dateien und Ordner werden überprüft... Die Volumebezeichnung lautet android. Es gibt mehr als einen Eintrag "MY DEMONS - STARSET (LYRICS).MP3" im Ordner "\Music\2017\march\". Der Eintrag wurde nach "MY DEMONS - STARSET (LYRICS).MP3-1" umbenannt Beim Überprüfen der Dateien im Verzeichnis "\Music\2017\march\" (78) wurde eine Beschädigung festgestellt. Es gibt mehr als einen Eintrag "STARSET - LAST TO FALL (OFFICIAL AUDIO).MP3" im Ordner "\Music\2017\march\". Der Eintrag wurde nach "STARSET - LAST TO FALL (OFFICIAL AUDIO)-1" umbenannt Beim Überprüfen der Dateien im Verzeichnis "\Music\2017\march\" (84) wurde eine Beschädigung festgestellt. ``` (skipping some more files like this. There were also some that did not rename entries, and only "wurde eine Beschädigung festgestellt." And there were also errors about whatever "Querverbindung" is supposed to be . ``` Beim Überprüfen der Volumebitmap wurde eine Beschädigung festgestellt. Dateizuordnungen werden von Windows überprüft... \Android\media\com.github.catfriend1.syncthingandroid\.stversions\eric_thesis_handin\Code\measurePy\env\Lib\site-packages\sklearn\preprocessing\_function_transformer.pyc ist auf Zuordnungseinheit 426689 querverbunden. Querverbindung durch Kopieren aufgelöst \Android\media\com.github.catfriend1.syncthingandroid\.stversions\eric_thesis_handin\Code\measurePy\env\Lib\site-packages\sklearn\preprocessing\_encoders.py ist auf Zuordnungseinheit 24455 querverbunden. Querverbindung durch Kopieren aufgelöst Die Datei- und Ordnerüberprüfung ist abgeschlossen. Es wurden Korrekturen am Dateisystem vorgenommen. Es sind keine weiteren Aktionen erforderlich. 485265664 KB Speicherplatz auf dem Datenträger insgesamt 379503104 KB in 83949 Dateien 728832 KB in 5680 Indizes 0 KB in fehlerhaften Sektoren 640 KB vom System benutzt 105033088 KB auf dem Datenträger verfügbar 131072 Bytes in jeder Zuordnungseinheit 3791138 Zuordnungseinheiten auf dem Datenträger insgesamt 820571 Zuordnungseinheiten auf dem Datenträger verfügbar ``` ### Fin Thanks for having a look, if you think it's possible that syncthing is causing the filesystem checks to fail / the filesystem to become corrupted. I have no way of telling if syncthing is indeed the culprit, all I know is that all the broken files were inside a synced folder.

A summary: My phone had issues with the external SD card three or four times now. It was working until a reboot, at which point I think it performs an fsck and notices that something is wrong. Manually mounting it r/w still works, it’s just lineageOS that does not want to automatically mount it.

Anyway, the latest time I came upon this issue, fsck.exfat was not able to fix all the issues. I had to use chkdsk on windows to fix it, and that reported more corruptions. But all of them inside folders that I am syncing with syncthing.

My external SD card is Exfat, and I sync to a windows NTFS computer and a synology NAS with whatever they use (probably ext-4 or such).

Is it possible that syncthing breaks my filesystem? (For the specific errors, please see the link above to the issue.)

Thank you for any insights!
Lucid

calmh · July 5, 2021, 5:49pm

In principle, no, as filesystems and storage devices are the operating system’s responsibility. No app should ever be able to corrupt a filesystem by using the fileystem APIs.

AudriusButkevicius · July 5, 2021, 6:06pm

If it’s the same card that got corrupted, it’s most likely the card that is to blame. SD cards don’t last forever, some are knock-offs of reputable brands that perform poorly and don’t last long.

lucidBrot · July 5, 2021, 6:50pm

I think the first time it got corrupted was a different one, but sure the SD card itself could also be to blame. It’s just curious then that the broken entries were only in syncthing-synced folders.

I’m aware that “in principle”, apps can not corrupt the filesystem but many things don’t work in principle but are still possible. I figure syncthing is not doing weird low-level hacks though, if that’s the response I’m getting.

If you have any idea how I could be certain what the culprit is (despite me not knowing how to reliably reproduce breaking my FS), let me know

and otherwise, I guess “pretty sure it’s not due to syncthing” is probably the best I could hope for, so thanks!

system · August 4, 2021, 6:51pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.