File with question mark in name not syncing

Hello

I have been experiencing an issue recently where a file with a question mark in its name refuses to sync between my computer and my Android phone. This issue appeared after I renamed some files, removing " characters from them. After this operation, one of these files did not re-sync and would not re-sync after several days and even after removing the directory and re-adding it on the Android device. It is the only file in the directory which contains a question mark (“?”) which leads me to believe that caused the issue. When I remove the question mark, the file syncs normally. Please see attached the screenshots below.

Before changing filename - computer

Bildschirmfoto vom 2024-09-12 08-48-36 After changing filename - computer

Before changing filename - phone

After changing filename - phone

Is this a bug or a misconfiguration on my part?

Please note that the wrong amount of files being shown on the Android version is an issue created by (I think) re-adding the directory and the way Android handles trashed files.

Android doesn’t support characters like "*/:<>?\| in filenames (see https://stackoverflow.com/questions/2679699/what-characters-allowed-in-file-names-on-android/64021421#64021421). If you want to sync those files to an Android device, you do need to remove these from their names.

Also, when taking screenshots on Android, please use the Web GUI (available from the left slide-out menu). The app’s interface shows only limited information in comparison to the full Web GUI.

1 Like

Alright, thanks for the clarification. Weird that it worked before, though.

That’s certainly possible if it’s either a different phone from before when it worked, or the Android version has been upgraded.

Note that technically speaking, Android OS internally has no issues with filenames/directories containing a question mark or any of the other characters from the following forbidden set: ? " / \ < > * | :

See also: Overview of FAT, HPFS, and NTFS File Systems

(In FAT32, when a file is deleted, the first character of a filename is changed to a question mark to flag the storage space used by the file as being available to be overwritten.)

But for portability and legacy reasons, Android typically formats user data storage (internal and SD card) with a FAT32 filesystem. If it didn’t, non-technical users who move a SD card to a laptop or other device running Windows and/or macOS wouldn’t be able to access the files without special software.

(In addition to portability, non-journaling filesystems such as FAT32 also cause less wear and tear on flash media.)

One of my Motorola phones running a non-rooted OEM Android allows filenames with colons, question marks, asterisks and so on for the internal storage, which in turn is synced by Syncthing to/from my Linux laptop without issues. However, if I format a SD card with FAT32, the usual restrictions apply.

I used to think the same, and this used to be the case in older Android versions, but it isn’t true any more in Android 11 and newer when it comes to the internal storage (see https://github.com/syncthing/syncthing/issues/9227#issuecomment-1816043530). However, regardless of the actual underlying filesystem, Android still applies the same restrictions to the user storage. The “illegal” characters are actually hard-coded in the Android source code (see https://cs.android.com/android/platform/superproject/+/master:frameworks/base/core/java/android/os/FileUtils.java;l=972?q=isValidFatFilenameChar).

Of course, this is for AOSP, so individual manufacturers (or custom ROMs, which is more likely) are free to modify the restrictions as they please. The ones like Google, Samsung, etc. don’t do that though, so the filenames are restricted on those. I cannot test other phone manufacturers myself, but in my experience of using Android for 14 years or so, I was able to use these characters only once, and that was on a single custom ROM. Everywhere else, they were forbidden.

That’s mostly a historical thing now. Flash storage without wear-levelling is rather uncommon these days.

Very true, but it’s still a common concern because wear-leveling just spreads out the damage more evenly. :smirk:

FAT32 has been superseded by F2FS on some mobile devices (last time I checked, Google Pixels are using it). Other filesystems including Btrfs and NTFS have flash-friendly features to reduce wear and even lend a helping hand for wear-leveling.

I was surprised when I stumbled across it being allowed on a stock OEM phone. I personally avoid including characters that might not be portable, but had inadvertently created one file with a colon because it was generated by a script (the filename included a timestamp). If I hadn’t noticed a difference of one file reported by Syncthings web UI, it likely would’ve gone unnoticed for a long time.

I haven’t seen it work on a Pixel or Galaxy, but it does on my Motorola Moto-series phone with a stock OEM ROM and one of my USB PC-on-a-stick running Android (also with an OEM ROM). I’ve created files with colons, question marks, asterisks, etc. If the devices were rooted I’d take a closer look.

None of the CyanogenMod, LineageOS or other distros derived from AOSP that I’ve tried so far support it.