caseSensitiveFS=false (the default) scan performance penalty scales up, and it can be extreme

If I remember right, CI produces apks from PRs. So you can update the syncthing submodule with that patch and PR that to the android repo. Otherwise of you have docker, building is very simple.

The listdir cache is intended to prevent that behaviour, but I suspect the cache time is too low. I think I mentioned that as some point.

1 Like

Sorry for necromancing this thread, but I finally managed to resurrect my old PR:

Feel free to test the current main branch to see if it has a meaningful impact for case-insensitive scanning :slightly_smiling_face:

1 Like

Thanks a lot for thinking about me, I’ll give it a test performance-wise (I hope I manage somehow to find all the one-liners for the previous test…); given that the use-case I’m mostly interested in is Android (due to unbelievably bad performance for just walking the directory) I’m migrating a bunch of setups I have to syncthing-fork (as the official Android Syncthing is frozen quite a bit in the past) and as soon as I see some updated syncthing-fork version (presumably pulling the discussed fixes) I’ll re-test everything.

I recently got a Samsung Galaxy Android tablet and file scanning was brutally slow. I also have an Android phone where file scanning on the same folder was much faster, which I found weird since the tablet has a much faster CPU.

I decided to look into it and found this post after some time. The speedup from setting caseSensitiveFS to true was massive on the tablet. I’m not running Syncthing all the time on my Android devices because of the battery usage and have them set to sync while charging. But if I want to access a file between charging I start Syncthing manually causing a full file scan and it will not sync until that is done. That makes the slow file scanning especially annoying for me since this is a situation where I want to quickly sync some files and stop Syncthing again.

I think the potential performance hit should be more clearly documented. It would also benefit more people if it’s possible to detect if a folder is case sensitive and automatically disable the safety checks.

Here are the scanning times with caseSensitiveFS on and off. The folder has around 27000 files, 350 folders and is 120GB. Syncthing v2.0.11 (64-bit ARM).

Samsung Galaxy tablet:

  • 1230s (or about 20min) with caseSensitiveFS false
  • 10s with caseSensitiveFS true (123x faster!)

Android phone:

  • 21s with caseSensitiveFS false
  • 16s with caseSensitiveFS true

P.S. Thank you so much for making and maintaining Syncthing! :blue_heart:

That’s a massive difference, but please keep in mind that Android’s user storage is case-insensitive, so enabling this option will eventually very likely lead to data loss at some point.

1 Like

You are right. I thought that Androids storage was case sensitive unless external media like an SD card was used. But turns out that internal app storage is case sensitive, but the storage shared between apps (/storage/emulated) is case insensitive.

After reading more up on the issue I can also see why Syncthing doesn’t want to detect if a filesystem is case sensitive/insensitive.

I still wonder why the difference between the scanning time on my Android tablet and my Android phone is so big. The performance hit on my phone is totally acceptable, but on the tablet it’s a problem.