I feel you, as a programmer myself, I totally get the KISS principle and the hidden complexity that arises from implementation details.
I also think this is a core problem that a tool like this should to solve one way or the other. (And I really love it already quite a lot to the point that I sent a donation. Nice job!)
Having said that, here my suggestions to your points.
We’d probably need to keep the
.syncthing.renames or whatever file in the same directory as the file in question, so that it would apply even if there are multiple Syncthing folders pointed at different levels of the same directory tree, and so that it would have a chance of coming along when a directory is renamed. We’d have to look at and possibly read that file for every file operation, which is costly
I really would put a single file into the root of a share and cache it’s content to do quick mappings for what you mentioned.
I don’t really see the issue of having such files in sub-folders in case of nested sync folders. Such files in all sub-directories should be simply ignored.
and we’d need to do it recursively upwards to the folder root (because we might have a path like
foo:/bar#/baz? where each level got renamed into
foo_/bar_ (2)/baz_ etc).
Correct, but I think this shouldn’t be too complicated given that a mapping is loaded from a single file and then each name, be it folder or file, is first checked if it has an entry in the map
When committing a file, there’s an unavoidable interval between performing the file operation and saving the
.syncthing.renames during which data loss might occur. Similarly when reading; if we look at a file name but the renames file hasn’t been written yet we’ll get bad information. We could perhaps invent some locking scheme, but that’s ugly and another layer of complexity we don’t have at the moment. In both cases we might be risking data loss or overwrites of the wrong file if we get the timing wrong, and that’s typically not acceptable.
I still think it would be totally acceptable if this would only work in SEND ONLY mode in addition with a warning “The source has files and folder names that contain characters that the target file system can not support. Should they get renamed on the target system, or excluded from sync?”
I assume that in most cases where a two way sync is necessary, the system on both ends is the same. And if we assume SEND ONLY, I think first writing the .syncthing.renames file would already be sufficient. Ofc, using rename, so .syncthing.renames.new, then, when fully written, replace the old one, then send the file. Worst thing could be dead entries in that file, because the original file was deleted in the exact wrong moment. But this could be corrected on next full scan or maybe even be ignored.
I do try to avoid the trap of having perfect be the enemy of good enough. A solution shouldn’t have to be theoretically perfect if it delivers significant value. However, when it seems like it may cause a lot of issues on top of potential confusion and there is a different solution (rename those files), then I’m a bit sceptical that it’s worth it.
Wise words, but just to give you some real data for your judgment. My Audiobook collection has 2000 file errors that I would need to rename to be able to fully sync it to my Android device. (Will end up writing a script to achieve exactly that, but mainly because I can)