I’ve encountered data-corruption which spread to other hosts through syncthing. That can never fully be prevented as syncthing is doing the job it is designed for, but could the risk be mitigated?
One idea would be to add file content validation through use of the external versioning script, using /rest/system/shutdown to immediately stop syncthing when detecting corrupt data. Given that it is not the intended use-case, is that an advisable thing to do?
To exemplify:
During my recent incident I had fsck transform text files into fully random gibberish, and a faulty journal replay made a .png file into an unrelated .tar-archive. These would both be trivially detectable error conditions from a small script.
An alternative would be to interface with the LocalIndexUpdated event and do a shutdown from a handler of that. I guess it could catch corrupted files earlier, but it would also be a trickier place to do validation as it requires actually digging out which files were changed with the new sequence.
It is clear that the purpose of this thread are outside of the goals of syncthing itself, but I hope syncthing still tries its best to be integrated with. All questions are regarding having an external component doing the validation and making decisions.
Plugging into the external versioning seems doable “today”, and is thus the main question of this thread.
I see that protection against data-corruption has been lifted before (t/1987, t/13096, t/13120, bug#5629 + kind of t/15276 and bug#1315), but seemingly only as requests for functionality within syncthing itself. Rejecting those seems reasonable giving the project goals, but what would the stand be towards adding another hook for externally validating files prior to adding them to the sync-queue (and possibly a corresponding error-condition about a user requested halt)? Are the odds good for a well-written such contribution getting merged?
Arguments about whether this ideally would be better handled on the filesystem layer or whether the performance hit is desired should in my opinion be for each user to decide on, given their reality.
Additionally:
I see there is also a /rest/system/error-endpoint, but its briefly documented. I have not dug any deeper in what it does, but my assumption is that it only does user feedback, not actually modifying sync-functionality.
Please let me know your thoughts,