Hey guys,
So I have been thinking about trying to add (at least partial if not complete) persistence to the global log and have come up with a few possible ideas.
1.) Have a node that, when just coming online and peering, queries the /events/disk API of the node he connected too for the current list and fill his own events buffer to match. This would probably happen only one time after client restart, during the first time he connects to another node, and not for every peered connection.
PROS:
- Probably easier to code and more straight forward (ie likely fewer bugs)
CONS:
-
No guarantee of true persistence, just what the other client has. In hopes that if enough clients are online most of the time, the persistence will pass from client to client.
-
Possible that the node connected too also doesn’t have much (or any) history, depending on how often all nodes in swarm go up or down (see previous point)
-
Requires API of all (most?) clients to listen on http://0.0.0.0:8384 and all API keys to be identical, instead of the default http://127.0.0.1:8384 (prob not good for security but maybe we can force TLS, also what info can someone really steal using ST API?).
2.) Have the node (the one that makes changes to file/folder object) write to a text file just as he is writing to event buffer for API. The file will be synced like other files but with special exception to ignore scanning it for changes to sync (so the act of writing down a change doesn’t trigger another which in turn triggers another, causing circular syncing)
PROS:
-
Doesn’t requires all clients to have API listening on http://0.0.0.0:8384 or same API key
-
Theoretically should maintain complete and utter persistence of all changes as far back as any users desires since the file is synced between all online clients shortly after a change is made
CONS:
-
Might be trickier to code to make exception for a file to not sync unless explicitly told to by a client to do so
-
Would likely need to sync the file in a system dir and not the standard ones that the users have setup
3.) Some hybrid where a client writes his own events to a (non synced) file on shutdown and upon restart would refill his event buffer with those events. But (in case some length of time has gone by) he could also try to gather missing events from other nodes via API.
.
That’s all I have so far… anyone have any other thoughts or ideas on how this feature could be made better or more easily? Am I just over-engineering this and missing some obvious easy thing instead?
Thanks