Idea for handling git repos and other similar directories.

Hi all,

Considering I spend most of my days in code, not being able to sync my work across machines pains me a great deal. Judging by many comments out there, I am not the only one. So, I have been thinking about the problem and came up with a rather simple idea. Before I start hacking away, I was curious what your opinions are.

The idea is rather simple. Git treats the local repo (.git) like a database. Rather then trying to figure out the details of how to sync databases, why not treat it like a single thing, single item that needs comparing/synchronizing.

So, the proposal is something like:

  • SyncThing should keep a list of well known directories that should have special treatment. This list can be added to by the user.
  • In these directories, any other settings for ignoring files etc. are disabled. So these directories are either synched completely or not at all.
  • When detecting changes, a change of any kind inside the tree, will flag the whole tree for synching.
  • To sync the “special” item, the receiver will first make a (hard) copy of the tree. Then all the changes will be synched into the copy. When the whole tree is finished, the receiver will unlink the old copy and place the new version in it’s place.

There are obviously some issues that need to be solved.

  • Aborted transfers would have to be dropped or optimised in some non trivial way.
  • Various conflicts are possible if any of the files in the old version of the directory (one that is removed).
  • I am sure more problems.

For the purpose of this conversation, let’s leave aside comments about internals of SyncThing and/or how hard this would be to implement in SyncThing. These problems will surely need to be addressed, but let’s not get into weeds before we decide if the approach has merit.

So, any thought, comments, questions on the idea? Has something like this been tried before? Do you know any reason this would not work? Is another way being considered?

Thanks all and keep up the great work on SyncThing.

Miki.

I think the idea of an atomic directory has some merit, if also technical difficulties.

I also think git isn’t a great example to be honest – it’s specifically designed to be used in a distributed manner and is all about pulling, pushing, and merging changes between devices. We should let git do what git does and not try to wedge a half-assed (compared to what git itself does) sync solution on top of it.

Other database like things are more suitable for the idea though. For example things like iTunes databases, photo management databases, etc. There I think the main technical issue would be to assure the source is stable / quiescent before attempting to sync it.

Also, off topic, but given the amount of times it’s repeated here (and I admire the effort and the fact it’s consistent): https://docs.syncthing.net/users/faq.html#is-it-syncthing-syncthing-or-syncthing

P.S. yeah, I agree with the concept, but I struggle to grasp the amount of work involved, which I suspect might be alot. It might be easier for you to pick a different tool for the job.

1 Like

I was actually looking the other day whether there was some sort of autocorrect possible in the forum software itself. (There wasn’t, although it can blacklist/sensor words.)