Sync X First (GH#174)

I am trying to implement I’ve made some pretty good progress; I can sort files and I can read sorter configuration. @calmh, could you point me to where the changed file list shows up? I am currently sorting in LoadIndexes but that is almost surely not where the files need to be sorted.

Hi! I answered this on github before I saw it here. But just to not leave it hanging here, there’s a Model.NeedFilesRepo that returns the list of files that are needed to bring a given repo up to sync. The returned list will be processed in order by the file puller, so I think this would be the cleanest place to hook in.

As a side note, the current order is pseudorandom, due to the files being extracted from a map and how those work in Go. This is nice because if suddenly 1000 files need to be synced from one source node to five other nodes, they will start pulling the files in random order.

This means that when a given node wants to pull file number 500, odds are that one of the other nodes have already synced this file and we can get it from there as well. If everyone pulls in the same order, the source node will have to send the same data five times. Preserving this pseudo-randomness in case the user hasn’t specified a priority order might be a good thing.

When done, would you mind writing up a short article in the documentation category about how this works and how to configure it (i.e. like excluding files etc)?

Feel free to edit the config file article as well.

Just saw this; sure I’ll do that, though I do plan on making a GUI for the priority stuff too.