I have a NFS mount with a sizable amount of data in many sub-directories that I want to synchronize to another server’s NFS mount. The source files is also continuously changing, although I am unable to get an exact measure of the rate of change. My question is, what is the best way for me to accomplish this with relatively low overhead. I will be running a rsync to get the bulk of the data over to the new NFS mount. I’ve previously used syncthing for something else with fairly good results. However, I am not sure how well it will deal with a NFS dataset so large. Writing a reiterative script using rsync is also possible, but AFAIK, rsync has a large overhead on runtime as it indexes and generates the metadata for the sync operation.
You’re probably better off running Syncthing on the actual servers where the files reside.
Unfortunately that’s not possible. The NFS shares originates from an appliance and I cannot install anything on it.
Then I’m not really sure what you are asking. Accessing files over NFS will of course give you more overhead than doing it natively. You will most likely not get filesystem notifications to work over NFS so Syncthing will depend on periodic scanning. Expect lots of metadata shuffling, hopefully some of it cached locally. I’m not sure what you can tweak to improve things. Probably this is similar whichever syncing solution you deploy.
If your two NFS mounts are mounted on the same NFS client there is no real advantage to using Syncthing at all, as far as I can tell.
It’s actually in an effort to migrate data from an old NFS appliance to a new one, hence the reason I am trying to explore ways to do this efficiently. I had expected it is not to be the best usecase of syncthing given that file changes in NFS does not notify the kernel.
This sounds like the perfect use case for rsync a/ b/ unless the appliances themselves provide something better (e.g., ZFS replication).
2 Likes
Agreed. This will be the path of least resistance to copy data from the old share to the new.