syncthing and rsync disagree. I wonder why

Support
1

I have three computers. My Mac and Win10 computer happily synchronize my documents using syncthing.

Separately, my Mac pushes documents to a Raspberry Pi using rsync. When I push the files from my Win10 to the Raspberry Pi, rsync sees all the files as a mismatch and proceeds to transfer all the files.

If syncthing believes the Mac and Win10 are in sync, why do the rsync programs on the two computers disagree what should be sent to a third computer?

Thanks for any ideas.

2

We can’t say, look closely at the files to see what differ - including modification time and permissions. Those are the aspects that are most likely to be handled differently would be my guess.

3

What could also be the reason are how modtimes are handled by the filesystem - from the rsync man page:

–modify-window
When comparing two timestamps, rsync treats the timestamps as being equal if they differ by no more than the modify-window value. This is normally 0 (for an exact match), but you may find it useful to set this to a larger value in some situations. In particular, when transferring to or from an MS Windows FAT filesystem (which represents times with a 2-second resolution), --modify-window=1 is useful (allowing times to differ by up to 1 second).

4

Calmh, so mod times and permissions could be different and syncthing still believes they are in sync? I didn’t understand this before and assumed all would be perfectly matched before syncthing quit. So I looked further…

When I look closer, I realize rsync (using the -n do nothing option) is listing only directories “to be sent” (no specific file is named). Directory times on the two source computers are not even close. Times on all the files match, however, file permissions are different. In the Win10 Ubuntu bash window, all files and directories show with rwx permission for owner, group, and everybody. Hmm…

So I nee to figure out why all the files and directories have wide open permissions under Win10 bash. It must be something about how the NTFS underlying filesystem presents to the bash shell.

But I’m still intrigued that syncthing could be happy leaving permissions and times different.

5

Syncthing only syncs the read only bit on windows in terms of permissions and does not consider a directory changed if only the modtime is different. So rsync will probably just update the modtimes (and potentially permissions, you can configure that) and then be done with it, which should be very fast.

6

Syncthing does not sync directory mtime, as it’s a hard thing to maintain. Some filesystems bump that when files inside get modified, some don’t etc, resulting in a never ending mess.

7

Thanks all for the guidance. Got it working. This allows either of two sync’d computers to push a directory to a third computer using rsync. I’m using rsync because syncthing currently offers no way to make the 3rd computer receive only. Looking forward to when this feature is added to syncthing!

Here’s the script file I ended up with (sorry about the double spacing; how do I enter CR limited lines into the forum post?)

#! /bin/bash

set -x

#usage: $ push-to-pi-offsite {subdir | null for all of Documents directory}

#to push from Mac OSX to raspberry off-site backup aggregator

ssh pi@10.0.0.113 test -f /home/pi/cleartextfiles/_encfs-is-mounted && rsync -rltvO --no-p --chmod=ugo=rwX --exclude-from ‘exclude.txt’ --delete-after ~/Documents/$1 pi@10.0.0.113:cleartextfiles/Documents

#( r ) recurse

#( l ) links as links (not diving into them)

#( t ) preserve time stamps

#( v ) verbose list filenames as processing

#( O ) ignore directory times (syncthing doesn’t do them so they’re different on different machines)

#(no-p) don’t copy permissions

#(chmod=ugo=rwX) all non-umasked bits set on destination for new file, existing destination files left alone

1 Like
8

You can format code like this:

`` `
your code
`` `

(without the space between the ticks)

9

Felix thank you for the triple-back-tick idea. I couldn’t find any way to edit a posting I already wrote, so I’ll repost here so others can see it better.

Thanks all for the guidance. Got it working. This allows either of two sync’d computers to push a directory to a third computer using rsync. I’m using rsync because syncthing currently offers no way to make the 3rd computer receive only. Looking forward to when this feature is added to syncthing!

Here’s the script file I ended up with:

#! /bin/bash
set -x
# usage:  $ push-to-pi-offsite {subdir | null for all of Documents directory}
# to push from Mac OSX to raspberry off-site backup aggregator
#
ssh pi@10.0.0.113 test -f /home/pi/cleartextfiles/_encfs-is-mounted && rsync -rltvO --no-p --chmod=ugo=rwX --exclude-from ‘exclude.txt’ --delete-after ~/Documents/$1 pi@10.0.0.113:cleartextfiles/Documents
#
# (r) recurse
# (l) links as links (not diving into them)
# (t) preserve time stamps
# (v) verbose list filenames as processing
# (O) ignore directory times (syncthing doesn’t do them so they’re different on different machines)
# (no-p) don’t copy permissions
# (chmod=ugo=rwX) all non-umasked bits set on destination for new file, existing destination files left alone

One other point of clarification. “_encfs-is-mounted” is a manually created zero-byte semaphore file sort of like a syncthing .stfolder folder. I use this file because the destination location is a virtual input for encfs encryption, and I don’t want rsync to run unless encfs has the mount mounted.

10

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.