Sync files between 2 Raspberry Pi devices

[cktechie]

Hi

I am trying to sync files (mostly photos and videos) between 2 RPis, only the directories are getting created however there are no files synced at all. can you please guide what the issue might be.

[gadget]

Welcome,

Unfortunately there’s not enough info yet to properly diagnose anything.

Screenshots of Syncthing’s web UI – with folder details visible so we can see any errors – would be most helpful.

Also, please provided at least the following about the RPi setup:

• OS
• Storage (e.g. USB drive, filesystem type)

(The more details shared, the easier it is for I and others on this forum to help.)

[cktechie]

RPi 1 (RPi which has files to be synced / backed up to RPi2)

OS: Linux DietPi from v9.5.1 Hardware: Raspberry Pi 4 Syncthing: v1.27.9, Linux (64-bit ARM) Storage: External Mounted 1TB SSD, USB Drive fuseblk 932G 558G 375G 60% /mnt/1TBSSD

RPi2 (RPi which is supposed to receive files from RPi1)

OS: Linux DietPi v8.25.3 Hardware: Device model : RPi 3 Model B+ (armv7l) Syncthing: v1.27.9, Linux (32-bit ARM) Storage: External Mounted 1TB SSD, USB Drive exfat 932G 80G 853G 9% /mnt/1TBSSD

[gadget]

RPi1…

Is the SSD connected via USB formatted as a NTFS volume?

RPi2…

(Click the blue “129,067 items, ~142 GiB” hyperlink in the “Photos” panel to access a pop-up dialog box with a list of the files that are out of sync and why.)

Because the SSD connected via USB has an exFAT volume, enable the “Ignore Permissions” since any filesystem permissions (including ownership) sent by RPi1 won’t be compatible.

Also, with exFAT (and the other related FAT-based filesystems), watch out for file/directory names that contain any of the following set of characters… \ : * ? " < > | … plus additional restrictions regarding where spaces can be used.

With filesystem watching enabled (i.e. “Watch for Changes”), there’s really no benefit to setting “Full Rescan Interval” to 60 seconds. Reverting it to the default 3600 seconds will reduce load on the system.

[cktechie]

RPi1… Is the SSD connected via USB formatted as a NTFS volume? → Yes

RPi2… (Click the blue “129,067 items, ~142 GiB” hyperlink in the “Photos” panel to access a pop-up dialog box with a list of the files that are out of sync and why.) —> Does not show the reason, see the screenshot below

Because the SSD connected via USB has an exFAT volume, enable the “Ignore Permissions” since any filesystem permissions (including ownership) sent by RPi1 won’t be compatible. —> Done

Full Rescan Interval reverted to the default 3600 seconds —> Thank you!

Still no luck, not syncing

[gadget]

On RPi2, check Syncthing’s run log (Actions → Logs).

According to the screenshot, the target directory is ~/Photos, is that where your 1TB SSD is mounted?

What’s the mount entry in /etc/fstab? (Since you’re using DietPi, I’m assuming there’s no desktop environment doing the auto-mounting.)

[cktechie]

On RPi2, check Syncthing’s run log (Actions → Logs). —> attached RPi2_logs.txt (6.7 KB)

According to the screenshot, the target directory is ~/Photos, is that where your 1TB SSD is mounted? —> symlink created for the Photos (ln -s /mnt/1TBSSD/photos/library Photos)

What’s the mount entry in /etc/fstab? (Since you’re using DietPi, I’m assuming there’s no desktop environment doing the auto-mounting.) —> Below is the fstab content for the 1TB SSD: UUID=FE9C-21AD /mnt/1TBSSD exfat noatime,lazytime,rw,gid=dietpi,fmask=0002,dma=0002,nofail,noauto,x-systemd.automount

[gadget]

@cktechie

On RPi2, check Syncthing’s run log (Actions → Logs). —> attached RPi2_logs.txt (6.7 KB)

Grabbing just a few of the relevant lines from the log:

2024-07-30 14:34:20 Failed initial scan of receiveonly folder "Photos" (uifvn-sjufx)
2024-07-30 14:34:20 Paused folder "Photos" (uifvn-sjufx) (receiveonly)

So far it looks like a permissions issue…

According to the screenshot, the target directory is ~/Photos, is that where your 1TB SSD is mounted? —> symlink created for the Photos (ln -s /mnt/1TBSSD/photos/library Photos)

That’s fine, but the combination of the mount point and exFAT means a bit of extra care managing permissions…

What’s the mount entry in /etc/fstab? (Since you’re using DietPi, I’m assuming there’s no desktop environment doing the auto-mounting.) —> Below is the fstab content for the 1TB SSD: UUID=FE9C-21AD /mnt/1TBSSD exfat noatime,lazytime,rw,gid=dietpi,fmask=0002,dma=0002,nofail,noauto,x-systemd.automount

hmmm… there’s a gid mount option but no uid, so everything under /mnt/1TBSSD will be effectively owned by root:dietpi.

If the user running Syncthing is the same one who owns ~/Photos, and that user isn’t root, there are going to be access issues.

[cktechie]

@gadget

hmmm… there’s a gid mount option but no uid, so everything under /mnt/1TBSSD will be effectively owned by root:dietpi.

Thank you much for this input, I was able to edit the fstab file to include uid=dietpi, rebooted the RPi2, after which the sync seems to be working now.

However there is a red box showing up – Revert Local Changes, do I need to do anything for it? ignore it?

[gadget]

@cktechie

However there is a red box showing up – Revert Local Changes, do I need to do anything for it? ignore it?

Since the target folder on RPi2 is receive-only, and RPi1 is set to send-only, there’s no risk changes on RPi2 will be pushed upstream to RPi1.

Chances are the local changes warning has to do with the permissions change, since Syncthing on RPi2 already had a chance to scan the target directory.

If you are certain that ~/Photos didn’t contain anything valuable prior to RPi1 adding new content, it’s safe to press that red button.