Simple Android backup setup issue

#1

Hi there, I just set up a simple backup solution with my Android phone and a Pi running raspbian/syncthing. I have one folder to backup photos with the ‘send-only’ option, another as a ‘send-only’ backup for my local device backups (MIUI feature) … and finally a ‘send-and-receive’ shared folder for two-way transfers.

I have everything working straight away … even over 4g and pretty impressed. I already have VNC on the Pi so didn’t have to set anything up beyond sharing the device Id’s with each other and setting up the folders (can access the GUI over VNC).

However there is one very annoying issue. Everytime the pi is rebooted and I connect from Android, two of my three folders always show in red as ‘out of sync’(photos and backups). This is not the case as the files are definitely the same on both sides. This happens even when I give the Pi its 15 required minutes to finish scanning the folders after each reboot and before connecting from Android.

The temporary solution I’ve found is to go into web GUI on the Android side and tap ‘force overrides’ a few times until it goes away. This makes no sense to me as both devices are already synced and they have both had the time to scan their folders before connecting.

If I don’t reboot the pi, I can leave it for days and connect just fine from Android without.any out of sync nonsense…

Could someone please explain what is going wrong on the reboots? I have a very simple setup and this behavior is quite strange to me.

thanks

Note: the only non-standard setup I did is to have syncthing start as an lxsessions autostart rather than though systemd but that should be no different than launching it from the terminal on every user login. Also, by the looks of it, it would seem that the ‘send -only’ option is a correlating factor but this is the default option when using synching to backup photos…

(Jakob Borg) #2

The “send only” setting is one the Android side, right, so the data flow is Android -> Pi, and when rebooting the Pi the Android side looks “out of sync”?

If so, does this happen if you just restart Syncthing on the Pi, or just if you reboot the whole Pi? If the latter it sounds like something is tweaking the files during reboot. Maybe resetting permissions or something?

#3

hi thanks for your reply! I just tested and it doesn’t happen when just restarting syncthing on the pi. This happens only when rebooting the pi.

My syncthing folders on the pi are located on a fat formatted USB. ‘ignore permissions’ is set for each folder (also the default for Android folders I believe).

The problem doesn’t occur on my shared folder which only has one file so far and is set for send/receive on both devices.

Only the Android device is set send-only for my photo and backup folders, the other ‘shared’ folder and the folders on the pi are set ‘send and receive’. When I reboot the pi, wait for it to finish scanning and then connect my Android this happens:

Android app shows the photo and backup folders as ‘out of sync’ in red. The Pi seems to be stuck at a random sync percentage of about 60%. Yet both devices have the exact same files and everything is unchanged. If I look at the Pi’s GUI it looks like its stuck syncing at an arbitrary percentage close to what’s shown in the Android app. No scanning folders or restarting solves anything. However, if I go to the web GUI of the Android app I can see that the photos and backup folders are in red with a button to override changes. My backup folder always has the same amount of changes but the photos folder seems to vary. Both are around 50% of the files. One tap is enough for backup folder, but overriding changes on photos must be tapped several times sometimes… It doesn’t seem to actually override anything as clearly no files are sent or received.

Once I’ve ‘overidden the changes’ then it works perfectly until the pi is rebooted. Upon reboot of the pi, syncthing does a lengthy scan (over 10 minutes for 7gb of photos) despite no changes and the issue returns.

I don’t necessarily have to have it set this way, it’s just very basic and thought that it should just work in these conditions. Any suggestions?..

#4

Also, whilst the Android app reports that the two send-only folders are out of sync, the correct local and global file count is shown in the web GUI of both devices. Literally pressing override changes seems to change nothing but unsticking the syncing process.

#5

the plot thickens… I set all folders to send and receive instead and the issue disappeared. When I returned the setting to send only on my photo and backup folder the issue was still at bay. However, upon taking a new picture I have the override changes issue return after reboot for that 1 single file.

I think if I set it to send and receive, the issue will disappear until I make some new files. Otherwise if I tap override changes the issue will be gone until reboot.

Simply if I set it to send and receive there will be no issue. But is it not standard practice to set backups to send only?

(Jakob Borg) #6

I still have a feeling this is mtime related, but it’s not clear to me why the reboot affects anything.

#7

Setting it to “send & receive” will not solve the problem, but the “changed” meta data of the pie will be synced to the android device.

You can probably solve the problem by using a a better fs. FAT cannot store the timestamps accurately. The OS maybe caches the timestamp when setting it, but that cache is cleared on reboot.

(Jakob Borg) #8

That’s what it feels like, but I can’t find any evidence of such by Googling…

There’s also the thing where the Pi doesn’t have a real time clock and will have wildly wrong time at startup, combined with how VFAT stores file times in local time, and the meaning of that changing over the year with daylight savings… So yeah, I don’t know. Weird stuff.

#9

thanks for the input guys. I tried ext4 and the issue seems to be solved… I had chosen vfat because I could just copy directly from my phone over OTG and there were no permissions issues so I thought that it would be simpler. Oh well, at least it’s working properly now! So vfat is to be especially avoided with the raspberry pi. Thanks again