Basic New-User Confusion About Folder Contents

[packetfire]

In short, how do I make Synchthing “forget” about past synchs?

I have syncthing installed on a Pixel 3 phone, and my older Original Pixel (“Pixel 1”) I also have a program from Nikon on the Pixel 3 which copies over photos from a Nikon SLR.

My goal is to move all photos to the Pixel 1 “DCIM” directory. Pixel 1s enjoy unlimited “Google Photos” storage at full rez forever, so the Pixel 1 is retired to simply being a middleman in a “cloud backup” for both the Nikon and the Pixel 3.

…but I can’t seem to make it work well:

1. Any photos taken with the Pixel 3 should be synched to the Pixel 1’s internal camera (“DCIM”) directory.

2. Any photos from the Nikon directory on the Pixel 3 should get the same treatment - a one-way copy to the Pixel 1’s DCIM directory

Both of these are one-way synchs - Pixel 1 only reads Pixel 3, never deletes anything on Pixel 3.

When I am done the synch to the Pixel 1, I will delete all photos from the Pixel 3 (or copy them to a different directory, as the Pixel 3 has 128GB, while the Pixel 1 has only 32GB.)

BUT - As the Pixel 1 has limited memory, so once “Backup” of the Pixel 1 DCIM directory is accomplished by the Google Photos app, I want to delete all the photos from the Pixel 1, to free up space for future photos, and still synch new photos on the Pixel 3 from whatever source (DCIM folder)

I am not sure how to do this, as there seem no limits on file selection, so I cannot (for example) set up a date-range limit to make it only concern itself with “new” photos that I know I have not yet synched.

Perhaps the better approach would be to copy over a limited number of photos at one time to a “file transfer” directory, but I doubt I will ever take 10 GB of photos and video in one day, so I was hoping to be able to set up a “synch today’s photos” configuration, or a “synch everything newer than the last sync date/time”.

But I am clueless on how to accomplish this. Right now, I have a synch process that is “41% Complete”, and has not progressed beyond that since last night.

[tomasz86]

The explanation is quite convoluted. Could you upload screenshots of the Syncthing Web GUI from all involved devices? Please make sure to unfold the folder info, so that all paths and shared devices are visible. The Web GUI can be accessed on Android through the left slide-out menu in the app.

At the moment, it sounds like you’re trying to sync two Send Only folders into a single folder on a remote device. While this will still upload the files, you will always end up with a permanent “out of sync” state on the sending side.

When it comes to deleting already synced files, etc., the simples solution is to just move them on the receiving side to a different folder. Alternatively, you could also utilise file versioning (see https://docs.syncthing.net/users/versioning.html) for this purpose, also on the receiving side.

There’s no option to ignore files by date, etc., but specifically with videos and pictures, which normally have dates in their filenames, you can utilise character ranges to target only specific time periods (see https://forum.syncthing.net/t/sync-files-from-a-specific-date-onwards/17651).

[packetfire]

At left, the Pixel 3, at right the OG Pixel. The OG Pixel wants everything in its DCIM folder, so that it will auto-backup to Google Photos.

Either phone can get the photos from the Nikon, but given the limited 32GB memory of the OG Pixel, with about half of that taken up by the OS and apps one cannot delete, I guessed it would be best to collect them on the Pixel 3, and let Syncthing send them to the OG Pixel.

So, from “move synched files to a different folder” sounds like advice to use a dedicated directory for synching, and then manually moving things to the DCIM folder for Google Photos to notice. That’s certainly not all that painful, but I was trying to make it all happen while I slept, from syncthing doing its thing, to Google snarfing up photos.

Can I be the only one trying to manage photos from multiple devices onto one central device for upload to “the cloud”? Someone else just has to have this scripted and nailed down into a bulletproof set up.

I’m sorry about my lack of proper terminology, I’m not disrespectful, just unfamiliar with Syncthing.

[tomasz86]

Thank you for the screenshots, but to have a clear overview, I still need to ask once more time for screenshots from the Web GUI, not the Android app interface. As mentioned above, if you open the left slide-out menu, there a button there called “Web GUI”. For the record, the GUI itself should look like the one at https://docs.syncthing.net/intro/gui.html, just adjusted to the phone screen.

I haven’t used Google Photos personally, so I’ve got only a general idea about how it works, but normally if you wanted to sync everything to a single, central location, and then delete the files on the sending devices, you could simply set up separate folders for each device, and then run a script on the “server” to move them somewhere else for further operations. This is the cleanest method, at least in my opinion.

Of course, there’re hacks to target multiple Send Only folders into one location on the “server” and merge their content there, but this will result in Syncthing being in a permanent “out of sync” state, so it’s not really recommended. It does work though, and you should be able to find at least a few forum topics about this method specifically if you’re interested.

[packetfire]

Screenshots from Pixel 1 - The recipient of Synch from Pixel 3 and Nikon

And screenshots from Pixel 3 - the sender of its DCIM folder to the Pixel 1

I am sorry, I had no idea what the “Web GUI” was, thought it was for PC/Mac, not phones

I will hunt around to see what I can see, but the main issue for me is knowing when the process is completed for any one day’s (or week’s) photos. I do not mind an “out of sync” warning, but I need to understand the mechanisms here well enough to verify that all the images have been moved over to the Pixel 1.

[tomasz86]

To be honest, it’s still not 100% clear what you’re trying to do, but the main problem here seems to be word “move”. The purpose of Syncthing isn’t to move files. It is to (continuously) synchronise them.

At the moment it appears that you’re simply trying to sync two different Send Only folders from Pixel 3 into a single folder on Pixel 1. Is this correct? If yes, then while hacky, the current setup should do the job. Both Send Only folders will display the “Override Remote Changes” button, but as long as you don’t click it, the synchronisation should go on.

What Syncthing isn’t going to do is to delete the files from Pixel 3. This means that you will end up with the files split between DCIM and Nikon folders on Pixel 3, and combined together in one folder on Pixel 1. You need to keep in mind that if you delete the original files on Pixel 3, then they will end up being deleted on Pixel 1 too. There exist less or more hacky workarounds to get around this, but they’re usually not recommended.

A clean solution is not to merge the folders like that, but rather to sync DCIM to one folder, and Nikon to yet another folder on Pixel 1, so that they stay separate. This way everything can be set to just Send & Receive, which means that you can then move the files to a different location on Pixel 1 later. Alternatively, if you prefer to have the folders merged into one on Pixel 1, you could also experiment with (external) file versioning to move the files on deletion to a specific folder.

In general, everything explained above is rather non-standard and quite convoluted. The normal use case for Syncthing is to simply sync files between specific folders both ways.

[packetfire]

OK, thanks for the insights!

“if you delete the original files on Pixel 3, then they will end up being deleted on Pixel 1 too”

That’s not what I understood - I am setting the sender folder(s) to “Send Only”, and the receiver folders to “Receive Only”. Doesn’t this block any deleting of files?

The hope is to use syncthing as an automatic and unconditional one-way file transfer utility. The reason for “sync” rather than some less smart “copy” program is that it should require minimal fussing, as the rules are set up to copy whatever is in certain folder(s) on one device to certain folder(s) on the other.

So, wander about with camera and phone, snapping photos, back to base camp and plug into the battery packs that have been charging up all day and:

a) The Pixel 1 plays “WiFi Hotspot” for the Nikon and the Pixel 3 b) Everyone gets plugged in to charge up c) Syncthing starts up because it sees WiFi d) Files move across to the Pixel 1 and also its attached USB-OTG flash drive (for a complete backup of all images)

This happens for several days, until the Pixel 1 is carried back to civilization and connects to WiFi that can connect to the actual interwebnet, at which point all the photos get synched up to Google Photos.

So I end up with:

(a) Multiple SD cards for the Nikon with its “original” photo images (b) The pixel 3, with its own photos (c) A flash drive backup of the Pixel 3 photos (d) The Pixel 1 with photos from both the Pixel 3 and the Nikon, in what it thinks is its own internal DCIM folder, but it actually a symbolic link to its external flash drive, ready to upload to Google Photos when within range of real WiFi.

So, upon return to civilization, one can eat and sleep and not worry about uploads, as they happen automagically. Mostly, one is tired upon return, as sleeping is hard on cliff faces and mountainsides. The goal is to automate, and eliminate the potential for a tried human to screw up the backups.

[calmh (Jakob Borg)]

@packetfire

I am setting the sender folder(s) to “Send Only”, and the receiver folders to “Receive Only”. Doesn’t this block any deleting of files?

Sending and receiving refers to sync “actions” - such as creating a new file, changing it, or deleting it. A delete is a change like any other: it gets sent from a send-only side and received by a receive-only side.