Syncthing as Google Photos replacement - way to stream pictures and videos from other device

General
1

I’m trying to develop a Google Photos replacement. What I need is:

  1. automatic sending of data to external, self hosted servers
  2. ability to remove photos from mobile devices (to free up space) without losing them completely (they need to persist somewhere else)
  3. ability to easily view photos/videos even after removal through some kind of streaming

Syncthing seem to solve points 1 and 2 perfectly (through One-Way/Send-only Directories), but I’m still looking for solution to 3. I need some Android app (or self-hosted web server?) that would display gallery of photos stored on one of Syncthing nodes without downloading them permamently. What would be the simplest and safest way to do this?

1 Like
2

Syncthing does not really implement 2) and 3). Syncthing is a file synchronization tool and neither 2) nor 3) are related to synchronizing.

Send-only folders do not mean that no files get deleted - if you delete them on your phone, they will be deleted on the server(s) too.

All of this is probably hackable using additional scripting, third party tools or other workarounds.

2 Likes
3

I have implemented something similar.

I have one sync drive per mobile device (this is for the whole family).

I have one connected harddrive (odroid HC1) that syncs the photos. this is kind of the central hub.

The HC1 have cron jobs that run daily, and basically move the photos from the mobile device sync folder to a common “media library” sync folder where all photos are stored and organized.

This effectively removes the photos from the mobile devices. I have it organized so that photos will stay on the device 30 days, mainly for giving you time to show recent photos and also for you to have the opportunity to delete bad pictures/videos before they start being propagated.

The process of moving from mobile device sync to media lib sync is handled by scripts that use exiftool, see below.

The media library sync folder is also synced with the PC where photos are post-processed, rated, cleaned, etc. (I use darktable for that),

The most complex part is the “moving” form phone sync folders to media library. I use exiftool for selecting the photos (or videos) moving them to the right folder and also adjusting metadata (who shot the picture), etc.

here is a snippet:

#!/bin/bash

ML_IMAGE="registry.lan/exiftool:edge"
ML_AGE_DAYS="30"
ML_ARTIST="Rafael"
ML_COPYRIGHT="All Rights Reserved"
ML_PREFIX="RV2"
ML_VOLUME="syncthing_sync"
ML_OPTIONS="--ext xmp"
ML_USER="1000"
ML_GROUP="1000"
ML_SOURCE="Camera Redmi Note 7 Rafael /Camera"
ML_DEST="ValleRandaMediaLib"

# Notes: Use FileModifyDate because is present in all media formats (XMP)
# Sidecars might have another date, which may result in copying half of the metadata
# Modify Date will change after migration which makes this step difficult to reverse

# Move Videos

docker run  --rm \
  -v ${ML_VOLUME}:/media \
  --name medialib_${ML_PREFIX}_video \
  --user ${ML_USER}:${ML_GROUP} \
  ${ML_IMAGE} exiftool -r \
  -artist="${ML_ARTIST}" \
  -copyright="${ML_COPYRIGHT}" \
  -overwrite_original \
  -ext mp4 \
  ${ML_OPTIONS} \
  -d "/media/${ML_DEST}/Video/%Y-%m" \
  "-Directory<FileModifyDate" \
  "-Filename=${ML_PREFIX}_%f.%e" \
  -if "time() - ${ML_AGE_DAYS}*86400 > "'${FileModifyDate#;DateFmt("%s")}' \
  "/media/${ML_SOURCE}"

logger -t mediasort_rv-redmi-note-7 Video Processing Finished with status $?

# Move Photos

docker run  --rm \
  -v ${ML_VOLUME}:/media \
  --user ${ML_USER}:${ML_GROUP} \
  --name medialib_${ML_PREFIX}_video \
  ${ML_IMAGE} exiftool -r \
  -artist="${ML_ARTIST}" \
  -copyright="${ML_COPYRIGHT}" \
  -overwrite_original \
  --ext mp4 \
  ${ML_OPTIONS} \
  -d "/media/${ML_DEST}/Photo/%Y-%m" \
  "-Directory<FileModifyDate" \
  "-Filename=${ML_PREFIX}_%f.%e" \
  -if "time() - ${ML_AGE_DAYS}*86400 > "'${FileModifyDate#;DateFmt("%s")}' \
  "/media/${ML_SOURCE}"

logger -t mediasort_rv-redmi-note-7 Photo Processing Finished with status $?

One of these jobs run per device.

My collection is now reaching 1TB.

My next step in the process is to get the photos out of Darktable into something easier to share…

I am looking at the following options.

https://pixelfed.org/ as a public federated sharing https://photoprism.app/ as an online album option.

I will probably take both. I think it makes sense to decice from photoprism what you want to make public via pixelfed.

6 Likes
4

Hi,

I’m also moving my pictures off the phones and then organize them in folders on a central server. I’ve found piwigo very useful as it’s completely open source, has plugins e.g. to show your videos too and I like the adapter to an existing folder structure very much. I just give www-data (aka piwigo) read-only permissions to my /home/me/pictures root folder and then push the quick sync button on it’s admin web ui when new pictures have been placed via SMB/windows explorer. Piwigo also has an open source Android app, ios app for mobiles to remotely view the gallery.

https://piwigo.org/demo/

5 Likes
5

Thanks for pointing that out, I misunderstood this feature. This complicates things a little. I wanted to minimize additional scripting to ensure stability of entire system. I just wanted to make it with ready-to-use building blocks, so I don’t need to care about maintaining entire system that much.

Thanks rvalle for recommendation of photoprism. I’ve just initially tested it and seems perfect for my needs.

1 Like
6

but have you tried the before mentioned solutions?

i’m curious to know the differences… and have a simple filter, if i’ll ever bother to pick one.

here’s a few more i also haven’t tried, to the mix:

https://www.reddit.com/r/selfhosted/comments/jsdxpl/google_photos_unlimited_storage_shutting_down/ (i couldn’t even find the alternative i had in mind, and i don’t think it’s included in this link)

freaking options overload!

2 Likes
7

This is solved using file versioning on the “STORAGE SERVER SIDE” then you can remove the pics fro the phone and still have them on the remote device.

8

File versioning its true will retain the deleted files on the “server” side, but depending on the style of versioning actually makes it difficult to manage/access.

Moving the files out of the server receiving directory/folder into a central organizing directory/folder accomplishes the objective much more efficiently.

With Linux/FreeBSD/*nix you can use hard link copies (cp -l) to make a copy with a later script that deletes them from the incoming Syncthing folder (which effectively only deletes one of the hard links). As long as there is more than one hard link to the file the original data will not be removed.

I use a variation of this for adding photos from Android devices to a XigmaNAS server running Syncthing.