Several people have asked for a native app to upload photos and videos from their phones to their PhotoStructure library.
There are several apps that do this already, and do this quite well, so this isn’t something we’re going to work on anytime soon.
I personally use Resilio Sync to copy files as I take them to my NAS at home. PhotoStructure sync is configured to import from the Resilio directory and automatically organize the new photos and videos into my library.
The three options presented on the page you referenced, Resilio Sync, SyncThing, and PhotoSync, appear not to simply upload images, but to synchronize multiple sets of files, such as those on one’s phone and those on another storage device. “Upload” implies a one-way transfer, generally from a smaller to larger device, which is a very different operation that synchronization.
This seems to present two problems for the upload-from-phone-to-PhotoStructure use case: 1) Changes made on the secondary device would be replicated to the phone, potentially breaking native photo apps by violating assumptions they make about naming, tagging, or organization of images, and 2) images deleted from the phone (as one often does as storage on the phone runs short) would be deleted on the secondary storage, resulting in permanent loss of images that one expected to be safely archived off-phone.
Did you intend to suggest an alternative to uploading images from one’s phone to one’s PhotoStructure library, and for that to be a multi-step process involving 1) continuously synchronizing the images on a phone to a secondary data store on another device, and 2) arranging for PhotoStore to periodically check for new images on the secondary store, and when found, copy them to a third, PhotoStore-specific, data store?
Resilio and SyncThing both support different modes:
“mirror” mode, where deletes on the source device are reflected on the destination device
“send only” mode, where deletes on the source device are ignored.
You want the second option, so if your phone decides to “smart delete” photos that were backed up somewhere else, the backup on your NAS doesn’t synchronize those deletions.
If you’re using SyncThing, you want “send only” mode. If you’re using Resilio Sync, you want “Read Only-Full Sync” and “Store deleted files in folder archive” on the server folder (which should be set by default).
With respect to how I suggest people set up files, it’s up to you: I personally like to have my photos in one directory hierarchy, so I use automatic organization, but I’ve retained all my Resilio archives separately, as well (disk is cheap).
If you’d rather maintain a single copy of your photos and videos, you can either opt out of automatic library organization, or you can periodically archive your Resilio backup directory (take care to not delete the files that the app uses to maintain sync state, and realize you shouldn’t archive files on dest that still exist on src, as the sync app will re-transfer everything you just archived).
Syncthing is amazing. I’d been trying halfheartedly for a while to get it working over Tailscale from various devices to my NAS. With my phone’s storage rapidly approaching full, and my concern about what happens if and when I let Google Photos delete images from the phone that “are backed up” to the app’s default and opaque off-site storage facility, I gave up on that and let Syncthing do its NAT-busting, connection-finding, no-screentapping-qrcode-scanning magic. Wow, it’s slick.
I moved my Keepass database out of SpiderOak (which I never got working on Android) and Keybase FS (an old copy of the Keepass DB, to sometimes awkwardly retrieve on the phone), and started syncing it around to Linux, Mac OS, Android, and Chrome OS. Yee-haw. I don’t know if I got this right and have mild concern about a sort of hall-of-mirrors syncing among the various machines, but it seems to be working.
Of relevance to this forum: Moving toward my hopes of trying out PhotoStructure, and wanting a copy of images on my phone that are not cycled through Google Takeout, not tied up in whatever Synology Moments does with them server-side, and not requiring me to fix whatever is going on with my Perkeep installation, I pointed Syncthing on the phone at at /storage/emulated/0/DCIM and shared it with Syncthing on the NAS. I set the phone’s config for this share to “send only” and that of the NAS to “receive only.” 29 GB transferred, and maybe I have a good copy of my images and videos on which for PhotoStructure to feast when I install it.
I’m left with some concerns. One is off-topic here (re: multi-shot “images”, whatever happens with edits in Google Photos that it says are undoable, etc.) so I’ll write up another post asking about it.
The other is that I see lots of empty directories named like images (e.g. IMG_20191226_173817) in the copy one-way-synced to my NAS. Spread among them are image files, named similarly, but with a .jpg suffix (e.g. IMG_20191226_174108.jpg). Prevalence is much higher for older images, so I wonder if these are placeholders for image files that were backed up, removed from the phone, and would be faulted in if I browsed that far back in my on-phone viewer (Pixel 3a with recent updates, BTW).
I’m not going to take the time to isolate what’s going on with the empty dirs and formulate a precise question, here, but I wonder if you are sufficiently familiar with Google Photos to know what this indicates, if PhotoStructure handles this scenario, and if there’s a race condition, whereby PhotoStructure would need to identify a new image and import it before a later sync action replaced the actual image with one of these placeholders, developing.
Yup! I’d also directly-connect my phone to my computer and do a backup via the Finder/File Explorer into a totally different backup directory before I delete everything, though, just to be safe.
(There may be a reason why I needed to write a good de-duper for PhotoStructure…)
I think pixels just do this. I don’t know why. PhotoStructure is happy walking into any directory, though, and if it’s empty, no big deal.
If the sync runs later, and there are suddenly files in that folder: again, no big deal, they’re be imported.
If you have a Pixel, you’ll have full-quality backups if you pull down a Google Takeout. If you’re on a Mac or Linux box, you can then use ratarmount to “mount” all the tarballs into a single, merged directory (without taking all the time and disk space to uncompress everything!), and let PhotoStructure sync from there too.
I discussed ratarmount in this post, but I’ll make a proper instructions page soon that outlines exactly how to do this.
Multi-shot images are not aggregated currently, but there’s a feature request for that if you want to vote for it.
Edits from Google Photos will show up in your Google Takeout. I made the deduper’s heuristics such that these edits will, most likely, get picked as the “best” variant for an asset.
This is exactly what I’m going to try, except with a VPS rather than a NAS.
My ideal flow is:
incoming or import or similar folder on my server.Currently PhotoStructure only copies files (does not move them) so I think I’d end up with duplicate copies of each photo (one in the incoming folder and one in the library), until either this feature request is implemented or I write my own script to remove files from incoming once they’re in the PhotoStructure library.
I feel like a sync feature directly built in to PhotoStructure could be more sophisticated though. For example, if the photo already exists in the library (exact hash match), do not bother sending the file to the server.
I’ve considered adding an “asset change observer” setting that is a path to a script or executable. It would get called whenever an asset is changed, and gets a json payload via stdin that contains:
Would that be helpful?
Is there a better API for this that you two can think up?
That would be great! I think it would be sufficient for what I need. It’s definitely a power user feature but you could probably distribute PhotoStructure with some example scripts to demonstrate how to use it.
Since PhotoStructure uses Node.js, support for JS scripts would be great - you could probably execute them with whatever Node executable PhotoStructure itself is using?
Did I understand you correctly: that you’d use it to copy new files to cloud storage?
Are there other attributes you’d want?
I want to be able to delete the ‘original’ file once the asset has been copied into the PhotoStructure library, so that I don’t have the same photo in two different folders. Basically I just want the original folder to be a temporary holding place until PhotoStructure imports the photo into its library and copies it into the nicely-organised 2020/01 directory. I could just have a cronjob that removes files from the incoming folder once they’re in the library, but some hook in PhotoStructure like what you’re describing would be nicer.
Ciao! I was looking to have the same thing (syncthing + photostructure), but I noticed that this is quite different from the “backup” strategy:
In fact, if you set the shared folder on the phone as “send only” and you delete a photo on your phone, it will be deleted on the NAS too.
I still have to find how to configure SyncThing in that “backup” mode.
Welcome to PhotoStructure, @willywongi !
OH YIKES. Sorry about that! (I use Resilio Sync most of the time, but it’s mostly due to inertia, not that it’s better than SyncThing).
I’ve just edited the comment you quoted to be clearer and added some relevant links. Thanks so much for pointing this out!
Today I learned that a mini USB-C dock with ethernet can be plugged into an Android device and it’ll use the wired ethernet. Fancy.
My LAN backup from my phone to my NAS went from ~30MB/s to ~100MB/s:
I studied a bit more and I found out that with SyncThing you should turn on file versioning on the NAS/server.
https://docs.syncthing.net/users/versioning.html
You can then safely delete photos from your phone and the synced folder will keep those, moving them to a separate folder.
I’ll share my complete setup as soon as I finish it!
Hi @willywongi,
How is your setup going ? I’d like to play with SyncThing, but I don’t want to corrupt my storage hierarchy and database. So… waiting for your explanations on how to you use ST with PS, between your phone and your NAS/Server.
Ciao @GoustiFruit!
My setup is still not finished - I was planning on uploading my Google Photos (from Google Takeout) but I was stunned from the sheer amount of data I piled up.
I’m writing some notes on a gist about my setup; please note that I’m backing my photos on a VPS (not on my NAS) and I use Traefik to securely expose services.
Syncing to the server has always been the relatively easy part of phone/library syncing. The real magic would be having the phone recognize the remote image library as if the files are local (similar to how Google Photos works on Android).
Any workarounds for this?
If you have enough space on your phone, you could set up another sync folder in SyncThing and sync the Photostructure library folder back to you phone.
Something like:
You might need to set you phone gallery app to display that photoslibrary folder on you phone.
Note: I’m not using this setup with Photostructure right now, but I’m using it manually.