Several people have asked for a native app to upload photos and videos from their phones to their PhotoStructure library.
There are several apps that do this already, and do this quite well, so this isnāt something weāre going to work on anytime soon.
I personally use Resilio Sync to copy files as I take them to my NAS at home. PhotoStructure sync is configured to import from the Resilio directory and automatically organize the new photos and videos into my library.
The three options presented on the page you referenced, Resilio Sync, SyncThing, and PhotoSync, appear not to simply upload images, but to synchronize multiple sets of files, such as those on oneās phone and those on another storage device. āUploadā implies a one-way transfer, generally from a smaller to larger device, which is a very different operation that synchronization.
This seems to present two problems for the upload-from-phone-to-PhotoStructure use case: 1) Changes made on the secondary device would be replicated to the phone, potentially breaking native photo apps by violating assumptions they make about naming, tagging, or organization of images, and 2) images deleted from the phone (as one often does as storage on the phone runs short) would be deleted on the secondary storage, resulting in permanent loss of images that one expected to be safely archived off-phone.
Did you intend to suggest an alternative to uploading images from oneās phone to oneās PhotoStructure library, and for that to be a multi-step process involving 1) continuously synchronizing the images on a phone to a secondary data store on another device, and 2) arranging for PhotoStore to periodically check for new images on the secondary store, and when found, copy them to a third, PhotoStore-specific, data store?
Resilio and SyncThing both support different modes:
āmirrorā mode, where deletes on the source device are reflected on the destination device
āsend onlyā mode, where deletes on the source device are ignored.
You want the second option, so if your phone decides to āsmart deleteā photos that were backed up somewhere else, the backup on your NAS doesnāt synchronize those deletions.
If youāre using SyncThing, you want āsend onlyā mode. If youāre using Resilio Sync, you want āRead Only-Full Syncā and āStore deleted files in folder archiveā on the server folder (which should be set by default).
With respect to how I suggest people set up files, itās up to you: I personally like to have my photos in one directory hierarchy, so I use automatic organization, but Iāve retained all my Resilio archives separately, as well (disk is cheap).
If youād rather maintain a single copy of your photos and videos, you can either opt out of automatic library organization, or you can periodically archive your Resilio backup directory (take care to not delete the files that the app uses to maintain sync state, and realize you shouldnāt archive files on dest that still exist on src, as the sync app will re-transfer everything you just archived).
Syncthing is amazing. Iād been trying halfheartedly for a while to get it working over Tailscale from various devices to my NAS. With my phoneās storage rapidly approaching full, and my concern about what happens if and when I let Google Photos delete images from the phone that āare backed upā to the appās default and opaque off-site storage facility, I gave up on that and let Syncthing do its NAT-busting, connection-finding, no-screentapping-qrcode-scanning magic. Wow, itās slick.
I moved my Keepass database out of SpiderOak (which I never got working on Android) and Keybase FS (an old copy of the Keepass DB, to sometimes awkwardly retrieve on the phone), and started syncing it around to Linux, Mac OS, Android, and Chrome OS. Yee-haw. I donāt know if I got this right and have mild concern about a sort of hall-of-mirrors syncing among the various machines, but it seems to be working.
Of relevance to this forum: Moving toward my hopes of trying out PhotoStructure, and wanting a copy of images on my phone that are not cycled through Google Takeout, not tied up in whatever Synology Moments does with them server-side, and not requiring me to fix whatever is going on with my Perkeep installation, I pointed Syncthing on the phone at at /storage/emulated/0/DCIM and shared it with Syncthing on the NAS. I set the phoneās config for this share to āsend onlyā and that of the NAS to āreceive only.ā 29 GB transferred, and maybe I have a good copy of my images and videos on which for PhotoStructure to feast when I install it.
Iām left with some concerns. One is off-topic here (re: multi-shot āimagesā, whatever happens with edits in Google Photos that it says are undoable, etc.) so Iāll write up another post asking about it.
The other is that I see lots of empty directories named like images (e.g. IMG_20191226_173817) in the copy one-way-synced to my NAS. Spread among them are image files, named similarly, but with a .jpg suffix (e.g. IMG_20191226_174108.jpg). Prevalence is much higher for older images, so I wonder if these are placeholders for image files that were backed up, removed from the phone, and would be faulted in if I browsed that far back in my on-phone viewer (Pixel 3a with recent updates, BTW).
Iām not going to take the time to isolate whatās going on with the empty dirs and formulate a precise question, here, but I wonder if you are sufficiently familiar with Google Photos to know what this indicates, if PhotoStructure handles this scenario, and if thereās a race condition, whereby PhotoStructure would need to identify a new image and import it before a later sync action replaced the actual image with one of these placeholders, developing.
Yup! Iād also directly-connect my phone to my computer and do a backup via the Finder/File Explorer into a totally different backup directory before I delete everything, though, just to be safe.
(There may be a reason why I needed to write a good de-duper for PhotoStructureā¦)
I think pixels just do this. I donāt know why. PhotoStructure is happy walking into any directory, though, and if itās empty, no big deal.
If the sync runs later, and there are suddenly files in that folder: again, no big deal, theyāre be imported.
If you have a Pixel, youāll have full-quality backups if you pull down a Google Takeout. If youāre on a Mac or Linux box, you can then use ratarmount to āmountā all the tarballs into a single, merged directory (without taking all the time and disk space to uncompress everything!), and let PhotoStructure sync from there too.
I discussed ratarmount in this post, but Iāll make a proper instructions page soon that outlines exactly how to do this.
Multi-shot images are not aggregated currently, but thereās a feature request for that if you want to vote for it.
Edits from Google Photos will show up in your Google Takeout. I made the deduperās heuristics such that these edits will, most likely, get picked as the ābestā variant for an asset.
This is exactly what Iām going to try, except with a VPS rather than a NAS.
My ideal flow is:
incoming or import or similar folder on my server.Currently PhotoStructure only copies files (does not move them) so I think Iād end up with duplicate copies of each photo (one in the incoming folder and one in the library), until either this feature request is implemented or I write my own script to remove files from incoming once theyāre in the PhotoStructure library.
I feel like a sync feature directly built in to PhotoStructure could be more sophisticated though. For example, if the photo already exists in the library (exact hash match), do not bother sending the file to the server.
Iāve considered adding an āasset change observerā setting that is a path to a script or executable. It would get called whenever an asset is changed, and gets a json payload via stdin that contains:
Would that be helpful?
Is there a better API for this that you two can think up?
That would be great! I think it would be sufficient for what I need. Itās definitely a power user feature but you could probably distribute PhotoStructure with some example scripts to demonstrate how to use it.
Since PhotoStructure uses Node.js, support for JS scripts would be great - you could probably execute them with whatever Node executable PhotoStructure itself is using?
Did I understand you correctly: that youād use it to copy new files to cloud storage?
Are there other attributes youād want?
I want to be able to delete the āoriginalā file once the asset has been copied into the PhotoStructure library, so that I donāt have the same photo in two different folders. Basically I just want the original folder to be a temporary holding place until PhotoStructure imports the photo into its library and copies it into the nicely-organised 2020/01 directory. I could just have a cronjob that removes files from the incoming folder once theyāre in the library, but some hook in PhotoStructure like what youāre describing would be nicer.
Ciao! I was looking to have the same thing (syncthing + photostructure), but I noticed that this is quite different from the ābackupā strategy:
In fact, if you set the shared folder on the phone as āsend onlyā and you delete a photo on your phone, it will be deleted on the NAS too.
I still have to find how to configure SyncThing in that ābackupā mode.
Welcome to PhotoStructure, @willywongi !
OH YIKES. Sorry about that! (I use Resilio Sync most of the time, but itās mostly due to inertia, not that itās better than SyncThing).
Iāve just edited the comment you quoted to be clearer and added some relevant links. Thanks so much for pointing this out!
Today I learned that a mini USB-C dock with ethernet can be plugged into an Android device and itāll use the wired ethernet. Fancy.
My LAN backup from my phone to my NAS went from ~30MB/s to ~100MB/s:
I studied a bit more and I found out that with SyncThing you should turn on file versioning on the NAS/server.
https://docs.syncthing.net/users/versioning.html
You can then safely delete photos from your phone and the synced folder will keep those, moving them to a separate folder.
Iāll share my complete setup as soon as I finish it!
Hi @willywongi,
How is your setup going ? Iād like to play with SyncThing, but I donāt want to corrupt my storage hierarchy and database. Soā¦ waiting for your explanations on how to you use ST with PS, between your phone and your NAS/Server.
Ciao @GoustiFruit!
My setup is still not finished - I was planning on uploading my Google Photos (from Google Takeout) but I was stunned from the sheer amount of data I piled up.
Iām writing some notes on a gist about my setup; please note that Iām backing my photos on a VPS (not on my NAS) and I use Traefik to securely expose services.
Syncing to the server has always been the relatively easy part of phone/library syncing. The real magic would be having the phone recognize the remote image library as if the files are local (similar to how Google Photos works on Android).
Any workarounds for this?
If you have enough space on your phone, you could set up another sync folder in SyncThing and sync the Photostructure library folder back to you phone.
Something like:
You might need to set you phone gallery app to display that photoslibrary folder on you phone.
Note: Iām not using this setup with Photostructure right now, but Iām using it manually.