Automatic face tagging

From what’s next

:robot: Add ML-powered face-and-thing tagging. As always with PhotoStructure, all computation happens locally on your computer. Your data stays yours.

Note that “face tagging” is actually a pretty substantial bit of work that will be shipped in several steps:

  1. Extract and browse by face/person tags already embedded in metadata/google takeouts (this will ship in v1.0.0)

  2. Add ML-powered face detection and aggregation (v1.1 – this will leverage existing face locii produced by 1., so importing a new Google Takeout that has person tags will be really handy here).

  3. Add UI to correct any face detection errors and manage person “aliases” (v1.2+)

4 Likes

v1.1 would work without any Google tags of any kind, correct? The “this will leverage existing…” phrase does not explain how Google tags would be helpful. Do you just mean they’d reduce the local computation necessary for PhotoStructure?

I’d rather Google never see any of my photos, so I’ve disabled automatic uploads to Google Photos and instead use the OneDrive app to instant backups from the phone, and periodically migrate photos off of OneDrive and into my NAS. Although some of my family does use Google Photos as well, so it’d be a mix in the long term.

Yes, absolutely. PhotoStructure doesn’t require Google Photos: but if you have a Google Takeout, it knows how to read the JSON files in those archives (and, frankly, I think Google Photos has the best face detection that I’m aware of–if anyone’s already uploaded any portion of their library to GP, they might as well leverage it).

I have used Google Photos and now transitioning to PhotoStructure. All my photos are stored locally, which is what I will use. I have used Google Photos from face/thing recognition in the past. I have a Google Takeout of all photos.

How will I be able to use the face recognition already undertaken, but not include the Google Takeout photos?

1 Like

Howdy, @harrisc: welcome to PhotoStructure!

Are you not wanting to import to “high quality” images into your library (because you have your originals)?

Know that PhotoStructure does de-dupe these images, and should merge the original and the downsampled google photo copy into a single visible asset.

I hadn’t thought of this scenario: you only want the metadata/sidecar from your takeout?

I guess PhotoStructure could skip copying asset file variants when they aren’t going to be the “primary” asset, but still copy over the sidecar?

Or do you mean something else?

1 Like

I was thinking of not importing the photos as I had the originals, although Google had done some useful edits and collages. I have kept the original photos in particular directories which I was planning to keep.

If I import the Google Takeout photos and de-duped are they added to my existing directories or I still need to keep all Google Takeout photos?

Chris

If you’re using automatic organization, the downsampled image from Takeout, along with their sidecars, will be copied into your library, because they have unique contents.

If the file is already in your PhotoStructure library directory (even if it’s a non-datestamped folder), it won’t be moved.

Lots more details here:

1 Like

I also suggest to add an option of automatic importing people from the keywords. Some well-organized collections might have people tagged in hierarchical keywords like Who > Friends > John Doe or People > Friends > John Doe etc. Giving a setting to choose this parent tag Who or People will be a good hint for the automated face tagging system

What you described was added to v1.0.0! https://photostructure.com/about/2021-release-notes/#new-curators

I’ll write up a more detailed post about it soon.

1 Like

PhotoStructure actually already does this! See the tagWhoSynonyms setting (it defaults to “People”, “Face”, and “Faces”).

I will be leveraging prior Region rects to bootstrap face centroids, as well.

1 Like

Being able to pull from faces already detected from Google Photos is a vastly appreciated feature. That will probably be the point where I stop uploading to Google Photos entirely.

PhotoStructure v1.0 does this!

If you have a recent Google Takeout, they started adding people tags to their proprietary JSON sidecars. Version 1.0 knows how to extract that metadata :+1:

As indicated section 1 has been done.

Is there any indication of the timetable for the rest?

For those wanting to get started on this before it’s added to PhotoStructure, two software solutions I found are:

TagThatPhoto / Tag That Photo is Windows-only and charges a subscription to recognize more than 10 people. It uses commercial face recognition software from Applied Recognition which review say is quite good, and you can also use some peer-to-peer stuff to collaborate on tagging with others.

digiKam is free, open-source, and cross-platform (Windows, macOS, Linux, and BSD). But I guess the open-source facial recognition libraries aren’t as good yet as commercial ones.

Both allow you to manually tag people who aren’t recognized, for example if you can only see the back of their head they aren’t automatically recognized. You could of course just add hierarchical tags for names of people in the photos, but I recommend using the manual tagging feature to add names for particular region of photos, so that extended family/friends/descendants can still know exactly which person is connected to which name.

Also see related forum thread Facial Recognition / Additional Photos / Google Takeout

I use TagThatPhoto, and it works great with Photostructure. If anybody has any questions, I’ll be happy to share what I know!

I just started using PhotoStructure and I’m really liking it so far. I tested out PhotoPrism as well and this feels much less clunky! I’m wondering if this is going to be implemented soon. I paid for a year of TagThisPhoto but it would be great to do everything in one program. I’m running PhotoStructure on Unraid with my photos on Dropbox mounted with rclone. This works great as new photos are uploaded from my phone right to Dropbox and pulled into Photostructure. I’m hoping eventually these will be able to be automatically tagged within PhotoStructure.

Hi there, I am also interested in understanding if these functionalities have already been developed or they are still in the pipeline. Thanks!

Not implemented yet. Matthew has said in Discord that first priority is another stable release, then next priority will be supporting multiple users.

1 Like