@mrm this discussion has me questioning the very foundation of my workflow!
Iāve indeed discovered photostructure re-adding photos that were already thereābecause I retroacively tagged version A of the photo in the ps library with digikam.
Ps circled back at some time later and found a different sha of that image and brought it in as *-1.jpg. I have 6035 of these now!
Iām trying to figure out how to keep photostructure in my workflowāI love its deduplication and presentation style. Digikam is great for deep management, but not great for displaying.
It would be neat to confg PS to overlook exifdata when deduplicating, but I imagine that would take an entirely different approach than just hashing the file. Iāve seen some python modules that do funky math magic to create some sort of common-pixel fingerprint on just the image dataā¦ that dedup work was able to identify the same image even if it were resized.
Do you have any suggestions on a sequence or strategy that allows retroactive tagging of things in a ps library?
Oof, sorry for PhotoStructure adding those duplicates!
Unfortunately, v2.1.0-alpha.7 only had one image hash algorithm (a āmean hashā triplet in L*a*b colorspace), which (by design) would collide with very similar image content.
Prior versions just looked at the image hash, which resulted in quite-differently-timestamped images being aggregated together if the shot was basically the same.
The next build includes two new image hash algorithmsāDCT and gradient/diff hashesāso if all three hashes match, itās very similar image content.
Soāwhat would be the ācleverā solution for PhotoStructure to not just generate more duplicates in your library? Basicallyāhow do I do the right thing if one variation of a photosā captured-at time was edited to be correct, but there are other variations with the old (incorrect) time?
Approach 0: PhotoStructure edits the captured-at time, and therefore knows whatās going on, and can do the right thing.
Approach 1: I could SHA just the image content (basically SHAāing the image after stripping all tags), but this can be ātrickedā by editing the image contents.
Approach 2: Some software applies a unique identifier (a āuuidā) to all imagesāand all edits, as long as the uuid persists, are considered the āsameā image. The issue with this approach is it hopes that other external software will retain these UUID tag values when editing/re-saving. Some image formats (like Appleās live-photo HEIC) already include UUIDs.
Can you think of another approach? Iām happy to talk through that (either here or on discord).
I think approach 1 seems the most reasonable among the three because:
ps already has a sha hashing framework in place so regression testing would be brief
exif is generally standardized in its placement and structure, so an exif stripper ought not be too difficult to implement in a way that it doesnāt damage sync durations
I think if the image itself changes (and thus the image sha), it would be reasonable to consider it a different photo. Even something relatively ordinary as a whitebalance correction would produce an image that looks different from its parent. Despite that, bloating a date with 10 near-alike pictures because a user was tinkering with filters will feel burdensome later when theyāre just looking through the album with friends. Having a ps option to group very-similar items together and visually display the most recent/oldest/biggest/exif-heaviest might be a nice way to coexist with such duplicates and reduce visual clutter while retaining recognition of their differences, should that level of distinction be desired now and then.
I think most users would find being able to retroactively edit the metadata of their library useful. Ps is a very fine cataloging and displaying application that stakes no claim on classification. This leaves a complimentary place open for such a thing (like digikam).
One thing I forgot to mention is that Iām still using v 1.1.0. Iāve tried 2.* versions with snapshots of this library and both times ended in ruin (though I no longer remember the specifics). If it would be of interest to you, I can give it another go if thereās a particular chunk of telemetry youād be interested in seeing when ps 2.x is placed in front of 180,000 pictures.
@tkohhh : Yesāautomatic organization has to be enabled for PhotoStructure to make copies of a given photo or video.
Automatic organization (as it stands) ensures thereās a copy of every unique file SHA in your library. Unfortunately, every metadata edit will result in a āuniqueā new file SHA.
Iām warming up to āapproach 1ā above. ExifTool just added ImageDataMD5 support for several filetypes (unfortunately, it doesnāt handle HEIC/HEIF or many video types yet, but every part helps), and I just included that into exiftool-vendored (which is the library PhotoStructure uses to do metadata I/O):