Incorrectly rotated .heic images

Bug
1

A beta tester reported an issue with incorrectly rotated .heic images (taken with an iPhone XS Max).

If you find any other mis-rotated .heic images, and they don’t have any private content, please send an example to support@photostructure.com so we can make sure the bugfix works on your files too.

2

Update: the next version of PhotoStructure will switch from in-process HEIC conversions (which requires a compilation) to using an external tool (much like how ffmpeg or VLC are utilized for transcoding jobs).

As long as heif-convert is in your PATH, HEIF images will be imported and viewed.

This method turns out handle these orientations correctly, be faster (!), and much easier to support from all of PhotoStructure’s various editions.

3

I’m afarid this issue persist in 1.0

All portrait HEIC files seem to be rotated 90 degrees right. It seems to be all portrait photos, all landscape photos are correct.

Viewing the file in imageglass or gimp shows the correct orientation.

I can provide example photos if you wish. I exported exif info via imageglass (uses exiftool)

ExifTool-----------------------------:
ExifTool Version Number : 12.09

File---------------------------------:
File Name : 20210107_084834515_iOS.heic
Directory : E:/PICTURES/DATE/2021/01
File Size : 2.7 MB
File Modification Date/Time : 2021:01:07 09:48:34+01:00
File Access Date/Time : 2021:01:12 12:16:06+01:00
File Creation Date/Time : 2021:01:07 09:56:06+01:00
File Permissions : rw-rw-rw-
File Type : HEIC
File Type Extension : heic
MIME Type : image/heic
Exif Byte Order : Big-endian (Motorola, MM)
Image Width : 4032
Image Height : 3024

QuickTime----------------------------:
Major Brand : High Efficiency Image Format HEVC still image (.HEIC)
Minor Version : 0.0.0
Compatible Brands : mif1, MiPr, miaf, MiHB, heic
Handler Type : Picture
Primary Item Reference : 49
Meta Image Size : 4032x3024
HEVC Configuration Version : 1
General Profile Space : Conforming
General Tier Flag : Main Tier
General Profile IDC : Main Still Picture
Gen Profile Compatibility Flags : Main Still Picture, Main 10, Main
Constraint Indicator Flags : 176 0 0 0 0 0
General Level IDC : 90 (level 3.0)
Min Spatial Segmentation IDC : 0
Parallelism Type : 0
Chroma Format : 4:2:0
Bit Depth Luma : 8
Bit Depth Chroma : 8
Average Frame Rate : 0
Constant Frame Rate : Unknown
Num Temporal Layers : 1
Temporal ID Nested : No
Image Spatial Extent : 4032x3024
Rotation : 270
Image Pixel Depth : 8 8 8
Media Data Size : 2820924
Media Data Offset : 3432

EXIF---------------------------------:
Make : Apple
Camera Model Name : iPhone 11
Orientation : Rotate 90 CW
X Resolution : 72
Y Resolution : 72
Resolution Unit : inches
Software : 14.2
Modify Date : 2021:01:07 09:48:35
Host Computer : iPhone 11
Y Cb Cr Positioning : Centered
Exposure Time : 1/121
F Number : 1.8
Exposure Program : Program AE
ISO : 50
Exif Version : 0231
Date/Time Original : 2021:01:07 09:48:35
Create Date : 2021:01:07 09:48:35
Offset Time : +01:00
Offset Time Original : +01:00
Offset Time Digitized : +01:00
Components Configuration : Y, Cb, Cr, -
Shutter Speed Value : 1/121
Aperture Value : 1.8
Brightness Value : 5.237780467
Exposure Compensation : 0
Metering Mode : Multi-segment
Flash : Off, Did not fire
Focal Length : 4.3 mm
Subject Area : 2002 1505 2213 1324
Sub Sec Time Original : 181
Sub Sec Time Digitized : 181
Flashpix Version : 0100
Color Space : Uncalibrated
Exif Image Width : 4032
Exif Image Height : 3024
Sensing Method : One-chip color area
Scene Type : Directly photographed
Exposure Mode : Auto
White Balance : Auto
Focal Length In 35mm Format : 26 mm
Scene Capture Type : Standard
Lens Info : 1.539999962-4.25mm f/1.8-2.4
Lens Make : Apple
Lens Model : iPhone 11 back dual wide camera 4.25mm f/1.8
Composite Image : General Composite Image
GPS Latitude Ref : North
GPS Longitude Ref : East
GPS Altitude Ref : Above Sea Level
GPS Speed Ref : km/h
GPS Speed : 0.159621492
GPS Img Direction Ref : True North
GPS Img Direction : ########
GPS Dest Bearing Ref : True North
GPS Dest Bearing : #######
GPS Date Stamp : 2021:01:07
GPS Horizontal Positioning Error : 8.872468793 m

MakerNotes---------------------------:
Run Time Flags : Valid
Run Time Value : 1889156185867958
Run Time Scale : 1000000000
Run Time Epoch : 0
Acceleration Vector : -0.07901333276 -0.4597193001 -0.865364373

ICC_Profile--------------------------:
Profile CMM Type : Apple Computer Inc.
Profile Version : 4.0.0
Profile Class : Display Device Profile
Color Space Data : RGB
Profile Connection Space : XYZ
Profile Date Time : 2017:07:07 13:22:32
Profile File Signature : acsp
Primary Platform : Apple Computer Inc.
CMM Flags : Not Embedded, Independent
Device Manufacturer : Apple Computer Inc.
Device Model :
Device Attributes : Reflective, Glossy, Positive, Color
Rendering Intent : Perceptual
Connection Space Illuminant : 0.9642 1 0.82491
Profile Creator : Apple Computer Inc.
Profile ID : ca1a9582257f104d389913d5d1ea1582
Profile Description : Display P3
Profile Copyright : Copyright Apple Inc., 2017
Media White Point : 0.95045 1 1.08905
Red Matrix Column : 0.51512 0.2412 -0.00105
Green Matrix Column : 0.29198 0.69225 0.04189
Blue Matrix Column : 0.1571 0.06657 0.78407
Red Tone Reproduction Curve : (Binary data 32 bytes, use -b option to extract)
Chromatic Adaptation : 1.04788 0.02292 -0.0502 0.02959 0.99048 -0.01706 -0.00923 0.01508 0.75168
Blue Tone Reproduction Curve : (Binary data 32 bytes, use -b option to extract)
Green Tone Reproduction Curve : (Binary data 32 bytes, use -b option to extract)

Composite----------------------------:
Run Time Since Power Up : 21 days 20:45:56
Aperture : 1.8
Image Size : 4032x3024
Lens ID : iPhone 11 back dual wide camera 4.25mm f/1.8
Megapixels : 12.2
Scale Factor To 35 mm Equivalent : 6.1
Shutter Speed : 1/121
Create Date : 2021:01:07 09:48:35.181+01:00
Date/Time Original : 2021:01:07 09:48:35.181+01:00
Modify Date : 2021:01:07 09:48:35+01:00
GPS Altitude : 18.8 m Above Sea Level
GPS Latitude : ##########
GPS Longitude : #############
Circle Of Confusion : 0.005 mm
Field Of View : 69.4 deg
Focal Length : 4.3 mm (35 mm equivalent: 26.0 mm)
GPS Position : 59 deg 21’ 37.14" N, 17 deg 54’ 18.61" E
Hyperfocal Distance : 2.04 m
Light Value : 9.6

4

That’d be super, thanks!

5

I also see this in some of my heic photos, I’ll send a few examples

6

Yep… same here.

And if I use the “rotate” option, it rotates it, doesn’t handle the aspect ratio correctly, and then after a fairly short time period, it rotates it back.

I can provide example images if you want them.

7

Experiencing the same issue here with 1.0 beta 12 installed via Docker Compose.

8

I am using version 1.1.0 and the latest stable docker image.

I did some testing to see which photos are affected (iPhone 12). I mixed portrait/landscape pictures with live and non-live photos and aspect ratio.
The result seems to indicate that 16:9 photos are correctly rotated, but 4:3 photos are not rotated properly.

9

I spent some time figuring out the differences between 4:3 and 16:9 photos. These are my findings with respect to the exif properties,

For 16:9 photos
Property ‘Orientation’ not present
Property ‘Rotation’ is always 0.
Image size reflects rotation, e.g. 4032x2268 for landscape or 2268x4032 for portrait

For 4:3 photos
Property ‘Orientation’ present, and correct, e.g. ‘Rotate 90 CW’
Property ‘Rotation’ present, and correct, e.g. ‘270’
Image size is always 4032x3024

Possible explanation
To me this seems to indicate that PhotoStructure does not apply rotation at all.
For 16:9 photos it just so happens that the iPhone encodes the picture “as it should be” already, so not applying a rotation gives the correct result.

10

It does look like my incorrectly rotated iPhone images are 4:3 aspect ratio, so this lines up with vgt’s comments.

11

I agree with the above commenters, and I want to add one more wrinkle: the incorrect rotation issue does not appear on macOS. I created two PhotoStructure instances - one Dockerized on a Linux machine and one desktop macOS. The same HEIC image is rotated incorrectly on the Linux version and correctly on the macOS version.

Same issues as vgt. When I try to rotate the HEIC on the Docker instance, the aspect ratio gets all messed up and I get an error, “Well darn, that didn’t work” and sometimes I get an error about failure to rotate.

I noticed that all the incorrectly rotated photos were taken by my wife’s iPhone. I think both phones are set to the same aspect ratio, but I’m not sure off the top of my head.

12

Thanks for that additional information! I’ll try again to reproduce this issue within docker.

13

The same HEIC image is rotated incorrectly on the Linux version and correctly on the macOS version.

I’m using PhotoSructure 1.1.0 for Mac and I’m seeing incorrectly rotated .heic images.
I think they all are 4:3 aspect ratio (4032 × 3024).
“Landscape” photos seem to be ok, “portrait” photos not.
Photos were taken with Samsung S20FE.

14

@clinis can you email support@photostructure.com an original image that isn’t oriented correctly (as long as there’s no private content in it?)

Thanks!

15

Howdy! Before I mark this bug as resolved, I wanted to save the analysis I had to do here: it might be interesting to (a tiny fraction) of people impacted by this issue, and it’d be nice to be able to refer to it in the release notes.

:camera: Background

PhotoStructure uses heif-convert to convert HEIF/HEIC images to browser-accessible JPEGs.

I’d verified that heif-convert worked properly on Mac and Linux. Mac uses the built-in sips, and Linux boxen ship with heif-convert from the libheif-examples package.

I couldn’t reproduce this issue because I would always try examples on PhotoStructure for Desktop on macOS and Ubuntu, on PhotoStructure for Servers using Node, and it worked on my machine.

On Ubuntu 20.04, the version of heif-convert, v1.6, emits correctly oriented JPEGs: the source orientation is ignored, the converted image is never rotated, and because the Orientation metadata is retained, everything works.

:man_detective: Clues

I (finally!) noticed that the Alpine/docker version of heif-convert was a different version. (Could that be it?)

I discovered that version 1.9 and later emit rotated JPEGs (as opposed to the older versions, that ignore rotation). What’s incorrect is that they also copy the source EXIF metadata, unadjusted, to the destination JPEG. This results in the encoded image being correctly oriented, but due to the retained Orientation tag, the image is rotated, again, and boom, we have this bug.

:scientist: Solutions

My initial thought was to only apply a bugfix to the buggy versions, but I can’t tell what version is installed because heif-convert doesn’t respond to --version and all versions link to /usr/lib/libheif.so.1 :weary:.

I thought of adding heuristics to detect if the converted JPEG from HEIC had incorrect dimensions, but this doesn’t handle when people hold their phone “upside-down”, nor does it handle square originals (which is an Instagram default).

I can tell heif-convert to emit PNG, which resolves this issue in all versions (!!), but that causes conversion time to take 3-5x longer (on my workstation, about 1s for heif->jpeg for a typical iPhone 13 image, and 4-5s for heif-png for the same image.

I think for now it’s safest to just take the performance hit to resolve the issue for all cases and all versions.

:hourglass: When will this bugfix be available?

Version 2.1 includes this fix, and a fancy-pants operation to automatically repair the previews for impacted assets as soon as you upgrade.

Alpha builds of Version 2.1 will be released soon (hopefully this week).

3 Likes