Poor transcoding of videos in wider color spaces (Rec. 2020)

Bug
,
#1

Expected Behavior

Rec.2020 videos should be transcoded to a lower bitrate video in the same color space, not Rec.709.

Current Behavior

Rec.2020 videos are transcoded to Rec.709, and have completely blown out colors.

Steps to Reproduce

  1. Enable MP4 transcoding with the environment variable
    “PS_DO_NOT_TRANSCODE_MIMETYPES=”
  2. Add a MP4 video with Rec.2020 color space
  3. In the web viewer, observe that colors are completely blown out. Light tones pure white, and colors are extremely saturated.

Environment

Operating system and version: Debian 10.7

PhotoStructure edition: PhotoStructure for Docker

Extra info

The colors in the videos were fine before I enabled transcoding for MP4 videos. Only after I enabled transcoding and rebuilt the photostructure library did the colors get wacky.

1.0.0-beta.9 is ready
Video transcoding problem with Apple HEVC.MOV files
Trouble transcoding iPhone8 video, possibly stalling progress of library import
#2

Thanks for reporting this!

Out of curiosity, what device is producing high-gamut colorspace video?

The arguments that PhotoStructure uses for ffmpeg are actually configurable via the ffmpegTranscodeArgs setting (configurable in the system settings.toml or via environment variable). The default is this:

-c:a aac -c:v libx264 -pix_fmt yuv420p -profile:v high

FWIW, it took way longer than I expected to come up with the current ffmpeg incantation that produces mp4s that play on mobile and desktop versions of Safari, Chrome, and Firefox. I’d happily trade you a coupon for a free year of PhotoStructure PLUS if you figure out the additional option(s) needed to make ffmpeg behave correctly and still retain cross-platform support!

#3

What device?

I used a Sony A7C in HLG mode to produce the original video, then color graded it in Catalyst Browse, but kept the output color space as Rec.2020 (why not, right?). The videos look normal when viewed on my phone or desktop natively.

#4

I googled a bit and found https://video.stackexchange.com/a/32398

Can you add this to your system settings.toml:

ffmpegTranscodeArgs = [
  "-c:a",
  "aac",
  "-c:v",
  "libx264",
  "-pix_fmt",
  "yuv420p",
  "-color_trc",
  "smpte2084",
  "-color_primaries",
  "bt2020",
  "-profile:v",
  "high",
]

Restart PhotoStructure, navigate to the video, pick “resync this asset”, wait until ffmpeg is done (it may take about as long as the video to transcode, depending on the speed of your CPU), and then reload the page.

Fingers crossed it won’t look like blown-out garbage…

#5

Do I need the 1.0 beta for that ffmpeg argument to work? It didn’t seem to do anything different.

#6

Yeah, it’s a fairly recent addition.

You can tell what settings are available for each version by looking at the defaults.env on GitHub:

#7

I can confirm this works, but only after a rebuild of the library. Manually syncing the asset, or syncing a directory with the CLI “sync” command, did not seem to re-create the transcoded video.

By the way, should the ffmpeg command also include “-hwacel”, “auto” to use the CPU or GPU hardware capabilities? It does take a significant amount of CPU time to transcode long 100 Mpbs videos.

#8

I’ll add this to the default in the next beta and we can see if it works for everyone. Thanks for the suggestion!

#9

sorry, should be “hwaccel”.

Source docs:
https://trac.ffmpeg.org/wiki/HWAccelIntro

#10

Dang, ffmpeg is picky with where -hwaccel auto is placed. This will require a new setting.

See Hardware-accelerated encoding/transcoding

#12
#13