Nvidia Drivers - Docker

adamf · June 7, 2022, 3:41am

Hey, I just wanted to circle back around on this ✅ Hardware-accelerated encoding/transcoding - #14 by adamf

I was doing some testing with Photostructure alpha1 alongside Frigate NVR and Plex. I compared some outputs of ffmpeg and nvidia-smi.

First - if I run nvidia-smi on my host, I can see the containers using various processes. In this case, I have five security cameras and two instances of Plex transcoding a video with my P2000 card.

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.129.06   Driver Version: 470.129.06   CUDA Version: 11.4     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Quadro P2000        Off  | 00000000:01:00.0 Off |                  N/A |
| 61%   64C    P0    24W /  75W |    912MiB /  5059MiB |      1%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A     20521      C   ffmpeg                            111MiB |
|    0   N/A  N/A     20523      C   ffmpeg                            111MiB |
|    0   N/A  N/A     20572      C   ffmpeg                            111MiB |
|    0   N/A  N/A     20574      C   ffmpeg                            111MiB |
|    0   N/A  N/A   2405990      C   ...diaserver/Plex Transcoder      177MiB |
|    0   N/A  N/A   2409847      C   ffmpeg                            111MiB |
|    0   N/A  N/A   2473381      C   ...diaserver/Plex Transcoder      177MiB |
+-----------------------------------------------------------------------------+

If I run the same incantation inside of Plex or Frigate, they show the RAM is in use but (and I’ve never figured this out) they do not display the processes that each container itself is using.

Frigate, for example:

docker exec -ti frigate sh -c 'nvidia-smi'
Mon Jun  6 20:26:07 2022       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.129.06   Driver Version: 470.129.06   CUDA Version: 11.4     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Quadro P2000        Off  | 00000000:01:00.0 Off |                  N/A |
| 61%   63C    P0    24W /  75W |    912MiB /  5059MiB |      1%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
+-----------------------------------------------------------------------------+

Now, the way I pass through the GPU to both of those containers (and attempt to with Photostructure) is via a runtime and an environment flag. I’m snipping the rest for brevity.

  frigate:
    image: blakeblackshear/frigate:stable-amd64nvidia
    privileged: true
    runtime: nvidia
    environment:
      - NVIDIA_VISIBLE_DEVICES=all
      - NVIDIA_DRIVER_CAPABILITIES=compute,utility,video

Similarly, Plex:

  plex:
    image: lscr.io/linuxserver/plex
    runtime: nvidia
    environment:
      - NVIDIA_VISIBLE_DEVICES=all

Note that Plex functions just fine with regard to hardware transcoding without the capabilities being specified and without privilege so I don’t believe that is the problem.

When I add the same flags (visible devices and runtime) to Photostructure, I partly get what I want. Nvidia devices show up inside of /dev.

Issue: nvidia-smi is missing from Photostructure. It returns an error when attempting to run. This would indicate the drivers do not exist. I’m sure you already know they are not included as we discussed it on the prior thread, but I’m just going through my thought process.

Next I tested hwaccel. Plex’s container does not include it but here is Frigate and Photostructure.

Frigate

docker exec -ti frigate sh -c 'ffmpeg -hwaccels'
ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --disable-debug --disable-doc --disable-ffplay --enable-shared --enable-avresample --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libfreetype --enable-libvidstab --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxcb --enable-libx265 --enable-libxvid --enable-libx264 --enable-nonfree --enable-openssl --enable-libfdk_aac --enable-postproc --enable-small --enable-version3 --enable-libzmq --extra-libs=-ldl --prefix=/opt/ffmpeg --enable-libopenjpeg --enable-libkvazaar --enable-libaom --extra-libs=-lpthread --enable-libsrt --enable-libaribb24 --enable-nvenc --enable-cuda --enable-cuvid --enable-libnpp --extra-cflags='-I/opt/ffmpeg/include -I/opt/ffmpeg/include/ffnvcodec -I/usr/local/cuda/include/' --extra-ldflags='-L/opt/ffmpeg/lib -L/usr/local/cuda/lib64 -L/usr/local/cuda/lib32/'
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Hardware acceleration methods:
cuda

Photostructure

docker exec -ti photostructure sh -c 'ffmpeg -hwaccels'
ffmpeg version 4.4.1 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 10.3.1 (Alpine 10.3.1_git20211027) 20211027
  configuration: --prefix=/usr --enable-avresample --enable-avfilter --enable-gnutls --enable-gpl --enable-libass --enable-libmp3lame --enable-libvorbis --enable-libvpx --enable-libxvid --enable-libx264 --enable-libx265 --enable-libtheora --enable-libv4l2 --enable-libdav1d --enable-postproc --enable-pic --enable-pthreads --enable-shared --enable-libxcb --enable-libsrt --enable-libssh --enable-libvidstab --disable-stripping --disable-static --disable-librtmp --enable-libaom --enable-libopus --enable-libsoxr --enable-libwebp --enable-vaapi --enable-vdpau --enable-vulkan --disable-debug
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Hardware acceleration methods:
vdpau
vaapi
vulkan

Issue: cuda does not appear. Again I’m sure this is driver related.

All that being said, Frigate’s solution to this is to host a separate dockerfile image for different use cases. He has an arch image and an arm image, but most importantly he has both an amd64 and an amd64 with nvidia support.

github.com

blakeblackshear/frigate/blob/master/docker/Dockerfile.amd64

FROM frigate-base
LABEL maintainer "blakeb@blakeshome.com"

# By default, use the i965 driver
ENV LIBVA_DRIVER_NAME=i965
# Install packages for apt repo

RUN wget -qO - https://repositories.intel.com/graphics/intel-graphics.key | apt-key add - \
    && echo 'deb [arch=amd64] https://repositories.intel.com/graphics/ubuntu focal main' > /etc/apt/sources.list.d/intel-graphics.list \
    && apt-key adv --keyserver keyserver.ubuntu.com --recv-keys F63F0F2B90935439 \
    && echo 'deb http://ppa.launchpad.net/kisak/kisak-mesa/ubuntu focal main' > /etc/apt/sources.list.d/kisak-mesa-focal.list

RUN apt-get -qq update \
    && apt-get -qq install --no-install-recommends -y \
    # ffmpeg dependencies
    libgomp1 \
    # VAAPI drivers for Intel hardware accel
    libva-drm2 libva2 libmfx1 i965-va-driver vainfo intel-media-va-driver-non-free mesa-vdpau-drivers mesa-va-drivers mesa-vdpau-drivers libdrm-radeon1 \
    && rm -rf /var/lib/apt/lists/* \
    && (apt-get autoremove -y; apt-get autoclean -y)

This file has been truncated. show original

github.com

blakeblackshear/frigate/blob/master/docker/Dockerfile.amd64nvidia

FROM frigate-base
LABEL maintainer "blakeb@blakeshome.com"

# Install packages for apt repo
RUN apt-get -qq update \
    && apt-get -qq install --no-install-recommends -y \
    # ffmpeg dependencies
    libgomp1 \
    && rm -rf /var/lib/apt/lists/* \
    && (apt-get autoremove -y; apt-get autoclean -y)


# nvidia layer (see https://gitlab.com/nvidia/container-images/cuda/blob/master/dist/11.1/ubuntu20.04-x86_64/base/Dockerfile)
ENV	    NVIDIA_DRIVER_CAPABILITIES compute,utility,video

RUN apt-get update && apt-get install -y --no-install-recommends \
    gnupg2 curl ca-certificates && \
    curl -fsSL https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/7fa2af80.pub | apt-key add - && \
    echo "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64 /" > /etc/apt/sources.list.d/cuda.list && \
    echo "deb https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu2004/x86_64 /" > /etc/apt/sources.list.d/nvidia-ml.list && \

This file has been truncated. show original

I’m wondering simply: How much of a lift would it be to do something similar here, either with a secondary image tag or the ability to trigger the drivers based on an environmental variable. (I’m not sure the latter is even a thing, but spitballing.)

Else, how would you recommend or what is the simplest way I go about adding them without having to fork and maintain the PS image myself? Currently to keep CPU load down I’ve disabled transcoding, and I could simply use CPU cycles to transcode, but I’d really prefer to leverage the P2000 if possible as it’s already serving me very well overall.

Thanks!

mrm · June 7, 2022, 4:20am

Dude, nice research!

So, I absolutely could publish another docker image that has cuda support.

I went through the (pretty big!) hassle of making PhotoStructure work nicely with Alpine so the docker image could be ~200MB instead of ~1.5GB.

The frigate-base image is Ubuntu. I just scanned all codepaths that are isDocker()-specific, and other than a couple minor bits (that will be fixed in alpha.2), things should work just fine.

FWIW, PhotoStructure’s Dockerfile is source-available: here’s the alpha version.

To make this happen, I’d need to convert base-tools and that Dockerfile to be Ubuntu-flavored. That shouldn’t be too painful.

I suspect the image, along with the CUDA driver, may push 1.5-2GB. That certainly isn’t a deal killer, but it’s certainly something to consider.

And holy crap do I suddenly want one of these, but yikes frigate-base install is scary.

adamf · June 7, 2022, 3:23pm

That would be great if it’s possible. (I figured you were super busy with everything else we were discussing around Alpha 2, and that this was the ideal time to post this )

I found one UK company with a US office that allegedly is shipping me a Coral accelerator this month - been waiting for 2 months already. On ebay they’re currently going (due to Frigate and Covid combined) for hundreds of dollars above MSRP.

What makes frigate base scary? I don’t know enough about docker files.

mrm · July 20, 2022, 3:28pm

Just found the official cuda image–not bad at all:

I’d also not realized before that I needed to add --gpus all to the docker command, or install a cuda plugin for docker (!?)

adamf · July 21, 2022, 6:05am

Is that from a development perspective you need to install the plugin?

To run things, I installed the container runtime (Installation Guide — NVIDIA Cloud Native Technologies documentation) after installing the drivers themselves to my OS. (But actually you can run those too, via another container, if you’re so inclined - Overview — NVIDIA Cloud Native Technologies documentation)