NVIDIA VA-API Hardware Video Decode in Firefox — Complete Debug Guide

System: Linux From Scratch, GTX 1660 Super (Turing TU116), NVIDIA 590.48.01, Firefox 148.0.2, kernel 7.0.0-rc3 Date: 2026-03-20 Result: VP9 hardware decode working — GPU NVDEC active, CPU decode dropped from 86% to 4.7%

The Problem

1080p/1440p YouTube VP9 videos stuttered badly. CPU pinned at 100% on both cores (Pentium E6500K @ 3.9 GHz). nvidia-smi pmon showed dec=0 — the GPU's NVDEC hardware decoder was completely idle. Firefox was software-decoding everything.

The Stack

Hardware video decode on NVIDIA with Firefox requires this chain to work end-to-end:

YouTube (VP9) → Firefox RDD process → FFmpeg (VAAPI hwaccel)
  → libva → nvidia-vaapi-driver → NVDEC hardware
  → DMA-BUF export → EGL import → WebRender GPU compositing

Every link must work. Our debugging found three separate breaks in this chain.

Issue 1: nvidia-vaapi-driver built without VP9 support

Symptom

vainfo showed H264, HEVC, VP8 profiles but no VP9:

VAProfileH264Main               : VAEntrypointVLD
VAProfileHEVCMain               : VAEntrypointVLD
VAProfileVP8Version0_3          : VAEntrypointVLD
# VP9 missing!

Firefox's vaapitest returned VAAPI_HWCODECS=4176 (bitmask without VP9 bit 8 = 256).

Root Cause

nvidia-vaapi-driver's VP9 support depends on gstreamer-codecparsers-1.0 (from GStreamer gst-plugins-bad). In meson.build:

gst_codecs_deps = dependency('gstreamer-codecparsers-1.0', required: false)

if gst_codecs_deps.found()
    sources += ['src/vp9.c']    # VP9 only compiled if GStreamer found!
    deps += [gst_codecs_deps]
endif

The dependency is required: false — when missing, VP9 is silently excluded. No warning, no error. The driver builds and works fine, just without VP9.

How to Check

pkg-config --exists gstreamer-codecparsers-1.0 && echo "Found" || echo "NOT FOUND"

Fix

Build the minimum GStreamer stack (all with --prefix=/usr --buildtype=release):

GStreamer core 1.28.1 — -Dgst_debug=false -Dtools=disabled -Dtests=disabled -Ddoc=disabled -Dintrospection=disabled -Dexamples=disabled -Dbenchmarks=disabled
gst-plugins-base 1.28.1 — -Dauto_features=disabled -Dtests=disabled -Ddoc=disabled -Dintrospection=disabled -Dexamples=disabled
gst-plugins-bad 1.28.1 — -Dauto_features=disabled -Dgpl=enabled -Dtests=disabled -Ddoc=disabled -Dintrospection=disabled -Dexamples=disabled

The -Dauto_features=disabled flag is key — it disables all optional plugins but still builds the core libraries including codecparsers.

Then rebuild nvidia-vaapi-driver. Verify:

LIBVA_DRIVER_NAME=nvidia NVD_BACKEND=direct vainfo --display drm --device /dev/dri/renderD128
# Should now show:
#   VAProfileVP9Profile0  : VAEntrypointVLD
#   VAProfileVP9Profile2  : VAEntrypointVLD

How to Verify in Firefox

LIBVA_DRIVER_NAME=nvidia NVD_BACKEND=direct /usr/lib/firefox/vaapitest -d /dev/dri/renderD128
# VAAPI_HWCODECS should be 4432 (4176 + 256 for VP9 decode)

Bitmask reference:

Bit	Value	Codec
4	16	H264 decode
6	64	VP8 decode
8	256	VP9 decode
10	1024	AV1 decode
12	4096	HEVC decode

Issue 2: Firefox gfxVars not detecting VP9 (red herring)

Symptom

Firefox RDD process logged: "Hw codec disabled by gfxVars for AV_CODEC_ID_VP9"

Investigation

We initially thought Firefox's glxtest/vaapitest subprocess wasn't inheriting LIBVA_DRIVER_NAME=nvidia. We patched Firefox source:

Removed NVIDIA blocklist in widget/gtk/GfxInfo.cpp (lines 1141-1147)
Changed UseVP9HwDecode default to true in gfx/config/gfxVars.h

Actual Cause

These patches were unnecessary. The gfxVars correctly reported no VP9 because vainfo genuinely had no VP9 profiles (Issue 1). Firefox's g_spawn_async_with_pipes() passes nullptr for environ, which means the subprocess does inherit the parent's environment. The detection code was correct all along.

Lesson

Always verify the lowest layer first. We should have checked vainfo output before patching Firefox. The VAAPI_HWCODECS=4176 value was the clue — 4176 doesn't include bit 8 (VP9 = 256).

Issue 3: EGL DMA-BUF import failure (nvidia-vaapi-driver 0.0.13)

Symptom

After fixing Issue 1, Firefox logged:

FFVPX: Choosing FFmpeg pixel format for VA-API video decoding.
FFVPX: Requesting pixel format VAAPI_VLD
...
DMABufSurfaceYUV::CreateTexture() UID 9218 plane 0
  EGLImageKHR creation failed, EGL error EGL_BAD_PARAMETER
  failed to create EGL image!
  failed to create texture over DMABuf memory!
CreateImageVAAPI(): failed to get VideoFrameSurface

VA-API decode succeeded, DMA-BUF export succeeded, but EGL couldn't import the surface as a texture.

Root Cause

Firefox imports NV12 surfaces as separate per-plane EGL images:

Plane 0: DRM_FORMAT_R8 (1280×720, luma Y)
Plane 1: DRM_FORMAT_GR88 (640×360, chroma UV interleaved)

This is correct for Mesa (AMD/Intel) but nvidia-vaapi-driver 0.0.13's direct backend exported DMA-BUFs with modifiers that NVIDIA's EGL rejected for these single-plane formats. The egl backend was even worse — vaExportSurfaceHandle failed entirely.

Fix

Upgrade nvidia-vaapi-driver from 0.0.13 to 0.0.16:

curl -kL -o nvidia-vaapi-driver-0.0.16.tar.gz \
  https://github.com/elFarto/nvidia-vaapi-driver/archive/refs/tags/v0.0.16.tar.gz
tar xf nvidia-vaapi-driver-0.0.16.tar.gz
cd nvidia-vaapi-driver-0.0.16
meson setup build --prefix=/usr --buildtype=release
ninja -C build -j$(nproc)
ninja -C build install

0.0.16 fixed the DMA-BUF modifier export for per-plane EGL import. After upgrade:

DMABufSurfaceYUV::CreateTexture() UID 11070 plane 0   # success!
DMABufSurfaceYUV::CreateTexture() UID 11070 plane 1   # success!
VA-API frame pts=0 dts=0 duration=17000 color space BT709

Complete Working Configuration

System Requirements

NVIDIA driver with NVDEC support (Kepler+)
nvidia-drm.modeset=1 kernel parameter
nvidia-uvm.ko loaded (for compute support)

Packages

Package	Version	Notes
NVIDIA driver	590.48.01	Patched for kernel 7.0-rc3 (CachyOS patches)
nvidia-vaapi-driver	0.0.16	Must be ≥0.0.16 for Firefox DMA-BUF compat
libva	2.23.0	VA-API runtime
gstreamer	1.28.1	Core library
gst-plugins-base	1.28.1	Base library
gst-plugins-bad	1.28.1	Provides codecparsers (required for VP9!)
FFmpeg	7.1.3	Built with `--enable-vaapi --enable-hwaccel=vp9_vaapi`
Firefox	148.0.2	Built from source

Environment Variables (`/etc/profile.d/nvidia-vaapi.sh`)

export LIBVA_DRIVER_NAME=nvidia
export NVD_BACKEND=direct
export MOZ_DISABLE_RDD_SANDBOX=1

Firefox Preferences (`user.js`)

user_pref("media.ffmpeg.vaapi.enabled", true);
user_pref("media.hardware-video-decoding.force-enabled", true);
user_pref("media.rdd-ffvpx.enabled", true);
user_pref("media.rdd-ffmpeg.enabled", true);
user_pref("widget.dmabuf.force-enabled", true);
user_pref("gfx.x11-egl.force-enabled", true);

Firefox Wrapper Script (`/usr/local/bin/firefox`)

#!/bin/sh
export XDG_RUNTIME_DIR=/tmp/pipewire-root
export PULSE_SERVER=unix:/tmp/pipewire-root/pulse/native
export LIBVA_DRIVER_NAME=nvidia
export NVD_BACKEND=direct
export MOZ_DISABLE_RDD_SANDBOX=1
export MOZ_X11_EGL=1
exec /usr/lib/firefox/firefox -P default-esr "$@"

Debugging Cheat Sheet

Layer 1: Does the GPU support NVDEC?

nvidia-smi -q | grep -A5 Decoder
# Turing (GTX 1660+): VP9, HEVC, H264
# Pascal (GTX 1060+): VP9, HEVC, H264 (no AV1)

Layer 2: Does vainfo show VP9?

LIBVA_DRIVER_NAME=nvidia NVD_BACKEND=direct vainfo --display drm --device /dev/dri/renderD128
# Look for: VAProfileVP9Profile0 : VAEntrypointVLD
# If missing: rebuild nvidia-vaapi-driver with gstreamer-codecparsers-1.0

Layer 3: Does Firefox's vaapitest detect VP9?

LIBVA_DRIVER_NAME=nvidia NVD_BACKEND=direct /usr/lib/firefox/vaapitest -d /dev/dri/renderD128
# VAAPI_HWCODECS should include 256 (VP9 bit)
# 4432 = H264(16) + VP8(64) + VP9(256) + HEVC(4096)

Layer 4: Does Firefox's RDD process support VP9 hw decode?

MOZ_LOG='PlatformDecoderModule:5' firefox 2>&1 | grep VP9
# Good: "Support AV_CODEC_ID_VP9 for hw decoding"
# Bad:  "Hw codec disabled by gfxVars for AV_CODEC_ID_VP9"

Layer 5: Does VA-API decode + DMA-BUF export work?

MOZ_LOG='FFmpegVideo:5,Dmabuf:5' MOZ_LOG_FILE=/tmp/ff-vaapi.log firefox
# Then play a VP9 video and check:
grep -i 'VA-API.*init\|CreateTexture\|EGLImage\|failed' /tmp/ff-vaapi.log.child-*.moz_log
# Good: "VA-API FFmpeg init successful" + "VA-API frame pts=..."
# Bad:  "EGLImageKHR creation failed" → upgrade nvidia-vaapi-driver
# Bad:  "vaExportSurfaceHandle failed" → don't use NVD_BACKEND=egl

Layer 6: Is the GPU actually decoding?

nvidia-smi pmon -d 2
# Look for "dec" column > 0 on the RDD process
# Type should show "C+G" (compute + graphics)

Quick Health Check (all in one)

# 1. vainfo has VP9
LIBVA_DRIVER_NAME=nvidia NVD_BACKEND=direct vainfo 2>&1 | grep VP9

# 2. vaapitest has VP9
LIBVA_DRIVER_NAME=nvidia NVD_BACKEND=direct /usr/lib/firefox/vaapitest -d /dev/dri/renderD128

# 3. GPU decoder active during playback
nvidia-smi pmon -c 5 -d 2

Common Pitfalls

nvidia-vaapi-driver VP9 silently disabled — No warning when gstreamer-codecparsers-1.0 is missing. Always verify with vainfo after building.
vainfo vs Firefox disagree — vainfo uses your shell env. Firefox's vaapitest inherits from the parent process. If Firefox is launched from a display manager that doesn't source /etc/profile.d/, the env vars won't be set. Use a wrapper script.
EGL_BAD_PARAMETER on NV12 planes — nvidia-vaapi-driver < 0.0.16 can't export DMA-BUFs that NVIDIA's EGL accepts for per-plane (R8/GR88) import. Upgrade to ≥ 0.0.16.
NVD_BACKEND=egl doesn't export DMA-BUF — The EGL backend fails vaExportSurfaceHandle. Use NVD_BACKEND=direct (requires nvidia-drm.modeset=1).
Firefox NVIDIA blocklist — Firefox has a hardcoded NVIDIA blocklist in widget/gtk/GfxInfo.cpp. Recent Firefox versions (148+) may not need patching if the vaapitest probe returns correct results. Check before patching.
FFmpeg must have VA-API — Verify: strings /usr/lib/libavcodec.so* | grep vaapi. Firefox's internal FFVPX does NOT have VA-API — it relies on the system FFmpeg library for hardware acceleration.
pkg-config path on LFS — GStreamer installs to /usr/lib64/pkgconfig/ on some systems. Ensure PKG_CONFIG_PATH includes this directory when building nvidia-vaapi-driver.

Timeline of This Debug Session

Time	Action	Result
Start	`vainfo` shows no VP9 profiles	nvidia-vaapi-driver missing VP9
+5m	Found `gstreamer-codecparsers-1.0` is optional dep	Root cause identified
+15m	Built GStreamer 1.28.1 (core + base + bad)	codecparsers available
+20m	Rebuilt nvidia-vaapi-driver	VP9 profiles appear in vainfo
+25m	Firefox vaapitest returns HWCODECS=4432	VP9 detected by Firefox
+30m	Play VP9 video — still software decode	New issue: EGL import fails
+35m	`MOZ_LOG` shows `EGL_BAD_PARAMETER` on R8 plane	DMA-BUF modifier incompatible
+40m	Tried NVD_BACKEND=egl — different failure	vaExportSurfaceHandle failed
+45m	Upgraded nvidia-vaapi-driver 0.0.13 → 0.0.16	EGL import succeeds
+50m	`nvidia-smi pmon` shows `dec: 3`	VP9 hardware decode working!

FilesExpand file tree

nvidia-vaapi-firefox-guide.md

Latest commit

History

nvidia-vaapi-firefox-guide.md

File metadata and controls

NVIDIA VA-API Hardware Video Decode in Firefox — Complete Debug Guide

The Problem

The Stack

Issue 1: nvidia-vaapi-driver built without VP9 support

Symptom

Root Cause

How to Check

Fix

How to Verify in Firefox

Issue 2: Firefox gfxVars not detecting VP9 (red herring)

Symptom

Investigation

Actual Cause

Lesson

Issue 3: EGL DMA-BUF import failure (nvidia-vaapi-driver 0.0.13)

Symptom

Root Cause

Fix

Complete Working Configuration

System Requirements

Packages

Environment Variables (/etc/profile.d/nvidia-vaapi.sh)

Firefox Preferences (user.js)

Firefox Wrapper Script (/usr/local/bin/firefox)

Debugging Cheat Sheet

Layer 1: Does the GPU support NVDEC?

Layer 2: Does vainfo show VP9?

Layer 3: Does Firefox's vaapitest detect VP9?

Layer 4: Does Firefox's RDD process support VP9 hw decode?

Layer 5: Does VA-API decode + DMA-BUF export work?

Layer 6: Is the GPU actually decoding?

Quick Health Check (all in one)

Common Pitfalls

Timeline of This Debug Session

Environment Variables (`/etc/profile.d/nvidia-vaapi.sh`)

Firefox Preferences (`user.js`)

Firefox Wrapper Script (`/usr/local/bin/firefox`)