overview.md

Architecture overview

Pelorus is a zero-copy GPU pre-encode pipeline. Frames stay in VRAM from decode, through one or more Vulkan compute filters, to a hardware encoder — the moment a frame drops to system RAM, PCIe bandwidth destroys throughput, so the whole design is built to avoid it. Vulkan encoders take Pelorus frames as they are; on Linux, VAAPI and QSV encoders take them through hwmap without hwdownload when the last filter runs with tiling=drm (Vulkan output pools).

It is codec-agnostic: deband, denoise, and motion-hint stages pre-process pixels and help any hardware encoder — HEVC (hevc_nvenc/hevc_qsv/ hevc_vaapi/hevc_amf, rivaling x265) and AV1 (av1_nvenc, …) alike. Only film-grain synthesis is codec-specific, and Pelorus covers both AV1 (AOM) and HEVC/VVC (H.274 / SEI FGC).

Context (C4 level 1)

            ┌────────────────────────────────────────────────────────┐
   source → │  FFmpeg filtergraph (frames in AV_PIX_FMT_VULKAN, VRAM)  │ → HW encoder → mux
            │   vf_pelorus_analyze → _deband → _denoise → _grain → _mc  │   (NVENC/AMF/QSV)
            └───────────────┬───────────────────────────┬─────────────┘
                            │ writes PelorusSideData      │ reads side data
                            ▼ (UUID-keyed AVFrame blob)    ▼
                    ┌───────────────┐            ┌──────────────────┐
                    │   libpelorus  │◄───────────│  vmafx (oracle)  │
                    │  interop ABI  │  vendored  │  vf_libvmaf*,     │
                    │  + contracts  │  by both   │  server, MCP,    │
                    └───────────────┘            │  vmaf-tune       │
                                                 └──────────────────┘

Components

• libpelorus — the shared core: the side-data interop ABI (interop.h), filter parameter contracts (deband.h), and FFmpeg-free encoder-stat adapters (encoder telemetry). Both Pelorus and vmafx link/vendor it.
• vf_pelorus_* filters (ffmpeg-patches/) — Vulkan compute stages. Each is frame-in/frame-out, runs a compute shader, and (optionally) annotates the frame with a Pelorus side-data section.
• vmafx — the quality oracle (separate repo). Reads Pelorus side data for perceptually-weighted scoring; drives VMAF-in-the-loop autotune.

The two seams (bidirectional interop)

1. Data plane (ADR-0103): a versioned, UUID-keyed AVFrame side-data blob both vf_pelorus_* and vmafx's vf_libvmaf* read/write in one filtergraph, zero-copy. Pelorus is the sole writer; the ABI is append-only.
2. Control plane (ADR-0106): Pelorus exposes filter strengths as AVOptions and the banding/variance maps as perceptual-weighting input; vmafx's libvmaf_tune / vmafx-server / MCP run the search and hand back recommended strengths + CRF.

Pipeline stages (roadmap)

The stance follows ADR-0193: every stage remains experimental until a measured result on the pinned public corpus links its evidence.

Stage	Filter	Effect	Stance
borderfix	pelorus_borderfix_vulkan	dirty-line / border repair — clamp the dirty edge band onto the clean interior rect (the zero-copy equivalent of fillborders=smear); all planes, band widths in each plane's own pixels; runs first, before any other stage	experimental
analyze	pelorus_analyze_vulkan	measure banding/variance/edge stats → side data	experimental
deband	pelorus_deband_vulkan	flatten contours + dither	experimental
deblock	pelorus_deblock_vulkan	re-encode deblock/dering — gated [1 2 1] low-pass across the prior codec's DCT block grid; smooths blocking so the new encoder does not code it as false residual (luma by default; optional selected chroma); runs early, before deband	experimental
dehalo	pelorus_dehalo_vulkan	anime/2D dehalo + dering — single-pass GPU port of DeHalo_alpha + FineDehalo; removes the ring next to line-art (luma by default; optional selected chroma); foundation of tune=anime	experimental
denoise	pelorus_denoise_vulkan	edge-preserving spatio-temporal denoise (biggest BD-rate lever)	experimental
grain	pelorus_grain_estimate_vulkan; pelorus_fgs (BSF)	estimate film-grain params → PEL_SEC_FILMGRAIN + native AV1 side data; HEVC uses separately configured static pelorus_fgs BSF AVOptions (no inline frame-side-data bridge, ADR-0117); av1_nvenc via -pelorus_film_grain (ADR-0118); per-frame HEVC/H.264/VVC legs deferred	experimental
mc	pelorus_mc_vulkan	block-matching motion estimator → per-block motion + confidence side data; denoise mc=1 consumes it for confidence-gated temporal warping; NVENC ME-hint consumer (-pelorus_me_hints, patch 0008) uses it for encode speed, with no gain on the measured 4090 case (~2–3% slowdown at p7 — see bench-results.md v0.9)	experimental
aa	pelorus_aa_vulkan	anime warp anti-aliasing (awarpsharp2) + optional line-darkening (FastLineDarken); luma by default with optional selected chroma, single-pass transform, no side data; second stage of the anime tune chain after dehalo (ADR-0124); defaults unproven on content (SSIMULACRA2 / edge-region VMAF-NEG proof deferred)	experimental
scenecut	pelorus_scenecut	scene-cut → forced IDR — metadata-only consumer (NOT a Vulkan filter, no GPU work) that reads PEL_SEC_MOTION.has_scene_cut (from mc) and sets pict_type=I so the encoder opens a fresh GOP at the cut; vendor-neutral encoder steering, no per-encoder patch (ADR-0126); the ffmpeg CLI needs -force_key_frames source (scenecut); runs on Vulkan frames, no hwdownload needed (ADR-0186); BD-rate A/B is a documented follow-up	experimental
telemetry	libpelorus x265 CSV and SVT-AV1 stat-file adapters	parse caller-supplied encoder-stat text into validated per-frame telemetry records; CPU-only and independent of FFmpeg	experimental

The anime tune composes these stages into one recommended GPU pre-encode chain — analyze-ROI + dehalo + aa + deband at 10-bit, then encoder ROI steering — each leg targeting a distinct anime artefact (starved flats, ringing, jaggies, banding). See docs/usage/anime.md (ADR-0125).

See docs/backends/vulkan.md for the compute-filter authoring model and docs/usage/ffmpeg.md for the end-to-end command form.