Skip to content
pipeline

feat: integrate dual-lane core canary #437

Sign in to view logs

feat: integrate dual-lane core canary

feat: integrate dual-lane core canary #437

Workflow file for this run

.github/workflows/pipeline.yml at 26803ec
# Pulsar pipeline -- one build, many jobs.
#
# pulsar.exe is built ONCE in the `build` job and uploaded as the
# `pulsar-rundir` artefact. Every downstream job downloads that
# artefact instead of rebuilding. Jobs are isolated so a failure in
# `offline-probes` doesn't take down `live-broadcast`, `package`,
# `npm-publish` etc -- you see exactly which gate broke and which
# ones still passed in the GitHub Actions UI.
#
# Job graph :
#
# lint ──► build ──┬─► binary-gate
# ├─► offline-probes
# ├─► live-broadcast (tag / dispatch ONLY, #132)
# │ ├─► publish-gh-pages
# │ └─► release-attach
# └─► package ──────────► release-attach
# ────► npm-publish
#
# Trigger / behaviour matrix (each job carries its own `if:`) :
# push to feature branch : lint, build, gates (NO broadcast)
# push to main (post-PR merge) : same (NO broadcast, NO gh-pages)
# push tag v*.*.* : + broadcast + gh-pages + package
# + release attach + npm publish
# pull_request to main : lint, build, gates (NO broadcast)
# workflow_dispatch : full pipeline + operator-chosen
# broadcast duration / fps + opt-in
# release-grade stages via inputs
#
# The Twitch broadcast left the per-commit path in #132 (CI wall-clock,
# porteur decision) -- rationale and the coverage it costs are written
# out above the `live-broadcast` job itself.
#
# docs-only diffs : the workflow ALWAYS triggers (no more paths-ignore
# at the trigger level, #205) so branch protection always gets a
# status for the 4 required contexts (lint / build / binary-gate /
# offline-probes). The cheap `changes` job (ubuntu, git diff against
# base) detects whether the diff touches anything outside docs / .md /
# CHANGELOG.md / .gitignore / LICENSE and exposes `outputs.code`.
# `lint` and `contract-tests` gate on it directly; `build` (and
# everything that needs: build) inherits the skip for free through
# GitHub's default `needs` success() check -- a skipped `lint` is not
# a success, so `build` (needs: lint) is skipped too, and so on down
# the graph. A skipped job still reports a conclusion (`skipped`) on
# its required-check context, which GitHub counts as passing -- so a
# docs-only PR gets all 4 required checks green without the MSVC
# build ever running. A code diff (`code == 'true'`) runs the full
# graph exactly as before, no regression.
#
# Concurrency : `pipeline-<ref>` group with cancel-in-progress=true
# means a fresh push to the same ref cancels the previous run, so you
# never have two builds queueing for the same change. Live broadcast
# additionally takes its own non-cancelling lock named `live-test`
# so Twitch never sees two streams at once.
name: pipeline
on:
workflow_dispatch:
inputs:
duration_seconds:
description: 'Live broadcast duration (seconds)'
type: string
default: '300'
fps:
description: 'Encoder fps target'
type: string
default: '60'
enable_package:
description: 'Run the package job (light + full zips)'
type: boolean
default: false
enable_release_attach:
description: 'Attach distros + proof to a GitHub Release (requires a tag context)'
type: boolean
default: false
enable_npm_publish:
description: 'Publish @clodocapeo/pulsar-* to npm (requires NPM_TOKEN + tag-matched VERSION)'
type: boolean
default: false
# Dedup strategy : `pull_request` fires for every PR-touching push,
# so we DON'T also fire `push` on feature branches -- that would
# double every PR run. push is reserved for main + tags (the events
# without a backing PR). Net : exactly one pipeline run per change.
# - feature branch + open PR : pull_request fires (once per push)
# - feature branch, no PR : nothing fires (open a draft PR if
# you want CI before review)
# - direct push to main : push fires
# - tag push v*.*.* : push fires
push:
branches: [main]
tags:
- 'v*.*.*'
pull_request:
branches: [main]
concurrency:
group: pipeline-${{ github.ref }}
cancel-in-progress: true
# Shared expressions reused across job-level `if:` guards.
env:
IS_TAG: ${{ startsWith(github.ref, 'refs/tags/v') }}
IS_MAIN: ${{ github.ref == 'refs/heads/main' }}
jobs:
# ── Stage 0 : docs-only detection (#205) ──────────────────────────
# Always runs (no paths-ignore on the trigger anymore) so it can
# itself report a status, and so the jobs below have something to
# gate on. Compares the PR head against its base (or the push
# before/after SHAs) and flags `code=true` the moment a single
# changed file falls outside docs/**, *.md, CHANGELOG.md,
# .gitignore, LICENSE. Missing/unusable base (new branch, force
# push with no prior SHA) fails safe to `code=true` -- never
# silently skip the real gates.
changes:
name: detect changed paths
runs-on: ubuntu-latest
timeout-minutes: 3
outputs:
code: ${{ steps.filter.outputs.code }}
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Detect non-docs changes
id: filter
shell: bash
run: |
set -euo pipefail
if [ "${{ github.event_name }}" = "pull_request" ]; then
base="${{ github.event.pull_request.base.sha }}"
head="${{ github.event.pull_request.head.sha }}"
else
base="${{ github.event.before }}"
head="${{ github.sha }}"
fi
if [ -z "${base}" ] || [ "${base}" = "0000000000000000000000000000000000000000" ] || ! git cat-file -e "${base}^{commit}" 2>/dev/null; then
echo "::notice::no usable base revision -- treating as a code change"
echo "code=true" >> "$GITHUB_OUTPUT"
exit 0
fi
changed="$(git diff --name-only "${base}" "${head}")"
if echo "${changed}" | grep -vE '(^|/)[^/]+\.md$|^docs/|^CHANGELOG\.md$|^\.gitignore$|^LICENSE$' | grep -q .; then
echo "::notice::non-docs changes detected"
echo "code=true" >> "$GITHUB_OUTPUT"
else
echo "::notice::docs-only diff"
echo "code=false" >> "$GITHUB_OUTPUT"
fi
# ── Stage 1 : lint ────────────────────────────────────────────────
# Cheap fail-fast checks that don't need the build. Source-grep for
# forbidden patterns, patch lint against pinned upstream SHA, plugin
# metadata, npm tarball audit. Runs on ubuntu (faster, cheaper).
#
# Skipped (not run) on a docs-only diff (#205) -- `build`,
# `binary-gate` and `offline-probes` all chain off `needs: lint`
# (directly or transitively) with no custom `if:` of their own, so
# GitHub's default `success()` requirement skips them automatically
# the moment lint is skipped. A skipped required check still counts
# as passing.
lint:
name: lint
needs: changes
if: needs.changes.outputs.code == 'true'
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Checkout (with submodules)
uses: actions/checkout@v4
with:
submodules: recursive
fetch-depth: 0
- name: Configure git identity
run: |
git config --global user.email "ci@pulsar.zablaboratory"
git config --global user.name "Pulsar CI"
- name: License isolation source-tree audit
run: bash scripts/check-license-isolation.sh
# #222 (finding H1, Bastion clearance #221) -- #220 showed assert()
# silently no-ops under NDEBUG in the RelWithDebInfo CI build. Guards
# against a new test file reintroducing assert()/<cassert>/<assert.h>
# instead of PULSAR_CHECK.
- name: assert() guard under tests/**
run: bash scripts/check-no-assert-in-tests.sh
# Hardware-free half of the QSV preset proof: the boot setter, the
# GetVideoSettings reader and obs-qsv11's OWN source must agree on the
# property name / values / default of the QSV preset knob. Runs here,
# before the patch-lint step touches the submodule tree, because no
# patch modifies obs-qsv11 sources. The runtime half needs an Intel QSV
# device (scripts/probe-qsv-preset.py, partial on this fleet).
- name: QSV preset property contract (source consistency)
run: python3 scripts/check-qsv-preset-contract.py
# Same proof for NVENC, where the knob name splits INSIDE the family:
# "preset" for the 31.0+ encoder, "preset2" for the jim_nvenc /
# ffmpeg_nvenc compat shims that resolveEncoderId can also select. Also
# hardware-free -- no NVIDIA GPU is required or used.
- name: NVENC preset property contract (source consistency)
run: python3 scripts/check-nvenc-preset-contract.py
# #158 / ADR Prism 028 §3.2 -- resolution criterion 1, in executable
# form: NO browser-source creation path may inherit obs-browser's
# default webpage_control_level. Build-free on purpose, so an unpinned
# path fails here in seconds instead of surviving to a -Full build the
# light variant would not even exercise. The RUNTIME half (what the page
# actually sees, plus the source lifecycle) is
# scripts/probe-webpage-control-level.py in the offline suite.
- name: Browser-source control-level pinning (source consistency)
run: python3 scripts/check-webpage-control-level.py
# #167 / Prism ADR 023 Amendment 3 -- criteria 1 and 5, in the two
# directions they pull: nv-filters IS in the bundle (and the header
# comment no longer justifies the strip it used to), and no NVIDIA SDK
# DLL or .trtpkg model IS. Build-free, so a packaging regression fails
# here in seconds instead of on a tag. The `package` job re-runs it
# with --dist against the real zip contents.
- name: nv-filters packaging (bundled module, no SDK payload)
run: python3 scripts/check-nv-filters-packaging.py
- name: Patch lint (apply against pinned upstream SHA)
run: |
set -e
shopt -s nullglob
patches=(patches/*.patch)
if [ ${#patches[@]} -eq 0 ]; then
echo "::notice::no patches to lint"; exit 0
fi
root_patches=()
obs_browser_patches=()
for p in "${patches[@]}"; do
if [[ "$p" == *obs-browser* ]]; then
obs_browser_patches+=("$p")
else
root_patches+=("$p")
fi
done
pinned=$(git submodule status --cached upstream | awk '{print $1}' | sed 's/^[+-]*//')
cd upstream
git reset --hard "$pinned"
git clean -fdx
for p in "${root_patches[@]}"; do
full="../$p"
echo "::group::git am $p"
if ! git am --3way "$full"; then
git am --abort || true
echo "::error::Patch $p does not apply on upstream pinned SHA."
exit 1
fi
echo "::endgroup::"
done
if [ ${#obs_browser_patches[@]} -gt 0 ]; then
obs_browser_pinned=$(git submodule status --cached plugins/obs-browser | awk '{print $1}' | sed 's/^[+-]*//')
pushd plugins/obs-browser
git reset --hard "$obs_browser_pinned"
for p in "${obs_browser_patches[@]}"; do
full="../../../$p"
echo "::group::git am $p in obs-browser"
if ! git am --3way "$full"; then
git am --abort || true
echo "::error::Nested patch $p does not apply on obs-browser pinned SHA."
exit 1
fi
echo "::endgroup::"
done
popd
fi
- name: Plugin metadata (CMakeLists + README per plugin)
run: |
set -e
fail=0
for d in plugins/*/; do
name=$(basename "$d")
for f in CMakeLists.txt README.md; do
if [ ! -f "$d/$f" ]; then
echo "::error::missing $f in plugins/$name/"
fail=1
fi
done
done
[ $fail -eq 0 ]
- name: npm tarball audit
run: bash scripts/check-npm-pack-audit.sh
# ── Stage 1b : cross-service contract tests ──────────────────────
# Pure-logic contract tests (no OBS binary, no network) — they prove
# the inter-service `scene_control` schema (M10, ADR 003 Amd 2 §A2.1)
# holds Blue→leaf→Prism/probe. Runs on ubuntu, parallel to `lint`, so
# a contract break fails fast WITHOUT the 30-min Windows build. Owned
# by Conduit (#59).
contract-tests:
name: contract tests (scene_control)
needs: changes
if: needs.changes.outputs.code == 'true'
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Python 3.11
uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Install test deps
# websockets: the M10 probe module imports it at top level, so the
# offline probe tests (test_m10_setup.py, test_probe_m10_real_orion.py)
# can only be collected with it present. No OBS binary / no VPS reach
# is needed for these — they assert pure logic (leaf path, scene_control
# default, the --real-orion URL the CEF loads).
run: python -m pip install --upgrade pip pytest websockets
- name: Run scene_control contract test
# Blue is not checked out in Pulsar CI: the test binds its
# in-test leaf_mapper MIRROR (identical 3-segment rule, ADR
# §A2.2) and prints a declared blind-spot note. The mirror is
# bound to the REAL Blue leaf_mapper when both repos are present
# (developer machine; Blue's own CI cross-checks the fixtures).
run: pytest scripts/contracts/scene_control/ -v
- name: Run offline M10 harness + probe tests
# Provable without pulsar.exe and without the VPS: the F2 Orion
# declaration fixture (scene_control leaf path + valid default) and
# the --real-orion true-wire URL the CEF browser_source loads. The
# on-air legs are the CTest integration suite (needs the build) +
# Keeper's antenna run (needs the VPS).
run: pytest scripts/test_m10_setup.py scripts/test_probe_m10_real_orion.py -v
- name: Run offline Pulsar probe contract tests
# The AC-12 RTMP boundary and AC-13 dual-lane probe contracts are
# hardware-free; keep them in the required Python gate so changes to
# their parser, trace, and evidence invariants cannot bypass CI.
run: pytest tests/probe-take-latency tests/probe-dual-lane -v
# ── Stage 2 : build pulsar.exe (THE ONE BUILD) ────────────────────
# Every downstream Windows job consumes the `pulsar-rundir` artefact
# this job uploads. retention-days is short -- the artefact only
# has to outlive the in-flight pipeline run.
build:
name: build pulsar.exe (-Full)
needs: lint
runs-on: windows-2022
timeout-minutes: 30
steps:
- name: Checkout (with submodules)
uses: actions/checkout@v4
with:
submodules: recursive
fetch-depth: 0
- name: Configure git identity
shell: pwsh
run: |
git config --global user.email "ci@pulsar.zablaboratory"
git config --global user.name "Pulsar CI"
- name: Cache obs-deps + Qt6 + CEF
uses: actions/cache@v4
with:
path: upstream/.deps
key: obs-deps-full-${{ runner.os }}-${{ hashFiles('scripts/build-win.ps1', '.gitmodules') }}
restore-keys: |
obs-deps-full-${{ runner.os }}-
obs-deps-${{ runner.os }}-
- name: Install CMake 3.28+
uses: lukka/get-cmake@latest
with:
cmakeVersion: '3.30.0'
- name: Setup MSVC toolchain
uses: ilammy/msvc-dev-cmd@v1
with:
arch: x64
- name: Build pulsar.exe with -Full (CEF + obs-browser)
shell: pwsh
run: ./scripts/build-win.ps1 -Full
- name: Upload pulsar-rundir artefact
uses: actions/upload-artifact@v4
with:
name: pulsar-rundir
# The whole RelWithDebInfo rundir : pulsar.exe + all DLLs +
# CEF runtime + the CMake build dir for ctest. Compressed
# at level 1 (fast) since downstream jobs download once.
#
# build/tests/nv-probe/** is there for #167 : that gate is a
# self-contained executable plus its four fixture DLLs, none of
# which live in the rundir. Without it `ctest --test-dir build`
# finds the test registered and the binary missing, which reads
# as an infrastructure failure rather than as the security check
# it is.
path: |
upstream/build_x64/rundir/RelWithDebInfo
build/CTestTestfile.cmake
build/**/CTestTestfile.cmake
build/**/*.vcxproj
build/tests/nv-probe/**
retention-days: 1
compression-level: 1
# ── Stage 3a : binary-export gate ────────────────────────────────
binary-gate:
name: binary export gate (#3)
needs: build
runs-on: windows-2022
timeout-minutes: 10
steps:
- name: Checkout (scripts only)
uses: actions/checkout@v4
# No submodules : we only need scripts/check-binary-exports.ps1.
- name: Download pulsar-rundir
uses: actions/download-artifact@v4
with:
name: pulsar-rundir
path: .
- name: Setup MSVC toolchain (dumpbin on PATH)
uses: ilammy/msvc-dev-cmd@v1
with:
arch: x64
- name: Verify binary export tables
shell: pwsh
run: ./scripts/check-binary-exports.ps1
# ── Stage 3b : offline probe suite ───────────────────────────────
offline-probes:
name: offline probe suite (CTest)
needs: build
runs-on: windows-2022
timeout-minutes: 15
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Download pulsar-rundir
uses: actions/download-artifact@v4
with:
name: pulsar-rundir
path: .
- name: Setup Python 3.11
uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Install probe deps
shell: bash
run: |
python -m pip install --upgrade pip
pip install websockets
- name: Detect physical GPU for accelerated CEF proof
id: hardware-gpu
shell: pwsh
run: ./scripts/detect-hardware-gpu.ps1 -GithubOutput $env:GITHUB_OUTPUT
- name: Run offline probe suite
# Retry-once : obs upstream has two known race-condition crash
# paths in the source/output destruction lifecycle (WASAPI
# source destroy + obs_output_release vs worker thread). They
# surface ~7 % of the time on the windows-2022 CI runner even
# with the graceful-stop drain in
# pulsar-multi-stream::release_destination_handles_locked.
# One automatic retry is enough to cover the residual flake ;
# if both attempts fail, that's a real regression.
#
# Reading a failure : run-probes.ps1 now names the two cases
# apart, so DON'T guess from the client-side error.
# - "==> FATAL: the shared pulsar.exe DIED" -> a server crash.
# The ConnectionClosedError / [WinError 1225] lines under it
# are consequences ; the suite stops there and prints the
# libobs stderr tail. Retrying that is not a diagnosis.
# - no FATAL banner -> the server stayed up and a probe
# assertion genuinely failed. Read the probe, not the infra.
# TODO(upstream-obs) : investigate the worker-thread vs
# service-ref-release race + WASAPI source destroy path,
# submit fixes upstream.
env:
# A GitHub-hosted Windows VM exposes no physical PCI display
# adapter. Keep the product GPU-on and skip only the visual M3
# oracle there; real hardware runs execute it by default.
PULSAR_SKIP_ACCELERATED_CEF_PROBE: ${{ steps.hardware-gpu.outputs.available == 'true' && '0' || '1' }}
uses: nick-fields/retry@v3
with:
timeout_minutes: 5
max_attempts: 2
retry_wait_seconds: 10
shell: pwsh
command: ctest --test-dir build --output-on-failure --output-junit ctest-junit.xml -C RelWithDebInfo
- name: Upload CTest output (passing and failing tests)
# --output-on-failure only echoes a test's internal stdout/stderr to
# this job's console log when that test fails, so a green run leaves
# no trace of what it actually exercised -- structurally unverifiable
# after the fact (Vigil, PR #211 review + ADR-005 Amendment 2 review).
# CTest's own Testing/Temporary/LastTest.log always captures every
# test's full output regardless of outcome, and --output-junit above
# adds a structured per-test index ; archive both instead of
# switching the whole suite to -V, which would just drown the normal
# case in noise. if: always() so a genuine regression's evidence is
# still attached alongside the retry's own console output.
if: always()
uses: actions/upload-artifact@v4
with:
name: pulsar-ctest-output
path: |
build/ctest-junit.xml
build/Testing/Temporary/LastTest.log
if-no-files-found: warn
retention-days: 14
# ── Stage 3d : capture <-> PGM compatibility (opt-in, real CEF) ──
#
# #234 -- #231 left this suite (packages/capture-pgm-compat, opt-in via
# PULSAR_LIVE_CAPTURE_COMPAT=1) unexercised by any CI job: it had only
# ever run on the porteur's own machine. It's the only proof in this
# repo that a FROZEN source -- spatially indistinguishable from a
# healthy one (measured: spatialStddevAvg 50.986 vs 48.200) -- gets
# caught at all, and only on the temporal axis (0.002 vs 4.271). A
# silent drift on that axis would otherwise reach main undetected.
#
# Reuses build's pulsar-rundir artefact exactly like offline-probes: a
# real pulsar.exe with CEF (obs-browser), so no separate ~150MB binary
# download via pulsar-bundle-full's own postinstall
# (PULSAR_BUNDLE_SKIP_POSTINSTALL=1) -- PULSAR_BUNDLE_FULL_BINARIES_PATH
# points spawn() straight at the artefact's rundir instead.
#
# The hosted windows-2022 runner has no physical GPU. The package tests
# still run there, but the opt-in accelerated CEF leg runs only when a
# physical Intel/AMD/NVIDIA PCI adapter is present. A declared skip is not
# release evidence: the real Prism/Twitch hardware E2E remains mandatory.
capture-pgm-compat:
name: capture <-> PGM compatibility (real CEF)
needs: build
runs-on: windows-2022
timeout-minutes: 15
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Download pulsar-rundir
uses: actions/download-artifact@v4
with:
name: pulsar-rundir
path: .
- name: Setup Node.js 20
uses: actions/setup-node@v4
with:
node-version: '20'
- name: Detect physical GPU for accelerated CEF proof
id: hardware-gpu
shell: pwsh
run: ./scripts/detect-hardware-gpu.ps1 -GithubOutput $env:GITHUB_OUTPUT
- name: Setup ffmpeg
uses: FedericoCarboni/setup-ffmpeg@v3
with:
ffmpeg-version: release
- name: Install npm workspaces
# The real pulsar.exe + CEF runtime is already on disk from the
# pulsar-rundir artefact downloaded above -- pulsar-bundle-full's
# own binaries/ download would just be redundant weight, not a
# different pulsar.exe.
env:
PULSAR_BUNDLE_SKIP_POSTINSTALL: '1'
shell: pwsh
run: npm ci --no-audit --no-fund
- name: Build workspace deps (topo order)
shell: pwsh
run: |
npm run build -w @clodocapeo/pulsar-client
npm run build -w @clodocapeo/pgm-correlator
npm run build -w @clodocapeo/pulsar-bundle-full
- name: Run capture <-> PGM compatibility suite
env:
PULSAR_LIVE_CAPTURE_COMPAT: ${{ steps.hardware-gpu.outputs.available == 'true' && '1' || '0' }}
PULSAR_BUNDLE_FULL_BINARIES_PATH: ${{ github.workspace }}/upstream/build_x64/rundir/RelWithDebInfo
shell: pwsh
run: npm run test -w @clodocapeo/capture-pgm-compat
- name: Report hardware CEF proof requirement
if: steps.hardware-gpu.outputs.available != 'true'
shell: pwsh
run: |
Write-Host "::warning::Accelerated capture <-> PGM proof skipped: this runner has no physical GPU. This is not a pass; release requires the real Prism/Twitch hardware E2E."
Add-Content -LiteralPath $env:GITHUB_STEP_SUMMARY -Value "## Accelerated CEF proof: NOT RUN`nThis host has no physical GPU. Product GPU acceleration remains enabled; the real Prism/Twitch hardware E2E is still required before release."
# ── Stage 3c : live broadcast (Twitch + record + diagnostic) ─────
#
# OFF the routine path since #132 (porteur decision : CI wall-clock).
# It was the second cost centre after the C++ build -- ~17 min on a
# push to main (10 min of real stream + submodule checkout + ffmpeg
# setup + VOD transcode + upload), and it serialises on a repo-wide
# non-cancelling Twitch lock, so two changes in flight queue behind
# each other.
#
# It is NOT deleted, because "rien n'est valide sans live Twitch
# complet" stays the project's validation criterion: the job still
# runs, unchanged, on a **tag push** (release-grade broadcast that
# feeds gh-pages + the Release attach) and on **workflow_dispatch**
# (operator-chosen duration/fps). What we give up is per-commit
# coverage of the real-ingest chain:
# - encoder -> real Twitch RTMPS ingest, sustained (regressions in
# rtmp_output / the service lifecycle now surface at tag time, or
# on the antenna run, instead of on the PR that caused them),
# - the VOD proof MP4 + diagnostic.json for that commit,
# - the gh-pages proof refresh on every push to main.
# The offline suite covers the wire contract, the encoders and the
# record path -- not a real ingest. Re-arm per-commit coverage by
# deleting the `if:` below.
#
# `live broadcast (Twitch)` is a REQUIRED status check on main: it
# must be dropped from the branch-protection contexts in the same
# move, or every PR waits forever on a check that no longer reports.
live-broadcast:
name: live broadcast (Twitch)
needs: build
if: >-
startsWith(github.ref, 'refs/tags/v')
|| github.event_name == 'workflow_dispatch'
runs-on: windows-2022
timeout-minutes: 60
permissions:
contents: write # gh-pages deploy is downstream, but release attach
# needs it on this job too if dispatch enables it.
# Twitch refuses two simultaneous streams ; serialise.
concurrency:
group: live-test-twitch
cancel-in-progress: false
steps:
- name: Checkout
uses: actions/checkout@v4
with:
submodules: recursive # the test scene + JSX live in scripts/live-test/
- name: Download pulsar-rundir
uses: actions/download-artifact@v4
with:
name: pulsar-rundir
path: .
- name: Setup Python 3.11
uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Install probe deps
shell: bash
run: |
python -m pip install --upgrade pip
pip install websockets
- name: Setup ffmpeg
uses: FedericoCarboni/setup-ffmpeg@v3
with:
ffmpeg-version: release
- name: Run Twitch live-broadcast probe
env:
TWITCH_STREAM_KEY: ${{ secrets.TWITCH_STREAM_KEY }}
# Duration matrix : workflow_dispatch input wins ; else
# 600 s (10 min) for a tag (release-grade broadcast that
# publishes to gh-pages + Release). The 60 s branch/PR arm is
# now dead code -- the job's `if:` keeps it off that path
# (#132) -- and is kept so re-arming per-commit coverage is a
# one-line revert, not a rewrite.
LIVE_TEST_DURATION: >-
${{
(github.event_name == 'workflow_dispatch' && github.event.inputs.duration_seconds)
|| ((github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v')) && '600')
|| '60'
}}
LIVE_TEST_FPS: ${{ github.event.inputs.fps || '60' }}
LIVE_TEST_VOD_DIR: ${{ runner.temp }}/pulsar-live-vod
shell: bash
run: |
echo "::notice::live-test duration=${LIVE_TEST_DURATION} s"
python scripts/probe-twitch-live.py 2>&1 | tee probe-output.log
- name: Compress + stage broadcast VOD
if: success()
shell: bash
run: |
set -euo pipefail
vod_path="$(grep '^LIVE_VOD_PATH=' probe-output.log | tail -1 | cut -d= -f2-)"
vod_path="${vod_path%$'\r'}"
if [ -z "${vod_path}" ] || [ ! -f "${vod_path}" ]; then
echo "::error::LIVE_VOD_PATH sentinel missing or file not found"
exit 1
fi
raw_size=$(stat -c%s "${vod_path}")
mkdir -p artefacts
short_sha="${GITHUB_SHA:0:7}"
stable="artefacts/pulsar-live-broadcast-proof.mp4"
versioned="artefacts/pulsar-live-broadcast-proof-${short_sha}.mp4"
ffmpeg -hide_banner -loglevel warning -y \
-i "${vod_path}" \
-c:v libx264 -preset fast -crf 23 -tune film -pix_fmt yuv420p \
-c:a aac -b:a 96k -ac 2 -ar 48000 \
-movflags +faststart \
"${stable}"
cp "${stable}" "${versioned}"
out_size=$(stat -c%s "${stable}")
ratio=$(awk -v a=${raw_size} -v b=${out_size} 'BEGIN{printf "%.2f", a/b}')
echo "::notice::VOD compression : raw=$(numfmt --to=iec ${raw_size}) -> compressed=$(numfmt --to=iec ${out_size}) (${ratio}× smaller)"
- name: Stage diagnostic JSON
if: success()
shell: bash
run: |
set -euo pipefail
diag="$(grep '^LIVE_DIAGNOSTIC_PATH=' probe-output.log | tail -1 | cut -d= -f2-)"
diag="${diag%$'\r'}"
if [ -n "${diag}" ] && [ -f "${diag}" ]; then
cp "${diag}" artefacts/diagnostic.json
echo "::notice::diagnostic.json staged ($(stat -c%s artefacts/diagnostic.json) bytes)"
else
echo "::warning::no diagnostic.json sentinel found"
fi
- name: Upload broadcast artefact (proof MP4 + diagnostic)
if: success()
uses: actions/upload-artifact@v4
with:
name: pulsar-live-broadcast-proof
path: |
artefacts/pulsar-live-broadcast-proof.mp4
artefacts/pulsar-live-broadcast-proof-*.mp4
artefacts/diagnostic.json
retention-days: 90
if-no-files-found: error
# The pulsar-websocket config carries `server_password` in clear.
# Uploading the raw file put that password in a 7-day, org-readable
# artefact, so we stage a REDACTED copy instead : every key that is
# not on the safe allow-list is replaced by a marker. The allow-list
# is a *deny-by-default* : a future obs-websocket key lands redacted
# unless someone vets it here.
#
# The diagnostic value is preserved on purpose -- what failure
# triage actually needs from this file is (a) did pulsar drop a
# config at all, (b) on which port, (c) was a password seeded
# (probe-twitch-live.py waits for a NON-EMPTY password, so
# "seeded but empty" is a real failure mode). None of that
# requires the password itself.
#
# NB : the previous path `obs-websocket/config.json` was workspace-
# relative and matched nothing (the file lives under the rundir),
# so `if-no-files-found: ignore` silently uploaded nothing. This
# step also fixes that blind spot.
- name: Stage redacted pulsar-websocket config on failure
if: failure()
shell: bash
run: |
set -euo pipefail
src="upstream/build_x64/rundir/RelWithDebInfo/bin/64bit/obs-websocket/config.json"
if [ ! -f "${src}" ]; then
echo "::warning::no pulsar-websocket config.json to redact (${src} absent)"
exit 0
fi
mkdir -p artefacts
python scripts/redact-websocket-config.py \
"${src}" artefacts/obs-websocket-config.redacted.json
echo "::notice::staged redacted pulsar-websocket config"
- name: Upload pulsar log on failure
if: failure()
uses: actions/upload-artifact@v4
with:
name: pulsar-failure-log
path: |
probe-output.log
artefacts/obs-websocket-config.redacted.json
retention-days: 7
if-no-files-found: ignore
# ── Stage 4 : publish proof MP4 to gh-pages ──────────────────────
publish-gh-pages:
name: publish proof to gh-pages
needs: live-broadcast
if: success() && (github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v'))
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: write
steps:
- name: Download broadcast artefact
uses: actions/download-artifact@v4
with:
name: pulsar-live-broadcast-proof
path: artefacts
- name: Publish to gh-pages
uses: peaceiris/actions-gh-pages@v4
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ./artefacts
publish_branch: gh-pages
force_orphan: true
user_name: 'github-actions[bot]'
user_email: 'github-actions[bot]@users.noreply.github.com'
commit_message: "live-test: refresh broadcast proof MP4 (${{ github.sha }})"
# ── Stage 5 : package light + full distros (tag only) ────────────
package:
name: package light + full distros
needs: build
if: startsWith(github.ref, 'refs/tags/v') || (github.event_name == 'workflow_dispatch' && github.event.inputs.enable_package == 'true')
runs-on: windows-2022
timeout-minutes: 15
steps:
- name: Checkout
uses: actions/checkout@v4
with:
submodules: recursive
- name: Download pulsar-rundir
uses: actions/download-artifact@v4
with:
name: pulsar-rundir
path: .
- name: Package light variant
shell: pwsh
run: ./scripts/package-win.ps1 -Variant light -Zip -SkipBuild
- name: Package full variant
shell: pwsh
run: ./scripts/package-win.ps1 -Variant full -Zip -SkipBuild
# #167 criteria 1 + 5 against the REAL packaged tree, not the script
# that produced it: nv-filters.dll present, and no NVIDIA SDK DLL or
# .trtpkg model anywhere inside. Control by absence -- what is being
# asserted is that Pulsar redistributes none of the SDK.
- name: nv-filters payload check on the packaged distros
shell: bash
run: |
set -e
for d in dist/pulsar-windows-x64-*/; do
echo "::group::$d"
python scripts/check-nv-filters-packaging.py --dist "$d"
echo "::endgroup::"
done
- name: Upload distro zips artefact
uses: actions/upload-artifact@v4
with:
name: pulsar-distros
path: dist/*.zip
retention-days: 30
if-no-files-found: error
# ── Stage 6 : attach distros + proof to GitHub Release (tag only) ─
release-attach:
name: GitHub Release attach
needs: [package, live-broadcast]
if: startsWith(github.ref, 'refs/tags/v') || (github.event_name == 'workflow_dispatch' && github.event.inputs.enable_release_attach == 'true')
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: write
steps:
- name: Download distro zips
uses: actions/download-artifact@v4
with:
name: pulsar-distros
path: artefacts
- name: Download broadcast proof
uses: actions/download-artifact@v4
with:
name: pulsar-live-broadcast-proof
path: artefacts
- name: Create Prism runtime release manifest
shell: bash
env:
TAG: ${{ github.ref_name }}
run: |
set -euo pipefail
base="$(printf '%s' "$TAG" | sed -E 's/^v([0-9]+\.[0-9]+\.[0-9]+).*/\1/')"
asset="pulsar-windows-x64-full-v${base}.zip"
path="artefacts/${asset}"
test -f "$path"
digest="sha256:$(sha256sum "$path" | cut -d' ' -f1)"
jq -n \
--arg version "$base" \
--arg tag "$TAG" \
--arg name "$asset" \
--arg digest "$digest" \
--arg url "https://github.com/${GITHUB_REPOSITORY}/releases/download/${TAG}/${asset}" \
'{schema_version:"prism.component.release.v1",component:"pulsar",version:$version,release_tag:$tag,artifact_name:$name,artifact_url:$url,artifact_sha256:$digest}' \
> artefacts/prism-pulsar-runtime-manifest.json
- name: Attach to release
uses: softprops/action-gh-release@v2
with:
files: |
artefacts/*.zip
artefacts/pulsar-live-broadcast-proof.mp4
artefacts/pulsar-live-broadcast-proof-*.mp4
artefacts/diagnostic.json
artefacts/prism-pulsar-runtime-manifest.json
fail_on_unmatched_files: true
# ── Stage 7 : npm packages publish (tag only, parallel to build) ─
# Independent of pulsar.exe build : different toolchain (Node, not
# MSVC), different runner (ubuntu-latest, faster). Only `needs: lint`
# so it doesn't sit waiting for the C++ build. Publishes pulsar-client
# first, then bundle + bundle-full which depend on it.
#
# #206 : the OR form this used to be (`startsWith(tag) ||
# (dispatch && flag)`) let workflow_dispatch publish from ANY ref --
# the tag check and the dispatch check were alternatives, not both
# required, so `enable_npm_publish=true` dispatched from an arbitrary
# branch bypassed the v*.*.* tag protection entirely. `github.ref` is
# AND-ed in now : whatever the event, the ref actually being run
# against must be a tag. workflow_dispatch still works exactly as
# before -- but only when the operator explicitly picks a `v*.*.*`
# tag as "Use workflow from" and flips the input on ; a push of a
# matching tag keeps publishing unconditionally, same as always.
npm-publish:
name: publish @clodocapeo/pulsar-* to npm
needs: lint
if: startsWith(github.ref, 'refs/tags/v') && (github.event_name != 'workflow_dispatch' || github.event.inputs.enable_npm_publish == 'true')
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: read
id-token: write # OIDC for npm provenance (kept for future use)
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
registry-url: 'https://registry.npmjs.org/'
- name: Setup ffmpeg
uses: FedericoCarboni/setup-ffmpeg@v3
with:
ffmpeg-version: release
- name: Verify tag matches VERSION
if: startsWith(github.ref, 'refs/tags/v')
run: |
set -e
v=$(cat VERSION | tr -d '[:space:]')
tag='${{ github.ref_name }}'
base="$(printf '%s' "$tag" | sed -E 's/^v([0-9]+\.[0-9]+\.[0-9]+).*/\1/')"
[ "$base" = "$v" ] || { echo "::error::tag $tag does not match VERSION (v$v)"; exit 1; }
- name: Install npm workspaces
# PULSAR_BUNDLE_SKIP_POSTINSTALL skips the binary download in
# pulsar-bundle's postinstall -- tsc-publish doesn't need pulsar.exe.
# --force bypasses npm's os/cpu check : pulsar-bundle declares
# os:["win32"] cpu:["x64"] but this runner is ubuntu-latest, and
# we only need tsc to compile the TS sources. The published tarball
# carries the os/cpu constraints unchanged for end users.
env:
PULSAR_BUNDLE_SKIP_POSTINSTALL: '1'
run: npm install --no-audit --no-fund --force
- name: Build npm packages (topo order)
run: |
npm run build -w @clodocapeo/pulsar-client
npm run build -w @clodocapeo/pgm-correlator
npm run build -w @clodocapeo/pulsar-bundle
npm run build -w @clodocapeo/pulsar-bundle-full
# pgm-correlator's tests shell out to a real ffmpeg/ffprobe (a
# synthetic-but-real fixture, not a live PGM claim -- see its
# README). Install ffmpeg explicitly because the runner image does not
# guarantee that ffmpeg/ffprobe are present.
- name: Test npm packages
run: |
npm run test -w @clodocapeo/pulsar-client
npm run test -w @clodocapeo/pgm-correlator
npm run test -w @clodocapeo/pulsar-bundle
npm run test -w @clodocapeo/pulsar-bundle-full
- name: Publish to npm
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
run: |
publish_if_missing() {
package_name="$1"
package_dir="${package_name##*/}"
package_version="$(node -p "require('./packages/${package_dir}/package.json').version")"
if npm view "${package_name}@${package_version}" version >/dev/null 2>&1; then
echo "${package_name}@${package_version} already exists; keeping the immutable npm release"
return 0
fi
npm publish -w "$package_name" --access public
}
publish_if_missing @clodocapeo/pulsar-client
publish_if_missing @clodocapeo/pulsar-bundle
publish_if_missing @clodocapeo/pulsar-bundle-full