Skip to content

Lua API: quarto.project, quarto.doc paths, comparable quarto.version, os.exit fails the document (GH #939-#942) - #949

Merged
cscheid merged 13 commits into
mainfrom
feature/lua-api-q1-parity
Oct 27, 2026
Merged

cscheid merged 13 commits into
mainfrom
feature/lua-api-q1-parity

Conversation

@cscheid

@cscheid cscheid commented Oct 27, 2026

Copy link
Copy Markdown
Member

Closes #939, closes #940, closes #941, closes #942.

Four Lua API gaps reported by @atheriel while porting the Posit Connect docs from Quarto 1.9.32 to q2 0.36.0. All four are things a Quarto 1 filter does on its first few lines, and all four reproduced on main at 3f7d4b0. Plan with findings and design: claude-notes/plans/2026-10-27-lua-api-q1-parity-gh-939-942.md (epic bd-pqj37chw; one --no-ff merge per sub-task).

What changes

#942 — quarto.version is a real pandoc.types.Version (bd-5qepzwst)
quarto.version was the literal table {0, 1, 0}. It is now built from quarto_util::cli_version() (nightly override included, trimmed to the leading dotted-numeric part as Q1 does) as a new LuaVersion userdata behind pandoc.types.Version: comparable with strings on either side (quarto.version >= "1.4"), tostring, must_be_at_least, 1-based component indexing, and table.concat(quarto.version, ".") keeps working (Lua 5.4's table.concat accepts a userdata with __index + __len; pinned by test). PANDOC_VERSION / PANDOC_API_VERSION are Versions too; pandoc.utils.type reports Version.

#939 — quarto.project and QUARTO_PROFILE (bd-ip1lrgra)
New HostContext on pampa's LuaOptions (input/output file, project directory / output directory / offset, active profiles, environment snapshot), filled by quarto-core's new lmd::lua_options_for_document for both the user-filter stage and the shortcode transform. quarto.project has Q1's shape: directory / offset / output_directory nil outside a project, profile always a pandoc.List. offset is Q1's projectOffset (relative path back to the root, forward slashes, "." at the root).

os.getenv now reads from an environment snapshot the host takes once in StageContext::new, on the thread that builds the context, before any script runs: the runtime's environment, then _environment file values for keys the real environment lacks, then QUARTO_PROFILE forced to the active list. Rationale: filters run on tokio worker threads under block_in_place, and libc getenv there races any setenv in the process; a snapshot also gives every script in a render one consistent environment. Without a host (pampa tests, conformance suite) the stdlib os.getenv is left alone. NativeRuntime::env_all switched to vars_os so a non-UTF-8 entry no longer panics.

#940 — quarto.doc.input_file / quarto.doc.output_file and a minimal PANDOC_STATE (bd-diu7rtes)
Absolute paths, as in Q1, in both the filter and shortcode environments. PANDOC_STATE = { input_files = pandoc.List{input}, output_file }; the other pandoc fields stay nil on purpose so a filter that needs one fails at its own line rather than reading a plausible wrong default.

#941 — os.exit fails the document, not the process (bd-9u10sb4b)
os.exit on the os table (native stdlib or WASM synthetic) now raises an external LuaExit { code } error with Lua's code mapping; apply_lua_filter reports it as LuaFilterError::Exit: Lua filter <path> called os.exit(<code>); the document was not rendered. The rest of the project renders, later errors are still reported, the summary prints, and the run exits non-zero. os.exit(0) also fails the document (Q1 parity: pandoc exiting without output was a failed render). pcall(os.exit, 1) catches it, like any error. error() is unchanged.

Docs: three new sections in docs/guides/authoring/lua-filters.qmd (where the filter is running; checking versions; failing a render).

Verification

  • cargo xtask verify: all steps green, including the hub-client tests against a WASM rebuilt from this branch (one smoke fixture, extensions/builtin-version-shortcode, hard-coded 0.1.0 and was updated to expect the real version; it is checked by both the native and the WASM smoke runs).
  • WASM Lua tests (cargo test -p pampa --test wasm_lua --target wasm32-unknown-unknown ...): 10/10, including the extended synthetic-os test (os.getenv, os.exit).
  • New tests: pampa unit tests (version.rs, host_context.rs, os_exit.rs), quarto-core tests for the snapshot layers and the offset, CLI e2e lua_host_context_cli.rs (two-page website under --profile with an _environment file; single-file render) and lua_os_exit_cli.rs (the issue's a/b/c project; os.exit(0)).
  • The four issue repros re-run by hand against the built binary; each now prints what the issue's "Expected" section asks for.
  • Not verified: Windows. cargo clippy --target x86_64-pc-windows-msvc cannot run from macOS for pampa, quarto-core or quarto (aws-lc-sys, wasmtime, tree-sitter compile C for the target). The new code uses no platform APIs; path assertions in the e2e tests are built with Path::join. A Windows run of cargo clippy -p pampa -p quarto-core -p quarto --all-targets -- -D warnings and the two new e2e tests would be welcome.

Follow-ups (not filed)

🤖 Generated with Claude Code

cscheid and others added 12 commits October 27, 2026 16:11
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
quarto.version was the literal table {0, 1, 0}: wrong version, not
comparable (`quarto.version >= "1.4"` raised "attempt to compare string
with table"), and pandoc.types did not exist so a filter could not wrap
it either. PANDOC_VERSION / PANDOC_API_VERSION had the same shape.

Add lua/version.rs with a LuaVersion userdata behind pandoc.types.Version:
constructor from string / integer / sequence / Version; __lt and __le as
meta functions over (Value, Value) so a string may be either operand;
__eq against Versions only (Lua never calls it for mixed types, as in
pandoc); __tostring; __index for 1-based components and
must_be_at_least; __len. With __index and __len, Lua 5.4's table.concat
accepts the userdata, so Q1-era `table.concat(quarto.version, '.')`
keeps working (pinned by test). pandoc.utils.type reports "Version".

quarto.version now comes from quarto_util::cli_version(), trimmed to its
leading dotted-numeric prefix (0.37.0-nightly.YYYYMMDD -> 0.37.0, Q1's
leadingDottedNumber). PANDOC_VERSION and PANDOC_API_VERSION are Version
objects with their previous components.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Quarto 1 filters index quarto.project (directory, offset, output_directory,
profile) on their first lines; q2 had no such table, and `--profile x` was
invisible to os.getenv("QUARTO_PROFILE") because q2 never writes it back
into the process environment.

pampa: LuaOptions gains a HostContext (input/output file, HostProject
{directory, output_directory, offset}, active profiles, environment
snapshot). lua/host_context.rs registers quarto.project with Quarto 1's
shape (nil paths outside a project, profile always a pandoc.List) and,
when the host supplied a snapshot, replaces os.getenv with a lookup in
it. The WASM synthetic os table gains a runtime-backed os.getenv.

quarto-core: StageContext::new takes the environment snapshot once, on
the thread that builds the context, before any script runs
(environment::snapshot_environment: runtime env_all, then _environment
values for missing keys, then QUARTO_PROFILE forced to the active list).
This keeps libc getenv off filter worker threads, where it would race a
setenv anywhere in the process, and gives every script in a render one
consistent environment. The snapshot rides on RenderContext for the
shortcode transform. lmd::lua_options_for_document fills the HostContext
(offset = Q1's projectOffset via pathdiff, forward slashes).

quarto-system-runtime: NativeRuntime::env_all uses vars_os so a
non-UTF-8 entry no longer panics, now that it runs per document.

Tests: pampa unit tests for the table and the getenv shadow; quarto-core
tests for the snapshot layers and the offset; a CLI e2e test rendering a
two-page website under --profile with an _environment file, plus a
single-file render for the Q1 nils. WASM os test extended.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…_STATE (GH #940)

A Quarto 1 filter learns which file it is processing from
quarto.doc.input_file / quarto.doc.output_file (absolute paths) or from
PANDOC_STATE; q2 had neither, and the working directory is the project
root for every page, so there was no way to compute a page-relative link.

Set both quarto.doc fields from the HostContext in the filter and
shortcode environments, and a minimal PANDOC_STATE { input_files =
pandoc.List{input}, output_file }. output_file is the resolved absolute
path: pandoc's is the writer's output name, but Quarto 1 joined it with
the output directory before any filter saw it, so this is what Q1 filters
ended up with. The other pandoc fields stay nil on purpose, so a filter
that needs one fails at its own line instead of reading a plausible wrong
default.

Tests: pampa unit tests with and without a host; the host-context CLI e2e
test now checks both pages' paths against quarto.project.directory with
Path::join, and PANDOC_STATE against quarto.doc.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ess (GH #941)

Quarto 1 filters call os.exit(1) to fail a render; pandoc was a child
process, so that failed one document. In q2 the native Lua state is
Lua::new() with the full stdlib, so os.exit was libc exit: the process
died mid-render with nothing in _site/, no summary, and parse errors on
pages not yet reached never reported.

lua/os_exit.rs replaces os.exit on the os table (native stdlib or WASM
synthetic) with a function raising an external LuaExit { code } error,
with Lua's code mapping (none/true -> 0, false -> 1, n -> n).
apply_lua_filter walks the mlua error chain for it and reports a new
LuaFilterError::Exit: "Lua filter <path> called os.exit(<code>); the
document was not rendered". Downstream already fails just that document,
continues, prints the summary, and exits non-zero. os.exit(0) also fails
the document (Q1 parity: pandoc exiting without output was a failed
render). The shortcode environment gets the same replacement. error() is
unchanged.

Tests: unit tests for the code mapping, chain walking and the pcall
caveat; a CLI e2e test with the issue's a/b/c website asserting b.html
renders, a.html does not, c.qmd's Q-2-12 is reported, the summary prints
and the status is non-zero; a second for os.exit(0). WASM os test
extended. Also carries rustfmt reflows of two files from the previous
sub-tasks.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…failing a render

Three new sections in the Lua filters guide for the API surface added for
GH #939-#942, and the plan's work items ticked with implementation notes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ender gate passes

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…t 0.1.0

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@posit-snyk-bot

posit-snyk-bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

✅ Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
✅ Open Source Security 0 0 0 0 0 issues
✅ Licenses 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

…ua filters guide

The docs render parsed `element`'s as an opening single quote (Q-2-7),
failing the docs build in CI.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@cscheid
cscheid merged commit 3971f69 into main Oct 27, 2026
17 checks passed
@cscheid
cscheid deleted the feature/lua-api-q1-parity branch October 27, 2026 23:44
cscheid added a commit that referenced this pull request Oct 28, 2026
…code-snapshot

Lua API: env shortcode reads the per-render environment snapshot (stacked on #949)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment