Studio 2.0

.agent/notes.md

@@ -0,0 +1,9 @@
+# Agent Notes
+
+This file is for durable repo-specific learnings that help future work but do not yet belong as hard rules.
+
+## Usage
+
+- Put normative, repo-level guidance in `.agent/rules.md` or `.agent/rules/`.
+- Put useful learnings, reminders, heuristics, and recurring gotchas here when they are worth remembering but are not yet stable enough to be promoted into rules.
+- When a note becomes clearly normative or repeatedly important, move it into the relevant rules file.

.agent/rules.md

@@ -1,35 +1,36 @@
 # Agent Rules
 
-This is the canonical entry point for repository rules.
-
-Use this file as the quick-reference summary, then consult the focused rule files in `.agent/rules/` for the details that match the task.
-
-## Read First
-
-- Verify all changes with the appropriate test and lint commands before considering a task done.
-- Use the local `./venv` for backend commands.
-- Update tests when logic changes; do not weaken tests to fit broken behavior.
-- Keep wiki docs and `wiki/Changelog.md` aligned with shipped behavior.
-- Prefer manual verification by the user for UI changes unless they explicitly ask for browser-driven verification.
-- Treat filesystem paths as a security surface; follow the backend security/path rules.
-- Push back when the requested implementation is weaker than the better pattern already available in the repo.
-
-## Rule Map
-
-- [`verification.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/verification.md)
-  Test-first expectations, verification commands, and definition of done.
-- [`workflow.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/workflow.md)
-  Ownership, pushback, documentation, and manual verification expectations.
-- [`backend.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/backend.md)
-  Progress/state consistency, worker sync, path safety, and backend structural guidance.
-- [`frontend.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/frontend.md)
-  UI consistency, accessibility, responsiveness, and frontend quality guidance.
-
-## Priority Order
-
-When rules overlap, follow them in this order:
-
-1. Verification and correctness
-2. Security and data safety
-3. User-visible behavior and documentation accuracy
-4. Maintainability and consistency
+This is the router for repository rules.
+
+## Task Map
+
+Use the smallest rule set that matches the task.
+
+1. Always read [`verification.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/verification.md) before calling code work complete.
+1. For frontend UI or UI-state work, read [`frontend.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/frontend.md), the relevant frontend subfile(s), and [`modular_architecture.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/modular_architecture.md).
+1. For frontend state ownership or overlays, read [`frontend-state.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/frontend-state.md) plus [`modular_architecture.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/modular_architecture.md).
+1. For frontend UX, recovery, waiting, or editor-flow work, read [`frontend-ux.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/frontend-ux.md) plus [`frontend-state.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/frontend-state.md) when state ownership is involved.
+1. For frontend interaction, styling, semantics, or responsive layout work, read [`frontend-interactions.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/frontend-interactions.md).
+1. For backend progress or ETA work, read [`backend.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/backend.md), [`backend-progress.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/backend-progress.md), and usually [`modular_architecture.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/modular_architecture.md).
+1. For artifact publication or recovery work, read [`backend.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/backend.md), [`backend-artifacts.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/backend-artifacts.md), and usually [`modular_architecture.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/modular_architecture.md).
+1. For backend path handling, read [`backend.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/backend.md) and [`backend-paths.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/backend-paths.md).
+1. For routing, service boundaries, queue policy, or migration shape, read [`backend.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/backend.md), [`backend-boundaries.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/backend-boundaries.md), and [`modular_architecture.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/modular_architecture.md).
+1. For large-file refactors or files over the architecture thresholds, read [`modular_architecture.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/modular_architecture.md) plus the relevant frontend/backend rule files.
+1. For plan, rollout, or documentation changes, read [`workflow.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/workflow.md) and usually [`modular_architecture.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/modular_architecture.md).
+
+## What Matters Most
+
+- Verify changes with the relevant tests and linting before calling work complete.
+- Use the local `./venv` for backend tooling.
+- Update tests when behavior or logic changes.
+- Fix concrete bugs or regression risks in the same pass.
+- Keep implementation docs, wiki pages, and `wiki/Changelog.md` aligned with shipped behavior.
+- Treat paths, assets, and output publication as security-sensitive.
+- Prefer the better long-term boundary when a quick fix would deepen worker-centric or UI-state coupling.
+
+## Rule Ownership
+
+- The agent owns the organization and maintainability of `.agent/rules.md`, `.agent/rules/`, and related agent guidance files.
+- The agent may restructure, split, rename, cross-link, and refine these rule files without asking for additional permission when doing so improves discoverability, routing, or long-term usability.
+- Normative, repo-level guidance should live in the rules files.
+- Durable repo learnings, heuristics, and reminders that are useful to the agent but are not yet normative rules may be stored in [`notes.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/notes.md).

.agent/rules/backend-artifacts.md

@@ -0,0 +1,12 @@
+# Backend Artifacts
+
+Use this file for completion checks, artifact publication, recovery, and cache safety.
+
+## Core Rules
+
+- Raw file existence alone is never enough to mark render work complete.
+- Validated artifact manifests plus canonical persistence state determine whether output is reusable.
+- Publish generated outputs atomically: temp file first, validate, write manifest, then promote into the final artifact location.
+- Shared cached artifacts must be immutable once published.
+- When job state is re-queued, recovered, or invalidated, clear stale runtime metadata and recalculate status from canonical state.
+- If a revision changes, mark prior artifacts stale explicitly rather than silently treating them as valid.

.agent/rules/backend-boundaries.md

@@ -0,0 +1,13 @@
+# Backend Boundaries
+
+Use this file for routing, service layers, queue policy, and legacy migration boundaries.
+
+## Core Rules
+
+- Route handlers should call domain services and orchestrator services, not engine-specific or worker-specific code directly.
+- Engine wrappers must not decide scheduling policy.
+- Queue tasks must declare resource needs; they must not acquire ad-hoc locks scattered through task code.
+- Keep domain logic, orchestration logic, engine integration, and infrastructure concerns in separate modules.
+- Prefer explicit repositories and services over ad-hoc data access from route handlers and workers.
+- During migration, compatibility adapters are acceptable, but they should live in explicit legacy or adapter layers.
+- Do not let temporary migration code redefine the long-term architecture.

.agent/rules/backend-paths.md

@@ -0,0 +1,47 @@
+# Backend Paths
+
+Use this file for request-derived filesystem paths, scanned directories, and containment checks.
+
+## Core Rules
+
+- Treat any filesystem path derived from request data, DB values, uploaded filenames, or user-editable names as untrusted.
+- For existing files or directories, prefer enumerating a trusted root and matching by `entry.name`.
+- For new paths, use the explicit containment pattern that CodeQL recognizes well:
+  1. validate with a strict regex
+  2. build with `os.path.join(...)`
+  3. normalize with `os.path.normpath(...)`
+  4. absolutize with `os.path.abspath(...)`
+  5. verify the result stays under the trusted root before reading or writing
+- Reject traversal-style input instead of silently “fixing” it.
+- Do not hide security-critical path creation in vague helpers.
+
+## CodeQL Lessons Learned
+
+- If CodeQL traces a route parameter into a filesystem sink, repeat the validation and containment proof in the same function as the sink. A shared helper may be safe at runtime but opaque to the scanner.
+- For each filesystem sink argument (`open`, `shutil.move`, `shutil.copy*`, `Path.exists`, `rmtree`, etc.), make the value passed to the sink a locally checked string:
+  1. derive it from a trusted root with `os.path.join(...)`
+  2. normalize with `os.path.normpath(...)`
+  3. absolutize with `os.path.abspath(...)`
+  4. compare against a trusted-root prefix before the sink
+- For moves/copies, prove both sides independently. The source and destination can produce separate CodeQL findings even when they appear on the same line.
+- When migrating legacy files named from DB values, prefer enumerating the trusted directory and matching `entry.name`; then normalize and contain-check the matched `entry` before using it.
+- If a destination filename does not need to preserve a user/DB identifier, generate a deterministic safe filename such as `seg_{index}.wav` and persist that mapping instead of embedding the untrusted ID in the path.
+- For speaker-profile resolution and metadata files, prove the profile root locally in the same function, then use trusted `os.scandir(...)` results or direct string-based `open(...)` on the proved path instead of `Path.exists()`, `Path.read_text()`, or `Path.write_text()`.
+- For storage-version probes, scan the trusted root first and match `entry.name` rather than scanning a derived child directory built from the user-supplied name, even if the child path is validated.
+- For voice bundle export/import helpers, resolve the voice root by scanning the trusted voices directory and matching `entry.name`, then prove the manifest path relative to that discovered root before calling `os.path.isdir()`, `os.path.exists()`, or `open()`.
+- For profile metadata reads and repairs, scan the parent directory and match the exact metadata filename by `entry.name` before opening it; do not rely on `os.path.exists()` on a derived metadata path when CodeQL is tracing the source.
+- If CodeQL still traces a metadata or discovery scan, prefer a direct `try/except os.scandir(validated_parent)` block over a separate `os.path.isdir()` guard on the same derived path, so the root proof and sink stay in the same control-flow block.
+- **Extreme Hardening for Sinks**: Standard `Path` object methods like `.exists()`, `.is_dir()`, and `.mkdir()` are treated as sinks. CodeQL often ignores containment proofs on the parent `Path` object if the child path is derived and then checked via these methods.
+  - **Solution**: Convert all `Path` sinks to string-based `os.path` sinks (`os.path.exists`, `os.path.isdir`, `os.makedirs`).
+  - **Pattern**: Resolve the final string using `os.path.abspath(os.path.realpath(os.fspath(path)))` and perform the `startswith(trusted_prefix)` check immediately before the sink in the same local block.
+- **Source Sanitization is King**: While sink-level proofs are required for completeness, sanitizing user-provided strings (e.g., validating UUIDs or using strict regex) at the **API entry point** is the most effective way to break the taint chain early.
+  - **Pattern**: Use a public helper like `config.canonical_chapter_id(id)` that raises `ValueError` on anything except a valid UUID before passing the ID to any domain logic.
+- **Unrolled Proofs for Linear Path Analysis**: CodeQL's path-sensitive analysis frequently fails to follow containment proofs through loops (e.g., `for root in trusted_roots:`). 
+  - **Solution**: Explicitly "unroll" these checks into `if/elif` blocks for each trusted root. This provides a clear, linear path for the scanner to follow from the proof to the sink.
+- **Taint Persistence in Objects**: CodeQL often tracks taint through `pathlib.Path` objects even after a containment check. 
+  - **Pattern**: For high-risk sinks, convert the `Path` to a string (`os.fspath`), resolve it fully (`abspath/realpath`), and perform a fresh `startswith()` check immediately before the sink in the same local block.
+- **Test-Mode Database Safety**: When running tests, a safety guard should prevent connecting to non-test databases. 
+  - **Pattern**: Resolve the `DB_PATH` and the system temp roots (including symlinks like `/tmp` -> `/private/tmp` on macOS) fully before comparison. Use `Path(path).resolve()` on both sides to ensure consistent behavior across platforms.
+- **Temp Dir Containment Traps**: Be careful when allowing `tempfile.gettempdir()` in containment proofs to support tests.
+  - **Risk**: Allowing the entire temp root can permit "traversal" between unrelated test directories (e.g., from `tmp/project_a` to `tmp/project_b`), which security tests correctly identify as a vulnerability.
+  - **Solution**: Instead of allowing the broad temp root, ensure that tests correctly patch the `PROJECTS_DIR` and `XTTS_OUT_DIR` to their specific temp workspaces. The standard `startswith(p_root)` check will then naturally allow the test paths without compromising security.

.agent/rules/backend-progress.md

@@ -0,0 +1,10 @@
+# Backend Progress
+
+Use this file for ETA, progress, and WebSocket state consistency rules.
+
+## Core Rules
+
+- WebSocket progress values must be rounded to exactly 2 decimal places.
+- Only broadcast progress updates when the value advances meaningfully, the status changes, or the event carries important new context.
+- Queue and progress services own progress math. Do not duplicate ETA or completion logic inside engine wrappers or route handlers.
+- Parent-child job behavior should be modeled explicitly rather than inferred from filenames or loose naming conventions.

.agent/rules/backend.md

@@ -1,33 +1,22 @@
 # Backend Rules
 
-## Progress And State Consistency
+Use this file when the task touches backend orchestration, queueing, progress, artifact publication, or filesystem path handling.
 
-- WebSocket progress values must be rounded to exactly 2 decimal places.
-- Only broadcast progress updates when the value advances by at least 1%.
-- When a job is re-queued, reset, or recovered, clear stale metadata such as logs, timestamps, progress, and warnings.
-- Disk state is the source of truth when UI status and actual files disagree.
+## Read The Right Subfile
 
-## Technical Environment
+- [`backend-artifacts.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/backend-artifacts.md) for manifest validation, publish flow, cache immutability, and recovery safety.
+- [`backend-progress.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/backend-progress.md) for progress math, ETA rules, rounding, and update throttling.
+- [`backend-boundaries.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/backend-boundaries.md) for route/service/engine boundaries and migration discipline.
+- [`backend-paths.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/backend-paths.md) for request-derived paths and containment checks.
 
-- Always use the local `./venv` for backend tooling.
-- When worker threads update `j` objects, follow up with `update_job` so the WebSocket bridge receives the change.
+## Load Order
 
-## Path Safety And Code Scanning
+1. [`backend-progress.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/backend-progress.md) for ETA, rounding, and progress consistency.
+1. [`backend-artifacts.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/backend-artifacts.md) for publish/recovery/immutability.
+1. [`backend-paths.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/backend-paths.md) for any request-derived path handling.
+1. [`backend-boundaries.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/backend-boundaries.md) for routing, orchestration, queue policy, and migration shape.
 
-- Treat any filesystem path derived from request data, DB values, uploaded filenames, or user-editable names as untrusted.
-- For existing files or directories, prefer enumerating a trusted root and matching by `entry.name`.
-- For new paths, use the explicit containment pattern that CodeQL recognizes well:
-  1. validate with a strict regex
-  2. build with `os.path.join(...)`
-  3. normalize with `os.path.normpath(...)`
-  4. absolutize with `os.path.abspath(...)`
-  5. verify the result stays under the trusted root before reading or writing
-- For filenames and profile names, require flat single-segment names unless nested paths are truly intentional.
-- Reject traversal-style input instead of silently “fixing” it.
-- Avoid hiding security-critical path creation in generic helpers unless the helper exactly mirrors the accepted containment shape and has already been proven scanner-safe.
-- When code scanning regresses, export the latest `code-scanning-alerts.json`, group by rule and file, and fix the current highest-concentration sink lines directly before broad refactors.
+## Pair With
 
-## Structural Guidance
-
-- If a file exceeds 500 lines, consider splitting it along real logical boundaries.
-- Refactor carefully to avoid circular dependencies or scattering tightly coupled logic across too many files.
+- [`modular_architecture.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/modular_architecture.md) for Studio 2.0 boundary rules.
+- [`verification.md`](/Users/stevendunn/GitHub-Steven/audiobook-factory/.agent/rules/verification.md) for the required backend test and lint verification.