Skip to content

feat: show per-video YouTube playlist ingestion across WebUI and extension - #2738

Open
rmusser01 wants to merge 85 commits into
devfrom
codex/youtube-playlist-ingest-plan
Open

feat: show per-video YouTube playlist ingestion across WebUI and extension#2738
rmusser01 wants to merge 85 commits into
devfrom
codex/youtube-playlist-ingest-plan

Conversation

@rmusser01

@rmusser01 rmusser01 commented Jul 15, 2026

Copy link
Copy Markdown
Owner

Change summary

Pending human-written summary.

Summary

  • What changed: Added the version-2 YouTube playlist inspection, occurrence materialization, bounded run/job transport, durable recovery, and shared per-item lifecycle UI for WebUI and browser-extension ingestion.
  • Why: A playlist submission previously appeared as one source link even though the backend expanded it; users now review and track every selected video occurrence.
  • The capability gate fails closed on older servers, and direct URL/file ingest continues through the existing paths.

Validation

  • Tests added/updated for behavior changes
  • Relevant unit/integration tests pass locally
  • Docs and Backlog records updated
  • Backend integration gate: 683 passed, 2 skipped
  • PostgreSQL-targeted gate: 4 passed, 24 skipped because PostgreSQL was unavailable in the local fixture environment
  • Frontend branch regression set: 25 files, 878 tests passed
  • Focused Quick Ingest modal gate: 208/208 passed
  • Chromium playlist reload/reattach journey: 1/1 passed
  • Touched-scope ESLint, scoped Prettier check, and range whitespace checks passed
  • Bandit: 0 findings across all changed production Python paths
  • Full frontend TypeScript was not rerun; TASK-12113 records the existing repository-baseline three-attempt cap

UX Audit Checklist (v2 Stage 5)

  • Full npm run e2e:smoke:stage5 was not run; the scoped playlist Chromium workflow passed
  • Full all-pages Playwright suite was not run; the scoped playlist Chromium workflow passed
  • No Maximum update depth exceeded warnings in the playlist regression gates
  • No unresolved interpolation placeholders in the deterministic playlist journey
  • WebUI/extension parity validated for the shared Quick Ingest flow
  • Flashcards checklist is not applicable

Watchlists Accessibility and Scale Checklists

  • Not applicable; no Watchlists files or behavior changed

Risk & Rollback

  • Risk level: Medium
  • Rollback plan: Revert the playlist feature commits. The version-2 capability gate is fail-closed, so clients without the complete contract remain on existing ingest behavior.

Tracking

  • TASK-12112: backend version-2 playlist ingestion contract
  • TASK-12113: shared WebUI and extension per-video playlist ingestion
  • Design: Docs/superpowers/specs/2026-07-12-youtube-playlist-per-item-ingest-design.md
  • Plans: Docs/superpowers/plans/2026-07-12-youtube-playlist-ingest-shared-frontend.md and Docs/superpowers/plans/2026-07-14-quick-ingest-explicit-submission-command.md

Summary by cubic

Adds per-video YouTube playlist ingestion across the WebUI and extension with server-side inspection, virtualized review, per-item metadata, and per-video lifecycle. Includes submission idempotency, strict PostgreSQL RLS, durable IndexedDB recovery, and review-driven accessibility and handoff fixes; requires server DB migrations while older servers keep existing ingest.

  • New Features

    • Quick Ingest inspects playlists server-side and lists items with @tanstack/react-virtual; users can select/deselect, set per-item metadata, and choose duplicate handling.
    • One explicit submission command creates a playlist run and binds each occurrence; outcomes are per video (completed, included existing, metadata updated, skipped, failed, cancelled).
    • Session persistence via IndexedDB (dexie) with submission leases enables recovery after refresh or extension restarts; extension handoff opens Quick Ingest directly from playlist URLs.
    • Backend v2 playlist endpoints for preflight, materialization, and runs; bounded jobs support SQLite/PostgreSQL, resolve duplicates safely under RLS, and capability-gate playlists (fail-closed on older servers).
  • Bug Fixes

    • Make playlist run creation idempotent and replayable by session; harden playlist session identity.
    • Fence playlist planning/cleanup and validate recovered identities; discard stale plans. Enforce exact cancellation bindings and retry safely; propagate retry backpressure.
    • Enforce PostgreSQL RLS across Jobs and playlist tables with dedicated privileges; normalize SQLite cleanup cutoffs (julianday), publish jobs with DB-clock admission, return Retry-After on pressure, and require MEDIA_READ plus rate limits on owner-scoped GETs; normalize bad cursors to 404.
    • Address PR review findings (TASK-12971): improve file upload control accessibility and playlist inspection i18n; align sidepanel handoff and open-request retention.

Written for commit f303f5e. Summary will update on new commits.

Review in cubic

rmusser01 added 30 commits July 15, 2026 06:45
Require MEDIA_READ and media.read rate limits on owner-scoped GET routes. Normalize empty, oversized, malformed, and tampered cursors to a generic 404 without echoing input. Verify 25 endpoint tests and 141 broad regressions (6 PostgreSQL fixture-policy skips); Black, Ruff, compileall, diff-check, and Bandit pass.
Publish validated sentinel-scheduled jobs with backend database time, enforce exact queue/payload/job identity, and reconcile old unbound jobs through JobManager cancellation. Use database-clock admission and SQLite-safe expiry predicates, expose bounded OpenAPI request schemas, derive root-path-aware links, and return Retry-After on admission pressure. Verification: 211 non-property tests passed with 9 PostgreSQL policy skips; 2 property tests passed in isolation; Black, Ruff, compileall, diff-check, and Bandit passed with zero findings.
Compare SQLite cleanup cutoffs through julianday for preflights, materializations, runs, and their child resources while preserving PostgreSQL timestamp predicates. Regression coverage keeps active DB-time reservations linked to acquirable jobs and deletes both historical ISO and DB-native timestamp formats. Verification: 214 Task 1-4 tests passed with 9 PostgreSQL policy skips; property assertions passed on isolated retry; Black, Ruff, compileall, diff-check, and Bandit passed.
@coderabbitai

coderabbitai Bot commented Jul 15, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: cf41fa43-4ac7-49ac-b8fb-dc5c2571901f

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/youtube-playlist-ingest-plan

Comment @coderabbitai help to get the list of available commands.

@rmusser01
rmusser01 marked this pull request as ready for review July 15, 2026 13:50

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements a comprehensive backend and frontend system for per-item YouTube playlist ingestion. The changes introduce durable owner-scoped storage, asynchronous inspection, and atomic run creation, ensuring that playlist items are tracked individually rather than as opaque jobs. The review feedback correctly identifies that the implementation plan contains hardcoded local file paths, which reduces portability and should be corrected to use relative paths or environment variables.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread Docs/superpowers/plans/2026-07-12-youtube-playlist-ingest-backend.md Outdated
@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

Per-video YouTube playlist ingestion across WebUI and extension (v2 contract)

✨ Enhancement 🧪 Tests 📝 Documentation ⚙️ Configuration changes 🕐 40+ Minutes

Grey Divider

AI Description

• Adds a v2 playlist ingestion contract (preflight, materialization, runs, events) to expose
 per-video occurrences end-to-end.
• Implements durable, owner-scoped persistence + bounded job execution with recovery, retry, and
 cancellation semantics.
• Updates shared Quick Ingest UI (WebUI + extension) to inspect playlists, select videos, and track
 per-item lifecycle.
• Adds IndexedDB session persistence and broad backend/frontend test coverage; older servers
 fail-closed to legacy ingest.
Diagram

graph TD
  U(["User"]) --> UI["Quick Ingest UI"] --> API["Playlist v2 API"] --> DB[("Playlist Store")]
  UI --> IDB[("IndexedDB Session")]
  API --> W["Jobs/Worker"] --> YT{{"YouTube/yt-dlp"}}
  subgraph Legend
    direction LR
    _db[(Database)] ~~~ _svc([Service]) ~~~ _ext{{External}}
  end
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Reuse legacy playlist preflight + expand client-side only
  • ➕ Less backend surface area (no new run resources/events).
  • ➕ Fewer DB migrations and worker changes.
  • ➖ Cannot provide durable recovery/retry/cancel per occurrence.
  • ➖ Client would still lack authoritative per-item lifecycle and duplicate evidence.
  • ➖ Worse parity between WebUI and extension due to runtime differences.
2. Model playlist run as a single ingest job with embedded child state
  • ➕ Fewer new API endpoints (one run job + job events).
  • ➕ Potentially simpler server routing.
  • ➖ Harder to page/query items, reconcile retries, and expose stable per-occurrence URLs.
  • ➖ More coupling to job event schema; riskier evolution for UI needs.
3. Use WebSocket-only transport for run events
  • ➕ Lower polling overhead; richer bi-directional control channel.
  • ➖ More operational complexity than SSE/poll; harder extension compatibility.
  • ➖ Requires robust reconnect semantics anyway; SSE+poll fallback already covers failures.

Recommendation: Current approach (explicit v2 playlist resources + durable store + SSE/poll eventing + IndexedDB recovery) is the best fit for the stated UX goal: users must review/track each video occurrence with robust refresh/restart recovery. The additional API surface is justified by pagination, immutable snapshots, and stable per-item lifecycle contracts. Alternatives either sacrifice durability/observability or increase operational complexity without eliminating the need for recovery semantics.

Files changed (114) +79322 / -5062

Enhancement (42) +21984 / -1893
option.jsonAdd playlist inspection/preflight i18n strings +65/-1

Add playlist inspection/preflight i18n strings

• Introduces new Quick Ingest copy for playlist inspection, filtering, selection, availability/duplicate statuses, and updates dropzone aria text.

apps/packages/ui/src/assets/locale/en/option.json

AddContentStep.tsxIntegrate playlist inspection into Add Content step +648/-330

Integrate playlist inspection into Add Content step

• Adds playlist candidate detection, inspection lifecycle UI, selection batching, and virtualization to efficiently review large playlists.

apps/packages/ui/src/components/Common/QuickIngest/AddContentStep.tsx

BatchMetadataPanel.tsxAlign batch metadata panel with playlist-aware wizard state +11/-3

Align batch metadata panel with playlist-aware wizard state

• Small updates to support new wizard state fields used by playlist runs and review flows.

apps/packages/ui/src/components/Common/QuickIngest/BatchMetadataPanel.tsx

FloatingProgressWidget.tsxShow durable run outcomes in floating progress summary +152/-29

Show durable run outcomes in floating progress summary

• Extends completion summary to use playlist run terminal outcomes and supports formatted/i18n-ready messages.

apps/packages/ui/src/components/Common/QuickIngest/FloatingProgressWidget.tsx

IngestWizardContext.tsxAdd playlist run/review state and transitions to wizard context +605/-77

Add playlist run/review state and transitions to wizard context

• Introduces playlist review state, pending run requests, processing blocks (review required/expired), and new actions to update/cancel occurrences.

apps/packages/ui/src/components/Common/QuickIngest/IngestWizardContext.tsx

ItemMetadataTable.tsxAdd playlist review metadata table with virtualization and focus management +481/-80

Add playlist review metadata table with virtualization and focus management

• Adds playlist-mode rendering, per-occurrence metadata patch inputs, duplicate-policy controls, and virtualized rows with keyboard/focus handling.

apps/packages/ui/src/components/Common/QuickIngest/ItemMetadataTable.tsx

PlaylistPreflightPanel.tsxReplace legacy preflight panel with v2 inspection UI +671/-112

Replace legacy preflight panel with v2 inspection UI

• Implements a virtualized playlist items panel with filtering (new/duplicates/unavailable), batch selection actions, and robust status/error messaging.

apps/packages/ui/src/components/Common/QuickIngest/PlaylistPreflightPanel.tsx

ProcessingStep.tsxTrack per-occurrence lifecycle during processing with retry/cancel hooks +609/-168

Track per-occurrence lifecycle during processing with retry/cancel hooks

• Updates the processing step to use server lifecycle states/outcomes, virtualizes long lists, and supports explicit submission/retry behavior.

apps/packages/ui/src/components/Common/QuickIngest/ProcessingStep.tsx

FileDropZone.tsxAdjust file dropzone behavior for playlist-aware flow +40/-57

Adjust file dropzone behavior for playlist-aware flow

• Updates validation and interaction details to work cleanly alongside playlist inspection and explicit submission semantics.

apps/packages/ui/src/components/Common/QuickIngest/QueueTab/FileDropZone.tsx

ReviewStep.tsxAdd playlist-specific review controls and guards +411/-49

Add playlist-specific review controls and guards

• Extends the review step to handle playlist item review requirements, metadata patches, and per-occurrence duplicate policies.

apps/packages/ui/src/components/Common/QuickIngest/ReviewStep.tsx

WizardResultsStep.tsxShow playlist run outcomes and navigation affordances +393/-53

Show playlist run outcomes and navigation affordances

• Updates results rendering to reflect durable run terminal outcomes, retry eligibility, and per-item statuses.

apps/packages/ui/src/components/Common/QuickIngest/WizardResultsStep.tsx

file-bytes.tsUtility to read file bytes for explicit submission path +27/-0

Utility to read file bytes for explicit submission path

• Adds helper to read file contents needed for explicit submission/runtime handoff logic.

apps/packages/ui/src/components/Common/QuickIngest/file-bytes.ts

types.tsAdd playlist run lifecycle types and source references +98/-2

Add playlist run lifecycle types and source references

• Introduces sourceRef/materialization identity, playlist review state, and durable lifecycle/outcome fields used across UI and runtime.

apps/packages/ui/src/components/Common/QuickIngest/types.ts

usePlaylistInspection.tsNew hook for playlist inspection queueing/polling/selection +869/-0

New hook for playlist inspection queueing/polling/selection

• Implements client-side inspection orchestration (concurrency limits, polling, cursor paging), default selection, and duplicate detection across queue/session.

apps/packages/ui/src/components/Common/QuickIngest/usePlaylistInspection.ts

QuickIngestWizardModal.tsxWire explicit playlist submission + recovery into wizard modal +2510/-525

Wire explicit playlist submission + recovery into wizard modal

• Adds run creation/retry/cancel messaging, review-required recovery, and authority handoff hooks for WebUI vs extension runtimes.

apps/packages/ui/src/components/Common/QuickIngestWizardModal.tsx

QuickIngestButton.tsxMinor Quick Ingest button updates +3/-3

Minor Quick Ingest button updates

• Small adjustments to keep Quick Ingest entrypoint aligned with updated wizard behavior.

apps/packages/ui/src/components/Layouts/QuickIngestButton.tsx

form.tsxUpdate sidepanel chat form quick-ingest integration +6/-84

Update sidepanel chat form quick-ingest integration

• Refactors the form integration to support quick ingest open events compatible with playlist inspection seeding.

apps/packages/ui/src/components/Sidepanel/Chat/form.tsx

useSidepanelQuickIngestOpen.tsAdd hook to open Quick Ingest from sidepanel with playlist context +140/-0

Add hook to open Quick Ingest from sidepanel with playlist context

• Introduces shared logic to construct and dispatch quick ingest open details for playlists and other sources.

apps/packages/ui/src/components/Sidepanel/Chat/useSidepanelQuickIngestOpen.ts

background.tsExtend extension background to support quick ingest playlist runtime +571/-15

Extend extension background to support quick ingest playlist runtime

• Adds message handling and runtime support required for playlist run persistence, reattach, and progress updates.

apps/packages/ui/src/entries/background.ts

quick-ingest-session-runtime.tsAdd durable quick ingest session runtime with replay/reattach hooks +2160/-19

Add durable quick ingest session runtime with replay/reattach hooks

• Implements a richer runtime session model (start/run/retrying/terminal/review) to support reload-safe playlist run processing.

apps/packages/ui/src/entries/shared/quick-ingest-session-runtime.ts

background-proxy.tsExtend background proxy for new quick ingest runtime messages +24/-3

Extend background proxy for new quick ingest runtime messages

• Adds/adjusts message forwarding needed for playlist session persistence and reattach operations.

apps/packages/ui/src/services/background-proxy.ts

media.tsMinor domain adjustments for playlist ingest integration +284/-1

Minor domain adjustments for playlist ingest integration

• Small changes to media domain methods used by playlist run creation and processing.

apps/packages/ui/src/services/tldw/domains/media.ts

openapi-guard.tsOpenAPI guard additions for playlist paths +10/-0

OpenAPI guard additions for playlist paths

• Adds guard coverage for new playlist-ingest endpoints when validating server OpenAPI shapes.

apps/packages/ui/src/services/tldw/openapi-guard.ts

playlist-ingest.tsNew v2 playlist ingest client (preflight/materialize/run/events) +2202/-0

New v2 playlist ingest client (preflight/materialize/run/events)

• Implements typed client wrappers for preflights, item paging, materialization, run creation/cancel/retry, SSE streaming, and public error mapping.

apps/packages/ui/src/services/tldw/playlist-ingest.ts

quick-ingest-batch.tsAdd explicit durable run submission path to quick ingest batch service +1489/-48

Add explicit durable run submission path to quick ingest batch service

• Integrates playlist run creation, pending chunk submission, retry and per-occurrence cancellation, and review-required recovery handling.

apps/packages/ui/src/services/tldw/quick-ingest-batch.ts

quick-ingest-session-reattach.tsReattach to playlist runs via poll/SSE and derive UI snapshot +257/-1

Reattach to playlist runs via poll/SSE and derive UI snapshot

• Adds durable run reattach logic, lifecycle derivation from run items, and signature-based snapshot updates.

apps/packages/ui/src/services/tldw/quick-ingest-session-reattach.ts

server-capabilities.tsDetect playlist ingest v2 contract via docs-info + OpenAPI +42/-1

Detect playlist ingest v2 contract via docs-info + OpenAPI

• Adds endpoint-set detection for v2 playlist ingest and a numeric docs-info contract-version gate; bumps cached key version.

apps/packages/ui/src/services/tldw/server-capabilities.ts

quick-ingest-session.tsPersist quick ingest session via IndexedDB and add submission leases +1316/-141

Persist quick ingest session via IndexedDB and add submission leases

• Integrates Dexie storage, adds submission ownership/authority bookkeeping, and exposes handoff/lease operations to coordinate reload-safe submission.

apps/packages/ui/src/store/quick-ingest-session.ts

quick-ingest-open.tsExtend quick ingest open detail types for playlist preflight seeding +27/-8

Extend quick ingest open detail types for playlist preflight seeding

• Adds type guards and structures to represent playlist-related open events (WebUI/extension entrypoints).

apps/packages/ui/src/utils/quick-ingest-open.ts

__init__.pyRegister playlist ingest routes under media endpoints +1/-0

Register playlist ingest routes under media endpoints

• Ensures new playlist_ingest router is included in the media endpoint package exports.

tldw_Server_API/app/api/v1/endpoints/media/init.py

ingest_jobs.pyHarden ingest job routing for run-bound staging and limits +1685/-16

Harden ingest job routing for run-bound staging and limits

• Adds HMAC/identity helpers and integrates playlist run staging cleanup/boundaries to support explicit submission with durable runs.

tldw_Server_API/app/api/v1/endpoints/media/ingest_jobs.py

playlist_ingest.pyNew v2 playlist ingest endpoints (preflight/materialize/run/events) +711/-0

New v2 playlist ingest endpoints (preflight/materialize/run/events)

• Introduces FastAPI routes for playlist preflights, item paging, materializations, run creation/status/items, cancel/retry, and SSE event streaming with safe error mapping.

tldw_Server_API/app/api/v1/endpoints/media/playlist_ingest.py

media_playlist_ingest.pyAdd v2 playlist ingest schemas and input bounds +820/-0

Add v2 playlist ingest schemas and input bounds

• Defines the shared request/response models, lifecycle/outcome enums, duplicate policies, and bounded JSON/metadata patch validation.

tldw_Server_API/app/api/v1/schemas/media_playlist_ingest.py

Collections_DB.pyCollection DB adjustments for playlist run reconciliation +552/-10

Collection DB adjustments for playlist run reconciliation

• Minor changes to collection persistence/query behaviors needed by run creation and per-item planning flows.

tldw_Server_API/app/core/DB_Management/Collections_DB.py

api.pyExtend media DB API for playlist dedupe lookups +29/-1

Extend media DB API for playlist dedupe lookups

• Small changes to support lookup-by-URLs flows used to compute duplicate evidence for playlist items.

tldw_Server_API/app/core/DB_Management/media_db/api.py

media_database_impl.pyMedia DB implementation hook for new lookup behavior +4/-0

Media DB implementation hook for new lookup behavior

• Adds small wiring to support new playlist ingest service usage patterns.

tldw_Server_API/app/core/DB_Management/media_db/media_database_impl.py

media_item_update_ops.pySupport playlist metadata patch updates in media item ops +143/-31

Support playlist metadata patch updates in media item ops

• Extends update operations to safely apply reviewed metadata patches and handle new edge cases.

tldw_Server_API/app/core/DB_Management/media_db/runtime/media_item_update_ops.py

query_ops.pyQuery helpers used by playlist ingest store/service +16/-0

Query helpers used by playlist ingest store/service

• Adds supporting query ops used by new playlist ingest reconciliation paths.

tldw_Server_API/app/core/DB_Management/media_db/runtime/query_ops.py

playlist_ingest_service.pyService layer for playlist preflight/materialization/runs +1418/-0

Service layer for playlist preflight/materialization/runs

• Implements owner-scoped validation, duplicate evidence/review-required gating, run reconciliation, retry/cancel behavior, and safe error code mapping.

tldw_Server_API/app/core/Ingestion_Media_Processing/Video/playlist_ingest_service.py

input_sourcing.pyAdjust input sourcing to support explicit submission and staging +17/-21

Adjust input sourcing to support explicit submission and staging

• Updates ingestion sourcing utilities to integrate with run-bound staging and explicit submission identities.

tldw_Server_API/app/core/Ingestion_Media_Processing/input_sourcing.py

exceptions.pyAdd typed exceptions used by playlist ingest endpoints +9/-0

Add typed exceptions used by playlist ingest endpoints

• Adds new exception types referenced by updated ingest job handling and trust-boundary validation paths.

tldw_Server_API/app/core/exceptions.py

media_ingest_jobs_worker.pyWorker support for playlist preflight jobs and cleanup +458/-4

Worker support for playlist preflight jobs and cleanup

• Adds playlist_preflight job execution, lease identity validation, cancellation checks, safe blocking snapshots, and expired-resource cleanup logic.

tldw_Server_API/app/services/media_ingest_jobs_worker.py

Bug fix (2) +122 / -15
media_lookup_repository.pyHarden URL lookup repository for dedupe normalization tests +54/-3

Harden URL lookup repository for dedupe normalization tests

• Adds minor enhancements needed for playlist dedupe evidence and new normalization coverage.

tldw_Server_API/app/core/DB_Management/media_db/repositories/media_lookup_repository.py

playlist_preflight.pyHarden playlist preflight normalization for availability and bounds +68/-12

Harden playlist preflight normalization for availability and bounds

• Adds explicit availability handling, detects yt-dlp unavailable placeholders, bounds identity/display text sizes, and introduces typed safe preflight errors.

tldw_Server_API/app/core/Ingestion_Media_Processing/Video/playlist_preflight.py

Tests (48) +47555 / -3052
AddContentStep.playlist-ingest.test.tsxComprehensive tests for playlist inspection in Add Content +2330/-0

Comprehensive tests for playlist inspection in Add Content

• Adds a focused test suite covering playlist detection, inspection lifecycle, selection defaults, filtering, and error paths.

apps/packages/ui/src/components/Common/QuickIngest/tests/AddContentStep.playlist-ingest.test.tsx

FileDropZone.acceptance.test.tsxUpdate dropzone acceptance expectations +17/-3

Update dropzone acceptance expectations

• Adjusts dropzone tests to match updated aria/behavior used by the refreshed Quick Ingest flow.

apps/packages/ui/src/components/Common/QuickIngest/tests/FileDropZone.acceptance.test.tsx

FloatingProgressWidget.test.tsxTest floating widget with playlist terminal outcomes +164/-7

Test floating widget with playlist terminal outcomes

• Updates tests to ensure summary counts and messaging reflect playlist run outcomes and mixed result states.

apps/packages/ui/src/components/Common/QuickIngest/tests/FloatingProgressWidget.test.tsx

IngestWizardContext.test.tsxExtend wizard context tests for playlist review/run state +1069/-10

Extend wizard context tests for playlist review/run state

• Adds coverage for new actions/state fields, including review-required transitions and per-occurrence cancellation semantics.

apps/packages/ui/src/components/Common/QuickIngest/tests/IngestWizardContext.test.tsx

PlaylistPreflightPanel.test.tsxUpdate playlist preflight panel tests for v2 UI +99/-100

Update playlist preflight panel tests for v2 UI

• Replaces/updates tests to validate new virtualized list UI, status handling, selection actions, and filtering behavior.

apps/packages/ui/src/components/Common/QuickIngest/tests/PlaylistPreflightPanel.test.tsx

PlaylistPreflightPanel.virtualization.test.tsxAdd virtualization regression tests for playlist list rendering +639/-0

Add virtualization regression tests for playlist list rendering

• Ensures large playlist rendering remains performant and stable with the virtualized panel implementation.

apps/packages/ui/src/components/Common/QuickIngest/tests/PlaylistPreflightPanel.virtualization.test.tsx

QuickIngestWizardModal.integration.test.tsxUpdate wizard modal integration tests for playlist v2 flow +2207/-26

Update wizard modal integration tests for playlist v2 flow

• Extends integration coverage to include playlist inspection/materialization and explicit run submission handoffs.

apps/packages/ui/src/components/Common/QuickIngest/tests/QuickIngestWizardModal.integration.test.tsx

QuickIngestWizardModal.session.test.tsxExpand session persistence/recovery tests for playlist runs +6565/-1245

Expand session persistence/recovery tests for playlist runs

• Adds extensive coverage for IndexedDB-backed session recovery, submission lease behavior, and reattach flows across reloads.

apps/packages/ui/src/components/Common/QuickIngest/tests/QuickIngestWizardModal.session.test.tsx

WizardResultsStep.navigation.test.tsxAdd results-step navigation tests for durable runs +317/-0

Add results-step navigation tests for durable runs

• Validates navigation and retry entrypoints from results for durable playlist runs.

apps/packages/ui/src/components/Common/QuickIngest/tests/WizardResultsStep.navigation.test.tsx

ControlRow.chat-handoff.test.tsxUpdate chat handoff tests for quick ingest open detail +39/-8

Update chat handoff tests for quick ingest open detail

• Adjusts sidepanel chat handoff tests to validate updated quick ingest open semantics used by the playlist flow.

apps/packages/ui/src/components/Sidepanel/Chat/tests/ControlRow.chat-handoff.test.tsx

form.queue.contract.test.tsxContract test adjustments for form queue integration +2/-2

Contract test adjustments for form queue integration

• Minor expectation updates to reflect new quick ingest open payloads.

apps/packages/ui/src/components/Sidepanel/Chat/tests/form.queue.contract.test.tsx

useSidepanelQuickIngestOpen.test.tsxNew tests for sidepanel quick ingest open behavior +249/-0

New tests for sidepanel quick ingest open behavior

• Adds coverage for opening Quick Ingest from sidepanel contexts, including playlist-preflight seed details.

apps/packages/ui/src/components/Sidepanel/Chat/tests/useSidepanelQuickIngestOpen.test.tsx

background.web-clipper.test.tsUpdate background integration tests for playlist quick ingest +1203/-1

Update background integration tests for playlist quick ingest

• Adjusts extension background test coverage to include playlist reload/reattach and new runtime messages.

apps/packages/ui/src/entries/tests/background.web-clipper.test.ts

quick-ingest-playlist-extension.integration.test.tsNew extension integration test for playlist quick ingest +100/-0

New extension integration test for playlist quick ingest

• Validates end-to-end extension flow for playlist inspection and durable processing integration.

apps/packages/ui/src/entries/shared/tests/quick-ingest-playlist-extension.integration.test.ts

quick-ingest-session-runtime.test.tsUpdate runtime tests for durable run sessions +3574/-8

Update runtime tests for durable run sessions

• Extends runtime state-machine tests to cover new compact run session formats, cancellation, retries, and event replay.

apps/packages/ui/src/entries/shared/tests/quick-ingest-session-runtime.test.ts

quick-ingest-playlist-locale.test.tsAdd locale coverage for playlist quick ingest strings +122/-0

Add locale coverage for playlist quick ingest strings

• Ensures new playlist-related i18n keys are present and resolvable.

apps/packages/ui/src/i18n/tests/quick-ingest-playlist-locale.test.ts

background-proxy.test.tsUpdate background proxy tests +30/-1

Update background proxy tests

• Minor updates to align proxy behavior with new quick ingest runtime messages.

apps/packages/ui/src/services/tests/background-proxy.test.ts

quick-ingest-batch.test.tsMajor updates to quick ingest batch tests for explicit playlist runs +5861/-1473

Major updates to quick ingest batch tests for explicit playlist runs

• Adds extensive coverage for run creation, chunk submission, retry/cancel behavior, and failure modes under the new playlist contract.

apps/packages/ui/src/services/tests/quick-ingest-batch.test.ts

quick-ingest-session-reattach.test.tsAdd tests for run reattach across poll/SSE transports +605/-0

Add tests for run reattach across poll/SSE transports

• Validates snapshot derivation and lifecycle computation when reconnecting to a durable playlist run.

apps/packages/ui/src/services/tests/quick-ingest-session-reattach.test.ts

server-capabilities.test.tsAdd capability tests for playlist ingest v2 contract detection +95/-0

Add capability tests for playlist ingest v2 contract detection

• Ensures client capability computation recognizes v2 playlist endpoints and docs-info numeric contract version gate.

apps/packages/ui/src/services/tests/server-capabilities.test.ts

tldw-api-client.media-ingest.test.tsUpdate API client tests for new media ingest paths +887/-1

Update API client tests for new media ingest paths

• Adjusts client contract tests to account for added playlist ingest v2 calls.

apps/packages/ui/src/services/tests/tldw-api-client.media-ingest.test.ts

playlist-ingest.test.tsNew client tests for playlist-ingest service layer +1303/-0

New client tests for playlist-ingest service layer

• Adds comprehensive unit tests for playlist v2 API wrappers, pagination loading, error mapping, and event streaming.

apps/packages/ui/src/services/tldw/tests/playlist-ingest.test.ts

quick-ingest-indexeddb.test.tsNew tests for IndexedDB quick ingest session storage +1021/-0

New tests for IndexedDB quick ingest session storage

• Validates Dexie persistence behavior, retention cleanup, lease acquisition/renewal, and envelope validation.

apps/packages/ui/src/store/tests/quick-ingest-indexeddb.test.ts

quick-ingest-session.test.tsUpdate quick ingest session store tests for persistence/authority +1608/-26

Update quick ingest session store tests for persistence/authority

• Adds coverage for new persistence status, submission owner/authority, and handoff semantics.

apps/packages/ui/src/store/tests/quick-ingest-session.test.ts

quick-ingest-open.test.tsAdd tests for quick ingest open detail parsing (playlist) +27/-0

Add tests for quick ingest open detail parsing (playlist)

• Validates new open-detail discriminators used to seed playlist inspection flows.

apps/packages/ui/src/utils/tests/quick-ingest-open.test.ts

quick-ingest-playlist-v2.tsAdd E2E fixture for playlist v2 quick ingest +435/-0

Add E2E fixture for playlist v2 quick ingest

• Provides deterministic playlist fixture data used by Playwright workflows.

apps/tldw-frontend/e2e/fixtures/quick-ingest-playlist-v2.ts

quick-ingest-playlist.spec.tsAdd Playwright workflow for playlist inspection and processing +163/-0

Add Playwright workflow for playlist inspection and processing

• Adds a browser workflow that validates playlist inspection, selection, and run tracking behaviors.

apps/tldw-frontend/e2e/workflows/quick-ingest-playlist.spec.ts

test_collections_postgres_integration.pyAdjust collections Postgres integration tests +139/-1

Adjust collections Postgres integration tests

• Minor updates to keep collections integration tests compatible with new playlist-related collection planning behavior.

tldw_Server_API/tests/Collections/test_collections_postgres_integration.py

test_conference_media_collections.pyAdjust conference collections tests for playlist-aware behavior +359/-1

Adjust conference collections tests for playlist-aware behavior

• Small expectation updates to reflect collection planning/reconciliation changes used by playlist runs.

tldw_Server_API/tests/Collections/test_conference_media_collections.py

test_docs_info_capabilities.pyTest docs-info capability gating for playlist ingest v2 +56/-35

Test docs-info capability gating for playlist ingest v2

• Adds assertions for the new capability flags and mediaPlaylistIngestContractVersion behavior.

tldw_Server_API/tests/Config/test_docs_info_capabilities.py

test_media_db_media_item_update_ops.pyTest media item update ops with reviewed patches +220/-14

Test media item update ops with reviewed patches

• Extends unit tests to cover new patch validation and update edge cases used by playlist runs.

tldw_Server_API/tests/DB_Management/test_media_db_media_item_update_ops.py

test_jobs_fault_injection_sqlite.pyUpdate jobs fault injection tests for new tables +12/-9

Update jobs fault injection tests for new tables

• Adjusts fault injection coverage to account for playlist ingest tables/migrations in the SQLite jobs DB.

tldw_Server_API/tests/Jobs/test_jobs_fault_injection_sqlite.py

test_jobs_manager.pyUpdate job manager tests for playlist admission/binding +107/-17

Update job manager tests for playlist admission/binding

• Adds/updates tests to cover new playlist job paths and manager behavior.

tldw_Server_API/tests/Jobs/test_jobs_manager.py

test_jobs_migrations_postgres.pyValidate Postgres migrations include playlist ingest schema +221/-7

Validate Postgres migrations include playlist ingest schema

• Extends migration tests to ensure new tables/indexes are created in PostgreSQL jobs DB.

tldw_Server_API/tests/Jobs/test_jobs_migrations_postgres.py

test_jobs_migrations_sqlite.pyValidate SQLite migrations include playlist ingest schema +460/-9

Validate SQLite migrations include playlist ingest schema

• Extends migration tests to ensure new tables/indexes are created in SQLite jobs DB.

tldw_Server_API/tests/Jobs/test_jobs_migrations_sqlite.py

test_jobs_quotas_postgres.pyUpdate Postgres quota tests for playlist workload +56/-10

Update Postgres quota tests for playlist workload

• Adjusts quota/admission tests for additional playlist-related job/resource activity.

tldw_Server_API/tests/Jobs/test_jobs_quotas_postgres.py

test_jobs_quotas_sqlite.pyUpdate SQLite quota tests for playlist workload +14/-10

Update SQLite quota tests for playlist workload

• Adjusts quota/admission tests for additional playlist-related job/resource activity.

tldw_Server_API/tests/Jobs/test_jobs_quotas_sqlite.py

test_media_upload_cleanup.pyUpdate media upload cleanup tests for run-bound staging +19/-2

Update media upload cleanup tests for run-bound staging

• Adds coverage for new staging cleanup boundaries used by explicit submission.

tldw_Server_API/tests/Media/unit/test_media_upload_cleanup.py

test_dedupe_url_normalization.pyAdd dedupe URL normalization tests for playlist inputs +84/-0

Add dedupe URL normalization tests for playlist inputs

• Introduces tests that validate URL normalization/dedupe behavior used for playlist duplicate evidence.

tldw_Server_API/tests/MediaDB2/test_dedupe_url_normalization.py

test_playlist_ingest_run_workflow.pyAdd integration test for playlist run workflow +343/-0

Add integration test for playlist run workflow

• Covers end-to-end run creation, job binding, progress updates, and terminal outcome behavior for playlist ingestion.

tldw_Server_API/tests/MediaIngestion_NEW/integration/test_playlist_ingest_run_workflow.py

test_playlist_ingest_store_postgres.pyAdd Postgres integration tests for playlist ingest store +1377/-0

Add Postgres integration tests for playlist ingest store

• Validates store persistence semantics, paging, concurrency, and cleanup behaviors against PostgreSQL backend.

tldw_Server_API/tests/MediaIngestion_NEW/integration/test_playlist_ingest_store_postgres.py

test_playlist_ingest_properties.pyAdd property-based tests for playlist ingest invariants +276/-0

Add property-based tests for playlist ingest invariants

• Adds property tests for selection, paging, snapshot immutability, and state transition invariants.

tldw_Server_API/tests/MediaIngestion_NEW/property/test_playlist_ingest_properties.py

test_media_ingest_jobs_endpoint.pyUpdate ingest jobs endpoint unit tests for playlist staging integration +3070/-5

Update ingest jobs endpoint unit tests for playlist staging integration

• Adjusts/extends endpoint tests to cover new staging/identity/cleanup behavior used by playlist runs.

tldw_Server_API/tests/MediaIngestion_NEW/unit/test_media_ingest_jobs_endpoint.py

test_media_ingest_jobs_worker.pyUpdate worker unit tests for playlist preflight job handling +1028/-11

Update worker unit tests for playlist preflight job handling

• Adds unit coverage for lease validation, snapshot finalization, cancellation, and safe error codes for playlist preflight jobs.

tldw_Server_API/tests/MediaIngestion_NEW/unit/test_media_ingest_jobs_worker.py

test_playlist_ingest_endpoint.pyAdd unit tests for playlist ingest v2 endpoints +1374/-0

Add unit tests for playlist ingest v2 endpoints

• Introduces extensive endpoint tests for preflight/materialization/run creation, paging, cancel/retry, and event streaming error mapping.

tldw_Server_API/tests/MediaIngestion_NEW/unit/test_playlist_ingest_endpoint.py

test_playlist_ingest_service.pyAdd unit tests for playlist ingest service layer +2029/-0

Add unit tests for playlist ingest service layer

• Covers URL classification, review-required gating, duplicate evidence behavior, run reconciliation, retry/cancel semantics, and safe error reporting.

tldw_Server_API/tests/MediaIngestion_NEW/unit/test_playlist_ingest_service.py

test_playlist_ingest_store.pyAdd unit tests for playlist ingest persistence store +5032/-0

Add unit tests for playlist ingest persistence store

• Validates schema contracts, cursor paging, compare-and-set concurrency, resource expiry/cleanup, and invariant enforcement in the store layer.

tldw_Server_API/tests/MediaIngestion_NEW/unit/test_playlist_ingest_store.py

test_playlist_preflight.pyUpdate playlist preflight unit tests for availability handling +548/-10

Update playlist preflight unit tests for availability handling

• Adds coverage for unavailable placeholders, bounded identity/display fields, and new error codes during normalization/extraction.

tldw_Server_API/tests/MediaIngestion_NEW/unit/test_playlist_preflight.py

Documentation (9) +2765 / -0
2026-07-12-youtube-playlist-ingest-backend.mdBackend plan for playlist ingest v2 +531/-0

Backend plan for playlist ingest v2

• Adds a detailed backend implementation plan covering v2 endpoints, persistence, job binding, capacity limits, and recovery semantics.

Docs/superpowers/plans/2026-07-12-youtube-playlist-ingest-backend.md

2026-07-12-youtube-playlist-ingest-shared-frontend.mdShared frontend plan for per-item playlist ingest +495/-0

Shared frontend plan for per-item playlist ingest

• Documents WebUI/extension UX flows, capability gating, virtualization strategy, and state transitions for playlist review and processing.

Docs/superpowers/plans/2026-07-12-youtube-playlist-ingest-shared-frontend.md

2026-07-14-quick-ingest-explicit-submission-command.mdPlan explicit submission command for ingest runs +734/-0

Plan explicit submission command for ingest runs

• Specifies the explicit run-creation command and handoff semantics to avoid ambiguous client/server submission during reloads.

Docs/superpowers/plans/2026-07-14-quick-ingest-explicit-submission-command.md

2026-07-12-youtube-playlist-per-item-ingest-design.mdDesign spec for per-video playlist ingest visibility +543/-0

Design spec for per-video playlist ingest visibility

• Defines the user-visible behavior (per-video selection and lifecycle tracking) and the v2 contract expectations across surfaces.

Docs/superpowers/specs/2026-07-12-youtube-playlist-per-item-ingest-design.md

task-12109 - Design-per-video-YouTube-playlist-ingestion-visibility-for-WebUI-and-extension.mdAdd TASK-12109 backlog record +61/-0

Add TASK-12109 backlog record

• Introduces backlog tracking for the per-video playlist ingest visibility design work.

backlog/tasks/task-12109 - Design-per-video-YouTube-playlist-ingestion-visibility-for-WebUI-and-extension.md

task-12110 - Plan-backend-contracts-for-per-video-YouTube-playlist-ingestion.mdAdd TASK-12110 backlog record +61/-0

Add TASK-12110 backlog record

• Adds backlog tracking for backend contract planning and implementation milestones.

backlog/tasks/task-12110 - Plan-backend-contracts-for-per-video-YouTube-playlist-ingestion.md

task-12111 - Plan-shared-WebUI-and-extension-per-video-playlist-ingestion.mdAdd TASK-12111 backlog record +63/-0

Add TASK-12111 backlog record

• Adds backlog tracking for shared frontend planning across WebUI and extension.

backlog/tasks/task-12111 - Plan-shared-WebUI-and-extension-per-video-playlist-ingestion.md

task-12112 - Implement-per-video-YouTube-playlist-ingestion-backend.mdAdd TASK-12112 backlog record +98/-0

Add TASK-12112 backlog record

• Tracks backend implementation work for playlist ingest v2 endpoints, persistence, and worker/job semantics.

backlog/tasks/task-12112 - Implement-per-video-YouTube-playlist-ingestion-backend.md

task-12113 - Implement-shared-WebUI-and-extension-per-video-playlist-ingestion.mdAdd TASK-12113 backlog record +179/-0

Add TASK-12113 backlog record

• Tracks shared frontend implementation for per-video playlist ingest inspection and lifecycle UI.

backlog/tasks/task-12113 - Implement-shared-WebUI-and-extension-per-video-playlist-ingestion.md

Other (13) +6896 / -102
quick-ingest.tsAdd Dexie-backed storage for Quick Ingest sessions and leases +576/-0

Add Dexie-backed storage for Quick Ingest sessions and leases

• Implements IndexedDB persistence for wizard sessions, retention cleanup, and submission lease/authority handoff helpers.

apps/packages/ui/src/db/dexie/quick-ingest.ts

schema.tsExtend Dexie schema for quick ingest sessions +32/-1

Extend Dexie schema for quick ingest sessions

• Adds schema entries/migrations required to store Quick Ingest session records in IndexedDB.

apps/packages/ui/src/db/dexie/schema.ts

types.tsAdd Dexie types for quick ingest session records +20/-0

Add Dexie types for quick ingest session records

• Introduces typed records used by the IndexedDB quick ingest storage layer.

apps/packages/ui/src/db/dexie/types.ts

vitest.setup.tsAdjust vitest setup for playlist-related tests +3/-3

Adjust vitest setup for playlist-related tests

• Minor updates to support the expanded test suites and runtime hooks used by quick ingest playlist flows.

apps/tldw-frontend/vitest.setup.ts

config_info.pyExpose playlist ingest v2 contract capability gate in docs-info +99/-71

Expose playlist ingest v2 contract capability gate in docs-info

• Adds granular capability flags and a numeric mediaPlaylistIngestContractVersion gate to fail-closed on older servers; also refactors formatting.

tldw_Server_API/app/api/v1/endpoints/config_info.py

playlist_ingest_staging.pyCanonical staging directory helpers for run-bound uploads +92/-0

Canonical staging directory helpers for run-bound uploads

• Adds validated staging path derivation and bounded cleanup helpers to avoid temp-root scanning and alias traversal.

tldw_Server_API/app/core/Ingestion_Media_Processing/Video/playlist_ingest_staging.py

playlist_ingest_store.pyDurable playlist ingest persistence store (owner-scoped) +5174/-0

<b...

@qodo-code-review

qodo-code-review Bot commented Jul 15, 2026

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 📜 Skill insights (0)

Context used
✅ Compliance rules (platform): 74 rules

Grey Divider


Action required

1. PlaylistPreflightBusyError outside core exceptions ✓ Resolved 📘 Rule violation ⌂ Architecture
Description
The PR introduces multiple custom exception classes in
tldw_Server_API/app/core/Ingestion_Media_Processing/Video/playlist_ingest_service.py instead of
centralizing them in tldw_Server_API/app/core/exceptions.py. This violates the
exception-centralization requirement and makes exception reuse/handling inconsistent across
features.
Code

tldw_Server_API/app/core/Ingestion_Media_Processing/Video/playlist_ingest_service.py[R77-123]

+class PlaylistPreflightBusyError(RuntimeError):
+    """Raised when no transactional preflight reservation is available."""
+
+
+class InvalidPlaylistUrlError(ValueError):
+    """Raised for input outside the trusted YouTube playlist boundary."""
+
+
+class PlaylistPreflightUnavailableError(RuntimeError):
+    """Raised when the resource cannot be durably bound to an internal job."""
+
+
+class PlaylistPreflightIncompleteError(RuntimeError):
+    """Raised when materialization is requested before a complete snapshot."""
+
+
+class PlaylistSelectionError(ValueError):
+    """Raised for an invalid server occurrence selection."""
+
+
+class PlaylistRunValidationError(ValueError):
+    """Raised with a stable code when a run request is invalid."""
+
+
+class PlaylistRunPendingError(PlaylistIngestConflictError):
+    """Raised when an existing run still has an ambiguous zero-job action."""
+
+    def __init__(self, run_id: str) -> None:
+        self.run_id = str(run_id)
+        super().__init__("duplicate_action_pending")
+
+
+class PlaylistRunStatusUnavailableError(PlaylistIngestConflictError):
+    """Raised when retry reconciliation cannot establish current owner media state."""
+
+
+class PlaylistPreflightRequiredError(PlaylistRunValidationError):
+    """Raised when a direct URL must first use playlist preflight."""
+
+
+class ReviewRequiredError(RuntimeError):
+    """Raised before persistence when refreshed duplicate evidence changed Review."""
+
+    def __init__(self, items: Sequence[ReviewRequiredItem]) -> None:
+        self.items = tuple(items)
+        super().__init__("review_required")
+
Evidence
PR Compliance ID 224217 requires new custom exceptions to be centralized in
/app/core/exceptions.py. The diff shows several new class ...Error(...) definitions added
directly in playlist_ingest_service.py.

Rule 224217: Centralize custom exceptions in core module
tldw_Server_API/app/core/Ingestion_Media_Processing/Video/playlist_ingest_service.py[77-123]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
New custom exceptions are defined outside `tldw_Server_API/app/core/exceptions.py`, violating the project rule that custom exceptions must be centralized.

## Issue Context
The playlist ingest feature defines domain errors such as `PlaylistPreflightBusyError` and `InvalidPlaylistUrlError` directly in the service module.

## Fix Focus Areas
- tldw_Server_API/app/core/Ingestion_Media_Processing/Video/playlist_ingest_service.py[77-123]
- tldw_Server_API/app/core/exceptions.py[64-72]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. create_playlist_preflight missing docstring ✓ Resolved 📘 Rule violation ✧ Quality
Description
The new endpoint handler create_playlist_preflight is added without a function docstring. This
violates the docstring requirement for modified/new functions and makes the new API behavior harder
to maintain and audit.
Code

tldw_Server_API/app/api/v1/endpoints/media/playlist_ingest.py[R276-285]

+def create_playlist_preflight(
+    request_scope: Request,
+    payload: Any = Body(...),
+    current_user: User = Depends(get_request_user),
+    job_manager: JobManager = Depends(get_job_manager),
+) -> PlaylistPreflightAcceptedResponse:
+    try:
+        request = PlaylistPreflightCreateRequest.model_validate(payload)
+    except ValidationError as exc:
+        raise HTTPException(status_code=422, detail="invalid_playlist_url") from exc
Evidence
PR Compliance ID 224214 requires docstrings for all functions. The new create_playlist_preflight
function is added and its first statement is executable code (try:) rather than a docstring.

Rule 224214: Require docstrings for all modules, classes, and functions
tldw_Server_API/app/api/v1/endpoints/media/playlist_ingest.py[276-295]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
New functions were added without docstrings, violating the requirement that all modules/classes/functions must have docstrings.

## Issue Context
The playlist ingest endpoint module is new and includes multiple helper/endpoint functions; at least one (`create_playlist_preflight`) has no docstring as its first statement.

## Fix Focus Areas
- tldw_Server_API/app/api/v1/endpoints/media/playlist_ingest.py[276-295]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


3. playlist_ingest_store raw SQL ✓ Resolved 📘 Rule violation ⌂ Architecture
Description
The PR adds extensive raw SQL usage in
tldw_Server_API/app/core/Ingestion_Media_Processing/Video/playlist_ingest_store.py, which is
outside /app/core/DB_Management/. It also interpolates table/column identifiers via f-strings
inside SQL, violating the project requirements for DB access layering and parameterized SQL
construction.
Code

tldw_Server_API/app/core/Ingestion_Media_Processing/Video/playlist_ingest_store.py[R751-771]

+        with self._connection(write=True) as db:
+            self._query(
+                db,
+                """
+                INSERT INTO playlist_preflights (
+                    preflight_id, owner_user_id, status, source_url, source_kind,
+                    playlist_id, job_id, created_at, updated_at, expires_at
+                ) VALUES (?, ?, 'pending', ?, ?, ?, ?, ?, ?, ?)
+                """,
+                (
+                    preflight_id,
+                    owner,
+                    str(source_url),
+                    str(source_kind),
+                    playlist_id,
+                    job_id,
+                    self._db_datetime(now),
+                    self._db_datetime(now),
+                    self._db_datetime(expires),
+                ),
+            )
Evidence
PR Compliance ID 224231 requires DB interaction (raw SQL) to be confined to
/app/core/DB_Management/. The diff shows raw SQL INSERT/SELECT statements added in
playlist_ingest_store.py under Ingestion_Media_Processing instead. PR Compliance ID 224220 also
forbids string interpolation in SQL; the diff shows f-string SQL with {parent_table} /
{item_table} interpolation.

Rule 224231: Enforce database access through /app/core/DB_Management/ only
Rule 224220: Use parameterized queries for all database interactions
tldw_Server_API/app/core/Ingestion_Media_Processing/Video/playlist_ingest_store.py[751-771]
tldw_Server_API/app/core/Ingestion_Media_Processing/Video/playlist_ingest_store.py[1431-1458]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Database access and raw SQL were introduced in a module outside `/app/core/DB_Management/`, and some queries use f-string interpolation for SQL identifiers.

## Issue Context
The playlist ingest store is implementing direct SQL reads/writes (e.g., INSERT/SELECT) inside `Ingestion_Media_Processing`, which violates the rule requiring DB access through `/app/core/DB_Management/` only.

## Fix Focus Areas
- tldw_Server_API/app/core/Ingestion_Media_Processing/Video/playlist_ingest_store.py[751-879]
- tldw_Server_API/app/core/Ingestion_Media_Processing/Video/playlist_ingest_store.py[1431-1458]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Remediation recommended

4. Windows skipif lacks issue ✓ Resolved 📘 Rule violation ▣ Testability
Description
A new @pytest.mark.skipif(os.name == "nt" ...) was added without any tracking issue reference.
This violates the rule requiring skips to include an issue/ticket (and discourages disabling tests
without a tracked plan).
Code

tldw_Server_API/tests/MediaIngestion_NEW/unit/test_media_ingest_jobs_endpoint.py[R2638-2639]

+@pytest.mark.skipif(os.name == "nt", reason="backslash is a separator on Windows")
+def test_posix_staging_filename_allows_nonseparator_backslash(tmp_path, monkeypatch):
Evidence
PR Compliance ID 224226 forbids disabling tests to hide failures and requires appropriate
justification for skips, including an issue reference. The new skip marker includes only a free-text
reason and no issue/ticket identifier.

Rule 224226: Do not skip or disable tests to hide failures
tldw_Server_API/tests/MediaIngestion_NEW/unit/test_media_ingest_jobs_endpoint.py[2638-2639]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
A new test skip (`@pytest.mark.skipif`) was introduced without referencing a tracking issue.

## Issue Context
Skips are allowed only with explicit justification and a linked issue/ticket so the skip is not used to mask failures indefinitely.

## Fix Focus Areas
- tldw_Server_API/tests/MediaIngestion_NEW/unit/test_media_ingest_jobs_endpoint.py[2638-2639]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


5. Sidecar downgrades playlist v2 🐞 Bug ≡ Correctness
Description
/config/docs-info computes mediaPlaylistIngestContractVersion=1 when TLDW_WORKERS_SIDECAR_MODE
is enabled because hasMediaIngestWorker is derived from
should_start_inprocess_worker(sidecar_mode=...). The WebUI/extension gates playlist v2 on
mediaPlaylistIngestContractVersion >= 2, so sidecar deployments can incorrectly disable per-video
playlist ingest even if the v2 endpoints and a sidecar worker are present.
Code

tldw_Server_API/app/api/v1/endpoints/config_info.py[R159-171]

+        playlist_ingest_v2_ready = all(
+            caps[name]
+            for name in (
+                "hasMediaPlaylistPreflight",
+                "hasMediaPlaylistPreflightResources",
+                "hasMediaPlaylistIngestRuns",
+                "hasMediaIngestJobs",
+                "hasMediaIngestJobEvents",
+                "hasMediaPlaylistIngestEvents",
+                "hasMediaIngestWorker",
+            )
+        )
+        caps["mediaPlaylistIngestContractVersion"] = 2 if playlist_ingest_v2_ready else 1
Evidence
hasMediaIngestWorker is false in sidecar mode due to should_start_inprocess_worker returning
false, and that capability is required for mediaPlaylistIngestContractVersion to be 2; the
frontend requires contract version >=2 to enable playlist ingest v2.

tldw_Server_API/app/api/v1/endpoints/config_info.py[151-171]
tldw_Server_API/app/services/worker_startup_policy.py[92-103]
apps/packages/ui/src/services/tldw/server-capabilities.ts[420-510]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`mediaPlaylistIngestContractVersion` is computed as 2 only if `hasMediaIngestWorker` is true. But `hasMediaIngestWorker` is currently derived from `should_start_inprocess_worker(..., sidecar_mode=env_flag_enabled("TLDW_WORKERS_SIDECAR_MODE"))`, which *always returns false* in sidecar mode. This makes `/config/docs-info` advertise contract version 1 in sidecar deployments, and the frontend disables playlist ingest v2.

### Issue Context
- `should_start_inprocess_worker` is a startup-policy helper (in-process vs sidecar) and is not the same thing as “a worker exists that can process jobs”.
- The frontend’s `hasMediaPlaylistIngestV2` depends on `mediaPlaylistIngestContractVersion >= 2`, so an incorrect `1` disables the feature.

### Fix Focus Areas
- Decide the semantic meaning of `hasMediaIngestWorker` for docs-info (recommended: represent *effective* worker availability/capability, not “in-process worker will be started”).
- Options:
 - Compute `hasMediaIngestWorker` using `worker_path_enabled(...)` (so sidecar doesn’t auto-false it), and/or
 - Remove `hasMediaIngestWorker` from `playlist_ingest_v2_ready` if it’s not a hard requirement for contract v2, and/or
 - Add a separate capability for “in-process worker enabled” vs “sidecar worker expected”.

#### Target locations
- tldw_Server_API/app/api/v1/endpoints/config_info.py[149-171]
- tldw_Server_API/app/services/worker_startup_policy.py[92-110]
- apps/packages/ui/src/services/tldw/server-capabilities.ts[420-510]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


6. Playlist routes ignore draining ✓ Resolved 🐞 Bug ☼ Reliability
Description
The new playlist preflight endpoint enqueues durable Jobs work
(PlaylistIngestService.create_preflight calls JobManager.create_job) without guarding against
app shutdown draining. Unlike /media/ingest/jobs (which uses assert_may_start_work), playlist v2
requests can start new work during draining and may strand preflights/runs depending on shutdown
ordering.
Code

tldw_Server_API/app/api/v1/endpoints/media/playlist_ingest.py[R267-294]

+@router.post(
+    "/playlist-preflights",
+    response_model=PlaylistPreflightAcceptedResponse,
+    status_code=status.HTTP_202_ACCEPTED,
+    summary="Create an asynchronous playlist preflight resource",
+    tags=["Media Playlist Ingest v2"],
+    dependencies=_PREFLIGHT_DEPENDENCIES,
+    openapi_extra=_request_body_schema(PlaylistPreflightCreateRequest),
+)
+def create_playlist_preflight(
+    request_scope: Request,
+    payload: Any = Body(...),
+    current_user: User = Depends(get_request_user),
+    job_manager: JobManager = Depends(get_job_manager),
+) -> PlaylistPreflightAcceptedResponse:
+    try:
+        request = PlaylistPreflightCreateRequest.model_validate(payload)
+    except ValidationError as exc:
+        raise HTTPException(status_code=422, detail="invalid_playlist_url") from exc
+    try:
+        created = PlaylistIngestService(job_manager).create_preflight(
+            _owner(current_user),
+            url=request.url,
+            max_items=request.max_items,
+            timeout_seconds=request.timeout_seconds,
+        )
+    except Exception as exc:  # noqa: BLE001 - trust boundary maps every failure to a safe code
+        _raise_http(exc)
Evidence
The playlist preflight endpoint directly calls service code that creates a durable job, but unlike
the existing ingest job submission endpoint it does not apply the lifecycle draining guard.

tldw_Server_API/app/api/v1/endpoints/media/playlist_ingest.py[267-307]
tldw_Server_API/app/core/Ingestion_Media_Processing/Video/playlist_ingest_service.py[350-367]
tldw_Server_API/app/api/v1/endpoints/media/ingest_jobs.py[2251-2263]
tldw_Server_API/app/services/app_lifecycle.py[49-56]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
Playlist v2 endpoints that create new durable work (at least `POST /playlist-preflights` and likely `POST /ingest/runs` and retries) do not call `assert_may_start_work`. This allows work to be enqueued during the server’s draining shutdown phase.

### Issue Context
- `assert_may_start_work` is used on the existing ingest-jobs submission path to reject new work with a 503 when draining.
- `PlaylistIngestService.create_preflight` creates a Jobs row (`create_job`), so it should follow the same lifecycle contract.

### Fix Focus Areas
- Import and call `assert_may_start_work(request_scope.app, "media.playlist.preflight.create")` (and similar) at the start of endpoints that enqueue jobs / mutate durable playlist ingest state.
- Apply consistently to:
 - `create_playlist_preflight`
 - `create_playlist_ingest_run`
 - `retry_playlist_ingest_run` (if it creates/enqueues work)

#### Target locations
- tldw_Server_API/app/api/v1/endpoints/media/playlist_ingest.py[267-492]
- tldw_Server_API/app/services/app_lifecycle.py[49-56]
- tldw_Server_API/app/core/Ingestion_Media_Processing/Video/playlist_ingest_service.py[277-387]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Qodo Logo

Comment thread tldw_Server_API/app/api/v1/endpoints/media/playlist_ingest.py
Comment thread tldw_Server_API/tests/MediaIngestion_NEW/unit/test_media_ingest_jobs_endpoint.py Outdated
Comment thread tldw_Server_API/app/api/v1/endpoints/config_info.py
Comment thread tldw_Server_API/app/api/v1/endpoints/media/playlist_ingest.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant