Skip to content

feat(craft): outputs artifact classifier - #14326

Open
nmgarza5 wants to merge 2 commits into
mainfrom
nikg/craft-artifact-classifier
Open

feat(craft): outputs artifact classifier#14326
nmgarza5 wants to merge 2 commits into
mainfrom
nikg/craft-artifact-classifier

Conversation

@nmgarza5

@nmgarza5 nmgarza5 commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Description

Why. Between the manifest (#14325, what exists in outputs/) and the artifact rows (#14319, what the panel renders) sits a decision: which entries are deliverables, what type each one is, and what hash a directory gets so version bumps fire exactly when content changes. The only classification in the codebase today is the fabricated webapp check (web/ directory exists, no package.json required). This PR makes that decision a set of pure, unit-testable functions so the reconciler (#14327) stays a thin diff loop. Stacked on #14319 because the classifier emits the widened ArtifactType values.

What.

  • artifact_classifier.py: derive_artifacts reduces an outputs listing to its artifacts — each visible top-level entry is one artifact, nested files belong to their top-level directory and never surface on their own. Files are typed by extension with a generic fallback (.ts is pinned to CODE before the mimetypes families, which register it as video/mp2t). The scaffolded webapp is recognized by the same web/package.json convention the runtime uses.
  • Directory hashes are deterministic over root-relative paths, so identical trees hash identically and a child rename, addition, removal, or content change all move the hash. Hidden names (node_modules, .next, dotfiles) never influence the result, so build churn cannot dirty an artifact.
  • Files past the manifest hash ceiling get a meta:<size>:<mtime> surrogate signal. Passing NULL instead would mean the version never bumps for large files, because feat(craft): artifact index and action receipt schema with DAL #14319's upsert compares with IS DISTINCT FROM and NULL never differs from NULL.
  • The shared hidden-name rule moves to configs.py so the classifier and the session manager filter identically without the classifier importing the manager, and the webapp path constants unify in nextjs_dev.py so the convention cannot drift across its three consumers.

Nothing consumes derive_artifacts yet; the reconciler in #14327 does.

How Has This Been Tested?

  • 36 unit tests (test_artifact_classifier.py): the full extension table including the .ts/mimetypes pitfall and legacy binary formats staying generic, top-level-only reduction, deep-nesting aggregation, directory hashes moving on content change/rename/add and not on hidden churn, identical trees hashing identically, hidden-only directories hashing like empty ones, the exact webapp convention (nested web/ and package.json-as-directory both rejected), surrogate behavior for unhashed files including partial-metadata degradation, and sort determinism.
  • Craft unit suite green alongside (session manager still filters identically after the constant move).

Additional Options

  • [Optional] Please cherry-pick this PR to the latest release version.
  • [Optional] Override Linear Check

@nmgarza5
nmgarza5 requested a review from a team as a code owner August 27, 2026 02:48
@greptile-apps

greptile-apps Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Adds pure classification functions that convert complete sandbox output listings into deterministic artifact records.

  • Classifies top-level files by extension or MIME family and aggregates directories over visible descendants.
  • Detects scaffolded web applications through the shared web/package.json convention.
  • Centralizes hidden workspace-name filtering and Next.js output-path constants.
  • Adds unit coverage for classification, visibility, hashing, metadata surrogates, and web application detection.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
backend/onyx/server/features/build/artifact_classifier.py Introduces artifact classification, visibility filtering, deterministic directory hashing, metadata surrogates, and scaffold-based web application detection.
backend/onyx/server/features/build/configs.py Centralizes the hidden workspace-name set and visibility predicate.
backend/onyx/server/features/build/sandbox/nextjs_dev.py Derives the canonical web application paths from shared root and relative-path constants.
backend/onyx/server/features/build/session/manager.py Reuses the centralized hidden workspace-name predicate without changing the existing filter behavior.
backend/tests/unit/onyx/server/features/craft/test_artifact_classifier.py Covers artifact typing, top-level reduction, hashing, visibility, metadata degradation, ordering, and web application recognition.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Complete outputs listing] --> B[Remove hidden paths]
    B --> C[Group entries by top-level root]
    C --> D{Top-level entry type}
    D -->|File| E[Classify extension or MIME family]
    D -->|Directory| F[Aggregate visible descendants]
    F --> G{web/package.json present?}
    G -->|Yes| H[WEB_APP artifact]
    G -->|No| I[DIRECTORY artifact]
    E --> J[Sort artifacts by path]
    H --> J
    I --> J
Loading

Reviews (2): Last reviewed commit: "docs(craft): state the partial-metadata ..." | Re-trigger Greptile

Comment thread backend/onyx/server/features/build/artifact_classifier.py
@nmgarza5

Copy link
Copy Markdown
Contributor Author

@greptile

Base automatically changed from nikg/craft-artifact-schema to main August 27, 2026 18:41
@nmgarza5
nmgarza5 force-pushed the nikg/craft-artifact-classifier branch from 178110b to 2735d38 Compare August 27, 2026 18:56
@github-actions

github-actions Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Full-stack Preview (frontend + backend)

Status Preview Commit Updated
https://7c972f3-onyx.preview.onyxcorp.dev/ 7c972f3 2026-08-27 21:42:25 UTC

@github-actions

github-actions Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

🖼️ Visual Regression Report

Project Changed Added Removed Unchanged Report
admin 7 0 0 180 View Report
exclusive 0 0 0 10 ✅ No changes

Pure functions from an outputs listing to the artifacts a reconciler should
hold: each visible top-level entry is one artifact, files typed by extension
with a generic fallback, directories aggregating visible descendants into a
deterministic content hash, and the scaffolded web directory recognized as a
webapp. Files past the manifest hash ceiling carry a size-and-mtime surrogate
so change detection still moves for them. The shared hidden-name rule moves
to configs so the classifier and the session manager filter identically
without the classifier importing the manager, and the webapp path convention
gains derived constants in nextjs_dev so its consumers cannot drift.
An adapter that omits size or mtime for an unhashed file degrades change
detection to the field it did supply. The docstring says so and a test pins
the degradation, since no metadata-only signal can see a change in data it
was never given.
@nmgarza5
nmgarza5 force-pushed the nikg/craft-artifact-classifier branch from 2735d38 to 7c972f3 Compare August 27, 2026 21:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants