Skip to content

perf: drop node positions after metadata - #1236

Open
ovflowd wants to merge 1 commit into
mainfrom
perf/drop-positions
Open

ovflowd wants to merge 1 commit into
mainfrom
perf/drop-positions

Conversation

@ovflowd

@ovflowd ovflowd commented Nov 3, 2026

Copy link
Copy Markdown
Member

Description

This PR drops position from every node of metadata's entries once they're parsed. Nothing after metadata reads where a node came from in the source: the type warnings that print positions run while it parses. On Node.js's docs that's ~480k objects, about 80MB, that the main thread no longer holds or copies to and from the workers, and builds are ~5% faster.

json-simple writes the metadata as it is, so it no longer has them either, and is about a third of its size.

Validation

Added drops where each node of an entry came from, which checks that no node of a parsed entry has a position. json-simple is the only target whose output changes, and the <api>.json files json and legacy-json both write already race on main.

Related Issues

Split out of #1205.

Check List

  • I have read the Contributing Guidelines and made commit messages that follow the guideline.
  • I have run node --run test and all tests passed.
  • I have check code formatting with node --run format:check & node --run lint.
  • I've covered new added functionality with unit tests if necessary.

Nothing after `metadata` reads where a node came from in the source:
the type warnings that print positions run while it parses. Dropping
them shrinks every entry the main thread holds and copies to and from
the workers. On Node.js's docs that is ~480k position objects, about
80MB, and builds are ~5% faster.

`json-simple`, which writes the metadata as it is, no longer has them
either, and is about a third of its size.

Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>
Assisted-by: Claude Sonnet 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
api-docs-tooling Ready Ready Preview Nov 3, 2026 7:34pm UTC

Request Review

@codecov

codecov Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.02326% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 94.14%. Comparing base (c8b228e) to head (b65c0b7).

Files with missing lines Patch % Lines
...generators/metadata/utils/__tests__/parse.test.mjs 90.32% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1236      +/-   ##
==========================================
- Coverage   94.14%   94.14%   -0.01%     
==========================================
  Files         291      291              
  Lines       28950    28993      +43     
  Branches     2944     2953       +9     
==========================================
+ Hits        27256    27296      +40     
- Misses       1672     1675       +3     
  Partials       22       22              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ovflowd ovflowd mentioned this pull request Nov 3, 2026
4 tasks done
@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

api-links Generator

Performance estimate (single CI run)

  • Generation time: 32.4% faster (1.05 s → 710.00 ms)
  • Peak memory: 0.2% higher (293.49 MB → 294.01 MB)

json Generator

Performance estimate (single CI run)

  • Generation time: 17.4% faster (9.48 s → 7.83 s)
  • Peak memory: 8.9% lower (997.38 MB → 909.03 MB)

legacy-html Generator

Performance estimate (single CI run)

  • Generation time: 1.8% slower (12.36 s → 12.58 s)
  • Peak memory: 5.7% lower (1.52 GB → 1.43 GB)

legacy-json Generator

Performance estimate (single CI run)

  • Generation time: 26.7% faster (7.23 s → 5.30 s)
  • Peak memory: 3.1% higher (934.98 MB → 964.41 MB)

llms-txt Generator

Performance estimate (single CI run)

  • Generation time: 1.8% slower (8.43 s → 8.58 s)
  • Peak memory: 2.1% higher (864.12 MB → 882.38 MB)

orama-db Generator

Performance estimate (single CI run)

  • Generation time: 23.1% faster (8.82 s → 6.78 s)
  • Peak memory: 6.8% higher (992.37 MB → 1.03 GB)

web Generator

Performance estimate (single CI run)

  • Generation time: 39.0% faster (23.90 s → 14.59 s)
  • Peak memory: 1.3% lower (1.48 GB → 1.46 GB)

@cloudflare-workers-and-pages

Copy link
Copy Markdown

🚀 Deploying Preview to Cloudflare 🚀

Preview Deployments by commit

Status Deployment URL Commit Updated (UTC) See this deployment's details
  • Build: Success ✅

View logs ↗
No Preview URL. Enable ↗ b65c0b7 2026-11-03T19:34:31.832Z View logs ↗

@ovflowd
ovflowd marked this pull request as ready for review November 3, 2026 20:39
@ovflowd
ovflowd requested a review from a team as a code owner November 3, 2026 20:39

This branch was successfully deployed

1 active deployment
Preview – api-docs-tooling — b65c0b71 Deployed Nov 3, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant