Repository navigation
Conversation
The generated AGENTS.md pointed at <block>/README.md, but Building Block docs folders are named after their packages (bb-kv-store/, ...) and core APIs live in core/README.md, so agents guessed paths like kv-store/ and missed.
🦋 Changeset detectedLatest commit: 32676a1 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Harbor agent check — advisory20 todo-app trajectories per arm. Pass rates carry Wilson 95% intervals; metric differences are candidate − baseline medians over passed phases with a seeded bootstrap 95% interval, shown only when both arms have at least 3 passes. The primary metric is cache-neutral cost; the storage section compares arms within one storage block. Local mock mode only; nothing is deployed.
Packed packages differ between arms. Trials
Failures
Pass rate
Passed-phase medians (IQR)
† Descriptive, not comparable across arms: new input (uncached input + cache writes) and cache read depend on prompt-cache warm-up order, so no difference is reported for them. Candidate − baseline
Δ is the difference of medians (percent of the baseline median) [bootstrap 95% interval]. A phase passes when it and every earlier phase passed with complete accounting in one session. Trials without a result or with an infrastructure error are excluded from rates. Cache-neutral cost is the primary metric: each phase at the pinned model’s list prices as if its own session wrote every context token to the prompt cache once (context growth at the cache-write rate, the rest of the input processed at the cache-read rate, output at the output rate), so it does not depend on which concurrent session warmed the cache. Input processed = uncached input + cache writes + cache reads. List-price cost is Claude Code’s estimate, not AWS billing, and is secondary: it moves with cache hits from other sessions. Failed or incomplete work is excluded from metrics: less usage there is not an efficiency win. Storage block
The storage block the create phase constructed, read from its snapshot. It moves cost by itself, so a different storage mix between arms shifts the pooled medians above. Costs here are whole-run medians over trials that passed every phase; a difference is shown only when both arms have at least 3 such passes in that storage. |
Replace the docs bullet with one command that lists the docs folders and prints the block catalog, so agents read folder names instead of guessing them, and say to import ApiNamespace/ApiError from @aws-blocks/blocks.
xyos
changed the title
Problem
The generated
AGENTS.mdtells agents to open<block>/README.mdin the AWS Blocks docs folder. The per-block folders are named after their packages (bb-kv-store/,bb-distributed-table/, ...), so agents drop thebb-prefix and guess paths likedistributed-table/orkv-store/that don't exist. The core APIs (ApiNamespace,ApiError, the typed client) live incore/README.md, which the bullet doesn't mention.Changes
packages/create-blocks-app/resources/AGENTS.md: the docs bullet now gives one command that lists the docs folders and prints the block catalog from the docsREADME.md(the table between theblock-catalogmarkers thatscripts/sync-catalog.mjsmaintains). Agents then read the real folder names (e.g.bb-distributed-table) instead of guessing them. The bullet points tocore/README.mdforApiNamespace,ApiErrorand the typed client, and says to import them from@aws-blocks/blocks.@aws-blocks/create-blocks-apppatch.Why this wording
We compared 12 versions of this bullet, plus no bullet at all, using headless Claude Code sessions on 31 feature tasks in a freshly scaffolded
bareapp. Each task is graded by running the app (dev server, JSON-RPC calls, restarts), never by reading the code.core/README.md) was neutral on both models, and so was a variant that only added the import hint.So this is an efficiency change (agents get oriented sooner), not a correctness fix.
Tradeoffs
node_modules/@aws-blocks/blocks/docsand drops therequire.resolvecommand, so agents in a non-flat install (pnpm, workspaces) lose that route. Every evaluated scaffold used the default flat npm install. The package README still documents therequire.resolvecommand.<!-- BEGIN:block-catalog -->/<!-- END:block-catalog -->markers, whichsync-catalog.mjswrites andsync-catalog.test.mjscovers.Validation
node_modules/@aws-blocks/blocks/docs), the bullet's command lists 25 docs folders (24 blocks pluscore) and prints the catalog table (25 rows, 29 lines with the markers).bb-distributed-table/README.mdandcore/README.mdexist.32676a14with Node 24.16.0:npm run build,npm run lint:deps(416 files),npm test(6148 pass, 0 fail, 7 skipped) andnpm run test:e2e:local(432 pass, 0 fail, 1 skipped) all exit 0.Checklist