Skip to content

Commit d4b9e4f

Browse files
crisnahinecrisnahine
authored
QA round 8: a name never rebuilds a path (#158)
QA round 8: a name never rebuilds a path
2 parents 60d28b1 + 6c82c5a commit d4b9e4f

57 files changed

Lines changed: 2345 additions & 150 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎CHANGELOG.md‎

Lines changed: 186 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,192 @@ All notable changes to this project will be documented in this file.
55
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
66
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
77

8+
## [5.24.0] - 2026-08-17
9+
10+
### Fixed
11+
12+
Defects found by a QA round against GitLab, OpenProject, Canvas LMS,
13+
Discourse and Mastodon. Every one of them exits 0.
14+
15+
- **The commands that read the app degrade the way `tool` does.** `context`,
16+
`inspect`, `facts`, `preset`, `watch` and `init` called a bare boot guard,
17+
so on a repo you have just cloned - the case an agent most needs a
18+
`CLAUDE.md` for, and the case least likely to boot - every tool answered
19+
and the command that writes the files exited 1 having written nothing.
20+
`init` was worse: it writes its config files first, so a boot failure left
21+
the app half set up and called that a failure. They allow the static tier
22+
now and each takes `--no-boot`; `doctor` still fails, because diagnosing
23+
the boot is its job. Writing under `--no-boot` then exposed its own bug -
24+
the context writer asked `Rails.application` for the output directory,
25+
which raises `NameError` on the path where Rails is never loaded at all.
26+
- **`tool --list` reads the app's config when the app cannot boot.** Boot is
27+
what normally loads `.rails-ai-context.yml`, so the listing fell back to
28+
the gem's defaults: on Mastodon it advertised 45 tools while the MCP server
29+
offered 43 and the CLI itself answered `Unknown tool 'query'` for one it
30+
had just listed.
31+
- **`search_extensions` is documented as the fallback's list, which is what
32+
it is.** Making ripgrep honour it did make the two backends agree, and it
33+
cost the reach that makes the tool useful: on Mastodon `gem 'devise'` went
34+
from 5 results to none, because a Gemfile carries no listed extension - and
35+
the same for a Rakefile, a `.md`, a `.sql`. The docs, the attr comment and
36+
the line the install generator writes now say plainly that the list is the
37+
Ruby fallback's and that ripgrep searches every file.
38+
- **A table the replay cannot name is not reported.** Canvas has a migration
39+
that calls `create_table table_name do |t|` with a local computed at run
40+
time; the replay kept the entry under a nil key, and the first serializer
41+
to sort the table names took the whole context run down. Reachable only
42+
once `context` could enter the static tier at all.
43+
- **Coverage says nothing when there is nothing to measure against.** With a
44+
`default_locale` the app configures but ships no file for, every locale
45+
scored zero and all of them were named as untranslated.
46+
- **A mixin in a nested concerns directory is not a model.** The skip only saw
47+
the top-level `concerns/` that Rails autoloads, so OpenProject's
48+
`app/models/queries/operators/concerns` contributed four mixins to a model
49+
count of 978 where the app has 974. A nested `concerns/` is an ordinary
50+
namespace, though, so the directory name alone does not decide it: a file
51+
there that declares a class is a model like any other.
52+
- **`docs/COMPATIBILITY.md` describes the static tier the gem actually has.**
53+
It said 6 introspectors answer without a booted app and "the other 34 have
54+
no static path", naming eight examples - all of which answer. The real
55+
split is 23 files-only, 9 alternate-source and 8 runtime-only: 32 of 40.
56+
A guard spec derives all three lists from `INTROSPECTOR_MAP`.
57+
58+
- **A controller is named by the constant its source declares.** Zeitwerk
59+
resolves a path through the app's own inflector, which the static tier
60+
never loads, so camelizing the path invented `Activitypub::` for the 12
61+
controllers Mastodon declares as `ActivityPub::` and `Oauth::` for the 4
62+
it declares as `OAuth::` - names that appear nowhere in an app that
63+
registers those acronyms, and a `NameError` for anyone who uses one. The
64+
path stays the answer where the source does not carry the whole name
65+
(`application_cable/channel.rb`) and where the declared name does not name
66+
that file, since Prism recovers a syntax error into a partial tree.
67+
- **An interceptor is told from a mailer by its framework hook.**
68+
`app/mailers` is also where ActionMailer interceptors live, and every `.rb`
69+
under it counted: OpenProject's `Interceptors::DefaultHeaders` arrived as a
70+
mailer with `delivering_email` - a hook the framework calls - listed as an
71+
email an agent could send, 2 of its 11 entries. What an interceptor has and
72+
a mailer does not is one of `delivering_email`, `previewing_email` or
73+
`delivered_email`; requiring a class instead would have dropped GitLab's 20
74+
`Emails::*` modules, which hold every notification it sends.
75+
- **Migration replay reads what a migration does, not what it undoes.** A
76+
`def down` says how to reverse the change, so replaying it alongside `up`
77+
cancelled the migration out. On a migrations-only app the ordinary
78+
up-creates/down-drops pair erased two tables OpenProject really has (35
79+
to 37), and the reverse pair would have invented one it dropped. The same
80+
now holds for the block spellings, `reversible { |dir| dir.down { ... } }`
81+
and `revert`, and for the `t.timestamps` inside them - those were found by
82+
a separate walk over the whole file, so a down body's timestamps landed on
83+
the last table created on the way up.
84+
- **i18n coverage groups the locales below its own rounding floor instead of
85+
listing them.** A language-name lookup table under `config/locales`
86+
contributes a top-level key per language, and Rails does load each as an
87+
available locale. Scoring them produced a row each saying every key was
88+
missing: on Discourse, 138 of 186 coverage rows described a translation
89+
effort nobody had started. Those 138 are now summarised in one line, with a
90+
shared key count stated once. The rows still exist - asking for one by name
91+
answers with its numbers, so a translation genuinely started below the floor
92+
is not written off as untranslated. The available list still matches a
93+
booted Rails, and a locale's keys are read from its own file and any shared
94+
file together, so one that lives outside the naming convention is scored.
95+
- **A recorded tool selection no longer switches off `.ai-context.json`.**
96+
The install generator always records one, and every later `rails
97+
ai:context` passed that list to the serializer. The machine artifact is
98+
not an AI tool and no install menu offers it, so it silently stopped being
99+
written on the one install path the docs describe - while the generator
100+
went on adding it to `.gitignore`.
101+
- **`skip_tools` accepts the symbol spelling its neighbours use.** Every
102+
other key in the generated initializer takes symbols, and `%i[]` here
103+
skipped nothing at all: both readers compare against a tool name, which is
104+
a String.
105+
106+
- **A controller carries the file it was read from.** Reading the name from
107+
the source fixed the name and broke every path derived from it:
108+
`ActivityPub::CollectionsController`.underscore is `activity_pub/...` and
109+
the file is under `activitypub/`, so `rails_get_context` answered "Could
110+
not extract source code" for a file that exists and asked the routes for a
111+
controller key Rails does not use. Both tiers emit `file:` - the booted one
112+
asks Ruby where the class was defined - and the consumers read it through
113+
`Payload`.
114+
- **The locale files are indexed once, not once per locale.** Asking every
115+
file about every locale is O(locales x files): on Discourse, 187 over 108,
116+
it took the i18n answer from 4.6 seconds to four and a half minutes. Each
117+
entry also records the locales it serves, so asking for one by name finds
118+
its files even when the filename spells the gem rather than the locale.
119+
120+
- **Every consumer that turns a controller name back into a path reads the
121+
file it was read from.** Carrying `file:` fixed six of them and left five
122+
answering a confident negative, which is the answer an agent acts on
123+
without checking. On Mastodon's 18 inflected controllers: `rails_test_info`
124+
reported 16 specs that exist as missing, `rails_get_routes` reported 4
125+
routes that exist as absent, `rails_get_view` 2 views, and
126+
`rails_generate_test` wrote a spec whose body was `skip "no routes found"`
127+
for a controller with a route. `rails_analyze_feature` listed them as
128+
untested. `Payload.controller_route_key` is the one derivation now, and a
129+
spec drives all five tools with an inflected name.
130+
- **The install path's files require their own stdlib.** `init` loads six
131+
files before Rails and before the entry file, on purpose - so the entry
132+
file's `require "set"` never runs for them. `legacy_cleanup`'s `[...].to_set`
133+
raised `NoMethodError` on Ruby 3.1, where `Set` is not autoloaded, and 3.2+
134+
hid it: only the CI matrix ever saw it.
135+
- **`rails_runtime_info` reports cache numbers, not the cache object.** The
136+
MemoryStore branch passed `cache.inspect` straight through, so the answer
137+
carried `#<ActiveSupport::Cache::MemoryStore entries=0, size=0, options={...}>`
138+
- the shape this gem has shipped as a defect before. Entries and size are
139+
now facts, and a store that answers `#stats` gets its hash rendered as pairs
140+
rather than a Ruby literal.
141+
- **A model's table comes from its file, not from its name.** Naming a model by
142+
the constant its source declares made `underscore` stop being the way back:
143+
`OAuthClientConfig` underscores to `o_auth_client_config`, and the file is
144+
`oauth_client_config.rb`. `rails_model_details` reported the table as
145+
`o_auth_client_configs`, which no app has, so the columns section vanished
146+
and the schema hint pointed at nothing - on Canvas that hit
147+
`OAuthClientConfig`, `OAuthRequest`, `AuthenticationProvider::OAuth` and
148+
`OAuth2`, on OpenProject `OAuthClient` and `OAuthClientToken`. The file's own
149+
name already carries the inflection, because Zeitwerk resolved the constant
150+
from it. `rails_generate_test`, `rails_get_callbacks`, `rails_test_info` and
151+
the five `rails_validate` checks that key models by path read it too.
152+
- **An abstract base class is not a model in the static tier either.** A
153+
booted Rails rejects `abstract_class?`, and a namespaced base class is one:
154+
GitLab's `Ci::ApplicationRecord`, `PackageMetadata::ApplicationRecord` and
155+
`SecApplicationRecord` were counted, so the same app answered 895 models
156+
without a boot and 888 with one. OpenProject: 974 against 971.
157+
- **A model carries its file too, and is named by the constant its source
158+
declares.** `rails_model_details` rebuilt `app/models/<underscored>.rb` from
159+
the name in four places, so for a model in a pack or an engine the custom
160+
validations, the source-defined methods, the method signatures and the class
161+
structure all went quietly missing from an answer that otherwise looked
162+
complete. The static tier also camelized the path into the name, so an app
163+
registering an inflection got a constant it does not have - and one the
164+
booted tier, which reads the real class, would never agree with.
165+
- **`rails_validate_semantics` checks the views under an inflected directory.**
166+
A view directory names the route key, so camelizing `app/views/oauth/` back
167+
gave a constant the app never declares and the undefined-ivar check stopped
168+
running for that whole tree without saying so.
169+
- **A test-gap claim is not made from a scan that stopped early.**
170+
`rails_analyze_feature` scans the first 500 files per glob; Discourse has
171+
1,672 specs, so the scan never reached the ones covering the feature and
172+
every controller in it was reported as having no test. The Tests section
173+
disclosed the cap, the gaps section asserted through it.
174+
- **"Global before_actions" in the generated files means global.** The scan
175+
matched `skip_before_action` too, and ignored `only:` / `except:` / `if:`.
176+
Mastodon's `CLAUDE.md` listed five, of which three were false - including
177+
`verify_authenticity_token`, which that controller skips.
178+
179+
### Added
180+
181+
- **`--no-boot` on `context`, `inspect`, `facts`, `preset`, `watch` and
182+
`init`.** The static tier was reachable from `tool` and `serve` only.
183+
- **`--no-boot` reads a repo that has source but no `config/`.** An engine
184+
keeps its dummy app under `spec/dummy`, so the guard against describing an
185+
empty directory has to measure source, not boot files.
186+
187+
### Changed
188+
189+
- **The generated initializer has one indent from top to bottom.** The AI
190+
Tools section was written at the configure body's indent and every section
191+
after it flush, so the file an app commits changed indent two lines in and
192+
stayed there.
193+
8194
## [5.23.0] - 2026-08-16
9195

10196
### Fixed

‎CONTEXT.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,14 @@ Two senses, one per module, and neither is bare "path" in a name.
2828

2929
**How a path is written down** - `PortablePath` rewrites one so it means the same thing on another machine, because what it touches ends up in `.ai-context.json` and the app commits that file. App paths go app-relative, gem paths keep the gem and version and drop the install prefix. "Relativize" always means this.
3030

31+
## Declared constant
32+
33+
What a source file calls its own class, as opposed to the **path name** - the constant its path camelizes to. The two differ wherever the app registers an inflection, because Zeitwerk resolves a path through the app's own inflector and the static tier has never loaded it: `app/controllers/activitypub/` is `ActivityPub` in Mastodon, and `Oauth` is a constant nothing defines. `DeclaredConstant` reads the class the source declares, and since an inflection only ever changes case, the declaration that names a file is the one equal to the path name ignoring case. Anything else - a second class in the file, a nested error class, a tree Prism recovered from a syntax error - is not this file's class, and there the path name stays the answer: it is the only thing carrying the namespace when the source does not.
34+
35+
Distinct from the **Static tier** sense of "declared": that one is about declaring a tier's capability up front rather than detecting it at runtime. This one is about a constant's spelling.
36+
37+
The other half of the same problem is the reverse trip. Once a name is the declared one, no consumer can rebuild the path from it - and a pack or an in-repo engine breaks that derivation too, inflection or not. So controllers and models carry `file:` from whichever tier found them, and consumers read it through `Payload` (`controller_file`, `controller_route_key`, `controller_for_route_key`, `model_file`). The rule: a tool that needs a path for a name reads it, never underscores it.
38+
3139
## Static tier
3240

3341
The mode where the app did not boot, or `--no-boot` was passed. What an introspector answers here is what it declared, never what a runtime check happened to detect. Every introspector in `INTROSPECTOR_MAP` extends `StaticTier` and names one of three kinds:

‎docs/CLI.md‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@ rails-ai-context serve --transport http --port 6029 # HTTP transport
3737
|:-------|:--------|:------------|
3838
| `--transport` | `stdio` | `stdio` or `http` |
3939
| `--port` | `6029` | HTTP listen port |
40+
| `--no-boot` | off | Skip booting the app; answer from source alone |
4041

4142
### `tool`
4243

@@ -58,6 +59,28 @@ rails-ai-context tool model_details --model User
5859
|:-------|:------------|
5960
| `--list` | List all available tools |
6061
| `--json` | Output as JSON |
62+
| `--no-boot` | Skip booting the app; answer from source alone |
63+
64+
### The static tier and `--no-boot`
65+
66+
Every command that reads the app takes `--no-boot`: `tool`, `serve`, `context`,
67+
`inspect`, `facts`, `preset`, `watch` and `init`. It skips the boot and answers
68+
from source alone, which is what you want on a repo you have just cloned, on an
69+
app whose boot is broken, and in CI where booting costs more than the answer.
70+
71+
The same tier is entered automatically when a boot fails, so you get an answer
72+
either way. Answers that need a running app are marked `[UNAVAILABLE: ...]`
73+
rather than guessed, and everything else is tagged `[STATIC]` instead of
74+
`[VERIFIED]`. `docs/COMPATIBILITY.md` lists which of the 40 introspectors answer
75+
in which tier.
76+
77+
`doctor` is the exception: diagnosing the boot is its job, so it refuses
78+
`--no-boot` and still exits 1 when the app cannot start.
79+
80+
```bash
81+
rails-ai-context tool models --no-boot # no boot, no database, no Gemfile
82+
rails-ai-context context --no-boot # writes CLAUDE.md from source
83+
```
6184

6285
### Tool name resolution
6386

‎docs/COMPATIBILITY.md‎

Lines changed: 26 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -70,23 +70,37 @@ Two tiers, both reachable over the CLI and MCP (stdio and HTTP):
7070
(ActiveRecord connections, `Rails.application.routes`, loaded classes) plus the
7171
Prism AST layer for source-level facts (scopes, callbacks, strong params).
7272

73-
**STATIC** - the app didn't boot, or `--no-boot` was passed. Only introspectors
74-
that define a `static_call` path can answer without a booted app:
73+
**STATIC** - the app didn't boot, or `--no-boot` was passed. 32 of the 40
74+
introspectors answer here; each declares which of three kinds it is
75+
(ADR-0002), so what a tier can say is a declaration rather than a guess.
76+
77+
**files-only** (23) run unchanged against the static app handle, because they
78+
only ever read files: `gems`, `views`, `view_templates`, `turbo`, `stimulus`,
79+
`active_storage`, `action_text`, `auth`, `tests`, `rake_tasks`, `assets`,
80+
`devops`, `action_mailbox`, `migrations`, `seeds`, `middleware`, `env_config`,
81+
`multi_database`, `components`, `performance`, `frontend_frameworks`,
82+
`credentials` and `env`.
83+
84+
**alternate-source** (9) have a `static_call` that reads a different source
85+
from the booted path:
7586

7687
| Introspector | Static source |
7788
|:---|:---|
78-
| `schema` | `db/schema.rb` / `db/structure.sql` / migration files |
79-
| `migrations` | migration file list + `db/structure.sql`'s trailing `schema_migrations` insert |
80-
| `routes` | `config/routes.rb` parsed with a dedicated Prism listener |
89+
| `schema` | `db/schema.rb` / `db/structure.sql` / migration replay |
8190
| `models` | `app/models/**/*.rb` (plus packs/engines/extra paths) parsed, not constantized |
91+
| `routes` | `config/routes.rb` parsed with a dedicated Prism listener |
8292
| `controllers` | `app/controllers/**/*.rb` (plus packs/engines/extra paths) parsed, not constantized |
83-
| `env_config` | `config/environments/*.rb` read from disk - file-based, so the static tier serves the same data as a booted app |
84-
85-
The other 34 introspectors (views, jobs, gems, turbo, i18n, active_storage,
86-
auth, api, and the rest) have no static path and report `{ unavailable: reason
87-
}` in this tier - by construction, not by shape: `Introspector#run_introspector`
88-
(`lib/rails_ai_context/introspector.rb`) falls back to the same message
89-
regardless of what triggered the static tier.
93+
| `jobs` | `app/jobs`, `app/mailers` and `app/channels` parsed for classes and their public methods |
94+
| `i18n` | every top-level key across `config/locales`, and the default locale read from `config/` |
95+
| `api` | serializers, API controllers and route constraints read from source |
96+
| `engines` | `config/routes.rb` mounts, plus the Gemfile |
97+
| `active_support` | concern and core-extension use read from source |
98+
99+
**runtime-only** (8) report `{ unavailable: reason }` here, and only these:
100+
`conventions`, `database_stats`, `config`, `initializers`, `autoload`,
101+
`connection_pool`, `security` and `observability`. The fallback message comes
102+
from `Introspector#run_introspector` (`lib/rails_ai_context/introspector.rb`)
103+
and is the same whatever put the gem in the static tier.
90104

91105
`doctor` never enters the static tier - its job is diagnosing why boot failed,
92106
so it always requires a bootable app.

0 commit comments

Comments
 (0)