test: pin behaviour the open-issue review found untested - #3933
Open
sh41 wants to merge 40 commits into
Open
Conversation
Test ResultsCommit ✅ Required legs: 0 failing tests, all 12 ran to completion.
Required legs
Counts only - for which tests failed, open a leg’s |
sh41
force-pushed
the
missing-tests
branch
13 times, most recently
from
August 31, 2026 15:30
a7259c4 to
90c0074
Compare
The Unicode character classes in Elixir.flex shipped with no test, so reverting them left the suite green. Assert that CJK, accented Latin and Cyrillic lex as whole identifiers and atoms, and that uppercase is an atom start but not an identifier start.
VariableSymbol.classify routes a var! name to Kind.VARIABLE via Callable.isVariable, and isDeclaration then asks whether it sits in the left operand of the nearest Match. Nothing covered var! on either path. Pins both directions so a classifier that blindly accepted anything inside var! would fail: var!(foo) = 1 declares foo, and x = var!(foo) does not. A third case resolves a plain binding inside the same quote, which is the control for a separate, still-failing resolution case that is not committed here.
…d clone Debug does not launch the configuration the user edited. Process mints a node name and cookie and asks for a copy carrying them, and it is that copy whose command line is built, so everything the user set has to survive the copy. For the environment it did not. exunit.Configuration.debuggedConfiguration populated the clone with `envs.putAll(envs)` -- the freshly created local map copying itself rather than the property -- so environment variables were dropped on the way to Debug while working correctly under Run. That is KronicDeth#2187. It was fixed in 5a72561 (v13.1.0) by adding the `this.` qualifier, and nothing has pinned it in the four years since. Reintroducing the defect fails two of these four tests, which is what makes them worth having. The second failure is a consequence of KronicDeth#2187 that its report does not mention: with the environment empty, the `putIfAbsent` that defaults MIX_ENV to "test" also silently replaces a MIX_ENV the user set explicitly.
…iew module An unqualified call in a `.eex` template under `templates/<name>/` resolves into `views/<name>_view.ex`, mirroring how Phoenix.View compiles the template into a function on the view module. Shipped in v11.11.0 (2021-06-22) and untested since: no fixture anywhere in the suite used the `.eex` file type for resolution, so a regression here would have been silent. Covers both halves separately so a failure says which one broke: viewFile() finding the view module, and the callable resolver walking past the template into it via maxScope(). Verified by negative control: renaming page_view.ex breaks the convention and the call resolves to nothing, so the assertion is not passing incidentally. Found by a review of the open issue backlog.
… ebin paths An SDK home whose lib directory holds a regular file alongside the real application directories threw NotDirectoryException out of the ebin walk, because it recursed into every entry without asking whether it was a directory. Fixed by 57d9624, shipped in v8.0.0 (2018-08-16), and untouched since - the class moved from org.elixir_lang.jps.HomePath to SdkEbinPaths and the directory filter came with it, but nothing ever asserted it, so dropping it would have been silent. Covers both walks separately: eachEbinPath and hasEbinPath each carry their own copy of the filter, so a fix applied to one and not the other would otherwise pass. A third case walks a lib whose only entry is the plain file, which separates "skipped the file" from "found an app and stopped looking". Verified by mutation: replacing both filters with one that accepts everything fails all three with NotDirectoryException on the reported filename, while the negative control - asserting the plain file really is in an unfiltered listing - still passes. Without that check the assertions could have been vacuous, since the walk's own catch(IOException) swallows the exception; it is LOGGER.error under TestLoggerFactory that then fails the test. Not platform-specific despite the issue title saying Linux: KronicDeth#1143 is the same exception on macOS. Found by a review of the open issue backlog.
KronicDeth#1077 reports Code > Auto-Indent Lines doing nothing in WebStorm, with no reproduction ever supplied in eight years. Nothing under tests/ exercised the action, so its behaviour was unasserted rather than known-good or known-broken, and the report could not be answered either way. It works: indenting from column 0, outdenting an over-indented line, indenting a def, indenting a case clause, and leaving already-correct indentation alone all behave. The action runs CodeStyleManager.adjustLineIndent, which is a different entry point from the whole-file reformat the existing formatter fixtures cover - worth pinning separately for that reason alone. Tests run on IntelliJ IDEA, so this cannot speak to WebStorm specifically. What it shows is that the Elixir side of the action is not the missing piece; the plugin's lang.formatter registration is not IDE-conditional. Refs KronicDeth#1077
…hts as an error Reviewing KronicDeth#1194 found the reported crash already fixed: e71b247 (2021-12-30, shipped in v12.1.0) collapsed the split string/charlist PSI classes into ElixirLine, and ModuleAttribute's type-highlighting dispatch has an explicit ElixirLine/ElixirHeredoc case that runs before the catch-all cannotHighlightTypes() the original report crashed through. Nothing in the suite pinned that case for a charlist literal specifically.
Modules.erlArgumentList interpolates a temporary file's path into an Erlang -eval string. Erlang reads \d and \s inside a <<"...">> binary as escape sequences, so an un-doubled Windows path turns C:\deps\src into control characters and the debugged node dies at boot with "init terminating in do_boot" before any breakpoint is reachable. The doubling shipped in v11.8.1 but nothing pinned it. Removing the .replace call fails testRequireFilePathsEscapeBackslashes on the windows-2025 leg; on the Linux legs the path carries no backslashes, so the round-trip assertion carries the weight there instead.
Matching stubs to decompiled call definitions by position made any difference in count fatal; name/arity matching skips the unmatched instead. Decompiler.definitionLimit puts both cases in existing fixtures: above it private functions are never decompiled so their stubs cannot match, below it every stub must. The second case is the one that can rot silently - the golden fixtures compare decompiler text and assertParseable only looks for parse errors, so matching could break entirely and leave a green suite.
…ypespec highlights Reviewing KronicDeth#1397 found the reported crash already fixed: 36887da (2022-05-21, shipped in v13.1.0) added a DotCall case to ModuleAttribute's type-highlighting dispatch, ahead of the catch-all cannotHighlightTypes() the original report reached. Typing toward `String.t()` passes through `String.()` once the IDE closes the parenthesis, and that parses as an ElixirUnmatchedDotCall, which nothing in the suite covered. checkHighlighting alone would only pin that the annotator does not throw, because Highlighter enforces attributes rather than naming a key and the expected-highlighting markup has no key to match. The dot call's argument is asserted through doHighlighting instead, so a branch that swallowed the element and highlighted nothing fails too.
KronicDeth#1387 reported that EEx files were not understood and that "Embedded Elixir" is absent from Template Data Languages. Both are the design working: onlyTemplateDataFileType strips the .eex suffix and asks what the remaining name is, so page.html.eex gets HTML and a bare page.eex falls back to plain text; the dropdown omits EEx because eex.Language implements TemplateLanguage. Nothing pinned that derivation - tests/org/elixir_lang/eex/ held only lexer look-ahead tests, so a regression would have been silent. The two derivation cases are each other's control. Found by a review of the open issue backlog.
A variable bound in one EEx tag and used in a later one, inside the same `<%= if ... do %>` ... `<% end %>`, resolves in .eex, .leex and .heex. Shipped in v11.11.0 and untested since - nothing asserted variable resolution in any template language, so a regression would be silent. Pinned beside it is the gap KronicDeth#1793, KronicDeth#1414, KronicDeth#1509 and KronicDeth#1661 report: Variable.execute has an arm for ElixirEexTag and none for ElixirEex, so an ElixirEex handed to it is logged rather than walked for declarations. The working path routes around that through ElixirEex.processDeclarations, which is why the second test drives the processor directly and why the reports are so sparse. It inverts when the arm is added. The markup before the first nested tag is load-bearing: without template data there the block does not parse at all. Asserted as a clean parse so the test cannot pass for the wrong reason. Found by a review of the open issue backlog.
The fixture grid keeps a matched declaration/usage pair per definition kind and the guard pair was the only one missing, so completion of a user-defined guard rested on reading CallDefinitionClause.`is` rather than on a run. Reverting that predicate to `isFunction || isMacro` fails all three of these and nothing else.
Emmet picks its context from the language of the leaf under the caret, so in a multi-rooted template file whether it fires depends on which root claims that offset - not on which IDE is running. A bare `.eex` has no base extension to derive a data language from and falls back to plain text, which is the one shape where Emmet genuinely has no context. Emmet is reached through the customLiveTemplate extension point rather than by naming ZenCodingTemplate: its classes sat in intellij.xml.impl up to 2025.3 and moved to their own intellij.xml.emmet module in 2026.1, which is not on the compile classpath. It is still loaded at runtime, so the extension point finds it on every supported platform. The plain-HTML case is a control: without it every other assertion here would pass for the wrong reason if Emmet were absent.
The configuration once passed a bare `-S mix`, which the child process resolved through PATH and so picked a version-manager shim rather than the SDK's own mix. Nothing else in the suite builds a run configuration's command line, so that argument was unasserted.
Reformatting, the Enter handler, quote auto-closing and HTML tag/attribute completion all resolve through the data language the template derives, and none of them was asserted anywhere. Each case is paired with the same gesture in a plain .html control, so a harness limitation cannot read as a defect. .leex is covered too, since it is a separate file type on the same EEx language.
EexDataAstFactory is registered for the Elixir language and keys on EEX_DATA, which the EEx lexer emits as well as the HEEx one, but every existing case is a .heex file or a ~H sigil. Without outer leaves the Elixir and HTML roots both answer at a markup offset and the winner follows an unordered language set.
One name bound twice in two map patterns of the same clause head is a reported rename crash shape that no matrix fixture covered. The second binding matches against the first, so both occurrences are one variable and must rename together from either caret.
A version manager installs Hex under its own Elixir install, not `~/.mix`, and only exports MIX_HOME inside the shim's exec-env, which these command lines bypass. SdkPathsTest pinned the derivation but nothing pinned that Mix.commandLine applies it, so dropping that call left the suite green.
The two existing dedup tests enqueue the same request repeatedly, so they pin Set semantics and nothing more. The reported freeze was ~90 different deps, each of which used to get its own background task and its own blocking write action, and nothing asserted that the whole burst is now a single snapshot-and-clear. Capping the drain at two requests per pass fails this and nothing else in the class - every other test enqueues at most two. The fix itself cannot be reverted to check, because it deleted the code that failed rather than changing it.
…decompiled dispatch 09df8ba fixed KronicDeth#1901/KronicDeth#1603 in v11.12.0 by adding the CallDefinitionHead case to Any.isDecompiled(), and nothing has referenced isDecompiled or SourcePreferredItems since - reverting that case fails no assertion in the suite. A second head for one module/name/arity is what sends the list through the dispatcher, so two defdelegates are enough and no .beam fixture is needed.
Nothing asserted the direction KronicDeth#1613 reports - a third-party call site reaching a declaration made by defdelegate, shipped in v11.11.0. The gesture and multiResolve disagree here, so both are pinned.
ab52f10 fixed External Libraries labelling every SDK root `ebin`, but nothing pinned it -- the commit's own tests cover the Mix-dep class-root split instead.
…overflowing KronicDeth#1648's StackOverflowError was Using recursing on itself: 027ded2 recorded each resolved call definition clause on the visited set and recursed without checking it, and had already shipped in 11.4.0, the version that crashed. 164e5bd added the check and first shipped in v11.11.0; nothing asserted it. `git revert 164e5bd` does not apply - all three of its files were rewritten in the five years since - so this is verified by a targeted disable instead: deleting the one `.filter { !resolveState.hasBeenVisited(it) }` expression the commit added that survives verbatim on main. UsingTest is then the only failure in 7075, with a StackOverflowError of 337 Using.treeWalkUp frames.
A variable in an EEx tag has no declaration to find, so its use scope is empty. KronicDeth#1831, KronicDeth#1849, KronicDeth#1851 and KronicDeth#1772 all report the plugin saying "Don't know how to find variable use scope" on ElixirEexTagImpl instead, fixed by 85c0951 and shipped in v14.0.1 with nothing asserting it since. The cond block wrapping each tag is load-bearing: isVariable answers false for a tag at file scope, so variableUseScope is never consulted there and a test on a bare top-level tag would pass with the EEx case removed.
The platform skips its stub-index content-length comparison for binary files, which is what keeps recompiled .beam artefacts under _build from raising "Outdated stub in index". Nothing asserted the registration.
… module An alias inside a module whose own name starts with the aliased name finds only that enclosing module in scope, as an invalid prefix match. Resolution must fall through to the module index to reach the exact sibling; nothing pinned that, and reverting the check turns these red.
Quick Documentation had no coverage for a call whose qualifier is a short name bound by an alias directive, which is the case that needs the alias followed before any @doc can be found. The `as:` case is here because a plain alias's short name is also a suffix of the module it names, so a suffix match could satisfy that test without following the alias at all; a renamed qualifier names no module and leaves no such escape.
…s once 6e2f11f added the groupBy in resolver.Module.expand() and shipped in v12.0.0; nothing asserted it. Removing it makes the shared `use` and `alias` calls come back once per path the walk took, six results where four are correct. Only testEachElementResolvedOnce is red-proofed that way. The other two pass with the dedup removed as well, because the duplication is deterministic - they guard against a regression to unstable resolution rather than reproducing one.
A module defined in both project source and a dependency must resolve to the project's copy, so Go To Declaration lands in editable code. Nothing covered that: the neighbouring source-over-decompiled preference has a test, the same-module step did not.
Request resolution and plan building take separate read actions, so a deps/ handle that passed the first check can be invalid by the time the second reads its children, which throws InvalidVirtualFileAccessException. A control case with a live handle keeps that guard from being satisfied by never reading the children at all.
…ader Any file named *.beam reaches the decompiler, so an empty or truncated one is ordinary input rather than a fault to report. Nothing asserted that, and the guard that keeps it quiet is a single null check that reads as redundant. Removing it makes both cases log an error again, which is what users saw before it was added.
`isValidSdkHome` is a `File.canExecute()` on the path `getExecutableFilepathWslSafe` builds, so the choice of `bin/erl` over `bin/erl.exe` for a WSL UNC home is the whole of its WSL-awareness and nothing covered it. Off Windows this would pass for the wrong reason twice over: `WslPath.isWslUncPath` short-circuits on `WSLUtil.isSystemCompatible`, and both branches agree when `OS.CURRENT` is not Windows. The fixture's WSL-ness is asserted rather than assumed, and a sibling test pins that the two conventions genuinely differ. `testErlangSdkFileChooser_ValidatesWslPaths` read as this coverage while asserting only the descriptor's folder and file flags, so it is renamed for what it checks and points at the new test.
…terminates KronicDeth#1861 and KronicDeth#2015 report "StackOverflowError when annotating Call" from resolving an unqualified, argument-less call. Their traces are captured at the catch site, so they name the entry point rather than the cycle; the cycle is the walk over call definition clauses reached through use/__using__, which recorded each resolved clause as visited without checking the record before recursing. 164e5bd added the check and shipped in v11.11.0, but nothing asserted it from the entry those reports name: existing coverage calls Use.treeWalkUp directly, while the reports arrive through Callable.multiResolve. resolveInScope swallows the StackOverflowError and returns an empty list, so the logged title is the only observable and is what the test asserts on. Removing that one filter turns the cyclic test red on the reported artefact while the control - the same kind of call in a module with no use - stays green. Removing the later RecursionManager.doPreventingRecursion wrap instead changes nothing here: it is keyed on the element passed to it, and this re-entry runs through a succession of different elements inside one resolveInScope call.
…et distinct module names Project.createModuleForOtpApp builds each .iml path from the name it is handed, and the platform derives the module name from that path, so two OTP apps mapped to one name become two identical paths and the second newModule call throws ModuleWithNameAlreadyExists. That was the crash on importing an umbrella whose root directory and one apps/ child declare the same app:, fixed in v23.0.5 by disambiguating in moduleNameForOtpApps -- with nothing covering it since. The uniquely-named sibling asserts the disambiguation stays off apps that do not collide, so mapping every app straight to its own name fails only the collision case.
SdkDecompileParseableTest sweeps the same beams but stops one step short of this: mirror mapping runs after parsing, so an exported definition the decompiled source never produced a matching clause for parses fine and passes that sweep silently. It surfaces only at runtime, as a "No decompiled source function with name" warning and a navigation target that goes nowhere. Unexported definitions are excluded deliberately - compiler-generated comprehension helpers are never emitted into decompiled source and setMirror skips them, so a mirror-less unexported definition is ordinary. Both SDKs are currently clean: Elixir 5079/5079 and Erlang 43255/43255 exported definitions get a mirror. Reuses SdkBeams, so beam discovery is already paid by the parseable sweep; the marginal cost is the mirror build, about 8s on top of that sweep's 48s.
… terminates KronicDeth#2015 reports the overflow from annotator.Callable, which the annotator extension point drives over every plain call; the sibling cases enter at Callable.multiResolve, below that door. Running the highlighting pass over the same fixture covers the entry the report names. Removing the visited-element filter from the use walk turns this case red on the reported title while the control - the same call in a module with no use - stays green, so the door does reach what 164e5bd guards.
prependQualifiers matches each ancestor it walks against a when, and an EEx tag was missing from it, so an alias in `<%= ... %>` fell to the catch-all and came back prefixed "?." - a name that resolves to nothing. Nothing exercised a QualifiableAlias inside an EEx tag, so the guard added for KronicDeth#3278 was unpinned.
…f throwing ElixirDocumentationProvider had a bare TODO() where the Deprecated section should render, so asking for docs on any function whose docs chunk carries deprecated metadata threw NotImplementedError. Shipped in v13.1.1 by "Implement Deprecated metadata handling for docs from BEAM files", untested until now. Regression test for the crash reported in KronicDeth#2412. :queue.lait/1 in the existing OTP 27 fixture already carries that metadata as an OtpErlangBinary, so no new BEAM was needed.
… terminates Deciding whether a `test` generated inside a `for` comprehension is an ExUnit.Case child resolves that call, which walks back up into the same `for` and down into its children again. Two generated tests are needed to close the loop: with one, the only element the walk would revisit is the entrance the resolve state was seeded with. Removing the RecursionManager guard on resolveInScope puts this fixture back on the errors reported in KronicDeth#3405, which nothing else on the path prevents - both hasBeenVisited guards read a ResolveState, and the reference boundary the cycle crosses carries none.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A long-lived collector for the regression tests written during the 2026-08 open-issue review. Each commit pins behaviour that already ships but that nothing asserted, so a revert of the original work would have left the suite green. One commit per issue reviewed.
Carries
no-changelogdeliberately:.github/workflows/changelog.ymlnames "a test-only fix" as the first of its two exemptions.Every row whose original fix can still be reverted and built was checked to pin rather than merely pass — the fix is reverted and the new tests must fail, and must be the only ones that do. Details are in each commit message.
Four rows are outside that guarantee, because no revert applies or builds — the fix deleted a file, or restores a line that no longer compiles, or moved since, so the revert conflicts modify/delete, or the files were rewritten around it so the revert conflicts on content. Each was checked by a narrow mutation of the mechanism instead, and the bar is that it fails the new test and nothing else — a broad mutation that also fails existing tests proves nothing about added coverage:
RenameMatrixTest#testVariableMapPatternRepeated— reverting the removal of the legacy in-place rename handler restoresHandler.kt, which no longer compiles against current source. That row pins the reported code shape through the replacement path instead.MixDepsSyncServiceTest#testEnqueue_manyDistinctDepRootsInOneBurst_drainInASinglePass— f4e399504 deletedDepsWatcher.ktandmix/Watcher.ktoutright. Capping the drain at two requests per pass fails this test and no other in the class — 25 pass — because every other test there enqueues at most two requests. Breaking theSetaccumulator instead also fails five existing tests, which is why that check was discarded.Issue1613Test— 811753ac3 added code, but 4818be66c later moved the files out ofgen/, so the revert conflicts modify/delete against paths that no longer exist. Passingfalsefor the head’svalidResultinMultiResolve.executeOnDelegationfails this test and no other — 1 of 7070. Deleting theaddToResolveResultscall instead fails 15, because it also skipskeepProcessing()and changes the scope walk, so that check was discarded.UsingTest#testTreeWalkUpCyclicCallDefinitionClauseChain— 164e5bda9 added guards toUsing.kt,QuoteMacro.ktandResolveState.kt, all three of which were rewritten in the five years since, so the revert conflicts on content in every file. Deleting the one expression it added that survives verbatim onmain,.filter { !resolveState.hasBeenVisited(it) }inUsing, fails this test and no other — 1 of 7075, with aStackOverflowError.Tests so far
UnicodeTokenTestVarBangDeclarationTestvar!(foo) = 1in aquoteclassifies as a variable declaration;x = var!(foo)does notDebuggedConfigurationTestMIX_ENVdefaulting,isPassParentEnvsand-name/-setcookieall survive the copy Debug actually launchesIssue926Testtemplates/page/index.html.eexresolves intoviews/page_view.exAutoIndentLinesTestModuleAttributeTest#testIssue1194@typeunion highlights instead of crashing the annotatorModulesTestrequire_filepaths escape Windows backslashes, and round-trip to real filesStubMirrorTestdefinitionLimitModuleAttributeTest#testIssue1397String.()— a dot call with empty parens — highlights instead of crashing the annotatorTemplateDataLanguageTestpage.html.eexderives HTML, barepage.eexfalls back to plain text, and EEx is excluded from the Template Data Languages dropdown by designIssue1793Test.eex,.leexand.heexCallDefinitionClauseTest,VariantsTestdefguardclause completes at its use site — qualified and unqualified — anddefguardpstays out of remote completionEmmetApplicabilityTestpage.html.eex— plain markup, inside a<%= ... do %>block, between two tags, inside a tag with an EEx attribute value — and unavailable in a barepage.eex, whose data language is plain textiex.MixTestmixrather than a bare-S mixthe child process would resolve throughPATH— picking a version-manager shimEditorServicesTest.htmlcontrol, in*.html.eexand*.html.leexEexDataOuterLanguageElementTest.html.eexfile's Elixir root gets outerEEx Dataleaves, so the HTML root wins the caret at a markup offset instead of whichever the view provider's unordered language set yields lastRenameMatrixTest#testVariableMapPatternRepeatedMixTestmixcommand line carryMIX_HOME=<sdk home>/.mixandMIX_ARCHIVESbeneath it, somixfinds the Hex archive the version manager installed rather than looking in~/.mix; a home no version manager owns leaves both unsetMixDepsSyncServiceTest#testEnqueue_manyDistinctDepRootsInOneBurst_drainInASinglePassSetsemanticsGotoSymbolContributorTest#testIssue1603CallDefinitionHeadfor one module/name/arity goes through the source-preferred decompiled dispatch instead of throwingNotImplementedErrorIssue1613Testdefdelegate: Go To Declaration lands on the delegateddef, and thedefdelegatehead resolves as a declaration in its own moduleElixirSdkLibraryNodeDecoratorTestebin,liborsrc— is labelled in External Libraries by the OTP application that owns it (iex,elixir,stdlib-7.1), while anebinoutside any SDK root keeps its own nameUsingTestuse/__using__chain reaching a cycle of mutually recursive call definition clauses terminates instead of overflowing, and still yields the definitions the non-cyclic clause injectsIssue1831TestDon't know how to find variable use scopeat the user and namingElixirEexTagImpl— covered as a bare tag, as a call's qualifier, and as a call argumentbeam.FileTypeTest.beamis registered as a binary file type — the one plugin-owned property that makes the platform skip its stub-index content-length comparison, so a.beamunder_buildrewritten bymix compileno longer raisesOutdated stub in indexf6953a69eModuleGotoDeclarationTest(prefix-sibling cases)alias MyModule.Internalinsidedefmodule MyModule.InternalTestresolves to a single target, the sibling module, not the enclosing one whose name merely starts with the aliased name|⚠️ |
FunctionCallQuickDocumentationTest(alias cases) | Quick Documentation on a call whose qualifier is a short name bound byalias, and byalias ..., as:, renders the declaring module's@doc— the aliased case the suite qualified by real module name only | Measured on v24.0.1; documentation moved onto the alias-aware reference path in fe5643b69 (v11.11.0) | #1806 ||
DuplicateResultsThroughUseTest| A module reachable by two paths through auseresolves to each element once: the sharedaliasandusecalls the walk records are collapsed rather than repeated per path | v12.0.0 · 6e2f11f49 | #1778|
PreferProjectOverMixDependencyTest| A module defined both in the project's own source and in a Mix dependency resolves to the project's copy — for a qualified call and for a module alias — so Go To Declaration lands in code the user can edit rather than indeps/| Measured on currentmain; the same-module preference shipped in v11.11.0 | #1814 ||
MixDepsSyncServiceInvalidatedDepsRootHeavyTest| Adeps/handle invalidated after its sync request was coalesced yields no library plans instead of throwingInvalidVirtualFileAccessExceptionout of the read phase, plus a live-handle control that still plans the dep | 24.0.0 · f4e399504 | #1815 ||
beam.HeaderTest| A.beamtoo short to hold a four-byte header — empty, or truncated mid-header — is skipped silently rather than reported as an error, so an editor scratch buffer or a 0-byte placeholder resource that merely ends in.beamcosts the user nothing | v11.1.0 · 88cc56783 | #1829 ||
cli.CliToolWslSafeTest|getExecutableFilepathWslSafebuildsbin/erlfor a\\wsl$\/\\wsl.localhost\SDK home and the.exe/.batform otherwise. That choice is the entirety ofisValidSdkHome's WSL-awareness — it is aFile.canExecute()on the filename this builds, so the wrong one rejected every WSL home | v23.0.4 · 7498afd03 / 224fdd224 | #1911, #2499, #3470 ||
CyclicUseTest| Resolving an unqualified, argument-less call from a module whoseusechain contains a cycle terminates instead of overflowing — entered throughCallable.multiResolve, and through the highlighting pass that drives the registeredannotator.Callable, the two doors the crash reports name, rather than through the tree walk directly | v11.11.0 · 164e5bda9 | #1861, #2015 ||
mix.ProjectModuleNameForOtpAppsTest| An umbrella whose root directory and oneapps/child declare the sameapp:gets two distinct IntelliJ module names, so the two.imlpathscreateModuleForOtpAppbuilds from them no longer collide; a uniquely-named sibling keeps its plain name | v23.0.5 · fc3c45133 | #2003 ||
beam.SdkMirrorCoverageTest| Every exported definition in every.beamthe resolved Elixir and Erlang SDKs ship decompiles with a mirror (5,079/5,079 and 43,255/43,255 today).SdkDecompileParseableTestsweeps the same beams but stops one step short — mirror mapping runs after the parse, so an exported definition the decompiled source never produced a matching clause for parses fine and passes that sweep silently. That gap is how 49 No decompiled source function with name reports reached users past an existing test. ReusesSdkBeams, so beam discovery is already paid; the marginal cost over that sweep is ~8s on ~48s | v23.0.7 · bab755e6b9 (__struct__/1) and v24.0.1 · #3850 (Elixir.Stringlexer) | #1910 + 48 others ||
EexTagFullyQualifiedNameTest| AQualifiableAliasinside an<%= … %>tag, in both.eexand.html.eex, qualifies to its own name instead of falling toprependQualifiers' catch-all and coming back prefixed"?.". Reverting the guard fails all three cases with the reporters' ownElement Class Name: org.elixir_lang.psi.impl.ElixirEexTagImplbody, sincePlatformTestCase's logger turns the catch-all'sLogger.errorinto a failure | v15.1.0 · ca0b1ae55 | #3278 + #2414, #2468, #3658 ||
ErlangAtomQualifierHoverDocumentationTest#testDeprecatedFunctionHoverShowsDeprecatedSection| Quick Documentation for a function whose BEAM docs chunk carriesdeprecatedmetadata renders aDeprecatedsection instead of throwingNotImplementedErrorfrom a bareTODO(). Reinstating theTODO()fails this case with the reporters' ownkotlin.NotImplementedError: An operation is not implemented.and leaves the two existing cases in the class green. Needed no new fixture: the OTP 27queue.beamalready inerlang_atom_qualifier_hovercarriesdeprecatedmetadata for:queue.lait/1as anOtpErlangBinary, the one shape the renderer converts rather than logs for | v13.1.1 · 362973da6 | #2412, #2569, #2704 ||
Issue3405Test| Resolving a call inside an ExUnittestgenerated by aforcomprehension terminates instead of overflowing. Two generatedtestcalls are needed to close the loop: with one, the only element the walk would revisit is the entranceresolveResultsseeds the state with. Red-proved by removing theRecursionManagerguard onresolveInScope, which puts the fixture back on the reported errors | v23.2.0 · abc6e0e8e | #3405 |VariableSymbolclassifiesvar!correctly, but the issue asks about resolution, a different path: a use offooaftervar!(foo) = 1resolves to zero declarations. The failing test is written and deliberately not on this branch, which has to stay green. The fix belongs inMultiResolve.templates/viewsdirectory names and ignores a customroot:passed toPhoenix.View.Variable.executehas an arm forElixirEexTagand none forElixirEex, so it logsDon't know how to resolve variable in matchand never walks the subtree. The second test drives the processor directly and inverts when the missing arm is added, so the fix is a visible change to this file rather than a silent one.Four notes worth keeping
StubMirrorTestis the one that could have failed silently. The golden fixtures compare decompiler text,assertParseableonly looks for aPsiErrorElement, and the mismatch cases assertunmatched.isNotEmpty()— so name/arity matching could break entirely, killing navigation and completion in every decompiled.beam, with a green suite. Each mismatch case asserts the gap still exists first and namesdefinitionLimitin its failure message, so raising the limit repoints the case rather than deleting it.EditorServicesTesthad to be reverted across JVMs, not within one. The view provider's root order is a function of threeLanguagesingletons' identity hash codes, so it is fixed for a JVM's life and reruns inside one run re-measure a single draw. Removingab808db09'slang.ast.factoryline and looping across Gradle invocations gives 7 failures in 12 runs, against 12 of 12 passing with it restored. The two reformat cases stay green in every failing run — which is how the test separates the two causes behind the indentation report, only one of which shipped in v11.8.0.testIssue1397is checked in three states, not two. With theDotCallcase present, 15/15 pass; replaced byis DotCall<*> -> Unitit fails with0 HighlightInfo(s); removed entirely it fails with the reported crash. The first version of this test covered only the third state and passed the second, so it read as coverage it did not have.Two defects were found while writing these and filed separately: #3956 (an EEx
doblock with no template data before its first nested tag never terminates) and #3958 (no Find Usages type label for adefguard).The alias Quick Doc cases were red-proved against alias handling, not against a commit. The planned proof — removing the two
isValidResultguards — was abandoned unrun after reading current source: theCallbranch ofgetCustomDocumentationElementhas been rewritten since fe5643b69 and takes the first matching clause with an exact-name fallback rather than the 2021singleOrNull, so no filter removal can produce the predicted null, and neither guard is alias-following in the first place. Insteadorg.elixir_lang.psi.scope.Modulewas mutated: makingexecuteOnAliasCallArgumenta no-op fails the plain-aliascase only (1 of 5), and additionally neuteringexecuteOnMaybeAliasedNamefails both new cases (2 of 5) —alias ..., as:reaches scope by the second path, which is why one mutation does not cover both. The three pre-existing methods stay green under both, separating "documentation broke" from "alias-following was removed". This shows the tests detect loss of alias-following; it does not measure which commit supplied it, so the release column above says where the behaviour was measured rather than claiming a fixing commit.ModuleGotoDeclarationTest's prefix-sibling cases reverse an attribution. The first version passed against source with bothisValidResultguards disabled, which is what exposed the error: for this shape the in-scope walk returns only the invalid prefix match, soResolver.preferredhas nothing valid to prefer and its filtering is a no-op. What actually fixes it is 380184adf (v12.0.0), taking an in-scope set only when it holds a valid result so an all-invalid one falls through to the module index. Reverting that one line gives 2 failures in 16. Anyone comparing this against the review's dossier will find it cites 5d09b3b0c / v11.11.0 instead — the dossier is wrong, and reading the code alone could not tell the two apart.PreferProjectOverMixDependencyTestred-proves the mechanism, and still names no fixing commit. MutatingResolver.preferElementUnderSameModuleto return its argument unfiltered fails both new cases, and the failure text names the dependency's own file — so the dependency copy was a live candidate the resolver rejected, not one that never reached the list, which is the difference between measuring the preference and measuring nothing.PreferSourceOverDecompiledTest's eight methods stay green under that same mutation: samepreferred()function, neighbouring step, so a mutation that broke resolution generally would have taken them too. The test carries aProjectFileIndex.isInLibraryguard so a future vacuous pass fails instead. As with the cases above, this measures that currentmainprefers the project definition and cannot show which commit supplied it, so the release column cites the mechanism rather than claiming 5d09b3b0c.MixDepsSyncServiceInvalidatedDepsRootHeavyTesthad to skip the outermost entry point to mean anything. Driving it throughenqueue+drainwould have been vacuous:resolvePathShapedRequestalready discards an invalidSyncRequest.DepsRoota layer above, so the request never reaches the guard inbuildSyncPlanand removing that guard would not have gone red. The guard covers a different window — request resolution and plan building take separate read actions, so a handle valid at the first can be invalid at the second. The test invalidates the directory between the two layers, with every argument still coming from production code. Removing the guard fails the case with the reportedInvalidVirtualFileAccessExceptionand itsoriginal:N; found:-; File.exists()=falsesignature; the control stays green.HeaderTest's red run reproduces the reported message verbatim. Deletingheader != nullfromBeam.fromfails both cases withheader typeID (null) did not match expected (FOR1) from Elixir.Empty.beam. There are 0 bytes available on the dataInputStream. File size is 0 bytes.— the text of #1829, clause for clause. That turns a close resting on dating a commit into one resting on a demonstration.PlatformTestCase.captureLoggedErrorsis what makes “logged nothing” an assertion rather than an unobserved absence: its default action set rethrows, so an error logged here fails the test on the error itself.CliToolWslSafeTesthad two ways to pass for the wrong reason off Windows, not one.WslPath.parseWindowsUncPathreturns null unlessWSLUtil.isSystemCompatible()(= SystemInfo.isWin10OrNewer), so on a Linux leg the WSL branch is never entered; and off WindowsOS.CURRENT != OS.Windows, so both branches return the same unsuffixed name and an equality assertion cannot discriminate at all — deleting the WSL branch outright would not move it. Handled withWSLUtil.setSystemCompatible(true)(@TestOnly, saved and restored per test) plus an explicitassertTrue(WslPath.isWslUncPath(fixture))precondition, which is what turns a silent wrong-branch pass into a loud failure — it is the assertion that fires, with that message, when the guard is removed. Two mutations, each with its own message: neutering the branch givesexpected:<...binerl[]> but was:<...binerl[.exe]>,setSystemCompatible(false)gives the precondition.testNonWslHomeGetsTheCurrentOsConventionstays green under both. Note the production call reachesWslPathdirectly rather than throughWslCompatService, so the WSL suite'sMockWslCompatServicenever touches it — which is how this survived a 1,400-line WSL suite. The same commit renamesElixirWslSdkTest.testErlangSdkFileChooser_ValidatesWslPaths, which claimed this coverage while asserting only the descriptor's folder and file flags.CyclicUseTestreverses which of two guards is load-bearing, and the reversal is the reason it exists. The dossier readUsing/Import/Modular's 2021hasBeenVisitedchecks and the 2026abc6e0e8eRecursionManager.doPreventingRecursionwrap as two stacked layers protecting this path. Measured, the wrap protects nothing here: removing it alone leaves both cases green, and during the mutation that does go red the wrap was present and the overflow happened anyway — it is keyed on the element handed to it, while the re-entry runs through a succession of different elements inside oneresolveInScopecall. So the release cited is v11.11.0, not v23.2.0. Neutering the onehasBeenVisitedfilter reproduces the reported artefact exactly — a loggedStackOverflowError when annotating Callunder categoryorg.elixir_lang.reference.resolver.Callable— while the control, the same kind of call in a module with nouse, stays green under that same mutation. The logged title is the only observable available:resolveInScopeswallows theStackOverflowErrorand returns an empty list rather than rethrowing. Both reported doors are pinned, not just the shared mechanism:multiResolvedirectly, anddoHighlightingdriving the annotatorplugin.xmlregisters, which goes red under that same mutation — so the annotator entry is shown to reach the guard rather than assumed to. This complementsUsingTest, which drivesUse.treeWalkUpdirectly; no entry covers another.ProjectModuleNameForOtpAppsTestcould not be red-proved by reverting its commit. Reverting fc3c45133 deletesmoduleNameForOtpAppsoutright, so the test stops compiling — a red that measures nothing, and one no control assertion can survive. The mutation keeps the signature and restores the pre-fix mapping instead,otpApps.associateWith { it.name }, which is whatcreateModuleForOtpAppdid when it built the path as"${otpApp.name}.iml". Under it the collision case fails and the uniquely-named sibling stays green, separating "the disambiguation was removed" from "app-name parsing broke". The fixture is three realmix.exsfiles, soOtpApp.namecomes from the same PSI read production uses.