Skip to content

2026.11.19.1: rule convergence -- the descriptor's two forms, the version group in the merge, link inputs, compiler-run state, transactional publication, actions, text as UTF-8, a resolution per build program, path dependencies in the cache (mcpp#931, #952 to #960, #965, #967, #970, #972, #974) - #966

Merged
Sunrisepeak merged 30 commits into
mainfrom
eco/rule-convergence
Nov 19, 2026

Conversation

@Sunrisepeak

@Sunrisepeak Sunrisepeak commented Nov 19, 2026 •

Copy link
Copy Markdown
Member

The round recorded in .agents/docs/2026-11-19-ecosystem-rule-convergence-design.md, released as mcpp 2026.11.19.1. One pull request; its commits are checkpoints, and each carries its specification clause, the usage chapter in both languages, its tests and its CHANGELOG entry.

Checkpoint Content Issues
1 design record: the rules behind the issues, five causes, seven decisions, the round's scope
2 a descriptor's mcpp field has two forms, read by one function; publish writes the pointer form mcpp#960
3 the version group's constraint enters the merge; a package's tables are read by the row it is built for mcpp#956
4 whether an artifact runs here is one predicate with no branch before it mcpp#957
5 a link's inputs are the files its words name and the files its linker reports reading; a project's own link dependency file is kept mcpp#931
6 a row a member inherits from its workspace is the root's statement mcpp#954
7 an action reads a link unit's objects from the engine; a plugin's feature carries its own program mcpp#959, mcpp#955
8 a package instance has a directory of its own mcpp#953
9 a build interrupted in another process space continues; a start refused for its size says so mcpp#952, mcpp#958
10 a compile writes only its own attempt, and the executor publishes what it wrote; a published output survives a transient hold on Windows
11 the default platform is openkal-platform 0.31.0; the tests run under mtest 0.10.0
11b cl.exe's /sourceDependencies record names what a compile read; the text of every program is decoded where it is read (mcpplibs.os 0.4.6); every JSON document replaces what is not UTF-8 mcpp#970
12 version 2026.11.19.1
13 the pack launchers build on the mcpplibs.os release the platform's build program links (0.4.5): one resolution held the consumer's graph and the platform's build-program dependencies (superseded by 19) mcpp#972
14 cl.exe's record reaches ninja as mcpp.inputs: lines read under deps = msvc; every JSON record of the build is UTF-8 text; a response file on a link line is split by the rules of the tool that reads it; the advice for a lost header reads C1083 in any quote marks mcpp#970, mcpp#931
15 the Windows bare-metal llvm rows build (llvm 23.1.3 revision 4 carries ld.lld.exe); e2e 663 reads the std record its plan names
16 a path or git dependency's build is cached under the name of its sources: the commit, or the digest of the tree's content; the project's own trees are built in place mcpp#974
17 a base's C++ layer and the graph's layers reach a clang driver with no cfg beside it mcpp#965
18 a row the platform column misses is an index miss, refreshed as a dependency's is mcpp#967
19 each build program resolves its own build dependencies: a program takes the newest version the build holds that its request admits, two programs may hold two versions (two build programs' graphs, a +b lock entry naming its programs), a conflict inside one program is refused naming it; a platform series skips a release whose version group the build does not admit; the pack launchers return to mcpp's own mcpplibs.os (0.4.6, checked), superseding 13; the global dependency cache, the rename's staging and the provisioning closures move to files of their own to keep every file and step under its limit mcpp#972
20 a workspace member is the project's own tree whichever members a command selects: a member reached through path from the selected one is built in place, so its arguments do not depend on the selection; SPEC-001 §10.4 states every member mcpp#974
21 a cache fill completes under the goals a command names: the fills are a phony goal every named goal set includes
22 build programs resolve after the target, in waves, and keep the versions they hold while their requests admit them; a merge that replaces a version walks the new one as the first and withdraws the old one's requests and build program; a package of the target takes its capability's default from the target's graph (SPEC-002 §3.6); a series restart no call answers is dropped; a development dependency's program does not move the build's programs to another graph (review of 19, four e2e cases that fail before and pass after) mcpp#972

Dependencies, each released and indexed: openkal 0.26.0, openkal-platform 0.31.0, mcpplibs.os 0.4.6, mtest 0.10.0.

… causes, seven decisions and the round's scope (for review)
…h writes the pointer form (mcpp#960)

- mcpp publish and mcpp emit xpkg write mcpp = "*/mcpp.toml": the archive is the
  package's source tree and its manifest states the modules and the dependencies
- mcpp::pm::descriptor_manifest decides the form once for the build's layout
  check, the pre-install dependencies, the manifest, the patch root, mcpp new
  --template and mcpp xpkg parse
- a table that names a manifest is refused with the line to write in its place
- mcpp new --template refuses a package for the templates/ directory it lacks
- SPEC-001 §3.5, docs/11 in both languages, the round trip as a unit test and
  as e2e index::local/960
…re read by the row it is built for (mcpp#956)

- a request the platform's version group binds carries the group's version into
  the resolution record and every merge, so a build program's request of the
  same package is answered by one version in whichever order the two arrive;
  a refusal lists every request and its requester (SPEC-013 §7.5)
- a package reached only through build dependencies reads its
  [target.<selector>.dependencies] by this machine's row; a package both sides
  ask for makes each side's requests, and mcpp.lock records both (SPEC-004 §4.8)
- the build programs' graph is given a project's local index by its resolved
  path, where a relative one was copied as written
- e2e openkal::platforms/956 (two cases), docs/05 and docs/35 in both languages
…it (mcpp#957)

- the branch that refused macOS and iOS rows off macOS, whatever runner was
  declared, is removed: those rows are decided by the row's execution system
  and the runner or execution host stated for it, as every row is
- the system-names check counts a triple's system predicates, so a branch of
  that kind raises a count above its baseline
- SPEC-016 §6, docs/04 in both languages, unit tests of the predicate
…nd is not reported as the member's own (mcpp#954)
… feature carries its own program to every build program that links it (mcpp#959, mcpp#955)
…y instance its commands ask for (mcpp#953)

A package's objects lie in obj/<package>/<digest>/ and its BMIs in
<bmi dir>/<package>/<digest>/. The digest covers what decides the package's
command lines: its include directories and flags, its features' defines among
them, and the digests of the packages whose modules it imports. It holds no
version and no compiler build. One build directory held one instance, so a
shared package that mcpp test or another selection compiled with other features
was compiled again at every switch; both instances are now kept.

instances.tsv records when a build last used each instance. mcpp clean --stale
removes the instances no build used within the window and keeps those of the
plan that wrote the graph; mcpp clean --report lists them. A GCC BMI records
its imports at their instance paths, so the build cache keys a package's entry
by its instance. A <package>@<digest> name was measured and not taken: ninja
quotes an output holding '@'.
…efused for its size says so (mcpp#952, mcpp#958)

An observation of a compiler unit has four answers: empty, occupied, not yet
decidable, and not observable from this process space (openkal 0.26, clause
7.18). An unobservable unit of a generation whose owner and helper locks are
free -- locks of the file system, shared by every process space that shares
the directory -- is settled as abandoned and reported once
(compiler-run/unobservable). A refusal for occupied or undecidable units names
their number, the state directory and the way out [compiler-run-occupied].
mcpp clean, --stale and --older-than retire a configuration's compiler runs
before removing it. The owner's refusal of an admission names its condition,
and every helper return other than the compiler's status or a cancellation
names the step that ended it. Where the system gives nothing to declare a unit
from, the build compiles without units, bounded by the generation's cap, and
says so once (compiler-run/no-unit).

mcpp builds against mcpplibs.os 0.4.5. A start the system refuses for the size
of its arguments and environment reads "the request is larger than the system
admits (N arguments, A bytes; M variables, E bytes) [spawn-too-large]".
…t it wrote (design 2026-11-19 §10.6)

Where a toolchain's outputs are relocatable, a compile writes below
<build dir>/.mcpp-attempts/<attempt>/ and the executor publishes by rename
once the compiler has ended, a module's BMI the moment it is complete; an
object or BMI equal to the published one is kept. The attempt is recorded
before its compiler is admitted, so recovery discards an interrupted build's
unpublished attempts without observing them: a compiler that outlives its
build writes nothing the next build trusts. GCC 11 and later relocate through
-dumpdir/-dumpbase and a per-attempt module map, the root package's module
units included; a clang driver relocates when it takes -dumpdir, which
detection asks of it once and keeps (driver_takes_dumpdir); CodeView targets,
NASM and the resource compilers write in place. SPEC-011 §5.1 clause 8.

Under clang's two-phase schedule a cache fill now asks for every BMI of its
package: a module no unit imported had no BMI and the fill was refused. The
tests follow the package instance layout (mcpp#953) and the link record
(mcpp#931): the module address test, graph on Windows, the 847 map lookup,
the 669 and 667 packs, and runners/330. The upgrade lane records 2026.11.19.1
as the last BMI spelling change. The role refusals of the action graph move
into one function, so the step stays within the prepare size bound.
@Sunrisepeak Sunrisepeak added ci:full Run the full e2e set on this pull request (read when a push starts the run) ci:canaries Run the canary matrix (canaries.yml) on this pull request labels Nov 19, 2026
…uild says what the link's record lacks (mcpp#931)

GNU ld and ld.lld write one dependency file, the last named, and the engine's option followed the project's [build] ldflags: a project that asked for --dependency-file got no file, with nothing said (found by the design record's reproduction 931d). When the project's link words name one, the engine adds no option of its own and reports build/link-report: the link's record then holds only the files its words name. SPEC-004 §8.1 rule 2, docs/30.
…asks the driver without creating a home

On a Windows runner a GCC BMI published from a cross build was refused once among some 350 ("being used by another process"): a scanner opens a compile's output the moment it is written, and Windows refuses a rename while the file is held. Publication retries such a refusal for up to 30 s (mcpp::home::replace_file_within). The -dumpdir probe ran at detection, which a query (self env) reaches too, and created the home's probe cache: a query now asks in a temporary directory and keeps no answer. The mtest form of e2e 667 follows the link record as the script did (mcpp#931), 669 accepts the runtime-closure refusal where the program's loader is private, and the response-file check of LinkWordInputs names what it read.
…der mtest 0.10.0

openkal-platform 0.31.0 states the openkal 0.26 family: a start bounded by the environment alone,
an observation of a unit that answers whether this process space can observe it, and a runner that
names the call each task of an image waits in when it is asked to end. mcpp's own tests and the
CI runners use mtest 0.10.0, which records a timed-out program's processes before it ends them.
The design record states the measurements of the replayed issues and two findings of the
implementation: a transient hold on a published file on Windows, and a probe that made a home.
…ram reaches mcpp as UTF-8 (mcpp#970)

A compile under cl.exe writes its /sourceDependencies record, and the
executor turns it into the Make rule every compile edge is read by
(deps = gcc): the headers of INCLUDE and EXTERNAL_INCLUDE are left out, a
path under the build directory is spelled as the build directory and the
case of each component on disk, and no line of the compiler's text is read
for a dependency. VSLANG is no longer set.

The text a program writes is decoded where it is read, by the dialect's
text encoding: UTF-8 for GNU, and for MSVC the output code page of the
attached console, else the system's ANSI code page (mcpplibs.os 0.4.6,
mcpplibs.os.text). Every JSON document mcpp writes replaces a sequence that
is not UTF-8 instead of failing. An MSVC diagnostic is read by its
structure, and C1083 names its header in any language.

Design 2026-11-19 section 20; SPEC-011 section 5.1 items 11 and 12;
SPEC-012 section 5.
@Sunrisepeak Sunrisepeak changed the title Rule convergence: the descriptor's two forms, the version group in the merge, link inputs, compiler-run state, actions (mcpp#931, #952 to #960) 2026.11.19.1: rule convergence -- the descriptor's two forms, the version group in the merge, link inputs, compiler-run state, transactional publication, actions, text as UTF-8 (mcpp#931, #952 to #960, #970) Nov 19, 2026
Both launchers `mcpp pack` writes (bin/<name>.exe and the single-file
launcher) named mcpplibs.os 0.4.4, whose openkal ends below 0.26; with
openkal-platform 0.31 as the platform of mcpp's openkal rows, a launcher
could not be built and e2e 988 failed. They name one constant,
kLauncherOs, the release mcpp builds on (0.4.6), and
.github/tools/check_version_pins.sh holds it equal to mcpp.toml's.
…exe's record reaches ninja as lines

- The pack launchers pin mcpplibs.os 0.4.5, the release the build program of
  openkal-platform 0.31 links through mcpp.plugins 0.26.1: one resolution holds
  the consumer's graph and the platform's build-program dependencies, so the
  launcher takes the platform's os (mcpp#972). The pin-equality invariant added
  in the previous commit is withdrawn.
- A compile under the msvc dialect gives ninja the files it read as
  `mcpp.inputs:` lines read under `deps = msvc`, so a path ninja's depfile
  parser would split (a quote, `&`, `;`, `^`, a backquote) arrives whole; a
  build directory spelled with non-ASCII names is matched by its respelling.
- Every JSON record the build writes (build-database envelopes, test archive
  manifests, run records, plan records, build-program graphs) is written as
  UTF-8 text.
- A response file on a link line is split by the rules of the tool that reads
  it: CommandLineToArgvW's under the MSVC dialect, a GNU driver's otherwise
  (mcpp#931).
- The advice for a lost header recognises C1083 whatever its quote marks and
  the spacing around its colon (mcpp#970).
…plan names

llvm 23.1.3 revision 4 carries ld.lld.exe in the Windows archive (design
2026-11-19 section 11.2), so i686-none-elf and x86_64-none-elf with llvm
build on windows-x86_64 where the table stated lld-required-absent (target
matrix scan of run 44531373098).

e2e 663 took the newest std-module.json that mentions libcxx from the home
the shard's programs share; an x86_64-openkal record written beside it
(openkal-llvm-runtime's llvm/libcxx/modules) was read in two runs of the
Linux shard. It now reads the record build.ninja names, as its mtest port
in toolchain::layers does.
…ces (mcpp#974)

A dependency's sources have one name, read by the global dependency cache,
the host tool store and the build programs' identity alike: an index
package's version, a git dependency's resolved commit, and for a path
dependency the digest of its tree's content (`<version>+path.<digest>`),
which leaves out target/, .git/, .mcpp/, compile_commands.json and nested
packages. The tool store's modification-time stamp is gone.

The project's own trees -- the root, the selected members, any package below
the build's directory or below the directory of a build above it (own_roots,
handed to build-program and host-tool sub-builds) -- are built in place.
Every other path and git dependency enters the global cache, so two projects
over one framework and a CI run that restores the build cache share one
build. A path tree is read again before its entry is written; one that
changed while it compiled is not stored and is reported as a degradation;
the entry the same tree left under its previous digest is removed. A cache
hit is reported under the package's qualified name, so it reads `Cached`.

SPEC-001 section 10.4, usage chapters 04 and 30 in both languages, design
record section 22, e2e bmi/19 rewritten for the new behaviour.
…o cfg (mcpp#965)

Which library is the C++ layer is the target's fact; a driver configuration
file beside the compiler is a fact of one installation. resolve_clang_driver
took a base package's C++ headers only after it found the payload's cfg, and
the isolation words were gated on bypassing that cfg, so on a Windows host,
whose LLVM payload has none, the std module of a libstdc++ base (the Hurd and
Haiku rows of mcpp-platforms) was precompiled with neither the base's
include_dirs nor -nostdinc++.

The driver model now names the base's headers and its library whatever the
cfg, host_compile_tokens states the clang driver's search whenever it
bypasses a cfg or there is none to trust: -nostdinc++ with the base's
directories and -stdlib=<the layer's library> for a base's C++ layer, and
-nostdinc++ and -nostdlibinc for the layers the graph supplies. Only
--no-default-config follows the file. GCC keeps its own search.

SPEC-002 v1.10 section 2.5, usage chapter 22 in both languages, unit case
HostFlags/ABasesCxxLayerReachesADriverWithNoCfg.
…endency's is (mcpp#967)

The platform column was read from the local copy of the index alone, so a row
the index listed after that copy was taken was refused as an unknown target
until `mcpp index update`. A miss for a row that is refused without an answer
(one the engine does not know, or a planned one) now goes through
decide_for_miss and apply -- `--offline`, `[index] auto_refresh`, the
debounce, one sync per command -- and the column is scanned again after a
refresh, the scan cache dropped. Decided once per row per process; the
unknown-target and planned refusals say whether the index was refreshed, and
why not. A row the table serves without a platform does not refresh.

The column scan is split into helpers (one descriptor's claims, the roots,
the cached scan), which also brings its loops to two levels.

SPEC-013 v1.14 section 14 item 6, usage chapters 05 and 35 in both
languages, e2e index::refresh/967; openkal::platforms/1106 leg D runs
offline, where a refresh would bring the shared index whose
openkal-platform lists the planned row.
…ries takes the newest release the build can use (mcpp#972)

The target is one resolution and each build program -- a package's
build.mcpp or its rules' program, a platform's plugin -- is another
(design 2026-11-19 §21 A1-A6). A program's request takes the newest
version the target or another program holds that admits it, the target
being resolved first, so a build without a conflict keeps its versions
and its build programs' identities. Programs that hold one package at
two versions build in two build programs' graphs named by those
versions, and mcpp.lock records the second version as an entry
`<name>+b<digest>` with `programs`. A conflict inside one program is
refused naming the program. An entry the plugin-imports compat reading
turns into a library's dependency belongs to each resolution that holds
the library.

A platform selected as a series skips a release whose version group a
request of the build does not admit, naming the release and the
request, and is refused listing each release when none is left. The
pack launchers build on mcpp's own mcpplibs.os (0.4.6), which
check_version_pins.sh holds equal to mcpp.toml's.

The global dependency cache moves from plan.cpp to dependency_cache.cpp,
divided at its own banners; graph.cpp's rename staging and provisioning
closures move to graph_stage.cpp and graph_provisioning.cpp, the
resolutions' helpers to graph_requests.cpp, and step13_lockfile's
version entries to a function of their own: every file and step stays
under its limit.
…and selects (mcpp#974)

A member reached through `path` from the selected one, and any package
below the project's directory, is built in place as the documented rule
states: `mcpp emit build-database -p app` writes elsewhere and named a
member its `--workspace` plan built in place, so the member's arguments
depended on the selection (e2e 872). e2e 912 and path/09 measure the
build programs' graph and a path dependency's compile edges under
`--cache local`: their library is a tree beside the project, which the
global cache now names by its content, and a home that built the same
tree before stages it.
The fills of the global cache are a phony of their own, which an
explicit goal set includes: `mcpp test`'s first phase and the build
programs' graph build no `default`, so under clang's two-phase schedule
the BMI of a module nothing imported was not built and the fill was
refused (`bmi cache populate failed for openkal@0.26.0: expected build
output missing: .../openkal.time.pcm` in mcpp's macOS `mcpp test`).
… round's design record is landed

SPEC-001 §10.4 item 3 and the design record's E2 named the selected
members, which 7583fc7 replaced by every member of the workspace: a
member's arguments do not depend on the members a command selects.
…ld; a replaced version takes its build program with it

A review of 3525899 measured three defects, each now an e2e case that
fails before this change and passes after it:

- A program's second request of a package it held was resolved alone,
  took the index's newest version, and the merge of the program's own
  ranges moved the program off the version the target holds (956 D). A
  program keeps the version it holds while its requests admit it, and
  otherwise takes the newest version another resolution holds that all
  its requests admit; it takes that version's slot, and a slot no
  resolution holds leaves the build with its link flags and its build
  program. The requests a left version made are withdrawn in the
  resolution that left it, not in every resolution.
- The target was resolved first within the first round only: the
  platform's layers and capability defaults arrive in later rounds, after
  the programs had taken the index's newest versions, and a program's
  package answered the target's capability (972, the target before its
  build programs). The resolutions run in waves: the target with every
  platform round, then the programs of the packages it holds, then
  theirs. SPEC-002 §3.6 states the target's graph as the one a default
  is decided by.
- A merge that replaced a slot's version walked `[dependencies]` alone,
  without the row, the forwards or the build dependencies, and kept the
  build program of the version that left (graph/972). The new version is
  walked by the function that walked the first, and the old version's
  program is withdrawn.

Also: a record copied into a library's resolutions carries no link
flags (each copy withdrew them once more); a platform-series restart a
call does not answer is dropped when it returns, and the pass that states
the toolchain restarts a series too; resolve_graph restarts from the
overrides it was given; a platform the build programs' graph pins is not
taken as a series; contested packages are counted among the build's
programs, a development dependency's apart (1088 D).
A partial upgrade installed OpenSSL 4 beside packages built against
libcrypto.so.3, and pacman refused the transaction on every run since
2026-11-07 (mcpp#953).
@Sunrisepeak Sunrisepeak changed the title 2026.11.19.1: rule convergence -- the descriptor's two forms, the version group in the merge, link inputs, compiler-run state, transactional publication, actions, text as UTF-8 (mcpp#931, #952 to #960, #970) 2026.11.19.1: rule convergence -- the descriptor's two forms, the version group in the merge, link inputs, compiler-run state, transactional publication, actions, text as UTF-8, a resolution per build program, path dependencies in the cache (mcpp#931, #952 to #960, #965, #967, #970, #972, #974) Nov 19, 2026
@Sunrisepeak
Sunrisepeak marked this pull request as ready for review November 19, 2026 22:33
@Sunrisepeak
Sunrisepeak merged commit d4f4447 into main Nov 19, 2026
338 of 358 checks passed
This was referenced Nov 19, 2026
@Sunrisepeak
Sunrisepeak deleted the eco/rule-convergence branch November 19, 2026 23:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci:canaries Run the canary matrix (canaries.yml) on this pull request ci:full Run the full e2e set on this pull request (read when a push starts the run)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant