Skip to content

Commit f4151de

Browse files
committed
runtime_search_dir, the prepare role, and a stamp newer than every input (mcpp#701, mcpp#702)
Re-implements #701/#702's two prior commits to design .agents/docs/2026-09-26-compile-database-and-issue-699-design.md §5.4, and adds P, the `prepare` role the design's O1 asks for. One protocol bump (12) and one cache-format epoch bump (3) cover all three. R1'. `mcpp::runtime_search_dir(dir)` (`mcpp:runtime-search-dir=`, protocol 12) replaces the branch's `runtime_library_dir`: a launch-time search directory a build.mcpp discovers (a vcpkg prefix's `bin/`, a Qt SDK's `bin/`, a directory a `prepare` action populates) rather than one an author writes into a fixed TOML array. It lands DIRECTLY on `LinkIntent::runtimeSearchDirs` -- the field `runtime_search_dirs` itself populates -- not the retiring `RuntimeConfig::libraryDirs` the branch had joined by mistake, so every existing reader (the plan merge, RUNPATH/rpath, `mcpp run`'s loader path, `mcpp pack`'s closure search) sees a directive-declared directory exactly as it sees the manifest key's, and a dependency's declaration reaches its consumer's executable through the same merge. The directory need not exist when the program runs. R2. After a `check` or `prepare` command succeeds, the engine creates every declared stamp that is missing and sets the modification time of every existing one to the present, whether or not the command wrote it; on failure it writes nothing. This replaces the branch's before/after time comparison with R2's simple form: a stamp feeds no compile or link edge, so touching one the command already wrote changes no build. P. The `prepare` role (`mcpp::roles::prepare`, alongside `source`, `check`, `object`, `artifact`) is for construction whose file names are unknown when the build program runs -- installing a vcpkg manifest or a CMake subproject into a prefix, unpacking an SDK. Outputs: one or more stamps, written by the engine through the same wrapper as `check`, and one declared directory (`mcpp::action::output_dir`) the command populates. When the command succeeds and that directory does not exist, the engine writes no stamp and fails the edge, naming the directory. Every compile edge of the declaring package waits for it, as a blocking `check` does today; so does every LINK edge in the plan, because `plan.linkIntent` -- and therefore the `$ldflags` every link rule interpolates -- is one value merged from every package's LinkGlobal directives, read alike by the declaring package's own link and by every package that depends on it. The #534 self-check (`check_action_ordering`) is extended to assert this for links, the same way it already does for compiles. Progress lines label it `PREPARE`. An unknown role string is refused, naming the value and the five roles, where it used to be read as `source` silently -- the cache-format epoch moves to 3 so a stale entry from before this guard existed is re-validated rather than replayed two different ways by two code paths. A build program whose `rerun_if_changed`/`rerun_if_changed_glob` names a path inside a `prepare` directory (of any package in the graph) is warned, naming both paths: that directory is populated at build time, after configuration has already run.
1 parent e2d0981 commit f4151de

9 files changed

Lines changed: 627 additions & 135 deletions

File tree

‎modules/buildmcpp/src/directives.cppm‎

Lines changed: 120 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -164,18 +164,23 @@ enum class Slot : std::size_t {
164164
// splitting it into two slots would let a build with N deploy directives
165165
// pair them up wrong the moment N > 1.
166166
Deploy,
167-
// A LAUNCH-TIME SEARCH DIRECTORY, THE BUILD-PROGRAM FORM OF `[runtime]
168-
// library_dirs` (docs/04 §2.11). A dependency that brings a prebuilt
169-
// shared library (a vcpkg prefix's `bin/`, a Qt SDK's `bin/`) knows where
170-
// it lives only at build-program time, and the manifest key cannot be
171-
// computed -- it is a fixed TOML array. `Transform::AbsPath`, not a new
172-
// path shape: this is the SAME field the manifest key feeds
173-
// (`RuntimeConfig::libraryDirs`), so every consumer of it -- `mcpp run`'s
174-
// loader path, `mcpp pack`'s closure search, and the ELF/Mach-O
175-
// `-Wl,-rpath` rendering -- sees a directive-declared entry exactly as it
176-
// sees a TOML one, through the one merge in `plan.cppm` that already
177-
// exists for the manifest key. No new downstream code path.
178-
RuntimeLibraryDir,
167+
// A LAUNCH-TIME SEARCH DIRECTORY, THE BUILD-PROGRAM FORM OF
168+
// `runtime_search_dirs` (docs/04 §2.11). A dependency that brings a
169+
// prebuilt shared library (a vcpkg prefix's `bin/`, a Qt SDK's `bin/`, a
170+
// directory a `prepare` action populates) knows where it lives only at
171+
// build-program time, and the manifest key is a fixed TOML array.
172+
// `Transform::AbsPath`: relative resolves against the package root, like
173+
// `include-dir`. Feeds `LinkIntent::runtimeSearchDirs` DIRECTLY --
174+
// `runtime_search_dirs`' own field, not the retiring `[runtime]
175+
// library_dirs` / `RuntimeConfig::libraryDirs` (docs/04 §2.11 keeps that
176+
// one "for one compatibility train" and gains no directive of its own) --
177+
// so every reader of `runtimeSearchDirs` (the plan merge, RUNPATH/rpath on
178+
// ELF and Mach-O, `mcpp run`'s loader path, `mcpp pack`'s closure search)
179+
// sees a directive-declared directory through the one merge that already
180+
// exists for the manifest key (`resolve_runtime_contract`, plan.cppm), no
181+
// second path to keep in sync. The directory need not exist when the
182+
// program runs: a `prepare` action may populate it later, at build time.
183+
RuntimeSearchDir,
179184
Count
180185
};
181186
inline constexpr std::size_t kSlotCount = static_cast<std::size_t>(Slot::Count);
@@ -425,29 +430,32 @@ inline constexpr std::array<Def, 27> kTable{{
425430
// before `apply` on both the run path and the cache-hit path, so a cached
426431
// replay refuses exactly what a fresh run would.
427432
{"deploy", "deploy", Slot::Deploy, Scope::LinkGlobal, Transform::Deploy, false, "", "", 11},
428-
// v12. The build-program form of `[runtime] library_dirs` (docs/04
429-
// §2.11): a launch-time search directory, for a dependency (a vcpkg
430-
// prefix's `bin/`, a Qt SDK's `bin/`) whose location a build.mcpp learns
431-
// rather than one an author can write into TOML. Scope::LinkGlobal,
432-
// exactly as `deploy` above, because `apply` folds it into
433-
// `RuntimeConfig::libraryDirs` -- the SAME field the manifest key
434-
// populates -- so `resolve_runtime_contract`'s per-package merge
435-
// (plan.cppm) carries it into a consumer's `linkIntent.runtimeSearchDirs`
436-
// through the one path that already exists for the manifest key, rather
437-
// than a second one this table would have to keep in sync. `mustExistAfterRun`
438-
// is FALSE: a directory is not the declared-output contract's shape
439-
// (`generated=`/`source=` name a file), and the directory need not even
440-
// exist yet -- a payload not installed on this machine is the manifest
441-
// key's own behaviour (`xpkg_dir()` returning empty configures no runner,
442-
// silently and correctly).
433+
// v12 (mcpp#701/#702, design §5.4 R1'). The build-program form of
434+
// `runtime_search_dirs` (docs/04 §2.11): a launch-time search directory,
435+
// for a dependency (a vcpkg prefix's `bin/`, a Qt SDK's `bin/`, a
436+
// directory a `prepare` action populates) whose location a build.mcpp
437+
// learns rather than one an author can write into TOML. Scope::LinkGlobal,
438+
// exactly as `deploy` above, because `apply` folds it DIRECTLY into
439+
// `LinkIntent::runtimeSearchDirs` -- NOT `RuntimeConfig::libraryDirs`,
440+
// which docs/04 §2.11 keeps readable "for one compatibility train" and
441+
// which gains no directive -- so `resolve_runtime_contract`'s per-package
442+
// merge (plan.cppm) carries it into a consumer's `linkIntent
443+
// .runtimeSearchDirs` through the one path that already exists for the
444+
// manifest key, rather than a second one this table would have to keep in
445+
// sync. `mustExistAfterRun` is FALSE: a directory is not the
446+
// declared-output contract's shape (`generated=`/`source=` name a file),
447+
// and the directory need not even exist when this program runs -- a
448+
// `prepare` action (§3) may populate it later, at build time, which is
449+
// the reason this directive exists at all rather than a plain manifest
450+
// array.
443451
//
444452
// kCacheEpoch is NOT bumped, the same reasoning `warning`/`pack-format`
445453
// state explicitly: an entry written before this row carries no
446-
// `d runtime-library-dir` line, and the program that wrote it could not
454+
// `d runtime-search-dir` line, and the program that wrote it could not
447455
// emit one -- so replaying it yields exactly what that program said, and
448456
// the entry is still correct. An older engine reading a newer entry
449457
// already discards the whole record through the unknown-tag path.
450-
{"runtime-library-dir", "runtime-library-dir", Slot::RuntimeLibraryDir, Scope::LinkGlobal, Transform::AbsPath, false, "", "", 12},
458+
{"runtime-search-dir", "runtime-search-dir", Slot::RuntimeSearchDir, Scope::LinkGlobal, Transform::AbsPath, false, "", "", 12},
451459
}};
452460

453461
// ── Collected output of one run ────────────────────────────────────────────
@@ -980,13 +988,14 @@ void apply(mcpp::manifest::Manifest& m, const Directives& d) {
980988
for (auto const& p : d.at(Slot::IncludeDirsAfter))
981989
bc.includeDirsAfter.emplace_back(p);
982990

983-
// `runtime-library-dir`: already absolute from the AbsPath
984-
// transform, like IncludeDirs above. Joins `RuntimeConfig::libraryDirs`,
985-
// the exact field `[runtime] library_dirs` populates from TOML -- so a
986-
// consumer sees it exactly as it sees the manifest key, folded in by the
987-
// SAME merge (plan.cppm's `resolve_runtime_contract`), never a second one.
988-
for (auto const& p : d.at(Slot::RuntimeLibraryDir))
989-
m.runtimeConfig.libraryDirs.emplace_back(p);
991+
// `runtime-search-dir`: already absolute from the AbsPath transform, like
992+
// IncludeDirs above. Joins `LinkIntent::runtimeSearchDirs` DIRECTLY, the
993+
// exact field `[runtime] runtime_search_dirs` populates from TOML -- not
994+
// the retiring `RuntimeConfig::libraryDirs` -- so a consumer sees it
995+
// exactly as it sees the manifest key, folded in by the SAME merge
996+
// (plan.cppm's `resolve_runtime_contract`), never a second one.
997+
for (auto const& p : d.at(Slot::RuntimeSearchDir))
998+
m.runtimeConfig.linkIntent.runtimeSearchDirs.emplace_back(p);
990999

9911000
// Claims join the runtime declarations the manifest could have carried
9921001
// itself, so the version-floor check in prepare reads ONE list and never
@@ -1049,15 +1058,55 @@ void apply(mcpp::manifest::Manifest& m, const Directives& d) {
10491058
}
10501059
}
10511060

1061+
// The five roles `mcpp::action::role` may name, in the order
1062+
// `mcpp::manifest::BuildAction::Role` declares them (mcpp#702). A build.mcpp
1063+
// SHOULD spell one of `mcpp::roles::{source, check, object, artifact,
1064+
// prepare}` (SPEC-007 R3.6): those constants exist only from protocol 12, so
1065+
// a program that misspells the role fails to COMPILE on an older engine and
1066+
// names the missing constant, instead of reaching this decoder as an
1067+
// unrecognised string. A hand-written `printf("mcpp:action=...")` program
1068+
// (the frozen surface, protocol 0) has no such guard, which is the case this
1069+
// list is for.
1070+
constexpr std::array<std::string_view, 5> kActionRoles{
1071+
"source", "check", "object", "artifact", "prepare"};
1072+
1073+
bool is_known_action_role(std::string_view role) {
1074+
return std::ranges::find(kActionRoles, role) != kActionRoles.end();
1075+
}
1076+
1077+
// The bare `role` an `mcpp:action=` payload named, or "source" when the key
1078+
// is absent (the same default `decode_action` uses) -- independent of
1079+
// whether the REST of the payload is well-formed, so `action_error` can name
1080+
// an unrecognised role even when the action is otherwise malformed, instead
1081+
// of folding both failures into one generic message.
1082+
std::optional<std::string> action_role_of(std::string_view payload) {
1083+
try {
1084+
auto j = nlohmann::json::parse(payload);
1085+
return j.value("role", std::string{"source"});
1086+
} catch (...) {
1087+
return std::nullopt;
1088+
}
1089+
}
1090+
10521091
std::optional<mcpp::manifest::BuildAction> decode_action(std::string_view payload) {
10531092
try {
10541093
auto j = nlohmann::json::parse(payload);
10551094
mcpp::manifest::BuildAction a;
10561095
a.id = j.value("id", std::string{});
10571096
auto role = j.value("role", std::string{"source"});
1097+
// An unrecognised role is refused here, not read as Source: before
1098+
// mcpp#702 this fell through to Source silently, changing an action's
1099+
// meaning without a word (SPEC-007 R3.6). `action_error` names the
1100+
// offending value before anything is applied; this path is also the
1101+
// one a cache-hit replay runs through `apply` (which calls no
1102+
// validator), so it has to refuse on its own rather than assume
1103+
// `action_error` already ran -- `epoch` 3 (program_protocol.cppm)
1104+
// makes sure no such entry survives from before this row existed.
1105+
if (!is_known_action_role(role)) return std::nullopt;
10581106
a.role = role == "check" ? mcpp::manifest::BuildAction::Role::Check
10591107
: role == "artifact" ? mcpp::manifest::BuildAction::Role::Artifact
10601108
: role == "object" ? mcpp::manifest::BuildAction::Role::Object
1109+
: role == "prepare" ? mcpp::manifest::BuildAction::Role::Prepare
10611110
: mcpp::manifest::BuildAction::Role::Source;
10621111
auto arr = [&](const char* k, std::vector<std::string>& dst) {
10631112
if (auto it = j.find(k); it != j.end() && it->is_array())
@@ -1073,7 +1122,18 @@ std::optional<mcpp::manifest::BuildAction> decode_action(std::string_view payloa
10731122
a.blocking = j.value("blocking", false);
10741123
a.depfile = j.value("depfile", std::string{});
10751124
a.description = j.value("description", std::string{});
1125+
// Omitted by every role but Prepare, exactly as `depfile` is omitted
1126+
// by an action that declares none -- the decoder's default (empty)
1127+
// is identical either way, so a cache entry written before this field
1128+
// existed replays as one that never set it.
1129+
a.outputDir = j.value("output_dir", std::string{});
10761130
if (a.command.empty() || a.outputs.empty()) return std::nullopt;
1131+
// Prepare only: `output_dir` is the whole point of the role (R3.2,
1132+
// R3.3) -- a `prepare` action with none declared would have a stamp
1133+
// and nothing else, indistinguishable from a `check` that merely
1134+
// forgot `blocking = true`.
1135+
if (a.role == mcpp::manifest::BuildAction::Role::Prepare && a.outputDir.empty())
1136+
return std::nullopt;
10771137
if (a.id.empty()) a.id = a.outputs.front();
10781138
return a;
10791139
} catch (...) {
@@ -1187,6 +1247,18 @@ std::string action_error(const Directives& d) {
11871247
"and cannot be used.\n"
11881248
" payload: {}", payload);
11891249
}
1250+
// Named BEFORE the general decode below, so the message identifies
1251+
// the unrecognised VALUE and lists the five roles the engine knows,
1252+
// rather than folding it into "malformed action" -- which is what an
1253+
// unknown role used to become (silently read as Source) until
1254+
// mcpp#702 (SPEC-007 R3.6).
1255+
if (auto role = action_role_of(payload); role && !is_known_action_role(*role)) {
1256+
return std::format(
1257+
"build.mcpp declared an action with role \"{}\", which is not "
1258+
"one of the five mcpp knows: source, check, object, artifact, "
1259+
"prepare.\n"
1260+
" payload: {}", *role, payload);
1261+
}
11901262
if (decode_action(payload)) continue;
11911263
// A malformed action is a hard error, never a skip: an action that
11921264
// silently does not exist produces a build missing generated sources,
@@ -1195,7 +1267,10 @@ std::string action_error(const Directives& d) {
11951267
"build.mcpp declared a malformed action.\n"
11961268
" Every action needs a non-empty `command` and at least one\n"
11971269
" declared `output` — mcpp fixes the source set during prepare,\n"
1198-
" so an output whose NAME is unknown cannot be built.\n"
1270+
" so an output whose NAME is unknown cannot be built. A\n"
1271+
" `prepare` action also needs a non-empty `output_dir`\n"
1272+
" (mcpp::action::output_dir), the directory its command\n"
1273+
" populates.\n"
11991274
" payload: {}", payload);
12001275
}
12011276
return {};
@@ -1235,6 +1310,14 @@ void prepare_actions(std::vector<mcpp::manifest::BuildAction>& actions,
12351310
// simply never match the file the command actually wrote.
12361311
if (!a.depfile.empty() && a.depfile.find("${mcpp.") == std::string::npos)
12371312
a.depfile = abs_against(pkgRoot, a.depfile);
1313+
// Prepare only: the same anchoring, for the same reason -- the
1314+
// directory the command populates and the post-condition the engine
1315+
// checks after that command succeeds (`__action-stamp`, src/cli.cppm)
1316+
// both need it absolute, or a package-relative spelling would be
1317+
// interpreted relative to the ninja build directory instead of the
1318+
// package root it was written against.
1319+
if (!a.outputDir.empty() && a.outputDir.find("${mcpp.") == std::string::npos)
1320+
a.outputDir = abs_against(pkgRoot, a.outputDir);
12381321
if (a.role != mcpp::manifest::BuildAction::Role::Source) continue;
12391322
for (auto const& o : a.outputs) {
12401323
if (o.find("${mcpp.") != std::string::npos) continue;

‎modules/buildmcpp/src/program_protocol.cppm‎

Lines changed: 26 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -81,13 +81,21 @@ export namespace mcpp::build::program_protocol {
8181
// at a path relative to the executable. Same cost as v5's: a package calling
8282
// `mcpp::deploy()` fails on an older engine at the build.mcpp COMPILE, because
8383
// that engine's bundled module has no such function.
84-
// v12: adds `runtime-library-dir` -- the build-program form of `[runtime]
85-
// library_dirs`: a launch-time search directory for a dependency (a vcpkg
86-
// prefix's `bin/`, a Qt SDK's `bin/`) whose location a build.mcpp learns
87-
// rather than one an author can write into TOML. Same cost as v5's: a
88-
// package calling `mcpp::runtime_library_dir()` fails on an older engine at
89-
// the build.mcpp COMPILE, because that engine's bundled module has no such
90-
// function.
84+
// v12 (mcpp#701/#702): adds `runtime-search-dir` -- the build-program form of
85+
// `runtime_search_dirs`: a launch-time search directory for a dependency (a
86+
// vcpkg prefix's `bin/`, a Qt SDK's `bin/`, a directory a `prepare` action
87+
// populates) whose location a build.mcpp learns rather than one an author can
88+
// write into TOML. Same cost as v5's: a package calling
89+
// `mcpp::runtime_search_dir()` fails on an older engine at the build.mcpp
90+
// COMPILE, because that engine's bundled module has no such function.
91+
//
92+
// v12 also adds the `prepare` role: construction whose file names are not
93+
// known when the build program runs. Its constants
94+
// (`mcpp::roles::{source, check, object, artifact, prepare}`) and its
95+
// `mcpp::action::output_dir()` method live in the bundled module for the
96+
// same reason `deploy` and `runtime_search_dir` do -- a program that uses
97+
// them fails to COMPILE on an older engine, naming the missing symbol,
98+
// rather than reaching that engine as a string it misreads.
9199
inline constexpr int kProtocolVersion = 12;
92100

93101
// ── Cache-format epoch ─────────────────────────────────────────────────────
@@ -101,7 +109,17 @@ inline constexpr int kProtocolVersion = 12;
101109
// Epoch 2 (#359): entries gained `glob` records. An engine that does not know
102110
// them would replay a strict subset of the declared inputs and call a stale
103111
// build fresh, which is exactly the silent-wrong-answer this guard exists for.
104-
inline constexpr int kCacheEpoch = 2;
112+
// Epoch 3 (mcpp#702): `decode_action` now refuses an `mcpp:action=` payload
113+
// whose `role` names none of the five the engine knows, where it used to read
114+
// an unrecognised string as `Source` silently. That is an INTERPRETATION
115+
// change on an existing tag (`action`, since v1) rather than a new one: an
116+
// entry cached under epoch 2 could hold such a role and decoded successfully
117+
// then; replaying it through `apply` (which runs no validator) would now
118+
// either refuse or, worse, silently drop the action, neither of which is what
119+
// a fresh run of the SAME build.mcpp would do. Bumping the epoch forces one
120+
// fresh run, which goes through `action_error` and reports exactly what
121+
// changed, instead of a stale entry reinterpreted two ways by two code paths.
122+
inline constexpr int kCacheEpoch = 3;
105123

106124
// ── Run bound ──────────────────────────────────────────────────────────────
107125
//

0 commit comments

Comments
 (0)