Skip to content

Commit 202aa9f

Browse files
committed
fix: bound split compilation and track active download progress
1 parent e530c80 commit 202aa9f

17 files changed

Lines changed: 633 additions & 51 deletions
Lines changed: 132 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,132 @@
1+
---
2+
subject: build-progress-and-scheduling
3+
status: active
4+
---
5+
6+
# BMI publication, compiler capacity and active download budgets
7+
8+
Audience: maintainers closing the 2026.11.5.2 candidate and reviewing PR #889.
9+
This record separates three failures exposed during candidate validation.
10+
It does not qualify a diagnostic object substitution as a released binary.
11+
12+
## Failure boundaries
13+
14+
| Failure | Confirmed cause | Owner |
15+
|---|---|---|
16+
| An importer cannot find a BMI after its interface is touched | GCC unlinks the old final path before renaming the new BMI; an absent identity was accepted as publication | Core detached compilation |
17+
| A build exceeds its requested compiler count | Extra Ninja slots serve publication and join edges, while ordinary compilers bypass the detached semaphore | Core Ninja scheduling |
18+
| A newly activated download is immediately considered stalled | Its queued state was observed earlier; starting without a source attempt or byte change retained the queue-entry timestamp | Core provider progress tracking |
19+
20+
No compiler toolchain payload, plugin semantic or timeout constant is changed
21+
by these corrections.
22+
23+
## BMI publication
24+
25+
GCC 16.1 tracing shows an `unlink(final)` immediately before
26+
`rename(temporary, final)`. The observed interval was approximately 26
27+
microseconds. A changed identity alone is therefore insufficient: publication
28+
requires a present file with a changed identity. The adapter still preserves
29+
the previous BMI for equivalence checking and failure recovery.
30+
31+
The causal control amplifies only that existing unlink interval to 200
32+
milliseconds. The previous core reproduces the candidate failure: the middle
33+
module cannot read the core module. Replacing only the detached-compilation
34+
implementation makes the same real GCC workload succeed. A separate
35+
deterministic publisher confirms the boundary: the old phase returns zero
36+
while the BMI is absent; the corrected phase returns only after publication.
37+
38+
An exit status of zero without the required BMI is diagnosed as a failure.
39+
The previous BMI is restored. E2E 234 retains its cold execution, graph-shape,
40+
incremental build and stale-token assertions, and adds both publication cases.
41+
42+
## Shared compiler permits
43+
44+
The candidate generated `sched_cap = 8`, while its scheduling policy allows
45+
48 Ninja slots. The BMI supervisor used that semaphore; ordinary C/C++
46+
compilation and compiler-driven scanning did not. A full local candidate
47+
build exceeded eight compiler processes and encountered memory-pressure
48+
kills. Its interrupted output is not build qualification.
49+
50+
The selected correction retains one actual compiler budget. Under the
51+
detached strategy, ordinary built-in compilation and scanning use an internal
52+
argv executor holding the same permit until the real child exits. BMI
53+
supervisors continue to hold their permit across early publication and code
54+
generation. Join edges acquire no permit. A holder never waits for another
55+
permit, so waiting joins cannot form a permit dependency cycle.
56+
57+
The internal executor introduces no user mode and no argument serialization.
58+
It receives the compiler's actual argv, inherits the existing streams and
59+
returns its status. Existing response files retain their original owner and
60+
encoding. Ordinary completion and interrupt cleanup release a permit once;
61+
release cannot remove a permit already acquired by another compiler.
62+
Failure to create or acquire the semaphore is reported rather than followed
63+
by an unbounded compile. A failed supervisor start releases its permit.
64+
65+
The mixed control uses a real generated Ninja graph and GCC 16.1. A driver
66+
records active invocations and remains alive briefly after BMI publication,
67+
amplifying overlap with ordinary C and C++ units. With a requested cap of
68+
one, the previous graph reaches six active drivers. Corrected controls use
69+
caps of one and two, retain the extra Ninja slots, and observe peaks of
70+
exactly one and two. They also check no-op builds, ordinary compiler failure,
71+
a module failure after BMI publication, subsequent recovery, ordinary
72+
compiler cancellation, child termination and permit cleanup. E2E 234 retains
73+
the full-set stale-token reclamation check.
74+
75+
These local controls use selected recompiled objects and an extracted copy of
76+
the production dispatch branch. They establish causality and small-load
77+
behavior. The complete current sources still require the ordinary CI build
78+
and all required checks before release.
79+
80+
## Download activation and deadline reasons
81+
82+
The pinned xlings 2026.10.26.1 provider reports queued files in each batch
83+
with `started = false`. Once a worker obtains a slot, `started` becomes true
84+
before a source attempt or byte count necessarily changes. That transition
85+
starts the active budget. Waiting in the queue consumes no active budget;
86+
repeated samples of an already active file do not renew it. A byte increase
87+
or source change retains its existing renewal semantics.
88+
89+
An isolated provider invocation retained raw NDJSON from one download stream.
90+
Replaying the same 347 snapshots through the actual C++ provider modules gives:
91+
92+
| Module | Expired snapshots | First expiration |
93+
|---|---:|---:|
94+
| Previous state machine | 11 | 95.751 seconds |
95+
| Corrected activation transition | 0 | none |
96+
97+
The first false expiration belongs to a queued entry activated at 95.751
98+
seconds with source attempt zero. Its inherited age was 95.651 seconds.
99+
With queue time excluded, the maximum active idle interval at the observed
100+
snapshots is 7.735 seconds. The isolated network invocation reached its
101+
150-second observer budget; it was stopped by the experiment, not by a
102+
provider progress deadline, and does not establish a completed graphics run.
103+
104+
The failed candidate CI log contains rendered status rather than raw NDJSON.
105+
Its timing is consistent with activation of the final queued entries. It
106+
does not identify which raw file state first expired, and is not presented as
107+
proof of a particular file's network behavior. Progress on an independent
108+
download remains insufficient to renew a genuinely stalled file.
109+
110+
The provider source was followed through its HTTP implementation: the
111+
connect and idle environment budgets are consumed below the downloader
112+
wrapper. They remain eight and thirty seconds, respectively; the parent's
113+
per-file budget remains forty-five seconds. No budget is increased to hide
114+
the state error. Progress timeouts now identify the file, interval and source
115+
attempt; process deadlines remain distinct. Time-injected unit cases cover
116+
queue waits longer than forty-five seconds, genuine per-file stalls, source
117+
fallback and completion.
118+
119+
## Qualification sequence
120+
121+
The local controls precede the complete PR CI. The graphics job is accepted
122+
only after its shader, rendered-image and pixel assertions actually run.
123+
After every required check succeeds, an independent ecosystem review
124+
precedes merge and release. Published tags remain unchanged.
125+
126+
Plugin PR #52 has independently passed all six jobs against the published
127+
2026.11.5.1 core. Its final public core pin and real subproject-sidecar fixture
128+
follow ordinary installation of the new release, in the same plugin PR.
129+
130+
Public records contain synthetic assertions and aggregate evidence. Raw
131+
provider streams, user logs, usernames and private paths remain outside the
132+
repository and pull-request description.

‎.agents/docs/README.md‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ superseded_by: 2026-09-07-....md # when status is superseded
1818
---
1919
```
2020

21-
349 records.
21+
350 records.
2222

2323
## By subject
2424

@@ -34,6 +34,10 @@ Records that declare one. Everything else is listed by date below.
3434
- [构建依赖的一条管线:配置与产物两轴、包的实例、库接口、兼容系统与工作空间收尾(#797)](2026-10-11-build-dependencies-one-pipeline-design.md) — active
3535
- [Four upstream asks from a UI framework: what each one is under mcpp's design, and the combined plan](2026-09-13-four-upstream-asks-from-a-ui-framework.md) — landed
3636

37+
### build-progress-and-scheduling
38+
39+
- [BMI publication, compiler capacity and active download budgets](2026-11-06-bmi-publication-and-compiler-cap-closeout.md) — active
40+
3741
### design
3842

3943
- [升级的连续性:打包修订不在使用路径上替换载荷、替换失败不拆掉可用状态、home 中的悬空记录自愈、真实的升级 lane](2026-10-17-upgrade-continuity-design.md) — active
@@ -167,6 +171,7 @@ Records that declare one. Everything else is listed by date below.
167171

168172
### 2026-11
169173

174+
- [BMI publication, compiler capacity and active download budgets](2026-11-06-bmi-publication-and-compiler-cap-closeout.md) — active
170175
- [异构构建语义与插件边界补充方案(mcpp#885、mcpp#887)](2026-11-05-heterogeneous-build-semantics-and-plugin-boundaries-supplement.md) — active
171176
- [异构构建补充方案的第二轮调研与实验](2026-11-05-heterogeneous-build-research-and-experiments.md) — active
172177
- [生态 885 与异构构建补充方案的实现依赖](2026-11-05-heterogeneous-build-implementation-plan.md) — active

‎CHANGELOG.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,15 @@
1818
an installation that the same subproject has yet to produce.
1919
- The macOS clean bootstrap keeps its stage-one compiler outside the output tree
2020
removed by the second stage.
21+
- GCC BMI publication requires a present replacement file; its unlink-before-rename
22+
interval no longer releases importers. A successful compiler that produces no
23+
BMI fails with a diagnostic and restores the previous BMI.
24+
- Ordinary compiler invocations share the detached GCC compilers' concurrency
25+
permits, so the additional Ninja scheduling slots do not exceed `--jobs`.
26+
- Queued downloads begin their progress budget when a worker starts them,
27+
rather than inheriting time spent waiting for a slot.
28+
- Provider progress timeouts identify the stalled download and source attempt,
29+
independently of progress on other files and provider process deadlines.
2130

2231
### Changed
2332

‎docs/04-mcpp-toml.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1048,6 +1048,10 @@ with two behaviours has to be.
10481048

10491049
`bmi_schedule` decides when importers are unblocked.
10501050

1051+
With GCC and `bmi_schedule = "on"`, ordinary compilation and detached code
1052+
generation share the same total `jobs` limit (2026.11.5.2+). Additional
1053+
scheduling and join tasks do not increase the number of compiler processes.
1054+
10511055
| value | |
10521056
|---|---|
10531057
| `"auto"` | **the default, and it currently means OFF** |

‎docs/25-home-indices-and-store.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -87,6 +87,14 @@ rules are in [SPEC-010](specs/home-initialization.md).
8787
without installing tools. `self config --mirror auto|CN|GLOBAL` updates only local
8888
configuration, so it remains usable while the network is broken.
8989

90+
An install tracks byte progress and source changes separately for each active
91+
download. Time spent waiting for a download slot does not consume that budget.
92+
Progress on another file does not renew a stalled file's budget.
93+
From 2026.11.5.2, a stalled-download diagnostic names the file, elapsed interval
94+
and source attempt. Provider preparation and process timeouts are reported
95+
separately. Mirror selection
96+
uses `mcpp self config --mirror auto`, `CN` or `GLOBAL`.
97+
9098
### Retention of build outputs (2026.10.31.3+)
9199

92100
The build cache under `build-cache/v1/` is shared by every project of the home;

‎docs/zh/04-mcpp-toml.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -972,6 +972,9 @@ bmi_schedule = "off" # auto (default, = off) | on | off
972972

973973
`bmi_schedule` 决定 importer 何时被解除阻塞。
974974

975+
使用 GCC 且 `bmi_schedule = "on"` 时,普通编译与分离的代码生成共用同一个 `jobs` 总上限
976+
(2026.11.5.2+)。额外的调度和等待任务不增加编译器进程数量。
977+
975978
| 取值 | |
976979
|---|---|
977980
| `"auto"` | **默认值,目前意味着关闭** |

‎docs/zh/25-home-indices-and-store.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,11 @@ Windows 首次显式 `self init` 时,若 D 是可写的本地固定磁盘,
7171
`self env`、`self config`、`self doctor` 和 `toolchain list` 读取本地状态,不安装工具。
7272
`self config --mirror auto|CN|GLOBAL` 仅更新本地配置,因此网络故障时仍可使用。
7373

74+
安装过程分别跟踪每个活动下载的字节进度与来源切换,等待下载槽位的时间不消耗该预算。
75+
其他文件取得进度不会延长停滞文件的预算。
76+
从 2026.11.5.2 起,下载停滞诊断会标明文件、停滞时长与来源尝试次数;provider 准备阶段与进程超时单独报告。
77+
镜像选择使用 `mcpp self config --mirror auto`、`CN` 或 `GLOBAL`。
78+
7479
### 构建产物的保留期(2026.10.31.3+)
7580

7681
`build-cache/v1/` 下的构建缓存由 home 中所有工程共享;工程的 `target/` 属于工程自身。两者各有保留期,

0 commit comments

Comments
 (0)