Skip to content

Commit 526be0c

Browse files
committed
fix(traces): create immutable raw objects atomically across processes
Use backend conditional object creation for raw and staging uploads under both upload strategies. Bound owned segments to the existing 64 MiB limit, retain the first successful object's bytes, and preserve raw readback identity checks. Classify a conflict still missing after one HEAD as a retryable static error; never fall back to overwriting. FIX-MTR-07 independent patch v0.42.68, following published FIX-MTR-10. Controlled old-source cross-process RED retained; A5: adapter7/7, process8/8, raw5/5, roundtrip3/3, full adapter20/20, original smoke19/19. Local Claude ER05/H0 PASS session 347ca1f3-cec3-4036-b400-b51fa0752afc. Review SHA256: 26766e140f328f5b74b0c1dfa6bc63a9a4a0e98588145f42dc0d465d1eb4bbf1 Final C: exact fmt/clippy/full-test/build/build-tests all exit0, 20 targets / 3063 passed / 0 failed / 3 existing ignored / 0 filtered. C terminal SHA256: b004bf419de7b71d52945af22ce5cadebbd58c6e7142e72bf93efb10899fc5c8 Frozen final-input manifest SHA256: fc71159c122502ad5061f5fe443e33c3aca2411efeee51fc50306b9de22d7094 Tag/Release and applicable main/tag Docker D are still pending. Signed-off-by: ELi Ma <eli@patch.sh>
1 parent dab66b6 commit 526be0c

13 files changed

Lines changed: 1210 additions & 24 deletions

File tree

‎Cargo.lock‎

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Cargo.toml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[package]
22
name = "mega2"
3-
version = "0.42.67"
3+
version = "0.42.68"
44
edition = "2024"
55
license = "MIT OR Apache-2.0"
66
description = "Monorepo engine and Git hosting server for AI agents: trunk-based storage, Git Smart HTTP, Git LFS, OCI registry, and Agent Session Capture"

‎docs/architecture.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,8 @@ Three storage kinds with distinct roles, all accessed through the `jupiter` laye
8585
2. **Object content: pluggable object storage**. Git objects and LFS object bytes are not stored in the database; `build_object_storage` (`src/jupiter/storage/object_storage.rs`) builds the backend through the orbit factory per `[object_storage].storage_type`: local / S3-compatible / GCS. Contract types live in `src/orbit_api/`, implementations in `src/orbit/`. S3-family credentials accept `vault://` SecretRefs (see §5). Large LFS objects may use FastCDC chunking; the implementation lives under `src/ceres/lfs/`. Traces raw Git objects use their own `traces` namespace and are not stored under `git` or `agent`; the key layout is in [`refactoring/traces.md`](./refactoring/traces.md) §3.
8686
3. **Redis: cache and distributed locks, not a queue**. The git object cache (`GitObjectCache`) and the `RedLock` mutex live in Redis (`src/jupiter/redis/`); the write queue is the Postgres `push_queue` table (see §4) — there is no FIFO/durable queue on the Redis side. The connection is shared via `connection-manager`, and `redis.url` accepts a SecretRef.
8787

88+
Traces raw/staging writes use complete-object conditional creation in the adapter to preserve the first stored content under both upload strategies; the raw layer reads back the bytes to distinguish idempotence from a content conflict. The fixed object limit, retryable conflict classification, memory scope, and qualification of each deployed backend are defined in the [Traces storage contract](./refactoring/traces.md). This preserves the Git/LFS/Artifact/Media upload strategies and does not extend immutability to unused mutable Traces write interfaces.
89+
8890
See the [Configuration Guide](./configuration.md) for storage-related settings and the [Deployment Guide](./deployment.md) for operating the service.
8991

9092
## 4. The write path: MonoWriteQueue

‎docs/architecture.zh.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,8 @@ mega2 是服务 Agent 工作流的第二代 Mega 引擎;第一代 Mega 项目
8484
2. **对象内容:可插拔对象存储**。Git 对象与 LFS 对象的二进制内容不入库,由 `build_object_storage`(`src/jupiter/storage/object_storage.rs`)经 orbit 工厂按 `[object_storage].storage_type` 构建后端:local / S3-compatible / GCS。契约类型在 `src/orbit_api/`,实现在 `src/orbit/`。S3 系凭证支持 `vault://` SecretRef(见第 5 节)。LFS 大文件的 FastCDC 分块见 [`refactoring/fastcdc-media.md`](./refactoring/fastcdc-media.md)。Traces 的原始 Git 对象使用独立的 `traces` namespace,不存放在 `git` 或 `agent` 下;key 布局见 [`refactoring/traces.md`](./refactoring/traces.md) 第 3 节。
8585
3. **Redis:缓存与分布式锁,不是队列**。git 对象缓存(`GitObjectCache`)与 `RedLock` 互斥锁在 Redis(`src/jupiter/redis/`);写入队列是 Postgres 的 `push_queue` 表(见第 4 节),Redis 侧没有 FIFO/持久化队列。连接经 `connection-manager` 共享,`redis.url` 支持 SecretRef。
8686

87+
Traces raw/staging 通过适配器的完整对象条件创建保留先写入内容,两种上传策略遵守同一规则;原始层读回比对决定幂等或内容冲突。固定对象上限、冲突期间的可重试分类、内存范围和各部署后端的资格要求统一见 [Traces 存储合同](./refactoring/traces.md)。这不改变 Git/LFS/Artifact/Media 的上传策略,也不为未使用的 Traces 可变写入接口提供不可变保证。
88+
8789
深入阅读:[`refactoring/orbit.md`](./refactoring/orbit.md)(对象存储契约与后端)、[`refactoring/storage-events.md`](./refactoring/storage-events.md)(存储事件外发)、[`refactoring/fastcdc-media.md`](./refactoring/fastcdc-media.md)。
8890

8991
## 4. 写入路径:MonoWriteQueue

‎docs/plan/README.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,7 @@
1+
> **FIX07回迁与新组合树A5实际结果(2026-10-24T10:47:15.980060+00:00):** 本地Claude ER03/H0/VERDICT均PASS,session `ce9e53bd-c582-468f-bc5d-907a660dbf1d`,输出SHA256 `12e135000f67ceb1c8fe77c03b7055635224cf873894d1379effaf0368eb7f34`;Markdown加粗导致首个提取器误判,原始PASS与parser失败记录均保留,root核实际原文和57输入无漂移后登记。原生stash/ff-only整合实际exit0,原HEAD5ee8f51→已发布FIX10 `dab66b68cbaa4e6a1f454d5eaf18b31df8aeca28`;stash `cd81d7dce9caf2e6df12f0ba85ab738e17ab65eb`保留且11blob已核,六非重叠保存像逐字节相同、五gov基于已发布树重建。root唯一publisher回到原工作区,独立clone退役;apply terminal SHA256 `918a4fad3840b89a5a7a082ce77c9dbb9b0132ebe260a41ee22a2b8fb7caae94`。新组合树A5一次实际exit0:fmt、strict all-target/all-feature Clippy、adapter7/7、分进程8/8、raw5/5、roundtrip3/3、完整adapter20/20、release build及原19smoke19PASS/0FAIL/0skip。各lib门0failed/0ignored;VER6/7实际包含在广前缀,按唯一case不重复累计;VER5锁定SDK五源码SHA与当前raw/staging消费链重核。A5终态SHA256 `816a9c39a1ec5ed290ee98ddb4cf99c9f8023f468cb4f965f30cb8c7cd828a9d`,smoke日志SHA256 `b507ccc24e2ed87b490624eb42c9836c53657b8dd6481ce5ce6d7a3d9ae2c058`。lib子进程退出/对象清理以fixture实际断言为据,不把空MEGA2_IT_PIDS_FILE伪当外部清理证明;smoke服务exit0、自有DB/Redis键/容器/工作目录清理与源码/runner无漂移均实际核实。FIX07源码/A-B现locally-accepted,自己的代码ER05/H0、patch bump、完整C、签名提交、推送与版本D仍待,当前版本0.42.67;FIX10完成事实不代本卡门。root接受整合P2-1~8,六前后baseline blob额外已核,Libra三文件待精确本地签名且不push;超时若发生必须人工核自有资源,不自动重试。旧A2 lint、smokeR1/R2、FIX3/STOP均保留。完成数仍Mega12/111、Libra0/26,后继98与H1/LTR01仍待本卡完整C/D。
2+
3+
> **FIX-MTR-10 实际发布闭合与 FIX-MTR-07 恢复入口(2026-10-24T10:29:04.138921+00:00):** FIX10源码7AC/4VER、受控RED/GREEN及本地Claude最终R3 ER05/H0/VERDICT均PASS,session `9b3bd01e-e05d-4e91-b6c7-d589818b6934`,输出SHA256 `63f6311224cf622552e60618c5429d576f5d53b120489bbe23fe879dea1099c4`。0.42.67最终树完整C五门exit0,20 targets/3048 passed/0 failed/3原有ignored/0 filtered,编译0警告/错误,C terminal SHA256 `f494216b14f764eec52fb952710e04a304a100e03c6d0e5181c576157ed30fab`。signed+DCO提交`dab66b68cbaa4e6a1f454d5eaf18b31df8aeca28`、main、标注`v0.42.67`及人工Release均实核;tag Docker `40085020955`双原生架构与manifest三job、main Harbor `40084993632`均实际success,D终态SHA256 `32b178514a5327fb6bda45ad7821dd7d6e2e1fdf8d5029642fab504c3086da8b`。FIX10现done/complete;Mega完成12/111(活动12/110)、Libra0/26(活动0/25)、联合12/137(活动12/135)。原FIX07仍是已保存候选,旧A2 lint FAIL、smoke R1无Bash及R2 18PASS/1FAIL均保留;此次另审原生stash/ff-only整合,真实Claude ER03/H0 PASS及精确应用终态前不写原工作区。整合后本卡才恢复in-progress,须重新执行受影响A/B、自身代码ER05、独立patch与完整C/D;未提前验收或发布。后继98→101→99→11→40→13→39→92→12/H1及LTR10→11→12→LTR01/H2不变,H0~H6不变。Libra三个治理文件属MTR01/LTR01,仅本地signed提交、不push。本轮新治理字节由FIX07后续完整C/D承载,不倒记入FIX10已经通过的被测树;以下带日期旧快照与FAIL/STOP/预算保留。
4+
15
> **FIX10代码审查R1事实与文档订正(2026-10-24,最终输入R2待):** 本地Claude session `235da2d1-f225-4c03-a85c-e84971911a44`,exit0,ER05/H0/最终VERDICT均PASS,无P0/P1;输出SHA256 `a5380d528ff218fe57c31644581059fd5ad0c172611ad65ce5426610ba1ede2a`。root选择在发布前关闭P2-A~C:订正陈旧RED/GREEN待执行文字,恢复误插FIX10的历史快照原文并保留当前依赖边,对齐Granularity写集措辞;上述变更另行R2审查,源码及A1输入不变,不冒用R1覆盖改后文档。root具名接受P2-D:头长度解析允许前导+但最终原解析器拒绝,至多回原EOF路径;P2-E:缺权限由未改exec入口先拒绝,现有完整SSH进程目标在最终C实跑,早处理不绕过此入口。无新scope/waiver。C/版本/提交/推送/D均未执行。
26
37
> **FIX10本地A/B结果(2026-10-24,代码ER05待):** 独立clone单生产文件SSH修复候选完成;旧源handler RED1PASS/4FAIL、真实Docker Git RED0PASS/2FAIL(原90秒期限)已保存。相同新测试GREEN5/5、真实Git2/2、完整SSH模块34/34,0failed/0ignored;nightly fmt、strict all-targets/all-features clippy、release build均exit0。原19项HTTP/SSH/PUSH/LFS smoke 19PASS/0FAIL/0skip,HTTP OpenAPI200/3.1.0/96paths、SSH就绪实测;自有服务exit0、DB/Redis键/容器/目录均清理,无source/runner漂移。A1 terminal SHA256 `bdfd4f90cb34791337ab0dbb84d9eaeae9c70c19db6fa050b304f2ec97b6f330`,原件`/tmp/fix-mtr10-A1-20261024/`;smoke log SHA256 `addaaa247a9809959889bd54deb08267e79c48c195f0e0372afd5a166624a59b`,原件`/tmp/fix-mtr10-entry-20261024/`。原始A/B不替代最终C:本卡尚无代码ER05、版本bump、签名提交、推送、tag/Release或D。原FIX07候选仍冻结/blocked,未随本卡发布。两仓当前完成数仍11/0/11,Libra不push。

0 commit comments

Comments
 (0)