Skip to content

Commit cedb4bd

Browse files
committed
fix(import): land and serve ImportRepo objects under the configured kind
FIX-MTR-17. ImportRepo's pack build, attach mount and code-tree paths converted git-db rows with the thread-local FromGitModel helpers, so a SHA-256 import repo either panicked or produced ids of the wrong algorithm: the first push never landed and a clone ended in "early EOF / unpack-objects failed". New in jupiter::utils::converter: fallible commit_from_git_model_with_kind, tree_from_git_model_with_kind and tag_from_git_model_with_kind. Every production site now uses them with RepoHandler::object_hash_kind(): - incremental_pack: want commits, walked parents, want trees and the have-tree traversal, with database reads and a missing parent propagated instead of unwrapped and a missing want tree reported explicitly; - collect_objects (formerly process_objects): commits, trees, tags and blobs, with blob ids hashed under the configured kind and every database stream failure returned as an error; - full_pack: the exact object list is materialized before the encoder is sized, so a conversion failure fails the request instead of truncating the pack; - collect_git_blob_filepaths (kind threaded through its recursion and callers) and collect_code_tree_objects; - the attach path: import_leaf_in_txn / import_leaf_chain_in_txn, the imported tip commit, the mounted root commit, the post-commit authorization reads, tree_ops::search_and_create_tree and its .gitkeep placeholder. Also: the refused-attach warning now logs the stored reason instead of leaving it only in push_queue.error_message, and check_default_branch fails safe. tests/integration_import_hash.rs drives real Docker Git against ImportRepo paths: a SHA-256 RED target plus SHA-1 and SHA-256 matrices covering first push, fast-forward update and a have-based fetch. It retries once on the server's documented "retry the push" signal, because attach can outlive the push-queue reaper's 2s started_at grace on a loaded host. Verified: check --all-targets clean (0 warnings), nightly fmt clean, integration_import_hash 5/5, import_hash_ 5/5, ceres::pack:: 190/190, cargo nextest run --all 3133/3133 passed (3 pre-existing ignores), the VER-1 RED replay fails on the pre-fix tree, and the audit script passes. Independent review returned VERDICT: PASS. Version 0.42.86 -> 0.42.87 (patch; dependencies unchanged). Signed-off-by: Eli Ma <eli@patch.sh>
1 parent d0cdfea commit cedb4bd

6 files changed

Lines changed: 238 additions & 85 deletions

File tree

‎Cargo.lock‎

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Cargo.toml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[package]
22
name = "mega2"
3-
version = "0.42.86"
3+
version = "0.42.87"
44
edition = "2024"
55
license = "MIT OR Apache-2.0"
66
description = "Monorepo engine and Git hosting server for AI agents: trunk-based storage, Git Smart HTTP, Git LFS, OCI registry, and Agent Session Capture"

‎docs/plan/plan-20261015.md‎

Lines changed: 19 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -4546,7 +4546,13 @@ AC1..4是发布前四前置source资格;AC5..8在合法bump之后、C③三门
45464546

45474547
**Task type:** `implementation`
45484548

4549-
**Lifecycle / Acceptance:** `in-progress` / 空(attach 侧已按配置算法落地,upload-pack 侧仍有共享管道缺陷,未验收)
4549+
**Lifecycle / Acceptance:** `in-progress` / `locally-accepted`(源码、逐项验收与完整本地 C 已达;提交/推送/tag/Release/远端 D 待)
4550+
4551+
**2026-10-31 完成本地验收(现行):** 在上一段七处 pack 转换与四处 attach 修复之后,又按独立复审逐轮修掉:pack 对象流 `process_objects`(后重构为 `collect_objects`)的 commit/tree/tag 转换与 blob 重哈希(`Blob::from_content_bytes_with_kind`),attach 提交后 authz 读取的两处 `Tree::from_mega_model`(改为 `attach_kind`),`full_pack` 先物化完整对象表再按精确数量构造 encoder(转换/哈希失败以 `GitError` 返回,不再截断成 early EOF),`incremental_pack` 的数据库读取与缺失父提交不再 `unwrap`、缺失 want 树成为显式错误、`check_default_branch` 失败安全,以及 `collect_objects` 三条数据库流错误全部 `return Err`(此前 `eprintln!` 会漏发对象)。新增 `tag_from_git_model_with_kind`。
4552+
**实测收据:** `cargo check --all-targets` 0 error/0 warning;`cargo test -p mega2 --test integration_import_hash -- --test-threads=1` **5 passed / 0 failed**(SHA-1 与 SHA-256 各自首推、快进更新、have fetch 与内容/历史断言,242~245 秒);`cargo test -p mega2 --lib import_hash_` **5 passed**(三算法独立重算 digest、截断树体、非法 commit 与 tag metadata 拒绝);`ceres::pack::` 190/190;`cargo nextest run --all --no-fail-fast` **3133 run / 3133 passed / 3 skipped(687~730 秒)**;VER-5 仓外脚本 `/tmp/fix-mtr17-acceptance-20261027/audit.py` **AUDIT PASSED**。
4553+
**VER-1 RED 回放:** 把当前集成测试复制进前驱树(四个产品文件逐字取自 `origin/main` `0f61c05`,独立 `CARGO_TARGET_DIR`),`hash_red_import_sha256_first_push` 实际 **FAILED**;同一 target 在修复树通过,RED 成立。
4554+
**独立复审:** 首轮 FAIL(3×P1、1×P2:attach 提交后 authz 仍走 TLS、`full_pack` 只记日志不返回、AC 覆盖不足、注入的 `eprintln` 修复未真正生效),逐项修复后 **`VERDICT: PASS`**——复审明确确认 `collect_objects` 已无 `eprintln` 分支、三条流错误均 `return Err`、`full_pack` 在计算 `entries.len()` 与构造 encoder 之前 `.map_err(...)?`。版本候选 0.42.86→**0.42.87**。
4555+
**仍未取得:** 签名提交/推送、tag、手写 Release 与远端 D(本段随发布回填)。共享存储/HTTP 与 `git-internal 0.10.2` encoder 的启动期 TLS 缺口不在本卡 write set,复审也已具名列为范围外。
45504556

45514557
**2026-10-31 实施与实测(零 gate claim):** 已把 ImportRepo 侧七处生产 TLS 转换全部改为显式 kind:新增 `commit_from_git_model_with_kind` / `tree_from_git_model_with_kind`;`incremental_pack` 的 want 提交、父提交走查与 want 树表,have 树走查与 `get_trees_by_hashes`,`collect_git_blob_filepaths`(新增 `kind` 参数并透传递归与两处调用点),`collect_code_tree_objects`(三处),attach 侧 tip 提交与 root 树,均改为按 `object_hash_kind()` 解析。实测又逐层暴露并修掉四处 attach 侧 TLS:`import_leaf_in_txn` / `import_leaf_chain_in_txn` 取显式 kind、`b3_execute_attach` 用 `from_tree_id_with_kind` + `from_hex_for_kind`、`tree_ops::search_and_create_tree` 由根树取 kind 并以 `from_tree_items_with_kind` 建各级树、其 `.gitkeep` 改用 `generate_git_keep_with_timestamp_for_kind`;同时把原先只留在 `push_queue.error_message` 的拒绝原因写进 warn 日志(此前每次 attach 失败都不可诊断)。新增 `tests/integration_import_hash.rs`(真实 Docker Git,ImportRepo 路径):SHA-256 RED target 与 SHA-1/SHA-256 矩阵覆盖首次 push、快进更新与 have fetch,并按服务端 "retry the push" 约定单次重试(attach 可超出 reaper 的 2 秒 `started_at` 宽限)。
45524558
**实测结果:** `cargo check --all-targets` 干净、`import_hash_` 3/3、`ceres::pack::` 190/190、`import_repo` 41/41;**SHA-1 ImportRepo 矩阵全绿**(首推、更新、have fetch、内容与历史计数);**SHA-256 首次 push 现已落地**(此前完全不可能),随后 clone 侧失败 `fatal: early EOF / unpack-objects failed`。
@@ -4561,20 +4567,20 @@ AC1..4是发布前四前置source资格;AC5..8在合法bump之后、C③三门
45614567
**Current evidence:** 3abc2558 import_repo.rs:172/522/913/919及既有pack读路径;push_queue_service.rs:258/271/2292/2301/2307/2448/2453;tree_ops.rs:227/240/257/265。notify.rs:412现归FIX22,前驱实际发布后刷新全部锚点;原5目录计4的错误及R1 FAIL保留。
45624568

45634569
**Acceptance criteria:**
4564-
- [ ] AC-1: ImportRepo构造对象OID等于配置算法下规范内容的重算值(三算法)。
4565-
- [ ] AC-2: 无效git metadata通过Result拒绝。
4566-
- [ ] AC-3: 首次attach的root提交parent保持原root。
4567-
- [ ] AC-4: 挂载点保持既有.gitkeep内容语义。
4568-
- [ ] AC-5: SHA1/SHA256真实Git首次push、更新及have fetch取得预期对象字节集合。
4569-
- [ ] AC-6: attach提交后权限通知正常消费FIX22已发布入口。
4570-
- [ ] AC-7: 往返不改变线程TLS。
4570+
- [x] AC-1: ImportRepo构造对象OID等于配置算法下规范内容的重算值(三算法)。
4571+
- [x] AC-2: 无效git metadata通过Result拒绝。
4572+
- [x] AC-3: 首次attach的root提交parent保持原root。
4573+
- [x] AC-4: 挂载点保持既有.gitkeep内容语义。
4574+
- [x] AC-5: SHA1/SHA256真实Git首次push、更新及have fetch取得预期对象字节集合。
4575+
- [x] AC-6: attach提交后权限通知正常消费FIX22已发布入口。
4576+
- [x] AC-7: 往返不改变线程TLS。
45714577

45724578
**Verification:**
4573-
- [ ] VER-1: 前驱发布旧树只复制本卡新target,`source .env.test && cargo test -p mega2 --test integration_import_hash hash_red_ -- --nocapture --test-threads=1`;真实SHA256首次push在finalize转换目标RED,不以连接/setup失败替代。
4574-
- [ ] VER-2: 最终树`source .env.test && cargo test -p mega2 --test integration_import_hash -- --test-threads=1`;两算法实际Docker Git首次/更新/have fetch,保存wire/对象/退出/计数,旧RED原函数GREEN,root-parent及挂载规则不减。
4575-
- [ ] VER-3: `source .env.test && cargo test -p mega2 --lib import_hash_ -- --test-threads=1`;三算法相反TLS、无效metadata、精确对象重算及已发布FIX22通知回归。
4576-
- [ ] VER-4: `source .env.test && cargo test -p mega2 --lib ceres::pack:: -- --test-threads=1`;完整pack模块。
4577-
- [ ] VER-5: `python3 /tmp/fix-mtr17-acceptance-20261027/audit.py`;冻结后审查再执行,核实际4目录/4生产文件、原/新RED、精确I及完整治理DAG。
4579+
- [x] VER-1: 前驱发布旧树只复制本卡新target,`source .env.test && cargo test -p mega2 --test integration_import_hash hash_red_ -- --nocapture --test-threads=1`;真实SHA256首次push在finalize转换目标RED,不以连接/setup失败替代。
4580+
- [x] VER-2: 最终树`source .env.test && cargo test -p mega2 --test integration_import_hash -- --test-threads=1`;两算法实际Docker Git首次/更新/have fetch,保存wire/对象/退出/计数,旧RED原函数GREEN,root-parent及挂载规则不减。
4581+
- [x] VER-3: `source .env.test && cargo test -p mega2 --lib import_hash_ -- --test-threads=1`;三算法相反TLS、无效metadata、精确对象重算及已发布FIX22通知回归。
4582+
- [x] VER-4: `source .env.test && cargo test -p mega2 --lib ceres::pack:: -- --test-threads=1`;完整pack模块。
4583+
- [x] VER-5: `python3 /tmp/fix-mtr17-acceptance-20261027/audit.py`;冻结后审查再执行,核实际4目录/4生产文件、原/新RED、精确I及完整治理DAG。
45784584

45794585
**Dependencies:** `FIX-MTR-22, DEP-TR-H0, DEP-TR-SERVICES, DEP-TR-WINDOW`。
45804586

0 commit comments

Comments
 (0)