Skip to content

Commit 5e1cbd0

Browse files
genednaclaude
andcommitted
v0.40.13: pin test Git LFS to 3.8.0; local test troubleshooting docs
- tests/integration_git_lfs.rs: host and git-cli runner assertions expect git-lfs/3.8.0. - Dockerfile.git-cli / Dockerfile.git-smoke: GIT_LFS_VERSION=3.8.0 with the amd64 / arm64 SHA-256 digests of the GitHub release assets. - docker compose (test and storage-only): runner image tags and healthchecks for 3.8.0. - docs/refactoring/test-infra.md: pins and image rows; retired CI workflows marked as historical; git-smoke profile corrected to smoke (with app). - docs/development.md: new chapter on local-environment test failures (git-lfs pin mismatch; Clash Verge / macOS system proxy breaking the S3 smoke, since mega2's HTTP stack ignores the proxy exception list and the test child runs with a cleared environment). - scripts/dev-test.sh, Dockerfile.git-smoke, docker-compose.test.yml: stale tips and comments fixed (the S3 smoke needs only RustFS up; runner networking; git-smoke needs --profile app with smoke). - docs/plan/plan-20260923.md: DEP-FU-05 progress; the final-tree full test run is green (18 targets, 0 failures), only the macOS ld note remains. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Eli Ma <eli@patch.sh>
1 parent 1394860 commit 5e1cbd0

11 files changed

Lines changed: 148 additions & 51 deletions

‎Cargo.lock‎

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Cargo.toml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[package]
22
name = "mega2"
3-
version = "0.40.12"
3+
version = "0.40.13"
44
edition = "2024"
55
# Two [[bin]] targets exist; keep `cargo run -p mega2` unambiguous.
66
default-run = "mega2"

‎Dockerfile.git-cli‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,15 @@
11
FROM alpine/git:v2.49.1@sha256:c0280cf9572316299b08544065d3bf35db65043d5e3963982ec50647d2746e26
22

33
ARG TARGETARCH
4-
ARG GIT_LFS_VERSION=3.7.1
4+
ARG GIT_LFS_VERSION=3.8.0
55

66
USER root
77

88
RUN set -eux; \
99
arch="${TARGETARCH:-$(uname -m)}"; \
1010
case "$arch" in \
11-
amd64|x86_64) arch=amd64; sha256=1c0b6ee5200ca708c5cebebb18fdeb0e1c98f1af5c1a9cba205a4c0ab5a5ec08 ;; \
12-
arm64|aarch64) arch=arm64; sha256=73a9c90eeb4312133a63c3eaee0c38c019ea7bfa0953d174809d25b18588dd8d ;; \
11+
amd64|x86_64) arch=amd64; sha256=e455e00f15d9b95661b8d53498ffb0c3367962cf1ec73c31ab7369516cd6ab8d ;; \
12+
arm64|aarch64) arch=arm64; sha256=ac9c8efac980bb0505ead384d087e2acb6486fd8498691a2165fa174ec6118c2 ;; \
1313
*) echo "unsupported git-lfs architecture: $arch" >&2; exit 1 ;; \
1414
esac; \
1515
archive="git-lfs-linux-${arch}-v${GIT_LFS_VERSION}.tar.gz"; \

‎Dockerfile.git-smoke‎

Lines changed: 9 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,26 +1,25 @@
11
# Linked git protocol smoke runner for the compose test stack.
22
#
3-
# Unlike `git-cli` (which uses `network_mode: host` for the cargo-native
4-
# black-box harness that spawns its own service on the host), this image is
5-
# designed to join `networks.default` and exercise push/pull against the
6-
# compose-hosted `mega2` service over the internal bridge network
7-
# (http://mega2:8000/).
3+
# Both runners sit on the default bridge network. `git-cli` reaches the
4+
# service that the cargo-native black-box harness spawns on the host through
5+
# `host.docker.internal`; this image instead exercises push/pull against the
6+
# compose-hosted `mega2` service (http://mega2:8000/).
87
#
98
# It extends the pinned alpine/git base (same git + git-lfs versions as
109
# `git-cli`) and adds:
1110
# - bash: `scripts/git_protocol_smoke.sh` uses bash features ([[ ]], local,
1211
# arrays) that Alpine's /bin/sh does not provide.
1312
# - postgresql-client: the smoke flow seeds a one-off access_token row into
1413
# the compose `postgres` service before running the push/delete matrix
15-
# (same pattern as the git-protocol-smoke CI job).
14+
# (same pattern as the former git-protocol-smoke CI job).
1615
#
1716
# Build context = mega2 repo root:
18-
# docker compose -p mega2-it -f docker/docker-compose.test.yml --profile git build git-smoke
17+
# docker compose -p mega2-it -f docker/docker-compose.test.yml --profile app --profile smoke build git-smoke
1918

2019
FROM alpine/git:v2.49.1@sha256:c0280cf9572316299b08544065d3bf35db65043d5e3963982ec50647d2746e26
2120

2221
ARG TARGETARCH
23-
ARG GIT_LFS_VERSION=3.7.1
22+
ARG GIT_LFS_VERSION=3.8.0
2423

2524
USER root
2625

@@ -38,8 +37,8 @@ RUN set -eux; \
3837
RUN set -eux; \
3938
arch="${TARGETARCH:-$(uname -m)}"; \
4039
case "$arch" in \
41-
amd64|x86_64) arch=amd64; sha256=1c0b6ee5200ca708c5cebebb18fdeb0e1c98f1af5c1a9cba205a4c0ab5a5ec08 ;; \
42-
arm64|aarch64) arch=arm64; sha256=73a9c90eeb4312133a63c3eaee0c38c019ea7bfa0953d174809d25b18588dd8d ;; \
40+
amd64|x86_64) arch=amd64; sha256=e455e00f15d9b95661b8d53498ffb0c3367962cf1ec73c31ab7369516cd6ab8d ;; \
41+
arm64|aarch64) arch=arm64; sha256=ac9c8efac980bb0505ead384d087e2acb6486fd8498691a2165fa174ec6118c2 ;; \
4342
*) echo "unsupported git-lfs architecture: $arch" >&2; exit 1 ;; \
4443
esac; \
4544
archive="git-lfs-linux-${arch}-v${GIT_LFS_VERSION}.tar.gz"; \

‎docker/docker-compose-storage-only.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -203,7 +203,7 @@ services:
203203
# For in-network LFS smoke (git-smoke → mega2), recreate with:
204204
# --env-file config/compose.env.storage-only.lfs-innetwork
205205
git-smoke:
206-
image: mega2-git-smoke:3.7.1
206+
image: mega2-git-smoke:3.8.0
207207
build:
208208
context: ..
209209
dockerfile: Dockerfile.git-smoke
@@ -222,7 +222,7 @@ services:
222222
- ..:/repo:ro
223223
working_dir: /work
224224
healthcheck:
225-
test: ["CMD-SHELL", "git --version >/dev/null && git lfs version | grep -q '^git-lfs/3[.]7[.]1 '"]
225+
test: ["CMD-SHELL", "git --version >/dev/null && git lfs version | grep -q '^git-lfs/3[.]8[.]0 '"]
226226
interval: 2s
227227
timeout: 5s
228228
retries: 20

‎docker/docker-compose.test.yml‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -95,7 +95,7 @@ services:
9595
# `export MEGA2_IT_GIT_UID=$(id -u) MEGA2_IT_GIT_GID=$(id -g)`
9696
# when the test process is not 1000.
9797
git-cli:
98-
image: mega2-git-cli:3.7.1
98+
image: mega2-git-cli:3.8.0
9999
build:
100100
context: ..
101101
dockerfile: Dockerfile.git-cli
@@ -118,7 +118,7 @@ services:
118118
# Override the image's `git` entrypoint so the container stays up for `exec`.
119119
entrypoint: ["sleep", "infinity"]
120120
healthcheck:
121-
test: ["CMD-SHELL", "git --version >/dev/null && git lfs version | grep -q '^git-lfs/3[.]7[.]1 '"]
121+
test: ["CMD-SHELL", "git --version >/dev/null && git lfs version | grep -q '^git-lfs/3[.]8[.]0 '"]
122122
interval: 2s
123123
timeout: 5s
124124
retries: 20
@@ -138,8 +138,8 @@ services:
138138
#
139139
# Then drive the smoke from the host:
140140
# docker compose -p mega2-it -f docker/docker-compose.test.yml \
141-
# --profile smoke exec -T git-smoke \
142-
# bash scripts/git_protocol_smoke.sh
141+
# --profile app --profile smoke exec -T git-smoke \
142+
# bash /repo/scripts/git_protocol_smoke.sh
143143
# (with MEGA2_HTTP_REPO_URL=http://mega2:8000/ and a seeded token).
144144
#
145145
# NOTE: git-smoke lives in its own `smoke` profile (not `git`) so that
@@ -148,7 +148,7 @@ services:
148148
# project invalid whenever the `git` profile is active but `app` is not
149149
# ("service git-smoke depends on undefined service mega2").
150150
git-smoke:
151-
image: mega2-git-smoke:3.7.1
151+
image: mega2-git-smoke:3.8.0
152152
build:
153153
context: ..
154154
dockerfile: Dockerfile.git-smoke
@@ -174,7 +174,7 @@ services:
174174
- ..:/repo:ro
175175
working_dir: /work
176176
healthcheck:
177-
test: ["CMD-SHELL", "git --version >/dev/null && git lfs version | grep -q '^git-lfs/3[.]7[.]1 '"]
177+
test: ["CMD-SHELL", "git --version >/dev/null && git lfs version | grep -q '^git-lfs/3[.]8[.]0 '"]
178178
interval: 2s
179179
timeout: 5s
180180
retries: 20

‎docs/development.md‎

Lines changed: 97 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -112,7 +112,7 @@ docker compose -p mega2-it -f docker/docker-compose.test.yml \
112112

113113
# 3) 注入连接串并跑全量(含黑盒 + 模块集成)
114114
cp -n .env.test.example .env.test
115-
# 若要跑 RustFS / S3-compatible smoke,取消 .env.test 中 MEGA_OBJECT_STORAGE__* 注释后重新 source
115+
# RustFS / S3-compatible smoke 只需 RustFS 已启动(上一步已含),无需改 .env.test(用例自行设置 S3 环境)
116116
source .env.test
117117
cargo test --all
118118
```
@@ -366,6 +366,102 @@ cp .env.test.example .env.test
366366

367367
零残留可按 compose project label 检查卷与网络为空(见 `test-infra.md`)。
368368

369+
## 调试汇总:本机环境导致的测试失败
370+
371+
下面两类失败都与代码无关,而是本机工具版本或网络代理造成的;表现像代码回归,排查时容易走弯路。先用本节的自检命令确认是不是环境问题,再看代码。
372+
373+
### git-lfs 版本与钉住版本不一致
374+
375+
**现象**:`integration_git_lfs` 的 4 个用例(`integration_git_lfs_storage_events_basic_upload`、`integration_git_lfs_storage_events_presigned_gap`、`integration_git_lfs_trunk_push_auth_none_round_trip`、`integration_git_lfs_trunk_push_auth_token_round_trip`)失败,断言信息为:
376+
377+
```text
378+
host git-lfs used by trunk LFS IT must be the pinned version
379+
left: "git-lfs/3.8.0"
380+
right: "git-lfs/3.7.1"
381+
```
382+
383+
**原因**:测试要求两处 git-lfs 与钉住版本完全相等——宿主机的 `git lfs version`(`tests/integration_git_lfs.rs` 的 `assert_host_git_lfs_pinned`,PATH 优先 `~/.local/bin`),以及 compose `git-cli` runner 容器内的版本(`assert_pinned_git_lfs`)。宿主 git-lfs 由 Homebrew 自动升级后,与仓库钉住的版本不再一致。
384+
385+
**解决**:钉住版本自 v0.40.13(2026-09-26)起为 `3.8.0`。以后宿主 git-lfs 再升级时,把钉住版本整体上调,而不是降级宿主:
386+
387+
1. 同步修改所有钉住位置:`tests/integration_git_lfs.rs` 的两处断言;`Dockerfile.git-cli` 与 `Dockerfile.git-smoke` 的 `GIT_LFS_VERSION` 和 amd64 / arm64 两个 `sha256`;两个 compose 文件的镜像标签与 healthcheck;[`test-infra.md`](./refactoring/test-infra.md) 的登记条目。注意 healthcheck 写成正则 `3[.]8[.]0`,按 `3.8.0` 字面搜索会漏掉。`sha256` 取自 GitHub release 的 asset digest(输出里已去掉 `sha256:` 前缀):
388+
389+
```bash
390+
version=3.8.0
391+
gh api "repos/git-lfs/git-lfs/releases/tags/v${version}" \
392+
--jq '.assets[] | select(.name | test("^git-lfs-linux-(amd64|arm64)-")) | "\(.name) \(.digest | ltrimstr("sha256:"))"'
393+
```
394+
395+
2. 重建 runner 镜像并只替换 `git-cli` 容器(构建会下载 git-lfs 发布包并用上面的 `sha256` 校验)。与上文「git 工作目录 EACCES / 挂载分叉」相同,先准备工作目录并导出 UID / GID:
396+
397+
```bash
398+
dir="${MEGA2_IT_GIT_WORKDIR:-/tmp/mega2-git}"
399+
mkdir -p "$dir" && chmod 1777 "$dir"
400+
export MEGA2_IT_GIT_UID="$(id -u)" MEGA2_IT_GIT_GID="$(id -g)"
401+
docker compose -p mega2-it -f docker/docker-compose.test.yml --profile git build git-cli
402+
docker compose -p mega2-it -f docker/docker-compose.test.yml --profile git up -d --wait --no-deps git-cli
403+
# 若也要跑 linked 栈级 smoke,同样重建 git-smoke(它依赖 profile app 的 mega2):
404+
docker compose -p mega2-it -f docker/docker-compose.test.yml --profile app --profile smoke build git-smoke
405+
```
406+
407+
**自检**:
408+
409+
```bash
410+
PATH="$HOME/.local/bin:$PATH" git lfs version # 宿主(与测试相同的 PATH 顺序)
411+
docker exec mega2-it-git-cli-1 git lfs version # compose runner
412+
```
413+
414+
两者的版本号都应等于钉住版本;只要有一个不等,`integration_git_lfs` 就会失败。
415+
416+
### 系统代理(Clash Verge)导致 S3 冒烟返回 502
417+
418+
**现象**:`integration_object_storage_s3_compatible_smoke` 失败,而直连 RustFS 正常:
419+
420+
```text
421+
Generic S3 error: Error performing PUT http://127.0.0.1:19000/mega2/... after 10 retries ...
422+
Server returned non-2xx status code: 502 Bad Gateway
423+
```
424+
425+
同样把对象存储指向 `127.0.0.1:19000` 的 `integration_git_lfs_storage_events_presigned_gap` 也可能因此失败(钉住版本不一致时,它会先在版本断言处失败,掩盖这个问题)。
426+
427+
**原因**(两层叠加):
428+
429+
1. **mega2 不读系统代理的例外列表。** 测试以 `env_clear()` 启动 mega2 子进程,`NO_PROXY` 传不进去;mega2 的 HTTP 栈(reqwest → hyper-util)在 macOS 上只读取系统代理的 HTTP / HTTPS 地址,不读「忽略这些主机与域的代理设置」(`ExceptionsList`)。因此即使系统设置里已把 `127.0.0.1` 列为例外,发往本机 RustFS 的请求仍会交给代理(例如 Clash Verge 的 `127.0.0.1:7897`)。
430+
2. **代理没有把回环地址设为直连。** 代理把 `127.0.0.1` 转发到远端节点,远端访问不到本机,于是返回 502。Clash Verge 2.x 中写在「扩展配置」(原 Merge)里的 `prepend-rules` **不会生效**——自 v1.7.x 起前置 / 后置规则移到订阅右键菜单的「编辑规则」里。
431+
432+
**解决**:在 Clash Verge 的订阅上右键 →「编辑规则」→「添加前置规则」,代理策略都选 `DIRECT`:
433+
434+
| 规则类型 | 规则内容 |
435+
|---|---|
436+
| IP-CIDR | `127.0.0.0/8`(必需,RustFS 在 `127.0.0.1:19000`) |
437+
| DOMAIN | `localhost` |
438+
| IP-CIDR6 | `::1/128` |
439+
| IP-CIDR | `192.168.0.0/16`、`10.0.0.0/8`、`172.16.0.0/12` |
440+
441+
也可以用「扩展脚本」把这些规则插到 `config.rules` 最前面,或在跑测试时临时关闭系统代理。
442+
443+
**自检**:
444+
445+
```bash
446+
scutil --proxy | grep -E 'HTTP(S)?(Enable|Proxy|Port)' # 系统代理是否开启、端口
447+
# RustFS 须已就绪,否则下面的冒烟会直接跳过;已在运行时这一步立即返回,未就绪时它会失败
448+
docker compose -p mega2-it -f docker/docker-compose.test.yml up -d --wait rustfs rustfs-init
449+
# 经系统代理访问 RustFS 健康检查:200 表示代理对回环地址直连,502 表示代理没有直连
450+
port="$(scutil --proxy | awk '/HTTPPort/{print $3}')"
451+
env -u NO_PROXY -u no_proxy curl -sS -x "http://127.0.0.1:${port}" -o /dev/null -w '%{http_code}\n' http://127.0.0.1:19000/health
452+
# 单独重跑 S3 冒烟
453+
source .env.test && cargo test -p mega2 --test integration_vault integration_object_storage_s3_compatible_smoke -- --nocapture
454+
```
455+
456+
该用例在连不上 `127.0.0.1:19000` 时会打印 `... requires RustFS ... skipping` 并直接返回,结果同样显示通过;只有输出里没有这行时,通过才说明 S3 路径真正跑过。
457+
458+
**调试陷阱**:`curl -x <代理>` 仍然遵守环境变量 `NO_PROXY`。当前 shell 的 `NO_PROXY` 含 `127.0.0.1` 时,curl 实际是直连,会得出「代理工作正常」的错误结论;验证代理时务必用 `env -u NO_PROXY -u no_proxy` 去掉它。
459+
460+
### 其它已知的本机噪音
461+
462+
- macOS 上 `cargo build --tests` 可能输出 `warning: linker stderr: ld: __eh_frame section too large ...` 的链接器提示。它是平台链接器的输出,不是代码问题,但 cargo 会把它计为 1 个 warning(`linker_messages`),因此仓库要求的 `cargo build --tests` 0 警告在 macOS 上仍不满足——这也是 plan-20260923「计划完成门」保持未勾的原因。
463+
- 这些环境问题在 [`plan-20260923.md`](./plan/plan-20260923.md) 中登记为 `DEP-FU-05`(本机基线环境失败)。
464+
369465
## 相关文档
370466

371467
- 编排事实源:[`docker/docker-compose.test.yml`](../docker/docker-compose.test.yml)

0 commit comments

Comments
 (0)