Skip to content

Commit 981a4f2

Browse files
FarnaHerryFarnaHerry
authored
fix(godot-cpp): repack.sh for the multi-API 10.0.0-stable layout (+2 reproducibility bugs) (#537)
10.0.0-stable no longer ships a single gdextension/extension_api.json: one tag carries extension_api-4-3..4-7.json and the build picks one (scons api_version=, cmake GODOTCPP_DEFAULT_API_VERSION -- both say 4.7 for this release). The script hardcoded the single-API path and cannot build the tag. - resolve the API json the way the tag itself declares: explicit ENGINE_API > CMakeLists' GODOTCPP_DEFAULT_API_VERSION > highest present; single-API tags short-circuit on the file existing, so 4.5.0 / 10.0.0-rc1 stay byte-identical - LC_ALL=C on the "upstream tree untouched" check: it matches diff's "Only in <dir>: gen" line, which is localized -- on a zh_CN host it reads "只在 <dir> 中存在:gen", matches nothing, and every archive is refused - umask 002 + tar -xzpf: gen/ is created at repack time, so its mode bits come from the packer's umask and ride into the archive (002 -> 775/664, 022 -> 755/644). Same content, different sha256 per host; the two published archives are the 002 spelling. Verified (both published sha256s reproduce exactly; 4.7 is new + deterministic): 4.5.0 b0c36e77... = the index entry 10.0.0-rc1 aaafbf50... = the index entry 10.0.0-stable 66aaafb6... (x2), gen version.hpp = 4.7.0 stable The 10.0.0-stable version entry / example member / README rows are deliberately NOT here: they need the repacked archive published to xlings-res/godot-cpp (and the gitcode mirror) plus a v10.0.0-stable tag in mcpplibs/godot-cpp-m, neither of which an outside contributor can do. Hashes + follow-up checklist: .agents/docs/2026-10-17-add-godot-cpp-10.0.0-stable-plan.md
1 parent 8299f88 commit 981a4f2

2 files changed

Lines changed: 168 additions & 7 deletions

File tree

‎.agents/docs/2026-10-17-add-godot-cpp-10.0.0-stable-plan.md‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
# godot-cpp 走向多 API:repack.sh 支持 10.0.0-stable(4.7),并修两处不可复现
2+
3+
日期:2026-10-17 · 承接 [2026-08-04-add-godot-cpp-10.0.0-rc1.md](2026-08-04-add-godot-cpp-10.0.0-rc1.md)(4.6,PR #145)与
4+
[2026-08-04-add-godot-cpp-plan.md](2026-08-04-add-godot-cpp-plan.md)(4.5.0,PR #143)
5+
6+
## 1. 上游变动:一个 tag 不再等于一个引擎
7+
8+
`10.0.0-stable`(2026-09-15)的 `gdextension/` **不再有单一的 `extension_api.json`**,改为每个引擎小版本一份:
9+
10+
```
11+
gdextension/extension_api-4-3.json … extension_api-4-7.json gdextension_interface.json
12+
```
13+
14+
选哪份由构建方决定,上游自己三个入口是一致的:scons `api_version=`(`Makefile` 与 `test/SConstruct` 都是 `4.7`)、
15+
cmake `GODOTCPP_DEFAULT_API_VERSION`(`CMakeLists.txt` = `"4.7"`)、`tools/godotcpp.py::supported_api_versions`
16+
(`4.3 … 4.7`)。即 **10.0.0-stable 默认 = Godot 4.7**,同仓的 4.3–4.6 JSON 是给仍要老引擎的消费者的。
17+
18+
| 索引版本 | 上游 tag | 绑定引擎 |
19+
|---|---|---|
20+
| `4.5.0` | `godot-4.5-stable` | Godot 4.5 |
21+
| `10.0.0-rc1` | `10.0.0-rc1` | Godot 4.6 |
22+
| `10.0.0-stable` | `10.0.0-stable` | **Godot 4.7** |
23+
24+
因此「索引版本 → 引擎」这层映射在 10.x 里不能再靠 tag 名推断,得看生成时喂了哪份 API JSON(`repack.sh` 现在会打印)。
25+
26+
## 2. repack.sh:三处修改
27+
28+
### (a) 选 API JSON(否则 4.7 根本造不出来)
29+
30+
旧脚本硬编码 `gdextension/extension_api.json`,在 10.0.0-stable 上直接 `FileNotFoundError`。现在按**这个 tag 自己的
31+
声明**解析,而不是按 tag 名或硬编码版本:
32+
33+
1. 存在 `gdextension/extension_api.json` → 用它(4.5.0 / 10.0.0-rc1 行为逐字节不变);
34+
2. 否则取 `extension_api-<maj>-<min>.json`:显式 `ENGINE_API` 参数优先(给「同 tag 要烤老引擎」留出口),
35+
其次解析 `CMakeLists.txt` 的 `GODOTCPP_DEFAULT_API_VERSION`(即上游默认),最后回退到最高版本。
36+
37+
`generate_bindings()` 仍走原入口:无 build profile 时它等价于上游 scons 路径(`generate_trimmed_api("")` 原样返回
38+
api,`generate_template_get_node` 默认 True),不引入第二条生成链路。hooks 保持不传(上游默认 None)。
39+
40+
### (b) 校验步骤被 locale 左右(中英文机器行为不同)
41+
42+
「上游树未被改动」的判定解析的是 `diff` 的 `Only in <dir>: gen` 文本。中文 locale 下这行是
43+
`只在 <dir> 中存在:gen`,过滤不掉 → **任何归档都被判为「上游文件被改动」并拒绝**,与内容无关。改为 `LC_ALL=C diff`,
44+
让被解析的语言固定,而不是去匹配一个翻译。
45+
46+
### (c) 归档哈希取决于打包者的 umask(不可复现)
47+
48+
`gen/` 是本脚本**现场生成**的,其权限位来自本进程 umask(`0o777&~umask` 目录 / `0o666&~umask` 文件),并原样进包:
49+
50+
| | 上游文件(`tar -x` 解包) | `gen/`(生成) | 结果 |
51+
|---|---|---|---|
52+
| 旧脚本 + umask 022 | 755/644(被 umask 削掉) | 755/644 | sha 与已发布不符 |
53+
| 旧脚本 + umask 002 | 775/664 | 775/664 | 恰好是已发布的那份 |
54+
| **本 PR** | 775/664(`tar -xzpf` 保留上游模式) | 775/664(`umask 002`) | 与已发布一致,且与打包者 umask 无关 |
55+
56+
即:同一份内容,umask 不同 → sha 不同。已发布的两个归档是 **002** 的写法,把 umask 与 `-p` 都钉死,既让它们可复现,
57+
也不再让「谁能复现」取决于谁的 shell 配置。**不改变已发布归档的字节**,故索引里现有 sha 无需改动。
58+
59+
## 3. 验证(本机实测)
60+
61+
```bash
62+
tools/godot-cpp/repack.sh godot-4.5-stable 4.5.0 /tmp/out
63+
tools/godot-cpp/repack.sh 10.0.0-rc1 10.0.0-rc1 /tmp/out
64+
tools/godot-cpp/repack.sh 10.0.0-stable 10.0.0-stable /tmp/out
65+
tools/godot-cpp/repack.sh 10.0.0-stable 10.0.0-stable /tmp/out2 # 确定性
66+
```
67+
68+
| tag | 产出 sha256 | 与索引已记录对比 |
69+
|---|---|---|
70+
| `godot-4.5-stable` | `b0c36e77f02c4181352cdd7547b209b93a833be1ad6197f8c650d92987221a00` | **一致**(未改字节) |
71+
| `10.0.0-rc1` | `aaafbf50d4b8469d610fdb2eb76c6f58d758dbabbc6b013f60464d99b20ceb6e` | **一致**(未改字节) |
72+
| `10.0.0-stable` | `66aaafb642e7a390482dc381818640b1d0e583da2ef0cc9709214bcd13b68264` | 新增,两次独立运行一致 |
73+
74+
新归档的绑定目标经包内文件核对:`gen/include/godot_cpp/core/version.hpp` = `4.7.0 stable`;生成 1052 个类头
75+
(对比 rc1 的 ~1022 TU),上游树逐文件字节一致(`gen/` 是唯一新增)。
76+
77+
## 4. 本 PR 不含的部分(需要维护者权限)
78+
79+
索引版本条目只有在**归档可下载**之后才成立,CI 会真的去装它,所以下列步骤不在本 PR,交由维护者在合适时机补:
80+
81+
1. **发布归档**:把 `godot-cpp-10.0.0-stable.tar.gz` 发到 `github.com/xlings-res/godot-cpp`(GLOBAL)与
82+
`gitcode.com/mcpp-res/godot-cpp`(CN),tag `10.0.0-stable`——本仓贡献者对该组织无写权限。
83+
2. **模块包 tag**:`mcpplibs/godot-cpp-m` 目前有 `v4.5.0` / `v10.0.0-rc1`,需要 `v10.0.0-stable`
84+
(bump 自身 version 与 `compat.godot-cpp` 依赖,并对 4.7 头文件复核 `hashfuncs.hpp` shim)。
85+
3. 之后才能落 `pkgs/c/compat.godot-cpp.lua` + `pkgs/g/godotengine.godot-cpp-m.lua` 的 `10.0.0-stable` 条目、
86+
`tests/examples/godot-cpp-v10-stable` 成员与 README 两表——这三样在归档与 tag 就位后是一次纯登记改动。
87+
88+
## 5. 为什么值得进(消费者视角)
89+
90+
消费侧(如 [mcpp-kaki](https://github.com/farna/mcpp-kaki))现在的组合是「Godot 4.7.2 引擎 + 4.6 绑定」,靠
91+
`.gdextension` 的 `compatibility_minimum` 向上兼容跑着,但拿不到 4.7 新增 API、且 ABI 布局漂移只能运行期暴露。
92+
有了 4.7 绑定,版本两边才能真正对齐。

‎tools/godot-cpp/repack.sh‎

Lines changed: 76 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -14,16 +14,37 @@
1414
# * the upstream tag tarball, byte-for-byte (verified below), plus
1515
# * gen/, produced by running upstream's unmodified binding_generator.py.
1616
#
17+
# WHICH engine: up to 10.0.0-rc1 a tag carried exactly one API
18+
# (gdextension/extension_api.json). 10.0.0-stable is multi-API -- one tag ships
19+
# extension_api-<major>-<minor>.json for every engine minor it supports
20+
# (4.3..4.7) and the build picks one (scons `api_version=`, cmake
21+
# `GODOTCPP_DEFAULT_API_VERSION`). So one godot-cpp release no longer means one
22+
# engine, and the package version alone no longer says which bindings are
23+
# inside: resolve the file the way the tag itself declares, and pass the engine
24+
# explicitly when a package has to bake something other than the tag default.
25+
#
1726
# Reproduce:
1827
# tools/godot-cpp/repack.sh godot-4.5-stable 4.5.0 /tmp/out
28+
# tools/godot-cpp/repack.sh 10.0.0-stable 10.0.0-stable /tmp/out # 4.7 (tag default)
29+
# tools/godot-cpp/repack.sh 10.0.0-stable 10.0.0-stable-4.6 /tmp/out 4.6
1930
# then publish /tmp/out/godot-cpp-<version>.tar.gz to
2031
# github.com/xlings-res/godot-cpp (GLOBAL) and gitcode.com/mcpp-res/godot-cpp
2132
# (CN), both under the bare-version tag.
2233
set -euo pipefail
2334

24-
TAG="${1:?usage: repack.sh <upstream-tag> <bare-version> <outdir>}"
35+
TAG="${1:?usage: repack.sh <upstream-tag> <bare-version> <outdir> [engine-api]}"
2536
VERSION="${2:?}"
2637
OUTDIR="${3:?}"
38+
ENGINE_API="${4:-${ENGINE_API:-}}" # e.g. 4.7; ignored by single-API tags
39+
40+
# umask 002 is load-bearing for reproducibility, not a preference. gen/ is
41+
# created here, at repack time, so its mode bits come from THIS process's umask
42+
# (0o777&~umask for dirs, 0o666&~umask for files) and ride into the archive:
43+
# 002 -> 775/664, 022 -> 755/644. Same content, different sha256 -- the two
44+
# already-published archives (4.5.0 = b0c36e77..., 10.0.0-rc1 = aaafbf50...)
45+
# are the 002 spelling, and an earlier revision of this script only reproduced
46+
# under an English locale and a 002 umask. Pin it here rather than document it.
47+
umask 002
2748

2849
PYTHON="${PYTHON:-python3}"
2950
WRAP="godot-cpp-${TAG}"
@@ -37,42 +58,90 @@ curl -L -fsS -o "$WORK/upstream.tar.gz" \
3758
"https://github.com/godotengine/godot-cpp/archive/refs/tags/${TAG}.tar.gz"
3859
echo " upstream sha256: $(sha256sum "$WORK/upstream.tar.gz" | cut -d' ' -f1)"
3960

40-
tar -xzf "$WORK/upstream.tar.gz" -C "$WORK"
61+
# -p (--preserve-permissions) is load-bearing: GitHub's tag tarballs carry
62+
# 775/664 (group-writable), and extracting as a non-root user WITHOUT -p makes
63+
# tar apply the extractor's umask (022 -> 755/644). The modes then ride into the
64+
# packed archive, so two hosts with different umasks produce different sha256s
65+
# for identical content -- the published 4.5.0 archive is 0000775 and does not
66+
# reproduce under umask 022 without -p. Take the modes from upstream's archive,
67+
# not from whoever happens to be packing.
68+
tar -xzpf "$WORK/upstream.tar.gz" -C "$WORK"
4169
test -d "$WORK/$WRAP" || { echo "unexpected wrap dir in archive" >&2; exit 1; }
4270

4371
echo "==> generating bindings with upstream binding_generator.py"
4472
(
4573
cd "$WORK/$WRAP"
74+
# Which extension_api.json (see the header): explicit ENGINE_API wins, else
75+
# the tag's own declared default (CMakeLists.txt GODOTCPP_DEFAULT_API_VERSION,
76+
# which is what scons' Makefile/test/SConstruct agree with), else the highest
77+
# version present. Single-API tags short-circuit on the file existing, so
78+
# 4.5.0 / 10.0.0-rc1 keep resolving to the same path they always did.
79+
API_FILE="$(ENGINE_API="$ENGINE_API" "$PYTHON" -c "
80+
import glob, os, re
81+
if os.path.exists('gdextension/extension_api.json'):
82+
print('gdextension/extension_api.json')
83+
raise SystemExit
84+
files = sorted(glob.glob('gdextension/extension_api-*.json'),
85+
key=lambda p: tuple(int(n) for n in re.search(r'-(\d+)-(\d+)\.json$', p).groups()))
86+
if not files:
87+
raise SystemExit('no extension_api json in the tag')
88+
want = os.environ.get('ENGINE_API', '').strip()
89+
if want:
90+
target = 'gdextension/extension_api-%s.json' % want.replace('.', '-')
91+
if target not in files:
92+
raise SystemExit('this tag has no api file for %s' % want)
93+
print(target)
94+
raise SystemExit
95+
try:
96+
declared = re.search(r'GODOTCPP_DEFAULT_API_VERSION\s+\"([0-9]+\.[0-9]+)\"',
97+
open('CMakeLists.txt', encoding='utf-8').read())
98+
except OSError:
99+
declared = None
100+
if declared:
101+
target = 'gdextension/extension_api-%s.json' % declared.group(1).replace('.', '-')
102+
if target in files:
103+
print(target)
104+
raise SystemExit
105+
print(files[-1])
106+
")"
107+
echo " api json: ${API_FILE}"
108+
46109
# generate_bindings() grew an `interface_filepath` parameter between
47110
# godot-4.5-stable and 10.0.0-rc1, where gdextension_interface.h stopped
48111
# being checked in and became generated from gdextension_interface.json.
49112
# Dispatch on the actual signature and on which file the tag ships, rather
50113
# than on the tag itself, so this keeps working across the change in both
51114
# directions (cmake's GODOTCPP_GDEXTENSION_INTERFACE_FILE does the same).
52-
"$PYTHON" -c "
115+
API_FILE="$API_FILE" "$PYTHON" -c "
53116
import inspect
54117
import os
55118
import binding_generator
56119
120+
api_file = os.environ['API_FILE']
57121
params = inspect.signature(binding_generator.generate_bindings).parameters
58122
if 'interface_filepath' in params:
59123
interface = 'gdextension/gdextension_interface.json'
60124
if not os.path.exists(interface):
61125
interface = 'gdextension/gdextension_interface.h'
62126
binding_generator.generate_bindings(
63-
'gdextension/extension_api.json', interface,
127+
api_file, interface,
64128
True, '64', 'single', '.')
65129
else:
66130
binding_generator.generate_bindings(
67-
'gdextension/extension_api.json', True, '64', 'single', '.')
131+
api_file, True, '64', 'single', '.')
68132
"
69133
# generate_bindings() imports the module, which leaves bytecode behind
70134
find . -name '__pycache__' -type d -prune -exec rm -rf {} +
71135
)
72136

73137
echo "==> verifying the upstream tree is untouched"
74-
tar -xzf "$WORK/upstream.tar.gz" -C "$WORK" --transform "s|^${WRAP}|${WRAP}.orig|"
75-
diff -r --no-dereference "$WORK/${WRAP}.orig" "$WORK/${WRAP}" > "$WORK/diff.txt" || true
138+
tar -xzpf "$WORK/upstream.tar.gz" -C "$WORK" --transform "s|^${WRAP}|${WRAP}.orig|"
139+
# LC_ALL=C is load-bearing: the filter below matches diff's "Only in <dir>: gen"
140+
# line, and that line is localized -- on a zh_CN host it reads
141+
# "只在 <dir> 中存在：gen", matches nothing, and every archive is refused as
142+
# "upstream files were modified". Force the message language rather than try to
143+
# match a translation.
144+
LC_ALL=C diff -r --no-dereference "$WORK/${WRAP}.orig" "$WORK/${WRAP}" > "$WORK/diff.txt" || true
76145
if grep -v "^Only in $WORK/${WRAP}: gen\$" "$WORK/diff.txt" | grep -q .; then
77146
echo "upstream files were modified -- refusing:" >&2
78147
grep -v "^Only in $WORK/${WRAP}: gen\$" "$WORK/diff.txt" >&2

0 commit comments

Comments
 (0)