Skip to content

Commit bf03219

Browse files
BotCoder254claude
andcommitted
fix(runner): honor pax extended headers when repackaging the source tarball
git's tar writer stores any path longer than the 100-byte ustar name field in a pax extended header (type-x pseudo-entry) - it never uses the ustar prefix field. With GitHub's owner-repo-<sha>/ wrapper adding ~60 chars, 176 of this repo's own files cross that limit. The repackager copied the pseudo-entries through with a stripped path but unmodified record data, so Docker's extractor re-applied the ORIGINAL unstripped path and nested those files under /workspace/owner-repo-<sha>/... - the regular-file count stayed correct, making "checkout complete (394 files)" look healthy while tsc failed with dozens of TS2307 Cannot-find-module errors. The repackager now consumes type-x headers (carrying their `path` record to the following entry) and drops every pseudo-entry (x, g, GNU long name/link) from the output. Regression test crafts a git-shaped archive: global pax header + extended header + truncated regular entry; asserts the file unpacks at its full stripped path and no pseudo-entries survive. Runbook gains section 7b with the diagnostic fingerprint. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent 2d7d941 commit bf03219

2 files changed

Lines changed: 159 additions & 6 deletions

File tree

‎docs/fix-empty-workspace-stale-runner.md‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -305,6 +305,30 @@ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
305305
Whatever image a job pins, add it to `RUNNER_PREPULL_IMAGES` (comma-separated) so it
306306
is warmed into the daemon instead of pulled mid-pipeline.
307307

308+
## 7b. Long-path files "missing" from /workspace (pax extended headers)
309+
310+
Fingerprint (hit 2026-07-16 running this repo's own CI on an overup runner): the
311+
checkout log shows a **healthy file count** (`checkout complete (394 files)` — matching
312+
`git ls-files | wc -l` exactly), yet the job fails with dozens of
313+
`TS2307: Cannot find module` / `No such file or directory` errors, and every missing
314+
file has a LONG path. Short-path siblings in the same directory are present.
315+
316+
Cause: GitHub tarballs come from git's tar writer, which stores any path longer than
317+
the 100-byte ustar name field in a **pax extended header** (a type-`x` pseudo-entry
318+
that applies to the next entry) — git never uses the ustar prefix field. With the
319+
`owner-repo-<40-char-sha>/` wrapper (~60 chars), even modest source paths cross 100
320+
bytes. Runner binaries before the 2026-07-16 fix copied those pseudo-entries into the
321+
repackaged archive with a stripped path but **unmodified record data**, so Docker's
322+
extractor re-applied the ORIGINAL unstripped path and nested the affected files under
323+
`/workspace/owner-repo-<sha>/…`. The regular-file count stays correct, which is what
324+
makes the log look healthy.
325+
326+
Fix: runner ≥ the commit adding `repackage_applies_pax_long_paths_and_drops_pseudo_entries`
327+
resolves the pax `path` for the following entry and drops all pseudo-entries from the
328+
output. Roll out per §2–§4 (rebuild → verify in-image → revoke + re-create). To confirm
329+
a failing workspace exhibits this: `ls /workspace` in a debug step — the wrapper dir
330+
sitting next to your source tree is the tell.
331+
308332
## 8. Troubleshooting quick reference
309333

310334
| Symptom | Diagnosis | Fix |
@@ -313,6 +337,7 @@ is warmed into the daemon instead of pulled mid-pipeline.
313337
| Rebuilt + pushed, jobs still old | Image built from a stale clone (fresh timestamp, old binary) | §2 `git log` check, §3 grep check |
314338
| Pulled new image on VPS, jobs still old | Runner containers were restarted, not re-created | §4 revoke + re-create |
315339
| `checkout failed: repository archive contained no usable files` | New binary working as designed: the tarball filtered to zero files (unexpected layout) | Inspect the repo/tarball; this replaces the old silent-empty behavior |
340+
| File count healthy but LONG-path files "missing"; wrapper dir visible inside /workspace | Pax extended headers mishandled by a pre-2026-07-16 runner binary | §7b, then roll out per §2–§4 |
316341
| `image pull failed (…); using locally cached image` (job continues) | New binary tolerating a transient pull failure | Nothing — informational; heal the host per §6 when frequent |
317342
| `failed to extract layer … Lchown … no such file or directory` | Docker-host snapshotter state | §6 |
318343
| `sh: 1: <tool>: not found` in a step | Job image lacks the toolchain | §7 |

‎runner/src/executor.rs‎

Lines changed: 134 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@
2121
//! container and comes up empty. The archive API is daemon-agnostic and works
2222
//! identically for on-host, containerized, and remote-`DOCKER_HOST` runners.
2323
24-
use std::path::{Component, Path};
24+
use std::path::{Component, Path, PathBuf};
2525
use std::sync::{Arc, Mutex};
2626
use std::time::{Duration, Instant};
2727

@@ -860,22 +860,52 @@ fn repackage_stripped(tar_path: &Path) -> anyhow::Result<RepackagedSource> {
860860
let mut builder = tar::Builder::new(Vec::new());
861861
let mut skipped_links: u64 = 0;
862862
let mut files: u64 = 0;
863+
// git's tar writer stores any path longer than the 100-byte ustar name
864+
// field in a pax extended header (a type-`x` pseudo-entry that applies
865+
// to the NEXT entry) — it never uses the ustar prefix field. Carry that
866+
// path across so long-path files keep their real names, and never copy
867+
// the pseudo-entries into the output: their record data still holds the
868+
// ORIGINAL unstripped path, which Docker's extractor would re-apply,
869+
// nesting those files under the wrapper dir inside /workspace.
870+
let mut pax_path: Option<PathBuf> = None;
863871
for entry in archive.entries()? {
864872
let mut entry = entry?;
865873

866-
// Symlink/hardlink entries are dropped outright: a link TARGET can't
867-
// be validated by the path check below, and one pointing outside the
868-
// workspace would let later entries (or the job) escape it. Source
869-
// archives from the Contents API don't need them.
870874
match entry.header().entry_type() {
875+
tar::EntryType::XHeader => {
876+
if let Some(extensions) = entry.pax_extensions()? {
877+
for extension in extensions.flatten() {
878+
if extension.key().ok() == Some("path")
879+
&& let Ok(value) = extension.value()
880+
{
881+
pax_path = Some(PathBuf::from(value));
882+
}
883+
}
884+
}
885+
continue;
886+
}
887+
// Global pax headers and GNU long-name pseudo-entries are
888+
// metadata, not files — never part of the output.
889+
tar::EntryType::XGlobalHeader
890+
| tar::EntryType::GNULongName
891+
| tar::EntryType::GNULongLink => continue,
892+
// Symlink/hardlink entries are dropped outright: a link TARGET
893+
// can't be validated by the path check below, and one pointing
894+
// outside the workspace would let later entries (or the job)
895+
// escape it. Source archives from the Contents API don't need
896+
// them.
871897
tar::EntryType::Symlink | tar::EntryType::Link => {
872898
skipped_links += 1;
899+
pax_path = None;
873900
continue;
874901
}
875902
_ => {}
876903
}
877904

878-
let path = entry.path()?.into_owned();
905+
let path = match pax_path.take() {
906+
Some(path) => path,
907+
None => entry.path()?.into_owned(),
908+
};
879909
// Strip GitHub's top-level wrapper, keep only plain segments, and
880910
// rebuild the path with `/` so it is portable into the container.
881911
// A leading `./` (some tar producers emit it) is consumed first so
@@ -1093,6 +1123,104 @@ mod tests {
10931123
assert!(!dest.path().join("repo-sha").exists());
10941124
}
10951125

1126+
/// One pax record in the `%len key=value\n` wire format.
1127+
fn pax_record(key: &str, value: &str) -> Vec<u8> {
1128+
// len counts its own digits plus ` key=value\n`.
1129+
let base = key.len() + value.len() + 3;
1130+
let mut len = base;
1131+
loop {
1132+
let candidate = base + len.to_string().len();
1133+
if candidate == len {
1134+
break;
1135+
}
1136+
len = candidate;
1137+
}
1138+
format!("{len} {key}={value}\n").into_bytes()
1139+
}
1140+
1141+
/// git's tar writer never uses the ustar prefix field: a path longer
1142+
/// than the 100-byte name field arrives as a pax extended header
1143+
/// (type `x`) followed by a regular entry with a truncated name. The
1144+
/// repackager must restore the real path and drop the pseudo-entry —
1145+
/// copying it through made Docker re-apply the ORIGINAL unstripped
1146+
/// path, nesting 176 of this repo's own files under the wrapper dir
1147+
/// (the empty-looking /workspace CI failure of 2026-07-16).
1148+
#[test]
1149+
fn repackage_applies_pax_long_paths_and_drops_pseudo_entries() {
1150+
let long_rel = format!("src/{}/useRepositories.ts", "d".repeat(110));
1151+
let long_path = format!("repo-sha/{long_rel}");
1152+
1153+
let tmp = tempfile::NamedTempFile::new().unwrap();
1154+
let gz = flate2::write::GzEncoder::new(
1155+
std::fs::File::create(tmp.path()).unwrap(),
1156+
flate2::Compression::default(),
1157+
);
1158+
let mut builder = tar::Builder::new(gz);
1159+
1160+
// Global pax header, the way git leads every archive.
1161+
let comment = pax_record("comment", "abc123");
1162+
let mut global = tar::Header::new_ustar();
1163+
global.set_entry_type(tar::EntryType::XGlobalHeader);
1164+
global.set_size(comment.len() as u64);
1165+
global.set_cksum();
1166+
builder
1167+
.append_data(&mut global, "pax_global_header", &comment[..])
1168+
.unwrap();
1169+
1170+
// Extended header carrying the real path, then the truncated entry.
1171+
let record = pax_record("path", &long_path);
1172+
let mut xheader = tar::Header::new_ustar();
1173+
xheader.set_entry_type(tar::EntryType::XHeader);
1174+
xheader.set_size(record.len() as u64);
1175+
xheader.set_cksum();
1176+
builder
1177+
.append_data(&mut xheader, "repo-sha/paxheader", &record[..])
1178+
.unwrap();
1179+
let contents = b"export {};";
1180+
let mut file = tar::Header::new_ustar();
1181+
file.set_entry_type(tar::EntryType::Regular);
1182+
file.set_size(contents.len() as u64);
1183+
file.set_mode(0o644);
1184+
file.set_cksum();
1185+
builder
1186+
.append_data(&mut file, "repo-sha/truncated", &contents[..])
1187+
.unwrap();
1188+
1189+
// A short-path sibling must be unaffected by the carried pax path.
1190+
let mut plain = tar::Header::new_ustar();
1191+
plain.set_entry_type(tar::EntryType::Regular);
1192+
plain.set_size(2);
1193+
plain.set_mode(0o644);
1194+
plain.set_cksum();
1195+
builder
1196+
.append_data(&mut plain, "repo-sha/package.json", &b"{}"[..])
1197+
.unwrap();
1198+
builder.into_inner().unwrap().finish().unwrap();
1199+
1200+
let source = repackage_stripped(tmp.path()).unwrap();
1201+
assert_eq!(source.files, 2);
1202+
1203+
// No pseudo-entries survive into the output archive.
1204+
let mut archive = tar::Archive::new(&source.tar[..]);
1205+
for entry in archive.entries().unwrap() {
1206+
let entry = entry.unwrap();
1207+
assert!(matches!(
1208+
entry.header().entry_type(),
1209+
tar::EntryType::Regular
1210+
));
1211+
}
1212+
1213+
let dest = tempfile::tempdir().unwrap();
1214+
tar::Archive::new(&source.tar[..]).unpack(dest.path()).unwrap();
1215+
assert_eq!(
1216+
std::fs::read(dest.path().join(&long_rel)).unwrap(),
1217+
contents
1218+
);
1219+
assert!(dest.path().join("package.json").is_file());
1220+
assert!(!dest.path().join("truncated").exists());
1221+
assert!(!dest.path().join("repo-sha").exists());
1222+
}
1223+
10961224
/// A tarball whose entries all get filtered out (here: links only) must
10971225
/// report zero files so the caller fails the checkout instead of
10981226
/// uploading an empty archive and logging success.

0 commit comments

Comments
 (0)