Skip to content

fix(docker): pick a git-capable isolation image, and classify container-toolchain failures - #1156

Merged
Dumbris merged 8 commits into
mainfrom
fix/1143-1144-docker-image-diag
Sep 1, 2026
Merged

Dumbris merged 8 commits into
mainfrom
fix/1143-1144-docker-image-diag

Conversation

@Dumbris

@Dumbris Dumbris commented Sep 1, 2026

Copy link
Copy Markdown
Member

Fixes #1143. Fixes #1144.

#1143 — the default uvx image cannot install most MCP servers

default_images mapped uvx/python/pip/pipx to ghcr.io/astral-sh/uv:python3.13-bookworm-slim, which has no git. A large share of MCP servers ship only as a git URL with no PyPI release, so under Docker isolation they could never install:

├─▶ Git operation failed
╰─▶ Git executable not found. Ensure that Git is installed and available.

GetDockerImage now resolves a git-capable image when a server's args carry git+ and no per-server isolation.image override is set, via a named default_images["uvx-git"] key operators can retarget. The global default stays slim — nobody pays ~700MB for a case they don't hit.

An operator's own image map wins. Review caught that the first attempt hardcoded a public ghcr.io URL that beat a mirrored/air-gapped operator's configured default_images.uvx, because the load-time merge seeds the key in every install so the "key absent" fallback never ran. Resolution now decides on values, not key presence: the operator's uvx-git wins outright; an explicitly-empty key opts out; a still-shipped built-in next to a retargeted runtime image stands down to the operator's image and logs a Warn naming the key to set. Pinned by a test that models the real load path — config file → LoadFromFile → merge → resolve — rather than hand-building the merged map.

#1144 — the resulting error told users to file a bug

The failure above classified as MCPX_UNKNOWN_UNCLASSIFIED, whose message is "Please file a bug report so we can add a specific code." — for a one-line config problem. New MCPX_DOCKER_MISSING_TOOLCHAIN covers container-toolchain failures, with a docs page following the existing MCPX_DOCKER_EXEC_NOT_FOUND pattern.

The hard part: not hijacking correct diagnoses

Container stderr is attacker-adjacent free text, and four separate review rounds found the new arms stealing classifications they shouldn't. Final behaviour, probed with the realistic wrapper shape mcpproxy actually produces (command="/usr/local/bin/docker", docker_isolation=true):

stderr code
Git executable not found (the real failure) MCPX_DOCKER_MISSING_TOOLCHAIN
git@github.com: Permission denied (publickey) MCPX_STDIO_EXIT_BEFORE_INITIALIZE
Cannot connect to the Docker daemon (on exit) MCPX_DOCKER_DAEMON_DOWN
same wording in a live container's stderr (timeout) MCPX_STDIO_HANDSHAKE_TIMEOUT
/bin/sh: 1: source: not found MCPX_STDIO_EXIT_BEFORE_INITIALIZE
docker: permission denied … daemon socket MCPX_DOCKER_NO_PERMISSION

Three of these were bugs found during review, each the same shape — a guard applied to one arm but not its sibling:

  • git SSH auth reported as "Docker permission denied." classifyDocker's arm is contains("permission denied") && contains("docker"), and mcpproxy's own error wrapper injects the docker command into the string — so under isolation that arm was near-unfalsifiable and swallowed any permission-denied text from child stderr.
  • A shell builtin (source: not found) read as a missing image toolchain, sending users to rebuild an image over a #!/bin/sh line. Now requires that every missing name be a non-builtin.
  • Daemon wording in a live container's log reported as DAEMON_DOWN. A handshake timeout means the container is alive, so a layer failure cannot also be true — the wording came from the server's own log. A genuine daemon refusal is unaffected: it surfaces on the exit path.

Verification

Four fix→re-review rounds (3-lens adversarial). The final round's "clean" verdict was wrong — its probes omitted the wrapper prefix, which is exactly how the git-auth misdiagnosis survived. Caught by hand-probing with the realistic shape; tests now pin that shape so it can't recur.

Gates: go build ./..., -tags server, go test ./internal/... -race, golangci-lint v2 → 0 issues.

…er-toolchain failures (#1143, #1144)

Docker isolation was unusable for the large share of MCP servers that ship
only as a git URL, and the resulting failure told the user to file a bug.

#1143 — the Python default image is Astral's slim uv image, which has no git:

  $ docker run --rm --entrypoint sh ghcr.io/astral-sh/uv:python3.13-bookworm-slim -c 'git --version'
  sh: 1: git: not found

so `uvx --from pkg@git+https://…` died at resolve time. Raising the global
default would cost every user a ~1.5GB pull for a case many never hit, so the
substitution is per-server: when a Python package runner's args reference a
`git+` URL and the server pins no `isolation.image`, resolution uses the new
`default_images.uvx-git` key (built-in `ghcr.io/astral-sh/uv:python3.13-bookworm`,
verified to ship git 2.39.5). Operators can retarget the key; an explicit
per-server override still wins; node's default (node:22) already has git and is
untouched. Configs written before the key existed are not re-seeded on load, so
the built-in value is also the hard fallback. ResolveDefaults reuses the same
resolution, so the REST/tray placeholder shows the image that will really run.

#1144 — container stderr is a different failure surface from mcpproxy's own
errors: anything the image lacks landed in MCPX_UNKNOWN_UNCLASSIFIED, whose
catalog entry asks the user to file a bug report. Adds
MCPX_DOCKER_MISSING_TOOLCHAIN, matched under the Docker-isolation hint for uv's
"Git executable not found" / "Git operation failed" wordings and the shell
`<tool>: not found` shape, ordered so the existing #696 CLI-missing and
OCI entrypoint codes keep winning their cases. "Git operation failed" with an
auth/DNS cause is excluded — that clone failed with git present. The
runtime-aware remediation says whether the automatic image selection already
covers the failure, and names a per-server override as the opt-out when one is
set.
…nd let the operator's image map win (#1143, #1144)

Follow-up to fc61a26 after a three-lens adversarial review. Five findings were
real; one was not.

Classifier (#1144):

- `shellToolNotFoundRe` matched a bare `<word>: not found` ANYWHERE in the
  error, and classifyDockerIsolatedSpawn runs before every generic stdio arm —
  so one line in the captured stderr tail claimed the whole failure. A server
  that died on its own config (`ERROR config profile production: not found`)
  was diagnosed as a missing container toolchain and sent to a docs page telling
  it to change images. The regex is now anchored on the shell prefix such a line
  always carries (`sh: 1:`, `/bin/sh: 1: exec:`, `bash: line 1:`). Being
  anchored, it is also specific enough to move above the ENTRYPOINT-missing arm,
  so the bash and dash wordings of one failure finally agree.

- A handshake TIMEOUT means the container is still running; a shell line written
  minutes earlier did not kill it. Toolchain attribution no longer overrides
  MCPX_STDIO_HANDSHAKE_TIMEOUT.

- `uv` collapses every clone failure into "Git operation failed", so the
  exclusion list is the only thing keeping a private-repo failure out of the
  missing-git diagnosis, and it covered three wordings. GitHub's
  password-authentication removal notice, 401/403, host-key verification,
  connection refused, name-resolution failure, `server certificate verification
  failed` and `repository '<url>' not found` all landed on "your image has no
  git". Broadened to the wordings a private repo actually produces; each one is
  text only git/ssh/curl can emit, which is itself proof git ran.

Image resolution (#1143):

- resolveDefaultImage reached past the operator's `default_images` to a
  hardcoded ghcr.io URL whenever the `uvx-git` key was absent — an image a
  mirrored or air-gapped host cannot pull at all, while their own `uvx` entry
  sat right there. The map is authoritative now: key set wins, key set to `""`
  is the opt-out, absent falls back to the runtime's configured image, and the
  built-in is used only where the map has no answer at all (the alternative
  being the alpine fallback, which has neither git nor python).

  The premise for the old fallback — "configs written before the key existed are
  not re-seeded on load" — is false: a config file's `default_images` is decoded
  INTO the built-in map, so the built-in entry survives an upgrade.
  TestDefaultImagesMergeOverBuiltInsOnLoad pins that.

- The "we swapped your image" log now tracks the substitution that actually
  happened rather than the server merely qualifying for one.

- The REST/tray `isolation_defaults.image` placeholder built a throwaway
  ServerConfig with no Args, so the git substitution never fired and the
  placeholder showed the slim image while the container ran the git one.

- hintsHaveGitDependency dropped the runtime-type half of
  config.NeedsGitCapableImage's gate, promising `node`/`go`/`ruby` users an
  automatic fix that never runs and naming a config key with no effect on them.

Not a defect: the docs' retarget snippet does not cost a user their other
runtime mappings — `default_images` merges over the built-in map rather than
replacing it. Documented that explicitly, along with the `""` opt-out.
…top stderr noise hijacking codes (#1143, #1144)

The git-capable image selection reached past the operator's own
`default_images` on the path that matters. A config file's map is decoded
INTO the built-in map, so `uvx-git` is populated in every install — the
"key absent" fallback the previous fix relied on never runs, and an
air-gapped or mirrored operator whose `uvx` entry points at their registry
still resolved the hardcoded public ghcr.io image their host cannot pull.

Presence cannot express intent after that merge, so the resolver now asks
whose VALUE each entry holds (config.IsBuiltInDefaultImage): an operator's
`uvx-git` wins outright; an emptied key still opts out; a shipped `uvx-git`
next to a retargeted runtime image stands down to the operator's image and
warns on spawn, naming the key that makes the choice explicit. The new test
drives the real path — config file without `uvx-git` → LoadFromFile → merge
→ resolve — instead of hand-building the merged map. The
MCPX_DOCKER_MISSING_TOOLCHAIN remediation mirrors the same rule so it stops
claiming a substitution that did not happen.

Classifier, same theme — a guard applied to one arm and not its siblings:

- the handshake-timeout suppression now covers arm (4) as well as (2)/(3),
  and the timeout arm moves above the spawn arms in classifyStdio, so a
  benign "no such file or directory" in the stderr tail of a container that
  is alive and silent no longer outranks MCPX_STDIO_HANDSHAKE_TIMEOUT;
- a shell reporting a BUILTIN missing (`/bin/sh: 1: source: not found`) is a
  bashism under dash, not a missing image toolchain: it no longer hijacks
  MCPX_DOCKER_MISSING_TOOLCHAIN / MCPX_DOCKER_EXEC_NOT_FOUND, restoring
  MCPX_STDIO_EXIT_BEFORE_INITIALIZE. A real missing tool alongside it still
  classifies;
- `Permission denied (publickey)` is git SSH auth, not a file mode on the
  server's executable: it lands with the other thirteen git-clone failure
  wordings on MCPX_STDIO_EXIT_BEFORE_INITIALIZE instead of
  MCPX_STDIO_SPAWN_EACCES.
…ot the word "docker" (#1144)

mcpproxy's own stdio wrapper formats every Docker-isolated failure as

  stdio transport (command="/usr/local/bin/docker", docker_isolation=true): %w

so the literal substring "docker" is present in ALL of them. The
`permission denied && docker` arm therefore fired on any "permission
denied" anywhere in the captured container stderr: a git deploy key
rejected by GitHub came back as MCPX_DOCKER_NO_PERMISSION, telling the
user to fix Docker socket permissions that were never broken — the exact
class of misleading diagnosis #1144 exists to remove. The arm was
effectively unfalsifiable under isolation.

- Gate the permission and image-pull arms on dockerDaemonEvidence: text
  only the docker CLI, the daemon or a socket dial can produce (docker
  daemon / docker.sock / pipe/docker_engine / dockerd / error response
  from daemon), none of which the wrapper injects.
- Share those arms as dockerLayerCode between classifyDocker and the
  docker-isolated spawn path, and consult it there BEFORE any stderr-tail
  arm. A REAL socket refusal previously fell through to the generic stdio
  "permission denied" arm and returned MCPX_STDIO_SPAWN_EACCES (chmod
  advice for a server binary that is fine); a daemon-down or pull-denied
  returned MCPX_STDIO_EXIT_BEFORE_INITIALIZE.
- Accept DockerIsolated on its own as implying the stdio rules.
  ResolveIsolation only reports mode=docker for a server that has a
  Command, so isolation is always stdio; without this a caller that filled
  only the isolation hint skipped every stdio arm and left mcpproxy's own
  wrappers on MCPX_UNKNOWN_UNCLASSIFIED and its "file a bug report" CTA,
  with the loose docker arms as the only thing that could fire.

Tests use the REAL wrapped message shape (command=…docker,
docker_isolation=true prefix) under both hint shapes; the previous
round's probes omitted the prefix, which is why the misfire survived.
…r daemon is down

A handshake timeout means the container is alive and merely silent, so
docker-daemon wording in its stderr tail is the server's own log line, not
evidence that our docker layer failed. A real daemon refusal never produces
a handshake timeout — no container would exist to time out — so gating the
layer arm on the same toolchainSuppressed guard the stderr-tail arms use
costs nothing and stops MCPX_DOCKER_DAEMON_DOWN sending users to restart
Docker over an application message.
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Sep 1, 2026 •

Copy link
Copy Markdown

Deploying mcpproxy-docs with  Cloudflare Pages  Cloudflare Pages

Latest commit: 891e892
Status: ✅  Deploy successful!
Preview URL: https://43a12814.mcpproxy-docs.pages.dev
Branch Preview URL: https://fix-1143-1144-docker-image-d.mcpproxy-docs.pages.dev

View logs

@Dumbris

Dumbris commented Sep 1, 2026

Copy link
Copy Markdown
Member Author

Correction: the air-gap claim in the PR description is not yet true

An independent opencode (gpt-5.6-sol) cross-review found five defects, and I reproduced the two High ones against this branch. One of them contradicts a claim I made above, so flagging it before anyone reads the description as fact.

The description says an operator's mirrored/air-gapped configuration wins. That holds only if they set default_images. An operator who sets only docker_isolation.registry still pulls public GHCR for a git-dependency server:

const DefaultGitCapableImage = "ghcr.io/astral-sh/uv:python3.13-bookworm"  // fully qualified

and buildFullImageName (isolation.go:433) returns any image that already contains a registry unchanged — so the configured registry is ignored. Earlier rounds fixed the default_images path and never covered the bare-registry path.

Second High: the arm-exemption pattern again. DockerCLINotFound and DockerOCIRuntime are still exempt from the handshake-timeout guard, so a live container's own log hijacks the diagnosis. Probed with the realistic wrapper:

child stderr during a handshake timeout got want
app: docker: command not found MCPX_DOCKER_CLI_NOT_FOUND MCPX_STDIO_HANDSHAKE_TIMEOUT
app: OCI runtime warning ignored MCPX_DOCKER_OCI_RUNTIME MCPX_STDIO_HANDSHAKE_TIMEOUT
app: exec format error while probing plugin MCPX_DOCKER_OCI_RUNTIME MCPX_STDIO_HANDSHAKE_TIMEOUT

The standing rationale for exempting these arms — "it means no container ever started" — is exactly as false for them as it was for the daemon arm that round 4 fixed: if the handshake timed out, a container is alive.

Three Mediums also stand: benign exit stderr stealing STDIOExitBeforeInitialize; Git operation failed defaulting to "missing toolchain" unless the cause is on a finite exclusion list (so fatal: could not read from remote repository claims git is absent even though git demonstrably ran); and an operator who explicitly sets uvx-git to the shipped value having that choice read as unset, because intent is inferred by value inequality.

Do not merge this PR until the follow-up lands. I will update the description once the claims in it are true.

@github-actions

github-actions Bot commented Sep 1, 2026 •

Copy link
Copy Markdown
Contributor

📦 Build Artifacts

Workflow Run: View Run
Branch: fix/1143-1144-docker-image-diag

Available Artifacts

  • archive-darwin-amd64 (29 MB)
  • archive-darwin-arm64 (26 MB)
  • archive-linux-amd64 (17 MB)
  • archive-linux-arm64 (15 MB)
  • archive-windows-amd64 (29 MB)
  • archive-windows-arm64 (26 MB)
  • frontend-dist-pr (0 MB)
  • installer-dmg-darwin-amd64 (23 MB)
  • installer-dmg-darwin-arm64 (21 MB)

How to Download

Option 1: GitHub Web UI (easiest)

  1. Go to the workflow run page linked above
  2. Scroll to the bottom "Artifacts" section
  3. Click on the artifact you want to download

Option 2: GitHub CLI

gh run download 33493222691 --repo smart-mcp-proxy/mcpproxy-go

Note: Artifacts expire in 14 days.

@codecov-commenter

codecov-commenter commented Sep 1, 2026 •

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 88.23529% with 24 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
internal/diagnostics/remediation.go 80.00% 6 Missing and 3 partials ⚠️
internal/diagnostics/classifier.go 90.58% 7 Missing and 1 partial ⚠️
internal/config/isolation_git.go 58.33% 5 Missing ⚠️
internal/diagnostics/hints/hints.go 0.00% 1 Missing ⚠️
internal/upstream/core/isolation.go 97.95% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

…ror own the git image

Five defects from a cross-model review of this branch, all of the same two
shapes: a diagnosis read out of the captured stderr TAIL outranking a fact
mcpproxy knows for certain, and operator intent inferred from a value that
cannot express it.

Classifier (internal/diagnostics):

- The handshake-timeout guard now covers EVERY arm of
  classifyDockerIsolatedSpawn that reads the stderr tail, not the two a
  previous round happened to fix. A timeout means the container is ALIVE, so
  "docker: command not found", "OCI runtime …", "exec format error" and "runc"
  in its stderr are the container's own log lines, not our docker layer
  failing. Only two things stay exempt, and the comment says which and why:
  the typed *exec.Error (the host's own failed execve of docker) and
  mcpproxy's own "docker not found in PATH / login shell" wording. The same
  guard is applied to classifyDocker, the sibling path a hint-less caller
  reaches, and the STDIO_HANDSHAKE_TIMEOUT arm now leads its switch instead of
  sitting below another tail-reading arm.

- On the EXIT path, the spawn-class codes (ENOENT / EACCES / ENOEXEC) are
  suppressed when mcpproxy's own premature-exit wrapper is present: that
  wrapper is written only when the transport closed, i.e. the child ran, so a
  "we never started it" code is provably wrong. "no such file or directory"
  now also requires exec/OCI context in the container arm — on its own it is
  an ordinary line about an optional file, and it was turning an unrelated
  cache warning into MCPX_DOCKER_EXEC_NOT_FOUND ahead of the real fatal error.

- "Git operation failed" no longer DEFAULTS to a missing git binary unless an
  exclusion list says otherwise. The claim now needs positive evidence: a
  wording naming the missing binary, or uv reporting the failure with no
  diagnostic of git's own beneath it. Anything git/ssh/curl themselves printed
  proves git ran. The old list could never be complete, so wordings it missed
  ("could not read from remote repository", an HTTP 429, a redirect) told the
  user to rebuild their image over a failure that proves the opposite.

Git-capable image (internal/config, internal/upstream/core):

- An operator who set only `docker_isolation.registry` still pulled public
  ghcr.io: the built-in image is fully qualified and buildFullImageName returns
  a qualified reference unchanged. The built-in image is now split into
  registry + repo and qualified with the configured registry ("" and the
  shipped "docker.io" both mean "the public default", and the image is not on
  Docker Hub).

- `uvx-git` is no longer seeded into the shipped `default_images`. SaveConfig
  writes the whole merged map back into the operator's file, so a seeded key's
  presence says nothing and value equality was the only signal left — which
  cannot express "I explicitly chose the image mcpproxy ships". With the key
  unseeded, its presence means exactly one thing, so an operator mirroring
  `uvx` who deliberately points `uvx-git` at the public image now gets it
  instead of their git-less mirror. The remediation text follows the same rule.

Docs updated for the new contract.

The key has never shipped in a release, so no existing install has it seeded;
a config written by an earlier build of this branch reads as an explicit
choice, which is what its value asks for.
…guard

isStdioHandshakeTimeout ORed in the loose "handshake timeout" substring,
which is unbounded child stderr: Go's own net/http TLS error carries it, so
a docker image pull over TLS printed it verbatim. On the exit path that let
the tail both suppress every docker arm and win the leading arm of
classifyStdio, turning a real pull or exec failure into "raise your
timeout".

mcpproxy never writes both of its wrappers, so the premature-exit wrapper
co-occurring with the loose substring is proof the text came from the tail,
not from us. Keyed on that.
…anence gate

main extracted classifier-hint building into internal/diagnostics/hints,
which already covers the Docker enrichment this branch was adding inline —
so the conflict resolves to main's hints.For call, with DockerArgs added
there because the #1143 git-dependency remediation reads it.

#1155's TestParkableCode_ChildStderrNeverProvesPermanence pinned the OLD
classifications to isolate its permanence gate; this branch fixes those
classifications, so the expected codes move. The gate's invariant is
unchanged and still asserted: two stderr-only rows that still reach a
PERMANENT code are added so it is never left untested.
@Dumbris

Dumbris commented Sep 1, 2026

Copy link
Copy Markdown
Member Author

Resolved — the do-not-merge hold is lifted

All five opencode findings are fixed, plus two more the follow-up review found. Verified by running the real code with the realistic wrapper shape:

child stderr code
Git executable not found (the original failure) MCPX_DOCKER_MISSING_TOOLCHAIN
git@github.com: Permission denied (publickey) MCPX_STDIO_EXIT_BEFORE_INITIALIZE
Cannot connect to the Docker daemon (on exit) MCPX_DOCKER_DAEMON_DOWN
same wording from a live container (timeout) MCPX_STDIO_HANDSHAKE_TIMEOUT
/bin/sh: 1: source: not found MCPX_STDIO_EXIT_BEFORE_INITIALIZE
docker: permission denied … daemon socket MCPX_DOCKER_NO_PERMISSION

The air-gap claim in the description is now true. DefaultGitCapableImage was split into registry + repo and is qualified with docker_isolation.registry, so an operator who sets only registry gets <registry>/astral-sh/uv:python3.13-bookworm. Pinned by a test that runs the real LoadFromFile → merge → resolve path with no default_images at all.

Two structural changes rather than more arm-patching:

  • dockerDaemonEvidence — the docker-permission arm now requires text only the daemon or a socket dial can produce. Previously it keyed on contains("permission denied") && contains("docker"), and mcpproxy's own wrapper injects command="/usr/local/bin/docker" into every isolated error, making that arm near-unfalsifiable.
  • uvx-git is no longer seeded into default_images, so its presence is the operator's intent. Value comparison could not express "explicitly chose the shipped default", because SaveConfig marshals the merged map back into the operator's file — anything mcpproxy seeds reappears there indistinguishable from something they typed.

git operation failed also no longer defaults to "missing toolchain": git is claimed absent only when it printed no diagnostic of its own, so fatal: could not read from remote repository is no longer reported as git being missing.

Merge-conflict resolution worth a look

main's internal/diagnostics/hints extraction supersedes this branch's inline hint building, so the conflict took main's side; DockerArgs was added there because the #1143 remediation reads it.

#1155's TestParkableCode_ChildStderrNeverProvesPermanence needed updating, and I want that visible rather than buried in a merge commit. It pinned the old classifications to isolate its permanence gate — e.g. a child printing ENOENT for its own data file was asserted to classify as MCPX_STDIO_SPAWN_ENOENT. This PR fixes exactly that misclassification, so the expected codes move. The gate's actual invariant (parkable == false for stderr-only evidence) is unchanged and still asserted, and I added two rows whose stderr still reaches a permanent code so the gate is never left incidentally untested.

@Dumbris
Dumbris merged commit 0909c4d into main Sep 1, 2026
39 checks passed
@Dumbris
Dumbris deleted the fix/1143-1144-docker-image-diag branch September 1, 2026 10:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants