Skip to content

feat(config): read the API key from MCPPROXY_API_KEY_FILE - #1716

Merged
Dumbris merged 5 commits into
smart-mcp-proxy:mainfrom
james-crowley:feat/api-key-file
Oct 22, 2026
Merged

Dumbris merged 5 commits into
smart-mcp-proxy:mainfrom
james-crowley:feat/api-key-file

Conversation

@james-crowley

Copy link
Copy Markdown
Contributor

Description

First of the two PRs for #1706 (the second adds ${file:} and builds on this one). Adds MCPPROXY_API_KEY_FILE, the *_FILE form used by Docker Compose/Swarm secrets, Kubernetes secret volumes and systemd LoadCredential=, so the proxy's own key need not sit in the process environment (docker inspect, /proc/<pid>/environ, crash dumps) or in mcp_config.json.

  • One helper. config.ResolveEnvAPIKey is now the only reader of the environment's key. The loader (Validate), EnsureAPIKey, the CLI client (resolveAPIKey), status, the tray and the core it launches all go through it.
  • Hard fail. A named file that cannot be used (missing, unreadable, relative path, over 64 KiB, empty after removing the trailing newline) stops startup with an error naming the variable and path, never the content. There is no fallback to MCPPROXY_API_KEY, the config file or a generated key, and nothing is saved. A real run with a missing file exits 4 and leaves the config byte-identical.
  • Never written back. The key from the file is a process-only override, like the env key.
  • Tray. When the key is in a file, the tray does not inject it into the core as MCPPROXY_API_KEY (that would put it back in the core's environment); the core inherits MCPPROXY_API_KEY_FILE and reads the file itself. The tray exits on an unusable file instead of generating a key.
  • secret.ReadFile holds the read rules (absolute path, regular file after following symlinks, 64 KiB cap, exactly one trailing LF or CRLF removed, no NUL, errors never include content) so the ${file:} provider in the follow-up behaves identically.
  • Config.EnsureAPIKey now also returns an error (it is internal; the five callers are updated).
  • Spec 111 FR-014, FR-030 and US7 are amended so the spec covers the proxy's own key. Docs and CHANGELOG updated.

@Dumbris, one question on precedence. When both MCPPROXY_API_KEY and MCPPROXY_API_KEY_FILE are set I made the file win and log a warning, since an explicit file is the more deliberate choice and keeps the key out of the environment. The alternatives are to prefer the env var (the order the issue first described) or to refuse to start with both set, which is what some official images do. Happy to switch; it is one condition in ResolveEnvAPIKey plus its tests.

Testing

  • I have tested these changes locally
  • I have added/updated tests that prove my fix is effective or my feature works
  • All existing tests pass

go test ./internal/config ./internal/secret ./cmd/mcpproxy ./internal/httpapi pass. By hand with a built binary: key from the file authenticates (200, no key 401), the key is absent from /proc/<pid>/environ and the config, a missing file exits 4 with the config unchanged, mcpproxy status finds the key.

Not run here: the tray is built only for macOS/Windows. It compiles for Windows; the new TestBuildCoreEnvironmentAPIKey in cmd/mcpproxy-tray/main_test.go is darwin-only and I could not compile or run it on this Linux host. The macOS native app's CoreProcessManager.swift is untouched (it removes MCPPROXY_API_KEY and passes its own key).

Docker Compose/Swarm secrets, Kubernetes secret volumes and systemd
LoadCredential= deliver a secret as a file. Add the *_FILE form for the
proxy's own API key so it does not have to sit in the process environment
(docker inspect, /proc/<pid>/environ, crash dumps) or in mcp_config.json.

One helper, config.ResolveEnvAPIKey, is now the only reader of the
environment's key, so the loader, EnsureAPIKey, the CLI client, status,
the tray and the core it launches agree:
- the file wins over MCPPROXY_API_KEY (reported, and logged as a warning);
- an empty MCPPROXY_API_KEY_FILE is unset;
- a named file that cannot be used (missing, unreadable, relative, over
  64 KiB, empty) stops startup. No fallback to the other variable, the
  config file or a generated key, and nothing is saved;
- the key is a process-only override and is never written back.

secret.ReadFile holds the read rules (absolute path, regular file after
symlinks, size cap, one trailing LF/CRLF removed, no NUL, errors never
include content) for the ${file:} provider that follows.

Config.EnsureAPIKey now returns an error. A tray-launched core reads the
file itself instead of receiving the key in its environment.

Spec 111 FR-014, FR-030 and US7 amended to cover the proxy's own key.

Refs smart-mcp-proxy#1706
The tray resolves MCPPROXY_API_KEY / MCPPROXY_API_KEY_FILE through
config.ResolveEnvAPIKey so it matches the core. The helper reads the
environment and the one key file the operator named; it never opens
mcp_config.json.
@codecov-commenter

codecov-commenter commented Oct 21, 2026 •

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 79.56989% with 19 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
cmd/mcpproxy/status_cmd.go 0.00% 7 Missing ⚠️
internal/secret/file_read.go 80.55% 4 Missing and 3 partials ⚠️
cmd/mcpproxy/main.go 0.00% 3 Missing ⚠️
internal/config/config.go 89.47% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

@Dumbris

Dumbris commented Oct 21, 2026

Copy link
Copy Markdown
Member

Thanks, this is nicely done, especially the tray part. On precedence: please refuse to start when both MCPPROXY_API_KEY and MCPPROXY_API_KEY_FILE are set, with an error naming both variables. One unambiguous rule is easier to document than a warning somebody will miss, and it matches what the official images do. The env-over-config order you kept in #1717 (an env key wins over a ${file:} reference in api_key) is the right one there, since that is how every other env override in the config works. Review is next on my side.

…KEY_FILE are both set

One rule instead of a precedence: when both variables are non-empty,
ResolveEnvAPIKey returns ErrAPIKeyBothSet, whatever the file holds. The
error names both variables and never contains a key or file contents.

The loader (also when the config file carries its own api_key),
EnsureAPIKey, the CLI client, status and the tray all stop on it; the
tray now resolves the environment key even over a socket so it does not
hide the conflict from the core it launches. The file-wins warning and
EnvAPIKeyResult.Shadowed are removed.
@james-crowley

Copy link
Copy Markdown
Contributor Author

Done: setting both MCPPROXY_API_KEY and MCPPROXY_API_KEY_FILE (both non-empty) now refuses to start. The error is MCPPROXY_API_KEY and MCPPROXY_API_KEY_FILE are both set; set only one of them. It is fixed text, never contains a key or file contents, and is raised without opening the file. The loader (even when the config has its own api_key), EnsureAPIKey, the CLI client, status and the tray all stop on it; the file-wins warning is gone. Tests: TestResolveEnvAPIKey_BothSetIsAnError, TestEnsureAPIKey_BothSetIsAnErrorAndNeverGeneratesAKey, TestLoad_BothSetFailsTheLoad, and the "both variables set" case in TestResolveAPIKey_Precedence. Docs, CHANGELOG and spec 111 FR-014 describe the single rule. Head is 28730ef; all checks green.

open(2) on a FIFO blocks until a writer appears, so a mis-pointed
MCPPROXY_API_KEY_FILE or ${file:} reference hung startup before the
regular-file check could run. Stat the path first and keep the check on
the open descriptor for a path swapped in between.
@Dumbris

Dumbris commented Oct 22, 2026

Copy link
Copy Markdown
Member

Thanks for the quick turnaround on the both-set rule. I verified the branch on a merge with main (build, the CI lint config, race tests for config, secret, tray and both binaries) and cross-reviewed it; one thing came out of that, and I pushed it on top of your branch rather than send you back for a ten-line change (77ee89e):

  • secret.ReadFile called os.Open before the regular-file check. open(2) on a FIFO blocks until a writer appears, so a mis-pointed MCPPROXY_API_KEY_FILE hung startup instead of returning the documented error. It now stats the path first (the check on the open descriptor stays, for a path swapped in between), with a FIFO regression test under the unix build tag.

The CLI behaviour (warn and take the no-daemon path when the resolver errors, rather than exit) is fine by me: the core itself refuses to start, which was the ask. Merging once CI is green on the new head.

@Dumbris Dumbris left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified on a merge with main: build, CI lint config, race tests for config, secret, tray and both binaries; cross-reviewed. The FIFO guard is on the branch; the CLI warn-and-continue behaviour is accepted.

@Dumbris Dumbris left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified on a merge with main: build, CI lint config, race tests for config, secret, tray and both binaries; cross-reviewed. The FIFO guard is on the branch; the CLI warn-and-continue behaviour is accepted.

@Dumbris
Dumbris merged commit 97c8c8f into smart-mcp-proxy:main Oct 22, 2026
50 checks passed
@Dumbris

Dumbris commented Oct 22, 2026

Copy link
Copy Markdown
Member

Merged. Thank you, @james-crowley! MCPPROXY_API_KEY_FILE through one shared helper, with a hard fail on an unusable file and a refusal when both variables are set, is exactly how container and systemd deployments want to receive the key, and the tray part was the hard half. Rebase #1717 whenever you like and I will take it next.

Contributions like this are what keep MCPProxy moving, and they matter a lot to the project.

One small ask, only if you feel like it: a short public note about your change (a tweet, a LinkedIn post, a line in a community you are part of) helps more people discover MCPProxy. No pressure at all, and I would be glad to repost it.

Dumbris pushed a commit that referenced this pull request Oct 25, 2026
)

## Description
Second of the two PRs for #1706 (Spec 111 FR-030), on top of #1716 (merged). Closes #1706. The path allow-list is the separate follow-up #1721.

Adds the `file` provider next to `env` and `keyring`. It is registered in `secret.NewResolver`, so `${file:/absolute/path}` works everywhere the existing references do: an upstream server's `env`, `args` and `headers` (resolved on a copy, so the stored config keeps the reference), `data_dir`, and the proxy's own `api_key`.

- **Read rules** are `secret.ReadFile`'s from #1716: absolute path, symlinks followed but the target must be a regular file, 64 KiB cap, one trailing LF or CRLF removed (an LF file and a CRLF file resolve to the identical value), NUL rejected, errors name the reference and reason and never the content. An empty file is an error, as an empty env var is for `${env:}`.
- **Read each resolve**, nothing cached, so a server picks up a rotated secret the next time it connects.
- **Servers** keep the existing behaviour: a missing file is a missing-secret error naming the reference and the proxy stays up (US7 scenario 2).
- **`api_key`**: a reference is resolved at load as a process-only override, so the process runs with the secret while every save, including the one a `PATCH /api/v1/config` triggers, writes the reference back and never the secret. One that does not resolve fails the load and never falls back to a generated key. A key from the environment overrides the file's, and the reference is then not read. Editing `api_key` itself is still persisted as the operator's edit.
- Spec 111 FR-030, docs (`keyring-integration.md`, `upstream-servers.md`, `environment-variables.md`) and CHANGELOG updated.

@Dumbris, one security question. `${file:}` reads any file the proxy process can read, so a server added through the `upstream_servers` tool with `env: {X: "${file:/data/mcp_config.json}"}` could hand that file to the server's process. `${env:}` already has the same exposure (it can name `MCPPROXY_API_KEY`), servers added that way are quarantined, and `allow_server_add` gates the tool, so I treated it as the same trust boundary and documented it. Would you like a path allow-list (for example a `secret_file_dirs` setting that defaults to `/run/secrets`, `/run/credentials` and the data directory's secrets folder) in this PR, or as a follow-up?

## Testing
- [x] I have tested these changes locally
- [x] I have added/updated tests that prove my fix is effective or my feature works
- [x] All existing tests pass

`go test ./internal/config ./internal/secret` pass. New tests cover the provider, LF/CRLF equality, error text without content, `api_key` resolving as an override and surviving a save and a `PersistableConfig` round trip, an edit being persisted, an unresolvable reference failing the load with the file untouched, an env key winning without reading the reference, and server `env`/`headers`/`args` expanding on a copy. By hand with a built binary: a reference in `api_key` authenticates, `PATCH {"tools_limit": 21}` is saved with `api_key` still the reference, a rotated file works after restart, and a missing file refuses to start.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants