Repository navigation
feat(config): read the API key from MCPPROXY_API_KEY_FILE - #1716
Conversation
Docker Compose/Swarm secrets, Kubernetes secret volumes and systemd
LoadCredential= deliver a secret as a file. Add the *_FILE form for the
proxy's own API key so it does not have to sit in the process environment
(docker inspect, /proc/<pid>/environ, crash dumps) or in mcp_config.json.
One helper, config.ResolveEnvAPIKey, is now the only reader of the
environment's key, so the loader, EnsureAPIKey, the CLI client, status,
the tray and the core it launches agree:
- the file wins over MCPPROXY_API_KEY (reported, and logged as a warning);
- an empty MCPPROXY_API_KEY_FILE is unset;
- a named file that cannot be used (missing, unreadable, relative, over
64 KiB, empty) stops startup. No fallback to the other variable, the
config file or a generated key, and nothing is saved;
- the key is a process-only override and is never written back.
secret.ReadFile holds the read rules (absolute path, regular file after
symlinks, size cap, one trailing LF/CRLF removed, no NUL, errors never
include content) for the ${file:} provider that follows.
Config.EnsureAPIKey now returns an error. A tray-launched core reads the
file itself instead of receiving the key in its environment.
Spec 111 FR-014, FR-030 and US7 amended to cover the proxy's own key.
Refs smart-mcp-proxy#1706
The tray resolves MCPPROXY_API_KEY / MCPPROXY_API_KEY_FILE through config.ResolveEnvAPIKey so it matches the core. The helper reads the environment and the one key file the operator named; it never opens mcp_config.json.
|
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
|
Thanks, this is nicely done, especially the tray part. On precedence: please refuse to start when both MCPPROXY_API_KEY and MCPPROXY_API_KEY_FILE are set, with an error naming both variables. One unambiguous rule is easier to document than a warning somebody will miss, and it matches what the official images do. The env-over-config order you kept in #1717 (an env key wins over a ${file:} reference in api_key) is the right one there, since that is how every other env override in the config works. Review is next on my side. |
…KEY_FILE are both set One rule instead of a precedence: when both variables are non-empty, ResolveEnvAPIKey returns ErrAPIKeyBothSet, whatever the file holds. The error names both variables and never contains a key or file contents. The loader (also when the config file carries its own api_key), EnsureAPIKey, the CLI client, status and the tray all stop on it; the tray now resolves the environment key even over a socket so it does not hide the conflict from the core it launches. The file-wins warning and EnvAPIKeyResult.Shadowed are removed.
|
Done: setting both |
open(2) on a FIFO blocks until a writer appears, so a mis-pointed
MCPPROXY_API_KEY_FILE or ${file:} reference hung startup before the
regular-file check could run. Stat the path first and keep the check on
the open descriptor for a path swapped in between.
|
Thanks for the quick turnaround on the both-set rule. I verified the branch on a merge with main (build, the CI lint config, race tests for config, secret, tray and both binaries) and cross-reviewed it; one thing came out of that, and I pushed it on top of your branch rather than send you back for a ten-line change (77ee89e):
The CLI behaviour (warn and take the no-daemon path when the resolver errors, rather than exit) is fine by me: the core itself refuses to start, which was the ask. Merging once CI is green on the new head. |
Dumbris
left a comment
There was a problem hiding this comment.
Verified on a merge with main: build, CI lint config, race tests for config, secret, tray and both binaries; cross-reviewed. The FIFO guard is on the branch; the CLI warn-and-continue behaviour is accepted.
Dumbris
left a comment
There was a problem hiding this comment.
Verified on a merge with main: build, CI lint config, race tests for config, secret, tray and both binaries; cross-reviewed. The FIFO guard is on the branch; the CLI warn-and-continue behaviour is accepted.
|
Merged. Thank you, @james-crowley! Contributions like this are what keep MCPProxy moving, and they matter a lot to the project. One small ask, only if you feel like it: a short public note about your change (a tweet, a LinkedIn post, a line in a community you are part of) helps more people discover MCPProxy. No pressure at all, and I would be glad to repost it. |
) ## Description Second of the two PRs for #1706 (Spec 111 FR-030), on top of #1716 (merged). Closes #1706. The path allow-list is the separate follow-up #1721. Adds the `file` provider next to `env` and `keyring`. It is registered in `secret.NewResolver`, so `${file:/absolute/path}` works everywhere the existing references do: an upstream server's `env`, `args` and `headers` (resolved on a copy, so the stored config keeps the reference), `data_dir`, and the proxy's own `api_key`. - **Read rules** are `secret.ReadFile`'s from #1716: absolute path, symlinks followed but the target must be a regular file, 64 KiB cap, one trailing LF or CRLF removed (an LF file and a CRLF file resolve to the identical value), NUL rejected, errors name the reference and reason and never the content. An empty file is an error, as an empty env var is for `${env:}`. - **Read each resolve**, nothing cached, so a server picks up a rotated secret the next time it connects. - **Servers** keep the existing behaviour: a missing file is a missing-secret error naming the reference and the proxy stays up (US7 scenario 2). - **`api_key`**: a reference is resolved at load as a process-only override, so the process runs with the secret while every save, including the one a `PATCH /api/v1/config` triggers, writes the reference back and never the secret. One that does not resolve fails the load and never falls back to a generated key. A key from the environment overrides the file's, and the reference is then not read. Editing `api_key` itself is still persisted as the operator's edit. - Spec 111 FR-030, docs (`keyring-integration.md`, `upstream-servers.md`, `environment-variables.md`) and CHANGELOG updated. @Dumbris, one security question. `${file:}` reads any file the proxy process can read, so a server added through the `upstream_servers` tool with `env: {X: "${file:/data/mcp_config.json}"}` could hand that file to the server's process. `${env:}` already has the same exposure (it can name `MCPPROXY_API_KEY`), servers added that way are quarantined, and `allow_server_add` gates the tool, so I treated it as the same trust boundary and documented it. Would you like a path allow-list (for example a `secret_file_dirs` setting that defaults to `/run/secrets`, `/run/credentials` and the data directory's secrets folder) in this PR, or as a follow-up? ## Testing - [x] I have tested these changes locally - [x] I have added/updated tests that prove my fix is effective or my feature works - [x] All existing tests pass `go test ./internal/config ./internal/secret` pass. New tests cover the provider, LF/CRLF equality, error text without content, `api_key` resolving as an override and surviving a save and a `PersistableConfig` round trip, an edit being persisted, an unresolvable reference failing the load with the file untouched, an env key winning without reading the reference, and server `env`/`headers`/`args` expanding on a copy. By hand with a built binary: a reference in `api_key` authenticates, `PATCH {"tools_limit": 21}` is saved with `api_key` still the reference, a rotated file works after restart, and a missing file refuses to start.
Description
First of the two PRs for #1706 (the second adds
${file:}and builds on this one). AddsMCPPROXY_API_KEY_FILE, the*_FILEform used by Docker Compose/Swarm secrets, Kubernetes secret volumes and systemdLoadCredential=, so the proxy's own key need not sit in the process environment (docker inspect,/proc/<pid>/environ, crash dumps) or inmcp_config.json.config.ResolveEnvAPIKeyis now the only reader of the environment's key. The loader (Validate),EnsureAPIKey, the CLI client (resolveAPIKey),status, the tray and the core it launches all go through it.MCPPROXY_API_KEY, the config file or a generated key, and nothing is saved. A real run with a missing file exits 4 and leaves the config byte-identical.MCPPROXY_API_KEY(that would put it back in the core's environment); the core inheritsMCPPROXY_API_KEY_FILEand reads the file itself. The tray exits on an unusable file instead of generating a key.secret.ReadFileholds the read rules (absolute path, regular file after following symlinks, 64 KiB cap, exactly one trailing LF or CRLF removed, no NUL, errors never include content) so the${file:}provider in the follow-up behaves identically.Config.EnsureAPIKeynow also returns an error (it is internal; the five callers are updated).@Dumbris, one question on precedence. When both
MCPPROXY_API_KEYandMCPPROXY_API_KEY_FILEare set I made the file win and log a warning, since an explicit file is the more deliberate choice and keeps the key out of the environment. The alternatives are to prefer the env var (the order the issue first described) or to refuse to start with both set, which is what some official images do. Happy to switch; it is one condition inResolveEnvAPIKeyplus its tests.Testing
go test ./internal/config ./internal/secret ./cmd/mcpproxy ./internal/httpapipass. By hand with a built binary: key from the file authenticates (200, no key 401), the key is absent from/proc/<pid>/environand the config, a missing file exits 4 with the config unchanged,mcpproxy statusfinds the key.Not run here: the tray is built only for macOS/Windows. It compiles for Windows; the new
TestBuildCoreEnvironmentAPIKeyincmd/mcpproxy-tray/main_test.gois darwin-only and I could not compile or run it on this Linux host. The macOS native app'sCoreProcessManager.swiftis untouched (it removesMCPPROXY_API_KEYand passes its own key).