Skip to content

Commit 6b78e4b

Browse files
nappa85Marco Napetti
andauthored
feat: Fir-429 Broker dispatch, accept loop, and secret_providers config (#625)
## Why The pieces from PR08/09 (dictionary, broker transport, gateway, intercept, shim client) are all standalone modules; this PR is where firma-run's config layer learns to parse a `secret_providers` table and where the broker gets an actual accept/serve loop turning shim connections into real subprocess launches. Sandbox mounting/injection is deliberately deferred to PR11 so this stays reviewable as "config + dispatch logic" without sandbox concerns mixed in. ## What Changed - `crates/firma-run/src/config.rs`: - `ResolvedProfile.secret_providers: BTreeMap<String, IntegrationSpec>` — CLI vaults keyed by binary basename, HTTP vaults keyed by `provider_id`; merged across `[run.defaults]` and the active profile, later entries winning (profile overrides defaults, custom overrides built-in). Presence in this map is itself the authorization to intercept — no separate Cedar check. - `DEFAULT_SIDECAR_ENDPOINT`: now `unix:///tmp/sidecar.sock` on Unix (was `tcp://127.0.0.1:8080` unconditionally) — cfg-gated per `AGENTS.md`'s Unix/Windows platform-support rule. - `SecretMatcherConfig` — TOML-friendly mirror of `firma_core::SecretMatcher` with an explicit `type` tag. - `SecretProviderSpec` (`Cli`/`Http`, explicitly tagged rather than untagged, so a CLI-only field like `name` can never leak onto an HTTP entry) and `SecretProviderPatch` for merging. - `resolve_secret_providers`: resolves the merged table into `IntegrationSpec` values, looking up built-ins by name and validating custom matcher specs. - `crates/firma-run/src/secret/accept.rs` (new): the broker's accept loop — accepts shim connections from `BrokerListener` and serves each synchronously via `serve::serve_request`. A binary only reaches this loop because it matched a configured `secret_providers` entry, so no separate authorization decision happens here. - `crates/firma-run/src/secret/serve.rs` (new): per-request dispatch — turns one shim request into an actual vault CLI subprocess execution plus the `intercept` transform (PR09), applying the integration spec's credential env vars and extractor. - `crates/firma-run/src/{routing.rs, runtime.rs, sidecar/config.rs, sidecar/supervisor.rs}`: thread `secret_gateway_addr` and `http_secret_providers` through `AutostartFlags`/`SynthesizeRequest`/ `SpawnRequest` so the autostarted Sidecar receives the mirrored HTTP vault config and gateway address (consumed by PR07's handler). - `crates/firma-run/tests/integration/sidecar_autostart_*.rs`, `sidecar_config_merge.rs`: updated for the new fields threaded above. ## Risks / Notes - `accept`/`serve` run subprocess execution with forwarded credential env vars outside the sandbox — this is the actual trust boundary of the whole CLI-vault design; worth close review even though it's "just dispatch." - `DEFAULT_SIDECAR_ENDPOINT`'s platform split follows the repo's Unix/ Windows-only support rule (no fallback path for other targets). ## AI Assistance Generated with AI (Claude Code); human-reviewed for correctness and scope. --------- Co-authored-by: Marco Napetti <marco.napetti@gmail.com>
1 parent a1d9674 commit 6b78e4b

33 files changed

Lines changed: 1928 additions & 315 deletions

File tree

Cargo.lock

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

crates/firma-config-schema/src/lib.rs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,5 +20,6 @@ pub mod broker;
2020
pub mod gateway;
2121
pub mod run;
2222
pub mod secret_matcher;
23+
pub mod secret_provider;
2324
pub mod sidecar;
2425
pub mod utils;

crates/firma-config-schema/src/run.rs

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ use std::time::Duration;
1111

1212
use serde::{Deserialize, Serialize};
1313

14-
use crate::utils::NonZeroDuration;
14+
use crate::{secret_provider::SecretProviderPatch, utils::NonZeroDuration};
1515

1616
/// Sandbox backend selected for a Run profile.
1717
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
@@ -119,6 +119,13 @@ pub struct ProfilePatch {
119119
/// How the sandbox CA trust store is assembled. `None` resolves to the
120120
/// default `CaTrustMode::Sole`.
121121
pub ca_trust_mode: Option<CaTrustMode>,
122+
pub secret_gateway_addr: Option<String>,
123+
/// Secret providers to activate: bare strings reference a built-in
124+
/// integration, tables define a custom one. Additive across
125+
/// `[run.defaults]` and the active profile (like `env_passthrough`);
126+
/// entries appearing later win on name collision.
127+
#[serde(default)]
128+
pub secret_providers: Option<Vec<SecretProviderPatch>>,
122129
}
123130

124131
/// Mount entry patch.
Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
use serde::{Deserialize, Serialize};
2+
3+
use crate::secret_matcher::SecretMatcher;
4+
5+
/// Deserializable configuration for a CLI secret-provider integration.
6+
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
7+
#[serde(deny_unknown_fields)]
8+
pub struct CliSecretProviderConfig {
9+
/// Executable basename used to select this integration.
10+
pub binary_name: String,
11+
/// Stable identifier recorded for secrets from this integration.
12+
pub provider_id: String,
13+
/// Environment variables forwarded to authenticate the provider CLI.
14+
pub credential_env_vars: Vec<String>,
15+
/// Options ignored while identifying command words. Their value arity is
16+
/// honored, and the options remain unchanged in the executed command.
17+
#[serde(default)]
18+
pub stripped_options: Vec<FlagSpec>,
19+
/// Options that make an otherwise permitted invocation unsafe. An
20+
/// invocation containing one is blocked rather than silently changed.
21+
#[serde(default)]
22+
pub forbidden_options: Vec<FlagSpec>,
23+
/// Rules that classify invocations and configure secret extraction.
24+
pub matchers: Vec<CliMatcherRuleConfig>,
25+
}
26+
27+
/// A command-line option that a CLI integration needs to recognize.
28+
///
29+
/// ```toml
30+
/// { name = "--format", takes_value = true }
31+
/// { name = "--offline", takes_value = false }
32+
/// { name = "-u", takes_value = true, allow_attached_value = true }
33+
/// ```
34+
#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
35+
#[serde(deny_unknown_fields)]
36+
pub struct FlagSpec {
37+
/// The option's spelling, such as `--server-url` or `-u`.
38+
pub name: String,
39+
/// Whether the option consumes the following argument as its value.
40+
pub takes_value: bool,
41+
/// Whether the spelling accepts an attached value without `=`, such as
42+
/// `-uhttps://example.com`.
43+
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
44+
pub allow_attached_value: bool,
45+
}
46+
47+
impl FlagSpec {
48+
/// Creates a specification for an option whose value is a separate
49+
/// argument or follows `=`.
50+
#[must_use]
51+
pub fn value(name: &str) -> Self {
52+
Self {
53+
name: String::from(name),
54+
takes_value: true,
55+
allow_attached_value: false,
56+
}
57+
}
58+
59+
/// Creates a specification for an option that takes no value.
60+
#[must_use]
61+
pub fn valueless(name: &str) -> Self {
62+
Self {
63+
name: String::from(name),
64+
takes_value: false,
65+
allow_attached_value: false,
66+
}
67+
}
68+
69+
/// Creates a specification for an option that also accepts a value
70+
/// attached directly to its name.
71+
#[must_use]
72+
pub fn attached_value(name: &str) -> Self {
73+
Self {
74+
name: String::from(name),
75+
takes_value: true,
76+
allow_attached_value: true,
77+
}
78+
}
79+
}
80+
81+
/// One candidate rule for an [`CliSecretProviderConfig`].
82+
///
83+
/// Tagged by `type` (`sensitive_command` / `safe_command` / `blocked_command`)
84+
/// so it nests as a flat TOML table.
85+
#[derive(Debug, Clone, PartialEq, Eq, Deserialize, Serialize)]
86+
#[serde(tag = "type", rename_all = "snake_case", deny_unknown_fields)]
87+
pub enum CliMatcherRuleConfig {
88+
/// Response whose body must be scanned and redacted using `matcher`.
89+
SensitiveCommand {
90+
/// Command words, excluding the binary name.
91+
argv: Vec<String>,
92+
/// Whether trailing positional arguments are accepted.
93+
#[serde(rename = "match", default)]
94+
match_kind: CommandMatch,
95+
/// Matcher used to extract secrets from the normalized output.
96+
matcher: SecretMatcher,
97+
/// Output-shaping options skipped during matching and removed before
98+
/// [`CliMatcherRuleConfig::SensitiveCommand::append_options`] is applied.
99+
#[serde(default)]
100+
stripped_options: Vec<FlagSpec>,
101+
/// Options and values added to normalize output into the expected form.
102+
/// They are inserted before an end-of-options (`--`) marker when present.
103+
#[serde(default)]
104+
append_options: Vec<String>,
105+
},
106+
/// Known-safe path whose response never carries secrets; forwarded
107+
/// unredacted.
108+
SafeCommand {
109+
/// Command words, excluding the binary name.
110+
argv: Vec<String>,
111+
/// Whether trailing positional arguments are accepted.
112+
#[serde(rename = "match", default)]
113+
match_kind: CommandMatch,
114+
},
115+
/// Path that must always be denied.
116+
BlockedCommand {
117+
/// Command words, excluding the binary name.
118+
argv: Vec<String>,
119+
/// Whether trailing positional arguments are accepted.
120+
#[serde(rename = "match", default)]
121+
match_kind: CommandMatch,
122+
},
123+
}
124+
125+
/// Whether a command pattern permits trailing positional arguments.
126+
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Serialize, Deserialize)]
127+
#[serde(rename_all = "snake_case")]
128+
pub enum CommandMatch {
129+
/// Only the listed command words may occur. Options may be interspersed.
130+
Exact,
131+
/// Additional positional arguments may follow the listed command words.
132+
#[default]
133+
Prefix,
134+
}

crates/firma-config-schema/src/sidecar/secret_provider.rs renamed to crates/firma-config-schema/src/secret_provider/http.rs

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,12 @@
1-
//! Schema for `[[sidecar.http_secret_providers]]`.
1+
//! Schema for an HTTP secret provider (`type = "http"`).
22
//!
33
//! Concrete, non-generic representation of exactly what an operator writes in
4-
//! `firma.toml` for an HTTP secret provider. `firma-secret-provider` converts
5-
//! these into its runtime `HttpIntegrationSpec<SecretMatcher>` type (generic
4+
//! `firma.toml` for an HTTP secret provider — either as a `secret_providers`
5+
//! entry under `[run.defaults]` / `[run.profiles.<id>]` (`{ type = "http",
6+
//! provider_id = "...", host = "...", matchers = [...] }`) which `firma-run`
7+
//! mirrors into `[sidecar].http_secret_providers`, or directly as
8+
//! `[[sidecar.http_secret_providers]]` in a sidecar config. `firma-secret-provider`
9+
//! converts these into its runtime `HttpIntegrationSpec<SecretMatcher>` type (generic
610
//! over the matcher, and carrying the extraction behavior).
711
812
use serde::{Deserialize, Serialize};
Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
use serde::Deserialize;
2+
3+
pub mod cli;
4+
pub mod http;
5+
6+
/// One entry in `secret_providers`
7+
///
8+
/// Either a bare string naming an existing built-in integration (e.g. `"bws"`),
9+
/// or a full table defining a new custom integration (CLI or HTTP).
10+
/// The outer dispatch is string-vs-table; the CLI-vs-HTTP distinction *within*
11+
/// the table form is resolved by [`SecretProviderConfig`]'s own `type` tag.
12+
#[derive(Debug, Clone)]
13+
pub enum SecretProviderPatch {
14+
Named(String),
15+
Custom(Box<SecretProviderConfig>),
16+
}
17+
18+
impl<'de> Deserialize<'de> for SecretProviderPatch {
19+
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
20+
where
21+
D: serde::Deserializer<'de>,
22+
{
23+
struct PatchVisitor;
24+
25+
impl<'de> serde::de::Visitor<'de> for PatchVisitor {
26+
type Value = SecretProviderPatch;
27+
28+
fn expecting(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
29+
formatter.write_str(
30+
"a built-in integration name (bare string) or a custom integration table tagged by `type`",
31+
)
32+
}
33+
34+
fn visit_str<E>(self, value: &str) -> Result<Self::Value, E>
35+
where
36+
E: serde::de::Error,
37+
{
38+
Ok(SecretProviderPatch::Named(value.to_owned()))
39+
}
40+
41+
fn visit_string<E>(self, value: String) -> Result<Self::Value, E>
42+
where
43+
E: serde::de::Error,
44+
{
45+
Ok(SecretProviderPatch::Named(value))
46+
}
47+
48+
fn visit_borrowed_str<E>(self, value: &'de str) -> Result<Self::Value, E>
49+
where
50+
E: serde::de::Error,
51+
{
52+
Ok(SecretProviderPatch::Named(value.to_owned()))
53+
}
54+
55+
fn visit_map<M>(self, map: M) -> Result<Self::Value, M::Error>
56+
where
57+
M: serde::de::MapAccess<'de>,
58+
{
59+
let config = SecretProviderConfig::deserialize(
60+
serde::de::value::MapAccessDeserializer::new(map),
61+
)
62+
.map_err(|error| {
63+
serde::de::Error::custom(format!(
64+
"invalid secret_providers entry (expected a table with `type = \"cli\"` or `type = \"http\"`): {error}"
65+
))
66+
})?;
67+
Ok(SecretProviderPatch::Custom(Box::new(config)))
68+
}
69+
}
70+
71+
deserializer.deserialize_any(PatchVisitor)
72+
}
73+
}
74+
75+
/// A custom secret-provider integration spec
76+
///
77+
/// One full-table entry in `secret_providers`, explicitly tagged by `type`
78+
/// so a CLI-only field (e.g. `binary_name`) and an HTTP-only field (e.g.
79+
/// `host`) can never be mixed on the same entry — an untagged CLI-vs-HTTP
80+
/// guess would also give worse parse errors for a malformed table than an
81+
/// explicit tag does.
82+
///
83+
/// Minimal CLI example (JSON output with `{ key, value }` pairs):
84+
///
85+
/// ```toml
86+
/// [run.defaults]
87+
/// secret_providers = [
88+
/// { type = "cli", binary_name = "mock-vault", provider_id = "mock-vault", credential_env_vars = [], matchers = [{ type = "sensitive_command", argv = ["secret", "list"], matcher = { type = "json", record_path = "$[*]", value_path = "$.value", name = { source = "path", path = "$.key" } } }] },
89+
/// ]
90+
/// ```
91+
///
92+
/// Minimal HTTP example:
93+
///
94+
/// ```toml
95+
/// [run.defaults]
96+
/// secret_providers = [
97+
/// { type = "http", provider_id = "aws-secrets-manager", host = "secretsmanager.*.amazonaws.com", matchers = [{ type = "sensitive_command", path = "/GetSecretValue", matcher = { type = "json", record_path = "$", value_path = "$.SecretString", name = { source = "path", path = "$.Name" } } }] },
98+
/// ]
99+
/// ```
100+
#[derive(Debug, Clone, Deserialize)]
101+
#[serde(tag = "type", rename_all = "snake_case")]
102+
pub enum SecretProviderConfig {
103+
Cli(cli::CliSecretProviderConfig),
104+
Http(http::HttpSecretProviderConfig),
105+
}

crates/firma-config-schema/src/sidecar/mod.rs

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ use std::collections::HashMap;
99

1010
use serde::{Deserialize, Serialize};
1111

12-
use crate::gateway::GatewayConfig;
12+
use crate::{gateway::GatewayConfig, secret_provider::http::HttpSecretProviderConfig};
1313

1414
pub mod audit;
1515
pub mod authority;
@@ -20,7 +20,6 @@ pub mod infra;
2020
pub mod interceptor;
2121
pub mod local_exec;
2222
pub mod revocation;
23-
pub mod secret_provider;
2423
pub mod tenancy;
2524

2625
pub use audit::{AuditConfig, AuditSink};
@@ -37,7 +36,6 @@ pub use infra::{
3736
pub use interceptor::{ConnectRelayConfig, HttpsMitmConfig, InterceptorConfig, InterceptorMode};
3837
pub use local_exec::{DefaultAction, LocalExecConfig};
3938
pub use revocation::RevocationConfig;
40-
pub use secret_provider::{HttpMatcherRuleConfig, HttpSecretProviderConfig};
4139
pub use tenancy::{TenancyConfig, TenancyMode};
4240

4341
/// Top-level sidecar configuration, deserialized from the `[sidecar]` section

crates/firma-run/Cargo.toml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@ firma-identifiers.workspace = true
2424
firma-process-orchestrator.workspace = true
2525
firma-protobuf.workspace = true
2626
firma-runtime-state.workspace = true
27+
firma-secret-provider.workspace = true
2728
firma-sidecar.workspace = true
2829
hex.workspace = true
2930
jiff = { workspace = true, features = ["serde"] }
@@ -32,7 +33,7 @@ serde.workspace = true
3233
serde_json.workspace = true
3334
sha2.workspace = true
3435
thiserror.workspace = true
35-
tokio = { workspace = true, features = ["net", "rt", "time"] }
36+
tokio = { workspace = true, features = ["net", "rt", "time", "sync", "process", "io-util", "macros"] }
3637
toml.workspace = true
3738
toml_edit.workspace = true
3839
tonic.workspace = true

0 commit comments

Comments
 (0)