Skip to content

Commit f0a66b9

Browse files
benthecarmanbenthecarman
committed
Replace API keys with scoped macaroons
Use standard v2 macaroons so clients can restrict permissions, RPC methods, and expiry without access to server root keys. Support separate credentials with independent revocation through gRPC, the CLI, and MCP. Carry caller restrictions into newly issued credentials to prevent privilege escalation. Replace request HMAC headers with bearer macaroons over TLS. Clients must use the new credentials; existing event streams retain admission- time authorization. Add reference vectors and tests for token parsing, attenuation, persistence, and live authentication. AI assistance: OpenAI Codex.
1 parent f9bf766 commit f0a66b9

38 files changed

Lines changed: 2931 additions & 475 deletions

‎CONTRIBUTING.md‎

Lines changed: 18 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -19,8 +19,14 @@ cargo run --bin ldk-server ./contrib/ldk-server-config.toml
1919
## Testing
2020

2121
```bash
22-
cargo test # Run all tests
23-
cargo test --all-features # Run tests with all features
22+
cargo test # Run workspace tests
23+
cargo test --all-features # Run workspace tests with all features
24+
```
25+
26+
The end-to-end tests use a separate workspace. Run them with:
27+
28+
```bash
29+
cargo test --manifest-path e2e-tests/Cargo.toml -- --test-threads=4
2430
```
2531

2632
## Code Quality
@@ -50,7 +56,16 @@ cargo fmt --all
5056
2. Regenerate protos (see above)
5157
3. Create handler in `ldk-server/src/api/` (follow existing patterns)
5258
4. Add route in `ldk-server/src/service.rs`
53-
5. Add CLI command in `ldk-server-cli/src/main.rs`
59+
5. Map the RPC to its required permission in `method_authorization` in `ldk-server/src/macaroons.rs`.
60+
Unmapped methods return `UNIMPLEMENTED`, including requests made with an admin key.
61+
6. Add CLI command in `ldk-server-cli/src/main.rs`
62+
7. For a unary RPC, add its MCP schema, handler, and registry entry in `ldk-server-mcp/src/tools/`.
63+
Update the expected tools in `ldk-server-mcp/tests/integration.rs` and add live coverage in
64+
`e2e-tests/tests/mcp.rs` when applicable.
65+
8. Test requests with and without the required permission, including access with an admin key.
66+
67+
If the RPC needs a new permission, add it to `ldk-server-grpc/src/permissions.rs` and
68+
`ALL_PERMISSIONS`. Update any relevant presets and document the permission in `docs/api-guide.md`.
5469

5570
## Configuration
5671

‎docs/api-guide.md‎

Lines changed: 99 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -15,24 +15,93 @@ underlying LDK Node documentation.
1515

1616
## Authentication
1717

18-
Every gRPC request must include an `x-auth` metadata header with an HMAC-SHA256 signature:
18+
Every gRPC request must include a `macaroon` metadata header containing a hex-encoded v2
19+
binary macaroon:
1920

21+
```text
22+
macaroon: <hex-encoded-macaroon>
2023
```
21-
x-auth: HMAC <unix_timestamp>:<hmac_hex>
22-
```
2324

24-
Where:
25+
Macaroons use the standard HMAC-SHA256 key derivation and signature chain. The server supports
26+
first-party caveats only. It rejects third-party caveats, unknown conditions, malformed tokens,
27+
and tokens larger than 4096 binary bytes (8192 hex characters), with at most 32 caveats.
28+
An optional location field is a routing hint and is never used for authorization.
29+
30+
A macaroon is a bearer credential: anyone who obtains it can use its permitted operations.
31+
TLS is required. There is no per-request signature, body binding, or automatic replay protection.
32+
The old `x-auth` HMAC scheme and API keys are no longer accepted. Upgrade clients together with
33+
the server and supply the generated `macaroons/admin.macaroon` file or a scoped macaroon.
34+
The old `api_key` file is not imported.
35+
36+
### Caveats and delegation
2537

26-
- `unix_timestamp` is the current time in seconds since the Unix epoch
27-
- `hmac_hex` is the hex-encoded result of
28-
`HMAC-SHA256(api_key_bytes, timestamp_be_bytes || grpc_request_body_bytes)`
29-
- `api_key_bytes` is the API key string encoded as UTF-8 bytes
30-
- `timestamp_be_bytes` is the timestamp as a big-endian 8-byte unsigned integer
31-
- `grpc_request_body_bytes` is the raw gRPC request body sent over HTTP/2, including
32-
the 5-byte gRPC message frame
38+
All caveats must pass. Supported conditions use these exact forms:
3339

34-
The server rejects requests where the timestamp differs from the server's clock by more than
35-
**60 seconds**.
40+
| Caveat | Meaning |
41+
|--------|---------|
42+
| `permissions = node:read,payments:read` | Permit only these capabilities |
43+
| `method = GetNodeInfo` | Permit only this RPC method name |
44+
| `time-before = 1800000000` | Require server Unix time to be strictly less than this value |
45+
46+
Additional permission caveats intersect existing permissions. Adding `permissions = admin`
47+
to a restricted token does not restore admin access. Additional expiry conditions can only
48+
shorten its lifetime. Unknown or malformed conditions deny access.
49+
50+
Restrict a token locally, without contacting the server:
51+
52+
```bash
53+
ldk-server-cli attenuate-macaroon "$MACAROON" \
54+
--caveat 'permissions = node:read' \
55+
--caveat 'method = GetNodeInfo' \
56+
--caveat "time-before = $EXPIRY_UNIX_SECONDS"
57+
```
58+
59+
The command prints a hex token. The Rust client provides `macaroon::attenuate_macaroon` for the
60+
same operation. Give the restricted copy to the application and keep the original private.
61+
62+
Each `CreateMacaroon` call creates an independent root ID. Locally restricted copies retain
63+
the parent's ID; revoking that ID invalidates all such copies. To revoke clients independently,
64+
issue a separate macaroon for each client. The server cannot list copies made locally.
65+
Tokens created through the API inherit all the caller's caveats as well as their requested
66+
permissions. They have independent revocation IDs, so revoking the issuing credential does not
67+
revoke those separately issued tokens. Root keys are never returned by the API.
68+
69+
Authorization, including expiry, is checked when a request or event subscription starts.
70+
Revocation and expiry do not close an existing event stream. Reconnecting requires a valid token.
71+
72+
### Macaroon Permissions
73+
74+
Each macaroon has one or more capabilities. New RPCs are denied to scoped macaroons until they have an
75+
explicit capability mapping. The `admin` capability grants unrestricted access and must be used by
76+
itself.
77+
78+
| Capability | Access |
79+
|------------------------|---------------------------------------------------------------|
80+
| `node:read` | Node information, balances, and pathfinding scores |
81+
| `onchain:receive` | Create on-chain receive addresses |
82+
| `onchain:send` | Send on-chain funds |
83+
| `invoices:create` | Create BOLT11/BOLT12 invoices and incoming refund requests |
84+
| `payments:read` | Read payments and forwarded payments |
85+
| `payments:claim` | Claim or fail held BOLT11 payments |
86+
| `payments:send` | Send BOLT11, BOLT12, spontaneous, unified, and refund payments |
87+
| `channels:read` | List channels |
88+
| `channels:manage` | Open, configure, or cooperatively close channels |
89+
| `channels:splice` | Splice funds in or out, including to an external address |
90+
| `channels:force_close` | Force-close channels |
91+
| `peers:read` | List peers |
92+
| `peers:manage` | Connect or disconnect peers |
93+
| `messages:sign` | Sign messages and create BOLT12 payer proofs |
94+
| `messages:verify` | Verify message signatures |
95+
| `graph:read` | Read network graph data |
96+
| `utilities:read` | Decode invoices and offers |
97+
| `events:read` | Subscribe to the event stream |
98+
| `macaroons:manage` | Create, list, and revoke macaroons without privilege escalation |
99+
100+
Use `CreateMacaroon`, `ListMacaroons`, `RevokeMacaroon`, and `GetPermissions` to manage credentials.
101+
`CreateMacaroon` returns the hex bearer credential in `token`; list operations return metadata
102+
only. `GetPermissions` reports the effective permissions and caveats of the calling token. The CLI also provides `readonly`, `invoice`, and
103+
`admin` presets. MCP exposes the same operations as `create_macaroon`, `list_macaroons`,
104+
`revoke_macaroon`, and `get_permissions` tools.
36105

37106
## TLS
38107

@@ -67,9 +136,10 @@ Errors are returned as standard gRPC status codes:
67136
| gRPC Code | Meaning |
68137
|---------------------------|------------------------------------------------------------------|
69138
| `INVALID_ARGUMENT` (3) | Malformed request or invalid parameters |
139+
| `PERMISSION_DENIED` (7) | Valid macaroon without the required capability or with an unsatisfied caveat |
70140
| `FAILED_PRECONDITION` (9) | Lightning operation error (e.g., insufficient balance, no route) |
71141
| `INTERNAL` (13) | Server-side bug |
72-
| `UNAUTHENTICATED` (16) | Missing or invalid `x-auth` header |
142+
| `UNAUTHENTICATED` (16) | Missing, invalid, or revoked macaroon |
73143

74144
The `grpc-message` trailer contains a human-readable error description.
75145

@@ -232,6 +302,21 @@ Use events as notifications. After reconnecting, reconcile recoverable state wit
232302
`GetPaymentDetails`, `ListPayments`, `ListForwardedPayments`, and `ListChannels`. Some event fields
233303
cannot be recovered through these APIs.
234304

305+
### Macaroon Management
306+
307+
| RPC | Description |
308+
|------------------|---------------------------------------------------------------|
309+
| `CreateMacaroon` | Create a scoped macaroon and return its token |
310+
| `ListMacaroons` | List root IDs, names, permissions, and inherited caveats |
311+
| `RevokeMacaroon` | Revoke a key for new requests |
312+
| `GetPermissions` | Return the calling token’s ID, name, effective permissions, and caveats |
313+
314+
The first three RPCs require `macaroons:manage` or `admin`. A scoped key manager cannot create or
315+
revoke a key with permissions that it does not have. The final admin key cannot be revoked.
316+
Revoking a key blocks new requests, including new event subscriptions. Existing event streams
317+
remain open and continue to receive events until the client disconnects or the server stops.
318+
Authorization is checked only when a subscription starts.
319+
235320
### Metrics
236321

237322
Metrics are served as a plain HTTP GET endpoint (not gRPC):

‎docs/configuration.md‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -207,7 +207,11 @@ Two resolution methods are supported via the `mode` field:
207207
tls.crt # TLS certificate (PEM)
208208
tls.key # TLS private key (PEM)
209209
<network>/ # e.g., bitcoin/, regtest/, signet/
210-
api_key # API key
210+
macaroons/
211+
admin.macaroon # Hex-encoded initial admin bearer token (0400)
212+
roots/ # Server-only root keys (0700); never share this directory
213+
admin.toml # Initial root key and metadata (0400)
214+
<id>.toml # Independently revocable root keys (0400)
211215
ldk-server.log # Log file
212216
ldk_node_data.sqlite # LDK Node state (channels, wallet, payments)
213217
ldk_server_data.sqlite # Forwarded-payment history

‎docs/getting-started.md‎

Lines changed: 16 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -73,28 +73,34 @@ gRPC service listening on 127.0.0.1:3536
7373
NODE_URI: <node_id>@<address>
7474
```
7575

76-
Two files are auto-generated on first run:
76+
The admin macaroon and TLS certificate are auto-generated on first run:
7777

78-
| File | Location | Purpose |
79-
|-----------------|-----------------------------------|------------------------------------------|
80-
| API key | `<storage_dir>/<network>/api_key` | 32-byte random key (stored as raw bytes) |
81-
| TLS certificate | `<storage_dir>/tls.crt` | Self-signed ECDSA P-256 certificate |
78+
| File | Location | Purpose |
79+
|-----------------|---------------------------------------------------|-------------------------------------|
80+
| Admin macaroon | `<storage_dir>/<network>/macaroons/admin.macaroon` | Unrestricted API credential |
81+
| TLS certificate | `<storage_dir>/tls.crt` | Self-signed ECDSA P-256 certificate |
8282

8383
The default storage directory is `~/.ldk-server/` on Linux and
8484
`~/Library/Application Support/ldk-server/` on macOS.
8585

86-
### Reading the API Key
86+
### Reading the Macaroon
8787

88-
The API key file contains raw bytes. To get the hex string the CLI and client library expect:
88+
The CLI reads the admin macaroon automatically from the configured storage directory. No
89+
manual extraction is needed. To use the admin macaroon with another client, read
90+
`<storage_dir>/<network>/macaroons/admin.macaroon`. The entire file is the hex-encoded token.
91+
Do not copy files from the server-only `macaroons/roots/` directory.
92+
93+
Create a restricted macaroon for an application instead of copying the admin macaroon:
8994

9095
```bash
91-
xxd -p -c 64 ~/.ldk-server/bitcoin/api_key
96+
ldk-server-cli create-macaroon my-app --preset readonly
97+
ldk-server-cli create-macaroon invoice-app --preset invoice
9298
```
9399

94100
## First Commands
95101

96102
If the CLI and server share the same machine and use the default storage directory, the CLI
97-
auto-discovers the API key and TLS certificate, so no flags are needed:
103+
auto-discovers the macaroon and TLS certificate, so no flags are needed:
98104

99105
```bash
100106
# Check the node is running
@@ -113,7 +119,7 @@ details explicitly:
113119
```bash
114120
ldk-server-cli \
115121
--base-url localhost:3536 \
116-
--api-key <hex_api_key> \
122+
--macaroon <hex_macaroon> \
117123
--tls-cert /path/to/tls.crt \
118124
get-node-info
119125
```

‎docs/operations.md‎

Lines changed: 17 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,7 @@ the following config to `/etc/logrotate.d/ldk-server` (adjust the log path to ma
6060

6161
- Network graph data (re-synced from gossip or RGS)
6262
- Fee rate cache (re-fetched from the chain backend)
63-
- The API key (can be regenerated, but clients will need the new one)
63+
- Macaroon credentials (can be replaced, but clients will need new tokens)
6464
- The TLS certificate (can be regenerated, but clients will need the new one)
6565

6666
> **Warning:** Do not restore a backup onto two running nodes simultaneously. Running the
@@ -69,13 +69,21 @@ the following config to `/etc/logrotate.d/ldk-server` (adjust the log path to ma
6969
7070
## Security
7171

72-
### API Key
73-
74-
- Auto-generated as 32 random bytes on first startup
75-
- Stored at `<network_dir>/api_key` with `0400` permissions (read-only for owner)
76-
- The hex-encoded form of this key is used for HMAC authentication
77-
- Treat it as a secret: anyone with the API key and network access to the gRPC port can
78-
control the node
72+
### Macaroons
73+
74+
- An unrestricted admin token is generated at `<network_dir>/macaroons/admin.macaroon`.
75+
- Root keys stay in `<network_dir>/macaroons/roots/`. Never give these files to clients.
76+
- Use `create-macaroon` for independently revocable clients and `attenuate-macaroon` for local
77+
restrictions. Use the minimum permissions each client needs.
78+
- Treat tokens as secrets. A copied token grants its capabilities to the holder.
79+
- Revoking a root ID blocks new requests from it and all its locally restricted copies.
80+
Existing event streams continue until the client disconnects or the server stops. Expiry is
81+
also checked at subscription start only.
82+
- Restoring old root-key backups can restore revoked access. Preserve current revocation state
83+
when restoring a node, or replace its credentials.
84+
- The last unrestricted admin root cannot be revoked through the API. To rotate the initial
85+
admin token, create a new admin macaroon, save its token securely, then revoke the old ID.
86+
Update the default `admin.macaroon` file or pass `--macaroon` to use the new token.
7987

8088
### TLS
8189

@@ -188,7 +196,7 @@ To allow clients to connect from other machines:
188196
(e.g., `0.0.0.0:3536`).
189197
3. **Distribute the TLS certificate:** Copy `<storage_dir>/tls.crt` to each client machine.
190198
Clients must pin this certificate since it is self-signed.
191-
4. **Share the API key:** Provide the hex-encoded API key to authorized clients.
199+
4. **Share the macaroon:** Provide the hex-encoded macaroon to authorized clients.
192200

193201
If you regenerate the TLS certificate (by deleting `tls.crt` and `tls.key` and restarting),
194202
all clients will need the new certificate.

‎e2e-tests/src/lib.rs‎

Lines changed: 15 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,6 @@ use std::process::{Child, Command, Stdio};
1414
use std::time::Duration;
1515

1616
use corepc_node::Node;
17-
use hex_conservative::DisplayHex;
1817
use ldk_server_client::client::{EventStream, LdkServerClient};
1918
use ldk_server_client::ldk_server_grpc::api::{GetNodeInfoRequest, GetNodeInfoResponse};
2019
use ldk_server_client::ldk_server_grpc::events::event_envelope::Event;
@@ -97,7 +96,7 @@ pub struct LdkServerHandle {
9796
pub p2p_port: u16,
9897
pub storage_dir: PathBuf,
9998
pub config_path: PathBuf,
100-
pub api_key: String,
99+
pub macaroon: String,
101100
pub tls_cert_path: PathBuf,
102101
pub node_id: String,
103102
client: LdkServerClient,
@@ -343,31 +342,29 @@ impl LdkServerHandle {
343342
}
344343
});
345344

346-
// Wait for the api_key and tls.crt files to appear in the network subdir
345+
// Wait for the admin macaroon and TLS certificate files to appear.
347346
let network_dir = storage_dir.join("regtest");
348-
let api_key_path = network_dir.join("api_key");
347+
let macaroon_path = network_dir.join("macaroons").join("admin.macaroon");
349348
let tls_cert_path = storage_dir.join("tls.crt");
350349

351-
wait_for_file(&api_key_path, Duration::from_secs(30)).await;
350+
wait_for_file(&macaroon_path, Duration::from_secs(30)).await;
352351
wait_for_file(&tls_cert_path, Duration::from_secs(30)).await;
353352

354-
// Read the API key (raw bytes -> hex)
355-
let api_key_bytes = std::fs::read(&api_key_path).unwrap();
356-
let api_key = api_key_bytes.to_lower_hex_string();
353+
let macaroon = std::fs::read_to_string(&macaroon_path).unwrap().trim().to_string();
357354

358355
// Read TLS cert
359356
let tls_cert_pem = std::fs::read(&tls_cert_path).unwrap();
360357

361358
let base_url = format!("127.0.0.1:{grpc_port}");
362-
let client = LdkServerClient::new(base_url, api_key.clone(), &tls_cert_pem).unwrap();
359+
let client = LdkServerClient::new(base_url, macaroon.clone(), &tls_cert_pem).unwrap();
363360

364361
let mut handle = Self {
365362
child: Some(child),
366363
grpc_port,
367364
p2p_port,
368365
storage_dir,
369366
config_path,
370-
api_key,
367+
macaroon,
371368
tls_cert_path,
372369
node_id: String::new(),
373370
client,
@@ -547,10 +544,14 @@ pub struct McpHandle {
547544

548545
impl McpHandle {
549546
pub fn start(server: &LdkServerHandle) -> Self {
547+
Self::start_with_macaroon(server, &server.macaroon)
548+
}
549+
550+
pub fn start_with_macaroon(server: &LdkServerHandle, macaroon: &str) -> Self {
550551
let mcp_path = mcp_binary_path();
551552
let mut child = Command::new(&mcp_path)
552553
.env("LDK_BASE_URL", server.base_url())
553-
.env("LDK_API_KEY", &server.api_key)
554+
.env("LDK_MACAROON", macaroon)
554555
.env("LDK_TLS_CERT_PATH", server.tls_cert_path.to_str().unwrap())
555556
.stdin(Stdio::piped())
556557
.stdout(Stdio::piped())
@@ -602,8 +603,8 @@ pub fn run_cli_raw(handle: &LdkServerHandle, args: &[&str]) -> String {
602603
let output = Command::new(&cli_path)
603604
.arg("--base-url")
604605
.arg(handle.base_url())
605-
.arg("--api-key")
606-
.arg(&handle.api_key)
606+
.arg("--macaroon")
607+
.arg(&handle.macaroon)
607608
.arg("--tls-cert")
608609
.arg(handle.tls_cert_path.to_str().unwrap())
609610
.args(args)
@@ -758,9 +759,7 @@ pub async fn setup_funded_channel(
758759
.open_channel(OpenChannelRequest {
759760
node_pubkey: server_b.node_id().to_string(),
760761
address: format!("127.0.0.1:{}", server_b.p2p_port),
761-
amount: Some(open_channel_request::Amount::ChannelAmountSats(
762-
channel_amount_sats,
763-
)),
762+
amount: Some(open_channel_request::Amount::ChannelAmountSats(channel_amount_sats)),
764763
push_to_counterparty_msat: None,
765764
channel_config: None,
766765
announce_channel: true,

0 commit comments

Comments
 (0)