Replace HMAC Auth with scoped macaroons

We initially implemented HMAC auth but it doesn't allow for scoped
permissions without creating a new token for each use case. Macaroons
support this natively and allows for users to do more complex things
without us having to build out tons of support.

This implements macaroons from scratch and replaces our existing HMAC
Auth.

AI assistance: OpenAI Codex; reviews by GPT-6 Astra and Claude Fable
5.1.

Author: benthecarman

CONTRIBUTING.md

@@ -19,8 +19,15 @@ cargo run --bin ldk-server ./contrib/ldk-server-config.toml
 ## Testing
 
 ```bash
-cargo test                     # Run all tests
-cargo test --all-features      # Run tests with all features
+cargo test                     # Run workspace tests
+cargo test --all-features      # Run workspace tests with all features
+```
+
+Run the end-to-end tests from their separate workspace. Raw gRPC tests require `curl` with
+HTTP/2 support (`curl --version` must list `HTTP2`):
+
+```bash
+cargo test --manifest-path e2e-tests/Cargo.toml -- --test-threads=4
 ```
 
 ## Code Quality
@@ -50,7 +57,15 @@ cargo fmt --all
 2. Regenerate protos (see above)
 3. Create handler in `ldk-server/src/api/` (follow existing patterns)
 4. Add route in `ldk-server/src/service.rs`
-5. Add CLI command in `ldk-server-cli/src/main.rs`
+5. Map the RPC to its required permission in `method_authorization` in `ldk-server/src/macaroons/authorization.rs`.
+   Unmapped methods return `UNIMPLEMENTED`, even for admin tokens.
+6. Add CLI command in `ldk-server-cli/src/main.rs`
+7. For a unary RPC, add the MCP tool in `ldk-server-mcp/src/tools/` and update the tool list test
+   in `ldk-server-mcp/tests/integration.rs`. Add a live test in `e2e-tests/tests/mcp.rs` if applicable.
+8. Test allowed and denied requests, including admin access.
+
+If the RPC needs a new permission, add it to `ldk-server-grpc/src/permissions.rs` and
+`ALL_PERMISSIONS`. Update the presets that need it and add it to `docs/api-guide.md`.
 
 ## Configuration
 

Cargo.lock

@@ -1,6 +1,6 @@
 [workspace]
 resolver = "2"
-members = ["ldk-server-cli", "ldk-server-client", "ldk-server-grpc", "ldk-server", "ldk-server-mcp"]
+members = ["ldk-server-cli", "ldk-server-client", "ldk-server-grpc", "ldk-server", "ldk-server-mcp", "ldk-server-macaroons"]
 exclude = ["e2e-tests"]
 
 [profile.release]

Cargo.toml

@@ -19,6 +19,7 @@ a Lightning node while exposing a robust, language-agnostic API via [Protocol Bu
 - `ldk-server-cli`: CLI client for the server API
 - `ldk-server-client`: Rust client library for authenticated TLS gRPC calls
 - `ldk-server-grpc`: generated protobuf and shared gRPC types
+- `ldk-server-macaroons`: shared token parsing, signing, derivation, and request binding
 - `ldk-server-mcp`: stdio MCP bridge exposing unary `ldk-server` RPCs as MCP tools
 
 ### Features

README.md

@@ -15,24 +15,107 @@ underlying LDK Node documentation.
 
 ## Authentication
 
-Every gRPC request must include an `x-auth` metadata header with an HMAC-SHA256 signature:
+Each request needs a hex-encoded v2 macaroon in the `macaroon` header:
 
+```text
+macaroon: <hex-encoded-request-macaroon>
 ```
-x-auth: HMAC <unix_timestamp>:<hmac_hex>
+
+Keep your original macaroon private. The client uses it to make a token for each request,
+as described below. All requests use TLS.
+
+You can add caveats to make a restricted copy of your macaroon without contacting the server.
+A caveat limits its permissions, allowed methods, or expiry time. Added caveats can only reduce
+access. See [Restrictions](#restrictions) for examples.
+
+### Request binding
+
+The Rust client, CLI, and MCP handle request binding automatically. They keep your macaroon
+private and send a copy tied to the request's method, body, and time.
+
+Custom clients must add one final caveat:
+
+```text
+request = <unix-seconds> <RpcMethod> <body-sha256>
 ```
 
-Where:
+Use Unix time in seconds, a method name such as `OnchainSend`, and the lowercase SHA-256 hash
+of the exact gRPC body, including its five-byte frame header. Rust clients can use
+`macaroon::bind_macaroon_to_request`.
+
+Client and server clocks must be within 60 seconds. The token cannot authorize a different
+request, but the same request can still be replayed while the token is valid.
+
+See [Request proof format](request-binding.md) for exact encoding rules and an example.
+
+### Restrictions
 
-- `unix_timestamp` is the current time in seconds since the Unix epoch
-- `hmac_hex` is the hex-encoded result of
-  `HMAC-SHA256(api_key_bytes, timestamp_be_bytes || grpc_request_body_bytes)`
-    - `api_key_bytes` is the API key string encoded as UTF-8 bytes
-    - `timestamp_be_bytes` is the timestamp as a big-endian 8-byte unsigned integer
-    - `grpc_request_body_bytes` is the raw gRPC request body sent over HTTP/2, including
-      the 5-byte gRPC message frame
+A caveat is a condition that limits what a macaroon can do. All caveats must pass:
 
-The server rejects requests where the timestamp differs from the server's clock by more than
-**60 seconds**.
+| Caveat | Meaning |
+|--------|---------|
+| `permissions = node:read,payments:read` | Allow only these permissions |
+| `method = GetNodeInfo` | Allow only this RPC method |
+| `time-before = 1800000000` | Expire at this Unix time in seconds |
+
+Added caveats can only reduce access. They cannot restore permissions or extend the expiry time.
+
+Derive a restricted copy without contacting the server:
+
+```bash
+ldk-server-cli derive-macaroon "$MACAROON" \
+  --caveat 'permissions = node:read' \
+  --caveat 'method = GetNodeInfo' \
+  --caveat "time-before = $EXPIRY_UNIX_SECONDS"
+```
+
+The command prints a hex token. Rust clients can use `macaroon::derive_macaroon`.
+Give the copy to the application and keep the original private.
+
+### Create and revoke tokens
+
+Use `CreateMacaroon` to give each client a token you can revoke separately. New tokens keep
+all the caller's restrictions. Revoking the caller's token does not revoke these new tokens.
+
+Copies made with `derive-macaroon` share the original token's ID. Revoking that ID blocks
+all those copies. The server cannot list copies made locally.
+
+Revocation and expiry block new requests. Existing event streams stay open until the client
+disconnects or the server stops. Reconnecting requires a valid token.
+
+See [Macaroon Management](#macaroon-management) for the RPCs and
+[Operations](operations.md#macaroons) for storage and recovery.
+
+### Macaroon Permissions
+
+Choose the permissions each client needs, or use `admin` by itself for full access.
+The CLI also has `readonly`, `invoice`, and `admin` presets.
+RPCs with no permission mapping return `UNIMPLEMENTED`, even for admin tokens.
+
+| Permission | Access |
+| ---------- | ------ |
+| `node:read` | Node information, balances, and pathfinding scores |
+| `onchain:receive` | Create on-chain receive addresses |
+| `onchain:send` | Send on-chain funds |
+| `invoices:create` | Create BOLT11/BOLT12 invoices and incoming refund requests |
+| `payments:read` | Read payments and forwarded payments |
+| `payments:claim` | Claim or fail held BOLT11 payments |
+| `payments:send` | Send BOLT11, BOLT12, spontaneous, unified, and refund payments |
+| `channels:read` | List channels |
+| `channels:manage` | Open, configure, or cooperatively close channels |
+| `channels:splice` | Splice funds in or out, including to an external address |
+| `channels:force_close` | Force-close channels |
+| `peers:read` | List peers |
+| `peers:manage` | Connect or disconnect peers |
+| `messages:sign` | Sign messages and create BOLT12 payer proofs |
+| `messages:verify` | Verify message signatures |
+| `graph:read` | Read network graph data |
+| `utilities:read` | Decode invoices and offers |
+| `events:read` | Subscribe to the event stream |
+| `macaroons:manage` | Create, list, and revoke macaroons within your permissions |
+
+MCP provides token management through `create_macaroon`, `list_macaroons`, `revoke_macaroon`,
+and `get_permissions`.
 
 ## TLS
 
@@ -67,9 +150,10 @@ Errors are returned as standard gRPC status codes:
 | gRPC Code                 | Meaning                                                          |
 |---------------------------|------------------------------------------------------------------|
 | `INVALID_ARGUMENT` (3)    | Malformed request or invalid parameters                          |
+| `PERMISSION_DENIED` (7)   | Missing permission or a caveat that does not pass                    |
 | `FAILED_PRECONDITION` (9) | Lightning operation error (e.g., insufficient balance, no route) |
 | `INTERNAL` (13)           | Server-side bug                                                  |
-| `UNAUTHENTICATED` (16)    | Missing or invalid `x-auth` header                               |
+| `UNAUTHENTICATED` (16)    | Missing, invalid, or revoked macaroon                               |
 
 The `grpc-message` trailer contains a human-readable error description.
 
@@ -232,6 +316,18 @@ Use events as notifications. After reconnecting, reconcile recoverable state wit
 `GetPaymentDetails`, `ListPayments`, `ListForwardedPayments`, and `ListChannels`. Some event fields
 cannot be recovered through these APIs.
 
+### Macaroon Management
+
+| RPC | Description |
+|-----|-------------|
+| `CreateMacaroon` | Create a macaroon and return its private hex token in `token` |
+| `ListMacaroons` | List IDs, names, permissions, and caveats, without secrets |
+| `RevokeMacaroon` | Revoke a macaroon by ID |
+| `GetPermissions` | Show the caller's ID, name, usable permissions, and caveats |
+
+The first three RPCs require `macaroons:manage` or `admin`. You can only create or revoke
+tokens whose permissions you have. The last unrestricted admin token cannot be revoked.
+
 ### Metrics
 
 Metrics are served as a plain HTTP GET endpoint (not gRPC):

docs/api-guide.md

@@ -207,7 +207,11 @@ Two resolution methods are supported via the `mode` field:
   tls.crt                # TLS certificate (PEM)
   tls.key                # TLS private key (PEM)
   <network>/                # e.g., bitcoin/, regtest/, signet/
-    api_key                # API key
+    macaroons/
+      admin.macaroon       # Admin token (0400)
+      roots/               # Private root keys (0700)
+        admin.toml         # Admin root key and permissions (0400)
+        <id>.toml          # Root key and permissions from CreateMacaroon (0400)
     ldk-server.log         # Log file
     ldk_node_data.sqlite   # LDK Node state (channels, wallet, payments)
     ldk_server_data.sqlite # Forwarded-payment history

docs/configuration.md

@@ -73,28 +73,32 @@ gRPC service listening on 127.0.0.1:3536
 NODE_URI: <node_id>@<address>
 ```
 
-Two files are auto-generated on first run:
+The server creates these files on first run:
 
-| File            | Location                          | Purpose                                  |
-|-----------------|-----------------------------------|------------------------------------------|
-| API key         | `<storage_dir>/<network>/api_key` | 32-byte random key (stored as raw bytes) |
-| TLS certificate | `<storage_dir>/tls.crt`           | Self-signed ECDSA P-256 certificate      |
+| File | Location | Purpose |
+|------|----------|---------|
+| Admin macaroon | `<storage_dir>/<network>/macaroons/admin.macaroon` | Full API access |
+| TLS certificate | `<storage_dir>/tls.crt` | Secure client connections |
 
 The default storage directory is `~/.ldk-server/` on Linux and
 `~/Library/Application Support/ldk-server/` on macOS.
 
-### Reading the API Key
+### Client Macaroons
 
-The API key file contains raw bytes. To get the hex string the CLI and client library expect:
+The CLI reads `admin.macaroon` automatically. This file contains a hex token.
+Keep it private, and never give clients files from `macaroons/roots/`.
+
+Create a restricted token for each application:
 
 ```bash
-xxd -p -c 64 ~/.ldk-server/bitcoin/api_key
+ldk-server-cli create-macaroon my-app --preset readonly
+ldk-server-cli create-macaroon invoice-app --preset invoice
 ```
 
 ## First Commands
 
 If the CLI and server share the same machine and use the default storage directory, the CLI
-auto-discovers the API key and TLS certificate, so no flags are needed:
+auto-discovers the macaroon and TLS certificate, so no flags are needed:
 
 ```bash
 # Check the node is running
@@ -113,7 +117,7 @@ details explicitly:
 ```bash
 ldk-server-cli \
   --base-url localhost:3536 \
-  --api-key <hex_api_key> \
+  --macaroon <hex_macaroon> \
   --tls-cert /path/to/tls.crt \
   get-node-info
 ```

docs/getting-started.md

@@ -60,7 +60,7 @@ the following config to `/etc/logrotate.d/ldk-server` (adjust the log path to ma
 
 - Network graph data (re-synced from gossip or RGS)
 - Fee rate cache (re-fetched from the chain backend)
-- The API key (can be regenerated, but clients will need the new one)
+- Macaroon credentials (can be replaced, but clients will need new tokens)
 - The TLS certificate (can be regenerated, but clients will need the new one)
 
 > **Warning:** Do not restore a backup onto two running nodes simultaneously. Running the
@@ -69,13 +69,39 @@ the following config to `/etc/logrotate.d/ldk-server` (adjust the log path to ma
 
 ## Security
 
-### API Key
+### Macaroons
 
-- Auto-generated as 32 random bytes on first startup
-- Stored at `<network_dir>/api_key` with `0400` permissions (read-only for owner)
-- The hex-encoded form of this key is used for HMAC authentication
-- Treat it as a secret: anyone with the API key and network access to the gRPC port can
-  control the node
+Keep `<network_dir>/macaroons/` private. It contains the default admin token in `admin.macaroon`
+and the server's root keys in `roots/`. Give clients tokens, never root keys.
+
+Use `create-macaroon` to give each client a token you can revoke separately. Use `derive-macaroon`
+to make a restricted copy. Give each client only the permissions it needs.
+See the [API guide](api-guide.md#authentication) for restrictions and request binding.
+
+To replace an admin token, create and save a new admin token, then revoke the old ID.
+Replace `admin.macaroon` with the new token or pass it with `--macaroon`.
+The API prevents revocation of the last unrestricted admin token.
+
+#### Recovery
+
+Back up `roots/` and `admin.macaroon` together. Old root files can restore revoked access.
+Deleting all roots invalidates every token; the server creates a new admin token on restart.
+
+At startup, the server repairs a missing or invalid `admin.macaroon` from `roots/admin.toml`
+and logs the change. It keeps valid tokens, even if restricted or expired. It warns if the
+file holds a request token; replace that file with a reusable token.
+
+If the original admin root is gone but other roots remain, the server warns instead of replacing
+it. Use another admin token to create a replacement and save it as `admin.macaroon`.
+
+Duplicate root names or IDs stop startup. Move conflicting files out of `roots/` and restart.
+Files ending in `.tmp` are ignored.
+
+Root-file caveat edits take effect after restart. `GetPermissions` shows them, and newly issued
+tokens inherit them. Tokens issued earlier have separate roots and do not change.
+
+The server logs successful token creation and revocation, including who made the change and
+which token it affects. Logs contain no tokens or root secrets.
 
 ### TLS
 
@@ -188,7 +214,7 @@ To allow clients to connect from other machines:
    (e.g., `0.0.0.0:3536`).
 3. **Distribute the TLS certificate:** Copy `<storage_dir>/tls.crt` to each client machine.
    Clients must pin this certificate since it is self-signed.
-4. **Share the API key:** Provide the hex-encoded API key to authorized clients.
+4. **Share the macaroon:** Provide the hex-encoded macaroon to authorized clients.
 
 If you regenerate the TLS certificate (by deleting `tls.crt` and `tls.key` and restarting),
 all clients will need the new certificate.