budgets 0.5.1: final-record challenge for expired auth tokens

§Budget Exhaustion 'auth token expired' + §Settlement: a genuine-but-
expired aa-auth+jwt (issuer signature + PoP verified, request fresh) now
gets 401 + plain requirement=auth-token (no reason) with a resource token
carrying the presented token's final {jti, consumed}. Without this
challenge the issuer never sees a final figure and accounts every
allocation as fully consumed forever.

verify(..., allow_expired_auth_token=True) is opt-in (default False) and
bounded by EXPIRED_AUTH_TOKEN_GRACE (2h); access decisions are unchanged
and the middleware never caches an expired result. 83 tests.

Author: abay94

CHANGELOG.md

@@ -1,5 +1,28 @@
 # Changelog
 
+## 0.5.1
+
+**Budgets — the final-record challenge for expired auth tokens** (draft
+§Budget Exhaustion "auth token expired" + §Settlement). The final/snapshot
+settlement rule needs a moment at which the resource *states* a token's final
+figure — and that moment is the challenge to an expired token. A verifier that
+refuses to look at an expired token can never build it, so the issuer never
+sees a final record and accounts every allocation as fully consumed forever.
+
+- `BudgetMiddleware` answers a genuine-but-expired `aa-auth+jwt` (issuer
+  signature and proof of possession verified, request signature fresh) with
+  `401` + plain `AAuth-Requirement: requirement=auth-token;resource-token=…`
+  (no `reason` — the budget didn't run out, the token did) and `code:
+  AUTH_TOKEN_EXPIRED`. The resource token carries the presented token's
+  `{jti, consumed}`; no `AAuth-Budget` header. Nothing is served or metered.
+- `HttpsigVerifier.verify(..., allow_expired_auth_token=True)` — **opt-in,
+  default False**: returns the token with `VerifiedSignature.expired=True`
+  for up to `EXPIRED_AUTH_TOKEN_GRACE` (2h, the meter's retention) after
+  `exp`. Access decisions through `verify()`/`require_signature` are
+  unchanged: an expired token is still `None` there, and the middleware never
+  caches an expired result for downstream dependencies.
+- `build_aauth_requirement(reason=None, …)` emits the reason-less challenge.
+
 ## 0.5.0
 
 **AAuth Budgets — per-token cap, one consumption record** (draft-hardt-aauth-budgets

README.md

@@ -144,6 +144,11 @@ production on [get4agent.com](https://get4agent.com).
 
 - `insufficient-budget` refusals carry **`required`** — the refused request's
   maximum cost — so the agent lowers its bound and retries instead of guessing.
+- An **expired** auth token gets the base protocol's plain challenge with a
+  resource token carrying that token's **final** consumption record — the
+  figure its issuer needs to settle the allocation (a record stated at or
+  after `exp` is final; one on a live token is a snapshot and releases
+  nothing).
 - **Streaming** responses run in the draft's cost-omitted mode: `reserved` in
   the header, commit when the stream ends (set `request.state.budget_cost`
   mid-stream if you learn the actual), and the agent recovers the exact cost

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "regent-httpsig"
-version = "0.5.0"
+version = "0.5.1"
 description = "Verify and sign AI agent HTTP traffic in Python — RFC 9421 HTTP Message Signatures: Web Bot Auth (what OpenAI ships) and AAuth."
 readme = "README.md"
 license = "Apache-2.0"

src/regent_httpsig/fastapi.py

@@ -191,6 +191,13 @@ async def dispatch(
             return await call_next(request)
 
         sig = await self._verified(request)
+        if sig is not None and sig.expired:
+            # §Budget Exhaustion, "auth token expired": the base protocol's plain
+            # 401 — and the resource token we attach carries the presented
+            # token's FINAL consumption record (stated at/after its exp), which
+            # is what lets its issuer settle the allocation exactly. Without
+            # this challenge an issuer never sees a final figure at all.
+            return await self._expired_challenge(sig)
         envelope: BudgetClaim | None = None
         if sig is not None:
             try:
@@ -307,11 +314,42 @@ async def _verified(self, request: Request) -> VerifiedSignature | None:
         result = None
         if "signature" in request.headers:
             result = await self._verifier.verify(
-                request.method, _public_url(request), dict(request.headers)
+                request.method, _public_url(request), dict(request.headers),
+                allow_expired_auth_token=True,
             )
-        request.state.regent_httpsig_result = result
+        # An expired token is never handed to the app: the middleware answers
+        # it with the final-record challenge and nothing downstream runs. The
+        # per-request cache still must not carry it — a later dependency
+        # reading the cache would otherwise see a token that is not valid.
+        request.state.regent_httpsig_result = (
+            None if result is not None and result.expired else result)
         return result
 
+    async def _expired_challenge(self, sig: VerifiedSignature) -> Response:
+        jti = str(sig.claims.get("jti") or "")
+        key: MeterKey = (
+            str(sig.claims.get("iss", "")),
+            str(sig.claims.get("sub", "")),
+            str(sig.claims.get("aud", "")),
+        )
+        token: str | None = None
+        if self._resource_token is not None and jti:
+            try:
+                record = await self._meter.consumed_record(key, jti)
+                token = await _maybe_await(self._resource_token(key, record))
+            except Exception:  # noqa: BLE001 — the challenge must not fail on the extras
+                logger.warning("resource_token_provider failed", exc_info=True)
+        return JSONResponse(
+            status_code=401,
+            content={
+                "code": "AUTH_TOKEN_EXPIRED",
+                "message": "The auth token has expired. Take the resource token in "
+                           "AAuth-Requirement to your PS for a fresh one.",
+            },
+            headers={"AAuth-Requirement": build_aauth_requirement(
+                reason=None, resource_token=token)},
+        )
+
     async def _refusal_with_token(
         self, *, reason: str, envelope: BudgetClaim,
         remaining: int, key: MeterKey, jti: str,

src/regent_httpsig/sfv.py

@@ -176,13 +176,18 @@ def build_aauth_budget_header(
     return ", ".join(members)
 
 
-def build_aauth_requirement(*, reason: str, resource_token: str | None = None) -> str:
-    """Serialize ``AAuth-Requirement`` for a budget refusal:
+def build_aauth_requirement(*, reason: str | None,
+                            resource_token: str | None = None) -> str:
+    """Serialize ``AAuth-Requirement`` for a budget challenge:
     ``requirement=auth-token;resource-token="eyJ…";reason=insufficient-budget``.
-    ``reason`` is an sf-token (``insufficient-budget`` | ``budget-exhausted``);
-    the resource token (when the resource issues one) rides as an sf-string."""
+    ``reason`` is an sf-token (``insufficient-budget`` | ``budget-exhausted``)
+    for an exhaustion refusal, or ``None`` for the base protocol's plain
+    challenge (an EXPIRED auth token — the budget did not run out, the token
+    did); the resource token (when the resource issues one) rides as an
+    sf-string."""
     out = "requirement=auth-token"
     if resource_token:
         out += f";resource-token={_sf_string(resource_token)}"
-    out += f";reason={reason}"
+    if reason:
+        out += f";reason={reason}"
     return out

src/regent_httpsig/verify.py

@@ -61,6 +61,10 @@
 AAUTH_AUTH_TYP = "aa-auth+jwt"  # PS-issued auth tokens — the budget carrier
 # -11: person and auth tokens live at most one hour — enforced with tolerance.
 PERSON_TOKEN_MAX_LIFETIME = 3600 + 90
+# How long after its exp an auth token still earns the budgets "final record"
+# challenge (opt-in, BudgetMiddleware only): the meter retains consumption for
+# about this long after last activity; beyond it there is no figure to carry.
+EXPIRED_AUTH_TOKEN_GRACE = 7200
 
 
 def _register_fully_specified_algs() -> None:
@@ -86,6 +90,11 @@ class VerifiedSignature:
     sub: str | None = None  # AAuth agent id (token `sub`)
     label: str = ""
     claims: dict[str, Any] = field(default_factory=dict)  # AAuth token claims (redacted)
+    # Set only when ``verify(..., allow_expired_auth_token=True)`` accepted an
+    # auth token PAST its exp: genuine (issuer signature + proof of possession
+    # verified) but NOT valid for access. BudgetMiddleware uses it to answer
+    # with the final-record challenge; nothing else should ever see it True.
+    expired: bool = False
 
     def context(self) -> dict[str, Any]:
         """A flat dict suitable for logging / policy engines / audit trails."""
@@ -164,18 +173,28 @@ def __init__(
         self._cache: dict[str, tuple[float, dict[str, Any] | None]] = {}
 
     async def verify(
-        self, method: str, url: str, headers: Mapping[str, str]
+        self, method: str, url: str, headers: Mapping[str, str],
+        *, allow_expired_auth_token: bool = False,
     ) -> VerifiedSignature | None:
         """Verify the request's agent signature. Returns ``None`` when there is
         no ``Signature`` header, the signature is invalid, or the signer's keys
-        cannot be (safely) fetched — never raises on untrusted input."""
+        cannot be (safely) fetched — never raises on untrusted input.
+
+        ``allow_expired_auth_token`` (default False — leave it so for access
+        decisions) lets a genuine ``aa-auth+jwt`` past its ``exp`` come back
+        with ``expired=True`` instead of ``None``, for up to
+        :data:`EXPIRED_AUTH_TOKEN_GRACE` seconds. Budgets need this: the
+        resource owes the holder of an expired token a challenge carrying that
+        token's final consumption record (draft §Budget Exhaustion), and it
+        cannot build one for a token it refused to look at."""
         hdrs = {str(k): str(v) for k, v in headers.items()}
         if not any(k.lower() == "signature" for k in hdrs):
             return None
         result: VerifiedSignature | None = None
         try:
             if any(k.lower() == "signature-key" for k in hdrs):
-                result = await self._verify_aauth(method, url, hdrs)
+                result = await self._verify_aauth(
+                    method, url, hdrs, allow_expired_auth_token=allow_expired_auth_token)
             if result is None:
                 result = await self._verify_web_bot_auth(method, url, hdrs)
         except Exception as exc:  # noqa: BLE001 — belt and braces
@@ -288,7 +307,8 @@ async def _verify_web_bot_auth(
     # ── AAuth (identity-based mode) ──────────────────────────────────────────
 
     async def _verify_aauth(
-        self, method: str, url: str, headers: dict[str, str]
+        self, method: str, url: str, headers: dict[str, str],
+        *, allow_expired_auth_token: bool = False,
     ) -> VerifiedSignature | None:
         try:
             import jwt as pyjwt  # the [aauth] extra
@@ -387,6 +407,7 @@ async def _verify_aauth(
                         continue
         if issuer_key is None:
             return None
+        tolerate_exp = allow_expired_auth_token and typ == AAUTH_AUTH_TYP
         try:
             claims = pyjwt.decode(
                 token,
@@ -396,11 +417,20 @@ async def _verify_aauth(
                 options={
                     "require": ["iss", "sub", "exp", "iat"],
                     "verify_aud": audience is not None,
+                    "verify_exp": not tolerate_exp,
                 },
             )
         except Exception as exc:  # noqa: BLE001
             logger.info("aauth token invalid iss=%s: %s", iss, str(exc)[:200])
             return None
+        expired = False
+        if tolerate_exp:
+            overdue = int(time.time()) - int(claims.get("exp", 0))
+            expired = overdue >= 0
+            if overdue > EXPIRED_AUTH_TOKEN_GRACE:
+                logger.info("aauth auth token expired %ss ago — past grace iss=%s",
+                            overdue, iss)
+                return None
 
         # -11: person and auth tokens live at most one hour.
         if typ in (AAUTH_PERSON_TYP, AAUTH_AUTH_TYP):
@@ -456,4 +486,5 @@ async def _verify_aauth(
                           "budget")  # budgets: the envelope rides in the token
                 if k in claims
             },
+            expired=expired,
         )

tests/test_budget.py

@@ -266,11 +266,13 @@ def _ps_pair() -> tuple[Ed25519PrivateKey, dict[str, Any]]:
 
 
 def _auth_token(ps_priv: Ed25519PrivateKey, agent: EgressSigner, *,
-                amount: int, jti: str = "at-1") -> str:
+                amount: int, jti: str = "at-1", age: int = 0) -> str:
+    """``age`` > 600 mints a token that has ALREADY expired (iat/exp shifted
+    into the past) — genuine, possession-bound, not valid for access."""
     from regent_httpsig.verify import _register_fully_specified_algs
 
     _register_fully_specified_algs()  # PyJWT knows "Ed25519" only after this
-    now = int(time.time())
+    now = int(time.time()) - age
     return pyjwt.encode(
         {
             "iss": PS_ISS, "sub": "owner-1", "aud": RESOURCE, "jti": jti,
@@ -499,3 +501,70 @@ async def test_streaming_cost_omitted_reserved_math(
     next_remaining = int(str(d2["remaining"].value)) + 300  # add back r2's own cost
     recovered = 1000 + 300 - 700 - (1000 - next_remaining)  # draft's subtraction…
     assert 700 + 300 - next_remaining == 120     # …prev + reserved − next = cost
+
+
+async def test_expired_token_gets_final_record_challenge(
+    monkeypatch: pytest.MonkeyPatch,
+) -> None:
+    """§Budget Exhaustion, "auth token expired": plain requirement=auth-token
+    (no reason) + a resource token carrying the token's FINAL record — the
+    figure the issuer needs to settle the allocation (§Settlement: a record
+    stated at/after exp is final)."""
+    ps_priv, ps_jwk = _ps_pair()
+    agent = EgressSigner(seed=generate_seed(), signature_agent=PS_ISS)
+    live = _auth_token(ps_priv, agent, amount=1000, jti="at-9")
+    provider_calls: list[Any] = []
+
+    def provider(key: Any, record: Any) -> str:
+        provider_calls.append((key, record))
+        return "resource.token.final"
+
+    meter = InMemoryMeter()
+    app = _app(_verifier(ps_jwk, monkeypatch), meter=meter,
+               resource_token_provider=provider)
+    # Spend 300 on the live token, so there is a figure to carry.
+    r = await _post(app, "/v1/search", _signed_headers(agent, live, "/v1/search"))
+    assert r.status_code == 200
+
+    # The same jti, now past its exp (same issuer, same key, same claims).
+    stale = _auth_token(ps_priv, agent, amount=1000, jti="at-9", age=700)
+    r = await _post(app, "/v1/search", _signed_headers(agent, stale, "/v1/search"))
+    assert r.status_code == 401
+    assert r.json()["code"] == "AUTH_TOKEN_EXPIRED"
+    assert (r.headers["AAuth-Requirement"]
+            == 'requirement=auth-token;resource-token="resource.token.final"')
+    assert "AAuth-Budget" not in r.headers  # the budget didn't run out; the token did
+    assert provider_calls[-1] == ((PS_ISS, "owner-1", RESOURCE),
+                                  {"jti": "at-9", "consumed": 300})
+    # Nothing was served and nothing more was metered.
+    assert await meter.consumed_record(
+        (PS_ISS, "owner-1", RESOURCE), "at-9") == {"jti": "at-9", "consumed": 300}
+
+
+async def test_expired_token_past_grace_is_no_envelope(
+    monkeypatch: pytest.MonkeyPatch,
+) -> None:
+    """Beyond the grace window there is no figure left to carry: the token is
+    just an unknown one and the request falls through as unsigned."""
+    ps_priv, ps_jwk = _ps_pair()
+    agent = EgressSigner(seed=generate_seed(), signature_agent=PS_ISS)
+    ancient = _auth_token(ps_priv, agent, amount=1000, jti="at-old", age=3 * 3600)
+    app = _app(_verifier(ps_jwk, monkeypatch), require=True)
+    r = await _post(app, "/v1/search", _signed_headers(agent, ancient, "/v1/search"))
+    assert r.status_code == 401 and r.json()["code"] == "AUTH_TOKEN_REQUIRED"
+
+
+async def test_plain_verify_still_rejects_expired_tokens(
+    monkeypatch: pytest.MonkeyPatch,
+) -> None:
+    """The tolerance is opt-in for the budgets middleware only: an access
+    decision through ``verify()`` never sees an expired token as verified."""
+    ps_priv, ps_jwk = _ps_pair()
+    agent = EgressSigner(seed=generate_seed(), signature_agent=PS_ISS)
+    stale = _auth_token(ps_priv, agent, amount=1000, jti="at-9", age=700)
+    verifier = _verifier(ps_jwk, monkeypatch)
+    headers = _signed_headers(agent, stale, "/v1/search")
+    assert await verifier.verify("POST", f"{RESOURCE}/v1/search", headers) is None
+    sig = await verifier.verify("POST", f"{RESOURCE}/v1/search", headers,
+                                allow_expired_auth_token=True)
+    assert sig is not None and sig.expired