Skip to content

feat: keep the credit hold alive while inference runs - #10

Merged
Marketen merged 1 commit into
mainfrom
fix/keep-credit-holds-alive
Oct 15, 2026
Merged

Marketen merged 1 commit into
mainfrom
fix/keep-credit-holds-alive

Conversation

@Marketen

Copy link
Copy Markdown
Contributor

Summary

Metering expires a credit reservation after its TTL even while the request is still running. When Metering reports a lease (lease_ms, from dappnode/dappnode-nexus#143), the Gateway now renews the hold for as long as the request runs.

  • Timing: renews at a third of the remaining lease. Transient failures are retried within the lease, at most once per second or once per tenth of the lease, whichever is more often.
  • Clock: the lease is measured on the Gateway's own clock from before the reserve request, so clock differences (for example in the enclave) can't make a hold look valid or expired when it isn't.
  • Lost hold: if Metering refuses a renewal because the hold is gone, or the lease runs out, inference stops with an internal error. It isn't reported as a user cancellation.
  • Metrics: renewals appear under the metering operation renew, without reservation IDs.

Compatibility

With a Metering that doesn't send lease_ms, nothing is renewed and behaviour is unchanged. Long requests need dappnode/dappnode-nexus#143 deployed first.

Tests

  • Unit:
    • renewal past the original expiry, and stopping cleanly;
    • transient errors don't stop a valid lease;
    • a refused renewal stops inference immediately;
    • retries stay spaced during an outage;
    • the lease is stored relative to the Gateway clock;
    • a lost hold stops inference with an internal error, and the keeper stops after completion.
  • End to end: a local Metering and Gateway with a 2-second lease. A 6-second response kept its hold renewed past the original expiry, and both requests settled as completed.

🤖 Generated with Claude Code

Metering expires a credit reservation after its TTL even while the request is
still running. Gateway now renews the hold for the life of the request when
Metering reports a lease (lease_ms):

- renewals at a third of the remaining lease, retried on transient errors no
  more often than once per second or a tenth of the lease;
- the lease is measured on the Gateway's own clock from before the reserve
  request, so clock skew (an enclave) cannot shorten or extend it;
- if Metering refuses a renewal (the hold is gone) or the lease runs out,
  inference stops with an internal error rather than a user cancellation.

Metering versions without lease_ms are unaffected: nothing is renewed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Marketen
Marketen merged commit 7df48cc into main Oct 15, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant