Skip to content

tech.md

Latest commit

 

History

History
167 lines (135 loc) · 8.07 KB

tech.md

File metadata and controls

167 lines (135 loc) · 8.07 KB
Raw

AegisRate Gateway - Technical Specification (TECH.md)

This document describes the technical architecture, algorithms, data structures, and failure handling mechanisms implemented in AegisRate Gateway.

1. Technology Stack

Component Technology Version Purpose
Runtime Node.js v24.x LTS Non-blocking event loop, native ESM support, and HTTP proxy forwarding.
Primary Storage Redis v7.x (Alpine) In-memory key-value store, atomic script execution, and Pub/Sub broadcasting.
Scripting Engine Redis Lua Engine Lua 5.1 / LuaJIT Executes atomic rate-limiting checks directly inside Redis via EVALSHA.
Reverse Proxy Express v4.21.x HTTP routing, proxy pipelining, and RFC 6585 header injection.
Telemetry WebSockets (ws) v8.18.x Real-time bi-directional metrics feed for the dashboard.
Local Cache In-Memory Map Built-in L1 token lease storage to serve high-frequency keys without network round-trips.
Containerization Docker & Compose v29.x / 3.8 Redis deployment with memory constraints (maxmemory 256mb, volatile-lru).
Dashboard UI Vanilla HTML5 / CSS3 / JS Web Standards Dark-mode interface for monitoring bucket levels and upstream health.

2. Algorithms and Mathematical Models

2.1. Continuous-Refill Token Bucket

Tokens are refilled lazily on each request based on elapsed time, avoiding background refill jobs.

Parameters:

  • $C$: Bucket capacity
  • $R$: Refill rate (tokens per second)
  • $M$: Adaptive health multiplier ($0.2 \le M \le 1.0$)
  • $t_{\text{now}}$: Current timestamp (milliseconds)
  • $t_{\text{last}}$: Last evaluation timestamp (milliseconds)
  • $T_{\text{stored}}$: Token balance at $t_{\text{last}}$
  • $K$: Requested cost

Calculations:

  • Effective Capacity: $$C_{\text{eff}} = \max(1, \lfloor C \times M \rfloor)$$
  • Effective Refill Rate: $$R_{\text{eff}} = \max(0.1, R \times M)$$
  • Elapsed Time: $$\Delta t = \max\left(0, \frac{t_{\text{now}} - t_{\text{last}}}{1000}\right)$$
  • Current Tokens: $$T_{\text{current}} = \min(C_{\text{eff}}, T_{\text{stored}} + \Delta t \times R_{\text{eff}})$$

Evaluation Rules:

  • If $T_{\text{current}} \ge K$: $$T_{\text{remaining}} = T_{\text{current}} - K \implies \text{ALLOWED (Green Tier)}$$
  • If $T_{\text{current}} < K$:
    • If $\text{allow_negative} = 1$ (post-settlement): $$T_{\text{remaining}} = T_{\text{current}} - K \implies \text{ALLOWED (Debt Recorded)}$$
    • Else if $T_{\text{current}} + \text{GraceBuffer} \ge K$: $$\text{SOFT LIMIT (Yellow Tier: Cache or 202 Deferred)}$$
    • Else: $$\text{BLOCKED (Red Tier: 429 Too Many Requests)}$$
  • Reset Duration (seconds): $$\text{Reset} = \left\lceil \frac{\max(0, C_{\text{eff}} - T_{\text{remaining}})}{R_{\text{eff}}} \right\rceil$$

2.2. Sliding Window Counter

Approximates request counts by weighting the current and previous fixed windows.

Parameters:

  • $W$: Window duration in seconds
  • $L_{\text{eff}} = \max(1, \lfloor L \times M \rfloor)$: Effective limit
  • $w_{\text{curr}} = \lfloor \frac{t_{\text{now}}}{W \times 1000} \rfloor$: Current window index
  • $w_{\text{prev}} = w_{\text{curr}} - 1$: Previous window index
  • $\rho = \frac{t_{\text{now}} \pmod{W \times 1000}}{W \times 1000}$: Position within the current window ($0.0 \le \rho < 1.0$)
  • $G = \lfloor L_{\text{quota}} \times 0.2 \rfloor$: Grace buffer, scaled against the quota the active algorithm enforces (slidingLimit here, bucket capacity for Token Bucket) so the Yellow band is never sized in the wrong unit.

Calculations:

  • Estimated Count: $$N = \lfloor C(w_{\text{prev}}) \times (1 - \rho) + C(w_{\text{curr}}) \rfloor$$
  • If $N + K \le L_{\text{eff}}$: $$C(w_{\text{curr}}) \leftarrow C(w_{\text{curr}}) + K \implies \text{ALLOWED}$$
  • Else: $$\text{BLOCKED (429)}$$

3. Architecture and Request Flow

Incoming Request
       │
       ▼
[Client Identification] ─── (API Key or Trusted Socket IP)
       │
       ▼
[Health Check] ───────────── (Applies Adaptive Multiplier M)
       │
       ▼
[L1 Memory Lease] ────────── (Hit: Deduct in RAM < 0.05ms)
       │ (Miss)
       ▼
[L2 Redis Lua Script] ────── (Atomic EVALSHA execution)
       │
       ├─► Green Tier  ───► Proxy to Upstream ──► Post-Execution Settlement
       ├─► Yellow Tier ───► Serve Cache / Return 202 Accepted
       └─► Red Tier    ───► Shadow Pass (if enabled) OR Return 429

4. Key Subsystems

4.1. L1 In-Memory Lease Store

To minimize Redis network round-trips:

  • Hot clients receive an in-memory lease (e.g., 10 tokens for 5,000ms).
  • Subsequent requests decrement local memory in $O(1)$ time without network calls.
  • When the lease is exhausted or expires, the gateway queries Redis for a new batch.
  • Refill coalescing: at most one batch request per key is in flight at a time. Concurrent requests for the same key join the pending refill (up to 3 rounds) instead of each debiting their own batch, and a grant that arrives while a lease is still live is added to it rather than replacing it. Without both rules a burst debits Redis several times over and discards the surplus batches.

4.2. Post-Execution Cost Settlement

For endpoints with variable resource consumption:

  • An initial base cost is deducted before forwarding the request.
  • After the upstream responds, the gateway inspects the X-Compute-Cost header or measures latency.
  • Any delta cost is deducted asynchronously via Redis Lua with allow_negative = 1. If the balance drops below zero, future requests are blocked until the deficit is cleared by the refill rate.

4.3. Adaptive Upstream Health Monitor

The gateway monitors upstream performance using a rolling sample of the last 30 requests:

  • Calculates $p95$ latency and $5xx$ error rate.
  • If $p95 &gt; 800\text{ms}$ or error rate $\ge 15%$, multiplier drops down to $0.2$.
  • When latency stabilizes below $300\text{ms}$ and errors clear, multiplier recovers by $+0.05$ per interval up to $1.0$.

4.4. Control Plane Authorization

POST /api/control and the SET_CONFIG WebSocket message can switch algorithms, enable Shadow Mode, and rewrite route quotas for the entire cluster over Pub/Sub. Both are therefore gated:

  • When AEGIS_ADMIN_TOKEN is set, a matching X-Admin-Token header is required.
  • When it is unset, only loopback callers are accepted (keeps the local dashboard working).
  • Route overrides -- whether from HTTP or from a Pub/Sub peer -- are sanitized: unknown route keys are dropped and numeric fields must be finite and positive.

4.5. Circuit Breaker Fallback

If Redis pings fail or latency exceeds $200\text{ms}$ across consecutive samples:

  • The gateway switches to an in-memory fallback token bucket (fallbackLimiter.js).
  • Requests continue to be processed locally with zero downtime.
  • When Redis recovers, the gateway automatically switches back to distributed mode.

5. Redis Data Schema

Key Pattern Redis Type Fields TTL Policy
ratelimit:tb:{route}:{id} Hash tokens: Float string
last_updated: Timestamp		 120s rolling TTL
ratelimit:sw:{route}:{id} Hash {window_idx}: Integer count $2 \times \text{WindowSize}$
aegis:config:updates Pub/Sub Dynamic config payload Transient broadcast

6. HTTP Headers

Header Description Example
RateLimit-Limit Maximum quota in the active window 20
RateLimit-Remaining Remaining tokens 14
RateLimit-Reset Seconds until quota is fully refilled 2
Retry-After Recommended back-off time in seconds 4
X-RateLimit-Tier Traffic tier classification green | yellow | red
X-RateLimit-Algorithm Active algorithm token_bucket | sliding_window
X-RateLimit-Adaptive-Multiplier Current upstream health scale 1.0
X-Graceful-Degradation Degradation strategy applied Served-From-Stale-Cache
X-RateLimit-Shadow-Exceeded Emitted when limit exceeded in Shadow Mode true