Throttlr

Adaptive Distributed Rate Limiting API Gateway

A high-performance reverse-proxy API Gateway with rate limiting at the network edge. Powered by Redis Lua scripts, two-tier in-memory leasing, and adaptive upstream concurrency control.

---

Key Features

• Dual-Algorithm Engine: Supports Token Bucket (with burst absorption and sub-millisecond continuous refill) and Sliding Window Counter (with weighted window approximation).
• Two-Tier Memory Lease (L1 + L2): Leases batches of tokens to local memory for hot keys, reducing Redis network calls by up to 90%.
• Cost-Aware and Post-Execution Settlement: Supports route-based costs and post-execution delta deductions with negative balance debt tracking.
• Adaptive Upstream Health: Monitors rolling p95 latency and 5xx error rates to dynamically scale bucket capacities under heavy load.
• Graceful Degradation: Classifies traffic into Green (allowed), Yellow (served from stale cache or 202 deferred), and Red (429 rejected).
• Shadow Mode: Evaluates and logs rate limiting violations with X-RateLimit-Shadow-Exceeded: true without dropping traffic.
• Circuit Breaker Fallback: Automatically switches to an in-memory token bucket if Redis is unreachable or exceeds 200ms latency.
• Guarded Control Plane: /api/control and the WebSocket config channel require X-Admin-Token when AEGIS_ADMIN_TOKEN is set, and fall back to loopback-only access when it is not.
• Real-Time Dashboard: Dark-mode visualizer with animated bucket levels, WebSocket telemetry, and an interactive traffic simulator.

---

Directory Structure

ratelimit/
├── src/
│   ├── config.js               # Route costs and cluster configuration
│   ├── redis/
│   │   ├── client.js           # Redis connection and circuit breaker
│   │   ├── luaManager.js       # Lua EVALSHA script manager
│   │   └── lua/
│   │       ├── token_bucket.lua    # Token bucket script
│   │       └── sliding_window.lua  # Sliding window counter script
│   ├── limiter/
│   │   ├── limiterService.js   # Limiter coordinator and Pub/Sub listener
│   │   ├── memoryLease.js      # L1 in-memory lease store
│   │   ├── fallbackLimiter.js  # In-memory fallback limiter
│   │   └── costCalculator.js   # Cost evaluation and delta calculator
│   ├── adaptive/
│   │   └── healthMonitor.js    # Upstream p95 latency and error observer
│   ├── gateway/
│   │   ├── server.js           # Reverse proxy HTTP and WebSocket server
│   │   └── middleware.js       # Rate limit middleware and RFC headers
│   ├── upstream/
│   │   └── mockUpstream.js     # Test upstream services
│   ├── metrics/
│   │   └── metricsCollector.js # Metrics aggregation
│   └── dashboard/              # Browser dashboard (port 8080/dashboard)
├── tests/
│   ├── verification.js         # Automated test suite (10 cases / 33 assertions)
│   └── benchmark.js            # Throughput benchmark
├── docker-compose.yml          # Redis service configuration
├── tech.md                     # Technical specifications and algorithms
├── test.md                     # Test execution, debug log, and audit
├── package.json
└── README.md

---

Quick Start

1. Start Redis

docker run -d --name aegis-redis -p 6379:6379 redis:alpine

Or using docker compose:

docker compose up -d

2. Install Dependencies

npm install

3. Start Gateway

npm start

To protect the control plane on a non-local deployment, set a shared secret. Without it, /api/control only accepts loopback callers:

AEGIS_ADMIN_TOKEN=your-secret npm start

4. Endpoints

• Dashboard: http://localhost:8080/dashboard
• Gateway Proxy: http://localhost:8080/api/*
• Mock Upstream: http://localhost:8081

---

Testing and Benchmarks

Run the integration test suite:

npm test

Run the throughput benchmark:

npm run benchmark

---

Standard Headers

Header	Description	Example
RateLimit-Limit	Maximum quota for current window	20
RateLimit-Remaining	Remaining quota	14
RateLimit-Reset	Seconds until quota is fully refilled	2
Retry-After	Back-off time sent with 429 and 202 responses	4
X-RateLimit-Tier	Traffic tier	green | yellow | red
X-RateLimit-Algorithm	Active algorithm	token_bucket | sliding_window
X-RateLimit-Adaptive-Multiplier	Dynamic upstream health factor	1.0
X-Graceful-Degradation	Triggered soft degradation method	Served-From-Stale-Cache
X-RateLimit-Shadow-Exceeded	Emitted when limit exceeded in Shadow Mode	true

---

Operational Notes

• Deploying behind a proxy: add every trusted hop to config.gateway.trustedProxies. The gateway reads X-Forwarded-For right-to-left and keys on the first untrusted hop, so a client-prepended value cannot mint a fresh quota.
• Degradable routes: a cached body is replayed to other clients during degradation, so only responses the upstream marks Cache-Control: public (with no Set-Cookie and no credential-sensitive Vary) are stored. Everything else soft-throttles with 202 instead.
• Identity: X-API-Key is taken at face value and is not authenticated. Put real authentication in front of the gateway before trusting per-key quotas.

---

Documentation Links

• Technical Specifications and Algorithms: tech.md
• Code Review, Issue Log, and Test Audit: test.md