Protect /api/execute with cache, rate limit, and backend timeout - #19
Open
alexandramartinez wants to merge 2 commits into
Open
Protect /api/execute with cache, rate limit, and backend timeout#19alexandramartinez wants to merge 2 commits into
alexandramartinez wants to merge 2 commits into
Conversation
The DataWeave compiler backend is a JVM that compiles + runs each script,
so /api/execute is the hot path that overwhelms a small/free instance —
especially since Submit fans out one call per test case. Add three
dependency-free, in-memory guards that cut load before it reaches the backend:
- Result cache: identical {script, inputs} runs are pure, so memoize them
(5 min TTL, 1000-entry LRU). High hit rate on a practice site where the
same reference solutions get resubmitted constantly.
- Per-identity rate limit: 30 runs/min keyed on Clerk userId (falls back to
IP for anonymous callers), returns 429 + Retry-After.
- Backend fetch timeout: abort after 15s so hung compiles can't pile up and
wedge the whole instance.
State lives on globalThis (same pattern as lib/db.ts) so it survives warm
invocations. Only successful runs are cached — never a backend outage.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
@alexandramartinez is attempting to deploy a commit to the bighnesh's projects Team on Vercel. A member of the Team first needs to authorize it. |
Submit used to fire 1 + N sequential fetches from the browser (custom-input
run + one per test case), each a full round-trip that also burned a
rate-limit token and opened a fresh backend connection. On a small instance
that fan-out is a big part of what tips the DataWeave compiler over.
- Extract the single-run core (validate → parse → cache → timeout backend
call → normalize) into lib/dataweave.ts so both routes share identical
behaviour and the SAME result cache.
- Add POST /api/execute/batch: { code, inputs[] } → { results[] }. Costs one
rate-limit token per Submit (a batch is one user action, not N), caps batch
size at 25, and runs inputs server-side through the shared cache. Cache hits
(repeat solutions, identical test cases) return instantly.
- Slim /api/execute down to the shared core + its per-run rate limit.
- Rewire Workspace handleSubmit to make one batch call and map results back to
test cases, preserving the existing pass/fail diff output.
Net: one Submit = one browser request + one rate-limit token, and repeat/
identical runs skip the backend entirely.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
DWCode went down under load. The bottleneck isn't the Next.js app or the data size — it's the DataWeave compiler backend (
https://dwlbackend.onrender.com), a JVM that compiles + runs each script per request./api/executeis the hot path that overwhelms a small/free instance, and it's amplified by Submit fanning out one backend call per test case (Workspace.tsx).This PR adds three dependency-free, in-memory guards that cut load before it reaches the backend — zero new infra required.
What
New module lib/executeGuards.ts, wired into app/api/execute/route.ts. Guard state lives on
globalThisso it survives warm invocations — the same pattern already used in lib/db.ts.1. Result cache (biggest win)
DataWeave scripts are pure, so an identical
{script, inputs}pair always yields the same output. On a practice site the same reference solutions get resubmitted constantly — those are served with no backend hit.2. Per-identity rate limit
30 runs/min keyed on Clerk
userId, falling back to client IP for anonymous callers (the route is public). Over the limit →429+Retry-After. Stops one client, a stuck retry loop, or a bot from saturating the JVM.3. Backend fetch timeout
The original
fetchhad no timeout — a hung compile would hold a connection indefinitely, and a pile-up of those turns a slow backend into a dead one. AddedAbortSignal.timeout(15s)with a distinct "compiler overloaded, retry" message.Verification
tsc --noEmitpasseslib/executeGuards.tslints cleananywarnings on the route are unrelated to this change (confirmed viagit stash)Caveats
Possible follow-up
Batch the per-test-case Submit loop into a single backend request (cuts submit load by the average test-case count). The rate limiter in
lib/executeGuards.tsis written to be reusable for that.🤖 Generated with Claude Code