Skip to content

fix: stop GitHub rate-limit lockouts and unblock Vercel deploys - #3528

Merged
Priyanshu-byte-coder merged 2 commits into
mainfrom
fix/github-rate-limits
Oct 15, 2026
Merged

Priyanshu-byte-coder merged 2 commits into
mainfrom
fix/github-rate-limits

Conversation

@Priyanshu-byte-coder

Copy link
Copy Markdown
Owner

Why

  • Dashboard loads issued ~95 Search API calls against GitHub's 30/minute search quota, so widgets regularly returned 429.
  • Paging a year of commit search tripped GitHub's secondary (anti-scraping) limit. The request layer then froze every GitHub API for that user. A 429 primary limit was also misread as secondary.
  • Every production deploy since Sept 12 failed: vercel.json had two hourly crons, and Vercel Hobby only allows daily ones.

What

  • Commit timeline via GraphQL:
    • contributionsCollection discovers the repos.
    • Each repo's default-branch history(author:) supplies the commits.
    • No Search quota is used. Totals now match GitHub's contribution graph (search double-counted mirrored commits).
  • One shared, memoized timeline feeds streak, consistency, contributions, productive hours, commit times, hourly, weekly summary, repos, languages, repo explorer, repo health and CI.
  • Repo health: one GraphQL query per repo replaces four searches per repo (24 per load before).
  • Request layer:
    • Bounded per-resource concurrency.
    • Per-resource freshness.
    • A 15s revalidation floor, so a forced refresh revalidates each shared query once.
    • A 429 with remaining=0 is now scoped to that resource only.
  • Crons:
    • goals-sync and weekly webhooks now run from .github/workflows/hourly-jobs.yml, which uses the APP_URL and CRON_SECRET repo secrets.
    • Vercel keeps only daily/weekly crons.

Measured

Forced refresh of all 24 dashboard endpoints, local, real account:

before after
Search API calls ~95 15
429 responses ~11 of 24 0

Tests

  • 2483 pass. New tests cover the GraphQL timeline (pagination, missing repos, windowing, no search use) and 429 primary-limit scoping.
  • test/commit-timeline-shim.ts lets existing route tests keep describing commits as search results.

Dashboard loads issued ~95 Search API calls against a 30/minute quota, and
paging a year of commit search tripped GitHub's secondary (anti-scraping)
limit, which then froze every API for the user.

- Build the commit timeline from GraphQL (contributionsCollection repo
  discovery + per-repo default-branch history filtered by author id).
  No Search API quota is spent and totals now match GitHub's own
  contribution graph instead of double-counting mirrored commits.
- Derive every commit window (streak, consistency, contributions,
  productive hours, commit times, hourly, weekly summary, repos,
  languages, repo explorer, repo health, CI) from that one shared,
  memoized timeline.
- Repo health: one GraphQL query per repo replaces four searches.
- Request layer: bounded per-resource concurrency instead of strict
  serialization, per-resource freshness, a short revalidation floor so a
  forced refresh revalidates shared queries once, and a 429 with
  remaining=0 is treated as a primary limit for that resource only.
- Move the two hourly crons to a GitHub Actions schedule; Vercel Hobby
  rejects sub-daily crons, which has failed every deploy since Sept 12.

Forced refresh of all 24 dashboard endpoints: ~95 -> 15 searches, 0 429s.
@github-actions github-actions Bot added type:bug GSSoC type bonus: bug fix type:devops GSSoC type bonus: devops (+15 pts) type:testing GSSoC type bonus: tests (+10 pts) labels Oct 15, 2026
@github-actions github-actions Bot added type:feature GSSoC type bonus: new feature type:performance GSSoC type bonus: performance (+15 pts) labels Oct 15, 2026
Comment thread src/app/api/metrics/contributions/route.ts Fixed
Comment thread src/app/api/metrics/contributions/route.ts Fixed
Comment thread src/app/api/metrics/contributions/route.ts Fixed
Comment thread src/app/api/metrics/repos/route.ts Fixed
…eline

Repo and org filters on contributions and top repos now slice the shared
commit timeline in memory instead of issuing their own commit searches.
This removes the last commit-search fan-out and the user-controlled
branch between two fetch paths that CodeQL flagged.
@Priyanshu-byte-coder
Priyanshu-byte-coder merged commit 6fd3f68 into main Oct 15, 2026
17 checks passed
@Priyanshu-byte-coder
Priyanshu-byte-coder deleted the fix/github-rate-limits branch October 15, 2026 14:22
@github-actions

Copy link
Copy Markdown

🎉 Merged! Thanks for contributing to DevTrack.

If the project has been useful to you, a ⭐ star on the repo is the easiest way to support it — it helps DevTrack get discovered by more developers.

Keep an eye on open issues for your next contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type:bug GSSoC type bonus: bug fix type:devops GSSoC type bonus: devops (+15 pts) type:feature GSSoC type bonus: new feature type:performance GSSoC type bonus: performance (+15 pts) type:testing GSSoC type bonus: tests (+10 pts)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants