Skip to content

Latest commit

 

History

2 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 

Repository files navigation

AI Leaderboard Tracker

One command for a current, side-by-side read on which AI models are actually winning — across LLMs, coding agents, image, video, and speech — plus what changed since you last looked.

This is an agent skill (a SKILL.md + a small Python helper) for AI coding assistants like Claude Code and opencode. It reads the public Artificial Analysis leaderboards and turns them into a short, dated briefing.


Why this exists

The frontier moves every week. The best text-to-video model today isn't the one from six weeks ago. The top open-weight LLM changes constantly. Image, speech, and coding-agent rankings churn just as fast.

Staying current usually means opening a dozen leaderboard pages and trying to remember what they said last time. Almost nobody does that consistently — so most of us are quietly working off a mental model that's a month or two stale, and making tooling decisions on top of it.

This skill does the checking for you and, crucially, remembers your last run so it can tell you what actually moved. The goal is simple: stay up to date in five minutes instead of an afternoon.

What you get

  • Ten categories in one pass — LLM intelligence, coding agents, text→image, image editing, text→video, image→video, video editing, text→speech, speech→text, speech→speech.
  • Just the signal — the top ~12–15 per board with the metric that matters (Intelligence Index, Arena Elo, word error rate…), not the raw 250-row table.
  • Change tracking — every run is snapshotted and diffed against the previous one, so you see movement over time: 👑 new #1s, ▲▼ rank moves, 🆕 new entrants, ⬇️ drop-offs.
  • A report you can keep — a timestamped Markdown file plus a quick chat summary.

How it works

  1. Your assistant loads the skill and fetches each leaderboard page.
  2. Pages are retrieved through the Bright Data MCP — these are JavaScript-heavy pages, and a reliable fetch layer is what makes the run repeatable instead of flaky.
  3. A bundled, dependency-free Python helper (scripts/aa_report.py) stores each run as a JSON snapshot under ~/.cache/, diffs it against the last one, and renders the Markdown report.
  4. You get the summary in chat and the full report on disk.

No API keys live in the code. No database. Snapshots are just local JSON files.

Requirements

  • Python 3.8+ (standard library only)
  • An assistant that supports skills — Claude Code or opencode
  • The Bright Data MCP server configured (see setup)

Install

Skills load from a folder whose name matches the skill, so clone into a folder named artificialanalysis-leaderboards:

# Claude Code
git clone https://github.com/yaronbeen/ai-leaderboard-tracker-skill-brightdata \
  ~/.claude/skills/artificialanalysis-leaderboards

# opencode
git clone https://github.com/yaronbeen/ai-leaderboard-tracker-skill-brightdata \
  ~/.config/opencode/skills/artificialanalysis-leaderboards

Restart your assistant so it picks up the new skill.

One-time setup: data access

The skill fetches pages through the Bright Data MCP server, which handles the awkward parts of retrieving modern, script-rendered pages so your runs stay consistent. It has a free tier of 5,000 requests/month — and a full ten-board run costs only ~10 requests, so ordinary use sits comfortably inside the free tier.

  1. Create a Bright Data account and copy your API token from Settings → API keys (https://brightdata.com/cp/setting/users).

  2. Add the MCP server to your assistant, naming it brightdata so the tool names line up with the skill:

    Claude Code

    claude mcp add --transport sse brightdata \
      "https://mcp.brightdata.com/sse?token=YOUR_TOKEN"

    opencode — in opencode.json:

    {
      "mcp": {
        "brightdata": {
          "type": "remote",
          "url": "https://mcp.brightdata.com/mcp?token=YOUR_TOKEN",
          "enabled": true
        }
      }
    }
  3. Restart your assistant.

That's the whole setup. If you ever outgrow the free tier, Bright Data's docs cover paid zones, but you're unlikely to hit it with a personal briefing habit.

Usage

Just ask, in natural language:

  • "Check the AI leaderboards."
  • "What changed on the leaderboards since last time?"
  • "Who's #1 on text-to-video right now?"
  • "Give me the leaderboard roundup."

The first run lays down a baseline snapshot (nothing to diff against yet). Every run after that shows you what moved.

What it tracks

Category Metric Better
LLM intelligence Artificial Analysis Intelligence Index higher
Coding agents Coding Agent Index higher
Text→Image Arena Elo higher
Image editing Arena Elo higher
Text→Video Arena Elo higher
Image→Video Arena Elo higher
Video editing Arena Elo higher
Text→Speech Arena Elo higher
Speech→Text word error rate / index see board
Speech→Speech Speech-to-Speech Index higher

Example output (excerpt)

## Changes since last run (2026-08-20)
- **Text→Video (Arena)**
  - 👑 New #1: Gemini Omni Flash (was Wan 3.0)
  - ▲ Gemini Omni Flash: #2#1 (+1)
  - 🆕 New in top list: NewModel X
  - ⬇️ Dropped out: MiniMax H3

Notes

  • Metrics differ per board — intelligence indices and Elo are higher-is-better; cost, latency, and word error rate are lower-is-better. The report always states the metric so a reader knows which way is good.
  • All fetched content is treated strictly as data. The skill never acts on instructions found inside a scraped page.
  • Data belongs to Artificial Analysis; this tool just reads their public leaderboards and summarizes them for personal use. Check their site for terms.

License

MIT — see LICENSE. Built by Yaron Been.

About

Agent skill: a dated, side-by-side briefing of the Artificial Analysis AI leaderboards (LLMs, coding agents, image, video, speech) with run-over-run change tracking. Fetches via the Bright Data MCP.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages