|
| 1 | +<p align="center"><a href="https://apify.com/DevilScrapes/github-trending-scraper"><img src="assets/banner.svg" alt="GitHub Trending Scraper — JavaScript example" width="100%"></a></p> |
| 2 | + |
| 3 | +# GitHub Trending API for JavaScript and Node.js: trending repositories as JSON |
| 4 | + |
| 5 | +<p align="center"> |
| 6 | +<a href="https://apify.com/DevilScrapes/github-trending-scraper"><img alt="Run on Apify" src="https://img.shields.io/badge/Run%20on-Apify-DC2626?logo=apify&logoColor=white"></a> |
| 7 | +<img alt="JavaScript" src="https://img.shields.io/badge/Node.js-20%2B-339933?logo=nodedotjs&logoColor=white"> |
| 8 | +<a href="LICENSE"><img alt="MIT license" src="https://img.shields.io/badge/license-MIT-0A0606"></a> |
| 9 | +<a href="https://github.com/devilscrapes/github-trending-scraper-javascript/actions/workflows/ci.yml"><img alt="CI" src="https://github.com/devilscrapes/github-trending-scraper-javascript/actions/workflows/ci.yml/badge.svg"></a> |
| 10 | +</p> |
| 11 | + |
| 12 | +GitHub never shipped an endpoint for its Trending list, so apps that want it end up parsing HTML. This repository skips that chore: it is a single ES module that calls the **[GitHub Trending Scraper](https://apify.com/DevilScrapes/github-trending-scraper)** through the official `apify-client` package and hands your code the list as plain objects. Each object carries the repository's rank, its `owner/name`, URL, tagline, language, forks and two star counts, one for the selected period and one for all time. The scraping happens on Apify's infrastructure, which means your Node service, serverless function or bot only starts a run and reads the result. Consider it a Trending endpoint you never had to sell your soul to GitHub for. Rows are also saved to `results.json` and `results.csv` when you run the script directly. |
| 13 | + |
| 14 | +> **Actor:** [GitHub Trending Scraper](https://apify.com/DevilScrapes/github-trending-scraper) · **Publisher:** [Devil Scrapes](https://apify.com/DevilScrapes) · **Also in Python:** [github-trending-scraper-python](https://github.com/devilscrapes/github-trending-scraper-python) |
| 15 | +
|
| 16 | +## 🔥 What you get |
| 17 | + |
| 18 | +- **One row for every repository on the list**, with its position, the `owner/name` pair split into separate columns, the link, the tagline, the primary language, star and fork counts and the contributor avatars GitHub shows on the card. |
| 19 | +- **Star velocity as its own number**: `stars_in_window` counts only the stars earned during the chosen period, kept apart from the lifetime `stars_total`, so you never diff two snapshots to estimate momentum. |
| 20 | +- **Three periods, two filters**: switch `since` between `daily`, `weekly` and `monthly`, and narrow by programming language, spoken language or both, all producing the same row shape. |
| 21 | +- **GitHub's ordering survives export**: the `rank` column keeps position 1 at position 1 after the rows pass through CSV, a spreadsheet or a database sort. |
| 22 | +- **No GitHub token or rate budget of your own**: requests go out from the Apify run, and throttled or failed page loads are retried with backoff before anything is written. |
| 23 | +- **Every row is stamped** with the window it came from and a UTC collection time, which makes stacking scheduled snapshots into a history table straightforward. |
| 24 | + |
| 25 | +## 🎯 Use cases |
| 26 | + |
| 27 | +### Post a daily trending digest to Slack or Discord |
| 28 | + |
| 29 | +Schedule `node index.js` in GitHub Actions or a cron container, take the top five rows and send them to an incoming webhook. Each message can link `html_url` and show `stars_in_window`, which tells readers why the project is on the list. |
| 30 | + |
| 31 | +### Serve trending data from your own API |
| 32 | + |
| 33 | +Wrap the core call in an Express or Fastify route, cache the response for a few hours and expose `/trending?lang=go` to your frontend. Since every field has a stable name, the response mapping is written once. |
| 34 | + |
| 35 | +### Alert when a watched repository lands on the list |
| 36 | + |
| 37 | +Keep a set of `full_name` values for your own projects or competitors and check each new run against it. A match can open an issue, ping a channel or log an event in your analytics stack. |
| 38 | + |
| 39 | +### Give an AI agent a fresh open source signal |
| 40 | + |
| 41 | +Expose the call as a tool in your agent framework so a briefing bot can ask what is climbing in TypeScript this week. The compact rows fit comfortably into a model's context. |
| 42 | + |
| 43 | +## ⚡ Quick start |
| 44 | + |
| 45 | +You need an Apify account and an API token from [Console → Settings → Integrations](https://console.apify.com/settings/integrations). Apify's free plan includes monthly usage credit; this Actor's event prices are listed under Pricing below. |
| 46 | + |
| 47 | +```bash |
| 48 | +git clone https://github.com/devilscrapes/github-trending-scraper-javascript.git |
| 49 | +cd github-trending-scraper-javascript |
| 50 | +npm install |
| 51 | +export APIFY_TOKEN=your_token_here |
| 52 | +node index.js # uses input.json |
| 53 | +node index.js my-input.json --out data --max-charge 2 |
| 54 | +``` |
| 55 | + |
| 56 | +The script starts the [GitHub Trending Scraper](https://apify.com/DevilScrapes/github-trending-scraper), waits for it to finish, prints a link to the run in Apify Console and saves every result to `results.json` and `results.csv`. It exits with code 1 and the run's status message if the run fails. |
| 57 | + |
| 58 | +## 🧩 The core call |
| 59 | + |
| 60 | +Four statements are all the integration needs. `call()` waits for the run to end, and `listItems()` returns the repositories from its default dataset as an array. |
| 61 | + |
| 62 | +```javascript |
| 63 | +import { ApifyClient } from 'apify-client'; |
| 64 | + |
| 65 | +const client = new ApifyClient({ token: 'YOUR_APIFY_TOKEN' }); |
| 66 | +const run = await client.actor('DevilScrapes/github-trending-scraper').call({"since": "weekly", "language": "rust", "maxResults": 10, "proxyConfiguration": {"useApifyProxy": true}}); |
| 67 | +const { items } = await client.dataset(run.defaultDatasetId).listItems(); |
| 68 | +console.log(items); |
| 69 | +``` |
| 70 | + |
| 71 | +## 📥 Input |
| 72 | + |
| 73 | +`index.js` reads `input.json` and passes it through untouched. The sample requests this week's Rust list and stops at 10 repositories, enough to wire up a prototype without spending much. Change `since`, `language` or `spokenLanguage` per request when you call it from a server. |
| 74 | + |
| 75 | +```json |
| 76 | +{ |
| 77 | + "since": "weekly", |
| 78 | + "language": "rust", |
| 79 | + "maxResults": 10, |
| 80 | + "proxyConfiguration": { |
| 81 | + "useApifyProxy": true |
| 82 | + } |
| 83 | +} |
| 84 | +``` |
| 85 | + |
| 86 | +| Field | Type | Required | Default | Allowed values | |
| 87 | +|---|---|---|---|---| |
| 88 | +| `since` | string | no | `"daily"` | `daily`, `weekly`, `monthly` | |
| 89 | +| `language` | string | no | `""` | — | |
| 90 | +| `spokenLanguage` | string | no | `""` | — | |
| 91 | +| `maxResults` | integer | no | `25` | — | |
| 92 | +| `proxyConfiguration` | object | no | `{"useApifyProxy": true}` | — | |
| 93 | + |
| 94 | +## 📤 Output |
| 95 | + |
| 96 | +The result is an array with one object per listed repository. Numeric fields arrive as integers, `description` and `language` may be `null`, and `built_by` is an array of avatar URL strings that can be empty. |
| 97 | + |
| 98 | +| Field | Type | Always present | |
| 99 | +|---|---|---| |
| 100 | +| `rank` | integer | yes | |
| 101 | +| `full_name` | string | yes | |
| 102 | +| `owner` | string | no | |
| 103 | +| `name` | string | no | |
| 104 | +| `html_url` | string | yes | |
| 105 | +| `description` | string or null | no | |
| 106 | +| `language` | string or null | no | |
| 107 | +| `stars_total` | integer | no | |
| 108 | +| `stars_in_window` | integer | no | |
| 109 | +| `forks` | integer | no | |
| 110 | +| `built_by` | array | no | |
| 111 | +| `scraped_at` | string | no | |
| 112 | +| `window` | string | no | |
| 113 | + |
| 114 | +## 💰 Pricing |
| 115 | + |
| 116 | +| Event | Price (USD) | |
| 117 | +|---|---| |
| 118 | +| Actor start | $0.2 | |
| 119 | +| Result emitted | $0.002 | |
| 120 | + |
| 121 | +Prices as configured on 2026-08-20. The [Actor page](https://apify.com/DevilScrapes/github-trending-scraper) always shows the current price. Runs are billed to the Apify account that owns the token. Use `--max-charge` to cap what a single run can cost. |
| 122 | + |
| 123 | +## 🛠️ Working with the data in JavaScript |
| 124 | + |
| 125 | +The snippet below loads the saved rows, orders them by stars earned in the period and writes a Markdown digest you could post to a chat or commit to a repository. Only Node.js built-ins are used. |
| 126 | + |
| 127 | +```javascript |
| 128 | +import { readFile, writeFile } from 'node:fs/promises'; |
| 129 | + |
| 130 | +const rows = JSON.parse(await readFile('results.json', 'utf8')); |
| 131 | +const top = [...rows] |
| 132 | + .sort((a, b) => (b.stars_in_window ?? 0) - (a.stars_in_window ?? 0)) |
| 133 | + .slice(0, 10); |
| 134 | + |
| 135 | +const period = rows[0]?.window ?? 'daily'; |
| 136 | +const lines = top.map((repo, i) => { |
| 137 | + const lang = repo.language ?? 'n/a'; |
| 138 | + const blurb = repo.description ? ` - ${repo.description}` : ''; |
| 139 | + return `${i + 1}. [${repo.full_name}](${repo.html_url}) (${lang}, +${repo.stars_in_window ?? 0} stars, ${repo.stars_total ?? 0} total)${blurb}`; |
| 140 | +}); |
| 141 | +
|
| 142 | +const digest = `# GitHub trending (${period})\n\n${lines.join('\n')}\n`; |
| 143 | +await writeFile('digest.md', digest, 'utf8'); |
| 144 | +console.log(digest); |
| 145 | +``` |
| 146 | +
|
| 147 | +## ❓ FAQ |
| 148 | +
|
| 149 | +### Is this the same as GitHub's search API sorted by stars? |
| 150 | +
|
| 151 | +No. Search sorted by stars ranks all-time popularity, while Trending ranks recent activity over a day, week or month. This Actor returns GitHub's own published Trending order, including the stars gained during that period. |
| 152 | +
|
| 153 | +### Will this fit inside a Lambda or Vercel function? |
| 154 | +
|
| 155 | +Usually, provided the platform lets the handler wait for the Actor to finish. Under a tight timeout, kick the run off with `start()` rather than `call()` and pick up the dataset later from a webhook or a second invocation. |
| 156 | +
|
| 157 | +### What about a TypeScript codebase? |
| 158 | +
|
| 159 | +The client package bundles its own declarations, so the core call drops into a `.ts` file as is. Declare an interface for the row fields in the Output table and cast the dataset items to it. |
| 160 | +
|
| 161 | +### How do I keep costs predictable in production? |
| 162 | +
|
| 163 | +Set `maxResults` to what you actually display and pass a cap such as `--max-charge 0.5`. Each run costs the start fee plus one result charge per repository written. |
| 164 | +
|
| 165 | +### Why is `built_by` sometimes an empty array? |
| 166 | +
|
| 167 | +GitHub shows contributor avatars on some cards and not on others. The Actor returns what the page includes, so treat the field as optional in your UI. |
| 168 | +
|
| 169 | +## 🧯 Troubleshooting |
| 170 | +
|
| 171 | +- **`Set APIFY_TOKEN first`**: export the token in the same shell, or put it in a local `.env` file your shell loads. Never commit it. |
| 172 | +- **Run finished with `FAILED` or `ABORTED`**: open the printed Console link; the run log says why. Input validation errors name the field. |
| 173 | +- **Fewer rows than expected**: a `--max-charge` cap also limits how many results can be charged. Raise the cap or lower the limits in your input. |
| 174 | +- **Something looks wrong in the data**: open an issue on the [Actor page](https://apify.com/DevilScrapes/github-trending-scraper) so the maintainers see it. |
| 175 | +
|
| 176 | +## 🔗 More from Devil Scrapes |
| 177 | +
|
| 178 | +- [GitHub Trending Scraper on Apify Store](https://apify.com/DevilScrapes/github-trending-scraper) |
| 179 | +- [Python version of this example](https://github.com/devilscrapes/github-trending-scraper-python) |
| 180 | +- [All Devil Scrapes Actors](https://apify.com/DevilScrapes) |
| 181 | +- [Apify web scraping cookbook](https://github.com/devilscrapes/apify-web-scraping-cookbook) |
| 182 | +
|
| 183 | +Related Actors: |
| 184 | +
|
| 185 | +- [GitHub Repo Scraper](https://apify.com/DevilScrapes/github-repo-scraper) — fuller metadata for any repository that catches your eye on Trending. |
| 186 | +- [GitHub Organization Scraper](https://apify.com/DevilScrapes/github-org-scraper) — profile data for the organizations behind the projects that keep climbing. |
| 187 | +- [Hacker News Scraper](https://apify.com/DevilScrapes/hacker-news-scraper) — see whether a climbing repository is also being discussed on Hacker News. |
| 188 | +
|
| 189 | +--- |
| 190 | +
|
| 191 | +MIT licensed example code. The Actor itself runs on [Apify](https://apify.com/DevilScrapes/github-trending-scraper) and is maintained by [Devil Scrapes](https://apify.com/DevilScrapes). Not affiliated with GitHub, Inc.. |
0 commit comments