Product Hunt Scraper API

A Product Hunt scraper reads the daily leaderboard and returns it as data instead of a rendered page. This repository shows the selector that survives Product Hunt's markup, the cost trick that keeps calls at one credit, and working code in cURL, Python and Node.

All figures below came from live calls on 2026-08-25.

Contents

• Finding a selector that holds
• The call
• One credit, not twenty-five
• Splitting rank from name
• Watching the board move
• Cost
• Ready-made packages

Finding a selector that holds

Product Hunt is built with Tailwind utility classes. Class names like flex min-w-0 flex-1 flex-col describe layout, not meaning, and they change whenever the design does, so any scraper anchored to them breaks on the next deploy.

There are data-test attributes in the page, but the useful ones sit on vote buttons rather than on the cards themselves. The attribute that does survive is the product link:

a[href^="/products/"]

Every entry on the board is one of these, the href contains the slug, and the link text carries the display name. That single anchor returned 70 products on a live run.

The call

curl -G "https://app.scrapingbee.com/api/v1" \
     -H "Authorization: Bearer YOUR-API-KEY" \
     --data-urlencode "url=https://www.producthunt.com/" \
     --data-urlencode "mode=auto" \
     --data-urlencode "max_cost=25" \
     --data-urlencode 'extract_rules={"posts":{"selector":"a[href^=\"/products/\"]","type":"list","output":{"name":{"selector":"a","output":"text"},"url":{"selector":"a","output":"@href"}}}}'

The response is JSON:

{"posts": [{"name": "1. akta.pro", "url": "/products/akta-pro"},
           {"name": "2. Diet Claude", "url": "/products/diet-claude"}]}

One credit, not twenty-five

The instinct with a modern JavaScript site is to switch on rendering and a premium proxy, which costs 25 credits a call. On Product Hunt listing pages that is wasted money, because the product links are already in the server-rendered HTML.

Rather than assume either way, send mode=auto. ScrapingBee tries its configurations from cheapest to most expensive, stops at the first that works, and charges only for that one. The credits actually spent come back in the Spb-auto-cost header:

curl -sS -G "https://app.scrapingbee.com/api/v1" \
     -H "Authorization: Bearer YOUR-API-KEY" \
     --data-urlencode "url=https://www.producthunt.com/" \
     --data-urlencode "mode=auto" \
     -o /dev/null -D - | grep -i "^spb-auto-cost"
# spb-auto-cost: 1

If every configuration fails, the request costs nothing at all. And when the page hardens later, Auto-Mode climbs to meet it without an edit, up to the max_cost ceiling you set.

Splitting rank from name

The link text is "1. akta.pro", so rank and name arrive fused. Separate them once, at parse time, rather than everywhere downstream:

import re, requests

RANK = re.compile(r"^\s*(\d+)\.\s*")
RULES = {"posts": {"selector": 'a[href^="/products/"]', "type": "list",
                   "output": {"name": {"selector": "a", "output": "text"},
                              "url": {"selector": "a", "output": "@href"}}}}

response = requests.get(
    "https://app.scrapingbee.com/api/v1",
    headers={"Authorization": "Bearer YOUR-API-KEY"},
    params={"url": "https://www.producthunt.com/", "mode": "auto",
            "extract_rules": __import__("json").dumps(RULES)},
    timeout=180,
)

for post in response.json()["posts"]:
    match = RANK.match(post["name"])
    print(int(match.group(1)) if match else None,
          RANK.sub("", post["name"]),
          "https://www.producthunt.com" + post["url"])

Watching the board move

Rankings shift all day, and the interesting signal is the movement rather than any single snapshot. At one credit a poll, a check every thirty minutes costs 48 credits a day:

const seen = new Map();

async function poll() {
  for (const p of await board()) {          // board() wraps the call above
    const before = seen.get(p.slug);
    if (before && before !== p.rank) console.log(`${p.name}: #${before} -> #${p.rank}`);
    seen.set(p.slug, p.rank);
  }
}
setInterval(poll, 30 * 60 * 1000);

Cost

Configuration	Credits
Auto-Mode on a listing page	1 observed
Rotating proxy with JavaScript	5
Premium proxy with JavaScript	25
Stealth proxy	75
Any HTTP 500	0

Check the balance before a long poll:

curl "https://app.scrapingbee.com/api/v1/usage" -H "Authorization: Bearer YOUR-API-KEY"

Plan tiers are listed at scrapingbee.com/pricing.

Ready-made packages

pip install product-hunt-scraper-api
npm install product-hunt-scraper-api

from product_hunt_scraper_api import ProductHuntScraper

products, charged = ProductHuntScraper("YOUR-API-KEY").leaderboard_with_cost()
print(len(products), "products for", charged, "credit(s)")

Both split the rank out of the name, absolutise URLs, and expose the charged credits so you can see Auto-Mode working.

Related

Data extraction rules documents the selector syntax used above. AI extraction is the fallback when a layout shifts and describing the fields beats repairing selectors. For launch coverage, Google News and Google search pair naturally with this.

Scope

Only pages Product Hunt serves to anonymous visitors are in scope. The ScrapingBee terms prohibit scraping anything that requires signing in. Keep the API key in an environment variable, not in source, and not in an AI assistant's context window.

License

MIT. See LICENSE.