|
| 1 | +<p align="center"><a href="https://apify.com/DevilScrapes/houzz-professionals-scraper"><img src="assets/banner.svg" alt="Houzz Professionals Scraper — Python example" width="100%"></a></p> |
| 2 | + |
| 3 | +# Houzz Professionals Scraper in Python: contractor and designer leads for pandas |
| 4 | + |
| 5 | +<p align="center"> |
| 6 | +<a href="https://apify.com/DevilScrapes/houzz-professionals-scraper"><img alt="Run on Apify" src="https://img.shields.io/badge/Run%20on-Apify-DC2626?logo=apify&logoColor=white"></a> |
| 7 | +<img alt="Python" src="https://img.shields.io/badge/Python-3.11%2B-3776AB?logo=python&logoColor=white"> |
| 8 | +<a href="LICENSE"><img alt="MIT license" src="https://img.shields.io/badge/license-MIT-0A0606"></a> |
| 9 | +<a href="https://github.com/devilscrapes/houzz-professionals-scraper-python/actions/workflows/ci.yml"><img alt="CI" src="https://github.com/devilscrapes/houzz-professionals-scraper-python/actions/workflows/ci.yml/badge.svg"></a> |
| 10 | +</p> |
| 11 | + |
| 12 | +This repository is a compact, tested Python wrapper around the **[Houzz Professionals Scraper](https://apify.com/DevilScrapes/houzz-professionals-scraper)** on Apify. Choose one trade, for instance `general-contractor` or `interior-designer`, add a `City, ST` filter if you only care about one market, and the script hands back the matching Houzz pros as a table: business name, trade, city and state, phone, star rating, review count, and, from each pro's own page, the license number, website and number of portfolio photos. The crawl happens on Apify's servers, so your laptop never requests a Houzz page. What you keep locally is `results.json` and `results.csv`, shaped for a DataFrame, a notebook or the CRM import you run on Mondays. |
| 13 | + |
| 14 | +> **Actor:** [Houzz Professionals Scraper](https://apify.com/DevilScrapes/houzz-professionals-scraper) · **Publisher:** [Devil Scrapes](https://apify.com/DevilScrapes) · **Also in JavaScript:** [houzz-professionals-scraper-javascript](https://github.com/devilscrapes/houzz-professionals-scraper-javascript) |
| 15 | +
|
| 16 | +## 🔥 What you get |
| 17 | + |
| 18 | +- **Twenty trades selectable by slug**: general contractors, architects, interior designers, roofers, HVAC, plumbing, solar installers and more, each mapped to the directory page Houzz itself uses for that trade. |
| 19 | +- **A city filter you can reason about**: `location` keeps only rows whose city, and optionally state, equal what you typed, ignoring case. When the page cap arrives before enough matches, the run's status message reports the shortfall in plain numbers. |
| 20 | +- **Optional second pass per pro**: with `fetchProfileDetails` on, every kept professional's own Houzz page is opened to add `license_number`, `website` and `project_or_photo_count`; switch it off and those three stay `null` at a lower price. |
| 21 | +- **One row per Houzz ID**: the numeric `houzz_professional_id` deduplicates pros that show up on more than one listing page, so a contractor is never billed twice in one run. |
| 22 | +- **Blank means blank**: a badge, rating or license number that Houzz does not show stays `null`. `average_rating` is empty for pros with zero reviews instead of a misleading 0. |
| 23 | +- **Loud failure over quiet emptiness**: if every listing page fails, or pages load but contain no professionals at all, the run ends as failed instead of reporting success with nothing in it. Throttled or flaky pages are retried a few times with growing pauses first. |
| 24 | + |
| 25 | +## 🎯 Use cases |
| 26 | + |
| 27 | +### Score a trade before a sales team calls it |
| 28 | + |
| 29 | +Pull one category, then let pandas compute what share of pros list a license, how many reviews the median firm has and who carries a phone number. Reps get a call sheet sorted by evidence, not by the order Houzz happened to display. |
| 30 | + |
| 31 | +### Compare markets for a franchise or acquisition thesis |
| 32 | + |
| 33 | +Run the same trade with different `location` values and stack the CSVs. Counts of pros, rating spread and portfolio depth per city give an analyst a first read on where a roll-up or a new territory has room. |
| 34 | + |
| 35 | +### Watch a category change month to month |
| 36 | + |
| 37 | +A cron entry or a scheduled notebook can repeat the run each month and append the rows with their `scraped_at` stamp. Review counts that climb, new IDs that appear and licenses that get added all become a short `merge` away. |
| 38 | + |
| 39 | +### Build a sample for home-services research |
| 40 | + |
| 41 | +Survey designers, insurers and materials suppliers often need a list of real firms in one trade to sample from. Houzz IDs make the draw reproducible, and the profile link lets a researcher check any row by hand. |
| 42 | + |
| 43 | +## ⚡ Quick start |
| 44 | + |
| 45 | +You need an Apify account and an API token from [Console → Settings → Integrations](https://console.apify.com/settings/integrations). Apify's free plan includes monthly usage credit; this Actor's event prices are listed under Pricing below. |
| 46 | + |
| 47 | +```bash |
| 48 | +git clone https://github.com/devilscrapes/houzz-professionals-scraper-python.git |
| 49 | +cd houzz-professionals-scraper-python |
| 50 | +python -m venv .venv && source .venv/bin/activate |
| 51 | +pip install -r requirements.txt |
| 52 | +export APIFY_TOKEN=your_token_here |
| 53 | +python main.py # uses input.json |
| 54 | +python main.py my-input.json --out data --max-charge 2 |
| 55 | +``` |
| 56 | + |
| 57 | +The script starts the [Houzz Professionals Scraper](https://apify.com/DevilScrapes/houzz-professionals-scraper), waits for it to finish, prints a link to the run in Apify Console and saves every result to `results.json` and `results.csv`. It exits with code 1 and the run's status message if the run fails. |
| 58 | + |
| 59 | +## 🧩 The core call |
| 60 | + |
| 61 | +Stripped to the minimum, the integration is the snippet below. `apify-client` launches the Actor, waits for the finished status, and yields the dataset rows one at a time. |
| 62 | + |
| 63 | +```python |
| 64 | +from apify_client import ApifyClient |
| 65 | + |
| 66 | +client = ApifyClient("YOUR_APIFY_TOKEN") |
| 67 | +run = client.actor("DevilScrapes/houzz-professionals-scraper").call(run_input={'category': 'general-contractor', 'maxResults': 5, 'maxPages': 1, 'fetchProfileDetails': True}) |
| 68 | +for item in client.dataset(run.default_dataset_id).iterate_items(): |
| 69 | + print(item) |
| 70 | +``` |
| 71 | + |
| 72 | +## 📥 Input |
| 73 | + |
| 74 | +`main.py` forwards `input.json` exactly as written. The bundled example asks for general contractors nationwide, reads a single listing page and stops at five profiles, so a first test costs a few cents. Houzz paginates its trade directory nationwide rather than by city, which is why a city filter can need more pages than a bare category run; raise `maxPages` when a smaller town comes back short. |
| 75 | + |
| 76 | +```json |
| 77 | +{ |
| 78 | + "category": "general-contractor", |
| 79 | + "maxResults": 5, |
| 80 | + "maxPages": 1, |
| 81 | + "fetchProfileDetails": true |
| 82 | +} |
| 83 | +``` |
| 84 | + |
| 85 | +| Field | Type | Required | Default | Allowed values | |
| 86 | +|---|---|---|---|---| |
| 87 | +| `category` | string | yes | `"general-contractor"` | `architect`, `carpenter`, `custom-closet-designers`, `design-build`, `electrical-contractors`, `general-contractor`, `handyman`, `home-builders`, `hvac-contractors`, `interior-designer`, `kitchen-and-bath-remodelers`, `kitchen-remodelers`, `landscape-architect`, `landscape-contractors`, `painters`, `plumbing-contractors`, `pools-and-spas`, `roofing-and-gutter`, `solar-energy-contractors`, `tile-and-stone-contractors` | |
| 88 | +| `location` | string | no | — | — | |
| 89 | +| `maxResults` | integer | no | `200` | — | |
| 90 | +| `maxPages` | integer | no | `20` | — | |
| 91 | +| `fetchProfileDetails` | boolean | no | `true` | — | |
| 92 | +| `proxyConfiguration` | object | no | `{"useApifyProxy": false}` | — | |
| 93 | + |
| 94 | +## 📤 Output |
| 95 | + |
| 96 | +Each item is a single professional. Fields that depend on the profile pass are `null` when `fetchProfileDetails` is off or that pro's page could not be read; the listing fields are still there. Phone numbers keep Houzz's own formatting, and ratings run from 0 to 5. |
| 97 | + |
| 98 | +| Field | Type | Always present | |
| 99 | +|---|---|---| |
| 100 | +| `houzz_professional_id` | integer | yes | |
| 101 | +| `name` | string | yes | |
| 102 | +| `category` | string | yes | |
| 103 | +| `city` | string or null | no | |
| 104 | +| `region` | string or null | no | |
| 105 | +| `phone` | string or null | no | |
| 106 | +| `website` | string or null | no | |
| 107 | +| `review_count` | integer | yes | |
| 108 | +| `average_rating` | number or null | no | |
| 109 | +| `project_or_photo_count` | integer or null | no | |
| 110 | +| `years_in_business` | integer or null | no | |
| 111 | +| `license_number` | string or null | no | |
| 112 | +| `houzz_profile_url` | string | yes | |
| 113 | +| `scraped_at` | string | yes | |
| 114 | + |
| 115 | +## 💰 Pricing |
| 116 | + |
| 117 | +| Event | Price (USD) | |
| 118 | +|---|---| |
| 119 | +| Actor start | $0.02 | |
| 120 | +| Result emitted | $0.002 | |
| 121 | +| Profile page scraped | $0.0005 | |
| 122 | + |
| 123 | +Prices as configured on 2026-08-06. The [Actor page](https://apify.com/DevilScrapes/houzz-professionals-scraper) always shows the current price. Runs are billed to the Apify account that owns the token. Use `--max-charge` to cap what a single run can cost. |
| 124 | + |
| 125 | +## 🛠️ Working with the data in Python |
| 126 | + |
| 127 | +The pandas snippet below (`pip install pandas`) does two jobs: it summarises license coverage and review depth per state, then writes `call_list.csv` with only the pros that list a phone number, licensed firms first. |
| 128 | + |
| 129 | +```python |
| 130 | +import json |
| 131 | + |
| 132 | +import pandas as pd |
| 133 | + |
| 134 | +leads = pd.DataFrame(json.load(open("results.json", encoding="utf-8"))) |
| 135 | +leads["licensed"] = leads["license_number"].notna() |
| 136 | +leads["rating"] = leads["average_rating"].fillna(0) |
| 137 | + |
| 138 | +coverage = leads.groupby("region").agg( |
| 139 | + pros=("houzz_professional_id", "nunique"), |
| 140 | + licensed_share=("licensed", "mean"), |
| 141 | + median_reviews=("review_count", "median"), |
| 142 | +) |
| 143 | +print(coverage.sort_values("pros", ascending=False).round(2).to_string()) |
| 144 | + |
| 145 | +callable_leads = leads[leads["phone"].notna()].sort_values( |
| 146 | + ["licensed", "review_count", "rating"], ascending=False |
| 147 | +) |
| 148 | +keep = ["name", "city", "region", "phone", "website", "license_number", "review_count", "rating", "houzz_profile_url"] |
| 149 | +callable_leads[keep].to_csv("call_list.csv", index=False) |
| 150 | +print(f"{len(callable_leads)} leads with a phone number written to call_list.csv") |
| 151 | +``` |
| 152 | + |
| 153 | +## ❓ FAQ |
| 154 | + |
| 155 | +### Do I need a Houzz account or a Houzz Pro subscription? |
| 156 | + |
| 157 | +No. The Actor reads the directory pages anyone can open in a browser. All you supply is an Apify API token; nothing about your Houzz account, if you have one, is involved. |
| 158 | + |
| 159 | +### Why are there no email addresses? |
| 160 | + |
| 161 | +Houzz does not publish them on directory or profile pages, so the dataset has no email column rather than a guessed one. Pair the `website` field with your own enrichment tool when you need inboxes. |
| 162 | + |
| 163 | +### Can I collect a trade that is not in the category list? |
| 164 | + |
| 165 | +Not yet. Houzz has roughly 78 trades and the input currently accepts the 20 listed in the schema. Any other value fails input validation before a single page is requested. |
| 166 | + |
| 167 | +### Does it cover Houzz sites outside the United States? |
| 168 | + |
| 169 | +No. The Actor reads the US directory on houzz.com only. Regional sites such as the UK or Australian Houzz are out of scope for this version. |
| 170 | + |
| 171 | +## 🧯 Troubleshooting |
| 172 | + |
| 173 | +- **`Set APIFY_TOKEN first`**: export the token in the same shell, or put it in a local `.env` file your shell loads. Never commit it. |
| 174 | +- **Run finished with `FAILED` or `ABORTED`**: open the printed Console link; the run log says why. Input validation errors name the field. |
| 175 | +- **Fewer rows than expected**: a `--max-charge` cap also limits how many results can be charged. Raise the cap or lower the limits in your input. |
| 176 | +- **Something looks wrong in the data**: open an issue on the [Actor page](https://apify.com/DevilScrapes/houzz-professionals-scraper) so the maintainers see it. |
| 177 | + |
| 178 | +## 🔗 More from Devil Scrapes |
| 179 | + |
| 180 | +- [Houzz Professionals Scraper on Apify Store](https://apify.com/DevilScrapes/houzz-professionals-scraper) |
| 181 | +- [JavaScript version of this example](https://github.com/devilscrapes/houzz-professionals-scraper-javascript) |
| 182 | +- [All Devil Scrapes Actors](https://apify.com/DevilScrapes) |
| 183 | +- [Apify web scraping cookbook](https://github.com/devilscrapes/apify-web-scraping-cookbook) |
| 184 | + |
| 185 | +Related Actors: |
| 186 | + |
| 187 | +- [Professional License Lookup Scraper — DE + CO Roster Export](https://apify.com/DevilScrapes/professional-license-lookup-scraper) — cross-check a contractor's license against a state roster. |
| 188 | +- [US Building Permit Leads Scraper (Socrata)](https://apify.com/DevilScrapes/us-building-permit-leads-scraper) — find who is pulling permits for new jobs in a city right now. |
| 189 | +- [New Business Filings — US State Registry Leads Feed](https://apify.com/DevilScrapes/new-business-filings-leads-scraper) — catch trade businesses the week they register with a state. |
| 190 | + |
| 191 | +--- |
| 192 | + |
| 193 | +MIT licensed example code. The Actor itself runs on [Apify](https://apify.com/DevilScrapes/houzz-professionals-scraper) and is maintained by [Devil Scrapes](https://apify.com/DevilScrapes). Not affiliated with Houzz. |
0 commit comments