Skip to content

Commit 365d5f3

Browse files
devilscrapesdevilscrapes
committed
OpenCorporates alternative API in Python
0 parents  commit 365d5f3

12 files changed

Lines changed: 577 additions & 0 deletions

File tree

‎.env.example‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
# Your Apify API token: https://console.apify.com/settings/integrations
2+
APIFY_TOKEN=

‎.github/workflows/ci.yml‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
name: CI
2+
3+
on:
4+
push:
5+
pull_request:
6+
7+
permissions:
8+
contents: read
9+
10+
jobs:
11+
test:
12+
runs-on: ubuntu-latest
13+
strategy:
14+
matrix:
15+
python-version: ["3.11", "3.13"]
16+
steps:
17+
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
18+
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
19+
with:
20+
python-version: ${{ matrix.python-version }}
21+
- run: pip install -r requirements-dev.txt
22+
- run: ruff check .
23+
- run: ruff format --check .
24+
- run: pytest -q

‎.gitignore‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
.env
2+
results.json
3+
results.csv
4+
__pycache__/
5+
.venv/
6+
.pytest_cache/
7+
.ruff_cache/

‎LICENSE‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2026 Devil Scrapes
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

‎README.md‎

Lines changed: 211 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,211 @@
1+
<p align="center"><a href="https://apify.com/DevilScrapes/opencorporates-alternative-scraper"><img src="assets/banner.svg" alt="OpenCorporates Alternative — US Business Registry Scraper — Python example" width="100%"></a></p>
2+
3+
# OpenCorporates Alternative API in Python: US state registry lookups into pandas
4+
5+
<p align="center">
6+
<a href="https://apify.com/DevilScrapes/opencorporates-alternative-scraper"><img alt="Run on Apify" src="https://img.shields.io/badge/Run%20on-Apify-DC2626?logo=apify&logoColor=white"></a>
7+
<img alt="Python" src="https://img.shields.io/badge/Python-3.11%2B-3776AB?logo=python&logoColor=white">
8+
<a href="LICENSE"><img alt="MIT license" src="https://img.shields.io/badge/license-MIT-0A0606"></a>
9+
<a href="https://github.com/devilscrapes/opencorporates-alternative-api-python/actions/workflows/ci.yml"><img alt="CI" src="https://github.com/devilscrapes/opencorporates-alternative-api-python/actions/workflows/ci.yml/badge.svg"></a>
10+
</p>
11+
12+
This repository is a compact Python client for the **[OpenCorporates Alternative Scraper](https://apify.com/DevilScrapes/opencorporates-alternative-scraper)** on Apify. You hand it a list of company names, and it hands back what the New York, Colorado, Connecticut and Oregon registries have on file for each one: the legal name the state recorded, the filing type, the standing, when the entity was formed, and who its registered agent is. The lookups run on Apify's servers; your side is a short script and a JSON file. It suits analysts who keep vendor or prospect lists in CSV and want a check they can rerun next quarter.
13+
14+
> **Actor:** [OpenCorporates Alternative — US Business Registry Scraper](https://apify.com/DevilScrapes/opencorporates-alternative-scraper) · **Publisher:** [Devil Scrapes](https://apify.com/DevilScrapes) · **Also in JavaScript:** [opencorporates-alternative-api-javascript](https://github.com/devilscrapes/opencorporates-alternative-api-javascript)
15+
16+
## 🔥 What you get
17+
18+
- **Records from the states themselves**: names are matched against the free open-data datasets that New York, Colorado, Connecticut and Oregon publish, not against a third-party aggregator or a paid search page.
19+
- **One schema for four registries**: each state names its columns differently; every row comes back with the same keys, so a single parser covers all of them.
20+
- **Comparable status bucket**: alongside the state's own wording, `status_normalized` puts each entity into `active`, `inactive` or `unknown`, which you can also filter on before rows are written.
21+
- **Connecticut officers on request**: with `fetchOfficers` on, matched Connecticut rows carry their principals. Home addresses from that dataset are left out on purpose; only business addresses are kept.
22+
- **A shaky state does not stop the batch**: temporary errors are retried with backoff, and a state dataset that keeps failing is skipped with a warning while the remaining names and states resolve.
23+
- **Traceable rows**: each record links to the raw state record and to the dataset's landing page, so an auditor can check the source in one click.
24+
25+
## 🎯 Use cases
26+
27+
### Screen a vendor list before procurement signs anything
28+
29+
Export supplier names from your ERP, feed them through `input.json` and read the output into a DataFrame. Names with no registry hit, or with only inactive hits, become your follow-up list. The rest get a formation date and an entity type next to them for the file.
30+
31+
### Clean a prospect spreadsheet ahead of a campaign
32+
33+
Join the rows back to your lead sheet on the `query` column. Companies that the state marks as dissolved or otherwise inactive can be dropped before anyone spends an email on them, and the registered agent column helps spot shell-like filings that share one agent.
34+
35+
### Run a quarterly re-check from cron
36+
37+
Point a cron entry or an Airflow task at `python main.py`, write each run into a dated folder with `--out`, and compare standings between quarters. A company that moves from active to inactive is exactly the change a credit or compliance team wants surfaced.
38+
39+
### Support an investigation with citable records
40+
41+
Reporters and researchers can collect formation dates and agent names for a list of entities, then cite `source_record_url` for every claim. The link points at the state's own record, which is what an editor or a lawyer will ask for.
42+
43+
## ⚡ Quick start
44+
45+
You need an Apify account and an API token from [Console → Settings → Integrations](https://console.apify.com/settings/integrations). Apify's free plan includes monthly usage credit; this Actor's event prices are listed under Pricing below.
46+
47+
```bash
48+
git clone https://github.com/devilscrapes/opencorporates-alternative-api-python.git
49+
cd opencorporates-alternative-api-python
50+
python -m venv .venv && source .venv/bin/activate
51+
pip install -r requirements.txt
52+
export APIFY_TOKEN=your_token_here
53+
python main.py # uses input.json
54+
python main.py my-input.json --out data --max-charge 2
55+
```
56+
57+
The script starts the [OpenCorporates Alternative — US Business Registry Scraper](https://apify.com/DevilScrapes/opencorporates-alternative-scraper), waits for it to finish, prints a link to the run in Apify Console and saves every result to `results.json` and `results.csv`. It exits with code 1 and the run's status message if the run fails.
58+
59+
## 🧩 The core call
60+
61+
Strip away the CLI handling and this is what remains. The official `apify-client` package starts a run, blocks until it completes, and iterates over the dataset rows for you.
62+
63+
```python
64+
from apify_client import ApifyClient
65+
66+
client = ApifyClient("YOUR_APIFY_TOKEN")
67+
run = client.actor("DevilScrapes/opencorporates-alternative-scraper").call(run_input={'searchQueries': ['Blue Sky Consulting LLC'], 'jurisdictions': ['NY', 'CO'], 'fetchOfficers': True, 'maxResultsPerQuery': 5, 'statusFilter': 'any', 'proxyConfiguration': {'useApifyProxy': False}})
68+
for item in client.dataset(run.default_dataset_id).iterate_items():
69+
print(item)
70+
```
71+
72+
## 📥 Input
73+
74+
`main.py` forwards `input.json` to the Actor as written. The sample looks up one common consulting-firm name in the New York and Colorado registries and keeps at most five matches, the same input the Actor's own release check runs, so a first run returns rows. Swap in your own list; `searchQueries` takes up to 500 names in a single run.
75+
76+
```json
77+
{
78+
"searchQueries": [
79+
"Blue Sky Consulting LLC"
80+
],
81+
"jurisdictions": [
82+
"NY",
83+
"CO"
84+
],
85+
"fetchOfficers": true,
86+
"maxResultsPerQuery": 5,
87+
"statusFilter": "any",
88+
"proxyConfiguration": {
89+
"useApifyProxy": false
90+
}
91+
}
92+
```
93+
94+
| Field | Type | Required | Default | Allowed values |
95+
|---|---|---|---|---|
96+
| `searchQueries` | array | yes | `["Blue Sky Consulting LLC"]` | — |
97+
| `jurisdictions` | array | no | `["NY", "CO", "CT", "OR"]` | — |
98+
| `fetchOfficers` | boolean | no | `true` | — |
99+
| `maxResultsPerQuery` | integer | no | `5` | — |
100+
| `statusFilter` | string | no | `"any"` | `any`, `active`, `inactive` |
101+
| `proxyConfiguration` | object | no | `{"useApifyProxy": false}` | — |
102+
103+
## 📤 Output
104+
105+
Each row is one entity that one state returned for one of your names, so a single input name can yield several rows across states. Dates arrive as ISO `YYYY-MM-DD` strings, and nullable columns stay `None` when a state does not publish that value rather than being filled with guesses.
106+
107+
| Field | Type | Always present |
108+
|---|---|---|
109+
| `query` | string | yes |
110+
| `entity_id` | string | yes |
111+
| `entity_name` | string | yes |
112+
| `jurisdiction` | string | yes |
113+
| `entity_type` | string or null | no |
114+
| `status` | string or null | no |
115+
| `status_normalized` | string | yes |
116+
| `formation_date` | string or null | no |
117+
| `dissolution_date` | string or null | no |
118+
| `principal_address` | string or null | no |
119+
| `registered_agent_name` | string or null | no |
120+
| `registered_agent_address` | string or null | no |
121+
| `officers` | array | yes |
122+
| `source_record_url` | string | yes |
123+
| `registry_url` | string | yes |
124+
| `scraped_at` | string | yes |
125+
126+
## 💰 Pricing
127+
128+
| Event | Price (USD) |
129+
|---|---|
130+
| Actor start | $0.05 |
131+
| Result row emitted | $0.005 |
132+
133+
Prices as configured on 2026-08-12. The [Actor page](https://apify.com/DevilScrapes/opencorporates-alternative-scraper) always shows the current price. Runs are billed to the Apify account that owns the token. Use `--max-charge` to cap what a single run can cost.
134+
135+
## 🛠️ Working with the data in Python
136+
137+
The snippet below, which needs pandas (`pip install pandas`), rolls the rows up to one line per input name: how many entities matched, in which states, whether any of them is active, and the earliest formation date. It also lists the names from `input.json` that found nothing at all.
138+
139+
```python
140+
import json
141+
142+
import pandas as pd
143+
144+
names = json.load(open("input.json", encoding="utf-8"))["searchQueries"]
145+
rows = pd.DataFrame(json.load(open("results.json", encoding="utf-8")))
146+
147+
rows["formed"] = pd.to_datetime(rows["formation_date"], errors="coerce")
148+
rows["is_active"] = rows["status_normalized"].eq("active")
149+
150+
summary = (
151+
rows.groupby("query")
152+
.agg(
153+
matches=("entity_id", "size"),
154+
states=("jurisdiction", lambda s: ",".join(sorted(set(s)))),
155+
any_active=("is_active", "any"),
156+
earliest_formed=("formed", "min"),
157+
)
158+
.reset_index()
159+
)
160+
summary.to_csv("verification_summary.csv", index=False)
161+
print(summary.to_string(index=False))
162+
163+
not_found = sorted(set(names) - set(rows["query"]))
164+
print("No registry match:", ", ".join(not_found) or "none")
165+
```
166+
167+
## ❓ FAQ
168+
169+
### Does this use OpenCorporates data or need an OpenCorporates key?
170+
171+
Neither. The Actor reads the four states' own open-data datasets, so the only credential you need is an Apify API token. It is an alternative source for the same kind of answer, not a wrapper around OpenCorporates.
172+
173+
### Which states are covered?
174+
175+
New York, Colorado, Connecticut and Oregon. Narrow the list with `jurisdictions` when you know where a company is registered; fewer states means fewer requests and a quicker run.
176+
177+
### Why is `status` empty for some New York or Oregon rows?
178+
179+
Those two states do not include a standing column in their open datasets. The Actor keeps `status` as `None` instead of inventing one, and its `status_normalized` value for those states is a default, not a verified standing. Treat an `active` filter on NY and OR rows as a weak signal and confirm important cases with the state.
180+
181+
### Why is the `officers` list empty?
182+
183+
Only Connecticut publishes a principals dataset that the Actor reads, so rows from the other three states always carry an empty list. For Connecticut, keep `fetchOfficers` set to `true`.
184+
185+
### How fresh is the data?
186+
187+
As fresh as each state's published snapshot. New York describes its dataset as current to the last business day of the month, so recent filings can lag there. None of the four sources is a live transactional lookup.
188+
189+
## 🧯 Troubleshooting
190+
191+
- **`Set APIFY_TOKEN first`**: export the token in the same shell, or put it in a local `.env` file your shell loads. Never commit it.
192+
- **Run finished with `FAILED` or `ABORTED`**: open the printed Console link; the run log says why. Input validation errors name the field.
193+
- **Fewer rows than expected**: a `--max-charge` cap also limits how many results can be charged. Raise the cap or lower the limits in your input.
194+
- **Something looks wrong in the data**: open an issue on the [Actor page](https://apify.com/DevilScrapes/opencorporates-alternative-scraper) so the maintainers see it.
195+
196+
## 🔗 More from Devil Scrapes
197+
198+
- [OpenCorporates Alternative — US Business Registry Scraper on Apify Store](https://apify.com/DevilScrapes/opencorporates-alternative-scraper)
199+
- [JavaScript version of this example](https://github.com/devilscrapes/opencorporates-alternative-api-javascript)
200+
- [All Devil Scrapes Actors](https://apify.com/DevilScrapes)
201+
- [Apify web scraping cookbook](https://github.com/devilscrapes/apify-web-scraping-cookbook)
202+
203+
Related Actors:
204+
205+
- [New Business Filings — US State Registry Leads Feed](https://apify.com/DevilScrapes/new-business-filings-leads-scraper) — a dated feed of newly formed companies from the same four state registries.
206+
- [UCC Lien Filing Scraper — CT + CO State Portals](https://apify.com/DevilScrapes/ucc-lien-filing-scraper) — lien filings against debtors in Connecticut and Colorado.
207+
- [GLEIF LEI Scraper — Legal Entity Identifier Records](https://apify.com/DevilScrapes/gleif-lei-records-scraper) — Legal Entity Identifier records when the counterparty sits outside these states.
208+
209+
---
210+
211+
MIT licensed example code. The Actor itself runs on [Apify](https://apify.com/DevilScrapes/opencorporates-alternative-scraper) and is maintained by [Devil Scrapes](https://apify.com/DevilScrapes). Not affiliated with OpenCorporates or any US state Secretary of State office.

‎assets/banner.svg‎

Lines changed: 41 additions & 0 deletions
Loading

‎input.json‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
{
2+
"searchQueries": [
3+
"Blue Sky Consulting LLC"
4+
],
5+
"jurisdictions": [
6+
"NY",
7+
"CO"
8+
],
9+
"fetchOfficers": true,
10+
"maxResultsPerQuery": 5,
11+
"statusFilter": "any",
12+
"proxyConfiguration": {
13+
"useApifyProxy": false
14+
}
15+
}

0 commit comments

Comments
 (0)