Ask a question about an organisation's open data in plain English and get a short data story with charts. Every number comes from a SQL query run on the real data, and every chart shows its query.
One engine, one config folder per client. Live: search.portaljs.com (DataHub, the default) · search.portaljs.com/demo/toronto
How it works in depth (pipeline, costs, hosting, decisions): ARCHITECTURE.md.
Datopian has two ways to build a client demo. They are different products, in different repos.
| PortalJS Ask (this repo) | Portal demo | |
|---|---|---|
| What the client sees | One page with a question box. A question gets a short data story with charts | Their own portal, modernised: catalogue, search, dataset pages, plus an optional data story and "Ask AI" |
| What it sells | The AI question-answering layer on its own | A portal revamp or migration (with AI as a bonus) |
| Best for | Organisations with open data but no portal they care about, or an AI add-on for an existing client | Anyone who already runs a portal: CKAN, ArcGIS Hub, EntryScape, Opendatasoft, Socrata |
| Effort | Hours: pick 5–20 datasets, write data notes and example questions | Days: connect the backend, brand it, write one story |
| Repo | datopian/portaljs-ask, one folder per client in portals/ |
datopian/data-portal-examples (internal), see its CLIENT-DEMO-GUIDE.md |
| URL | search.portaljs.com/demo/<slug> or the client's own domain |
<slug>.arc.portaljs.com or <slug>.portaljs.com |
They combine well: a portal demo's story page can link to the client's PortalJS Ask page for follow-up questions.
It is:
- A Next.js app on Vercel. A visitor's question goes to Claude, which writes SQL. The SQL runs in the visitor's browser (DuckDB) over the client's data as parquet files, and Claude then writes the story from the result rows.
- Fact-checked automatically: a second AI call compares each claim in a new story with the rows and fixes wrong ones before the story is shown or cached.
- Cheap to show: example questions are pre-computed (free and instant), and repeated questions are cached for 30 days. Only new typed questions cost money, about 5–8 US cents each.
- Configuration, not code, per client: a folder in
portals/<slug>/. Never a fork.
It is not:
- Not a document search or chatbot. It answers from tables only. The document demos
(
/security,/governance) were removed on 2026-10-12. - Not a data portal. There's no catalogue, dataset pages or publishing. For that, build a portal demo (§1).
- Not for private or big data. Data files are public, and it works up to tens of MB per table. Non-public or multi-GB data needs a server-side setup, which is a separate offer.
- Not a live feed. Each portal is a snapshot we download, clean and describe. Refreshing it
is a deliberate step (
portal-refresh). - Not logins or permissions. Everyone sees the same data.
| What | Where |
|---|---|
| Code | This repo. main is production |
| Hosting | Vercel, Datopian team, project portaljs-ask. Every pull request gets a preview URL |
| Client configs | portals/<slug>/: portal.json, sources.json, notes.md, profile.md, instant.json |
| Which portal is the homepage, client domains | portals/index.json |
| Data files | Vercel Blob store portaljs-ask-data, one folder per portal and snapshot. Not in git |
| Team skills for Claude Code | .claude/skills/: portal-new, portal-verify, portal-refresh, ask-cost-report |
Existing portals: datahub (17 DataHub core datasets, the homepage) and toronto (20 City of
Toronto datasets).
You need Node 20+ and an Anthropic API key (ask for the team key; never commit it).
gh repo clone datopian/portaljs-ask
cd portaljs-ask
npm install
cp .env.example .env.local # set ANTHROPIC_API_KEY at minimum
npm run dev # builds the portal configs, then starts Next.js- http://localhost:3000 shows the default portal; http://localhost:3000/demo/toronto shows Toronto.
- Example questions load from
instant.jsonand cost nothing. Typed questions call the API and cost money. - Without the Upstash Redis variables, limits and the answer cache are per process. That's fine locally.
- After editing anything in
portals/, re-runnpm run portals(dev and build do it for you).
Easiest: in Claude Code, from this repo, run the portal-new skill and give it the
organisation, its catalogue URL and (optionally) the datasets. It follows the steps below and
stops at a preview link.
By hand (full detail in ARCHITECTURE.md §6 and .claude/skills/portal-new/SKILL.md):
- Branch:
git checkout -b portal/<slug> - Config: create
portals/<slug>/portal.jsonby copyingportals/toronto/portal.jsonand changing every field (name, owner, headline, theme colours, logo, source, catalogue,data.base=/data/<slug>/,data.snapshot= today). Addsources.jsonwith 5–20 tables. Pick tabular datasets with dates and categories people ask about; skip map-only, PDF-only or very large datasets. - Data:
node scripts/portal-data.mjs <slug>downloads, cleans and converts the data and writesprofile.md. Resolve every Check item it lists. - Notes: write
portals/<slug>/notes.md: for every table, what one row is, every column with unit and range, how to count correctly, and traps. This is where answer quality comes from. Check every number you write with DuckDB. - Questions: add
datasetsand 15–20examplestoportal.json. The first 3 are the headline ones. - Upload data:
BLOB_READ_WRITE_TOKEN=... node scripts/upload-data.mjs <slug>, then put the printed URL indata.remoteinportal.json. Deploys fail without it. - Preview:
npm run portals && npx tsc --noEmit -p ., commit, push, open a draft PR and wait for the Vercel preview. - Pre-compute and verify:
ASK_ADMIN_TOKEN=... node scripts/precompute.mjs <slug> --api <preview URL>, then run theportal-verifyskill. Read every saved story against its rows (counts, ratios, "highest", periods), not only the headline numbers. - Merge once the portal owner agrees. It's live at
search.portaljs.com/demo/<slug>.
Zero-risk mode: set "live": false in portal.json. The question box disappears, only
the pre-computed examples show, and the AI cost is zero. Use it for demos sent by email.
Client's own domain (e.g. ask.example.org): add "<host>": "<slug>" to domains in
portals/index.json, add the domain to the Vercel project, and have the client point a CNAME
at cname.vercel-dns.com. See ARCHITECTURE.md §6.
| Limit | Default | Env var |
|---|---|---|
| Monthly AI spend for live questions | $40 (Anthropic account hard cap: $50) | ASK_MONTHLY_BUDGET_USD |
| Live questions per day, all portals | 30 | ASK_DAILY_TOTAL |
| Live questions per visitor per day | 5 | ASK_DAILY_PER_VISITOR |
When a limit is hit, live questions pause and the examples keep working. Check spend with the
ask-cost-report skill, or:
curl -H "x-ask-admin: $ASK_ADMIN_TOKEN" https://search.portaljs.com/api/data/status
- Run every example question on production; each answer loads and reads correctly.
- Check one headline number per portal against an official published figure.
- Open it on a phone.
- Open one shared answer link (
?q=...) and check the link preview. - Decide
"live": trueorfalsefor this audience. - After a data refresh: bump
data.snapshot(this starts a fresh cache), re-check the notes, re-run pre-compute.
| Problem | Fix |
|---|---|
| "Live questions are paused" | A daily or monthly limit was hit. Examples still work. See §6 |
| Deployment fails | A portal is missing data.remote in portal.json (step 6), or npx tsc --noEmit -p . fails |
| Wrong or odd numbers in answers | Fix notes.md (units, how to count, traps), then re-run precompute.mjs (--restory rewrites stories from existing rows) |
| Question outside the data | Expected: the page offers a free catalogue search linking to matching datasets |