Build a COM-480 Milestone 3 interactive D3 dashboard that helps investors compare AI value-chain companies across infrastructure, compute, AI infrastructure software, foundation models, and AI-enabled applications.
src/contains application source code and data-pipeline source code.src/data/contains data-pipeline code, seed files, collectors, source adapters, validators, LLM extraction logic, and pipeline CLI entrypoints.data/contains generated machine-readable outputs consumed by the visualization.doc/contains human-facing documentation, milestone PDFs, process-book material, screencast planning, data explanations, and pipeline explanations.agents/contains agent-facing notes and plans that are not the root instruction file.agents/impl.mdis the implementation handoff for the next pipeline work. Keep it current when implementation direction changes.README.mdis the public GitHub entry point required by Milestone 3..env_exampledocuments expected API keys and provider toggles. Never commit.envor real secrets.- Do not put generated app data under
doc/; explain the data there instead.
- Select companies by strategic relevance and source availability, not by layer alone.
- Preserve each company's AI value-chain
layeras a visualization/story field. - Route collection by company-level source coverage across SEC EDGAR, Financial Datasets MCP, FMP MCP, TradingView MCP, Qieman MCP, yfinance, FinanceToolkit, public web, and LLM extraction.
- Treat 36Kr PitchHub (
https://pitchhub.36kr.com/) as a China startup discovery and enrichment source for project pages, project sets, financing events, and investor-interest lists when pages are publicly accessible. - Treat
18.com.cn/ 东方财富 as a China public-market news and theme-discovery source for listed companies, sector/concept pages, financing news, and market articles; do not treat it as a private startup database. - Use public structured sources before public web extraction.
- For public companies, enrich valuation, quality, safety, growth/stability, and timing-overlay metrics before frontend integration. Missing metrics must remain null with review issues rather than guessed.
- Store acquired metrics as time-indexed observations in
data/metrics.sqlite, keyed by company, metric name, provider/source, period, and ISOfetched_at. Export JSON snapshots for the frontend, but do not treat JSON as the metric source of truth. - Use LLMs only to extract structured facts from public text when structured APIs are unavailable or incomplete.
- Cache source fetches and LLM extraction results by source URL/content hash and prompt version.
- Log LLM provider, model, prompt version, source, extracted fields, confidence, and timestamp.
- Do not impose an arbitrary LLM call limit; control usage with caching and source discipline.
- Seed discovery may propose candidates, but it must not silently rewrite seed CSVs.
- Keep
src/data/scripts/pipeline.pyas a thin CLI wrapper. Put implementation modules undersrc/data/scripts/<module>/, such asschema/,sources/,llms/,prompts/, andio/. - Do not wire the dashboard to raw seed exports as if they are final investment data. Build enrichment first, then connect
src/main.tsto enriched generated data.
- Use acquisition-class seed files:
src/data/seeds/public_listed.csvsrc/data/seeds/private_public_web.csvsrc/data/seeds/manual_review.csv
- Keep
personal_interestfor companies the team specifically wants to include. - Use
priorityto distinguish must-have, strong, normal, and candidate companies. - Keep
native_namefor Chinese, Hong Kong, Taiwan, Japan, and Korea companies when a local-language/common local name is available. - For Layer 0 companies, populate
layer_substageandvalue_chain_orderusing the physical AI infrastructure chain:0A Materials & wafers0B Fab equipment0C Memory & storage0D Packaging & substrates0E Optical & electrical interconnect0F Server & rack manufacturing0G Data-center power0H Cooling & thermal0I Construction & deployment
- For listed companies, keep both machine fields (
exchange,ticker) and the display fieldstock_code, formatted likeNASDAQ:MRVL. - Keep source URLs and include reasons close to the seed rows.
Use this format:
[<component>]: <summary>
- <change description line 1>
- <change description line 2>
Rules:
- Summary is imperative mood, lowercase, no trailing period, and no more than 72 characters.
- Body uses
-bullet lines describing what changed and why. - Keep one logical change per commit.
- Reference issues at the end of the body when applicable, for example
Refs: #42.
Common component tags:
misc- minor changes, formatting, config tweaks.bugfix- bug fixes.refactor- restructuring without behavior changes.docs- documentation-only changes.test- tests or benchmarks.ci- CI/CD pipeline changes.agent- agent-side orchestration and notes.scripts- utility scripts, benchmarking, data prep, deployment helpers.data- data pipeline, seed files, generated data schemas, or data docs.viz- dashboard and visualization changes.