Skip to content

Latest commit

 

History

History
300 lines (216 loc) · 15.2 KB

File metadata and controls

300 lines (216 loc) · 15.2 KB

FinAI Research Workflow

End-to-end AI agent pipeline for economic and financial academic research — from research idea to manuscript draft. Integrates MCP data acquisition, causal inference (DID/IV/PSM/GMM), LaTeX typesetting, and AI-assisted review loops.

⚠️ Important: Drafts generated by this tool require human review before submission. All causal identification strategies, statistical results, and citations must be verified by a researcher.

⚠️ Legal Risk Servers: 3 MCP servers (user-cnki, user-wanfang, user-chinese-literature) scrape websites that prohibit automated access. They are disabled by default for all users. See LEGAL_CONSENT.md to understand the risk and opt-in with CLI_ACCEPT_RISK.

FinAI Research Workflow — research that can show its work

Python License: MIT Install GitHub release GitHub stars arXiv CI docs codecov

DOI

🇨🇳 中文文档 · 🇬🇧 English Documentation


🚀 One-Liner Pitch

"Tell me your research topic, I'll help you: from literature review → idea generation → empirical design → paper draft → LaTeX compilation — fully automated."

📌 Quick Navigation

I'm looking for... Go here
One-line publish script python scripts/release.py
API reference scripts/ modules with type hints and docstrings
18 AI skills knowledge/skills/
Troubleshooting FAQ FAQ.md
Chinese comprehensive guide 使用指南.md (1049 lines, 13 chapters)

Quick Demo

FinAI guided workflow preview

This guided interface walkthrough shows supported agent hosts, the research-brief checkpoint, separate writing and empirical tracks, fail-closed data routing, and a verifiable delivery package. It is not a live research run and contains no mock coefficients, citations, or statistical claims. Regenerate it with python scripts/demo/gen_quick_demo.py.


👥 Who Is This For?

Audience Use Case
PhD students / researchers Design empirical studies, run econometric analysis, generate LaTeX manuscripts for JF/JFE/RFS/经济研究/金融研究
Finance professors Automate literature reviews, track policy experiments, benchmark against published papers
Graduate students Learn econometric methods (DID/IV/RDD) with automated validation and robustness checks
Quantitative analysts Access A-share data, run factor analysis, generate institutional-grade research reports
AI/ML researchers Explore LLM applications in financial research automation, provenance tracking, HITL design

Not sure? If you've ever spent days downloading data, running regressions, formatting LaTeX tables, or searching for related work — this tool is for you.


✨ Core Capabilities

📊 Data Acquisition (43 MCP server directories: 28 no-key + 12 API-key + 3 opt-in legal)

Note: 28 servers are classified as no-key, 12 require API keys, and 3 opt-in legal-risk sources are disabled by default. The classification is maintained by scripts/count_assets.py.

What you need MCP server
A-share quote/financial/margin user-tushare (need TUSHARE_TOKEN)
China macro (GDP/CPI/M2) user-financial (free)
Federal Reserve / FOMC user-fed-data (free)
World Bank macro user-wb-data (free)
IMF data user-imf-data (free)
OECD data user-oecd-data (free)
US Treasury yield / economic calendar user-eodhd (need key)
US stocks / ESG user-yfinance (free)
Research reports / news user-eastmoney-reports (free)
Forex / shipping / commodities user-enhanced-finance (free)
Academic papers user-arxiv, user-nber-wp, user-semantic-scholar
Chinese literature user-brave-search

Fail-closed routing for every data request: local empirical root → validated cache → compatible MCP/official source → visible stop. Mock data is available only after explicit opt-in.

🧮 Econometric Methods (~33 individual estimators / ~13 method families, JF/JFE/RFS standard)

Note: Numbers below count independent estimators. Some methods depend on optional maintained backends such as linearmodels (marked 🔗); ⭐ denotes self-contained Python implementations. Unavailable backends fail visibly.

  • ⭐ Standard DID + Event Study (2): 2x2 OLS, cluster-robust SE (HC0/HC1/CR0/CR1/CGM)
  • ⭐ Bacon Decomposition (1): Goodman-Bacon (2021) weight diagnostic
  • Staggered DID diagnostics (4): Callaway-Sant'Anna (QJE 2021), Sun-Abraham (REStud 2021), Borusyak (REStud 2024), dCdH — availability depends on the selected implementation; no unmaintained package is installed implicitly
  • 🔗 Synthetic Control (2): Abadie (JASA 2016), Arkhangelsky (Science 2021)
  • 🔗 IV / 2SLS (2): panel IV, Jackknife IV — requires linearmodels
  • 🔗 Panel GMM (2): Arellano-Bond, Blundell-Bord — requires linearmodels
  • ⭐ Other (~20): RDD, triple-diff, panel quantile, interactive fixed effects, local projections, spatial regression, Causal Forest, TVP-VAR, sensitivity analysis (Wild Bootstrap, Leamer bounds)
    • Honest DiD (Rambachan-Roth 2023): requires pip install honestdid (the official Python port of the original R HonestDiD package)

📄 Paper Writing (30 journal templates, English/Chinese)

  • English top: JF · JFE · RFS · JAE · JFQA · JPE · Econometrica
  • Chinese top: 经济研究 · 金融研究 · 管理世界 · 会计研究 · 中国工业经济
  • LaTeX compilation · Figures ≥300 DPI · BibTeX · PRISMA compliance

🤖 18 AI Skills (Claude Code / Cursor / Copilot)

  • Discovery: fin-idea-discovery · fin-generate-idea · fin-novelty-check · fin-lit-review
  • Design: fin-experiment-design · fin-data-acquisition
  • Writing: fin-paper-plan · fin-paper-draft · fin-paper-figure · fin-paper-writing · fin-paper-convert
  • Review: fin-review-loop · fin-submit-check · fin-ref-paper · fin-brief-generator · fin-viz-launch

🏗 Engineering Quality

  • ✅ 677 test files, 7 CI jobs, 3-OS matrix (Ubuntu + macOS + Windows)
  • ✅ Coverage report, codecov badge
  • ✅ Pre-commit hooks (ruff + mypy + codespell + commitlint)
  • ✅ Dependabot (pip + GitHub Actions)
  • ✅ Sigstore-signed releases
  • ✅ Security: SECURITY.md, bandit, 48h ack SLA
  • ✅ Sandbox: AST validation, halt rules
  • ✅ Full data provenance tracking

🚀 Quick Start

# 1. Install
git clone https://github.com/csmar432/finai-research.git
cd finai-research
pip install -e ".[dev, econometrics]"  # optional maintained econometrics backends

# 2. Configure API keys
cp .env.example .env
# At minimum, fill DEEPSEEK_API_KEY (free tier available at console.deepseek.com)

# 3. Health check
python scripts/health_check.py

# 4. Run a research pipeline
python scripts/agent_pipeline.py --topic "Carbon trading policy and corporate green innovation"

🏛 Research Pipeline (preflight + 8 research stages)

Step 0  Health check        → scripts/health_check.py
Step 1  Research ideas      → 8-12 candidate ideas, ranked
Step 2  Idea ↔ Data verify  → scripts/idea_data_checker.py (HITL checkpoint)
Step 3  Literature review   → MCP multi-source, citation network, gap analysis
Step 4  Novelty check       → JF/JFE/RFS/arXiv search
Step 5  Empirical design    → DID/IV/RD/PSM/18 robustness checks
Step 6  Data acquisition    → local/cache/source routing across 43 MCP directories; fail closed on gaps
Step 7  Paper writing       → outline → draft → figures → LaTeX
Step 8  Adversarial review  → multi-round, until publishable

Each step is independently callable and has its own output file as a state carrier.


🏆 Highlights

Metric Value Note
MCP data servers 43 43 directories: 28 fully free, 12 API-key, 3 opt-in legal-risk (CNKI/Wanfang/Chinese Literature)
Econometric method modules 58 56/58 modules have tests; ⭐ self-contained, 🔗 requires an optional maintained backend
Journal templates 30 English + Chinese top journals (JF/JFE/RFS/经济研究/金融研究/管理世界/...)
AI skills 18 .cursor/skills/ operational source, 5 fully automated, 13 prompt-driven
Research directions 45 registered digital_finance / green_finance / carbon_economics / ...
Test files / functions 674 / 12,783 pytest; count maintained by scripts/count_assets.py
Python lines ~206K scripts + ~98K tests
CI jobs 7 batches × ~40 steps lint + 3× smoke + mypy + security + coverage + docker
Coverage 60.0% aggregate / 87.4% critical aggregate gate fail-under=60; critical-path gate 80%

Coverage is reported to Codecov; the next aggregate milestone is 65% while critical research paths remain above 80%.


🤝 Contributing

See CONTRIBUTING.md. All contributions must:

  1. Pass ruff check scripts/
  2. Pass pytest tests/
  3. Use Conventional Commits format: feat:, fix:, docs:, refactor:, test:
  4. Link an issue

We use PR labeler (.github/labeler.yml) for automatic labels.


📜 License

MIT License — Copyright (c) 2026 FinAI Research Workflow Contributors


🔗 Links


👤 Maintainer

This project is maintained by @csmar432.

Contributions of all sizes are welcome — see CONTRIBUTING.md for the workflow.


🙏 Acknowledgments

  • JF / JFE / RFS for econometric methodology standards
  • OpenAlex / ArXiv / Semantic Scholar for academic data
  • MCP (Model Context Protocol) for the tool integration standard
  • Cursor / Claude Code / GitHub Copilot for the AI coding platform



📐 Numerical Correctness Tests

Beyond interface tests, we verify that our econometric estimators recover known coefficients from synthetic data with known DGP:

  • OLS (n=500, σ=0.5): β recovered within 0.2 of truth
  • DID (n=400, β_true=1.5): estimator within 0.3 of true effect
  • IV/2SLS (n=2000, strong instrument): β within 0.2 of structural
  • Kleibergen-Paap F > 10 (instrument strength)

Run with: pytest tests/test_numerical_correctness.py -v


✨ Why FinAI Research Workflow?

  • Built for economists, not generic AI demos — every default is calibrated for the Journal of Finance / 经济研究 standard (DID with heterogeneous treatment effects, cluster-robust SEs at firm level, 19 robustness checks, parallel-trend plots).
  • 43 MCP server directories — covers A-share financials, US equities, global macro (FRED/World Bank/IMF/OECD/BEA), and 400M+ academic papers (OpenAlex). The registry contains 28 no-key, 12 API-key, 0 stub, and 3 opt-in legal-risk directories; classification is maintained by scripts/count_assets.py.
  • 58 econometric method modules, not just OLS — standard DID, event study, Bacon decomposition, staggered DID (Callaway-Sant'Anna/Sun-Abraham/Borusyak/Goodman-Bacon), synthetic control, instrumental variables, panel GMM, RDD, mediation, and more.
  • 30 journal templates (EN/ZH/JP/DE) — JF, JFE, RFS, Econometrica, 经济研究, 金融研究, 管理世界, JPE, RES, AEJ Applied, ZWiSt — all with bilingual (English/Chinese) section headings.
  • Reproducibility first — every step has a script, every script has a --seed flag, every output has data provenance (source, timestamp, hash).

🚦 Quality Gates

This project maintains 3 layers of defense against LLM-generated audit reports containing hallucinated claims:

Layer Mechanism Catches
1 scripts/audit_guard.py (25 deterministic checks) Fake badges, stale metrics, missing tests, version drift
2 pre-commit hook (5 cheap checks <3s) Regression on critical state
3 Manual verification protocol Anything that survives layers 1–2

⚖️ Comparison with Alternatives

Tool Strength FinAI Difference
ChatGPT / Claude Code (bare) Broad knowledge No replication; no MCP live data; no journal templates
Stata + ado packages Battle-tested Manual-only; no LLM-assisted review; no auto-LaTeX
R + tidyverse Rich econometric ecosystem Same limitations as Stata
FinAI Research Workflow End-to-end pipeline with MCP data + journal templates + adversarial review

Built with ❤️ by the open-source financial AI community · MIT License · 2026