Intelligent two-layer validation system that catches the data quality issues rules can't handle.
Payment data pipelines flag thousands of anomalies daily. ~60% are false positives — legitimate edge cases that waste analyst time. This system uses deterministic rules for clear-cut errors and LLM-powered classification for the ambiguous 30% that rules can't resolve, reducing manual review queues by routing high-confidence classifications automatically. Runs with free LLM providers (Groq, Gemini, DeepSeek) or OpenAI.
| Metric | Result |
|---|---|
| Records processed | 132 synthetic card events |
| Rules engine | 16/16 seeded deterministic errors caught |
| AI layer | 13 ambiguous anomalies classified |
| Tests | 22/22 passing |
| Safety | Read-only advisory, no payment flow touched |
Payment platforms process millions of card transaction events (authorizations, captures, settlements, refunds, chargebacks). When data quality breaks down:
- Finance teams can't reconcile — phantom transactions, duplicate charges
- Fraud models fire on noise — training on dirty data degrades detection
- Compliance reports contain errors — regulatory risk
Current solutions are binary: either no validation (bad data flows through) or rigid rule-based systems that generate massive false-positive queues. The gap is the ambiguous middle — anomalies that look suspicious but might be legitimate (partial captures, international settlement delays, split shipments).
A two-layer architecture that uses AI only where rules fail:
flowchart TD
A[Payment Events CSV] --> B[Rules Engine]
B --> C[Clean Records]
B --> D[Flagged Records]
D --> E[Definite Errors]
D --> F[Ambiguous Anomalies]
F --> G[LLM Classifier]
G --> H[Auto-Classified]
G --> I[Needs Human Review]
C --> J[Validation Report]
E --> J
H --> J
I --> J
Key product insight: Don't replace rules with AI. Use AI only where rules fail — the ambiguous middle. This keeps costs at zero (free-tier providers), maintains trust (deterministic where possible), and focuses AI on high-value classification.
| Category | Examples | DQ Dimension |
|---|---|---|
| Schema violations | Missing transaction_id, empty amount | Completeness |
| Value errors | Negative amounts, invalid currency (XYZ), bad timestamps | Validity |
| Duplicates | Duplicate event_id, duplicate auth per transaction | Uniqueness |
| Sequence breaks | Capture before auth, settlement before capture | Sequence integrity |
| Relationship mismatches | Capture > auth amount, currency/merchant mismatch | Consistency |
| Scenario | What AI Determines |
|---|---|
| Capture = 95% of auth amount | Partial capture (edge case) vs. data error |
| Same merchant + amount, 2 min apart | Duplicate charge (error) vs. separate purchases |
| Refund referencing missing transaction | Pre-migration data gap vs. orphan refund |
| Settlement 7 days after capture | International processing delay vs. stuck settlement |
| Settlement 15 days after capture | Likely stuck (error) — far outside normal window |
| Chargeback + refund on same transaction | Double-recovery risk (error) |
| Chargeback filed 100 days after transaction | Within scheme window (edge case) but unusual |
| Settlement = 97% of capture sum | Processing fees (edge case) — 2-3% is standard |
# Clone and setup
git clone https://github.com/Rick-developer/AI-Payment-Data-Quality-Agent.git
cd AI-Payment-Data-Quality-Agent
# Install dependencies
pip install -r requirements.txt
# Run in rules-only mode (no API key needed)
python main.py --mode rules-only
# Run with AI classification (set ONE API key in .env)
cp .env.example .env
# Edit .env → add GROQ_API_KEY (free at console.groq.com)
python main.py --mode rules+ai
# Run tests
python -m pytest tests/ -vTime to first result: < 2 minutes.
Click to expand project tree
AI-Payment-Data-Quality-Agent/
├── main.py # CLI entry point
├── src/
│ ├── models.py # Data models + validation result types
│ ├── rules/
│ │ ├── schema_rules.py # Schema & value validation
│ │ ├── cross_record_rules.py # Uniqueness, sequences, relationships
│ │ └── validator.py # Rules engine orchestrator
│ ├── ai/
│ │ ├── prompts.py # LLM prompt templates
│ │ └── classifier.py # AI anomaly classifier + confidence routing
│ └── pipeline/
│ └── orchestrator.py # End-to-end pipeline + report generation
├── tests/
│ ├── test_rules.py # 16 unit tests + acceptance criteria
│ └── test_pipeline.py # 6 integration tests
├── data/
│ ├── payment_events.csv # 132 synthetic records
│ └── ground_truth.csv # 29 labeled issues (16 det + 13 ambiguous)
├── scripts/
│ └── generate_dataset.py # Reproducible dataset generator
├── docs/
│ ├── PRD.md # Product Requirements Document (v1.2)
│ ├── data_contract.md # Payment event schema specification
│ ├── notion_case_study_for_import.md # Portfolio case study
│ └── sample_outputs/ # Curated run artifacts
│ ├── validation_report.txt
│ ├── ai_classifications.csv
│ └── run_metadata.json
└── output/ # Generated on run (gitignored)
├── validation_report.txt # Human-readable report
├── run_metadata.json # Run statistics
└── ai_classifications.csv # AI results with explanations
| Decision | Choice | Why |
|---|---|---|
| Rules vs. AI split | 70% rules, 30% AI | AI only where it adds unique value. Rules are cheaper, faster, deterministic |
| Confidence routing | >= 0.85 auto-classify, < 0.85 human review | Balances automation with safety. Threshold is configurable |
| Offline-first | System works fully without API key | Never blocked by budget or API outages |
| Advisory only | Classify and label, never modify or block | System is read-only — it never touches payment flows |
| Card events only | Auth, capture, settlement, refund, chargeback | Focused scope. Each payment rail is a separate product decision |
| Metric | Result |
|---|---|
| Deterministic rule recall | 100% — all 16 seeded errors caught |
| False positive rate | 0% — zero clean records incorrectly rejected |
| Ambiguous anomaly detection | 100% — all 13 seeded anomalies flagged for AI |
| Offline mode | Works — full rules validation without API key |
| Test coverage | 22/22 tests passing |
| Cost per run | $0.00 with Groq/Gemini free tier (~$0.008 with OpenAI gpt-4o-mini) |
- Python 3.10+ — type hints, dataclasses, pathlib
- Multi-provider LLM — Groq (free), Gemini (free), DeepSeek, OpenAI via OpenAI-compatible API
- pytest — 22 automated tests
- python-dotenv — environment configuration
- CSV — input/output format (no database dependency)
This project includes complete product management documentation:
- Notion Case Study — Importable Notion portfolio case study documenting decisions, trade-offs, and outcomes
- PRD v1.2 — Product requirements with persona, effort/impact analysis, MoSCoW prioritization, AI design spec, and production gap register
- Data Contract — Formal schema specification for all 5 card payment event types
- Sample Validation Report — Example generated output
- Sample AI Classifications — Example AI reasoning
This is a portfolio demonstration system, not a production platform:
- All data is synthetic — no real payment data is processed
- Card payment events only (ACH, SWIFT, SEPA excluded)
- Batch processing (no real-time streaming)
- AI accuracy validated on 13 synthetic records (directional, not statistically meaningful)
- Production deployment would require: immutable audit trail, data residency controls, PCI-DSS compliance, drift monitoring, calibrated thresholds — documented in the Production Gap Register