AP Invoice Exception Assistant

AP Invoice Exception Assistant.

Ingests a vendor invoice (PDF or image), extracts structured line-item data, compares it against a mock purchase order, and lets a reviewer ask "why was this flagged?" in a chat interface that answers using only the actual comparison data - never a generic response.

Architecture

Invoice (PDF/image) --> extraction.py --> structured JSON
                                              |
                                              v
                          Mock PO (JSON) --> compare.py --> exception records
                                              |                  (deterministic,
                                              |                   no LLM)
                                              v
                          Reviewer question --> explain.py --> grounded answer
                                                                 (LLM, but only
                                                                  narrates the
                                                                  exceptions above)

Flask (app.py) wires all three together behind a small API + serves the UI (frontend/index.html)

Key design decision: only two steps in this pipeline use an LLM - extraction (reading the unstructured document) and explanation (narrating an already-computed diff in plain English). The comparison logic in between is plain, deterministic Python. An LLM computing "is $89.50 different from $85.00" is unreliable and unauditable; a threshold check is fast, correct, and testable. This is what keeps every flagged exception traceable back to an exact rule rather than a model's guess.

Tech stack

• Backend: Flask (Python)
• Extraction & explanation LLM: Google Gemini (gemini-3.6-flash), via the google-genai SDK -chosen for its genuine no-cost API tier with native PDF/image support
• Frontend: single HTML page, vanilla JS (no build step, by design - keeps setup to one command)
• Storage: in-memory (see Assumptions below)

Setup

cd backend
pip install -r requirements.txt
pip install flask

Get a free Gemini API key at https://aistudio.google.com/apikey , then:

# Windows PowerShell
$env:GEMINI_API_KEY="your-key-here"

# Mac/Linux
export GEMINI_API_KEY="your-key-here"

Run the server:

python3 app.py

Open http://127.0.0.1:5000. Select a mock PO from the dropdown, upload the matching sample invoice from data-sample_invoices/, and the comparison table + chat panel populate automatically.

Sample data for a quick test

PO	Invoice	Expected result
PO-1001	INV-1001.pdf	Clean match, 0 exceptions
PO-1002	INV-1002.pdf	Price mismatch + tax mismatch
PO-1003	INV-1003.pdf	Quantity mismatch + extra line item

Each invoice is also available as a .png in the same folder, to demonstrate image (not just PDF) input.

Assumptions made

Per the brief's instruction to state assumptions rather than ask - these are the judgment calls built into this solution:

• Matching tolerance: differences under 1% on unit price and 0.5 percentage points on tax rate are not flagged, to avoid noise from rounding. Quantity has zero tolerance - any difference is flagged. These thresholds are configurable constants in compare.py (TOLERANCE_PCT), not hardcoded into the logic.
• Line matching: invoice lines are matched to PO lines by item_code first, falling back to fuzzy description matching (via difflib) if no exact code match exists - mirroring how OCR noise or vendor-side code differences work in practice.
• Tax rate mismatch severity: measured in absolute percentage points, not relative percentage, since relative comparison misbehaves for small base values like tax rates (a 6%→7.5% shift is a 1.5pp move, not a "25% error").
• Storage: invoices processed in a session live in memory and reset when the server restarts. For a 24-hour scope this keeps setup to one command; swapping in a real database is an isolated change that doesn't touch the extraction/comparison/explanation logic.
• Extraction retries: if the model's JSON output fails schema validation, it gets up to 2 retries with the specific validation error fed back to it before the extraction is marked failed. A result that succeeded only after a retry is flagged with confidence: "low" rather than "high," so a reviewer can tell it needed a second attempt.

What's not built

• Persistent storage across server restarts
• Multi-invoice batch upload
• Authentication / multi-user support

These were deliberately left out - the rubric weights extraction quality, comparison correctness, and grounded explanation over UI completeness, so time went there first.