A computer vision service that analyzes facial geometry and visible skin characteristics from a selfie — built with FastAPI, MediaPipe Face Mesh (478 landmarks), classical OpenCV heuristics, and a rules-gated LLM explanation layer.
The system produces measurable metrics, confidence scores, visual overlays, and plain-language observations. It reports what is visibly present in an image — it does not diagnose medical conditions.
| 🚀 Current version | v0.3.0 — Recommendation Engine + LLM Explanations |
| 🛠️ Stack | Python, FastAPI, MediaPipe, OpenCV |
| ✅ Status | Active development — core pipeline stable, mobile client not yet built |
A live scan through the full pipeline — image quality gate, facial geometry via 478 MediaPipe landmarks, classical OpenCV skin heuristics, and the rules-gated LLM explanation layer.
Each recommendation card shows the deterministic rule ID that triggered
it (e.g. REDNESS_MODERATE) alongside the LLM's plain-language
explanation — generated only from that approved observation, never
invented independently.
⚕️ Sample scores shown are from a test image and are for demonstration purposes only — this analysis reflects visible characteristics, not a medical diagnosis.
Building on the hardened, calibrated pipeline from v0.2.0, this release adds the layer that turns raw scores into readable, safe explanations.
- 🧩 Deterministic recommendation rules engine — a pure, unit-tested module that maps calibrated CV scores to a fixed catalog of recommendation IDs. No ML, no LLM, no ambiguity: same input always produces the same output.
- 💬 LLM explanation layer — takes only the recommendation IDs already decided by the rules engine and turns them into calm, factual, non-alarming prose. The LLM never sees raw scores unsupervised and never invents a new observation.
- 🛡️ Schema-validated LLM output with automatic fallback to canned template text if the API call fails, times out, or returns malformed output — the report is never dependent on a third-party call succeeding.
- 🚫 Medical-language safety net — a keyword filter runs on generated text as a backstop to the prompt-level constraints.
- ⚡ Response caching by recommendation-ID combination to avoid redundant LLM calls across scans that trigger the same rule set.
- 📄 Extended report section in the API response with per- observation explanations, a summary, and a standing disclaimer.
See Changelog for the full version history.
📸 Selfie
↓
🔍 Image Quality Gate (reject blurry / dark / no-face / multi-face input)
↓
🧑💻 Face Detection & Landmarks (MediaPipe, 478 points)
↓
📐 Geometry & Shape Analysis (relative ratios, symmetry, shape heuristic)
↓
🩹 Skin Analysis (redness, pigmentation, texture, spots, under-eye — skin-masked)
↓
📊 Structured Metrics JSON
↓
🧩 Recommendation Rules Engine (deterministic — decides WHAT to report)
↓
💬 LLM Explanation Layer (explains what was already decided — never invents)
↓
📄 Final Report
Core design rule: CV models detect. The rules engine decides what's reportable. The LLM only explains. This separation is intentional and should not be bypassed — see Design Principles.
- 📐 Face geometry — proportional ratios, symmetry score, heuristic face shape classification with confidence.
- 🩹 Skin analysis — redness, pigmentation variation, texture, spot-like region detection, under-eye darkness — all computed on a landmark-derived skin mask that excludes eyes, brows, lips, and nostrils.
- 🚦 Quality gate that rejects unreliable input (blur, poor lighting, extreme angle, no face, multiple faces) with a specific reason.
- 🖼️ Visual debug overlay — skin mask boundary, flagged regions with a confidence gradient, and shape classification labels.
- 🧩 Deterministic recommendation engine with a fixed, auditable recommendation catalog.
- 💬 LLM-generated explanations — non-medical, non-alarming, with automatic fallback and a safety-net content filter.
- 🧪 Automated test harness with relative-ordering invariants (e.g. a visibly redder test image must score higher than a fairer one).
- 🖥️ Web UI with score cards, skin meters, a geometry table, an overlay toggle, and a printable/exportable PDF report.
- 🔬 CV models detect; they do not diagnose.
- 🧩 The rules engine decides what recommendations are allowed — never the LLM.
- 💬 The LLM explains approved results; it never invents observations.
- 📏 Facial measurements are relative/normalized — no exact real-world dimensions are claimed without calibrated depth.
- 🚫 Poor-quality selfies are rejected rather than analyzed unreliably.
- 🏷️ Every response carries a
pipeline_versionfor reproducibility. - ⚕️ No medical or diagnostic language appears anywhere in the output.
ai-face-analyzer/
├── app/
│ ├── main.py # FastAPI service, structured logging & timing
│ ├── schemas.py # Pydantic request/response models
│ ├── config.py # Centralized thresholds & constants
│ ├── pipeline/
│ │ ├── quality.py # Image quality gate
│ │ ├── face_detect.py # MediaPipe Face Landmarker
│ │ ├── geometry.py # Ratios, symmetry, shape classifier
│ │ ├── skin.py # Skin mask + redness/pigmentation/texture/spots
│ │ ├── regions.py # Normalized region output
│ │ └── overlay.py # Debug overlay & rejection diagnostics
│ ├── rules/
│ │ ├── engine.py # Deterministic recommendation engine
│ │ ├── recommendations.py # Fixed recommendation catalog
│ │ └── thresholds.py # Score bands that trigger each recommendation
│ └── llm/
│ ├── client.py # LLM API wrapper
│ ├── prompt.py # Prompt construction
│ └── schema.py # Validated LLM output shape
├── models/ # MediaPipe face_landmarker.task
├── tests/
│ ├── sample_images/ # Categorized test fixtures
│ ├── expected/notes.md # Qualitative expectations & invariants
│ └── test_pipeline.py
├── static/
│ └── index.html # Web UI + PDF report export
├── requirements.txt
├── .env.example
└── README.md
- 🐍 Python 3.11+
- 🔑 An LLM API key (set in
.env, see.env.example)
git clone https://github.com/Souvikkundu0901/ai-face-analyzer.git
cd ai-face-analyzer
python -m venv venv
source venv/bin/activate # Windows: venv\Scripts\Activate.ps1
pip install -r requirements.txt
cp .env.example .env # then add your LLM API keyuvicorn app.main:app --reload --port 8000Open http://localhost:8000 🌐 to use the web UI, or call the API
directly:
curl -X POST http://localhost:8000/api/analyze \
-F "image=@path/to/selfie.jpg"python -m unittest tests/test_pipeline.pyAccepts a selfie (multipart/form-data, field image). Returns a
structured JSON report (metrics, regions, confidence scores, and
explained recommendations), or a 422 with a specific quality-gate
rejection reason.
Returns an annotated debug image showing landmarks, skin mask boundary, and flagged regions.
Liveness check. ❤️
Full request/response schemas are in app/schemas.py.
- 💡 Warm/incandescent lighting can skew redness detection (CIELAB
a*channel); results are most consistent under neutral or daylight lighting. - 🧔 Heavy facial hair or low bangs can compress geometry ratio estimates by obscuring chin/hairline landmarks.
- 🌓 Under-eye contrast detection is currently less reliable on very
dark skin tones — calibration thresholds are adjustable in
config.py. - 🗄️ No persistent scan history yet — each analysis is stateless (planned for a later release).
- 🧠 Core CV pipeline (geometry + skin heuristics)
- 🧪 Automated test harness & calibration
- 🧩 Recommendation rules engine + LLM explanation layer
- 🗄️ Persistent scan history & longitudinal comparison
- 📱 Flutter mobile client
- 🔐 Authentication & multi-user support
- 🏭 Production hardening & bias evaluation across broader skin-tone and lighting datasets
- 🧩 Added deterministic recommendation rules engine.
- 💬 Added LLM explanation layer with schema validation and fallback.
- 🚫 Added medical-language safety-net filter.
- ⚡ Added recommendation-based response caching.
- 🧪 Automated test harness with relative-ordering invariants.
- ⚙️ Centralized all thresholds/config into
app/config.py. - 🖼️ Upgraded debug overlay (skin mask boundary, confidence gradient, rejection diagnostics).
- 🎯 Realistic, non-flat confidence scoring for shape and region detection.
- 🛡️ Robustness improvements (downscaling, angle tolerance, structured logging).
- 🎨 Redesigned web UI with printable PDF report export.
- 🌱 Initial pipeline: image quality gate, MediaPipe face detection and landmarks, geometry ratios, face-shape heuristic, skin heuristics (redness, pigmentation, texture, spots, under-eye darkness).
This project reports visible facial and skin characteristics only. It is not a medical device, does not perform diagnosis, and should not be used as a substitute for consulting a dermatologist or other qualified professional.
Add your chosen license here (e.g. MIT) and include a LICENSE file at
the repo root.
🧑💻 Directed & Created by Souvik Kundu