Skip to content

README.md

Latest commit

 

History

History
330 lines (257 loc) · 11 KB

README.md

File metadata and controls

330 lines (257 loc) · 11 KB
Raw

Media Compliance AI

An end-to-end AI pipeline that audits YouTube video ads against brand and regulatory compliance rules. Paste a YouTube URL, the system fetches the transcript, retrieves relevant compliance rules via RAG from Azure AI Search, and uses GPT-4o to identify violations — returning a structured report with timestamps, severity scores, and confidence ratings.

Live Demo

URL
Streamlit UI https://media-compliance-ai-4gqi5gghhitxrvn9fzjhbq.streamlit.app/
FastAPI Docs https://media-compliance-ai-api.orangewave-6fa5e008.eastus.azurecontainerapps.io/docs
Health Check https://media-compliance-ai-api.orangewave-6fa5e008.eastus.azurecontainerapps.io/health

Architecture

v2 — Current Architecture

flowchart TD
    subgraph ENTRY["Entry Points"]
        UI["Streamlit UI\nstreamlit_app.py"]
        API["FastAPI\nPOST /audit"]
    end

    subgraph CLIENT["Client Side"]
        YTT["YouTube Transcript API\nFetch captions"]
    end

    subgraph LANGGRAPH["LangGraph StateGraph"]
        direction TB
        IDX["Indexer Node\nIngest transcript"]
        AUD["Auditor Node\nGPT-4o Analysis"]
    end

    subgraph AZURE["Azure Infrastructure"]
        SEARCH["Azure AI Search\nVector Store · RAG"]
        AOAI["Azure OpenAI\nGPT-4o + Embeddings"]
        ACA["Azure Container Apps\nHosted API"]
    end

    subgraph OBS["Observability"]
        LS["LangSmith\nTracing"]
    end

    OUT["Audit Report\nPASS / REVIEW / FAIL\nViolations · Timestamps · Confidence"]

    UI -->|"1. fetch transcript"| YTT
    YTT -->|"2. transcript text"| UI
    UI -->|"3. POST /audit + transcript"| API
    API -->|"4. invoke graph"| IDX
    IDX -->|"transcript in state"| AUD
    AUD -->|"similarity search k=3"| SEARCH
    SEARCH -->|"rule chunks"| AUD
    AUD -->|"GPT-4o prompt"| AOAI
    AOAI -->|"violations JSON"| AUD
    AUD --> OUT
    AUD -.->|traces| LS

    classDef entry    fill:#4A90D9,stroke:#2c5f8a,color:#fff
    classDef client   fill:#F39C12,stroke:#d68910,color:#fff
    classDef pipeline fill:#7B68EE,stroke:#5a4dc4,color:#fff
    classDef azure    fill:#0078D4,stroke:#005a9e,color:#fff
    classDef obs      fill:#20B2AA,stroke:#148f89,color:#fff
    classDef output   fill:#27AE60,stroke:#1e8449,color:#fff

    class UI,API entry
    class YTT client
    class IDX,AUD pipeline
    class SEARCH,AOAI,ACA azure
    class LS obs
    class OUT output
Loading

v1 — Original Architecture (Azure Video Indexer)

The first version processed videos entirely server-side through Azure Video Indexer, which also extracted OCR (on-screen text like logos and text overlays) in addition to speech transcripts.

flowchart TD
    subgraph ENTRY1["Entry Points"]
        UI1["Streamlit UI\nstreamlit_app.py"]
        API1["FastAPI\nPOST /audit"]
    end

    subgraph LANGGRAPH1["LangGraph StateGraph"]
        direction TB
        IDX1["Indexer Node\nDownload + Index"]
        AUD1["Auditor Node\nGPT-4o Analysis"]
    end

    subgraph AZURE1["Azure Infrastructure"]
        YTDLP["yt-dlp\nVideo Download"]
        BLOB["Azure Blob Storage\nTemp Video Upload"]
        AVI["Azure Video Indexer\nTranscript + OCR"]
        SEARCH1["Azure AI Search\nVector Store · RAG"]
        AOAI1["Azure OpenAI\nGPT-4o + Embeddings"]
    end

    subgraph BLOCKED1["❌ Failure Point"]
        BLOCKED["YouTube blocks\nAzure datacenter IPs"]
    end

    OUT1["Audit Report\nPASS / REVIEW / FAIL\nViolations · Timestamps · Confidence"]

    UI1 -->|"1. POST /audit URL only"| API1
    API1 -->|"2. invoke graph"| IDX1
    IDX1 -->|"3. download video"| YTDLP
    YTDLP -->|"Sign in to confirm\nyou are not a bot"| BLOCKED
    BLOCKED -.->|"if unblocked"| BLOB
    BLOB -->|"upload"| AVI
    AVI -->|"transcript + OCR"| AUD1
    AUD1 -->|"similarity search k=3"| SEARCH1
    SEARCH1 -->|"rule chunks"| AUD1
    AUD1 -->|"GPT-4o prompt"| AOAI1
    AOAI1 -->|"violations JSON"| AUD1
    AUD1 --> OUT1

    classDef entry    fill:#4A90D9,stroke:#2c5f8a,color:#fff
    classDef pipeline fill:#7B68EE,stroke:#5a4dc4,color:#fff
    classDef azure    fill:#0078D4,stroke:#005a9e,color:#fff
    classDef blocked  fill:#E74C3C,stroke:#c0392b,color:#fff
    classDef output   fill:#27AE60,stroke:#1e8449,color:#fff

    class UI1,API1 entry
    class IDX1,AUD1 pipeline
    class YTDLP,BLOB,AVI,SEARCH1,AOAI1 azure
    class BLOCKED blocked
    class OUT1 output
Loading

Architecture Evolution

v1 — Azure Video Indexer v2 — YouTube Transcript API
Transcript source Azure Video Indexer (server-side) YouTube Transcript API (client-side)
OCR Yes — Azure Video Indexer extracts on-screen text No — captions only
Video download yt-dlp → Azure Blob Storage Not required
Failure point YouTube blocked Azure datacenter IPs None (captions fetched from client)
Fallback — Manual transcript paste in UI

Why it changed: YouTube's bot detection blocks download requests from cloud provider IPs (Azure, AWS, GCP). Every audit attempt failed with "Sign in to confirm you're not a bot". Moving transcript fetching to the client side (Streamlit) bypassed this entirely — Streamlit Cloud uses non-datacenter IPs that YouTube permits.

What was lost: OCR of on-screen text (brand logos, text overlays, disclaimer text). A future improvement would route video frames through Azure Computer Vision for text extraction, independent of the transcript fetch.

Tech Stack

Layer Technology
Orchestration LangGraph (StateGraph)
Transcript Extraction YouTube Transcript API
Compliance Rules Store Azure AI Search (Vector DB + RAG)
LLM Azure OpenAI GPT-4o
Embeddings Azure OpenAI text-embedding-3-small
API FastAPI (async, job-polling pattern)
UI Streamlit
Containerisation Docker + Azure Container Registry
Hosting Azure Container Apps
Observability Azure App Insights + OpenTelemetry + LangSmith
CI GitHub Actions

Pipeline Flow

  1. Transcript Fetch — Streamlit fetches the YouTube transcript client-side using YouTube Transcript API
  2. Submit — Transcript is sent to the FastAPI backend via POST /audit along with the video URL
  3. Ingest — LangGraph Indexer node receives the transcript and loads it into shared state
  4. Retrieve — Auditor node embeds the transcript and runs similarity search (k=3) against Azure AI Search to retrieve the most relevant compliance rule chunks
  5. Audit — GPT-4o receives the transcript + retrieved rules, returns structured violations in JSON
  6. Post-process — Violations are filtered by confidence score:
    • ≥ 0.75 → confirmed violation (CRITICAL or WARNING)
    • 0.50–0.74 → downgraded to REVIEW NEEDED
    • < 0.50 → dropped
  7. Output — PASS / REVIEW / FAIL verdict + violations list with timestamps, confidence scores, and rule source citations

API

POST /audit

Submit a video for compliance auditing. Returns a job_id immediately — the audit runs in the background.

{
  "video_url": "https://www.youtube.com/watch?v=...",
  "transcript": "[00:00:01] Welcome to today's video...",
  "transcript_segments": [
    {"text": "Welcome to today's video", "timestamp": "00:00:01"}
  ]
}

GET /audit/{job_id}

Poll for results. status progresses: PENDING → RUNNING → COMPLETED / FAILED

{
  "job_id": "...",
  "status": "COMPLETED",
  "final_status": "FAIL",
  "final_report": "The video contains two critical violations...",
  "compliance_results": [
    {
      "category": "FTC Disclosure",
      "severity": "CRITICAL",
      "description": "Sponsored content not disclosed within the first 30 seconds.",
      "timestamp": "00:00:05",
      "confidence": 0.95,
      "source": "1001a-influencer-guide-508_1.pdf"
    }
  ]
}

GET /health

{"status": "healthy", "service": "Media Compliance AI"}

Compliance Rules

Rules are chunked, embedded, and indexed into Azure AI Search from:

Document Coverage
FTC Influencer Guide Disclosure requirements for sponsored content, paid partnerships
YouTube Ad Specs & Policies Ad length, format, headline limits, prohibited content

Project Structure

media-compliance-ai/
├── streamlit_app.py               # Streamlit UI — transcript fetch + audit display
├── main.py                        # CLI entry point for local testing
├── Dockerfile                     # Two-stage build (uv + python:3.11-slim)
├── pyproject.toml                 # Dependencies
├── scripts/
│   └── redeploy.ps1               # One-command redeploy to Azure
├── backend/
│   ├── src/
│   │   ├── graph/
│   │   │   ├── state.py           # VideoAuditState — typed shared LangGraph state
│   │   │   ├── nodes.py           # Indexer node + Auditor node
│   │   │   └── workflow.py        # Graph wiring (indexer → auditor → END)
│   │   ├── services/
│   │   │   └── video_indexer.py   # Azure Video Indexer wrapper (fallback)
│   │   └── api/
│   │       ├── server.py          # FastAPI — async job queue, polling endpoints
│   │       └── telemetry.py       # OpenTelemetry + App Insights setup
│   └── data/
│       ├── 1001a-influencer-guide-508_1.pdf   # FTC compliance rules
│       └── youtube-ad-specs.pdf               # YouTube ad policy rules
└── .github/
    └── workflows/
        ├── ci.yml                 # Run tests on every push
        └── cd.yml                 # Manual deploy to Azure Container Apps

Running Locally

Prerequisites

  • Python 3.11+
  • uv package manager
  • Docker Desktop
  • Azure subscription with: OpenAI, AI Search deployed
  • .env file (copy from .env.example and fill in your keys)

Setup

# Create virtual environment and install dependencies
uv sync

# Activate
source .venv/Scripts/activate      # Git Bash / Mac
.venv\Scripts\activate.ps1         # PowerShell

Start the API

uvicorn backend.src.api.server:app --host 0.0.0.0 --port 8000 --reload

Start the UI

streamlit run streamlit_app.py

Open http://localhost:8501, paste a YouTube URL (max 20 minutes), and click Run Audit.

Note: If YouTube blocks automatic transcript fetch (cloud IPs are often blocked), the UI will prompt you to paste the transcript manually. Open the video on YouTube → click ... → Show transcript → copy and paste.

Deployment

Manual redeploy to Azure

.\scripts\redeploy.ps1

This builds the Docker image, pushes to Azure Container Registry, and updates the Azure Container App.

CI/CD

  • CI — runs automatically on every push to main
  • CD — manual trigger only (workflow_dispatch) via GitHub Actions

Constraints

  • Maximum video length: 20 minutes (enforced in the auditor node to control API costs)
  • Transcript must have captions available on YouTube