This document outlines the tools, libraries, and frameworks used to build the ClipMind AI platform.
- Backend: Python 3.11 with FastAPI (for high-performance async APIs)
- Frontend: TypeScript, React.js, and Next.js 14 (App Router, Standalone mode for Docker)
- Styling: Tailwind CSS
- PostgreSQL: Primary relational database for user management, roles, classrooms, and video metadata. (Hosted on Render)
- MongoDB: NoSQL document database used for storing large, unstructured data such as generated transcripts, summaries, and key moments. (Hosted on MongoDB Atlas)
Note: To achieve the sub-second processing speeds demonstrated in our evaluation (1.02s per summarization), we utilized the Groq LPU API.
- Speech-to-Text: Whisper model architecture (via Groq for speed)
- NLP Summarization & Intelligence: Large Language Models (LLM) for generating concise summaries, study guides, and quizzes.
- Computer Vision / Video Processing: FFmpeg (for extracting audio tracks from video files prior to transcription).
- Local / Ephemeral Storage: File processing and uploads use the local filesystem during AI inference.
- Tools:
ffmpeg-pythonfor parsing video duration and extracting.mp3audio chunks.
- Framer Motion: For fluid, spring-based animations across the Next.js frontend.
- Recharts: For rendering analytics data and top keywords on the dashboard.
- Lucide React: Iconography.
- Containerization: Docker & Docker Compose (Multi-stage builds for both frontend and backend)
- Cloud Hosting: Render.com (Web Services & Managed Postgres)
- Version Control: Git & GitHub
- Testing Framework:
pytestfor backend API and pipeline testing. - Evaluation: Jupyter Notebooks (
backend/ai/evaluation.ipynb) for testing AI model latency and summarization quality.