Convert videos and YouTube links into AI-enhanced markdown notes automatically.
Live on Tailscale Network:
- Server: rainydaze (Oracle Cloud, 22GB RAM)
- Access:
http://100.123.132.122:5000 - Model: Whisper large-v3 (best quality)
- Workers: 8 Gunicorn workers
- Status: ✅ Production Ready
# Clone and setup
git clone git@github.com:bindrap/Voice2Note.git
cd Voice2Note
# Install dependencies
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt gunicorn
# Build whisper.cpp
cd Whisper
git clone https://github.com/ggml-org/whisper.cpp.git .
cmake -B build && cmake --build build -j --config Release
bash ./models/download-ggml-model.sh large-v3
cd ..
# Configure environment
cp .env.example .env
# Edit .env with your SECRET_KEY and settings
# Initialize database
python init_db.py
# Configure firewall (Oracle Linux)
sudo firewall-cmd --permanent --add-port=5000/tcp
sudo firewall-cmd --permanent --add-interface=tailscale0 --zone=trusted
sudo firewall-cmd --reload
# Start production server
gunicorn -w 8 -b 0.0.0.0:5000 --timeout 1800 app:app --daemon- OS Firewall: Port 5000/tcp open, Tailscale interface trusted
- Workers: 8 (optimal for 22GB RAM, multi-user)
- Timeout: 1800s (30 minutes for long videos)
- Model: large-v3 (3GB, best transcription quality)
- Upload Limit: 1GB (configurable in .env)
- ✅ YouTube Support: Paste any YouTube URL to generate notes
- ✅ Local Video Upload: Upload video files (MP4, AVI, MOV, MKV, WebM)
- ✅ Audio Files: Also supports MP3, WAV, M4A
- ✅ AI-Powered: Uses Ollama for intelligent note generation
- ✅ Fast Transcription: Powered by whisper.cpp
- ✅ Markdown Export: Download notes in markdown format
- ✅ User Accounts: Secure authentication system with personal data isolation
- ✅ Background Processing: Videos process in background - navigate away anytime
- ✅ Real-time Progress: Granular progress tracking with detailed status updates
- ✅ Video Management: Cancel active processing, restart failed videos, delete entries
- ✅ Storage Statistics: Track total videos, notes, transcripts, and storage used
- ✅ Processing History: View all your processed videos with status indicators
- ✅ Minimalistic UI: Clean, modern interface with smooth animations
- ✅ Responsive Design: Works seamlessly on desktop and mobile
``` Video/YouTube → Audio Extraction → Transcription → AI Notes → Database → Web UI (yt-dlp/ffmpeg) (whisper.cpp) (Ollama) (SQLite) (Flask)
Features:
- User authentication with session management
- Background processing with threading
- Real-time status polling (2-second intervals)
- Cooperative cancellation with threading.Event()
- Granular progress tracking (5% → 100% with detailed sub-steps) ```
With 24GB RAM (Oracle VM):
- ✅ 1 hour: ~20-60 minutes processing (excellent)
- ✅ 2 hours: ~60-120 minutes processing (excellent)
- ✅ 6 hours: ~3-6 hours processing (supported, long wait)
- ✅ Unlimited: No hard limit - scales linearly with video length
Use large or medium Whisper model for best results with 24GB RAM.
- Python 3.8 or higher
- FFmpeg
- Git
- RAM: 4GB minimum (8GB recommended, 24GB optimal for large videos)
- Disk: 1GB + 100-500MB per hour of video processed
-
FFmpeg: For audio/video processing ```bash
sudo apt update && sudo apt install ffmpeg
brew install ffmpeg
Download from https://ffmpeg.org/download.html
```
-
whisper.cpp: For audio transcription ```bash
git clone https://github.com/ggml-org/whisper.cpp.git cd whisper.cpp cmake -B build cmake --build build -j --config Release
bash ./models/download-ggml-model.sh medium
bash ./models/download-ggml-model.sh base
bash ./models/download-ggml-model.sh large ```
```bash cd /path/to/Voice2Note ```
```bash python3 -m venv venv
source venv/bin/activate
venv\Scripts\activate ```
```bash pip install -r requirements.txt ```
Edit `config.py` and update the paths to match your whisper.cpp installation:
```python WHISPER_PATH = '/path/to/whisper.cpp/build/bin/whisper-cli' WHISPER_MODEL_PATH = '/path/to/whisper.cpp/models/ggml-medium.bin'
WHISPER_MODEL_PATH = '/path/to/whisper.cpp/models/ggml-large.bin' ```
```bash
export OLLAMA_API_KEY="your-api-key"
export SECRET_KEY="your-secret-key" ```
```bash python init_db.py ```
You should see: ``` ✓ Database initialized successfully! ```
```bash python app.py ```
The application will start on `http://localhost:5000\`
For production, use a WSGI server like Gunicorn:
```bash pip install gunicorn
gunicorn -w 8 -b 0.0.0.0:5000 --timeout 1800 app:app
gunicorn -w 8 -b 0.0.0.0:5000 --timeout 1800 app:app --daemon ```
Production Server Management: ```bash
ps aux | grep gunicorn | grep 5000
pkill -f "gunicorn.*5000.*app:app"
cd /path/to/Voice2Note source venv/bin/activate gunicorn -w 8 -b 0.0.0.0:5000 --timeout 1800 app:app --daemon ```
Or use the provided systemd service (Linux):
```bash
sudo cp voice2note.service /etc/systemd/system/
sudo nano /etc/systemd/system/voice2note.service
sudo systemctl enable voice2note sudo systemctl start voice2note ```
- Open `http://localhost:5000\`
- Click "Register" to create your account
- Enter username, email, and password
- Login with your credentials
- Login to your account
- Paste a YouTube URL in the input field
- Click "Process Video"
- Processing happens in background - you can navigate away!
- Check "History" to monitor progress with real-time updates
- View or download your notes when complete
- Login to your account
- Click "Choose a file" or drag & drop a video file
- Click "Process Video"
- Processing continues in background - check History page anytime
- View or download your notes when complete
In the History page, you can:
- Monitor Progress: Real-time progress bars with detailed status (e.g., "55% • Transcribing audio...")
- Cancel Processing: Stop videos currently being processed
- Restart Failed: Retry processing for failed or cancelled YouTube videos
- Delete Videos: Remove any video and its associated data
- View Statistics: See total videos, storage used, and processing history
| Video Length | Whisper Model | Processing Time | Status |
|---|---|---|---|
| 10 minutes | base | ~5-10 minutes | ✅ Fast |
| 10 minutes | medium | ~10-20 minutes | ✅ Recommended |
| 1 hour | base | ~30-60 minutes | ✅ Good |
| 1 hour | medium | ~60-120 minutes | ✅ Quality |
| 1 hour | large (24GB) | ~30-60 minutes | ✅ Excellent |
| 2 hours | base | ~60-120 minutes | ✅ Supported |
| 2 hours | medium | ~2-4 hours | ✅ Supported |
| 2 hours | large (24GB) | ~60-120 minutes | ✅ Excellent |
| 6 hours | base | ~3-6 hours | |
| 6 hours | medium | ~6-12 hours | |
| 6 hours | large (24GB) | ~3-6 hours | ✅ Feasible |
Times are for CPU processing. GPU acceleration can be 5-10x faster.
Maximum Video Length:
- Technical limit: No hard limit
- Practical considerations:
- RAM requirements: 4-8GB for base, 8-16GB for medium, 16-24GB for large
- Disk space: ~100-500MB per hour of video for temporary audio files
- Processing time: Scales linearly (2x length = 2x time)
- API token limits: Very long transcripts (>6 hours) may need chunking
Happy Note-Taking! 📝
``` voice2note/ ├── app.py # Main Flask application with auth & threading ├── config.py # Configuration ├── requirements.txt # Python dependencies ├── init_db.py # Database initialization ├── reset_database.py # Database reset utility │ ├── database/ │ ├── schema.sql # Database schema (with users table) │ └── db_manager.py # Database operations & user management │ ├── processors/ │ ├── video_handler.py # Video/audio extraction (yt-dlp/ffmpeg) │ ├── transcriber.py # Whisper.cpp integration │ └── note_generator.py # Ollama note generation with streaming │ ├── templates/ │ ├── base.html # Base template with navbar │ ├── index.html # Main upload page with stats │ ├── notes.html # Notes viewer with markdown rendering │ ├── history.html # Processing history with real-time updates │ ├── login.html # User login page │ ├── register.html # User registration page │ └── profile.html # User profile with statistics │ ├── static/ │ ├── css/ │ │ └── style.css # Minimalistic styling with animations │ ├── js/ │ │ └── app.js # JavaScript utilities │ └── favicon.svg # Application favicon │ ├── temp/ # Temporary audio/video files └── voice2note.db # SQLite database ```
Edit `config.py` to change the Whisper model:
```python WHISPER_MODEL = 'base' # Fast, good accuracy (142 MB) WHISPER_MODEL = 'small' # Better accuracy (466 MB) WHISPER_MODEL = 'medium' # Best balance (1.5 GB) - RECOMMENDED for 8GB RAM WHISPER_MODEL = 'large' # Best accuracy (2.9 GB) - RECOMMENDED for 24GB RAM ```
The Ollama API key is already configured in `config.py`. To use a different model:
```python OLLAMA_MODEL = 'gpt-oss:120b' # Current model ```
Default maximum file size is 500MB. To change:
```python MAX_CONTENT_LENGTH = 1000 * 1024 * 1024 # 1GB ```
Perfect for: Long videos (6+ hours), maximum quality, parallel processing
```python
WHISPER_MODEL = 'large' # Best accuracy (2.9GB)
```
Capabilities with 24GB RAM:
- ✅ Process 6+ hour videos with ease
- ✅ Use large Whisper model for best accuracy
- ✅ Handle multiple videos simultaneously
- ✅ Run local Ollama models (fully offline)
- ✅ No memory constraints
Processing Performance (24GB RAM, multi-core CPU):
| Video Length | Model | Processing Time | Quality |
|---|---|---|---|
| 1 hour | large | ~30-60 min | Excellent |
| 2 hours | large | ~60-120 min | Excellent |
| 6 hours | large | ~3-6 hours | Excellent |
| 1 hour | medium | ~20-40 min | Very Good |
```python WHISPER_MODEL = 'medium' # Recommended (1.5GB) ```
```python WHISPER_MODEL = 'base' # Fast and efficient (142MB) ```
For much faster transcription, build whisper.cpp with GPU support:
```bash cd whisper.cpp cmake -B build -DGGML_CUDA=1 cmake --build build -j --config Release ```
```bash cd whisper.cpp cmake -B build -DWHISPER_COREML=1 cmake --build build -j --config Release ```
Use quantized models for faster processing with slightly reduced accuracy:
```bash cd whisper.cpp ./build/bin/quantize models/ggml-medium.bin models/ggml-medium-q5_0.bin q5_0 ```
Update config.py: ```python WHISPER_MODEL_PATH = '/path/to/whisper.cpp/models/ggml-medium-q5_0.bin' ```
Register User ```http POST /register Content-Type: application/x-www-form-urlencoded
username=&email=&password=&confirm_password= ```
Login ```http POST /login Content-Type: application/x-www-form-urlencoded
username=&password= ```
Logout ```http GET /logout ```
Get Profile ```http GET /profile ```
Process Video (requires authentication) ```http POST /process Content-Type: multipart/form-data
youtube_url=https://youtube.com/watch?v=... OR video_file=
Response: { "success": true, "video_id": 123, "message": "Processing started in background...", "processing": true } ```
Get Processing Status (requires authentication) ```http GET /status/<video_id>
Response: { "status": "transcribing", "progress": 55, "error_message": null } ```
Cancel Processing (requires authentication) ```http POST /cancel/<video_id>
Response: { "success": true, "message": "Processing cancelled" } ```
Restart Processing (requires authentication) ```http POST /restart/<video_id>
Response: { "success": true, "video_id": 123, "message": "Processing restarted" } ```
Delete Video (requires authentication) ```http POST /delete/<video_id>
Response: { "success": true, "message": "Video and all associated data deleted" } ```
Get Notes (requires authentication) ```http GET /notes/<video_id> ```
Download Notes (requires authentication) ```http GET /download/<video_id> ```
Get All Videos (requires authentication) ```http GET /api/videos
Response: Array of video objects with status, progress, etc. ```
Search Videos (requires authentication) ```http GET /api/search?q= ```
Videos go through these states:
- `pending` (5%) - Queued for processing
- `extracting` (10-30%) - Extracting audio from video
- `transcribing` (30-60%) - Converting audio to text
- `generating` (60-100%) - Creating AI-enhanced notes
- `completed` (100%) - Notes ready
- `failed` - Processing error occurred
- `cancelled` - User cancelled processing
Error: `whisper.cpp not found at /path/to/whisper.cpp`
Solution:
- Verify whisper.cpp is installed and built
- Update `WHISPER_PATH` in `config.py` to the correct path
- Ensure the binary is executable: `chmod +x /path/to/whisper.cpp/build/bin/whisper-cli`
Error: `Whisper model not found at /path/to/model`
Solution:
- Download the model: `cd whisper.cpp && bash ./models/download-ggml-model.sh medium`
- Update `WHISPER_MODEL_PATH` in `config.py`
Error: `Failed to extract audio: ffmpeg not found`
Solution: Install FFmpeg using your package manager (see Prerequisites)
Error: `Failed to generate notes: API error`
Solution:
- Verify the API key is correct
- Check your internet connection
- Verify the Ollama API is accessible
Error: Process killed or memory errors
Solution:
- Use a smaller Whisper model (base instead of medium)
- Close other applications
- Process shorter videos
- Add more RAM to your system
- For 24GB RAM systems: Use large model for best results
Solutions:
- Use a smaller Whisper model (`base` instead of `medium`)
- Enable GPU acceleration for whisper.cpp
- Use a machine with more CPU cores
- Process shorter videos
- For 24GB RAM: Upgrade to `large` model with more CPU cores
Q: How long does processing take? A: Typically 2-5x the video length for transcription, plus 1-3 minutes for note generation. See processing time table above for details.
Q: Can I process videos longer than 1 hour? A: Yes! The system supports 1-6+ hour videos. Processing scales linearly: 2-hour videos take ~2-4 hours with medium model on CPU. With 24GB RAM and large model, you can efficiently process 6+ hour videos.
Q: Can I navigate away while processing? A: Yes! Processing happens in the background. You can close the tab, navigate away, or even restart your browser. Check the History page anytime to see progress.
Q: Can I cancel a video that's processing? A: Yes! Go to History page and click "Cancel" on any processing video. You can also restart failed or cancelled YouTube videos.
Q: Is my data private? A: Transcription happens locally on your machine. Only the transcript is sent to Ollama API for note generation. Each user's data is isolated - you only see your own videos.
Q: How is user data protected? A: Passwords are hashed with SHA-256. Session-based authentication keeps you logged in. Each user can only access their own videos and notes.
Q: Can I use a different AI model? A: Yes, you can modify `note_generator.py` to use Claude, GPT-4, or local Ollama models. The Ollama API key is already configured.
Q: Does it work offline? A: Transcription works fully offline. Note generation requires internet for Ollama API, or you can run local Ollama for fully offline operation.
Q: What video formats are supported? A: Video: MP4, AVI, MOV, MKV, WebM, FLV, WMV | Audio: MP3, WAV, M4A
Q: What happens if processing fails? A: Failed videos show in History with a "Failed" status. For YouTube videos, you can click "Restart" to try again. You can also delete failed entries.
Q: How much storage does it use? A: Check your Profile or homepage for storage statistics. Approximately 100-500MB per hour of video for audio files, plus database entries for notes and transcripts.
Q: What's the best setup for an Oracle VM with 24GB RAM? A: Use the `large` Whisper model for best accuracy. You can easily process 6+ hour videos, handle multiple videos simultaneously, and optionally run local Ollama models for fully offline operation.
- YouTube URL processing
- Local file upload
- Audio transcription (whisper.cpp)
- AI note generation (Ollama)
- Web interface
- Processing history
- User authentication system
- Background processing with threading
- Real-time progress tracking (polling-based)
- Video management (cancel/restart/delete)
- Storage statistics
- Minimalistic UI design
- Granular progress updates
- Cooperative cancellation system
- Real-time progress with WebSockets (upgrade from polling)
- Note editing interface
- Multiple export formats (PDF, DOCX)
- Batch processing (multiple videos at once)
- Custom note templates
- Advanced search functionality
- Email notifications on completion
- Chapter detection from video/description
- Speaker identification (diarization)
- Multi-language support (auto-detect)
- Video timestamp linking in notes
- Integration with note-taking apps (Notion, Obsidian, Evernote)
- Mobile app (iOS/Android)
- Browser extension for quick captures
- Collaborative notes (sharing with teams)
- API rate limiting and quotas
- Admin dashboard
Contributions are welcome! Please feel free to submit a Pull Request.
MIT License - See LICENSE file for details
- whisper.cpp: https://github.com/ggml-org/whisper.cpp
- yt-dlp: https://github.com/yt-dlp/yt-dlp
- Ollama: https://ollama.com
- Flask: https://flask.palletsprojects.com
For issues and questions:
- Check the troubleshooting section above
- Review existing issues in the issue tracker
- Create a new issue with details about your problem
Happy Note-Taking! 📝