🤖 Offline LLM Chat + 👾 AI Overlord

Run a real LLM 100% inside your browser — free, private, and fully offline after the first load. No server, no API key, no install. Includes a chat app and an arcade game where the AI is the villain.

Built with WebLLM (WebGPU) and wllama (llama.cpp → WASM, CPU) — automatic fallback, so it works on any modern browser, with or without a GPU.

💬 Chat	👾 AI Overlord

✨ What's inside

Page	What it does
index.html	💬 Minimal chat UI with streaming replies
game.html	👾 AI Overlord — a canvas space shooter where the LLM designs every enemy wave live (count, speed, attack pattern, taunt) and gloats when you die
engine.js	Shared loader: probes for a real WebGPU adapter → WebLLM; otherwise llama.cpp WASM on CPU
sw.js	Service worker so the app shell works offline (models are cached by the engines themselves)

🧠 Models (free, Apache 2.0)

Model	Size (Q4)	Best for
SmolLM2-360M-Instruct	~260MB	Fast replies, low-end machines
Qwen2.5-1.5B-Instruct	~1GB	Noticeably smarter chat

Weights download once from Hugging Face, then live in browser storage (Cache API / IndexedDB) — turn off wifi and everything keeps working.

🚀 Quick start

git clone https://github.com/oyekamal/offline-llm-chat.git
cd offline-llm-chat
python3 -m http.server 8787   # any static server works (service workers need localhost or https)

Open http://localhost:8787 → pick a model → Load model → chat. Or open http://localhost:8787/game.html → Load AI → Start game.

🎮 How the game uses the LLM (design notes)

Small models are slow (seconds per reply on CPU) and unreliable (they ignore "reply with JSON only" half the time). The game is designed around that:

• JS owns all game state — the model can never break a round
• The next wave is generated while you fight the current one — latency is hidden, you never wait
• Lenient parsing — strict JSON first, then regex field-scraping, then clamped defaults ("backup wave") if the model rambled
• The model contributes creativity (wave design, taunts, game-over gloats), not correctness

This pattern — LLM as content director, JS as referee — is the useful takeaway for building on top of tiny in-browser models.

🖥️ Compatibility

• With GPU (WebGPU): Chrome/Edge 113+, recent Firefox/Safari → fast (WebLLM, ~80% native speed)
• Without GPU: any browser with WASM SIMD → slower but works (wllama, CPU). Force it with ?cpu in the URL
• Keep models ≤2GB: Firefox caps per-site storage at 2GB

🛠️ Ideas to build on top

• Chat history persistence (localStorage)
• More game modes (LLM-generated boss fights, power-up descriptions)
• Voice input via Web Speech API
• PWA install prompt + proper icons
• RAG over local files with transformers.js embeddings

📄 License

MIT — do whatever you want.

---

#webllm #wllama #llamacpp #webgpu #wasm #offline-ai #local-llm #browser-llm #smollm #qwen #no-server #privacy #ai-game