|
1 | 1 | # memory.md — AtlasAI (active state) |
2 | 2 |
|
3 | | -**Updated:** 2026-07-20 · **Milestone:** Week 2 done + real backend added → Week 3 next |
| 3 | +**Updated:** 2026-07-21 · **Milestone:** Week 3 (Docker Platform) in progress — CPU path built, GPU path written (user-verified) |
4 | 4 |
|
5 | 5 | ## Progress |
6 | 6 | - Wk1 API Gateway ✅ — auth, validation, JSON logging, rate limiting; 17 tests. |
7 | 7 | - Wk2 Inference Engine ✅ — pluggable backends, BatchManager, SSE streaming, |
8 | 8 | engine-gated `/ready`; gateway services async over engine. |
| 9 | +- Wk3 Docker Platform 🚧 — `deployment/docker/`: two separate Dockerfiles |
| 10 | + (`Dockerfile.cpu` mock+hf, multi-stage slim non-root venv; `Dockerfile.gpu` |
| 11 | + CUDA 12.1 + py3.12/deadsnakes + vllm) and `docker-compose.yml` (CPU api |
| 12 | + service + hf-cache volume + healthcheck; commented Postgres/Redis/MLflow |
| 13 | + stubs). Root `.dockerignore` trims context to source. Images COPY only |
| 14 | + api/+shared/+inference/ (verified: no imports of future-week packages); |
| 15 | + HEALTHCHECK curls `/health`. Chosen scope: two Dockerfiles + API-only compose |
| 16 | + with commented stubs. GPU build NOT run in Claude env — user verifies on GPU. |
| 17 | + CPU+hf image verified running in container by user. Volume-perms fix: image |
| 18 | + now `mkdir -p` + `chown atlas` on `/home/atlas/.cache` so the named hf-cache |
| 19 | + volume initializes writable for the non-root user (was root-owned → Errno 13). |
| 20 | +- Wk3 CI ✅ — `.github/workflows/ci.yml`: job `lint-test` (ruff + pytest, py3.12, |
| 21 | + mock backend) → job `docker-cpu` (build CPU image `INCLUDE_HF=0` for speed, |
| 22 | + run container, smoke-test `/health`=ok and `/ready`=mock). First feature run |
| 23 | + through the pipeline: **Stop button** on `/tester` — AbortController cancels |
| 24 | + the in-flight Generate/Stream fetch (`stopRun()`, buttons enable/disable, |
| 25 | + AbortError → "stopped"). New test `test_tester_page_has_stop_control` (→ 40 |
| 26 | + tests). Verified: tester.py compiles + Stop tokens present (mock check). |
9 | 27 | - Real backend ✅ — added **HFBackend** (`inference/backends/transformers_backend.py`): |
10 | 28 | real transformers model on CPU/MPS/CUDA, async (to_thread), streaming via |
11 | 29 | TextIteratorStreamer, lazy torch import. Backends now: `mock` | `hf` | `vllm`, |
|
18 | 36 | - GPU server: `requirements-inference.txt` → `ATLAS_INFERENCE_BACKEND=vllm uvicorn api.main:app --host 0.0.0.0` |
19 | 37 | - Browser test: `/docs` (Swagger, Authorize `dev-key-local`) OR `/tester` (custom |
20 | 38 | page, supports Generate + live Stream). |
| 39 | +- Docker (CPU): from repo root `docker build -f deployment/docker/Dockerfile.cpu |
| 40 | + -t atlasai:cpu .` then `docker run --rm -p 8000:8000 atlasai:cpu` (mock). Or |
| 41 | + `cd deployment/docker && docker compose up --build` (add |
| 42 | + `ATLAS_INFERENCE_BACKEND=hf` for real gen). GPU: `Dockerfile.gpu` on a CUDA |
| 43 | + host with `--gpus all` (user-verified only). |
21 | 44 |
|
22 | 45 | ## Notes / fixes |
23 | 46 | - requirements-hf.txt uses version FLOORS (torch>=2.2.0 …): macOS-x86_64 has no |
|
44 | 67 | excluded from OpenAPI + rate limit). 39 tests pass. |
45 | 68 |
|
46 | 69 | ## Right now |
47 | | -Real-backend workflow implemented + documented. Next: user browser-tests `hf` |
48 | | -locally. Then resume **Week 3 — Docker Platform** (image strategy/compose still TBD). |
| 70 | +Week 3 Docker Platform + CI done. CPU+hf image runs in container; CI workflow |
| 71 | +lints, tests, builds the CPU image, and smoke-tests the running container. Stop |
| 72 | +button shipped as the first change through the pipeline. Next: user pushes so CI |
| 73 | +runs green on GitHub, runs `pytest` locally (expect 40), and validates the GPU |
| 74 | +image on CUDA hardware. Then checkpoint Wk3 (tag `v0.3.0-week3`). |
49 | 75 |
|
50 | 76 | ## Open follow-ups |
51 | 77 | - First `hf` request downloads model (~1GB) → cached; `/ready` 503 until loaded. |
52 | 78 | - HF/vLLM real runs not testable in Claude's env (no torch/GPU). |
53 | 79 | - Later: dedicated embedding backend; LoRA via `GenerationRequest.adapter` (Wk5). |
54 | 80 |
|
55 | 81 | ## Immediate next steps |
56 | | -1. Checkpoint Wk2: branch `week-2-inference-engine` → PR → tag `v0.2.0-week2`. |
57 | | -2. Start **Week 3 — Docker Platform**: Dockerfile(s) + docker-compose, reproducible |
58 | | - builds; likely split CPU (mock) vs CUDA (vllm) images. Confirm scope first. |
| 82 | +1. User builds/runs CPU image (`docker compose up --build`), verifies `/health`, |
| 83 | + `/ready`, `/tester`; then verifies GPU image (`Dockerfile.gpu`) on CUDA host. |
| 84 | +2. Checkpoint Wk3: conventional commit `feat(docker): ...`, tag `v0.3.0-week3`. |
| 85 | +3. Ask before advancing to Week 4. |
59 | 86 |
|
60 | 87 | ## Open follow-ups |
61 | 88 | - Cleanup pending: remove superseded Wk1 files (deletion was declined earlier): |
|
0 commit comments