Smallest AI Speed Test

An AI agent built this phonics-teaching AI from an empty folder in 8 minutes 50 seconds, using two tutorial repos as its only instructions. This repo is the proof: the code, the tests, the trained baseline models, and the timestamped build log.

The experiment (2026-08-20): give Claude Code (model Fable 5) an empty folder and one prompt: read smallest-ai-built-from-the-ground-up and smallest-ai-tutorial, write a spec, then build an AI model that teaches phonics. Pure Python, zero dependencies, backpropagation written by hand.

Full timestamped timeline: BUILD_LOG.md

Phase	Time
Reading both tutorial repos	~4.5 min
Writing SPEC.md + all code + 15 tests	~4 min
Training both models	16.6 s
Debugging	0 (gradient checks passed on first run)
Empty folder to trained, verified baseline	8 min 50 s

It's not just frontier models: a local 8B model built it too

Same day, same experiment, run with Qwen3 8B running fully on local hardware at ~25 tokens/second with reasoning disabled (model served from an NVIDIA DGX Spark on the local network; the build and its training ran on a Mac). It took about 2 hours instead of 9 minutes, and it got there: 26/26 letters, 97.2% held-out phoneme accuracy, 29/29 of its own tests passing, including a per-parameter BPTT gradient check. Its first training run collapsed (lr too high); it diagnosed the failure, fixed it, and documented the decision. The entire loop, from agent to trained model, never touched the cloud.

Its complete build, verbatim, plus independent verification of every claim: results/qwen3-8b-local/RESULTS.md.

Run	Agent	Prompt (PROMPT.md)	Observable build time	Letter model	Blender held-out
Baseline	Claude Fable 5 (cloud)	1: discovery	8 min 50 s	26/26	100% (36/36 phonemes)
Local	Qwen3 8B, ~25 tok/s, reasoning off	2: codified recipe	~2 h 06 min	26/26	97.2% (35/36 phonemes)

Comparability caveats (different prompts, different held-out word lists, single runs): see results/qwen3-8b-local/RESULTS.md.

The baseline models (included in models/)

Model	Task	Params	Train time	Measured accuracy
letter_sound.json (MLP 26-32-24)	letter to primary phoneme	1,656	1.2 s	26/26 (100%)
blender.json (LSTM, 32 hidden)	CVC word to phoneme sequence	8,245	15.4 s	100% phoneme + word, train (48 words) AND held-out (12 words)

Configs, seed, split, and full eval numbers: models/training_summary.json. Design spec written before the build: SPEC.md.

Honest caveat: in this dataset each letter's phoneme mapping is deterministic and every letter appears in training, so the held-out result proves generalization to unseen letter combinations, not unseen mappings. Small domain, single seeded run. This is a baseline, not a product.

Try it

No dependencies at runtime (pytest only for the tests). Python 3.10+.

python3 tutor.py sound-out cat ship make rain     # sound out any word
python3 tutor.py lesson                           # printed phonics lesson
python3 tutor.py quiz 5                           # interactive letter-sound quiz
python3 train.py                                  # retrain from scratch (~20 s)
python3 -m pytest tests/ -q                       # 15 tests incl. gradient checks

Example output:

     cat  c->/k/   a->/æ/   t->/t/
          blend it: /kæt/   [neural-lstm]
    make  m->/m/   a->/eɪ/   k->/k/   e->(silent)
          blend it: /meɪk/   [rules]

How it works

word -> WordReader (router)
          |-- digraph / silent-e / vowel team?  -> PhonicsRuleEngine (symbolic)
          |-- 3-letter CVC shape?               -> PhonicsBlender LSTM (neural)
          '-- anything else                     -> rule-engine letter fallback

• phonics_ai/nn.py: vectors, matrices, softmax, cross-entropy, MLP with hand-derived backpropagation.
• phonics_ai/lstm.py: LSTM cell (input/forget/output gates + candidate), per-timestep softmax head, full backpropagation through time, gradient clipping.
• phonics_ai/rules.py: deterministic phonics rules loaded from the data files.
• phonics_ai/models.py: the two trainable models plus the neuro-symbolic router; JSON (not pickle) serialization.

Why should you believe the gradients are right?

tests/ contains central-difference numerical gradient checks that verify every hand-derived gradient of a small MLP and a small LSTM, parameter by parameter, against (L(w+eps) - L(w-eps)) / 2eps. Anyone who has written BPTT by hand knows this is where the hours usually go. Run them yourself:

python3 -m pytest tests/test_nn.py tests/test_lstm.py -v

Provenance

• Tutorials followed: smallest-ai-built-from-the-ground-up and smallest-ai-tutorial by Nathan Maine.
• Phonics data in data/phonics/ copied from the tutorial repo (MIT).
• Code written by a Claude Code agent (Claude Fable 5) following those tutorials; timed and directed by Nathan Maine.
• Environment: MacBook Pro M3 Max, CPython 3.12, macOS. Seed 42 throughout.

The point of the experiment is not that the agent was fast. It is that documentation written clearly enough for a human beginner turns out to be an executable specification for an AI agent. The months of thinking live in the tutorials; the 9 minutes were execution.

License

MIT. See LICENSE.