Skip to content

README.md

Latest commit

 

History

History
74 lines (58 loc) · 3.03 KB

README.md

File metadata and controls

74 lines (58 loc) · 3.03 KB
Raw

Model card: uniform-f12.mgw

uniform-f12.mgw is the fixed, tiny character-level model used by microgpt.sql. It generates name-like strings and exists to make the bit-exact inference gate reproducible.

Architecture

Field Value
Parameters 14,272
Layers 1
Embedding width 32
Attention heads 4
Head width 8
MLP width 128
Context length 8
Vocabulary 26 lowercase characters + BOS

All runtime activations and arithmetic use signed Q16.48 fixed point. “F12” describes the stored weight grid: each Q16.48 weight was uniformly rounded to 12 fractional bits, while remaining encoded as an int64 MGW value.

Provenance

  1. model-wide.mgw is the trained Q16.48 model from nmicic/int-llm at commit 0b4b6d04eb3e9e969d125804a154309eed3be9de. It was trained for 5,000 steps by the integer MicroGPT path on Karpathy's public makemore names dataset, SHA-256 0a30b5557f192f32ab962680889aac5f6fda0f4cecf40a6d0b5694f58ea8cc4d, distributed under the upstream MIT license.
  2. tools/mgw_precision.py, retained from nmicic/int-llm-precision-ladder revision 7cc4b9400185b9774a59c3228ab4c8af60ee770b, rounds every weight tensor uniformly to the F12 grid. Tokenizer bytes and RNG state are copied unchanged.
  3. The result is uniform-f12.mgw, which the SQL loader validates and reads.

The source model also has a published int-llm model card.

Checksums and reproduction

466cfe9dba7b888cdaa23dedf4b10351826795793448c8e95dcb0f7a61ed33eb  model-wide.mgw
742cbd6d0b750bf3d164a23d97390171e3fe545ee9d87a2b0e843d3d8d1ae9f4  uniform-f12.mgw

From the repository root:

make model

This rebuilds the F12 file in a temporary directory, verifies both SHA-256 pins, and compares the rebuilt file byte-for-byte with the committed artifact.

Intended use and limitations

This is a deterministic systems-demo and regression model, not a general language model or production name generator. It accepts no user prompt, has an eight-character context, and emits only lowercase ASCII name-like samples. Its output may reflect patterns or individual strings present in the training data. No quality, fairness, safety, or real-world suitability claim is made.

The SQL and C loaders are intended only for these committed, checksum-verified MGW files. They are not hardened parsers for arbitrary or adversarial model input; a substituted oversized or malformed file may exhaust resources or trigger unsafe failure paths in the reference tooling.

The model artifacts are distributed under the repository's Apache-2.0 license.