File tree Expand file tree Collapse file tree
Expand file tree Collapse file tree Original file line number Diff line number Diff line change @@ -100,10 +100,12 @@ python -m meow.inference \
100100### Training Data
101101
10210220,000 synthetic (input, output) samples across 15 categories.
103- See the [ dataset card] ( dataset_card.md ) for full details.
103+ See the [ dataset card] ( https://huggingface.co/datasets/hunt3rx99/meowllm-miso ) for full details.
104104
105105### Training Procedure
106106
107+ The recommended training regime (for a fresh checkpoint):
108+
107109- ** Optimizer** : AdamW (β₁=0.9, β₂=0.95, weight_decay=0.1)
108110- ** Learning rate** : 3e-4 peak, linear warmup + cosine decay
109111- ** Warmup** : 200 steps
@@ -115,6 +117,10 @@ See the [dataset card](dataset_card.md) for full details.
115117- ** Hardware** : single T4 GPU (free Colab tier sufficient)
116118- ** Wall-clock time** : ~ 20 minutes
117119
120+ > ** Note:** The bundled ` best.pt ` was trained on CPU for 2000 steps
121+ > (batch size 32, ~ 4 minutes). See the Measured numbers section below
122+ > for its real eval results. A full GPU run will produce higher numbers.
123+
118124### Training hyperparameters (MeowConfig)
119125
120126``` python
You can’t perform that action at this time.
0 commit comments