Skip to content

Commit be5e7f6

Browse files
feat: teach numpy view/copy on the canvas (curriculum phase 1)
Add the numpy series. An ndarray is a window over a separate buffer box, so .copy() mints a new buffer, a slice is a view that shares the buffer, a + 1 vectorizes into a new buffer, and .sum() folds the buffer to a scalar. New isNdarray / sharesBuffer / independentBuffer predicates and a 5-level series. Also, as part of this increment: - split the level catalog into per-series modules under src/levels/ - tag every level with an ELI band and make the level tests invariant-based - add the RAM-view data model (addr / byteSize / refCount / seg) to heap objects
1 parent ec823d8 commit be5e7f6

21 files changed

Lines changed: 1483 additions & 808 deletions

‎README.md‎

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,7 @@ Type Python-subset statements. Meta-commands:
4141
| Command | Effect |
4242
|---------|--------|
4343
| `help` | subset + meta overview |
44-
| `levels` | list the 35 levels / 9 series |
44+
| `levels` | list the 40 levels / 10 series |
4545
| `level <id>` | load a level (e.g. `level first-name`) |
4646
| `lesson` | open the current level's lesson text |
4747
| `goal` / `guide` | focus the right-hand guide panel |
@@ -63,22 +63,23 @@ A deliberately small, pedagogically sharp subset — not CPython:
6363
- **Methods:** `.append` `.insert` `.extend` `.pop` `.remove` `.copy` `.get` `.update` `.keys` `.clear`
6464
- **Functions:** `def` / call / `return`, local frames, nested calls
6565
- **Builtins:** `print` `len` `type` `id` `str` `int` `bool` `list` `dict` `set`
66-
- **Modules:** `import copy as cp` — `cp.copy` (shallow) and `cp.deepcopy`; `import collections as c` — `c.defaultdict`; `import functools as f` — `f.reduce`
66+
- **Modules:** `import copy as cp` — `cp.copy` (shallow) and `cp.deepcopy`; `import collections as c` — `c.defaultdict`; `import functools as f` — `f.reduce`; `import numpy as np` — `np.array`, `.copy()`, slicing (views), `+`, `.sum()`
6767
- **Iterators:** `iter(x)` and `next(it)` — a stateful iterator object you advance by hand
6868
- **Functional:** `map(fn, it)` / `filter(pred, it)` — lazy iterator objects; `f.reduce(fn, it)` — an eager fold
69+
- **NumPy:** `np.array` → an ndarray *window* over a separate **buffer** box; `.copy()` mints a new buffer; a slice is a *view* sharing the same buffer; `a + 1` vectorizes into a new buffer; `a.sum()` folds to a scalar
6970
- **Plotting:** `scatter(xs, ys, "title")` `line(xs, ys, "title")`
7071
- **Operators:** arithmetic, comparisons, `is`, `==`
7172

72-
Not in v0: classes, real module imports (only the simulated `copy`, `collections`, `functools`), control flow (`if`/`for`/`while`), exceptions (beyond clean parse/runtime errors), comprehensions, I/O, async, `range`/`sorted`/`min`/`max`, full CPython semantics.
73+
Not in v0: classes, real module imports (only the simulated `copy`, `collections`, `functools`, `numpy`), control flow (`if`/`for`/`while`), exceptions (beyond clean parse/runtime errors), comprehensions, I/O, async, `range`/`sorted`/`min`/`max`, full CPython semantics.
7374

74-
A planned curriculum extends this with numpy, pandas, re, datetime and more — all drawn on the same memory canvas. See `docs/ENGINE-EXTENSION.md` and `docs/adr/002-library-modules.md`.
75+
A planned curriculum extends this with pandas, re, datetime and more — all drawn on the same memory canvas. See `docs/ENGINE-EXTENSION.md` and `docs/adr/002-library-modules.md`.
7576

7677
## Architecture
7778

7879
```
7980
src/engine/ simulated runtime (tokenizer, parser, evaluator, memory, session)
8081
+ goal checks (checks.ts) — the "TreeCompare" analogue
81-
src/levels/ 9-series / 35-level catalog + per-level goals, par, solutions
82+
src/levels/ 10-series / 40-level catalog + per-level goals, par, solutions
8283
src/ui/ app shell, memory canvas, plot board, REPL, dialog, script editor
8384
src/style.css design tokens + layout
8485
tests/ vitest unit + level-solution tests
@@ -90,7 +91,7 @@ The engine is intentionally a **teaching subset**, not CPython. Every state tran
9091

9192
## Levels
9293

93-
Nine series — Names & Bindings, Aliasing & Copy, Containers in Memory, **Copy Semantics** (shallow vs deep), **Iterators & Iterables**, **Defaults & Functional** (defaultdict, map, filter), Mutation & Methods, Functions & Frames, Plotting & Capstone. Each of the 35 levels has a brief, optional hint, a goal (predicate or target snapshot), a statement-par for golf, and a worked solution.
94+
Ten series — Names & Bindings, Aliasing & Copy, Containers in Memory, **Copy Semantics** (shallow vs deep), **Iterators & Iterables**, **Defaults & Functional** (defaultdict, map, filter), **NumPy: Buffers & Views** (ndarray, buffer, `.copy()`, slice-as-view), Mutation & Methods, Functions & Frames, Plotting & Capstone. Each of the 40 levels has a brief, optional hint, a goal (predicate or target snapshot), a statement-par for golf, and a worked solution.
9495

9596
## Tech
9697

‎docs/ENGINE-EXTENSION.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -165,7 +165,7 @@ Every library `L` is built by filling this contract:
165165
| Library | `type` | `value` shape | Canvas box |
166166
|---------|--------|---------------|------------|
167167
| `copy` | *(none — reuses list/dict)* | — | *(existing)* |
168-
| `numpy` | `ndarray` | `{ buffer: id, shape, dtype, view?: bool }` | ndarray box → buffer box (grid) |
168+
| `numpy` | `ndarray` / `buffer` | ndarray `{ buffer: id, offset, shape, dtype, view }` → buffer `{ data: number[], dtype }` | ndarray box → buffer box; a slice shares the buffer (view), `.copy()` mints a new one |
169169
| `pandas` | `Series` / `DataFrame` | `{ buffer: id, index, columns?, dtype }` | box → buffer; DF shows column headers |
170170
| `beautifulsoup` | `DomNode` | `{ tag, children: id[] }` | **heap tree** (parent→child edges) |
171171
| `re` | `Pattern` / `Match` | `{ source }` / `{ span, groups: id[] }` | box + group chips |

‎src/engine/checks.ts‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -233,6 +233,38 @@ export const preds = {
233233
};
234234
},
235235

236+
/** `name` is a numpy ndarray. */
237+
isNdarray(name: string) {
238+
return (snapshot: Snapshot, _runtime: MemoryState): boolean | string => {
239+
const view = slotView(name, snapshot);
240+
return view?.type === 'ndarray' ? true : `'${name}' is not an ndarray.`;
241+
};
242+
},
243+
244+
/**
245+
* Two ndarrays point at the SAME buffer (a view shares its source's buffer).
246+
*/
247+
sharesBuffer(name: string, other: string) {
248+
return (snapshot: Snapshot, _runtime: MemoryState): boolean | string => {
249+
const a = ndarrayBufferId(name, snapshot);
250+
const b = ndarrayBufferId(other, snapshot);
251+
if (a === null || b === null) return `Bind both '${name}' and '${other}' as ndarrays.`;
252+
return a === b;
253+
};
254+
},
255+
256+
/**
257+
* Two ndarrays point at DIFFERENT buffers (an independent copy).
258+
*/
259+
independentBuffer(name: string, other: string) {
260+
return (snapshot: Snapshot, _runtime: MemoryState): boolean | string => {
261+
const a = ndarrayBufferId(name, snapshot);
262+
const b = ndarrayBufferId(other, snapshot);
263+
if (a === null || b === null) return `Bind both '${name}' and '${other}' as ndarrays.`;
264+
return a !== b;
265+
};
266+
},
267+
236268
/**
237269
* A plot has been drawn (optionally of a specific kind).
238270
*/
@@ -303,6 +335,18 @@ export const preds = {
303335
},
304336
};
305337

338+
/** Read an ndarray's normalized `buffer` id from the snapshot, or null. */
339+
function ndarrayBufferId(name: string, snapshot: Snapshot): string | null {
340+
const module = snapshot.frames[snapshot.frames.length - 1];
341+
if (!module) return null;
342+
const id = module.locals[name];
343+
if (typeof id !== 'string' || !snapshot.heap[id]) return null;
344+
const obj = snapshot.heap[id];
345+
if (obj.type !== 'ndarray') return null;
346+
const v = obj.value as { buffer?: unknown };
347+
return typeof v.buffer === 'string' ? v.buffer : null;
348+
}
349+
306350
/** Read an iterator's `{ source, index }` from the normalized snapshot. */
307351
function iteratorValue(name: string, snapshot: Snapshot): { source: string; index: number } | null {
308352
const module = snapshot.frames[snapshot.frames.length - 1];

‎src/engine/evaluator.ts‎

Lines changed: 177 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
* Mutates a memory state (frames + heap) and records stdout / errors.
55
*/
66

7-
import { allocObject, cloneState, createModuleState } from './memory';
7+
import { allocObject, cloneState, computeRefCounts, createModuleState } from './memory';
88
import { parse } from './parser';
99
import type {
1010
AstNode,
@@ -30,6 +30,8 @@ export function run(source: string, state: MemoryState): MemoryState {
3030
// leaving a parent container's cached repr stale. Recompute container
3131
// reprs so display, print, and goal checks all see the current value.
3232
refreshReprs(state);
33+
// Recompute the refcount column for the RAM view from the now-final graph.
34+
computeRefCounts(state);
3335
} catch (err) {
3436
const message = err instanceof Error ? err.message : String(err);
3537
state.errors.push(message);
@@ -47,6 +49,21 @@ function refreshReprs(state: MemoryState): void {
4749
if (obj.type === 'list' || obj.type === 'dict' || obj.type === 'tuple' || obj.type === 'set' || obj.type === 'defaultdict') {
4850
obj.repr = formatObject(obj, state);
4951
}
52+
if (obj.type === 'ndarray') {
53+
obj.repr = numpyRepr(ndarrayWindow(obj.id, state));
54+
}
55+
}
56+
}
57+
58+
/**
59+
* Recompute the cached repr of every ndarray window AND its buffer after a
60+
* buffer write, so a mutation through one window is visible through every
61+
* other window (the whole point of a view).
62+
*/
63+
function refreshNdarrayReprs(state: MemoryState): void {
64+
for (const obj of Object.values(state.heap)) {
65+
if (obj.type === 'ndarray') obj.repr = numpyRepr(ndarrayWindow(obj.id, state));
66+
if (obj.type === 'buffer') obj.repr = `[${(obj.value as NumpyBufferValue).data.join(', ')}]`;
5067
}
5168
}
5269

@@ -162,6 +179,17 @@ function bindTarget(target: AstNode, value: RtValue, state: MemoryState): void {
162179
obj.repr = formatObject(obj, state);
163180
return;
164181
}
182+
if (obj.type === 'ndarray') {
183+
const v = obj.value as NdarrayValue;
184+
const n = Number(value);
185+
if (Number.isNaN(n)) throw new Error(`TypeError: cannot assign non-number to ndarray (line 0)`);
186+
const i = index as number;
187+
const data = (state.heap[v.buffer].value as NumpyBufferValue).data;
188+
data[v.offset + i] = n;
189+
// A buffer write is seen by every ndarray window over it — refresh them all.
190+
refreshNdarrayReprs(state);
191+
return;
192+
}
165193
if (obj.type === 'dict' || obj.type === 'defaultdict') {
166194
const keyRepr = dictKeyRepr(index, state);
167195
const entries = dictEntries(obj);
@@ -444,6 +472,17 @@ function callMethod(objId: RtValue, method: string, args: RtValue[], state: Memo
444472
throw new Error(`AttributeError: module 'copy' has no attribute '${method}' (line ${line})`);
445473
}
446474

475+
if (obj.type === 'module' && (obj.value as { name: string }).name === 'numpy') {
476+
if (method === 'array') return makeNdarray(args, state, line);
477+
throw new Error(`AttributeError: module 'numpy' has no attribute '${method}' (line ${line})`);
478+
}
479+
480+
if (obj.type === 'ndarray') {
481+
if (method === 'copy') return copyNdarray(objId as string, state, line);
482+
if (method === 'sum') return ndarraySum(objId as string, state, line);
483+
throw new Error(`AttributeError: 'ndarray' object has no attribute '${method}' (line ${line})`);
484+
}
485+
447486
if (obj.type === 'list') {
448487
const value = obj.value as RtValue[];
449488
if (method === 'append') {
@@ -849,6 +888,123 @@ function reduce(args: RtValue[], state: MemoryState, line: number): RtValue {
849888
}
850889
}
851890

891+
/**
892+
* A numpy buffer's payload: a flat run of numbers. Rendered as a separate
893+
* box on the canvas so a view-vs-copy distinction becomes visible — the
894+
* ndarray points at a buffer, and a copy mints a *second* buffer.
895+
*/
896+
interface NumpyBufferValue {
897+
data: number[];
898+
dtype: string;
899+
}
900+
901+
/**
902+
* An ndarray's payload: a *window* onto a buffer. `buffer` is the id of the
903+
* heap buffer box; `offset` + `shape` carve the view's slice out of that
904+
* buffer's flat run. `view` is true when this window is not the buffer's full
905+
* extent (i.e. it is a slice) and false when it owns its own full buffer.
906+
*/
907+
interface NdarrayValue {
908+
buffer: string;
909+
offset: number;
910+
shape: number;
911+
dtype: string;
912+
view: boolean;
913+
}
914+
915+
/** numpy-style repr of a number run: `[1. 2. 3.]` (float-ish, trailing dot). */
916+
function numpyRepr(data: number[]): string {
917+
const body = data.map((n) => (Number.isInteger(n) ? `${n}.` : String(n)));
918+
return `[${body.join(' ')}]`;
919+
}
920+
921+
/** The numeric window an ndarray currently shows (offset+shape over buffer). */
922+
function ndarrayWindow(ndId: string, state: MemoryState): number[] {
923+
const v = state.heap[ndId].value as NdarrayValue;
924+
const data = (state.heap[v.buffer].value as NumpyBufferValue).data;
925+
return data.slice(v.offset, v.offset + v.shape);
926+
}
927+
928+
/** Allocate a buffer object holding a flat numeric run. */
929+
function makeBuffer(data: number[], dtype: string, state: MemoryState): string {
930+
const obj = allocObject('buffer', { data, dtype } as NumpyBufferValue, `[${data.join(', ')}]`);
931+
state.heap[obj.id] = obj;
932+
return obj.id;
933+
}
934+
935+
/**
936+
* `np.array([...])` → a fresh buffer + an ndarray that owns its full buffer.
937+
*/
938+
function makeNdarray(args: RtValue[], state: MemoryState, line: number): RtValue {
939+
if (args.length !== 1) throw new Error(`TypeError: array() takes exactly one argument (line ${line})`);
940+
const data = readNumberSeries(args[0], state);
941+
const bufId = makeBuffer(data, 'float64', state);
942+
const obj = allocObject('ndarray', { buffer: bufId, offset: 0, shape: data.length, dtype: 'float64', view: false } as NdarrayValue, '');
943+
state.heap[obj.id] = obj;
944+
obj.repr = numpyRepr(data);
945+
return obj.id;
946+
}
947+
948+
/**
949+
* `a.copy()` → a new buffer holding the array's current window. The new
950+
* buffer is fully independent: mutating one is invisible to the other.
951+
*/
952+
function copyNdarray(ndId: string, state: MemoryState, line: number): RtValue {
953+
const nd = state.heap[ndId];
954+
if (!nd || nd.type !== 'ndarray') throw new Error(`TypeError: ${typeName(ndId, state)} has no .copy() (line ${line})`);
955+
const data = ndarrayWindow(ndId, state);
956+
const bufId = makeBuffer([...data], (nd.value as NdarrayValue).dtype, state);
957+
const obj = allocObject('ndarray', { buffer: bufId, offset: 0, shape: data.length, dtype: 'float64', view: false } as NdarrayValue, '');
958+
state.heap[obj.id] = obj;
959+
obj.repr = numpyRepr(data);
960+
return obj.id;
961+
}
962+
963+
/**
964+
* `a[start:stop]` → a *view*: a new ndarray window onto the SAME buffer. This
965+
* is the flagship identity idea — slicing does not copy the data, it makes a
966+
* second window onto it. Writing through the view edits the shared buffer.
967+
*/
968+
function ndarraySlice(ndId: string, start: RtValue, stop: RtValue, state: MemoryState, line: number): RtValue {
969+
const nd = state.heap[ndId];
970+
if (!nd || nd.type !== 'ndarray') throw new Error(`TypeError: ${typeName(ndId, state)} is not sliceable (line ${line})`);
971+
const v = nd.value as NdarrayValue;
972+
const full = (state.heap[v.buffer].value as NumpyBufferValue).data;
973+
const s = start === null ? 0 : Number(start);
974+
const e = stop === null ? v.shape : Number(stop);
975+
const offset = v.offset + s;
976+
const shape = Math.max(0, e - s);
977+
const obj = allocObject('ndarray', { buffer: v.buffer, offset, shape, dtype: v.dtype, view: true } as NdarrayValue, '');
978+
state.heap[obj.id] = obj;
979+
obj.repr = numpyRepr(full.slice(offset, offset + shape));
980+
void line;
981+
return obj.id;
982+
}
983+
984+
/**
985+
* `a + scalar` (vectorized) → a brand-new buffer holding the element-wise
986+
* result. The operand is read; a new buffer is minted for the result.
987+
*/
988+
function vecAdd(left: RtValue, right: RtValue, state: MemoryState, line: number): RtValue {
989+
const a = state.heap[left as string];
990+
if (!a || a.type !== 'ndarray') throw new Error(`TypeError: unsupported operand (line ${line})`);
991+
const b = Number(right);
992+
if (Number.isNaN(b)) throw new Error(`TypeError: cannot add ${typeName(right, state)} to ndarray (line ${line})`);
993+
const data = ndarrayWindow(left as string, state).map((n) => n + b);
994+
const bufId = makeBuffer(data, 'float64', state);
995+
const obj = allocObject('ndarray', { buffer: bufId, offset: 0, shape: data.length, dtype: 'float64', view: false } as NdarrayValue, '');
996+
state.heap[obj.id] = obj;
997+
obj.repr = numpyRepr(data);
998+
return obj.id;
999+
}
1000+
1001+
/** `a.sum()` → fold the array's window into a scalar. */
1002+
function ndarraySum(ndId: string, state: MemoryState, line: number): RtValue {
1003+
const nd = state.heap[ndId];
1004+
if (!nd || nd.type !== 'ndarray') throw new Error(`TypeError: ${typeName(ndId, state)} has no .sum() (line ${line})`);
1005+
return ndarrayWindow(ndId, state).reduce((acc, n) => acc + n, 0);
1006+
}
1007+
8521008
/**
8531009
* Python-style string repr using single quotes.
8541010
*/
@@ -917,6 +1073,7 @@ function lengthOf(v: RtValue, state: MemoryState): number {
9171073
const obj = state.heap[v];
9181074
if (['list', 'tuple', 'set', 'dict'].includes(obj.type)) return (obj.value as unknown[]).length;
9191075
if (obj.type === 'defaultdict') return dictEntries(obj).length;
1076+
if (obj.type === 'ndarray') return (obj.value as NdarrayValue).shape;
9201077
if (obj.type === 'str') return (obj.value as string).length;
9211078
}
9221079
if (typeof v === 'string') return v.length;
@@ -963,6 +1120,15 @@ function binOp(op: string, left: RtValue, right: RtValue, state: MemoryState, li
9631120
// string concatenation
9641121
const lt = typeName(left, state);
9651122
const rt = typeName(right, state);
1123+
1124+
// vectorized ndarray + scalar → a fresh buffer
1125+
if (op === '+' && (lt === 'ndarray' || rt === 'ndarray')) {
1126+
const scalar = lt === 'ndarray' ? right : left;
1127+
const arr = lt === 'ndarray' ? left : right;
1128+
if (typeof scalar !== 'number') throw new Error(`TypeError: unsupported operand for + (line ${line})`);
1129+
return vecAdd(arr, scalar, state, line);
1130+
}
1131+
9661132
if (op === '+' && lt === 'str' && rt === 'str') {
9671133
const lv = state.heap[left as string].value as string;
9681134
const rv = state.heap[right as string].value as string;
@@ -1049,6 +1215,13 @@ function subscriptGet(objId: RtValue, index: RtValue, state: MemoryState, line:
10491215
}
10501216
return value[idx];
10511217
}
1218+
if (obj.type === 'ndarray') {
1219+
const v = obj.value as NdarrayValue;
1220+
const i = Number(index);
1221+
const idx = i < 0 ? v.shape + i : i;
1222+
if (idx < 0 || idx >= v.shape) throw new Error(`IndexError: index out of range (line ${line})`);
1223+
return (state.heap[v.buffer].value as NumpyBufferValue).data[v.offset + idx]!;
1224+
}
10521225
if (obj.type === 'dict' || obj.type === 'defaultdict') {
10531226
const keyRepr = dictKeyRepr(index, state);
10541227
const entries = dictEntries(obj);
@@ -1086,6 +1259,9 @@ function sliceGet(objId: RtValue, start: RtValue, stop: RtValue, state: MemorySt
10861259
out.repr = formatObject(out, state);
10871260
return out.id;
10881261
}
1262+
if (obj.type === 'ndarray') {
1263+
return ndarraySlice(objId as string, start, stop, state, line);
1264+
}
10891265
if (obj.type === 'str') {
10901266
return allocString((obj.value as string).slice(s, e), state);
10911267
}

0 commit comments

Comments
 (0)