You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: AGENTS.md
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -109,6 +109,7 @@ stream-json/
109
109
-**Keep `.js` and `.d.ts` files in sync** for all modules under `src/`.
110
110
-**Token-based architecture.** The parser produces a stream of `{name, value}` tokens. All filters, streamers, and utilities operate on this token protocol.
111
111
-**Backpressure must be handled correctly.** All stream components rely on Node.js stream infrastructure via `stream-chain`.
112
+
-**Intended input is data the user owns or trusts** (dumps, exports, logs). The library is not designed for hostile input; docs say so, and code changes are not hardened against adversarial JSON beyond `JSON.parse` parity (`__proto__` becomes an own property) and the filters' `maxDepth` guard.
112
113
113
114
## Architecture
114
115
@@ -118,6 +119,7 @@ stream-json/
118
119
-**Assembler** (`src/assembler.js`, implementation in `src/core/assembler.js`) interprets the token stream and reconstructs JavaScript objects. Plain class — no `EventEmitter` inheritance in 3.x.
119
120
- Used internally by all streamers via `streamBase`.
120
121
- Reads only packed tokens (`keyValue`, `stringValue`, `numberValue`); streamed chunks are ignored.
122
+
- Materializes like `JSON.parse`: a `__proto__` key becomes an own property via `Object.defineProperty`, never the prototype (plain assignment would hit the inherited setter). Same in `FlexAssembler`.
121
123
-`Assembler.connectTo(stream, {onDone: asm => …})` is substrate-aware: accepts either a Node `Readable` (attaches `'data'` listener) or a Web `ReadableStream` (pumps via `getReader()`). Detection via `typeof stream.getReader === 'function'`. `asm.onDone(fn)` can set/clear the callback after construction.
122
124
- For hot paths, prefer a manual `for await (const tok of readable) asm.consume(tok)` loop over `connectTo` — no async-closure overhead, errors propagate directly. `FlexAssembler` has the same shape.
123
125
-`asm.tapChain` is a function for use in `chain()`.
Copy file name to clipboardExpand all lines: ARCHITECTURE.md
+1Lines changed: 1 addition & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -128,6 +128,7 @@ The typings name the stage shapes as exported aliases on the parser entries (`co
128
128
-`Assembler.connectTo(stream, {onDone})` — accepts either a Node `Readable` or a Web `ReadableStream`; detects the substrate via `typeof stream.getReader === 'function'` and either pumps via `getReader()` (Web) or listens on `'data'` (Node). The `onDone(asm)` callback fires when a top-level value is assembled. The 2.x `EventEmitter` shape (`asm.on('done', …)`) is removed in 3.0 — use the `onDone` option or `asm.onDone(fn)`.
129
129
-`asm.tapChain` — a function for use in `chain()` that returns assembled values or `none`.
- Writes keys like `JSON.parse`: a `__proto__` key becomes an own data property (`Object.defineProperty`); plain assignment would invoke the inherited setter and replace the object's prototype. `FlexAssembler` does the same for plain objects.
131
132
- Supports `reviver` option (like `JSON.parse` reviver) and `numberAsString`.
Copy file name to clipboardExpand all lines: README.md
+4Lines changed: 4 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -14,6 +14,10 @@ Why it might be for you:
14
14
-**Performance-minded.** The parser and assemblers are measured and tuned along the hot paths. Real numbers depend on your data and hardware, so [benchmark](https://github.com/uhop/stream-json/wiki/Benchmarks) on your own.
15
15
-**Solid.** ESM, bundled TypeScript typings, and a broad test suite exercised across Node, Bun, Deno, and the browser.
16
16
17
+
## Intended input
18
+
19
+
`stream-json` is built for data you own or trust — database dumps, exports, logs, and files produced by your own systems. It is not designed for hostile input: do not feed it JSON from the open internet or from untrusted users. Untrusted JSON needs validation of its own before it reaches a pipeline.
20
+
17
21
## Example
18
22
19
23
Pull one array out of a JSON document larger than memory and tally it — one record at a time, in constant memory:
Copy file name to clipboardExpand all lines: llms-full.txt
+3-1Lines changed: 3 additions & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -14,6 +14,8 @@
14
14
- Proper backpressure handling via Node.js stream infrastructure
15
15
- Works with `stream-chain` for pipeline composition
16
16
17
+
**Intended input:** data you own or trust — database dumps, exports, logs, files produced by your own systems. `stream-json` is not designed for hostile input: do not feed it JSON from the open internet or from untrusted users; untrusted JSON needs validation of its own before it reaches a pipeline.
`Assembler` — a plain class (no `EventEmitter` inheritance) that interprets the token stream and reconstructs JavaScript objects. It reads only packed tokens (`keyValue`, `stringValue`, `numberValue`); streamed chunks are ignored, so the parser must pack keys, strings, and numbers (its default). 3.0 dropped the `'done'` event in favor of an `onDone` callback option.
201
+
`Assembler` — a plain class (no `EventEmitter` inheritance) that interprets the token stream and reconstructs JavaScript objects. It reads only packed tokens (`keyValue`, `stringValue`, `numberValue`); streamed chunks are ignored, so the parser must pack keys, strings, and numbers (its default). It materializes like `JSON.parse`: a `__proto__` key becomes an own property; the object's prototype is never touched. 3.0 dropped the `'done'` event in favor of an `onDone` callback option.
200
202
201
203
Constructor options:
202
204
- `reviver` (function) — like `JSON.parse` reviver. Called as `reviver(key, value)`.
Copy file name to clipboardExpand all lines: llms.txt
+3-1Lines changed: 3 additions & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -2,6 +2,8 @@
2
2
3
3
> Micro-library of Node.js stream components for creating custom JSON processing pipelines with a minimal memory footprint. Parse JSON files far exceeding available memory using a SAX-inspired streaming token API. One dependency: `stream-chain`.
4
4
5
+
**Intended input:** data you own or trust (database dumps, exports, logs, your own systems' files). Not designed for hostile input — do not feed it JSON from the open internet or from untrusted users; validate untrusted JSON before it reaches a pipeline.
6
+
5
7
## Install
6
8
7
9
npm i stream-json
@@ -67,7 +69,7 @@ For the SAX-style event API on Web, use the `EventTarget`-based variants from `s
67
69
68
70
### Assembler
69
71
70
-
`Assembler` — class that reconstructs JS objects from tokens. Reads only packed tokens (`keyValue`, `stringValue`, `numberValue`); streamed chunks are ignored. Receives a per-value callback via the `onDone` option.
72
+
`Assembler` — class that reconstructs JS objects from tokens. Reads only packed tokens (`keyValue`, `stringValue`, `numberValue`); streamed chunks are ignored. Materializes like `JSON.parse`: a `__proto__` key becomes an own property, never the prototype. Receives a per-value callback via the `onDone` option.
0 commit comments