Skip to content

Commit 2f2d35b

Browse files
committed
Added checks for __proto__.
1 parent 1a66390 commit 2f2d35b

14 files changed

Lines changed: 184 additions & 5 deletions

AGENTS.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -109,6 +109,7 @@ stream-json/
109109
- **Keep `.js` and `.d.ts` files in sync** for all modules under `src/`.
110110
- **Token-based architecture.** The parser produces a stream of `{name, value}` tokens. All filters, streamers, and utilities operate on this token protocol.
111111
- **Backpressure must be handled correctly.** All stream components rely on Node.js stream infrastructure via `stream-chain`.
112+
- **Intended input is data the user owns or trusts** (dumps, exports, logs). The library is not designed for hostile input; docs say so, and code changes are not hardened against adversarial JSON beyond `JSON.parse` parity (`__proto__` becomes an own property) and the filters' `maxDepth` guard.
112113

113114
## Architecture
114115

@@ -118,6 +119,7 @@ stream-json/
118119
- **Assembler** (`src/assembler.js`, implementation in `src/core/assembler.js`) interprets the token stream and reconstructs JavaScript objects. Plain class — no `EventEmitter` inheritance in 3.x.
119120
- Used internally by all streamers via `streamBase`.
120121
- Reads only packed tokens (`keyValue`, `stringValue`, `numberValue`); streamed chunks are ignored.
122+
- Materializes like `JSON.parse`: a `__proto__` key becomes an own property via `Object.defineProperty`, never the prototype (plain assignment would hit the inherited setter). Same in `FlexAssembler`.
121123
- `Assembler.connectTo(stream, {onDone: asm => …})` is substrate-aware: accepts either a Node `Readable` (attaches `'data'` listener) or a Web `ReadableStream` (pumps via `getReader()`). Detection via `typeof stream.getReader === 'function'`. `asm.onDone(fn)` can set/clear the callback after construction.
122124
- For hot paths, prefer a manual `for await (const tok of readable) asm.consume(tok)` loop over `connectTo` — no async-closure overhead, errors propagate directly. `FlexAssembler` has the same shape.
123125
- `asm.tapChain` is a function for use in `chain()`.

ARCHITECTURE.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -128,6 +128,7 @@ The typings name the stage shapes as exported aliases on the parser entries (`co
128128
- `Assembler.connectTo(stream, {onDone})` — accepts either a Node `Readable` or a Web `ReadableStream`; detects the substrate via `typeof stream.getReader === 'function'` and either pumps via `getReader()` (Web) or listens on `'data'` (Node). The `onDone(asm)` callback fires when a top-level value is assembled. The 2.x `EventEmitter` shape (`asm.on('done', …)`) is removed in 3.0 — use the `onDone` option or `asm.onDone(fn)`.
129129
- `asm.tapChain` — a function for use in `chain()` that returns assembled values or `none`.
130130
- Tracks `depth`, `path`, `current`, `key`, `stack`.
131+
- Writes keys like `JSON.parse`: a `__proto__` key becomes an own data property (`Object.defineProperty`); plain assignment would invoke the inherited setter and replace the object's prototype. `FlexAssembler` does the same for plain objects.
131132
- Supports `reviver` option (like `JSON.parse` reviver) and `numberAsString`.
132133

133134
### Disassembler

README.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,10 @@ Why it might be for you:
1414
- **Performance-minded.** The parser and assemblers are measured and tuned along the hot paths. Real numbers depend on your data and hardware, so [benchmark](https://github.com/uhop/stream-json/wiki/Benchmarks) on your own.
1515
- **Solid.** ESM, bundled TypeScript typings, and a broad test suite exercised across Node, Bun, Deno, and the browser.
1616

17+
## Intended input
18+
19+
`stream-json` is built for data you own or trust — database dumps, exports, logs, and files produced by your own systems. It is not designed for hostile input: do not feed it JSON from the open internet or from untrusted users. Untrusted JSON needs validation of its own before it reaches a pipeline.
20+
1721
## Example
1822

1923
Pull one array out of a JSON document larger than memory and tally it — one record at a time, in constant memory:

llms-full.txt

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,8 @@
1414
- Proper backpressure handling via Node.js stream infrastructure
1515
- Works with `stream-chain` for pipeline composition
1616

17+
**Intended input:** data you own or trust — database dumps, exports, logs, files produced by your own systems. `stream-json` is not designed for hostile input: do not feed it JSON from the open internet or from untrusted users; untrusted JSON needs validation of its own before it reaches a pipeline.
18+
1719
## Quick start
1820

1921
Install:
@@ -196,7 +198,7 @@ parserStream.on('data', token => console.log(token.name));
196198

197199
## Assembler
198200

199-
`Assembler` — a plain class (no `EventEmitter` inheritance) that interprets the token stream and reconstructs JavaScript objects. It reads only packed tokens (`keyValue`, `stringValue`, `numberValue`); streamed chunks are ignored, so the parser must pack keys, strings, and numbers (its default). 3.0 dropped the `'done'` event in favor of an `onDone` callback option.
201+
`Assembler` — a plain class (no `EventEmitter` inheritance) that interprets the token stream and reconstructs JavaScript objects. It reads only packed tokens (`keyValue`, `stringValue`, `numberValue`); streamed chunks are ignored, so the parser must pack keys, strings, and numbers (its default). It materializes like `JSON.parse`: a `__proto__` key becomes an own property; the object's prototype is never touched. 3.0 dropped the `'done'` event in favor of an `onDone` callback option.
200202

201203
Constructor options:
202204
- `reviver` (function) — like `JSON.parse` reviver. Called as `reviver(key, value)`.

llms.txt

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@
22

33
> Micro-library of Node.js stream components for creating custom JSON processing pipelines with a minimal memory footprint. Parse JSON files far exceeding available memory using a SAX-inspired streaming token API. One dependency: `stream-chain`.
44

5+
**Intended input:** data you own or trust (database dumps, exports, logs, your own systems' files). Not designed for hostile input — do not feed it JSON from the open internet or from untrusted users; validate untrusted JSON before it reaches a pipeline.
6+
57
## Install
68

79
npm i stream-json
@@ -67,7 +69,7 @@ For the SAX-style event API on Web, use the `EventTarget`-based variants from `s
6769

6870
### Assembler
6971

70-
`Assembler` — class that reconstructs JS objects from tokens. Reads only packed tokens (`keyValue`, `stringValue`, `numberValue`); streamed chunks are ignored. Receives a per-value callback via the `onDone` option.
72+
`Assembler` — class that reconstructs JS objects from tokens. Reads only packed tokens (`keyValue`, `stringValue`, `numberValue`); streamed chunks are ignored. Materializes like `JSON.parse`: a `__proto__` key becomes an own property, never the prototype. Receives a per-value callback via the `onDone` option.
7173

7274
```js
7375
import Assembler from 'stream-json/assembler.js';

src/core/assembler.d.ts

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,9 @@ export interface AssemblerOptions<T = unknown> {
3737
* chunks are ignored, so the upstream parser must pack keys, strings, and numbers
3838
* (its default).
3939
*
40+
* Materializes like `JSON.parse`: a `__proto__` key becomes an own property; the
41+
* object's prototype is never touched.
42+
*
4043
* Generic in `T` (default `unknown`) — the type of the fully assembled value.
4144
* Declare `new Assembler<MyShape>()` to type `current` and `tapChain()`. Read
4245
* `current` in the `onDone` callback (when it actually holds the completed `T`);

src/core/assembler.js

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -157,7 +157,12 @@ class Assembler {
157157
if (this.current instanceof Array) {
158158
this.current.push(value);
159159
} else {
160-
this.current[this.key] = value;
160+
// like JSON.parse: an own property, never the inherited __proto__ setter
161+
if (this.key === '__proto__') {
162+
Object.defineProperty(this.current, this.key, {value, writable: true, enumerable: true, configurable: true});
163+
} else {
164+
this.current[this.key] = value;
165+
}
161166
this.key = null;
162167
}
163168
}
@@ -177,7 +182,11 @@ class Assembler {
177182
} else {
178183
value = this.reviver.call(this.current, this.key, value);
179184
if (value !== undefined) {
180-
this.current[this.key] = value;
185+
if (this.key === '__proto__') {
186+
Object.defineProperty(this.current, this.key, {value, writable: true, enumerable: true, configurable: true});
187+
} else {
188+
this.current[this.key] = value;
189+
}
181190
}
182191
this.key = null;
183192
}

src/core/utils/flex-assembler.d.ts

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,9 @@ type TokenSource = ReadableStream<Token> | ReadableLike;
2222
* chunks are ignored, so the upstream parser must pack keys, strings, and numbers
2323
* (its default).
2424
*
25+
* Materializes like `JSON.parse`: a `__proto__` key becomes an own property; the
26+
* object's prototype is never touched.
27+
*
2528
* Generic in `T` (default `unknown`) — the type of the fully assembled value
2629
* (`current` / `tapChain()`). Orthogonal to the per-rule container type `C` on
2730
* `ObjectRule<C>` / `ArrayRule<C>`: `T` is the whole result, `C` is one custom

src/core/utils/flex-assembler.js

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -253,6 +253,9 @@ class FlexAssembler {
253253
}
254254
if (this.rule) {
255255
this.rule.add(this.current, this.key, value);
256+
} else if (this.key === '__proto__') {
257+
// like JSON.parse: an own property, never the inherited __proto__ setter
258+
Object.defineProperty(this.current, this.key, {value, writable: true, enumerable: true, configurable: true});
256259
} else {
257260
this.current[this.key] = value;
258261
}

tests/node/test-assembler.js

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -278,3 +278,42 @@ test.asPromise('assembler: chain', (t, resolve, reject) => {
278278
});
279279
});
280280
});
281+
282+
const PROTO_VECTORS = [
283+
'{"__proto__":{"isAdmin":true},"name":"bob"}',
284+
'{"__proto__":null,"name":"bob"}',
285+
'{"__proto__":{"hasOwnProperty":1},"name":"bob"}',
286+
'{"user":{"__proto__":{"isAdmin":true},"name":"alice"}}'
287+
];
288+
289+
const checkProtoParity = (t, actual, expected) => {
290+
const inner = actual.user || actual,
291+
innerExpected = expected.user || expected;
292+
t.deepEqual(actual, expected);
293+
t.deepEqual(Object.keys(inner), Object.keys(innerExpected), 'same own keys as JSON.parse');
294+
t.ok(Object.hasOwn(inner, '__proto__'), '__proto__ is an own property');
295+
t.equal(Object.getPrototypeOf(inner), Object.prototype, 'prototype untouched');
296+
t.equal(inner.isAdmin, undefined, 'nothing inherited');
297+
t.equal(typeof inner.hasOwnProperty, 'function', 'Object.prototype methods intact');
298+
};
299+
300+
const assembleProto = (text, options) =>
301+
new Promise((resolve, reject) => {
302+
const asm = assembler(options),
303+
pipeline = chain([readString(text), parser(), asm.tapChain]);
304+
pipeline.on('error', reject);
305+
pipeline.on('end', () => resolve(asm.current));
306+
pipeline.resume();
307+
});
308+
309+
test.asPromise('assembler: __proto__ key becomes an own property, like JSON.parse', async (t, resolve, reject) => {
310+
try {
311+
for (const text of PROTO_VECTORS) {
312+
checkProtoParity(t, await assembleProto(text), JSON.parse(text));
313+
checkProtoParity(t, await assembleProto(text, {reviver: (_key, value) => value}), JSON.parse(text));
314+
}
315+
resolve();
316+
} catch (e) {
317+
reject(e);
318+
}
319+
});

0 commit comments

Comments
 (0)