-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathindex.html
More file actions
234 lines (221 loc) · 10.5 KB
/
Copy pathindex.html
File metadata and controls
234 lines (221 loc) · 10.5 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>zero-tvm — LLM inference in the browser, on hand-written WGSL</title>
<meta name="description" content="zero-tvm is an LLM inference engine written by hand in WGSL and TypeScript. It runs entirely in the browser: weights download once, cache locally, and inference runs on your own GPU.">
<link rel="icon" href="/favicon.svg" type="image/svg+xml">
<link rel="canonical" href="https://zerotvm.com/">
<meta property="og:type" content="website">
<meta property="og:url" content="https://zerotvm.com/">
<meta property="og:title" content="zero-tvm — LLM inference in the browser, on hand-written WGSL">
<meta property="og:description" content="An LLM inference engine written by hand in WGSL and TypeScript. Weights download once, cache locally, and inference runs on your own GPU.">
<!-- The canonical URL is the one that actually gets posted, and it was the
only page declaring summary_large_image without an image to put in it —
so every share of https://zerotvm.com/ unfurled as a bare text card
while /docs.html unfurled richly. og.png ships already. -->
<meta property="og:image" content="https://zerotvm.com/og.png">
<meta name="twitter:card" content="summary_large_image">
<meta name="twitter:image" content="https://zerotvm.com/og.png">
<meta name="twitter:title" content="zero-tvm — LLM inference in the browser, on hand-written WGSL">
<meta name="twitter:description" content="An LLM inference engine written by hand in WGSL and TypeScript. Weights download once, cache locally, and inference runs on your own GPU.">
<link rel="preload" href="/entrance/bg.svg" as="image" type="image/svg+xml">
<link rel="preload" href="/entrance/ring.svg" as="image" type="image/svg+xml">
<link rel="preload" href="/fonts/f3-fraunce.woff2" as="font" type="font/woff2" crossorigin>
<link rel="preload" href="/fonts/f4-plinean.woff2" as="font" type="font/woff2" crossorigin>
<link rel="stylesheet" href="/fonts.css">
<link rel="stylesheet" href="/tokens.css">
<!-- chat-ui.css: the shared message surface — ENTER opens the chat IN the
entrance (landing-chat.ts), so this page renders bubbles and the
composer too. Before landing.css, which carries the panel chrome. -->
<link rel="stylesheet" href="/chat-ui.css">
<link rel="stylesheet" href="/landing.css">
<!-- swarm.css: the NO-JS fallback section only. The swarm is a mode of the
character-select scene now (landing.css carries it); this dresses the
#swarm section a browser without JavaScript still gets. -->
<link rel="stylesheet" href="/swarm.css">
<script type="application/ld+json">
{
"@context": "https://schema.org",
"@type": "SoftwareSourceCode",
"name": "zero-tvm",
"url": "https://zerotvm.com/",
"description": "An LLM inference engine written by hand in WGSL and TypeScript, running entirely in the browser.",
"programmingLanguage": [
"WGSL",
"TypeScript"
],
"codeRepository": "https://github.com/abgnydn/zero-tvm",
"license": "https://opensource.org/licenses/MIT",
"isPartOf": {
"@type": "Collection",
"name": "WebGPU research line",
"url": "https://kernelfusion.dev"
},
"author": {
"@type": "Person",
"name": "Ahmet Barış Günaydın",
"url": "https://zerotvm.com/",
"sameAs": [
"https://barisgunaydin.com",
"https://github.com/abgnydn",
"https://huggingface.co/abgunaydin",
"https://x.com/abgnydn_",
"https://www.linkedin.com/in/abgnydn/"
]
}
}
</script>
</head>
<body>
<nav class="nav">
<div class="wrap nav-in">
<a href="/" class="nav-logo">zero<span>-tvm</span></a>
<div class="nav-links">
<a href="#models">Models</a>
<a href="#swarm">Swarm</a>
<!-- "Rooms" pointed at share.html's bare host page (a floating modal
over black). Hosting lives in the entrance now — enter a character,
then ⟁ Room. share.html remains the GUEST surface links point at,
and the footer still lists it. -->
<!-- "Agent" was here. agent-host.html only works against a local
agent-server on 127.0.0.1:8017, so a public visitor paid a ~2 GB
download for a page that then sat on "booting…" forever. It is a
repo workflow (`npm run agent`), documented in the README, not a
destination on the site. -->
<a href="docs.html#vs-webllm">Proof</a>
<a href="docs.html">Docs</a>
<!-- "Open chat →" left with the in-place chat: the page below IS the
chat entrance, and the CTA duplicated ENTER while routing through
the old standalone gate. zero-tvm.html remains the deep-link
surface (footer, docs, ?model= links). -->
</div>
</div>
</nav>
<section id="models">
<!-- The game entrance. Full-viewport character select — landing.ts renders
the whole screen (splash, stage, roster, sheet) into this root, and the
SWARM is a mode of it: the "Too big for one machine? Split it" button
under ENTER turns the sheet into the link builder and the summoning
ring into one arc per machine. Every figure still comes from the
registry; docs.html carries the rest of the why. -->
<div id="model-browser" class="cs-root"></div>
</section>
<!-- THE NO-JS FALLBACK for that mode, and nothing else. landing.ts hides this
section the moment it has rendered the scene, so it is what a browser
without JavaScript gets: the prose, the four explainers, a plain
share.html link and the full reach paragraph. Hidden from the markup
rather than in it, so a browser that never reaches that line keeps it. -->
<section id="swarm">
<div class="wrap">
<div class="sec-head">
<span class="eyebrow"><span class="pulse" aria-hidden="true"></span>Swarm</span>
<h2>A model bigger than the machine in front of you</h2>
<p>
A transformer is a stack of layers run in order, and nothing requires
one machine to hold all of them. Cut the stack into ranges: this laptop
loads the first layers, another machine loads the next, and each token
travels the chain as a single hidden state hopping from one browser to
the next over a peer-to-peer connection. Whoever holds the link just
chats, and never learns the model was assembled out of parts.
</p>
</div>
<div class="band sw-band">
<div>
<h3>Split by layer</h3>
<p>
Each machine downloads only the layers it holds, and keeps the
attention cache for those layers alone. The one that starts the model
carries the embedding; the one that ends it carries the output head.
</p>
</div>
<div>
<h3>Tokens hop</h3>
<p>
What crosses between machines is the bare residual — one hand-off per
token, on its own data channel so it never queues behind a reply
being streamed to a guest.
</p>
</div>
<div>
<h3>Guests just talk</h3>
<p>
A guest opens the room link and types. No WebGPU, no download, nothing
runs on their machine; the conversation lives in their tab and the
machines serving it are stateless between requests.
</p>
</div>
<div>
<h3>Weights spread</h3>
<p>
A guest can copy the cached weights straight from the machine serving
it, over a second channel, instead of fetching them again. What it
gets is whatever that machine holds — in a split, one layer range.
</p>
</div>
</div>
<div class="sw-builder">
<p class="note">
The link builder needs JavaScript. Without it: <a href="share.html">share.html</a>
is the room surface — open it as
<code>share.html?model=<model>&layers=0-k</code> on the first
machine, then <code>share.html?model=<model>&layers=k-N#<room></code>
on the next, and <code>share.html#<room></code> for anyone who
only wants to chat.
</p>
</div>
<p class="note sw-reach">
<b>What this needs.</b> The machines connect directly to each other; the
relay only passes connection details, and prompts and tokens never touch
it. There is no TURN relay in the connection config, so machines on the
same network, or behind ordinary home routers, find each other — a
corporate or hotel network usually will not let them. Splitting also
needs an MLX checkpoint: the MLC builds fetch whole shards, so skipping
layers would save no download, and only the models the builder lists can
be cut. Every serving tab should stay awake — share.html has a switch for
that, and a machine whose tab is asleep serves nobody.
</p>
</div>
</section>
<footer>
<div class="wrap">
<div class="foot-grid">
<div>
<a href="/" class="nav-logo">zero<span>-tvm</span></a>
<p class="foot-about">
An LLM inference engine written by hand in WGSL and TypeScript. Part
of a research line on WebGPU kernels by Ahmet Barış Günaydın.
</p>
</div>
<div>
<h4>This site</h4>
<ul>
<li><a href="zero-tvm.html">Chat</a></li>
<li><a href="share.html">Host a room</a></li>
<li><a href="#swarm">Split a model across machines</a></li>
<li><a href="docs.html">Docs</a></li>
<li><a href="docs.html#vs-webllm">Proof</a></li>
<li><a href="validate.html">Validate</a></li>
</ul>
</div>
<div>
<h4>Research line</h4>
<ul id="sibling-links">
<li><a href="https://kernelfusion.dev">kernelfusion.dev</a></li>
<li><a href="https://github.com/abgnydn">github.com/abgnydn</a></li>
<li><a href="https://huggingface.co/abgunaydin">huggingface.co/abgunaydin</a></li>
</ul>
</div>
</div>
<div class="foot-bottom">
<span>MIT licensed · weights are fetched once from HuggingFace and cached in your browser; prompts and generation stay on your machine in the chat. In a room, a guest’s prompts run on the host’s GPU.</span>
<a href="https://github.com/abgnydn/zero-tvm">github.com/abgnydn/zero-tvm ↗</a>
</div>
</div>
</footer>
<!-- landing-swarm.ts has no tag of its own any more: it is a mode of the scene,
imported by landing.ts on the click that opens it. -->
<script type="module" src="/src/landing.ts"></script>
</body>
</html>