Merge branch 'feat/web-edition' into feat/web-look

Author: ndemianc

docs/WEB.md

@@ -150,28 +150,39 @@ The extension's existing PKCE flow, with the browser's pieces in place of the OS
 1  extension   accountSignIn(): verifier → SecretStorage; builds
                <account>/ai/login?redirect_uri=<callback>&code_challenge=<S256>
 2  extension   host.openAuth(uri) → command levelcode.web.openAuthUrl (registered by main.js)
-3  main.js     checks the address is <account>/ai/…, marks the tab (sessionStorage), location.assign
+3  main.js     checks the address is <account>/ai/…, notes where the tab is (sessionStorage), location.assign
 4  account     already signed in there → mints a one-time code bound to the challenge, 302 to
                <editor>/callback.html?vscode-reqid=N&vscode-scheme=levelcode
                  &vscode-authority=levelcode.levelcode-ai&vscode-path=%2Fauth%2Fcallback&code=…
                (not signed in → the login page first)
-5  callback    writes localStorage "vscode-web.url-callbacks[N]" (dated), sees the mark → goes back to /
+5  callback    checks it is a sign-in to levelcode.levelcode-ai/auth/callback, writes localStorage
+               "vscode-web.url-callbacks[N]" (dated, the code only), goes back to where the tab was
 6  main.js     at start-up, once the workbench is listening, delivers a fresh result as
                levelcode://levelcode.levelcode-ai/auth/callback?code=… to the URL service
-7  extension   handleAuthCallback → POST /api/levelcode/v1/auth/exchange {code, verifier} → tokens
+7  extension   handleAuthCallback: a sign-in is pending (a verifier is stored) →
+               POST /api/levelcode/v1/auth/exchange {code, verifier} → tokens
 ```
 
 Why the tab leaves instead of opening a pop-up: a window opened from a chain that began in a
 webview is the first thing a strict browser (Safari, a phone) refuses. Leaving the tab and
 returning needs no permission. The PKCE verifier lives in SecretStorage, so it survives the
-navigation.
+navigation — and where the browser lets nothing persist (no IndexedDB or WebCrypto, or a refused
+write), the tab's secrets ride through that one navigation in sessionStorage and are taken back,
+and removed, when the editor starts again. The tab comes back to the address it left: a folder or
+workspace opened from the computer is opened again, not the scratch workspace.
 
 - **`?signin=1`** (the account site's "Open LevelCode" link) makes step 1 happen by itself, once:
   `main.js` removes the parameter, then delivers `levelcode://…/launch`, which focuses the chat and
   starts the sign-in only when there is no session.
 - **A result is delivered only if it is fresh** (written in the last two minutes — the one-time
   code lives for a minute) and is removed either way. One left by a callback page whose editor
   tab is gone is delivered when the editor next starts.
+- **`callback.html` is a page anyone can link to**, and what it leaves is delivered at the next start.
+  So it passes on a LevelCode sign-in and nothing else: for `levelcode.levelcode-ai/auth/callback`
+  only, and only the one-time `code` — never a token in the link (the old flow, which the desktop
+  still accepts) or the rest of the query. The extension then takes a code only while a sign-in of
+  its own is pending (`host.caps.unaskedSignIn` is off in a browser): a link nobody asked for cannot
+  sign the user in to someone else's account.
 - **`levelcode.web.openAuthUrl` is an open-redirect if it is loose**, and any extension in the page
   can call it. It goes only to `<account origin>/ai/…`.
 - The address handed over is `encodeURI(uri.toString(true))` — the string the editor's own opener
@@ -198,7 +209,7 @@ node web/serve.mjs --dist dist-web            # http://127.0.0.1:8800
 | `--webview-origin 'https://{{uuid}}.view.example.com'` (`LEVELCODE_WEB_WEBVIEW_ORIGIN`) | Self-host the webview origin. Without it the page uses Code-OSS's CDN at this build's own commit. Needs wildcard DNS and TLS — see below. |
 | `--ext-host-origin 'https://{{uuid}}.ext.example.com'` (`LEVELCODE_WEB_EXTHOST_ORIGIN`) | Run the extension host on an origin of its own. Optional while no third-party code is loaded. |
 | `--static <dir>` | Use a prebuilt client (skips the gulp build). |
-| `--id <id>` | Override the build id (default: a fingerprint of the client, the page and the extension). |
+| `--id <id>` | Override the build id (default: a hash of everything the build put under `/_/<id>/`, staged first and named after it). |
 
 **Dev server** (`web/serve.mjs`): `--static <vscode-web dir> --extensions <dir> --extension-names
 levelcode-ai,levelcode-web --account <origin>`, plus `--product <json>` and `--config-defaults
@@ -288,7 +299,9 @@ causing the effect). Use another registrable domain.
 | Editor settings, layout | the workbench's own storage | editor origin |
 
 Clearing the browser's site data removes all of it. A private window may discard it on close; the
-secret store then falls back to memory for the session.
+secret store then falls back to memory for the session. The key the secrets are sealed with is created
+once per browser profile: two tabs opened together on a fresh profile store it in one IndexedDB
+transaction, so both end with the same key.
 
 ## Keeping the desktop and the browser in step
 

extensions/levelcode-ai/agent.js

@@ -78,10 +78,6 @@ const TOOLS = [
 	{ name: 'use_skill', description: 'Load an expert playbook (SKILL.md) for a task type, chosen from the "Available skills" list in your system prompt. Returns the skill\'s step-by-step instructions as the tool result — then follow them. Read-only and instant (no approval). Call it once, early, when the goal matches a skill\'s description.', input_schema: { type: 'object', properties: { name: { type: 'string', description: 'the exact skill name from the Available skills list' } }, required: ['name'] } }
 ];
 
-// Rough token estimates (chars/4) for the static prompt segments, so the context popover can break
-// down "what's filling the window" — system + tools are sent on every request, the rest is messages.
-const SYSTEM_TOKENS_EST = Math.round(SYSTEM_BASE_HOST.length / 4);
-
 /** Append an "Available skills" name+description menu to the base prompt. Bodies NEVER go here —
  *  progressive disclosure means only name+description are ever in the prompt (~100 words for a dozen). */
 function buildSystem(menu) {
@@ -167,7 +163,8 @@ async function scanSearch(term, root) {
 	const out = [];
 	let size = 0;
 	let uris = [];
-	try { uris = await host.findFiles('**/*', FILE_EXCLUDES, 2000); } catch { return ''; }
+	// Scoped to `root` BEFORE the cap: across the whole workspace, another folder's files could use up all 2,000.
+	try { uris = await host.findFiles(new vscode.RelativePattern(host.uriFor(root), '**/*'), FILE_EXCLUDES, 2000); } catch { return ''; }
 	const rootPrefix = root.endsWith('/') ? root : root + '/';
 	for (const uri of uris) {
 		if (size >= 6000) { break; }

extensions/levelcode-ai/extension.js

@@ -931,7 +931,8 @@ async function contentHits(terms, dir) {
 	if (!dir) { return hits; }
 	const prefix = dir.endsWith('/') ? dir : dir + '/';
 	let uris = [];
-	try { uris = await host.findFiles('**/*', '{**/node_modules/**,**/.git/**,**/out/**,**/dist/**,**/*.map,**/*.min.*}', 400); } catch { return hits; }
+	// Scoped to `dir` BEFORE the cap: across the whole workspace, another folder's files could use up all 400.
+	try { uris = await host.findFiles(new vscode.RelativePattern(host.uriFor(dir), '**/*'), '{**/node_modules/**,**/.git/**,**/out/**,**/dist/**,**/*.map,**/*.min.*}', 400); } catch { return hits; }
 	const needles = terms.map((t) => t.toLowerCase());
 	const deadline = Date.now() + 4000;   // the same budget rgFiles gives ripgrep
 	for (const uri of uris) {
@@ -1094,7 +1095,10 @@ function recordDiagramStat(ev) {
 const diagramFolders = () => (vscode.workspace.workspaceFolders || []).map((f) => ({ name: f.name, root: f.uri.fsPath }));
 /** This conversation's diagrams. One for the life of the extension; reset with the conversation. */
 const diagrams = createDiagrams({
-	resolveLink: (link) => diagramLinks.resolveLink(link, diagramFolders()),
+	// A diagram's links are checked as it is drawn, and that cannot wait. Where files can only be looked at
+	// asynchronously (LevelCode in a browser) they are checked against the files the extension has indexed
+	// (buildFileIndex); the desktop reads the disk, as before. The click checks the file itself.
+	resolveLink: (link) => diagramLinks.resolveLink(link, diagramFolders(), { fs: host.knownFilesView([...workspaceFiles.values()].map((u) => u.path)) || undefined }),
 	onStat: recordDiagramStat
 });
 /** True once a diagram's spec has left the model's context (a compaction, or a resume that did not fit) —
@@ -1117,7 +1121,7 @@ async function openDiagramLink(id, nodeId) {
 	const node = record && record.spec ? record.spec.nodes.find((n) => n.id === String(nodeId)) : null;
 	if (!node || !node.link) { dbg('diagram.link.unknown', { id }); return; }
 	// Checked again at the moment of the click: the file may have moved, or become a link, since it was drawn.
-	const r = diagramLinks.resolveLink(node.link, diagramFolders());
+	const r = await host.withStats((fs) => diagramLinks.resolveLink(node.link, diagramFolders(), { fs }));
 	if (!r.ok) { vscode.window.showWarningMessage('LevelCode: that link cannot be opened — ' + r.reason + '.'); dbg('diagram.link.refused', { reason: r.reason }); return; }
 	recordDiagramStat({ type: 'link' });
 	try {
@@ -3567,10 +3571,16 @@ async function handleAuthCallback(uri) {
 	try {
 		const q = new URLSearchParams(uri.query || '');
 		const code = q.get('code');
+		const verifier = ctx ? await ctx.secrets.get(ACCOUNT_VERIFIER_KEY) : null;
+		// Where the callback is a page anyone can link to (LevelCode in a browser), a link nobody asked for must not
+		// sign the user in to someone else's account: only a code that answers a sign-in this editor started is taken.
+		if (!host.caps.unaskedSignIn && (!code || !verifier)) {
+			dbg('account.callback', { ignored: code ? 'no sign-in pending' : 'not a sign-in code' });
+			return;
+		}
 		// Preferred: PKCE code → exchange for tokens using the verifier we stashed at sign-in.
 		if (code) {
 			const endpoint = cloudApiUrl(); // token exchange is a BACKEND (Rails) call, not the website
-			const verifier = ctx ? await ctx.secrets.get(ACCOUNT_VERIFIER_KEY) : null;
 			try {
 				const res = await fetch(endpoint + '/api/levelcode/v1/auth/exchange', {
 					method: 'POST',

extensions/levelcode-ai/host.js

@@ -35,6 +35,12 @@ const caps = Object.freeze({
 	mcpStdio: !isBrowser,
 	/** Search file contents with the ripgrep binary the editor ships. */
 	ripgrep: !isBrowser,
+	/**
+	 * Take a sign-in result that no sign-in of this editor asked for: the old token-in-the-link form, or a code
+	 * with no sign-in pending. The desktop does what it always did. In a browser the callback is a page of the
+	 * editor's own site that anyone can link to, so only a code answering a sign-in this browser started counts.
+	 */
+	unaskedSignIn: !isBrowser,
 });
 
 const TEXT_SNIFF_BYTES = 8000;
@@ -63,6 +69,21 @@ function uriFor(abs) {
 	return best ? best.uri.with({ path: abs }) : vscode.Uri.file(abs);
 }
 
+/**
+ * Is this document one of the user's own files — the kind inline completion is offered in? On the desktop a
+ * `file` or an `untitled` document, as ever. In the browser also a document inside a workspace folder, whatever
+ * its scheme: the scratch workspace's files are `levelcode-scratch:` ones.
+ * @param {vscode.Uri} uri
+ */
+function isUserDocument(uri) {
+	if (uri.scheme === 'file' || uri.scheme === 'untitled') { return true; }
+	if (!isBrowser) { return false; }
+	return folders().some((f) => {
+		const root = f.uri.path.endsWith('/') ? f.uri.path : f.uri.path + '/';
+		return f.uri.scheme === uri.scheme && (f.uri.authority || '') === (uri.authority || '') && (uri.path === f.uri.path || uri.path.startsWith(root));
+	});
+}
+
 /* ----- reading and probing ------------------------------------------------------------------- */
 
 /** @param {string} abs @returns {Promise<Uint8Array>} rejects when the file is missing or unreadable. */
@@ -141,6 +162,66 @@ async function withReads(run) {
 	return result;
 }
 
+/* ----- checking paths synchronously --------------------------------------------------------- */
+
+/**
+ * The part of Node's `fs` a synchronous path check uses — `statSync` and `realpathSync` — answered from `lookup`.
+ * In the browser nothing is a symbolic link, so a path's real path is the path.
+ * @param {(abs: string) => ({isFile: boolean, isDirectory: boolean}|null)} lookup
+ */
+function statView(lookup) {
+	const missing = (p) => Object.assign(new Error('ENOENT: no such file or directory, ' + p), { code: 'ENOENT' });
+	return {
+		statSync(p) {
+			const s = lookup(String(p));
+			if (!s) { throw missing(p); }
+			return { isFile: () => s.isFile, isDirectory: () => s.isDirectory };
+		},
+		realpathSync(p) { if (!lookup(String(p))) { throw missing(p); } return String(p); },
+	};
+}
+
+/**
+ * Run a check written against `fs.statSync` / `fs.realpathSync` (a diagram's code links: diagram/links.js) where
+ * files can only be looked at asynchronously. Like withReads: the check runs once to learn which paths it asks
+ * about, those are stat'd, and it runs again with the answers (at most four passes).
+ * On the desktop it simply runs once against `fs`, as before.
+ * @template T
+ * @param {(fs: { statSync: Function, realpathSync: Function }) => T} run
+ * @returns {Promise<T>}
+ */
+async function withStats(run) {
+	if (!isBrowser) { return run(nodeFs); }
+	/** @type {Map<string, {isFile: boolean, isDirectory: boolean}|null>} */
+	const known = new Map();
+	let result;
+	for (let pass = 0; pass < 4; pass++) {
+		const asked = new Set();
+		result = run(statView((abs) => {
+			if (known.has(abs)) { return known.get(abs); }
+			asked.add(abs);
+			return null;
+		}));
+		if (asked.size === 0) { break; }
+		await Promise.all([...asked].map(async (p) => { known.set(p, await stat(p)); }));
+	}
+	return result;
+}
+
+/**
+ * The same view over files already known to exist, for a check that cannot wait for the file system (a diagram's
+ * links are checked as it is drawn). Folders are the files' parents. Null on the desktop: the check reads the disk.
+ * @param {Iterable<string>} files absolute paths
+ */
+function knownFilesView(files) {
+	if (!isBrowser) { return null; }
+	const set = new Set(files);
+	const dirs = new Set();
+	for (const f of set) { for (let d = f.slice(0, f.lastIndexOf('/')); d; d = d.slice(0, d.lastIndexOf('/'))) { dirs.add(d); } }
+	dirs.add('/');
+	return statView((abs) => (set.has(abs) ? { isFile: true, isDirectory: false } : dirs.has(abs) ? { isFile: false, isDirectory: true } : null));
+}
+
 /* ----- sending the user to sign in ------------------------------------------------------------- */
 
 /** Registered by the page (web/main.js); it takes the tab to the account site and the sign-in comes back to it. */
@@ -175,7 +256,7 @@ async function openAuth(uri) {
 /**
  * `vscode.workspace.findFiles`, which is what the editor's own search service answers. The browser
  * build uses it too: LevelCode's scratch workspace registers a file search provider.
- * @param {string} glob
+ * @param {vscode.GlobPattern} glob  a string, or a vscode.RelativePattern to search under one folder
  * @param {string} exclude
  * @param {number} max
  */
@@ -185,7 +266,7 @@ function findFiles(glob, exclude, max) {
 
 module.exports = {
 	isBrowser, caps,
-	uriFor, readBytes, readText, readTextOrNull, stat, exists, isBinary, withReads, findFiles, openAuth,
+	uriFor, isUserDocument, readBytes, readText, readTextOrNull, stat, exists, isBinary, withReads, withStats, knownFilesView, findFiles, openAuth,
 	// For tests.
 	_path: path,
 };

extensions/levelcode-ai/inlineComplete.js

@@ -10,6 +10,7 @@
 
 const vscode = require('vscode');
 const { sendWithAuthRetry } = require('./providers/authRetry');
+const host = require('./host');
 
 const PREFIX_MAX = 2500; // chars of context before the cursor (smaller = faster first token)
 const SUFFIX_MAX = 1000; // chars of context after the cursor
@@ -98,7 +99,7 @@ function registerInlineComplete(context, deps) {
 		async provideInlineCompletionItems(document, position, ctx, token) {
 			const cfg = aiConfig();
 			if (!cfg.get('completions.enabled', true)) { return null; }
-			if (document.uri.scheme !== 'file' && document.uri.scheme !== 'untitled') { return null; }
+			if (!host.isUserDocument(document.uri)) { return null; }   // file and untitled; in a browser, the workspace's own schemes too
 
 			// Debounce: if the user keeps typing, this token is cancelled and we bail.
 			const debounce = Math.max(0, cfg.get('completions.debounce', 150));

extensions/levelcode-ai/reviewSession.js

@@ -386,6 +386,9 @@ function registerReview(context, post, dbg, recordTouch) {
 				try { current.set(s.uri, ((await host.readTextOrNull(vscode.Uri.parse(s.uri).fsPath)) || '').replace(/^/, '')); } catch { /* */ }
 			}
 			rehydrate((uri, s) => current.get(s.uri) || '');
+			// The chat may have asked for the cards (resync, on `ready`) while these reads were still out, and was
+			// sent none. Send them now; a chat that is not listening yet asks again when it is.
+			if (pending.size) { resync(); }
 		})();
 	}
 

extensions/levelcode-ai/test/diagramHost.test.js

@@ -136,7 +136,8 @@ function boot(over) {
 	const env = {
 		vscode,
 		// The host module (host.js) as the desktop sees it: workspace paths are files on disk.
-		host: { isBrowser: false, caps: { shell: true, mcpStdio: true, ripgrep: true }, uriFor: (abs) => vscode.Uri.file(abs) },
+		host: { isBrowser: false, caps: { shell: true, mcpStdio: true, ripgrep: true, unaskedSignIn: true }, uriFor: (abs) => vscode.Uri.file(abs),
+			withStats: async (run) => run(fs), knownFilesView: () => null },
 		fs, path, os, catalog, sessionMemory, diagrams, diagramLinks, diagramExport, diagramText, diagramTool, diagramBundle: bundle,
 		diagramAscii: (over && over.ascii) || ascii,
 		aiConfig: () => ({ get: (k, d) => (k in settings ? settings[k] : d) }),