Repository navigation
feat(share): keep shares in dfo-apis and proxy cards and shares - #77
Merged
Merged
Conversation
The public /api/v1 proxy now forwards /presentation/components and /presentation/shares, passing the write-token, ETag and CORS headers the share data route needs. Analysis, anchor, pin and derived reads already sit under the datasets surface. Every forward to dfo-apis carries x-dfo-caller and x-dfo-caller-principal (DFO_CALLER_PRINCIPAL), so dfo-apis can count allowances per client. The caller digest now hashes the folded address (IPv6 to its /64, v4-mapped to IPv4), the same preimage dfo-apis uses for direct callers.
scripts/export-shares.ts --since writes every share as one JSONL line in the format docs/share-export.md sets out: id, kind, mode with today's null-mode defaults applied, owner, created_at, title, a slot per referenced data card, and the website-owned payload that renders what slots cannot (search queries and parts, conversations, inline cards, canvas layout). It reads only, orders by created_at then id, and exits non-zero naming each share it cannot place. The share_token table (migration 0022) keeps a signed-in user's write token for a share held in dfo-apis.
Creating, fixing and deleting a share goes through dfo-apis' share routes with the caller named on every write. A signed-in owner's write token is kept in share_token and sent on later writes; guests may make live shares, which they cannot change afterwards. /s/<id> and /embed/<id> render the dfo-apis record: card shares from their slots, search, canvas and conversation shares from the stored payload. sharePayload places cards in slots for new shares and the exporter alike, and the exporter writes frame: null for a card whose frame the site does not hold. scripts/import-share-tokens.ts loads the token file dfo-apis' importer writes, idempotently. The per-guest share allowance moves to dfo-apis, so spendGuestShare is gone.
A live card on /s/<id> and /embed/<id> reads its slot's card from dfo-apis, drawn on the frame the slot serves, so the site needs no render principal; the refresh route is a GET that passes the ETag both ways, so the browser revalidates instead of redrawing. A live share's cards are checked before the share is created, so a failing card never leaves a share behind. Share create sends a minted write token and retries once with the same body, and a payload over dfo-apis' limit answers 413 payload_too_large.
A live card's downloads, bundles and spreadsheets included, come from dfo-apis' slot data route in the format and bundle the reader asked for, so a download holds the data the card shows. The browser's refresh reads answer once, with no pending loop, and live shares need no credential.
A snapshot slot draws and downloads the frame its slot recorded at the switch, through the same slot routes a live slot uses, so the page, its downloads and dfo.fetch(share) agree; a card whose read fails is fetched again by the browser and shows the retryable error state. A stored write token is dropped only when dfo-apis says the share is gone or the token is wrong. Every share call runs under a 35 s deadline, a create whose token cannot be stored deletes the new share, and switching a search or conversation share to a snapshot is refused as before. The proxy refuses path segments that decode to "." or ".." or hold a slash.
The graph and search catch-alls and the component export route refuse a path segment that decodes to "." or ".." or holds a slash, as the datasets and presentation routes do. A test finds every forwarding catch-all under /api/v1 and checks each carries the guard.
scripts/share-migration-round-trip.ts seeds one share per legacy case, exports them, imports them into a local dfo-apis and the tokens back, renders /s and /embed for each, checks an owner can write with the imported token and a guest's share has none, and checks a second run of both imports changes nothing.
Putting the token a share already holds for the same owner changes nothing, so a second run of import-share-tokens leaves share_token as it was.
Plain sentences in the share, proxy and studio comments and in docs/share-export.md and docs/development.md, and one typo.
Every forwarding route builds its upstream path through forwardablePath, which refuses dot segments and encodes each segment, so a double-encoded ".." or a "?" or "#" segment stays inside the surface it was sent to; the component read route takes the same check, and a test finds every route under /api/v1 and /api/presentation that forwards. Slot data read through the site keeps dfo-apis' x-dfo-freshness and x-dfo-checked-at headers.
…nest Deleting an account takes each of the user's shares down in dfo-apis with its stored token before the user row goes; a share already gone counts as done, and any other failure keeps the account and the remaining tokens. A create refusal dfo-apis marks permanent answers share_refused with its code and no retry; a transient one keeps the retry. A create whose token cannot be stored, because another user's row holds the id, takes the new share down. Titles are cut by code point, GET /api/share/<id> answers a problem document when dfo-apis fails, and the share export reads shares and parts from one snapshot.
…ctly A share made through the Presentation API with no website payload draws one card per filled slot. A share page waits at most 4 s for its slot cards and leaves the rest for the browser to fill. Card refresh and download URLs carry the slot, so their routes read that slot and never the whole share record; a card no slot draws, as on a conversation share, downloads through the component export route. A component read refused with anything but 404 is transient, so a busy dfo-apis no longer withdraws a live link.
A snapshot card's citation dates from when its slot first served the frame it shows, read once from the share's history within the page budget, and falls back to the share's creation time. The share dialog shows a permanent refusal, or a share too large, without a retry. A lost write token no longer blocks account deletion, a token the site cannot keep answers a retryable 502, and the download-pin path encodes its segments.
The round trip orders share ids with collate "C", as the JavaScript sort it is compared with does, so ids of mixed case compare equal in either order.
This was referenced Oct 29, 2026
readShareHistory reads one 500-entry page and returns its next cursor, and servedSince follows the cursor until every framed slot has its entry or the history ends, within the share page's 4 s deadline. A cursor that repeats the one just sent counts as an invalid reply. A snapshot citation takes its date from the slot's served-since entry alone. When that entry is not found, the citation prints "(snapshot)" with no date; the share's creation date could name newer data.
…e-downs takeDownUserShares commits user.deleting_at before it lists the user's share tokens. putShareToken inserts through a select of the user row FOR SHARE that requires the fence unset (or older than 15 minutes, since Better Auth runs no hook after a failed user delete), so a share created during deletion either lands before the list or is refused and taken down by createOwnedShare's compensation. A failure that keeps the account clears the fence. A take-down that dfo-apis answers 204 or not_found is done. One refused with write_token_* is written to the new share_cleanup table, which has no foreign key to user, logged as an error, and deletion continues; an operator removes those shares with datasets-maintenance take-down-share. A failed compensation take-down is recorded there too. Migration 0023 adds the table and the column.
fenceShareTokens returns the attempt's fence value; takeDownUserShares refreshes that value before each take-down, re-lists tokens until none remain, and on failure lifts only a fence that still holds its value, so a failed attempt cannot open another attempt's fence. createOwnedShare reads the fence first and refuses a deleting account with 403 account_deleting before any dfo-apis call. A failed share_cleanup write logs share_cleanup_not_recorded and keeps token_not_kept. The share_cleanup_recorded log line is the alert for shares an operator takes down; deploy/README.md gives the drain steps. History reads 100 entries on the first page and stops on a repeated cursor. The token import reports a deleting owner as ownerDeleting.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Shares move to dfo-apis, and the website proxies the cards and shares surfaces and names each caller to dfo-apis. A share is created, replaced, switched to a snapshot and taken down through dfo-apis'
/api/presentation/sharesroutes with the owner's write token, whichshare_token(migration 0022) keeps beside the user;/s/<id>and/embed/<id>render from dfo-apis' share record, including shares made with the SDK, which draw one card per slot. Each slot draws through the slot card route and downloads come from the slot data route in the format and bundle the reader asks for, so the page, its files and the SDK'sfetch(share)agree. A snapshot shows what each slot showed at the switch and cites the time its slot first served that frame, read by paging the share's history; when that time cannot be found the citation gives no date. A share page waits at most 4 s for its slot cards and leaves the rest for the browser.Deleting an account takes the user's shares down in dfo-apis first, fenced against shares made meanwhile. The attempt sets
user.deleting_at, refreshes it before each take-down and re-lists tokens until none remain, andputShareTokenstores a token only while no fence holds (migration 0023). A deleting account's share create answers 403account_deletingbefore calling dfo-apis. A share whose token dfo-apis refuses is kept inshare_cleanupand logged asshare_cleanup_recorded, the alert an operator acts on with dfo-apis'datasets-maintenance take-down-share, asdeploy/README.mdsets out.The proxy (
src/lib/proxy) forwards/api/v1/datasets,/presentation/componentsand/presentation/shares, and sendsx-dfo-caller, an HMAC of the client address with IPv6 folded to /64, together withx-dfo-caller-principal, so dfo-apis counts allowances per visitor. Browsers cannot set either header. Every forwarding route builds its upstream path throughforwardablePath, which refuses dot segments and encodes each segment. Deploying needsCALLER_HMAC_KEYequal to dfo-apis'DFO_CALLER_HMAC_KEYandDFO_CALLER_PRINCIPALequal to its principal;LIVE_REFRESH_PRINCIPALis gone.Existing shares move with
scripts/export-shares.ts, dfo-apis'datasets-maintenance import-shares, andscripts/import-share-tokens.ts, asdocs/share-export.mdsets out; each step can run again without changing anything. This merges after the dfo-apis PR is deployed, since every share call goes to routes that PR adds.Left for a follow-up: the old server-side refresh stack (
src/lib/studio/liveShare.ts,liveShareResults.ts, the refresh and render parts ofliveBackend.ts, their tests, theshare_card_resulttable, and thelive_unavailablewithdrawal inshareDialog.ts) is unreachable and stays until production runs the new path.Testing: the CI jobs, including
pnpm test(2130 passed, against Postgres 18, including the deletion race on a held row lock) and thebuild:migrateandbuild:export-eventssmoke runs, asci.ymlruns them. Against a local dfo-apis stack,scripts/share-migration-round-trip.tsseeded one share per legacy kind, exported and imported them, rendered/sand/embedfor each, wrote as the owner with the imported token, and found a second run of both imports changed nothing; every check passed. The SDK's Python share journey ran through this proxy (3 of 3), and dfo-apis counted the allowance under the website's caller digest./code-review xhighfound 15 issues, and a second review of the deletion fence and history paging found 7 more; all are fixed on this branch.