Repository navigation
feat(mcp): a leaner public tool list, bare spec_id frames, relative years, one refusal envelope and export_data - #124
Merged
Merged
Conversation
The nine analysis verbs send a one-node recipe tree to POST /api/datasets/analysis, and resolve_anchor asks POST /api/datasets/analysis/anchor for the latest period, so the MCP, the website and the SDKs share one engine. Replies are built from the runner's summary: on every recorded tool call the digest text and structured content match the in-process tools. Runner refusals map to the existing ToolRefusal codes, a pending root replies service_unavailable with the runner's retry time, and any other code replies upstream_error naming it. The website's caller digest is forwarded with the caller principal. presentation-mcp reads an ops_ frame from the platform when no publish principal is configured. The in-process engine and tool bodies stay until the production recipe diff passes.
…uted them Studio's harness reads a verb result from the shared store when it holds the frame and otherwise from dfo-apis: publish_result takes the derived manifest's recipe and replay key and publishes nothing, chart work reads the derived manifest and Parquet file, rolling cards and recipe replay read stored recipes from dfo-apis, so Studio works on frames the runner computed. rendering_not_recorded is a stop code. Harness tests run verbs through the recorded runner replay; a strict xfail pins the runner refusing a rolling call over one side of a paired population.
A frame fetched with a breakdown or a stat_vars sum is a verb input again: runner_tree copies the mint record's groupby into the root's rendering, which dfo-apis reads through the same head rendering as a definition read. Only mint options dfo-apis cannot read still refuse rendering_not_recorded. The harness decodes a derived file as the flat frame-layout Parquet dfo-apis now serves. Both packages' runner replies are recorded again against dfo-apis.
A rolling growth, rank or filter sends the roots resolve_anchor was asked about as anchor_roots, built as the tree's own roots are, so dfo-apis takes the latest period over the same population; a key with no stored anchor record refuses rolling_anchor_unresolved before any request. recipe_replay takes a stored tree's period over its anchor_roots. data_mcp's Parquet frame decoder is public as frame_from_parquet_bytes. Runner replies are recorded again.
… exact The harness decodes a derived file whose unit is composed, such as a quotient of dollars by people, so a chart over a quotient_of result reads its series; a decode failure is logged with the frame pair. A retryable recipe read withholds a rolling card as pending instead of fixing its periods, and describe_frame answers it as transient. analysis-mcp sends a root's columns sorted and once each, as dfo-apis writes them; refuses a stored anchor record naming a derived root; maps a lost root behind a derived input to derived_input_unrecoverable on that input; and refuses a failed mint record read as retryable. presentation-mcp sends its first-party principal on card create and revise, so harness conversations spend no caller's card allowance. The recorder starts each recording from an empty fixture, and both packages' runner replies are recorded again.
Comments and docstrings for the runner client, verbs, derived frames and recipe replay read as plain statements, and the recorder names the dfo-apis sync it needs.
Every package pins spec-kit 03d8f5a. The harness classes the new presentation refusal codes, frame_codec reads the stored two-member unit layout as spec-kit's Frame.from_table now does, and both runner reply fixtures are recorded against dfo-apis 605f87bed.
…al dfo-apis Runner reply fixtures in analysis-mcp and the harness are recorded against dfo-apis e152d0b2d, whose output digest names a computed variable by what was computed.
presentation-mcp tells Studio's card writes from outside MCP clients by transport: a call through the harness's loopback mounts sends the first-party principal, and any other sends the website-vouched caller under DFO_CALLER_PRINCIPAL, so dfo-apis counts that client's card allowance; with no named client, outside writes share one allowance. Every rolling call checks its population and the roots its answer names against the anchor record. A lost anchor root refuses rolling_anchor_unresolved, a lost root names only the inputs read from it, and two sides over one version recover in one refusal. Derived reads take their own 15 s timeout, summaries ignore fields dfo-apis adds and refuse malformed parameters as upstream_error, and join and filter summaries keep only what the digest shows. make_charts reads each derived frame once. The docs and .env.example state both principals as they work.
CALLER_PRINCIPAL_HEADER and ADDRESS_CALLER live in common.caller, which analysis-mcp's runner_client and presentation-mcp's client both import. .env.example and the analysis-mcp README document DERIVED_READ_TIMEOUT_S (15 s). The runner recorder passes version_writer to DerivedReads, which dfo-apis requires, and both runner_replies.json fixtures are re-recorded against dfo-apis 1f28a9405. Only trace_id values changed.
data-mcp's frame_ref_for puts "without_party_totals": true in the cv_ preimage when the cut is set and sorts groupby.by, matching dfo-apis; both repos pin cv_bb722d06259f357e6961a081f1c1f141 for one input. mint_recovery tries each candidate with and without the cut, so refs named before this change still recover. MintRecord sorts options.groupby.by and get_data sends by sorted, so one breakdown has one recipe normal form in both repos. The runner recorder resolves the cut the way dfo-apis does, and both runner_replies.json fixtures are re-recorded against dfo-apis b8b270ea9 with the same status mix: analysis-mcp 83 replies, harness 164.
b7a40ad names minted variables mv_ plus a fixed-size digest and admits a derived ref root that carries the groupby DFO read it with.
recipe_replay checks a breakdown root against its record's cut and against each order of groupby.by named_groupbys lists: sorted, as recorded, then every order up to four axes. mint_recovery tries the same orders, so pairs named before the sort recover. A mint record keeps its options as written and refuses a malformed groupby; data-mcp sorts by where it writes options, so a stored recipe's normal form does not change when it is read back.
0cb55e3 adds a nested minted-name vector with name documents and records the grouped-root ref vectors from a dfo-apis run, with by sorted.
# Conflicts: # services/harness/src/harness/orchestrator_tools.py # services/harness/tests/executor/test_wide_frame.py
…im the declared schemas presentation-mcp refusals carry code, retry_mode, message and fix in structuredContent and the same facts in content, with isError. common.declaration.declared_schema applies the declaration rules to each tool's schema at mount and refuses a release whose references it cannot inline. A create or describe_frame naming a bare spec_id refuses frame_ref_incomplete.
…ared schemas get_skill takes path, offset and digest and, with a path, returns what get_skill_file returns, through one implementation; paging lines name get_skill. common.declaration applies the declaration rules to every data-mcp tool at registration.
…are_of, and shape the public list compare takes operation difference or sum, and share_of takes share or quotient, each form keeping its own engine path. Frame arguments are typed objects. The combined server hides sum_of, quotient_of, get_skill_file and resolve_anchor and answers the removed names with the call to use, returns schema failures as request_invalid with problems and accepted, strips the rolling arguments from the public growth, rank and filter, and gives the public get_data its type columns. A ratchet test bounds the public list in o200k tokens.
Each child server exports PUBLIC_SHAPES: paragraphs the public description adds, properties it hides and arguments it forces. The combined server applies them to every tool alike, replacing sentence matching. Text about a hidden argument lives in that argument's own description. Re-pins search, get_data, growth, rank and filter, re-records the context baseline, and widens the schema and naming checks to the whole public list.
export_data prepares the file through POST /api/datasets/exports, forwarding the caller, and returns the public link on DFO_SITE_ORIGIN with the file's facts, citation and licences. Refusals map to the shared envelope.
# Conflicts: # services/data-mcp/src/data_mcp/server.py
# Conflicts: # services/analysis-mcp/src/analysis_mcp/server.py # services/analysis-mcp/src/analysis_mcp/tool_pipeline.py # services/analysis-mcp/src/analysis_mcp/verbs.py # services/harness/src/compiler/capabilities.py # services/harness/tests/test_combined_mcp.py
The pin adds the rebuild refusals on datasets.exports and spec-kit dev's presentation.shares import codes; the harness classes share_import_not_permitted as unavailable and share_id_taken as a system fault.
datasets.exports never answers not_found for a frame, so a 404 not_found from the prepare route means the route is absent.
…to calls that roll over the tree recipe_not_found on a derived input becomes rolling_reanchor_unavailable, and inputs are read for the computed-again sentence, only when the request sent rolling "tree"; harness calls keep frame_not_held and make no manifest reads. An input whose manifest cannot be read is logged and left out. A wire test pins a public rolling rank with place_level.
…lic server _PublicArguments raises the unexpected-keyword error for any argument a tool's PublicShape hides, forced or not, so the caller gets request_invalid with the public argument list and the child never validates its hidden schema.
… from The public get_data names its frame with a content_version dfo-ai derives from the columns, which dfo-apis never stored. export_data maps a spec_ pair through its mint record to the dataset pair, sends that, and names both the caller's frame and the exported version.
…unforced ones A public call that sends place_types or a verb flag keeps working, with the forced value written over it; rolling and anchor_key still refuse as unknown. The get_skill_file redirect names the path argument.
…n public calls describe_frame and write_component create fall back to the bare spec_id route when dfo-apis no longer holds the pair, and name both the version asked for and the one read. A hidden rebuild_lost_frames argument, forced true by presentation-mcp's PUBLIC_SHAPES, limits this to the public server; the Studio mounts still refuse frame_unavailable.
The public get_data sends enrichment "default" and reads the columns from the one rendering the mint served, so a flow identity that binds no role gets its party columns and dfo-ai no longer derives columns from the identity. The party-kinds check reads the frame's coverage roles. Frame names for frames that worked before are unchanged.
…e helper common.materialisation.dataset_pair holds the rule export_data and presentation-mcp's PlatformAssembler both applied.
A frame whose mint record carries a groupby can share its base version, so exporting the record's dataset pair would hand over unsummed rows. export_data maps only records that add columns or cut party totals, and refuses the rest with capability_unavailable naming the aggregate route.
…calls describe_frame and write_component create, with rebuild_lost_frames set, answer a 404 frame_unavailable by reading a spec_ id at its newest version or replaying an ops_ frame's recipe on current data, and name both versions. common.derived_replay holds the replay call analysis-mcp also uses.
frame_mint_record holds the spec_ guard and failure logging; dataset_pair and export_data's summed-frame check both read through it.
jm-rivera
changed the title
common.declaration collapses a nullable only where the property states default null, so a presentation patch can still clear a field with null. The public list measures 18,018 o200k tokens. A malformed bare spec_id refuses as an argument error. request_invalid names the tool's own arguments on export_data and get_data. place_levels_mixed advises place_level only on tools that take it, and a public relative year with no latest year asks for fixed years in place of the hidden resolve_anchor. frame_ref_incomplete caps the id it echoes, through truncate_value, now in common.echoed_text.
common.materialisation.rendering answers a frame's rendering from its mint record. analysis-mcp, presentation-mcp and export_data all recover or export through it, so a lost frame keeps its groupby and party-total cut. A frame whose rendering cannot be established refuses rendering_not_recorded, and a record that cannot be read refuses retryably. A bare read records the pair it resolved, so a later call can name that pair. presentation-mcp rebuilds only a 404 frame_unavailable whose cause.code is unknown, expired or ops_unsupported. A revoked, quarantined or corrupted frame stays a refusal. A rebuilt frame's sentence names the whole pair, since a replay can mint a new ops_ id. A transient failure of the resent call keeps its delay. A pooled denominator keys by place in a place_level step. The export_data digest states the SHA-256.
# Conflicts: # services/presentation-mcp/src/presentation_mcp/client.py
test_every_refusal_code_a_server_writes_is_in_the_pinned_registry reads the codes of data-mcp, analysis-mcp, presentation-mcp and the combined server from their source and refuses any the registry lacks. The spec-kit pin moves to 52e4e42, which registers the codes those servers write.
…ilt inputs A bare read's mint record holds enrichment default, so a later verb on the pair reads the type columns a get_data pair holds. The digest reads a rebuilt input at the pair the replay computed, and mint record writes share one bound, so a stalled store leaves a computed answer standing.
Skill text names get_skill_file on the mounts, and a hidden argument the public shape forces names get_skill. A two-unit share refusal is rewritten on public calls only and keeps its numerator and denominator. A call with no anchor key is told which derived inputs recompute. Echoed ids, arguments and tool names are capped everywhere, and a failed resolve_definition names that tool.
The spec-kit pin moves to 2dd2400, which maps every code the servers write onto mcp.tools, so the registry test now checks that mapping in place of plain membership.
… own refusal The public server registers a bare spec_id's stored definition before the tool runs, so an id fresh from search works on the verbs, describe_frame, write_component and export_data. A resend after a rebuild keeps its own refusal, adding not_rebuilt only when it names a frame still lost. A failed bare read refuses frame_not_held or frame_expired naming the bare frames. Spec reads map to the ids sent through the recipe's tree, so a served id under another name is recorded, and root records decode leniently. Public digests head compare and share_of by the public tool and its operation.
…ters A bare read writes the plain default record, the same whichever of it and get_data writes first, and a public verb over a plain record sends a root that takes the default enrichment. A Studio call and a replay read as before. Refusals keep their pointer in fix, and _problems caps the validator's message. A transient registration failure lets the tool run, a 4xx refuses naming the calling tool, and ids register together under a process memo. export_data's prepare deadline takes the time left in the call. The digest says when an input's latest year could not be confirmed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The public MCP server at
/mcplists 16 tools at 18,018 o200k tokens, down from 19 tools at 27,054. It addsexport_data, takes a barespec_idwherever a frame goes, rolls windows over relative years, and gives every refusal one envelope whosefixnames only arguments the tool accepts.Surface
comparetakesoperation(differenceorsum) andshare_oftakesoperation(shareorquotient). Each form runs today'ssum_oforquotient_ofpath with the same frame id, digest and refusal codes. A share refusedcross_frame_unit_mismatchbetween two plain units setsfixtooperation: "quotient".get_skilltakespath,offsetanddigest, and with apathreturns whatget_skill_filereturns.sum_of,quotient_of,get_skill_fileandresolve_anchorstay on the per-server mounts the Studio harness calls, and leave the public list. A call to one of the first three refusestool_not_foundwith afixnaming the merged tool and the argument to set.common.declarationapplies the declaration rules to every tool at registration. It removes$comment, emptyrequiredand the null branch of a property that statesdefault: null, inlines every definition, and keeps each sentence once. It refuses a schema whose references it cannot inline.PUBLIC_SHAPES): paragraphs the public description adds, properties it hides and arguments it forces. The combined server applies them to every tool alike. A public call that sends a hidden, unforced argument refuses it as unknown; a forced argument the caller sends is accepted and overwritten.get_dataforcesplace_typesand asks the dfo-apis mint forenrichment: "default", so every public frame carries the type and party columns of the roles its version holds; dfo-ai reads the columns from the rendering the mint served.rank,aggregateandshare_oftakeplace_level, sent to dfo-apis as a filter step under the verb._PUBLIC_LIST_TOKEN_BOUND), and another holds every public schema to the declaration rules. A test holds every code the servers write to itsmcp.toolsmapping in the pinned registry.Frames
spec_idrequired,content_versionoptional. A barespec_id reads the newest version, the one the dfo-apis analysis runner's Spec root reads: the verbs send it as a Spec root and record the pair it names;describe_frame,write_componentandexport_datasend it to the dfo-apis routes that resolve it. The public server registers a bare id's stored definition first, so an id fresh fromsearchworks on all four paths. A bare id of any other kind refusesframe_ref_incomplete.spec_frame by fetching it again in the rendering its mint record states, anops_frame by replaying its recipe withlatest: true) and retries;describe_frameandwrite_componentcreate do the same for a 404frame_unavailablewhosecause.codeisunknown,expiredorops_unsupported, and each reply names the pair asked for and the one read.common.materialisation.renderinganswers a frame's rendering for all three. A frame whose rendering cannot be established, and in presentation-mcp a summed or cut frame, refusesrendering_not_recordedwith afixnamingget_data, since dfo-apis has no route that reads a Spec at a rendering outside an analysis tree. A bare read records the pair it resolved. The harness mounts keepframe_not_heldandframe_unavailable, so Studio runs its own replay. A hiddenrebuild_lost_framesargument, forced true byPUBLIC_SHAPES, carries the difference;common.derived_replayholds the one replay call.Rolling periods
growthandrankwindow bounds and thefilterthreshold value take{"anchor": "latest", "offset": n}beside a four-digit year. dfo-apis resolves the latest year from the tree, and the digest names it and the years it resolved.ops_input sendsrolling: "tree", so dfo-apis rolls those inputs' periods over the whole tree; the digest names any input computed again. A derived input with no stored recipe refusesrolling_reanchor_unavailable. The harness sends norollingfield and keeps its own populations.Refusals
code,retry_mode,messageandfixinstructuredContentand the same facts incontent, withisError, as data-mcp and analysis-mcp do. Advice moves into the message;_metakeeps the assembler's refusal for the harness, so Studio's refusal is unchanged.request_invalidcarriesfix{tool, problems, accepted}on every server, with dfo-apis pointers rebased onto the tool's own arguments. Tests sample every refusal code of data-mcp, analysis-mcp and presentation-mcp and check eachfixagainst the tool's declared schema.export_data
export_data(frame_ref, format, bundle)prepares the file through dfo-apisPOST /api/datasets/exports, forwarding the caller, and returnshttps://developmentfinance.org/api/datasets/exports/{export_id}. That link answers 302 to a five-minute signed URL and does not expire. The result and its digest carry the file's name, size, rows, columns, SHA-256, citation and licences. Aget_dataframe exports the stored version its mint record names, and the result names both. A summed or broken-downget_dataframe refusescapability_unavailableand names theaggregateroute, because the export route takes no groupby. A frame with no mint record refusesrendering_not_recorded. A frame dfo-apis no longer holds refusesframe_unavailablewith the call that registers it again; an export never moves to another version by itself.Order
2dd2400, the commit this PR and feat(datasets,presentation): export any frame to a lasting link, read frames by spec_id, and roll a pairing over both sides apis#194 pin, in spec-kit's dev history. A squash merge drops it, and both PRs then need re-pinning to the squash commit.rolling: "tree", and its export and bare-spec_idroutes are whatexport_data,describe_frameandwrite_componentcall, and itscause.codeonframe_unavailableis what presentation-mcp's rebuild reads; dev dfo-apis refuses therollingfield until then.