Surface Monarch's Flexible budget bucket in get_budgets, with a safe fallback

[alex-zwingli]

Fixes #103.

Under Monarch's fixed_and_flex budget system, the Flexible section carries a single amount covering every category beneath it. get_budgets never returned that number — only the individual categories inside the bucket — so spending-vs-budget for Flexible could not be computed from the tool's output.

Why it was missing

BUDGET_QUERY requested only budgetData { monthlyAmountsByCategory }. The query had been deliberately narrowed to avoid categoryGroups fields (budgetVariability, rolloverPeriod) that Monarch rejects for some accounts.

That narrowing cut wider than it needed to. The rejected fields are on the top-level categoryGroups selection; the flex data lives under budgetData, a different subtree. So the flex fields can be restored with the categoryGroups selection left byte-for-byte unchanged — which is what this PR does.

Approach: extended-first, with a safe fallback

Rather than change BUDGET_QUERY, this adds BUDGET_QUERY_FLEX alongside it — the same document plus monthlyAmountsForFlexExpense and totalsByMonth. It is tried first; if Monarch rejects those fields, the original narrow query runs and the tool behaves exactly as it does today.

The important detail is which failures count as "unsupported", and there are two traps here worth calling out.

1. Monarch's rejection text is generic. A refused field does not come back with standard GraphQL wording — probing the live API with a deliberately unknown field returns:

{'message': 'Something went wrong while processing: None on request_id: None.'}

So matching on Cannot query field / Unknown field would never fire against the real service. Detection keys on the exception type instead: gql raises TransportQueryError when the server answers with GraphQL errors, while HTTP failures raise TransportServerError and connection problems raise their own types. Text markers are kept only as a backstop for transports that do use the usual wording.

2. A failure that hits both queries isn't evidence about flex. The unsupported latch is set only after the narrow query succeeds:

except Exception as exc:
    if not _is_query_rejection(exc):
        raise                    # HTTP / connection -> surface it
    rejection = exc              # not latched yet

data = await client.gql_call(... BUDGET_QUERY ...)   # raises -> propagates
if rejection is not None:
    _flex_supported = False      # only now: flex failed, narrow worked

An expired session refuses both queries. Latching on the first failure would disable flex for the rest of the process because of an auth problem; this way it propagates and flex stays un-probed. An account that genuinely lacks the fields still pays only one failed round-trip per process rather than one per call.

Also confirmed while designing this: @include(if: false) does not let a single document carry optional flex fields — GraphQL validates the whole document before directives apply, and the live API rejects it. Two documents are genuinely required, so both render from one template to keep the categoryGroups selection structurally identical.

Output shape

get_budgets now returns an object instead of a bare list:

{
  "tool": "get_budgets",
  "args": {"start_date": null, "end_date": null},
  "data":   [ "... unchanged per-category rows ..." ],
  "budget_system": "fixed_and_flex",
  "flex":   {"status": "ok", "budget_variability": "flexible", "monthly": [ "..." ]},
  "groups": [ {"id": "...", "name": "...", "planned": 0, "actual": 0, "month": "..."} ],
  "goals":  [ {"id": "...", "name": "...", "archived": false,
               "planned_contributions": [], "actual_contributions": []} ],
  "totals": [ {"month": "...", "income": {}, "expenses": {},
               "flexible": {}, "fixed": {}, "non_monthly": {}} ]
}

Each per-category row in data also gains set_aside, rollover and rollover_type.

Two further gaps above the per-category level

Auditing what else Monarch returns above individual categories turned up two more omissions, both in data the API already sends:

• totalsByMonth.totalIncome and .totalExpenses were never requested. Income was missing from the tool altogether — only the three expense sub-buckets were mapped. On a live account both come back fully populated.
• monthlyAmountsByCategoryGroup was never requested. Groups budgeted at the group level rather than per category hold their amount there; it now surfaces as groups.

Both ride the same extended selection and therefore the same fallback, so an account that refuses these fields still gets every category row with flex/groups/totals marked unavailable.

groupLevelBudgetingEnabled is deliberately not requested — it sits on categoryGroups, which this PR keeps narrow. Group names are resolved from the existing narrow selection instead.

One asymmetry worth recording so it isn't mistaken for a further gap: Flexible is the only pooled bucket. Fixed and Non-Monthly are budgeted per category, so their per-category rows in data were already complete and their totals are plain sums — which is why Monarch exposes a dedicated monthlyAmountsForFlexExpense and no fixed/non-monthly equivalent. That's noted in the tool docstring.

Rollover: per-category numbers that did not add up

A further pass over the per-category fields found the most consequential omission. Monarch returns previousMonthRolloverAmount and rolloverType per category; neither was requested. For any category with rollover enabled, planned - actual therefore did not equal remaining, and nothing in the response explained the difference — it just looked like bad data.

On a live account, 5 of 86 rows were affected. With rollover included:

rows: 86
  reconcile only once rollover is included: 5
  still unexplained:                        0

Also in this pass:

• plannedSetAsideAmount was already being requested and then silently dropped during formatting. Now mapped as set_aside.
• budgetSystem is now requested and returned as top-level budget_system (e.g. "fixed_and_flex"), so a caller can distinguish "this account does not use flex budgeting" from "the flex bucket is empty" rather than inferring it from an empty result.

These go into the extended query only, through new template slots. The narrow fallback stays byte-for-byte the document already proven to work, and test_fallback_query_requests_nothing_beyond_the_proven_set asserts none of the extended fields leak into it — that query is the safety net, so it must never become the thing that breaks.

flex.status is deliberately explicit, because silently omitting the field is indistinguishable from a flex budget of zero — and an LLM reading it that way would state a confidently wrong number:

status	meaning
ok	bucket present and populated
not_configured	query succeeded, but this account has no flex bucket
unsupported	Monarch rejected the flex fields; fallback query ran

This is the one breaking change: callers reading the old top-level list now read data. It matches the envelope get_transactions already returns, and the README is updated accordingly.

Write side

Added set_flexible_budget, wrapping the client's existing update_flexible_budget(). This is needed because the Flex bucket is not a category group — of the 18 category groups returned on my account, none corresponds to it — so set_budget_amount(category_group_id=...) cannot reach it by construction. It refuses early with a clear message when flex is known-unsupported, instead of surfacing a raw GraphQL exception, and is added to the README's approval-required list since it mutates the ledger.

Testing

Unit: 69 budget tests, up from 6. Full suite 284 passed.

Coverage spans the flex-present / not_configured / rejection→fallback paths; that a transient rejection recovers rather than sticking; that TransportServerError propagates instead of degrading; that a failure hitting both queries surfaces as an error; rejection detection against the real generic Monarch message rather than idealized GraphQL wording; both list and bare-object shapes for monthlyAmountsForFlexExpense and the refusal to merge multiple buckets; explicit JSON nulls at every level; the income/expense marker; group-level budgets; paired-date validation in both directions at both the tool and helper level; and every branch of the two mutating tools including their confirming-payload guards.

These were validated by mutation, not by inspection. Each of the following was applied to the implementation and confirmed to fail a test: swapping the two query documents (both as constants and at the call sites), returning used_flex_query=True on the fallback path, making json_error return a success-looking payload, hardcoding a wrong current_month_range, dropping budget_system= from the destructive client call, dropping the category_type key, removing the null-guards, and disabling either confirming-payload guard. Several of those mutations passed an earlier version of this test suite, which is why the tests were rewritten.

test_fallback_query_requests_nothing_beyond_the_proven_set guards the regression this PR is most careful about: the narrow document is the safety net, so no extended field may leak into it.

Full suite: 284 passed. One unrelated failure, test_secure_session.py::TestGetAuthenticatedClient::test_no_session_returns_none, is pre-existing on main — confirmed by stashing these changes and re-running.

Worth flagging separately, since it will not reproduce in CI: that test fails only on a machine where someone has actually authenticated. _TOKEN_DIR resolves from Path.home() at import, and the fixture fakes the keyring but not the file fallback, so the test reads the developer's real ~/.monarch-mcp-server/token and gets a live session where it expects None. Running it with HOME pointed at a temp dir passes. Happy to fix that in a separate PR — it's untouched here.

Live: verified against a real fixed_and_flex account throughout. get_budgets returns 86 category rows with flex.status="ok", a populated bucket, group roll-ups, goals and all five totals; every row reconciles once rollover is included (5 previously could not, 0 remain); an explicit date range resolves to the right month; and forcing the fallback still returns all 86 rows with flex.status="unsupported" and the extended keys null — the no-regression guarantee.

Neither mutating tool was executed against the live account, since both write to a real ledger. They are covered by unit tests only, which is the main limitation of this PR's verification.

Reviewed by four independent audits (correctness, test quality, GraphQL mapping vs the vendored client, docs consistency) over three rounds. Everything they raised in scope is fixed; findings outside this PR's scope were filed separately as #105, #106 and #107 rather than folded in here.

Goals

goalsV2 comes back from the same query and was not requested either. Goal contributions are precisely what plannedSetAsideAmount refers to, so a "what is spoken for this month" answer that ignores them understates the plan. Surfaced as goals, with planned_contributions and actual_contributions per month.

Archived and completed goals are included but flagged rather than filtered, so a caller can exclude them deliberately instead of never learning they existed. Requested without upstream's @include(if: $useV2Goals) guard, since the extended document already falls back wholesale if Monarch refuses any part of it.

update_flex_rollover_settings, with its defaults removed

Also exposed, since it is the fix for a Flex bucket that has accumulated a large negative rollover over many months.

It needs care. The client signature is (rollover_start_month=None, rollover_starting_balance=0.0, rollover_enabled=True), and its own docstring offers this as the usage example:

Example (reset flex rollover to $0 starting this month): await mm.update_flex_rollover_settings()

So a zero-argument call silently discards accumulated rollover. Inheriting those defaults would hand an LLM a no-argument reset button, so both destructive arguments are required at the tool boundary, the docstring states plainly what is discarded and asks that the current rollover be reported first, and it is on the README's approval-required list. A test asserts the two arguments stay required.

Deliberately left out

• get_cashflow_summary() — redundant. Verified against the live API that the existing get_cashflow tool already returns the same block (sumIncome, sumExpense, savings, savingsRate); this method returns that and nothing else. A second tool returning a strict subset of an existing one is surface an agent has to choose between for no gain.
• reset_budget() — clears a whole month of planned amounts with no undo. Different blast radius from the scoped, re-settable mutations here; it deserves its own discussion before being handed to an LLM.
• reset_budget() is the one upstream budget capability deliberately left unexposed, for the reason above.

Happy to follow up on any of these separately.

If you'd rather this were smaller

This is a large PR because the pieces share one query, one fallback path and one response shape, and splitting them means a rebase chain. But it is deliberately structured so it can be split, and I'm happy to do that instead of asking you to review it whole. Each commit is self-contained and the natural cut lines are:

Slice	Contents	Breaking
A	Extended query + fallback machinery + per-category rollover / rollover_type / set_aside — return stays a bare list, rows just gain keys	no
B	The {data, flex, groups, goals, totals, budget_system} envelope	yes
C	Write tools: set_flexible_budget, update_flex_rollover_settings	no

The one I'd most suggest, if you want only one: pull A out from under B. The rollover fix is the highest-value part — without it, planned - actual silently fails to equal remaining for any rollover category — and it needs no shape change at all. Landing A alone fixes real wrong numbers today, and leaves the list→object question to be settled separately on its own merits.

C depends only on A. B is the only piece anyone should have to argue about.

Two unrelated follow-ups I'd send separately rather than smuggle in here:

• The test suite reads the developer's real ~/.monarch-mcp-server/token (see Testing below) — a small fixture fix.
• The README's Claude Desktop launch command resolves mcp[cli] unpinned, which now pulls SDK v2 and breaks a cold-cache install (that's Server fails to start: ModuleNotFoundError: No module named 'mcp.server.fastmcp' #84; I opened Launch config: resolve from uv.lock via the monarch-mcp-server console script instead of an unpinned ephemeral env (root cause of #84) #102 with the details).

🤖 Generated with Claude Code

[alex-zwingli]

@robcerda This PR is ready for your review. It fixes some major gaps that prevent users from effectively budgeting with it. Thanks for getting started on this MCP server!

Let me know if you'd also like help maintaining this repo.

[robcerda]

Surfacing the Flexible bucket is clearly right and I like that the destructive rollover arguments are required rather than defaulted, which is the pattern I followed in #125 for update_category. It conflicts with main now, so please rebase and add README rows for any new tools.