Skip to content

Stage 3 (Architecture & Design) takes ~1h50m wall time on a 1-module, 7-page demo slice; gate-check + closeout alone ~12 min #583

Closed

Description

@OlafFede
No description provided.

Activity

  1. MendixMau commented on Oct 28, 2026

    @MendixMau
    Owner

    Triage: DISCUSS track. Performance. UNVERIFIED: the issue has a title only (no body, logs or timings), and there is no project here to reproduce on.
    Problem: Stage 3 took about 1h50m of wall time on a 1-module, 7-page slice, and gate-check plus closeout alone took about 12 minutes.

    What I could check on master c28fc0d:

    • check_stage_3 (bin/gate-check.sh:1161) is file-presence only: fit-gap, design system, wireframes and blueprint, with no mxbuild and no mxcli. The per-stage check itself should take seconds.
    • Each run also does the full evaluation around it: the obligation check, the artifact check, the source ledger, a dashboard rebuild and doctor/stamp notes.
    • Runbook §1b rule 7 asks for a plain gate-check.sh run and then gate-check.sh --closeout (bin/gate-check.sh:78, :3306). That is two full evaluations per gate.
    • When closeout.sh runs directly, it calls gate-check.sh a third time for the verdict line (bin/lib/closeout.sh:396).
    • There is no per-check timing anywhere (no SECONDS/timing flag in gate-check.sh). Today nobody, reporter or maintainer, can tell which part of the 12 minutes is script time and which is agent turn time.

    Value: Medium, possibly High. If most of the 12 minutes is script time, every gate in every entry mode pays it. If most of the 1h50m is interview and agent drafting, the fix is in skill text, not in gate-check. We can't tell yet.

    Options:

    1. Measure first. Add an opt-in MXTK_GATE_TIMING=1 that prints seconds per section (stage checks, obligations, artifacts, source ledger, dashboard) to stderr. Ask the reporter to rerun one Stage 3 gate with it.
      • No change to behaviour or output when the flag is unset, in any entry mode. Cost: S.
    2. Evaluate once per gate. --closeout reuses the verdict of a plain run made seconds earlier, e.g. cached and keyed on the register's and artifacts' mtimes. closeout.sh stops spawning a third gate-check.
      • This halves or thirds gate cost in every entry mode.
      • Risk: a stale cache shows the wrong verdict. Mitigation: the key includes every file the gates read. Cost: M.
    3. Stage-scoped evaluation. gate-check.sh <dir> 3 skips other stages' checks, and skips the dashboard rebuild unless asked.
      • Biggest saving, but it changes what a stage-scoped run prints and when the dashboard refreshes, which other tools and agents read. Cost: M/L.

    Recommendation: Option 1, then decide between 2 and 3 on numbers, not guesses. Optimising without a breakdown risks speeding up the part that wasn't slow.

    Decision needed from you: Should we instrument first (1), or go straight to removing the duplicate evaluations (2)? Option 3 should probably wait for data. Reply "go option N" to build it.

    @OlafFede, if you can, please add the agent session's timeline: when Stage 3 started, each gate-check/closeout command, and how long each took. That tells us whether this is script time or agent time.


    Generated by Claude Code

  2. MendixMau commented on Nov 3, 2026

    @MendixMau
    Owner

    Thanks @OlafFede — closing as fixed. The Stage 3 speed work (#586, #587, #589, #590, #591) is on master. A re-run on a comparable medium-scope app (5 entities, 9 pages) took ~4 min wall time for Stage 3 end to end (three serial runs: 217–252 s), versus the ~1h50m reported here. Please reopen if you still see slow Stage 3 runs on current master.


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions