Skip to content

Manual placement: log the --fit verdict the launch actually carries - #10831

Open
deepspace28 wants to merge 4 commits into
unslothai:mainfrom
deepspace28:fix/manual-fit-decision-log
Open

Manual placement: log the --fit verdict the launch actually carries#10831
deepspace28 wants to merge 4 commits into
unslothai:mainfrom
deepspace28:fix/manual-fit-decision-log

Conversation

@deepspace28

Copy link
Copy Markdown

Summary

Follow-up to my report in #10821. In Manual GPU-memory mode with an explicit layer count, the backend's GGUF decision line interpolated use_fit before the Manual branch turned it off — so every Manual load logged --fit: on while the launch it described six milliseconds later carried --fit off. Paired with the deliberately emptied GPU probe, the two lines read "failed GPU enumeration, proceeding unprotected" instead of "Manual mode, the user owns placement".

Fix

load_model decides Manual placement from a condition that is knowable well before the log: the function arguments gpu_memory_mode and gpu_layers. So the change:

  1. Hoists the verdict to before the decision line, from the same predicate the launch branch applies (gpu_memory_mode == "manual" and gpu_layers >= 0), so the log can no longer disagree with the argv.
  2. Labels a deliberately empty list — GPUs free: [] (manual placement) — so an intentional stand-down is distinguishable from a failed probe at a glance. The dedicated empty-probe warning (correctly gated on _detected_gpus) keeps catching the genuinely failed case.
  3. Re-asserts use_fit = False in the Manual launch branch, so the placement try/except's --fit on fallback can never override Manual between the hoist and the launch.

A side effect worth noting: _placement_verdict_partial reads use_fit right after the log, so under the old order a Manual load recorded a partial-placement verdict ("the planner ran and could not fit"). With the hoist it reads the Manual override — False, which is the correct claim for a mode where nothing was fitted.

Tests

New studio/backend/tests/test_manual_mode_decision_log.py — source-level on load_model, since the log site cannot be driven from a unit test:

  • Manual turns the fit off above the decision line (this trio fails on main with the order reverted)
  • The hoist consults the same predicate the launch branch applies
  • An empty probe can carry the (manual placement) label
  • The launch branch still re-asserts use_fit = False before emitting --gpu-layers, so the placement fallback cannot leak into Manual
  • The partial-placement verdict reads the post-hoist flag
  • Manual + Auto layers (the fitter-carrying sibling branch) stays untouched

All 6 pass; test_empty_gpu_probe_reason.py and test_log_filter_no_truncation.py stay green (23 + 9).

Closes #10821

…nslothai#10821)

The GGUF decision line interpolated use_fit before the Manual branch set it
False, so every Manual-mode load logged --fit: on while spawning --fit off
(unslothai#10821). Decide Manual placement before the log from the same predicate the
launch branch applies, label an empty probed list with (manual placement) so
it reads as a discarded-on-purpose enumeration rather than a failed one, and
re-assert use_fit=False in the launch branch so the placement try/except's
--fit-on fallback can never override Manual between the two.

_source-level tests pin both the hoist-above-log order and the re-assert.
@deepspace28

Copy link
Copy Markdown
Author

@pre-commit-ci[bot] re-run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Manual GPU mode logs --fit: on while passing --fit off, making a normal launch look like a failed GPU probe

1 participant