Skip to content

Validate job options before writing the job script - #1837

Draft
Mohit-Ak wants to merge 1 commit into
MFlowCode:masterfrom
Mohit-Ak:fix/validate-job-options-before-job-script
Draft

Validate job options before writing the job script#1837
Mohit-Ak wants to merge 1 commit into
MFlowCode:masterfrom
Mohit-Ak:fix/validate-job-options-before-job-script

Conversation

@Mohit-Ak

Copy link
Copy Markdown
Contributor

run() currently does its option validation after it has already rendered and written the job script:

__generate_job_script(targets, case)
__validate_job_options()
__generate_input_files(targets, case)

__validate_job_options() raises for --no-mpi with more than one rank, nodes <= 0, tasks_per_node <= 0, or a malformed --email. Because it runs late, any of those aborts the run but leaves a freshly written <name>.sh in the case directory — and, since --clean and build() also run before it, a rejected invocation has by then already wiped the previous run's outputs and possibly triggered a build.

This moves __validate_job_options() to the top of run(), right after the targets are resolved and before the case file is loaded (loading executes case.py, so it's a side effect too). Validation only reads ARG(...), so the reorder has no other effect; valid invocations behave exactly as before.

Testing. Added toolchain/mfc/run/test_run.py, which drives run() under a sandboxed state.gARG with each rejected option combination and asserts that build, case.clean(), the job-script generator and the input-file generator were all not called, plus a control case that a valid invocation still reaches generation. On pristine master the four rejection tests fail (AssertionError: job script was written before option validation); with this change all five pass.

$ cd toolchain && python -m pytest mfc/run/test_run.py -q
5 passed
$ python -m pytest . -q          # the ./mfc.sh lint suite
375 passed, 8 warnings, 4 subtests passed in 8.68s
$ ruff check . && ruff format --check mfc/run/run.py mfc/run/test_run.py   # ruff 0.6.5
All checks passed! / 2 files already formatted

This is part (b) of #1511 only, per the issue's own suggestion to split it from the (a)/(c)/(d) tidy-ups.

Fixes #1511

run() rendered and wrote <name>.sh (and, with --clean, wiped the previous
outputs and kicked off a build) before __validate_job_options() had a chance
to refuse the invocation, so `--no-mpi` with several ranks, `nodes <= 0`, or a
malformed `--email` aborted the run but left a stale job script behind.

Move the validation to the top of run(), right after the case is loaded, so
an invalid invocation is rejected before any side effect. Add unit tests that
drive run() with each rejected option combination and assert that neither
build, case.clean(), the job-script generator nor the input-file generator
was invoked, plus a control that a valid invocation still reaches generation.

Fixes MFlowCode#1511 (part b).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

build/run/state cleanups: stale job script on invalid options, duplicated build parse, twin make_options/make_slug, quit shadows builtin

1 participant