Repository navigation
Commit 0923d8a
BUG: re-raise KeyboardInterrupt after an interrupted Monte Carlo run (#1177)
* BUG: re-raise KeyboardInterrupt after an interrupted Monte Carlo run
simulate() returned normally after Ctrl-C. Both execution modes caught the
interrupt and neither re-raised it, so simulate() went on to
__terminate_simulation() and returned exactly as it does after a complete
study. A caller could not tell a partial run from a finished one without
opening the output file and counting rows.
Five more holes sat next to that one, found in review and in the
interrupt-point audit it prompted:
- __run_in_serial bound inputs_json inside the loop body, so Ctrl-C during
the first keep_simulating() call reached the handler with nothing bound
and the run died with UnboundLocalError from inside the cleanup.
- It also never cleared inputs_json after a successful append, so Ctrl-C
landing in the progress print — or in the next keep_simulating() call —
wrote the row that had just committed into the error file as though it
never finished.
- _append_simulation_record rolled back on Exception, and
KeyboardInterrupt does not derive from Exception, so Ctrl-C between the
two appends left a one-sided record.
- Its rollback also only truncated the inputs file, so an interrupt inside
either write left a torn partial row for the reload to die on with
JSONDecodeError instead of the interrupt.
- The parallel cleanup began only after every worker had started, so
Ctrl-C during the startup loop left the already-started workers running
with nobody signalling or joining them.
Bind inputs_json before the loop and clear it after each committed append,
roll both files back on BaseException best-effort, cover the startup loop
with the same cleanup path over a started-workers list, and re-raise in
both handlers. Catch the interrupt in simulate() so
__terminate_simulation() still runs before it leaves: it reloads the logs
through the file setters, and set_num_of_loaded_sims is what the
documented append=True continuation reads.
The shutdown join stays unbounded, and the docstring now says so: the
interrupt propagates once every worker finishes the simulation it is in
and exits on its own. A worker stuck inside one simulation blocks the
interrupt as it already blocked the run; killing it here could tear a
half-written row into logs it holds the mutex for, and the bounded fleet
shutdown belongs to #1054. The ordinary exception path is unchanged.
Add ten regression tests covering both modes, the early interrupt, the
preserved rows, the reload, an interrupted run continued with append=True,
the between-appends rollback, the torn-write rollback, the committed row
staying out of the error file, and the startup-loop cleanup. They run the
real __terminate_simulation and assert the state it produces; all fail on
develop, and each fix was also reverted individually with only its own
test failing.
* BUG: preserve interrupted Monte Carlo cleanup
---------
Co-authored-by: Gui-FernandesBR <guilherme_fernandes@usp.br>1 parent d864d4d commit 0923d8a
2 files changed
Lines changed: 469 additions & 15 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
321 | 321 | | |
322 | 322 | | |
323 | 323 | | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
324 | 329 | | |
325 | 330 | | |
326 | 331 | | |
| |||
349 | 354 | | |
350 | 355 | | |
351 | 356 | | |
352 | | - | |
353 | | - | |
354 | | - | |
355 | | - | |
356 | | - | |
357 | | - | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
358 | 364 | | |
359 | 365 | | |
360 | 366 | | |
| |||
410 | 416 | | |
411 | 417 | | |
412 | 418 | | |
| 419 | + | |
| 420 | + | |
| 421 | + | |
| 422 | + | |
413 | 423 | | |
414 | 424 | | |
415 | 425 | | |
416 | 426 | | |
417 | 427 | | |
418 | 428 | | |
419 | 429 | | |
420 | | - | |
| 430 | + | |
421 | 431 | | |
422 | 432 | | |
| 433 | + | |
| 434 | + | |
423 | 435 | | |
424 | 436 | | |
425 | 437 | | |
| |||
435 | 447 | | |
436 | 448 | | |
437 | 449 | | |
| 450 | + | |
438 | 451 | | |
439 | 452 | | |
440 | 453 | | |
| |||
445 | 458 | | |
446 | 459 | | |
447 | 460 | | |
| 461 | + | |
448 | 462 | | |
449 | 463 | | |
450 | 464 | | |
451 | 465 | | |
452 | 466 | | |
453 | 467 | | |
454 | 468 | | |
455 | | - | |
456 | | - | |
| 469 | + | |
| 470 | + | |
| 471 | + | |
| 472 | + | |
457 | 473 | | |
458 | 474 | | |
459 | 475 | | |
| |||
530 | 546 | | |
531 | 547 | | |
532 | 548 | | |
533 | | - | |
534 | | - | |
535 | | - | |
| 549 | + | |
| 550 | + | |
| 551 | + | |
| 552 | + | |
536 | 553 | | |
537 | 554 | | |
538 | 555 | | |
539 | 556 | | |
540 | | - | |
541 | | - | |
542 | | - | |
| 557 | + | |
543 | 558 | | |
544 | 559 | | |
545 | 560 | | |
| |||
0 commit comments