AMR: the batched fine advance turns on by default where the case admits it, decided by the toolchain -- an amr case that leaves amr_batched_advance unset gets it with amr_device_pack and amr_bat_pad = 0.1 when a trial validation with them passes (the same prohibitions that guard an explicit request); the Fortran defaults stay off because a default set there bypasses the validator (ledger 99); case.md row and three validator unit tests

Author: sbryngelson

docs/documentation/case.md

@@ -734,7 +734,7 @@ To restart the simulation from $k$-th time step, see @ref running "Restarting Ca
 | `amr_subcycle`          | Logical | Advance the coarse level at the case dt and the fine level at dt/2 (two substeps; Berger-Colella refluxing). Requires `amr`; incompatible with `cfl_dt`. |
 | `amr_device_pack`       | Logical | Pack and unpack the per-stage coarse-patch gather (F1/F2) over the plan's flat transfer list instead of one launch per transfer. The sends fuse to one kernel per family per stage; the receives fuse per contiguous (box, peer) run, so measured at np=8 the pack dispatches fall about 89x and the unpack about 4.3x. Wire bytes and floating-point values are unchanged. Requires `amr`; incompatible with `amr_subcycle`; the non-polytropic QBMM pb/mv twin keeps its per-transfer path. Default F. |
 | `amr_batched_gather`    | Logical | Consume the per-stage coarse-patch gather (F1/F2) for all of a rank's blocks at once: the gathered patches live in one pool, and each wave runs one fused own-copy, one fused unpack and one batched ghost fill instead of one set of launches per block. Wire bytes and floating-point values are unchanged. Requires `amr` and `amr_device_pack`; incompatible with `amr_subcycle`. Default F. |
-| `amr_batched_advance`   | Logical | Advance owned fine blocks of equal level and extent in batches of up to 8, stacked two ghost shells apart along the last active dimension, in one RHS call per batch. Requires `amr`; lock-step, Cartesian, uniform grid only; incompatible with the per-block fine-advance hooks (relaxation, IB, QBMM, IGR, chemistry, hypoelasticity, bubbles, MHD, relativity, damage, surface tension) and with Riemann-extrapolation BCs under `null_weights`; requires `amr_max_grid_size` > 0. Bit-identical to the per-block advance on a grid whose cell spacing is bitwise uniform (stacked blocks share the batch leader's coordinate arrays); roundoff-level differences otherwise, announced once at startup. Default F. |
+| `amr_batched_advance`   | Logical | Advance owned fine blocks of equal level and extent in batches of up to 8, stacked two ghost shells apart along the last active dimension, in one RHS call per batch. Requires `amr`; lock-step, Cartesian, uniform grid only; incompatible with the per-block fine-advance hooks (relaxation, IB, QBMM, IGR, chemistry, hypoelasticity, bubbles, MHD, relativity, damage, surface tension) and with Riemann-extrapolation BCs under `null_weights`; requires `amr_max_grid_size` > 0. Bit-identical to the per-block advance on a grid whose cell spacing is bitwise uniform (stacked blocks share the batch leader's coordinate arrays); roundoff-level differences otherwise, announced once at startup. Default F. Left unset on an `amr` case, the toolchain turns it on (with `amr_device_pack` and `amr_bat_pad` = 0.1) whenever these rules admit it; set `amr_batched_advance = F` to force the per-block advance. |
 | `amr_max_blocks`       | Integer | Upper bound on the GLOBAL refined-block count. Sizes replicated per-rank METADATA (~11 kB/block); block slots themselves are allocated lazily for blocks a rank owns, so this is not N x device memory. Exceeding it silently truncates the refined region (the clusterer warns). Must be >= 1 (default 1024) |
 | `amr_max_grid_size`    | Integer | Absolute cap on a refined block's coarse-cell extent per dimension, the AMReX max_grid_size concept; must be >= 2 when set (default 0). With 0 the cap is derived from the decomposition and so shrinks as ranks are added, which tiles a fixed feature into more blocks the further you scale and makes the box set depend on the rank count. Setting it pins the cap, so the box set is identical at every rank count. The value may exceed half a rank subdomain: the solver scratch is then sized to the cap rather than to the subdomain, so per-rank memory grows as the cap raised to the number of dimensions |
 | `amr_max_level`        | Integer | Maximum AMR refinement depth (number of refined levels above L0); must be >= 1 (default 1). Multi-level nesting (>= 2) is supported: static AMR (`amr_regrid_int = 0`) nests up to level 2, dynamic regrid (`amr_regrid_int > 0`) nests deeper (see @ref amr_multilevel) |

toolchain/mfc/case_validator.py

@@ -3212,6 +3212,28 @@ def _format_errors(self) -> str:
         return "\n".join(lines)
 
 
+# The batched fine advance and its two companions, turned on for an AMR case that leaves amr_batched_advance unset
+# whenever the case admits it. The Fortran defaults stay F on purpose: a default set there bypasses every rule above
+# (ledger 99: ten of 58 AMR cases ran the documented unsupported combinations unguarded), so the decision lives here,
+# under the same prohibitions that guard an explicit amr_batched_advance = T.
+BATCHING_DEFAULTS: Dict[str, Any] = {"amr_batched_advance": "T", "amr_device_pack": "T", "amr_bat_pad": 0.1}
+
+
+def apply_batching_default(params: Dict[str, Any]) -> bool:
+    """Set the batching defaults in place when the case is an AMR case, leaves amr_batched_advance unset, and passes
+    simulation validation with them on. Returns True when they were applied."""
+    if params.get("amr", "F") != "T" or "amr_batched_advance" in params:
+        return False
+    trial = dict(params)
+    trial.update({k: params.get(k, v) for k, v in BATCHING_DEFAULTS.items()})
+    try:
+        validate_case_constraints(trial, "simulation")
+    except CaseConstraintError:
+        return False
+    params.update({k: params.get(k, v) for k, v in BATCHING_DEFAULTS.items()})
+    return True
+
+
 def validate_case_constraints(params: Dict[str, Any], stage: str = "simulation") -> List[str]:
     """Convenience function to validate case parameters
 

toolchain/mfc/run/input.py

@@ -123,6 +123,8 @@ def validate_constraints(self, target) -> None:
 
     # Generate case.fpp & [target.name].inp
     def generate(self, target) -> None:
+        if case_validator.apply_batching_default(self.params):
+            cons.print("[dim]amr_batched_advance left unset and admissible: batching on (amr_batched_advance = F forces the per-block advance)[/dim]")
         # Validate constraints before generating input files
         self.validate_constraints(target)
         self.generate_inp(target)

toolchain/mfc/test_case_validator.py

@@ -9,7 +9,7 @@
 
 import unittest
 
-from .case_validator import CaseConstraintError, CaseValidator
+from .case_validator import CaseConstraintError, CaseValidator, apply_batching_default, validate_case_constraints
 
 # A minimal 1D case that passes simulation validation.
 BASE = {
@@ -387,5 +387,33 @@ def test_not_tripped_without_alt_soundspeed(self):
         self.assertNotIn(self.MSG, self.errors_for({**TWO_FLUID, "riemann_solver": 4}))
 
 
+class TestBatchingDefault(unittest.TestCase):
+    """The batched advance turns on by default only where the validator would admit an explicit request."""
+
+    AMR = {**BASE, "amr": "T", "amr_regrid_int": 0, "amr_max_grid_size": 16, "time_stepper": 3, "amr_block_beg(1)": 0.25, "amr_block_end(1)": 0.75}
+
+    def test_admissible_amr_case_gets_batching(self):
+        p = dict(self.AMR)
+        self.assertTrue(apply_batching_default(p))
+        self.assertEqual(p["amr_batched_advance"], "T")
+        self.assertEqual(p["amr_device_pack"], "T")
+        self.assertEqual(p["amr_bat_pad"], 0.1)
+        validate_case_constraints(p, "simulation")
+
+    def test_prohibited_combination_stays_per_block(self):
+        for k in ("ib", "igr", "stretch_x"):
+            p = {**self.AMR, k: "T"}
+            self.assertFalse(apply_batching_default(p), k)
+            self.assertNotIn("amr_batched_advance", p, k)
+
+    def test_explicit_setting_and_non_amr_untouched(self):
+        p = {**self.AMR, "amr_batched_advance": "F"}
+        self.assertFalse(apply_batching_default(p))
+        self.assertEqual(p["amr_batched_advance"], "F")
+        q = dict(BASE)
+        self.assertFalse(apply_batching_default(q))
+        self.assertNotIn("amr_batched_advance", q)
+
+
 if __name__ == "__main__":
     unittest.main()