Skip to content

Commit 256355c

Browse files
committed
Ledger (82) correction: unallocated module arrays abort under present in every form, null components do not; m_amr re-audited under the corrected rule (comments updated)
1 parent 00caa29 commit 256355c

3 files changed

Lines changed: 33 additions & 7 deletions

File tree

docs/documentation/amr_action_plan.md

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -327,6 +327,30 @@ byte-identical to before. Landed on up/mega as ONE squashed lane commit (bbec045
327327
commits and the gated hashes stay on origin/task13/host-profile); no flag, since a compiler-lane default is not a behaviour
328328
change of the solver (bit-identity above).
329329

330+
**CORRECTION (same day, 17:30) to the audit rule above, before any other unit opts in.** Two sentences in this entry
331+
are wrong and are retracted: "declare-target allocatables are exempt" and "the 08da1931 abort was the chemistry-only
332+
COMPONENT q_T_sf%%sf". Microbenchmarks N1-N4 (amr-bench/ubench, outputs archived): a kernel naming an UNALLOCATED module
333+
allocatable array aborts under the clause in every form -- bare, ``declare target`` plain/link/enter/to, after an
334+
``enter data map(alloc:)``, from its own module or use-associated -- while a null allocatable or pointer COMPONENT of a
335+
module scalar or of a dummy runs fine (``scalar_field%%sf`` is a pointer). The amr_rvw "counter-example" was a misread:
336+
the kernel that names it is the ``if (cyl_coord)`` sibling in s_amr_restrict_overwrite_device_sf, and the launch trace of
337+
00a7c569 shows the launched kernel's 35 arguments hold no amr_rvw. So the 08da1931 abort was the bubbles-only weight/R0
338+
(declare-target via GPU_DECLARE, allocated only under bubbles_euler) named by the conversion kernel. **Corrected rule for any unit that opts
339+
in: every kernel that names a conditionally allocated module array must launch only under that array's own allocation
340+
condition.** m_amr re-audited under it with the audit tool (amr-bench/audit_present.py, generated-Fortran based): amr_rvw
341+
(cyl_coord), sw_jac/jac (``if (igr) call s_amr_igr_swap_sigma``), amr_cg_pb/mv (do_pbmv / pull_host paths), amr_gst_a/b
342+
(amr_subcycle, the lerp path), amr_prim_st/amr_bt_* (``if (amr_prim_batch) call s_amr_convert_prim_batch``) -- all
343+
guarded; amr_cg, amr_cons_st, amr_stor_st, amr_cons_br allocated before first use. The shipped opt-in stands; the goldens
344+
that gated it ran on the GPU lane (staging ``gpu-mp-0e981924c0`` built 15:31, lock ``gpu: mp``), including AMR with IGR
345+
(2 tests), viscous (2), hypoelastic (2), IB (8), MHD (3), chemistry (3), bubbles (2), cylindrical (1); none with QBMM or
346+
alt_soundspeed. Pre-audit of m_rhs under the corrected rule: its kernels name alpha1/2, blkmod1/2, Kterm (alt_soundspeed),
347+
tau_Re_vf (viscous), rhs_hatL/R_vf and flux_gsrc_n (hypo_nc dual pass), nc_iface_vel_n (use_nc_iface_vel), qL/qR_prim
348+
(.not. igr) -- each behind its own condition on first reading, but m_rhs's rhs kernels are few and large (the census
349+
put weno/riemann at ~9 launches/rank/step of 2.5 ms each), so its per-launch gain is bounded by ~3 ms/step; the
350+
per-block rhs term (~4.5 ms/block/step) is per-batch work, not this tax. Whether the remaining 540 launches/rank/step
351+
pay the walk elsewhere (m_time_steppers, m_variables_conversion, m_cbc, m_weno, m_riemann_solvers) is the next
352+
measurement, not assumed.
353+
330354
**What this does NOT claim.** Nothing about NVHPC or CCE changes (their lanes emit what they emitted). The per-block cost
331355
outside gather/gfill (rhs per-batch ~4.6, regrid ~2, reflux 2.0, seam 1.6 ms/block/step, ledger 80) is not yet
332356
re-measured under the clause; the census counted 540 launches/rank/step at cap 64, so the same tax sits in the

src/common/include/omp_macros.fpp

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -33,10 +33,11 @@
3333
#! kernel touches (e.g. amr_cg(i)%sf) on EVERY launch, walking and re-attaching each component -- ~0.3 ms
3434
#! per launch for a 10-component array, linear in the component count, and the per-element mapper it
3535
#! generates for the type then taxes every kernel in that compilation unit (amr-bench/ubench, 2026-09-05).
36-
#! Not the default: a kernel naming an allocatable VARIABLE OR COMPONENT that is unallocated at launch
37-
#! (fine under the implicit map, which maps 0 bytes) aborts under `present` unless it is declare-target
38-
#! (e.g. m_variables_conversion's conversion kernel names the chemistry-only q_T_sf%sf), so a file opts in
39-
#! only once its kernels are audited for that.
36+
#! Not the default: a kernel naming a module allocatable ARRAY that is unallocated at launch (fine under
37+
#! the implicit map, which maps 0 bytes) aborts under `present`, declare-target or not; null allocatable
38+
#! or pointer COMPONENTS are fine (amr-bench/ubench N1-N4). m_variables_conversion's conversion kernel
39+
#! names the bubbles-only weight/R0, so a file opts in only once every kernel that names a conditionally
40+
#! allocated array is shown to launch only under that same condition.
4041
#:if getvar('MFC_OMP_PRESENT_ALLOCATABLE', False)
4142
#:set default_val = 'defaultmap(present:allocatable) '
4243
#:else

src/simulation/m_amr.fpp

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,10 @@
22
!!@file
33
!!@brief Contains module m_amr
44

5-
#! AMD OpenMP lane: assert allocatables present on every kernel here (see OMP_DEFAULT_STR). Audited 2026-09-05: the only
6-
#! bare module allocatables this unit's kernels name are amr_cg and amr_cons_br (allocated before first use); every
7-
#! conditionally allocated one carries GPU_DECLARE (exempt), and no kernel names an allocatable component. Keep it that way.
5+
#! AMD OpenMP lane: assert allocatables present on every kernel here (see OMP_DEFAULT_STR). Audited 2026-09-05: every
6+
#! conditionally allocated module array a kernel here names launches only under its allocation's own condition (amr_rvw:
7+
#! cyl_coord; sw_jac/jac: igr; amr_cg_pb/mv: do_pbmv; amr_gst_a/b: amr_subcycle; amr_prim_st/amr_bt_*: amr_prim_batch);
8+
#! amr_cg and amr_cons_br/stor_st are allocated before first use. A kernel naming an UNALLOCATED array aborts. Keep it so.
89
#:set MFC_OMP_PRESENT_ALLOCATABLE = True
910
#:include 'macros.fpp'
1011

0 commit comments

Comments
 (0)