Skip to content

Commit 92d2a92

Browse files
committed
Merge master into cont_damage_fix
Conflict in m_data_output.fpp at three s_compute_speed_of_sound call sites: master added the alpha_rho argument for per-phase EOS evaluation, this branch renamed G_local to G_damaged. Kept both - this branch's name, master's argument.
2 parents 4f20ac4 + d2d8cac commit 92d2a92

68 files changed

Lines changed: 4773 additions & 282 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.claude/rules/common-pitfalls.md

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,19 @@ covered in `docs/documentation/contributing.md`.
6060
QBMM/viscous and MHD HLLD, while both Lagrange bubble cases *complete* with out-of-tolerance
6161
answers. Measured 2026-08-29 on MI210. A compile-only check returns green, so any future attempt to
6262
drop these must run the tests, not just build.
63+
- **CCE OpenACC (19.0.0 through 21.0.2, `-O2`; `-O0`/`-O1` correct; OpenMP offload unaffected): a device
64+
routine that contains any `GPU_LOOP` (itself or in anything it calls) must be called with scalars,
65+
never with an array element as an actual argument.** Every `routine` level is affected, including the
66+
conforming `loop vector` inside `routine vector`. With both ingredients present the
67+
element is misaddressed: an `intent(in)` element reads as garbage, an `intent(out)` element is
68+
never written. Either ingredient alone is fine, which is why master's `s_compute_pressure(q%sf(j,k,l),...)`
69+
works (no loop) and `s_compute_mixture_coefficients` works (scalar actuals). PR #1811 added the
70+
Newton and RK4 loops to the EOS helpers and every call that passed `%sf(j,k,l)` or `blkmod1(k,l,q)`
71+
ended in `NaN(s) in timestep output` on the Frontier CCE OpenACC lanes only, bit-identical on every
72+
other backend. Fix: copy elements to locals before the call, receive into a local. 37-line
73+
reproducer and the bisection: sbryngelson/compiler-bugs `cce/acc-routine-element-by-reference`,
74+
MFC #1815. Do not "fix" it by deleting the `seq` directives instead: they are the idiom master
75+
uses in every device routine.
6376
- The same "call it from the loop body" rule covers `m_thermochem`: calling `get_species_*` from
6477
inside a `GPU_ROUTINE` rather than from the kernel gave CCE OpenMP a runtime
6578
`Memory access fault by GPU node-N ... Reason: Unknown` on the first step (exit 134), while every

.github/scripts/submit-slurm-job.sh

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -301,6 +301,17 @@ while :; do
301301
# another node. Note bench-pair.sh probes only after building both trees, so
302302
# a fault there discards those builds and the resubmit repeats them.
303303
faulted_node=$(bash "$SCRIPT_DIR/node-exclude.sh" node-from "$output_file")
304+
# Fall back to SLURM's own record when the MFC_FAULT_NODE marker is
305+
# unreadable. A job that dies before its .out is flushed (or before NFS
306+
# makes it visible) leaves no marker, so node-from returns empty; the
307+
# merge below then adds nothing and SLURM re-draws the SAME bad node. A
308+
# dead-GPU V100 (atl1-1-02-006-34-0, cuInit 999) ate both attempts of run
309+
# 34183404644 exactly this way. sacct knows the node whether or not the
310+
# .out exists, so identification no longer depends on the marker.
311+
if [ -z "$faulted_node" ]; then
312+
faulted_node=$(sacct -j "$job_id" -X -n -o NodeList 2>/dev/null | head -n1 | tr -d ' ')
313+
case "$faulted_node" in ""|None*|*[,\[]*) faulted_node="" ;; esac
314+
fi
304315
if [ "$node_attempt" -lt "$MFC_MAX_NODE_RESUBMITS" ]; then
305316
node_attempt=$((node_attempt + 1))
306317
node_exclude=$(bash "$SCRIPT_DIR/node-exclude.sh" merge "$node_exclude" "$faulted_node")

.github/workflows/common/build.sh

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,9 +41,16 @@ source .github/scripts/retry-build.sh
4141
# aborts in MPI_Init ("OPAL ERROR: Unreachable in file ext3x_client.c"). mpirun
4242
# is unaffected, and it is how MFC launches every binary anyway. Output is left
4343
# on stdout so a future failure is diagnosable from the CI log.
44+
#
45+
# --bind-to none is required, not cosmetic: on a busy GPU node SLURM hands the job
46+
# an offset/partial cpuset (e.g. cores 32-47), and Open MPI's default --bind-to
47+
# core then dies with "hwloc_set_cpubind Error for bitmap" before the binary even
48+
# starts. The run templates, coverage_build.py and preflight.sh all already pass
49+
# it; this smoke-test was the lone launch that did not, and it red-crossed jobs
50+
# that backfilled onto shared GPU nodes.
4451
validate_cmd=""
4552
if [ "$job_cluster" = "phoenix" ]; then
46-
validate_cmd='syscheck_bin=$(find build/install -name syscheck -type f 2>/dev/null | head -1); [ -z "$syscheck_bin" ] || mpirun -np 1 "$syscheck_bin"'
53+
validate_cmd='syscheck_bin=$(find build/install -name syscheck -type f 2>/dev/null | head -1); [ -z "$syscheck_bin" ] || mpirun --bind-to none -np 1 "$syscheck_bin"'
4754
fi
4855

4956
# --- Variant selection ---

docs/documentation/case.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -732,6 +732,7 @@ To restart the simulation from $k$-th time step, see @ref running "Restarting Ca
732732
| `pi_inf_wrt` | Logical | Add the liquid stiffness function to the database |
733733
| `pres_inf_wrt` | Logical | Add the liquid stiffness to the formatted database |
734734
| `c_wrt` | Logical | Add the sound speed to the database |
735+
| `T_wrt` | Logical | Add each fluid's temperature to the database (needs `cv` > 0) |
735736
| `omega_wrt(i)` | Logical | Add the $i$-direction vorticity to the database |
736737
| `schlieren_wrt` | Logical | Add the numerical schlieren to the database|
737738
| `qm_wrt` | Logical | Add the Q-criterion to the database|

docs/documentation/contributing.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -464,8 +464,8 @@ means supplying these, not grepping for `gammas`:
464464
| `f_pressure` / `s_compute_energy` | \f$p(e)\f$ and \f$E(p)\f$ |
465465
| `f_bulk_modulus` | \f$K(p)\f$ - every sound speed in MFC is \f$K/\rho\f$, differing only in how phases are mixed |
466466
| `s_compute_speed_of_sound` / `_avg` | that mixing: Wood's law, 6-equation, bubble-diluted |
467-
| `f_phase_internal_energy` | per-phase internal energy (6-equation model) |
468-
| `f_isentrope_exponent` / `f_isentrope_pressure` / `f_pressure_on_isentrope` | the isentrope \f$p + B = \textrm{const}\,\rho^n\f$ |
467+
| `s_phase_internal_energy` | per-phase internal energy (6-equation model) |
468+
| `f_isentrope_exponent` / `f_isentrope_pressure` | the isentrope \f$p + B = \textrm{const}\,\rho^n\f$ |
469469
| `f_sg_thermal` | the thermal law \f$p + B = (n-1)c_v\rho T\f$ |
470470

471471
The first six are *mechanical* - they need only \f$p, \rho, e, c\f$. The last two are *caloric* and
Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
"""
2+
Isentropic release of a fluid whose reference curve is an isentrope (JWL or Vinet): a Riemann problem
3+
between two states of the same fluid. Everything left of the contact keeps the left state's entropy,
4+
so the fan and the left star state must lie on the closed-form isentrope through (rho0, p0).
5+
"""
6+
7+
import argparse
8+
import json
9+
import math
10+
11+
parser = argparse.ArgumentParser(description="1D JWL isentropic release")
12+
parser.add_argument("--mfc", type=json.loads, default="{}", metavar="DICT")
13+
parser.add_argument("-N", type=int, default=400)
14+
parser.add_argument("--cfl", type=float, default=0.4)
15+
parser.add_argument("--eos", choices=["jwl", "vinet"], default="jwl", help="reference curve: both are isentropes")
16+
args = parser.parse_args()
17+
18+
rho0, p0, rho_r, p_r = 1.0, 1.0, 0.3, 0.1
19+
if args.eos == "jwl":
20+
fluid = {f"fluid_pp(1)%jwl_{k}": v for k, v in {"a": 6.0, "b": 0.15, "r1": 4.0, "r2": 1.0, "omega": 0.3, "rho0": rho0}.items()}
21+
else:
22+
fluid = {f"fluid_pp(1)%vinet_{k}": v for k, v in {"k0": 2.0, "k0p": 4.0, "gruneisen": 0.3, "rho0": rho0}.items()}
23+
N, L, T_end = args.N, 1.0, 0.15
24+
c_max = math.sqrt((1.3 * p0 + 6.15) / rho0) # generous bound on c + |u| for either fit
25+
dt = args.cfl * (L / N) / c_max
26+
Nt = math.ceil(T_end / dt)
27+
dt = T_end / Nt
28+
29+
case = {
30+
"run_time_info": "F",
31+
"x_domain%beg": 0.0,
32+
"x_domain%end": L,
33+
"m": N - 1,
34+
"n": 0,
35+
"p": 0,
36+
"dt": dt,
37+
"t_step_start": 0,
38+
"t_step_stop": Nt,
39+
"t_step_save": Nt,
40+
"num_patches": 2,
41+
"model_eqns": 2,
42+
"num_fluids": 1,
43+
"time_stepper": 3,
44+
"recon_type": "weno",
45+
"weno_order": 5,
46+
"weno_eps": 1.0e-16,
47+
"mapped_weno": "T",
48+
"riemann_solver": 2,
49+
"wave_speeds": 1,
50+
"avg_state": 2,
51+
"bc_x%beg": -3,
52+
"bc_x%end": -3,
53+
"format": 1,
54+
"precision": 2,
55+
"prim_vars_wrt": "T",
56+
"parallel_io": "F",
57+
"fluid_pp(1)%eos": args.eos,
58+
**fluid,
59+
}
60+
for pid, (x_c, rho, pres) in enumerate([(0.25, rho0, p0), (0.75, rho_r, p_r)], start=1):
61+
case.update(
62+
{
63+
f"patch_icpp({pid})%geometry": 1,
64+
f"patch_icpp({pid})%x_centroid": x_c,
65+
f"patch_icpp({pid})%length_x": 0.5 * L,
66+
f"patch_icpp({pid})%alpha_rho(1)": rho,
67+
f"patch_icpp({pid})%alpha(1)": 1.0,
68+
f"patch_icpp({pid})%vel(1)": 0.0,
69+
f"patch_icpp({pid})%pres": pres,
70+
}
71+
)
72+
print(json.dumps(case))

examples/1D_mg_acoustic/case.py

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
"""
2+
Right-moving acoustic pulse in a single Mie-Gruneisen fluid at its reference state.
3+
A rectangle patch carries a simple-wave perturbation (drho, dp = c^2 drho, du = c drho/rho0), so
4+
only the right-going characteristic is excited; the harness tracks the centroid of drho against
5+
the general analytic c. Patches rather than an analytic IC: an IC expression is compiled in, and
6+
every distinct one costs the test suite a full rebuild.
7+
"""
8+
9+
import argparse
10+
import json
11+
import math
12+
13+
parser = argparse.ArgumentParser(description="1D Mie-Gruneisen acoustic pulse")
14+
parser.add_argument("--mfc", type=json.loads, default="{}", metavar="DICT")
15+
parser.add_argument("-N", type=int, default=200)
16+
parser.add_argument("--cfl", type=float, default=0.4)
17+
parser.add_argument("--a", type=float, default=0.0, help="Gruneisen slope: Gamma_G = Gamma_0 + a mu")
18+
args = parser.parse_args()
19+
20+
rho0, p0, c0, s, gruneisen = 1.0, 1.0, 1.0, 1.5, 0.4
21+
c = math.sqrt(c0**2 + (1.0 + gruneisen) * p0 / rho0 + args.a * p0 / (rho0 * gruneisen)) # the frozen speed at the reference state
22+
amp, x0, width = 1.0e-4, 0.25, 0.1
23+
N, L, T_end = args.N, 1.0, 0.3
24+
dt = args.cfl * (L / N) / c
25+
Nt = math.ceil(T_end / dt)
26+
dt = T_end / Nt
27+
28+
print(
29+
json.dumps(
30+
{
31+
"run_time_info": "F",
32+
"x_domain%beg": 0.0,
33+
"x_domain%end": L,
34+
"m": N - 1,
35+
"n": 0,
36+
"p": 0,
37+
"dt": dt,
38+
"t_step_start": 0,
39+
"t_step_stop": Nt,
40+
"t_step_save": Nt,
41+
"model_eqns": 2,
42+
"num_fluids": 1,
43+
"time_stepper": 3,
44+
"recon_type": "weno",
45+
"weno_order": 5,
46+
"weno_eps": 1.0e-16,
47+
"mapped_weno": "T",
48+
"riemann_solver": 2,
49+
"wave_speeds": 1,
50+
"avg_state": 2,
51+
"bc_x%beg": -3,
52+
"bc_x%end": -3,
53+
"format": 1,
54+
"precision": 2,
55+
"prim_vars_wrt": "T",
56+
"parallel_io": "F",
57+
"num_patches": 2,
58+
"patch_icpp(1)%geometry": 1,
59+
"patch_icpp(1)%x_centroid": 0.5,
60+
"patch_icpp(1)%length_x": L,
61+
"patch_icpp(1)%alpha_rho(1)": rho0,
62+
"patch_icpp(1)%alpha(1)": 1.0,
63+
"patch_icpp(1)%vel(1)": 0.0,
64+
"patch_icpp(1)%pres": p0,
65+
"patch_icpp(2)%geometry": 1,
66+
"patch_icpp(2)%alter_patch(1)": "T",
67+
"patch_icpp(2)%x_centroid": x0,
68+
"patch_icpp(2)%length_x": width,
69+
"patch_icpp(2)%alpha_rho(1)": rho0 + amp,
70+
"patch_icpp(2)%alpha(1)": 1.0,
71+
"patch_icpp(2)%vel(1)": c / rho0 * amp,
72+
"patch_icpp(2)%pres": p0 + c**2 * amp,
73+
"fluid_pp(1)%eos": "mie_gruneisen",
74+
"fluid_pp(1)%mg_rho0": rho0,
75+
"fluid_pp(1)%mg_c0": c0,
76+
"fluid_pp(1)%mg_s": s,
77+
"fluid_pp(1)%mg_gruneisen": gruneisen,
78+
**({"fluid_pp(1)%mg_gruneisen_a": args.a} if args.a else {}),
79+
}
80+
)
81+
)

examples/1D_mg_impact/case.py

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
"""
2+
Symmetric impact of two Mie-Gruneisen slabs approaching at relative speed U.
3+
Each slab is brought to rest by a shock with particle-velocity jump U/2, so the shock state
4+
lies exactly on the Hugoniot u_s = c0 + s u_p; the harness checks the shock speed and the
5+
plateau density against that relation.
6+
"""
7+
8+
import argparse
9+
import json
10+
import math
11+
12+
parser = argparse.ArgumentParser(description="1D Mie-Gruneisen symmetric impact")
13+
parser.add_argument("--mfc", type=json.loads, default="{}", metavar="DICT")
14+
parser.add_argument("-N", type=int, default=800)
15+
parser.add_argument("--U", type=float, default=1.0, help="closing speed of the two slabs")
16+
parser.add_argument("--cfl", type=float, default=0.4)
17+
parser.add_argument("--s2", type=float, default=0.0, help="quadratic Hugoniot coefficient u_s = c0 + s u_p + s2 u_p^2 + s3 u_p^3")
18+
parser.add_argument("--s3", type=float, default=0.0, help="cubic Hugoniot coefficient")
19+
args = parser.parse_args()
20+
21+
rho0, p0, c0, s, gruneisen = 1.0, 1.0e-3, 1.0, 1.5, 0.4
22+
N, L, T_end = args.N, 1.0, 0.2
23+
dt = args.cfl * (L / N) / (c0 + (s + 1.0 + (args.s2 + args.s3 * args.U) * args.U) * args.U)
24+
Nt = math.ceil(T_end / dt)
25+
dt = T_end / Nt
26+
27+
case = {
28+
"run_time_info": "F",
29+
"x_domain%beg": 0.0,
30+
"x_domain%end": L,
31+
"m": N - 1,
32+
"n": 0,
33+
"p": 0,
34+
"dt": dt,
35+
"t_step_start": 0,
36+
"t_step_stop": Nt,
37+
"t_step_save": Nt,
38+
"num_patches": 2,
39+
"model_eqns": 2,
40+
"num_fluids": 1,
41+
"time_stepper": 3,
42+
"recon_type": "weno",
43+
"weno_order": 5,
44+
"weno_eps": 1.0e-16,
45+
"mapped_weno": "T",
46+
"riemann_solver": 2,
47+
"wave_speeds": 1,
48+
"avg_state": 2,
49+
"bc_x%beg": -3,
50+
"bc_x%end": -3,
51+
"format": 1,
52+
"precision": 2,
53+
"prim_vars_wrt": "T",
54+
"parallel_io": "F",
55+
"fluid_pp(1)%eos": "mie_gruneisen",
56+
"fluid_pp(1)%mg_rho0": rho0,
57+
"fluid_pp(1)%mg_c0": c0,
58+
"fluid_pp(1)%mg_s": s,
59+
"fluid_pp(1)%mg_gruneisen": gruneisen,
60+
**({"fluid_pp(1)%mg_s2": args.s2} if args.s2 else {}),
61+
**({"fluid_pp(1)%mg_s3": args.s3} if args.s3 else {}),
62+
}
63+
for pid, (x_c, vel) in enumerate([(0.25, 0.5 * args.U), (0.75, -0.5 * args.U)], start=1):
64+
case.update(
65+
{
66+
f"patch_icpp({pid})%geometry": 1,
67+
f"patch_icpp({pid})%x_centroid": x_c,
68+
f"patch_icpp({pid})%length_x": 0.5 * L,
69+
f"patch_icpp({pid})%alpha_rho(1)": rho0,
70+
f"patch_icpp({pid})%alpha(1)": 1.0,
71+
f"patch_icpp({pid})%vel(1)": vel,
72+
f"patch_icpp({pid})%pres": p0,
73+
}
74+
)
75+
print(json.dumps(case))

src/common/m_constants.fpp

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -118,6 +118,11 @@ module m_constants
118118
!! cannot be auto-generated, so these are hand-written.
119119
integer, parameter :: eos_stiffened_gas = 1
120120
integer, parameter :: eos_ideal_gas = 2
121+
integer, parameter :: eos_mie_gruneisen = 3
122+
integer, parameter :: eos_jwl = 4
123+
integer, parameter :: eos_vinet = 5
124+
integer, parameter :: eos_rk4_steps = 8 !< fixed-step RK4 along a phasic isentrope or a reference temperature
125+
integer, parameter :: ode_isentrope = 1, ode_reference_temperature = 2 !< the two ODEs s_rk4 integrates
121126
integer, parameter :: num_synth_shells_max = 50 !< Max energy shells for synthetic turbulence
122127
integer, parameter :: num_turb_sources_max = 10 !< Max Gaussian forcing zones for synthetic turbulence
123128

0 commit comments

Comments
 (0)