Skip to content

Commit 1e7e638

Browse files
committed
Merge master into TerminalOutput: resolve conflicts with #1762
2 parents cb0e59d + 814f7a5 commit 1e7e638

341 files changed

Lines changed: 9133 additions & 6052 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.claude/rules/common-pitfalls.md

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,35 @@ covered in `docs/documentation/contributing.md`.
3636
- `@:ACC_SETUP_VFs(...)`/`@:ACC_SETUP_SFs(...)` GPU pointer setup compiles only under
3737
Cray. Around MPI: `GPU_UPDATE(host=...)` before send, `GPU_UPDATE(device=...)` after
3838
receive.
39+
- An array whose bound is a device global (`dimension(num_fluids)`, `dimension(num_species)`) may be
40+
passed to a device routine **from a parallel-loop body, but not from inside another
41+
`GPU_ROUTINE(parallelism='[seq]')`**. CCE OpenACC rejects the second form with
42+
`ftn-7066 ... Global in accelerator routine without declare -- num_fluids`, and reports it at
43+
whatever line it gave up on: remove one trigger and the message *walks forward* to the next call,
44+
so the reported line is not the cause. Only the plain lanes fail - under `--case-optimization`
45+
those bounds are `parameter`s, so a green Case Opt lane beside a failing plain one is the
46+
signature. Every accepted call site in the tree already obeys this (`m_cbc`, `m_ibm`,
47+
`m_bubbles_EL`, `s_compute_cell_state`): form such a call in the loop body and pass scalars
48+
deeper. Neither `cray_inline` nor a `num_fluids_max` bound nor dropping optional dummies helps -
49+
all three were measured.
50+
- nvfortran 23.11/24.1 segfault (`fort2 TERMINATED by signal 11`) on a caller that passes a
51+
`parameter` array from `m_thermochem` (e.g. `molecular_weights`) into a declare-target routine.
52+
Read such arrays directly in the kernel, or pass a plain local computed from them.
53+
- The `USING_AMD` fypp guards (86 sites, `#:set` in `src/common/include/shared_parallel_macros.fpp`) are
54+
load-bearing, not a stale workaround - do not "modernize" them away. They swap a device-global array
55+
bound for a literal: `dimension(3)` for `num_dims`/`num_fluids` when case optimization is off (64
56+
sites), and `dimension(20)` for `sys_size` in `m_compute_cbc` (21 sites, with a matching
57+
`@:PROHIBIT` in `m_start_up` capping `sys_size <= 20` under AMD+CBC). Setting `USING_AMD = False`
58+
and rebuilding amdflang `--gpu mp` without case optimization compiles CLEAN - 728 s, zero
59+
diagnostics - and then NaNs at step 50 in CBC, riemann `wave_speeds=2`, IBM, surface tension,
60+
QBMM/viscous and MHD HLLD, while both Lagrange bubble cases *complete* with out-of-tolerance
61+
answers. Measured 2026-08-29 on MI210. A compile-only check returns green, so any future attempt to
62+
drop these must run the tests, not just build.
63+
- The same "call it from the loop body" rule covers `m_thermochem`: calling `get_species_*` from
64+
inside a `GPU_ROUTINE` rather than from the kernel gave CCE OpenMP a runtime
65+
`Memory access fault by GPU node-N ... Reason: Unknown` on the first step (exit 134), while every
66+
other backend ran. Evaluate them at the call site and pass the arrays in. Note this one only shows
67+
at runtime, and only on a case that reaches the path - the build is clean.
3968

4069
## Parameters
4170

@@ -105,6 +134,27 @@ covered in `docs/documentation/contributing.md`.
105134
- Tests are generated programmatically in `toolchain/mfc/test/cases.py` (parameter
106135
modifications on `BASE_CFG` via the `CaseGeneratorStack` push/pop pattern); test UUID =
107136
CRC32 of the trace string; `./mfc.sh test -l` lists all.
137+
- `--only` matches whole trace *elements*, not substrings, and `_filter_only`
138+
(`toolchain/mfc/test/test.py`) **ANDs labels while ORing UUIDs**. So `--only bubbles` matches
139+
nothing (the element is `Bubbles`), and `--only low_Mach=1 low_Mach=2` asks for cases carrying
140+
both and also matches nothing. It then exits **143**, which reads like an external kill rather
141+
than an empty filter. Pass UUIDs whenever you want the union of several groups.
142+
- Sibling `define_case_d` calls off the same stack level are never *combined*. Two switches that
143+
only matter together (`avg_state=1` needs `wave_speeds=2` to be read at all) therefore get zero
144+
effective coverage unless something pushes one and defines the other beneath it. Check
145+
reachability before trusting that a flag is tested.
146+
- `--no-build` silently runs whatever binary is on disk for a configuration it did not build.
147+
Chemistry has its own config (`gpu-mp-chem-*`) that a plain `./mfc.sh build` never produces, so
148+
a `--no-build` run reports failures from stale binaries and hides real compile breaks. Run
149+
chemistry-touching sets without it.
150+
- Pick the newest binary by the *binary's* mtime (`ls -t build/install/*/bin/simulation`), not the
151+
install directory's - a stale config's directory can be newer than a fresh build's.
152+
- The pre-commit hook lives in the main repo's `.git/hooks/` and git exports `GIT_DIR` there
153+
during a commit, so from a worktree the toolchain lint enumerates the *other* checkout and
154+
fails. Reproduce with `GIT_DIR=<main>/.git ./mfc.sh precheck`. Run precheck by hand and commit
155+
with `--no-verify`.
156+
- `/tmp` is node-local: scratch does not survive a compute-node change, and its absence is
157+
silence, not an error. Keep patches and resource baselines on a shared filesystem.
108158
- Golden files are tolerance-compared. Regenerate only the affected tests
109159
(`./mfc.sh test --generate --only <tests>`) — an unexplained golden-file diff is a bug
110160
report, not noise to be regenerated away.

docs/documentation/contributing.md

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -452,6 +452,31 @@ $:GPU_DECLARE(create='[my_array]')
452452

453453
If an array is allocated inside an `if` block, its deallocation must follow the same condition.
454454

455+
### How to Add an Equation of State
456+
457+
Every stiffened-gas expression lives in `src/common/m_variables_conversion.fpp`. Adding a second EOS
458+
means supplying these, not grepping for `gammas`:
459+
460+
| Operator | Gives |
461+
|---|---|
462+
| `s_compute_mixture_coefficients` / `_dt` | mixture \f$\Gamma, \Pi_\infty, q_v\f$ from the phase fractions, and their time derivative |
463+
| `f_pressure` / `s_compute_energy` | \f$p(e)\f$ and \f$E(p)\f$ |
464+
| `f_bulk_modulus` | \f$K(p)\f$ - every sound speed in MFC is \f$K/\rho\f$, differing only in how phases are mixed |
465+
| `s_compute_speed_of_sound` / `_avg` | that mixing: Wood's law, 6-equation, bubble-diluted |
466+
| `f_phase_internal_energy` | per-phase internal energy (6-equation model) |
467+
| `f_isentrope_exponent` / `f_isentrope_pressure` / `f_pressure_on_isentrope` | the isentrope \f$p + B = \textrm{const}\,\rho^n\f$ |
468+
| `f_sg_thermal` | the thermal law \f$p + B = (n-1)c_v\rho T\f$ |
469+
470+
The first six are *mechanical* - they need only \f$p, \rho, e, c\f$. The last two are *caloric* and
471+
additionally need \f$c_v\f$ and \f$q'_v\f$. An EOS that supplies only the mechanical set cannot support
472+
phase change (`m_phase_change` also needs entropy and enthalpy) or reactive burn, so those features
473+
must be prohibited for it in `case_validator.py`.
474+
475+
The coefficients arrive in two parameterizations of the same EOS: `gammas`/`pi_infs` are the stored
476+
forms the user supplies (see @ref sec-stored-forms), and `isentrope_n`/`isentrope_B` are the same EOS
477+
as \f$p + B = \textrm{const}\,\rho^n\f$, derived once at start-up. Convert with the `f_isentrope_*`
478+
operators rather than open-coding either relation.
479+
455480
### How to Add a Test Case
456481

457482
**Step 1: Create a case file**

docs/documentation/equations.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -644,7 +644,7 @@ Reaction mechanisms are code-generated via Pyrometheus (\cite Cisneros26), which
644644

645645
## 10. Surface Tension (`surface_tension = .true.`) (\cite Schmidmayer17; \cite Wilfong26 Sec. 4.1.8)
646646

647-
**Source:** `src/simulation/m_surface_tension.fpp`, `src/simulation/include/inline_capillary.fpp`
647+
**Source:** `src/simulation/m_surface_tension.fpp`
648648

649649
**Color function advection:**
650650

docs/documentation/gpuParallelization.md

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -614,15 +614,14 @@ Does not do anything for OpenMP currently
614614
pure, sequential per-thread helpers is:
615615

616616
```fortran
617-
subroutine s_accumulate_mixture_properties(nf, alpha_rho_K, alpha_K, rho_K, gamma_K, pi_inf_K, qv_K)
617+
subroutine s_compute_mixture_coefficients(alpha_rho_K, alpha_K, rho_K, gamma_K, pi_inf_K, qv_K)
618618
619-
$:GPU_ROUTINE(function_name='s_accumulate_mixture_properties', parallelism='[seq]', cray_inline=True)
619+
$:GPU_ROUTINE(function_name='s_compute_mixture_coefficients', parallelism='[seq]', cray_inline=True)
620620
621-
integer, intent(in) :: nf
622-
real(wp), dimension(nf), intent(in) :: alpha_rho_K, alpha_K
623-
real(wp), intent(out) :: rho_K, gamma_K, pi_inf_K, qv_K
621+
real(wp), dimension(num_fluids), intent(in) :: alpha_rho_K, alpha_K
622+
real(wp), intent(out) :: rho_K, gamma_K, pi_inf_K, qv_K
624623
...
625-
end subroutine s_accumulate_mixture_properties
624+
end subroutine s_compute_mixture_coefficients
626625
```
627626

628627
**When to use it.** Extract a block into a `GPU_ROUTINE` helper when:

examples/0D_bubblecollapse_adap/case.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -122,6 +122,7 @@
122122
"nb": nb,
123123
# Fluids Physical Parameters
124124
"fluid_pp(1)%gamma": 1.0e00 / (gam_l - 1.0e00),
125+
"fluid_pp(1)%eos": "stiffened_gas",
125126
"fluid_pp(1)%pi_inf": gam_l * (pi_inf_l / p0) / (gam_l - 1.0),
126127
}
127128
)

examples/1D_acoustic_dipole/case.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,7 @@
6464
"acoustic(1)%delay": 1e-7,
6565
# Fluids Physical Parameters
6666
"fluid_pp(1)%gamma": 1.0e00 / (4.4e00 - 1.0e00),
67+
"fluid_pp(1)%eos": "stiffened_gas",
6768
"fluid_pp(1)%pi_inf": 4.4e00 * 5.57e08 / (4.4e00 - 1.0e00),
6869
}
6970
)

examples/1D_acoustic_gauss_sigmadist/case.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,7 @@
6363
"acoustic(1)%delay": 1e-7,
6464
# Fluids Physical Parameters
6565
"fluid_pp(1)%gamma": 1.0e00 / (4.4e00 - 1.0e00),
66+
"fluid_pp(1)%eos": "stiffened_gas",
6667
"fluid_pp(1)%pi_inf": 4.4e00 * 5.57e08 / (4.4e00 - 1.0e00),
6768
}
6869
)

examples/1D_acoustic_gauss_sigmatime/case.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,7 @@
6363
"acoustic(1)%delay": 1e-7,
6464
# Fluids Physical Parameters
6565
"fluid_pp(1)%gamma": 1.0e00 / (4.4e00 - 1.0e00),
66+
"fluid_pp(1)%eos": "stiffened_gas",
6667
"fluid_pp(1)%pi_inf": 4.4e00 * 5.57e08 / (4.4e00 - 1.0e00),
6768
}
6869
)

examples/1D_acoustic_sine_frequency/case.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,7 @@
6363
"acoustic(1)%delay": 0,
6464
# Fluids Physical Parameters
6565
"fluid_pp(1)%gamma": 1.0e00 / (4.4e00 - 1.0e00),
66+
"fluid_pp(1)%eos": "stiffened_gas",
6667
"fluid_pp(1)%pi_inf": 4.4e00 * 5.57e08 / (4.4e00 - 1.0e00),
6768
}
6869
)

examples/1D_acoustic_sine_wavelength/case.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,7 @@
6363
"acoustic(1)%delay": 0,
6464
# Fluids Physical Parameters
6565
"fluid_pp(1)%gamma": 1.0e00 / (4.4e00 - 1.0e00),
66+
"fluid_pp(1)%eos": "stiffened_gas",
6667
"fluid_pp(1)%pi_inf": 4.4e00 * 5.57e08 / (4.4e00 - 1.0e00),
6768
}
6869
)

0 commit comments

Comments
 (0)