Skip to content

VALIDATION.md

Latest commit

 

History

History
156 lines (121 loc) · 7.55 KB

VALIDATION.md

File metadata and controls

156 lines (121 loc) · 7.55 KB
Raw

Clean-room validation walkthrough

Goal: prove the shareable bundle builds OPM Flow (serial + MPI) from scratch on a fresh tree, exactly as a recipient would experience it.

Prereqs assumed already present (machine-level, can't be "clean"): VS 2022 C++ toolset + Windows SDK + CMake + Ninja, git, and MS-MPI runtime+SDK.

Steps (each validated below):

  1. Create an empty directory and unzip opm-flow-windows-mpi-build.zip into it.
  2. ./build-all.ps1 -Mpi -OpmOrg gitPaean -OpmBranch windows
    • generates compat shims
    • clones vcpkg, DUNE (v2.10.0), OPM (fork/branch), Trilinos
    • applies the DUNE patches from patches/
    • vcpkg installs deps (restored from binary cache if warm)
    • builds Zoltan, then DUNE + opm-common/grid/simulators with MPI
  3. Run serial deck (SPE1) — confirm exit 0 + outputs.
  4. mpiexec -n 2 ... — confirm parallel run with Zoltan partitioning.

Results are appended at the bottom as each step completes.

Results (clean run in C:\Users\paean\Desktop\opm_clean)

  • Step 1 (unzip): bundle extracted to a fresh dir — 9 files, nothing pre-built. OK.
  • Step 2 (build-all -Mpi -OpmOrg gitPaean -OpmBranch windows):
    • compat shims generated (getopt/unistd/sys/* /FCMacros) — OK
    • cloned vcpkg, DUNE (v2.10.0), OPM (gitPaean/windows fork), [Trilinos next] — OK
    • DUNE patches auto-applied to fresh clone — verified: gmshreader.hh has _ftelli64; mpitraits.hh has the #ifdef MPI_CXX_DOUBLE_COMPLEX guard; DuneCommonMacros.cmake has find_package(MPI COMPONENTS C) (3.0 gate gone). OK
    • deps restored from vcpkg binary cache — OK
    • built (MPI): DUNE x4, Zoltan (via Trilinos), opm-common, opm-grid — all OK
    • FINDING: opm-simulators failed at ReservoirCouplingSpawnSlaves.cpp:188 (path::c_str()=wchar_t* vs char*). Cause: the fork gitPaean/windows is missing the pushed opm-simulators "MPI build with MS-MPI" commit — it is committed locally but is 1 commit ahead of origin/windows (not pushed). opm-common and opm-grid MPI commits ARE pushed; only opm-simulators is behind. ACTION FOR USER: git -C opm-simulators push origin windows.
    • Applied that one commit's diff to the clean tree and resumed.
    • Resumed build: SUCCESS — build-mpi\opm-simulators\bin\flow_blackoil.exe (≈22.9 MB).
  • Step 3 (serial / 1 rank): flow_blackoil SPE1.DATA → exit 0, 123 timesteps.
  • Step 4 (parallel / 2 ranks): mpiexec -n 2 flow_blackoil SPE1.DATA → Using 2 MPI processes, ZOLTAN Load balancing method = 9 (GRAPH), 123 timesteps, exit 0, UNRST written. PASS.

Verdict (first run)

The bundle + guide build and run OPM Flow (serial + parallel) from a clean tree. The ONE blocker for a from-fork build was unpushed: the opm-simulators "MPI build with MS-MPI" commit. (Now pushed — see re-validation below.)

Re-validation — fresh tree, complete fork, ZERO manual steps (opm_clean2)

After the opm-simulators commit was pushed to gitPaean/windows, re-ran the whole thing in a brand-new directory from the bundle, with no manual patching:

Expand-Archive opm-flow-windows-mpi-build.zip -DestinationPath opm_clean2
.\build-all.ps1 -Mpi -OpmOrg gitPaean -OpmBranch windows
  • DUNE patches auto-applied (log: "dune-common: applied dune-common-windows.patch", "dune-grid: applied dune-grid-windows.patch"). OK
  • Clone (incl. complete fork) + Zoltan + all modules (MPI) built. SUCCESS: build-mpi\opm-simulators\bin\flow_blackoil.exe (~22.9 MB).
  • mpiexec -n 2 flow_blackoil SPE1.DATA: "Using 2 MPI processes", "ZOLTAN Load balancing method = 9 (GRAPH)", 123 timesteps, exit 0, UNRST written.

VERDICT: PASS with zero manual steps. A recipient needs only:

  1. install toolchain + MS-MPI (one-time),
  2. unzip the bundle,
  3. .\build-all.ps1 -Mpi -OpmOrg gitPaean -OpmBranch windows. (-OpmOrg/-OpmBranch point at the forks carrying the Windows fixes.)

Full all-targets re-validation (fresh clone, gitPaean/windows, zero manual steps)

After the compositional fixes landed (opm-common Constants.hpp pushed to the fork; DUNE yaspgrid fix added to patches/), cloned the harness repo into a fresh dir and ran the WHOLE all-targets build twice — serial and MPI:

git clone <repo> opm_clean3
cd opm_clean3
.\build-all.ps1 -SimTarget all -OpmOrg gitPaean -OpmBranch windows         # serial
.\build-all.ps1 -Mpi -SimTarget all -OpmOrg gitPaean -OpmBranch windows    # parallel
  • DUNE patches (gmshreader + yaspgrid) auto-applied to the fresh DUNE clone. OK
  • Fork carried every OPM fix (incl. Constants.hpp) — no manual patching.
  • Serial: all 48 opm-simulators executables built (every flow_* variant, incl. compositional flow_comp*/flowexp_comp*). 0 failures.
  • MPI: Zoltan built, then all 48 executables built with MPI. 0 failures. mpiexec -n 2 flow_blackoil SPE1.DATA -> "Using 2 MPI processes", "ZOLTAN Load balancing method = 9 (GRAPH)", 123 timesteps, exit 0, UNRST written.

VERDICT: the instructions + patches build the entire opm-common/opm-grid/ opm-simulators target set (GPU off), serial and parallel, clean from the fork with zero manual steps. Parallel jobs default to 4 (RAM); pass -Jobs N to build-all.ps1/build-module.ps1 to raise it (a ≥ 32 GB machine handles 6–8). compositional+MPI compiles cleanly.

Re-verifying the patch series after a rebase

The Windows patches are maintained on the windows branch of each fork and are rebased onto upstream master as it moves. Every rebase is a chance to lose a hunk — it happened once in this work, where a stash dance silently dropped a string-literal split from both opm-common branches and the next build failed on it. So after rebasing, check the result against the tree that was actually built and tested:

git diff --stat <reference> windows

Empty output means the rebase reconstructed the tested tree byte for byte. Anything else needs explaining before pushing.

To re-verify on Windows, build each module with testing on:

./build-module.ps1 <module> -Mpi -OpenMP -Target all -Extra '-DBUILD_TESTING=ON','-DBUILD_EXAMPLES=ON'

BUILD_EXAMPLES=ON is required alongside BUILD_TESTING=ON: opm-simulators' modelTests.cmake registers tests against obstacle_immiscible and obstacle_pvs, which are built from EXAMPLE_SOURCE_FILES. That is an upstream constraint, not something these patches introduce.

Neither module reaches a green ctest on Windows, and that is expected:

  • opm-common: 226/229. ParserIncludeTests needs git symlinks; rst_deck_test and rst_deck_test2 are driven by a shell script.

  • opm-simulators: 73/139. All 66 failures are BAD_COMMAND — ctest could not launch the process at all — and the failing set is exactly the set of tests whose ctest command is a .sh script (run-vtu-test.sh, run-parallel-unitTest.sh). Windows cannot execute those directly. Every test that does start passes.

That last point is worth reproducing rather than taking on trust, because it makes the result independent of any recorded baseline. Compare the failing names against the shell-driven ones:

ctest --show-only=json-v1 | jq -r '.tests[] | select((.command|join(" "))|test("\.sh")) | .name' | sort > sh.txt

If the two sets match exactly, no test regressed. If a name appears in the failures that is not in sh.txt, that is a real failure and needs investigating.

Beware of truncating ctest's output when checking this. | Select-Object -Last N drops the "N tests failed out of M" summary line and leaves only the tail of the failure list, which reads as a much smaller failure count than the real one.