Raw beam dumps keep growing in file size, then crash in H5Dset_extent, when a file beam sits across a pipeline-stage boundary

[Chaojie-Zhang]

What we saw. Two-beam run (driver + witness, both profile_type: "file"). The witness's
raw dump files grew every dump (3.1 → 4.3 MB) although its particle number was constant. The
run then died during a raw dump with H5Dset_extent ... unable to set dataset extent, and an
MPI abort "message sizes do not match across processes" (cray-mpich, Perlmutter). The last raw
file contained almost no particles. All of this makes it look like particles are being created
or lost — they are not; the particle bookkeeping is fine. Only the raw-file writing is broken.

What actually happens. Our witness happened to lie across a stage boundary: 143 particles
just below ξ = 9.0 (stage 5), the other 55k just above (stage 6). When a raw dump is written,
the first stage holding particles creates the datasets, and the HDF5 chunk size is set to that
stage's particle count (ldim(1) = tp before h5pset_chunk_f; hdf5io_class.f03 lines
1389 / 1447 / 1494 / 1538 on main). So the whole 55k-particle file is stored in chunks of 143.
Slow particles drift backward in ξ (dξ/step = dt/2γ²), so this small group shrinks every dump,
and the chunk shrinks with it. The file fills with chunk-index overhead, and when the group is
nearly gone the write fails and the ranks fall out of sync. Measured on our run:

dump	true N	chunk	particles below ξ=9.0	file size
0	55,333	(143,)	143	3.23 MB
20	54,083	(25,)	25	3.61 MB
40	53,973	(14,)	14	4.07 MB
60	53,960	(10,)	10	4.47 MB
80	—	(8,)	8	crash mid-dump

Since the drift rate falls as 1/γ², low-energy beams hit this within tens of steps while
high-energy beams only plateau — so it masquerades as beam physics.

How we fixed it on our side (demonstration). We shifted driver and witness together by
+0.1 c/ωp in ξ so the witness sits entirely inside one stage (relative phase unchanged).
Result: file size flat at 3.0 MB, no crash, run healthy.

Suggested fix in QPAD. Don't let the chunk follow a possibly tiny leading-stage count —
floor it. Chunks larger than the current extent are legal for extendible datasets:

ldim(1) = max( tp, 16384 )    ! instead of: ldim(1) = tp   (the 4 create sites in pwpart_3d_pipe)
call h5pset_chunk_f(dcplID, 1, ldim, ierr)

We run with this change (compile-tested): flat files, no crash, any beam placement. Related:
the HDF5 ierr codes in this routine are never checked, which is why the failure shows up as
a cryptic MPI collective abort instead of a clear error.

Two small unrelated bugs noticed while debugging, for completeness:

1. part3d_comm.f03 (~line 292): the pipeline receive-buffer resize uses recv_cnt(p_fwd)
   where it should be recv_cnt(p_bwd).
2. write_err calls mpi_finalize + stop on one rank instead of mpi_abort, so the
   remaining ranks hang after a fatal error.