Skip to content

Feature: extend simulate() to residual covariances (~~) and HSGP terms #434

Description

@drbenvincent

Problem

simulate() currently raises NotImplementedError for:

  • Residual covariances (Y1 ~~ Y2) — correlated equation errors
  • hsgp() smooth terms — useful for flexible trends in MMM

Both compile in model() for cross-sectional data (HSGP); ~~ works cross-sectionally in estimation.

Scope

Phase 1: Cross-sectional ~~ in simulate() — draw correlated residuals via Cholesky of fixed covariance params in params dict.

Phase 2: Cross-sectional hsgp() in simulate() — clamp HSGP basis coefficients via pm.do(), draw smooth contribution.

Phase 3 (separate / deferred): ~~ and HSGP in scan-compiled panel models (currently limited in estimation too).

Acceptance criteria

  • Remove NotImplementedError guards in simulate() for supported cases
  • Tests mirroring tests/test_simulate_data.py validation patterns
  • Clear errors for unsupported panel+HSGP combinations (already enforced in compile)

Related

Decision (2026-08-24)

Resolved during meta-issue run #426 pre-flight:

  • Phase 1 accepts chol_{block} as the packed lower-triangular Cholesky vector (PyMC LKJCholeskyCov packing), matching what introspect already advertises. Block-member noisy draws propagate as mu_{var} deterministics into descendant equations within one draw pass (documented behavior, mirrors estimation). Phase-scoped docstring edits only: hsgp guard stays until Phase 2.
    427:Cold start at zero for temporal state (matches compile_scan_panel init semantics), documented in the API. params must supply every free RV including alpha{var} vectors under pooling='partial'; scalar values are broadcast to the required shape. Implementation may reuse the scan step functions directly rather than run_do_panel_unified's idata machinery — whichever is simpler.
    429:Template returns a plain dict[str, shape/dtype descriptor] built from the same zero-filled placeholder compile simulate() already performs internally; a typed dataclass wrapper is optional sugar, not required. Placeholder data needs real column names/dtypes but not plausible values.
    432:New module pathmc/scaling.py (AGENTS.md module list gains it); Scaling exported from pathmc/init. Methods: max / mean / fixed / divide-by-external-grid, computed per panel dimension group. simulate() applies inverse scaling only when scaling= is passed; default None = no scaling.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestgenerative-parityForward simulation (simulate) lags behind estimation (model, do)stress-testSurfaced by demanding end-to-end workflows that expose general capability gaps

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions