Skip to content

feat(v4): item 6 of the 4.0 cut - unify the DDD results container (M-014, M-140, M-141) - #878

Merged
igerber merged 2 commits into
mainfrom
feat/v4-pr6-ddd-container-unification
Oct 28, 2026
Merged

igerber merged 2 commits into
mainfrom
feat/v4-pr6-ddd-container-unification

Conversation

@igerber

@igerber igerber commented Oct 27, 2026

Copy link
Copy Markdown
Owner

Summary

  • 4.0 cut, item 6 (docs/v4-design.md section 9; rows M-014, M-140, M-141): the DDD container unification. TripleDifference.fit() returns ONE TripleDifferenceResults for both designs. The surviving 2x2x2 class IS the unified container - its positional layout is byte-stable (the four cell counts and estimation_method gained defaults), the staggered structure (group_time_effects, groups, time_periods, unit counts, control_group, base_period, anticipation, bootstrap_results, ...) is appended with defaults, and design ("2x2x2" / "staggered") discriminates (__post_init__ validates the design-specific invariants). The canonical quintet is native in both designs (born canonical, after item 5); group_time_effects is {} in 2x2x2; event_study_effects / group_effects / cband_crit_value are carrier fields, None on every 4.0 fit; the five fit-time overall_*_es fields and the never-populated influence_functions are dropped; new df_inference records the 2x2x2 engine's t df (the LWDiDResults.df_inference contract) so aggregate('simple')'s df column is honest in both designs. The module moved staggered_triple_diff_results.py -> triple_diff_results.py (git mv; re-exported from triple_diff.py, the 3.x pickle class path) so the shared staggered engine imports the container without a cycle. StaggeredTripleDiffResults is removed from diff_diff / __all__ (3.x staggered pickles cannot load, like every removed class; 3.x 2x2x2 pickles load via __setstate__ backfill).
  • Post-fit aggregate() (rows M-140 / M-141; the fit-time aggregate= / balance_e= keywords are REMOVED - TypeError in both designs): _AGGREGATE_SUPPORTED = ('simple', 'event_study', 'group'), no 'total'. 'simple' relays the stored headline in both designs (n_kind units / obs; df from resolve_inference_df); 'event_study' / 'group' recompute from the retained CallawaySantAnna kit in the staggered design (staggered.py's _build_aggregation_kit on the eligibility-zeroed precomputed copy, so the REGISTRY "Deviation from R" weighting is preserved exactly) and on bootstrapped fits REPLAY the fit-time multiplier bootstrap from the kit's BootstrapReplaySpec (the CS / DMLDiD / EfficientDiD mechanism); the 2x2x2 design fails those levels (and weights= / balance_e=) closed naming first_treat=. The paper's Eq. 4.14 overall (the 3.x overall_att_es family) is aggregate('simple', weights='event_time') - a weights= SELECTOR on the existing type (the LWDiD / Wooldridge hook), not a vocabulary word (user decision 2026-10-27) - accepting balance_e=, reproducing the engine's df rule (per-statistic effective df, else the fit's sentinelled survey df carried as the new ddd_df_survey_original bookkeeping key, so NaN-at-fit stays NaN post-fit) and both fit-time warnings; weights='event_time' on other levels and balance_e= on the plain relay are rejected rather than ignored. summary() no longer prints the event-study / cohort / Eq. 4.14 blocks (it points at the post-fit calls); to_dataframe(level=None) defaults to 'group_time' (staggered) / 'overall' (2x2x2), 'event_study' / 'group' raise with the post-fit pointer. power.py's staggered-boundary guard drops its by-value pair (an estimator_kwargs carrying aggregate surfaces as fit()'s TypeError).
  • Consumers dispatch on the DESIGN, never on a class name: _reporting_helpers.results_design / is_staggered_ddd (scoped to TripleDifferenceResults - the HAD containers carry an unrelated design field); DR keeps one class-name key per class in _APPLICABILITY / _PT_METHOD (the exhaustiveness guard builds a stub from every key) and adds _DESIGN_APPLICABILITY / _DESIGN_PT_METHOD read through _applicability_for / _pt_method_for; the "re-fit with aggregate=" derivation branch is gone (the generic aggregate('event_study') auto-derivation serves the staggered design; the 2x2x2 design has no PT row, exactly as today); BR's never-enabled narration and the practitioner handler (new staggered branch) branch on the design; headline_attribute reads "att" in both designs (the LAST "overall_att"); DR gains a top-level design key and BR estimator.design (None for every other estimator). NO schema bump (user decision 2026-10-27): DR 3.0 / BR 4.0 have not shipped - PR 5b's REPORTING.md Note is extended. results_base._ABSENT_SURFACE_HINTS gains the TripleDifferenceResults entry; build_event_study_surface(cross) now raises that hint instead of a TypeError; plot_group_effects and its two siblings raise ValueError on the empty 2x2x2 table; DR's group_time_effects branches use truthiness.
  • Ledger: M-014 / M-140 / M-141 planned -> removed with test_ref: tests/test_v4_unify_ddd.py (notes carry the mechanics, the module move, the pickle policy, the weights='event_time' decision and the power change); M-013 notes close item (6); M-050 / M-093 / M-146 code_refs follow the module rename. Spec: section 4.2 "Results" and the section-6 amendment get SHIPPED notes, the vocabulary paragraph records the selector decision, item 6 gets its SHIPPED note. REGISTRY: the "Result-class field naming" exception sentence goes (every container canonical), the TripleDifference Dispatch bullet and power Note drop the by-value pair, the results-container Note is rewritten for the 4.0 state, a new - **Note (post-fit aggregate() - rows M-014/M-140/M-141):** records the vocabulary / relay / replay / Eq. 4.14 / df-rule / warning contract, the StaggeredTripleDifference "Aggregation" prose and the R cross-validation sentence name the post-fit call, the seed Note says the unification landed without seed. TODO.md: the "post-fit aggregate() for the staggered DDD container" row is retired; the seed-provenance row is reworded. docs/doc-deps.yaml maps the renamed module (drift medium; triple_diff.rst in, staggered.rst out).
  • Docs: docs/api/triple_diff.rst (post-fit example, one-container sentence, aggregate / epv_summary in the methods list), staggered.rst loses the StaggeredTripleDiffResults section, index.rst its autosummary entry; docs/api/_autosummary/ regenerated via make -C docs html (the TripleDifferenceResults stub changes, the StaggeredTripleDiffResults stub is deleted; no other churn); llms-full.txt (staggered signature block without the two keywords, usage with the post-fit calls, the TripleDifferenceResults table with design and the staggered fields, methods incl. aggregate(...), the DR "Schema top-level keys" list gains design), llms.txt catalog line, llms-autonomous.txt BR / DR schema bullets; docs/migration-4.0.md (worked post-fit block in the DDD merge section, pickle sentence, appendix Fix cells for M-014 / M-140 / M-141 - the *(not yet implemented)* markers are gone); METHODOLOGY_REVIEW.md staggered-DDD rows; tutorial 08 cells 22-23 ported and the notebook re-executed under the Rust backend (cell 19's 2x2x2 to_dataframe() output gains the design / pscore_trim / alpha / df_inference columns).
  • tests/test_naming_guard.py: _CS_GROUPS_CLASSES gains TripleDifferenceResults (its groups names the ATT(g,t) cohort; the dying class passed only through the M-014 class exemption), the _CS_GROUPS_READER path follows the rename, the aggregate-token allowlist reasons say no fit-time aggregate= survives. tests/test_guides.py gains the TripleDifference *_fit_blocks_have_no_fit_time_aggregate pin (both signature blocks).

Methodology references (required if estimator / math changes)

  • Method name(s): N/A - no estimator or math changes. Every number is bit-identical to the 3.x fit-time surfaces: the engine still computes the same ATT(g, t) cells and the same CS-simple overall; the event-study / cohort / Eq. 4.14 aggregations run post-fit through the SAME CallawaySantAnnaAggregationMixin methods on the SAME eligibility-zeroed bookkeeping, and the bootstrap replay regenerates the SAME weight stream (pinned against literals captured on the unmodified tree at rtol 1e-9: event-study rows, cohort rows, the Eq. 4.14 quintet on the analytical / balance_e=1 / survey-pweight lanes, the bootstrap sup-t critical value, percentile SEs and the replayed Eq. 4.14 inference).
  • Paper / source link(s): Ortiz-Villavicencio & Sant'Anna (2025), arXiv:2505.09942v3 - Eq. 4.14 (the event-study-average overall), unchanged; REGISTRY ## TripleDifference / ## StaggeredTripleDifference.
  • Any intentional deviations from the source (and why): None new. The Eq. 4.14 overall moves from a fit-time field to aggregate('simple', weights='event_time') (user decision 2026-10-27, recorded in the spec vocabulary paragraph and the REGISTRY Note).

Validation

  • Tests added/updated: NEW tests/test_v4_unify_ddd.py (removal pins incl. the TypeError on both designs and the signature; shape, degenerate 2x2x2 view, positional layout, __post_init__ guards, df_inference provenance; aggregate vocabulary and guards; python-backend-guarded literal oracles - analytical ES / group / Eq. 4.14 quintet, balance_e=1, the n_bootstrap=49, seed=7, cband=True replay (cband, percentile SEs, Eq. 4.14 percentile inference), the survey-pweight lane incl. the df column; both Eq. 4.14 warnings incl. the single-PSU bootstrap lane; read surfaces under FutureWarning=error, to_dataframe levels, to_dict blocks, summary / repr / epv_summary, the duck-typed consumers; pickle round trip, simulated 3.x 2x2x2 state, the 3.x class path; describe_target_parameter, DR (staggered derives PT post-fit; 2x2x2 has no PT row; design key; None elsewhere), BR (both control groups, both designs), practitioner routing). Ported to the post-fit route: tests/test_v4_merge_ddd.py (oracle gates read aggregate(...); Gate E inverted; the removed-keyword TypeError pins; the power boundary test), test_staggered_triple_diff.py, test_survey_staggered_ddd.py, test_methodology_staggered_triple_diff.py (R golden overall_att_es keys read through aggregate('simple', weights='event_time')), test_aggregate_contract.py, test_event_study_surface.py (the legacy_standin carrier like CS), test_staggered_aggregation.py; test_result_aliases.py (SDDD exception gone), test_results_serialization.py roster, test_target_parameter.py (design-aware stub + the _DESIGN_APPLICABILITY exhaustiveness mirror), test_business_report.py stubs, test_diagnostic_report.py (_TOP_LEVEL_KEYS + the derived-PT test), test_naming_guard.py, test_guides.py.
  • Run locally: DIFF_DIFF_BACKEND=python step-1 group (unify_ddd, merge_ddd, triple_diff, result_aliases, results_serialization: 366 passed, every oracle lane un-skipped); report / consumer group under -W error::FutureWarning (target_parameter, diagnostic_report, business_report, practitioner, aggregate_contract, event_study_surface, event_study_consumers: 1310 passed); estimator group under -W error::FutureWarning (staggered_triple_diff, survey_staggered_ddd, methodology_staggered_triple_diff, survey_r_crossvalidation, power, staggered, staggered_aggregation, base_estimator: 905 passed); ledger / guard / tracking / changelog / doc-deps / guides / doc-snippets (933 passed) and docs_ia (20 passed); full default pytest and --nbmake on tutorial 08: see the run note below. CI-pinned mypy diff_diff clean (111 files); black / ruff clean; py3.9 compileall of the touched tests clean; make -C docs html exit 0.
  • Sweep gates: StaggeredTripleDiffResults / staggered_triple_diff_results survive only in provenance prose, the removal pins and the frozen capture script; overall_att_es / overall_se_es survive only in the live CSBootstrapResults.overall_att_es_* payload fields, provenance prose, the R-golden JSON keys and the inline oracle literals; the AST sweep for fit(aggregate=|balance_e=) over diff_diff/, tests/ and the notebooks returns only pytest.raises(TypeError) removal pins (verified per site) and the frozen capture script.
  • Backtest / simulation / notebook evidence (if applicable): tutorial 08 re-executed under DIFF_DIFF_BACKEND=rust; the post-fit aggregate('event_study') frame and the Eq. 4.14 row render in cell 23.

Security / privacy

  • Confirm no secrets/PII in this PR: Yes

Changelog

  • changelog.d/ fragment added (or N/A - no user-visible change): Yes - changelog.d/20261027-v4-pr6-ddd-container-unification.md (### Breaking Changes x2 + ### Internal)

Follow-ups

  • Cut sequence: 7 (df_convention + M-080 + the uniform cluster=False), 8 (M-093), 9 (M-090 docs incl. the REGISTRY archive framing for the removed classes), 10 release.
  • TODO.md seed-provenance row (both multiplier-bootstrap containers) stays open.

…014, M-140, M-141)

TripleDifference.fit returns ONE TripleDifferenceResults for both designs.
The surviving 2x2x2 class IS the unified container: its positional layout
is byte-stable (the four cell counts and estimation_method gained
defaults), the staggered structure is appended with defaults, and a
design discriminator ("2x2x2" / "staggered") says which engine fit it.
The canonical quintet is native in both designs; the group-time table is
empty in 2x2x2; event_study_effects / group_effects / cband_crit_value
are carrier fields that are None on every 4.0 fit; the five fit-time
overall_*_es fields and the never-populated influence_functions are gone;
df_inference records the 2x2x2 engine's t df (the LWDiDResults contract).
The module moved from staggered_triple_diff_results.py to
triple_diff_results.py (re-exported from triple_diff.py, the 3.x pickle
class path) so the shared staggered engine imports it without a cycle.
StaggeredTripleDiffResults is removed.

Post-fit aggregate() (rows M-140/M-141, the fit-time aggregate=/balance_e=
keywords are removed - TypeError): the container is an AggregationMixin
adopter. 'simple' relays the stored headline in both designs; 'event_study'
and 'group' recompute from the retained CallawaySantAnna kit in the
staggered design (built on the eligibility-zeroed precomputed copy so the
eligible-treated weighting is preserved) and REPLAY the fit-time multiplier
bootstrap on bootstrapped fits; the 2x2x2 design fails those levels closed
naming first_treat=. The paper's Eq. 4.14 overall (the 3.x overall_att_es
family) is aggregate('simple', weights='event_time') - a weights selector,
not a vocabulary word (user decision 2026-10-27) - reproducing the engine's
df rule (per-statistic effective df, else the fit's sentinelled survey df
carried as the ddd_df_survey_original bookkeeping key) and both fit-time
warnings. power.py's staggered-boundary guard drops its by-value pair.

Consumers dispatch on the design through _reporting_helpers.results_design
/ is_staggered_ddd (DR's _DESIGN_APPLICABILITY / _DESIGN_PT_METHOD tables
beside the class-name tables; BR's never-enabled narration; a staggered
practitioner handler); headline_attribute reads att in both designs (the
last overall_att); DR gains a top-level design key and BR
estimator.design (additive; no version bump - DR 3.0 / BR 4.0 have not
shipped, folded into PR 5b's REPORTING.md Note). plot_group_effects and
its siblings raise on the empty 2x2x2 table.

Ledger M-014 / M-140 / M-141 -> removed with tests/test_v4_unify_ddd.py as
test_ref (removal pins, shape and positional layout, the aggregate
vocabulary and guards, python-backend-guarded literal oracles for the
analytical, balance_e, bootstrap-replay and survey lanes captured on the
unmodified tree, both Eq. 4.14 warnings, read surfaces under
FutureWarning=error, the duck-typed consumers, pickles incl. the 3.x class
path, the report surface and practitioner routing). The DDD suites port
to the post-fit route; the aggregate-contract, event-study-surface,
staggered-aggregation, alias, serialization, target-parameter, report and
naming-guard suites follow. Spec section 4.2 / section 6 / item 6 SHIPPED
notes, the REGISTRY post-fit Note, REPORTING.md, the migration guide,
guides, API rst, autosummary stubs, tutorial 08 (re-executed) and the
tracking rows follow.
@github-actions

Copy link
Copy Markdown

⛔ Blocker

  • One P0: the newly enabled staggered-DDD diagnostic path silently merges fractional event-time labels, potentially hiding pre-trend violations.
  • Core aggregation changes appear consistent with the Registry and the repository’s review of Ortiz-Villavicencio & Sant’Anna (2025), v3.
  • Documented weighting deviations and tracked provenance limitations are informational.
  • A changelog fragment is present. Full tests could not run because pytest and numerical dependencies are unavailable; the diagnostic failure was reproduced using the actual source functions in isolation.

Methodology

P0 — Preserve event-time labels through automatic diagnostics.

Locations: diff_diff/diagnostic_report.py:L284-L298, L4084-L4101, L4183-L4189.

The new staggered-DDD dispatch enables automatic post-fit event-study derivation. That path reaches _surface_to_event_study_dict(), which stores rows under int(k). Accepted fractional horizons therefore collide: -1.5 and -1.0 become the same key, while -0.5 becomes zero.

Impact: Pre-treatment evidence disappears without a warning. An isolated reproduction with three pre-period p-values [1e-20, 0.92, 1e-30] retained only the middle row. The diagnostic returned p=0.92 and no_detected_violation; the full-family Bonferroni p-value is 3e-30. The adapter is unchanged, but this PR newly routes staggered-DDD results through it.

Concrete fix: Preserve numeric horizon keys and compare their actual values against the pre-period cutoff. Remove the lossy conversion from both the adapter and _collect_pre_period_coefs(), preserving covariance-index alignment. Alternatively, explicitly reject unsupported fractional horizons before publishing diagnostics.

P3 — Documented aggregation conventions; no action required.

Location: docs/methodology/REGISTRY.md:L4411-L4434.

Eligible-treated cohort weighting, per-group WIF inference, and the distinction between the headline ATT and Eq. 4.14 are explicitly documented. The retained aggregation machinery follows these conventions; they are not defects.

Code Quality

No additional findings beyond the event-time conversion above.

Performance

No blocking findings identified.

Maintainability

No additional findings identified.

Tech Debt

P3 — Bootstrap seed provenance remains tracked.

Location: TODO.md:L30.

Impact: Serialized results do not directly expose the bootstrap seed. This predates the PR and is explicitly tracked.

Concrete fix: No action required for this PR; retain the coordinated container-provenance follow-up.

Security

No security findings identified in the reviewed changes.

Documentation/Tests

P2 — The balancing oracle does not exercise partial cohort removal.

Locations: tests/test_v4_unify_ddd.py:L87-L90, L363-L366.

Impact: The balance_e=1 oracle equals the unbalanced oracle because both cohorts remain eligible. It cannot catch incorrect reweighting or replay when balancing removes some cohorts.

Concrete fix: Add a balance_e=3 case using the existing cohorts [3, 4] and periods 1..6, retaining cohort 3 while excluding cohort 4. Check event-study and Eq. 4.14 estimates and inference for analytical and bootstrapped fits. This is nonblocking.

The changelog requirement is satisfied. Methodology verification used the Registry and repository’s version-pinned scholarly review; the primary paper PDF was not independently retrieved.

Path to Approval

  1. Preserve fractional horizons throughout the diagnostic adapter and pre-period collector, or fail closed explicitly.
  2. Add a regression with horizons [-1.5, -1.0, -0.5] verifying that all three pre-periods survive, covariance labels remain aligned, and the diagnostic cannot return the demonstrated false reassurance. Include a staggered-DDD fit on a half-step calendar grid to exercise the newly enabled route.

… and heterogeneity collectors

CI review on PR #878 (P0): the DR adapter keyed the derived
aggregate('event_study') surface through int(k) and the pre-period /
heterogeneity collectors compared int(k) against the anticipation-aware
boundary. On a half-step calendar (e = t - g in -1.5, -1.0, -0.5, ...)
-1.5 merged into -1 and -0.5 into the treatment horizon 0, so pre-periods
silently left the PT family (a staggered DDD fit on a halved calendar
reported "no pre-periods"; a CS fit on the same calendar kept one of its
three). Pre-existing for every post-fit event-study producer; this PR's
staggered DDD route is the newest adopter.

_event_time_value / _event_time_key replace the four int() sites:
integer-valued horizons keep Python int keys (the ImputationDiD/TwoStageDiD
raw dtype), fractional horizons stay float, pre/post comparisons use the
numeric value, and the labels stay hash-equal to the surface's vcov_index
so the joint Wald still aligns.

Tests: adapter unit pins (horizons [-1.5, -1.0, -0.5] with p-values
[1e-20, 0.92, 1e-30] all survive, covariance index alignment, raw-field
route, helper edge cases), the heterogeneity collectors' numeric split,
and a staggered DDD half-step-calendar fit whose DiagnosticReport PT runs
the covariance-backed joint Wald over every negative horizon. Also the
review's P2: balance_e=3 oracles (cohort 4 unobserved at e=3 -> dropped,
e=-2 gone) for the analytical event study / Eq. 4.14 overall and the
bootstrap replay (per-horizon se, cband, percentile overall), captured on
the unmodified tree. REPORTING.md derived-route Note and the changelog
fragment record the fix.
@github-actions

Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: 085f3b9c401b0508c4a6699cc5d8ab7613243a17


✅ Looks good

Executive summary

  • The previous P0 is resolved: fractional event-time labels remain distinct and retain covariance alignment.
  • Isolated execution of the actual diagnostic functions preserved all three pre-periods and returned p ≈ 3e-30, with clear_violation.
  • The previous P2 balancing gap is addressed by analytical and bootstrap tests that remove cohort 4 with balance_e=3.
  • No unmitigated P0/P1 findings identified. The changelog requirement is satisfied.
  • Full numerical tests could not run because NumPy, SciPy, pandas, and pytest are unavailable.

Methodology

Previous P0 — Resolved.
Locations: diff_diff/diagnostic_report.py:L4052-L4075, L4124-L4132, L4214-L4226.

The adapter preserves fractional keys, and the collector compares their actual values against the anticipation-aware boundary. The reported false reassurance was eliminated in isolated source execution; retained labels also mapped to covariance positions [0, 1, 2]. No further fix required.

P3 — Documented aggregation conventions; no action required.
Location: docs/methodology/REGISTRY.md:L4411-L4434.

Impact: Eligible-treated weighting, per-cohort WIF inference, and the distinction between the headline ATT and Eq. 4.14 remain explicit conventions. The post-fit implementation retains the shared aggregation machinery.

Concrete fix: None. Cross-checking used the Registry, source docstrings, and the repository’s version-pinned review of Ortiz-Villavicencio & Sant’Anna (2025), v3, particularly Eqs. 4.13–4.14 and Corollary 4.2. The primary PDF was not independently retrieved.

Code Quality

No additional findings. The shared event-time helpers consistently replace truncation in the affected diagnostic collectors.

Performance

No new findings within the re-review scope.

Maintainability

No new findings within the re-review scope.

Tech Debt

P3 — Bootstrap seed provenance remains tracked.
Location: TODO.md:L30.

Impact: Results do not directly expose the seed behind bootstrap inference. This existing observability limitation remains tracked for both affected containers.

Concrete fix: No action required for this PR; retain the coordinated follow-up.

Security

No security findings identified in the reviewed changes.

Documentation/Tests

Previous P2 — Resolved.
Location: tests/test_v4_unify_ddd.py:L411-L453.

The new balance_e=3 cases exercise partial cohort removal and check event-study estimates, Eq. 4.14 inference, bootstrap SEs, and simultaneous-band critical values. No further fix required.

Fractional-label regression coverage includes adapter/covariance alignment and an actual half-step staggered-DDD diagnostic route: tests/test_diagnostic_report.py:L3726-L3790, tests/test_v4_unify_ddd.py:L674-L692.

Syntax checks passed for 15 affected source/test files. Full estimator, bootstrap, and joint-Wald integration tests remain unverified in this environment.

@igerber igerber added the ready-for-ci Triggers CI test workflows label Oct 27, 2026
@igerber
igerber merged commit 7cea471 into main Oct 28, 2026
43 of 44 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-ci Triggers CI test workflows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant