Skip to content

Commit 7cea471

Browse files
authored
feat(v4): item 6 of the 4.0 cut - unify the DDD results container (M-014, M-140, M-141) (#878)
1 parent 2437e80 commit 7cea471

47 files changed

Lines changed: 3352 additions & 1993 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎METHODOLOGY_REVIEW.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -987,7 +987,7 @@ These three are feature deferrals (paper-supported extensions that the library h
987987

988988
| Field | Value |
989989
|-------|-------|
990-
| Module | `_staggered_triple_diff_engine.py` (the engine behind `TripleDifference`'s staggered mode; the `staggered_triple_diff.py` class surface was removed at 4.0), `staggered_triple_diff_results.py` |
990+
| Module | `_staggered_triple_diff_engine.py` (the engine behind `TripleDifference`'s staggered mode; the `staggered_triple_diff.py` class surface was removed at 4.0), `triple_diff_results.py` (the unified `TripleDifferenceResults`, both designs, since item 6) |
991991
| Primary Reference | Ortiz-Villavicencio & Sant'Anna (2025) — same paper as TripleDifference, staggered case |
992992
| R Reference | `triplediff::ddd(panel=TRUE)` + `agg_ddd()` (per `benchmarks/R/benchmark_staggered_triplediff.R`) |
993993
| Status | **Complete** |
@@ -1005,7 +1005,7 @@ These three are feature deferrals (paper-supported extensions that the library h
10051005
- **Three-term DDD decomposition (Eq. 4.1):** post-treatment ATT(g,t) recover a known constant effect.
10061006
- **GMM combination (Eqs. 4.11-4.12):** optimal weights sum to one; a single comparison group reduces to `w=[1]`.
10071007
- **Event study (Eq. 4.13):** ES(e) equals the eligible-treated cohort-share-weighted average of ATT(g, g+e).
1008-
- **Overall (Eq. 4.14 / Cor. 4.2):** opt-in `overall_att_es` = unweighted mean of post-treatment ES(e), cross-validated against R `agg_ddd(type="eventstudy")$overall.att`/`overall.se`.
1008+
- **Overall (Eq. 4.14 / Cor. 4.2):** post-fit `aggregate("simple", weights="event_time")` (the 3.x fit-time `overall_att_es`) = unweighted mean of post-treatment ES(e), cross-validated against R `agg_ddd(type="eventstudy")$overall.att`/`overall.se`.
10091009

10101010
**R Comparison Results** (`benchmarks/R/benchmark_staggered_triplediff.R`; `triplediff::ddd(panel=TRUE)` + `agg_ddd()`; CSV fixtures gitignored / regenerated on-the-fly, JSON golden committed):
10111011

@@ -1014,8 +1014,8 @@ These three are feature deferrals (paper-supported extensions that the library h
10141014
| Group-time ATT(g,t) | rtol 0.1% | exact |
10151015
| Group-time SE(g,t) | rtol 1% | matches |
10161016
| Event-study ES(e) | rtol 25% | within (per-e eligible-treated weighting deviation) |
1017-
| Overall ATT, Eq. 4.14 (`overall_att_es`) | rtol 10% | ≤5% (weighting deviation averages out in the mean) |
1018-
| Overall SE, Eq. 4.14 (`overall_se_es`) | rtol 3% | ≤0.5% |
1017+
| Overall ATT, Eq. 4.14 (`aggregate("simple", weights="event_time")`) | rtol 10% | ≤5% (weighting deviation averages out in the mean) |
1018+
| Overall SE, Eq. 4.14 (same call, `se`) | rtol 3% | ≤0.5% |
10191019

10201020
The paper-equation-anchored Verified Components above are deterministic and run without R.
10211021
The R cross-validation in this table runs only when local `R` + `triplediff` are available

‎TODO.md‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,9 +26,8 @@ Related tracking surfaces:
2626
| Numeric between-period cohorts (e.g. `first_treat=4.5` with integer times) are rejected by LWDiD while CallawaySantAnna estimates them and LWDiD's own datetime/Period cohorts map to the next observed period — close the dtype asymmetry by adopting the next-observed-period mapping for numeric cohorts too (contract documented in REGISTRY cohort-encodings Note + `docs/api/lwdid.rst` Input Contract). Lands only after PR #588 merges | `diff_diff/lwdid.py` | #588 | Quick | Low |
2727
| Implement the LW 2026 eq. 7.9/7.10 unit-average cohort estimand (regress per-unit post-average transformed outcomes on `[1, D_g]` vs never-treated) as an alternative to the documented cell-mass `cohort_effects` convention (REGISTRY within-cohort aggregation Note; the two differ on unbalanced panels, where cell-mass weights units by observed post periods). Needs the 7.10 regression + its covariance on the NT path. Lands only after PR #588 merges | `diff_diff/lwdid_staggered.py` | #588 | Quick | Low |
2828
| Expose cell-mass overall ATT (Stata `Post_avg` convention; = CS-simple on balanced panels) as an aggregate extra on LWDiD results — the fit's `.att` is the paper's `tau_omega` (cohort-mean-then-treated-weight, eq. 7.18); the authors' large-N display uses cell-mass weighting instead, and both are legitimate estimands (see the REGISTRY LWDiD Aggregation note). Lands only after PR #588 merges | `diff_diff/lwdid_results.py` | #588 | Quick | Low |
29-
| Post-fit `aggregate()` for the staggered DDD container: `StaggeredTripleDiffResults` carries no `AggregationMixin`, which is why the phase-3(b) merge had to carry fit-time `aggregate=`/`balance_e=` onto the surviving `TripleDifference` (rows M-140/M-141) as the ONE documented exception to the section-6 aggregate-postfit program. Porting the container onto the M-122 aggregation contract retires both rows; note the bootstrapped-fit recompute levels will need replay or a fail-closed relay — solved for CS, DMLDiD, and EfficientDiD via the BootstrapReplaySpec state replay (the container port can adopt the same mechanism); ImputationDiD/TwoStageDiD/ContinuousDiD still track theirs. Until it lands, the DDD docs deliberately keep teaching the fit-time kwarg (the canonical route there) | `diff_diff/staggered_triple_diff_results.py`, `diff_diff/aggregation.py`, `docs/api/triple_diff.rst`, `docs/tutorials/08_triple_diff.ipynb` | 3(b) | Heavy | Medium |
3029
| Staggered-DDD power support: `simulate_power`/`simulate_mde`/`simulate_sample_size` now REJECT a staggered-configured `TripleDifference` (both registered DDD generators emit 2x2x2 data and fit with `(group, partition, post)`, so a staggered config would be simulated under the wrong design). Support needs a staggered DDD DGP profile plus fit-kwargs builder, and a decision on whether the mode is selected by profile or by the estimator's own config | `diff_diff/power.py` | 3(b) | Mid | Low |
31-
| Bootstrap-`seed` provenance on multiplier-bootstrap results containers: neither `StaggeredTripleDiffResults` nor `CallawaySantAnnaResults` carries the `seed` that generated its bootstrap SEs / p-values / sup-t bands, so a serialized result cannot report the random configuration behind its inference. NOT a 3(b) regression - `seed` reaches the engine and `get_params()` correctly (same seed reproduces the SE bit-exactly, a different seed moves it), the gap is results-object observability only, it predates the merge, and both containers inherit it from the shared `CallawaySantAnnaBootstrapMixin`. Add `seed` (and consider `n_bootstrap`/`bootstrap_weights`/`cband`) to BOTH containers plus `to_dict()`, with seeded and unseeded pins; sequence it with the M-014 container unification rather than schema-changing one container mid-merge. Precedent for exposing it: `ContinuousDiDResults`, `EfficientDiDResults`, `SyntheticDiDResults` already do | `diff_diff/staggered_triple_diff_results.py`, `diff_diff/staggered_results.py` | 3(b) | Quick | Low |
30+
| Bootstrap-`seed` provenance on multiplier-bootstrap results containers: neither `TripleDifferenceResults` (staggered design) nor `CallawaySantAnnaResults` carries the `seed` that generated its bootstrap SEs / p-values / sup-t bands, so a serialized result cannot report the random configuration behind its inference. NOT a 3(b) regression - `seed` reaches the engine and `get_params()` correctly (same seed reproduces the SE bit-exactly, a different seed moves it), the gap is results-object observability only, it predates the merge, and both containers inherit it from the shared `CallawaySantAnnaBootstrapMixin`. Add `seed` (and consider `n_bootstrap`/`bootstrap_weights`/`cband`) to BOTH containers plus `to_dict()`, with seeded and unseeded pins; the M-014 container unification (4.0, item 6) landed without it - the unified container records `n_bootstrap` / `inference_method` but not `seed` - so add it to BOTH containers in one PR. Precedent for exposing it: `ContinuousDiDResults`, `EfficientDiDResults`, `SyntheticDiDResults` already do | `diff_diff/triple_diff_results.py`, `diff_diff/staggered_results.py` | 3(b) | Quick | Low |
3231
| Staggered-mode cluster-robust ANALYTICAL SEs: `cluster=` raises in `TripleDifference`'s staggered mode, so clustered inference there is bootstrap-only. Implementing a clustered analytical path for the GMM-combined influence function would let the raise become a real lane | `diff_diff/_staggered_triple_diff_engine.py` | 3(b) | Heavy | Low |
3332
| diagnostic_report admission for `EventStudyResults` surfaces (the TWFE event-study mode + `aggregate('event_study')` containers): DiagnosticReport/BusinessReport now REJECT the surface explicitly (Phase 3(a); previously a silent zero-check report / all-null headline) and practitioner_next_steps serves the generic fall-through - admission needs source-aware routing (the type-name-keyed `_APPLICABILITY`/`_HANDLERS` registries cannot discriminate the unified container's producers) and a scalar-vs-per-period headline design; MPD-native results received {parallel_trends, pretrends_power, sensitivity, bacon, design_effect}. PR 4c-ii (2026-10-25) DELETED the MPD-native branches that logic lived in (DiagnosticReport's `interaction_indices`+`vcov` joint-Wald covariance source, the `post_period_effects` heterogeneity source, the `pre_period_effects` Bonferroni/NaN-filter collector branch, the `avg_att` alias row, practitioner's `_handle_multi_period`) - recover them from git history (`git show 5a10b847^:diff_diff/diagnostic_report.py`) when admission lands; event-study results reach `_handle_generic` meanwhile | `diff_diff/diagnostic_report.py`, `diff_diff/business_report.py`, `diff_diff/practitioner.py` | 3(a) | Mid | Medium |
3433
| DiagnosticReport public skip bookkeeping omits RUNNER-level skips: `applicable_checks` reflects only gate outcomes, so a check whose runner returns `status="skipped"` (heterogeneity's empty/non-finite branches, `_pt_event_study`'s empty-coefs branch - a pre-existing pattern, now also reachable via a failed post-fit event-study derivation on bootstrapped / kit-less ImputationDiD/TwoStageDiD/ContinuousDiD fits) stays listed as applicable while `skipped_checks`/`schema["skipped"]` omit it, so automation reading the public fields can misclassify the check as completed; reconcile runner-returned skipped sections into the public bookkeeping (all checks, one convention) or resolve those availabilities at the gate | `diff_diff/diagnostic_report.py` | derived-ES review | Mid | Low |
Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
### Breaking Changes
2+
3+
- **4.0 cut, DDD container unification (`M-014`, `M-140`, `M-141`).** `TripleDifference.fit()` returns ONE `TripleDifferenceResults` for both designs: `results.design` is `"2x2x2"` or `"staggered"`, the canonical `att` / `se` / `t_stat` / `p_value` / `conf_int` are native in both, the 2x2x2 positional layout is unchanged, and the staggered structure (`group_time_effects`, `groups`, `time_periods`, unit counts, `control_group`, `base_period`, `anticipation`, `bootstrap_results`, ...) is appended with defaults (`group_time_effects` is `{}` in the 2x2x2 design). The 3.x `StaggeredTripleDiffResults` class, its module, its `overall_*` fields and `to_dict()` keys, and its fit-time `overall_att_es` / `overall_se_es` / `overall_t_stat_es` / `overall_p_value_es` / `overall_conf_int_es` fields are REMOVED (3.x staggered pickles cannot load; 3.x 2x2x2 pickles load unchanged). The fit-time `aggregate=` / `balance_e=` keywords of `TripleDifference.fit()` are REMOVED (`TypeError`): aggregation is post-fit - `results.aggregate('event_study')` (`EventStudyResults`, `balance_e=`), `results.aggregate('group')` (`AggregationResult`) and `results.aggregate('simple')` (a one-row relay of the stored headline, both designs); bootstrapped fits REPLAY the fit-time multiplier bootstrap so sup-t bands and percentile inference match a 3.x fit-time aggregation; the 2x2x2 design fails the staggered levels closed. The paper's Eq. 4.14 overall is `results.aggregate('simple', weights='event_time')` (accepts `balance_e=`; both 3.x warnings preserved). `summary()` no longer prints the event-study / cohort / Eq. 4.14 blocks; `to_dataframe('event_study' | 'group')` raise with the post-fit pointer; `to_dataframe(level=None)` defaults to `'group_time'` in the staggered design and `'overall'` in 2x2x2 (whose one-row frame gains `design`, `pscore_trim`, `alpha`, `df_inference`). New field `df_inference` records the 2x2x2 engine's t df. `plot_group_effects` and its siblings raise `ValueError` on a 2x2x2 fit (no cells).
4+
- **Report schemas.** For staggered DDD fits `DiagnosticReport.to_dict()['estimator']` and `BusinessReport.to_dict()['estimator']['class_name']` read `TripleDifferenceResults`, and `target_parameter.headline_attribute` reads `"att"` (the last `"overall_att"`). Both schemas gain an additive key - DR a top-level `design`, BR `estimator.design` - `"2x2x2"` / `"staggered"` for `TripleDifferenceResults`, `None` otherwise. No version bump: `DIAGNOSTIC_REPORT_SCHEMA_VERSION` 3.0 and `BUSINESS_REPORT_SCHEMA_VERSION` 4.0 had not shipped (folded into the PR 5b bump). `power.py`'s staggered-boundary guard no longer keys on `aggregate` / `balance_e` (an `estimator_kwargs` carrying them surfaces as `fit()`'s `TypeError`).
5+
6+
### Fixed
7+
8+
- **`DiagnosticReport` kept every event-study horizon on fractional calendars.** The parallel-trends and heterogeneity collectors keyed the derived `aggregate('event_study')` surface (and compared raw `event_study_effects` keys) through `int(k)`, so on a half-step calendar (`e = t - g` in `-1.5, -1.0, -0.5, ...`) `-1.5` merged into `-1` and `-0.5` into the treatment horizon `0`: pre-periods silently left the PT family and a violated horizon could read as a clean verdict. Integer-valued horizons keep their `int` keys; fractional horizons stay `float`, compare at their real value against the anticipation-aware boundary, and align with the covariance index for the joint Wald. Affected every post-fit event-study producer routed through the report (CallawaySantAnna included), not only the staggered DDD route this release enables.
9+
10+
### Internal
11+
12+
- Ledger rows `M-014`, `M-140`, `M-141` flip to `removed` with `tests/test_v4_unify_ddd.py` as their `test_ref`; `diff_diff/staggered_triple_diff_results.py` is renamed to `diff_diff/triple_diff_results.py` (hosting the unified container, re-exported from `triple_diff.py`); the staggered engine attaches the CallawaySantAnna aggregation kit (`staggered.py`'s `_build_aggregation_kit` on the eligibility-zeroed precomputed copy, plus the `ddd_df_survey_original` bookkeeping key for the Eq. 4.14 df rule); report consumers dispatch on the design through `_reporting_helpers.results_design` / `is_staggered_ddd` (DR's `_DESIGN_APPLICABILITY` / `_DESIGN_PT_METHOD`); the practitioner handler gains a staggered branch; `docs/api/_autosummary` regenerated; the spec's section 4.2 / section 6 / item-6 SHIPPED notes and the REGISTRY `## TripleDifference` post-fit Note record the mechanics and the `weights='event_time'` decision.

‎diff_diff/__init__.py‎

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -253,9 +253,6 @@
253253
CSBootstrapResults,
254254
GroupTimeEffect,
255255
)
256-
from diff_diff.staggered_triple_diff_results import (
257-
StaggeredTripleDiffResults,
258-
)
259256
from diff_diff.sun_abraham import (
260257
SABootstrapResults,
261258
SunAbraham,
@@ -383,7 +380,6 @@
383380
"TwoStageBootstrapResults",
384381
"SpilloverDiDResults",
385382
"TripleDifferenceResults",
386-
"StaggeredTripleDiffResults",
387383
"TROPResults",
388384
"SyntheticControlResults",
389385
"StackedDiDResults",

‎diff_diff/_reporting_helpers.py‎

Lines changed: 35 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -19,11 +19,29 @@
1919

2020
from __future__ import annotations
2121

22-
from typing import Any, Dict
22+
from typing import Any, Dict, Optional
2323

2424
from diff_diff.results_base import Diagnostic
2525

2626

27+
def results_design(results: Any) -> Optional[str]:
28+
"""The fitted DESIGN of a results container that serves two designs from one class.
29+
30+
Scoped to ``TripleDifferenceResults`` (4.0, row M-014: ``"2x2x2"`` or
31+
``"staggered"``); ``None`` for every other class. Scoped on purpose - the
32+
HAD containers carry an unrelated ``design`` field (their dose-support
33+
mode), which must not leak into the report schemas' ``design`` key.
34+
"""
35+
if type(results).__name__ != "TripleDifferenceResults":
36+
return None
37+
return getattr(results, "design", None)
38+
39+
40+
def is_staggered_ddd(results: Any) -> bool:
41+
"""True for a ``TripleDifferenceResults`` fitted in the staggered design."""
42+
return results_design(results) == "staggered"
43+
44+
2745
def describe_target_parameter(results: Any) -> Dict[str, Any]:
2846
"""Return the target-parameter block for a fitted result.
2947
@@ -423,6 +441,22 @@ def describe_target_parameter(results: Any) -> Dict[str, Any]:
423441
}
424442

425443
if name == "TripleDifferenceResults":
444+
# One container, two designs since 4.0 (row M-014): dispatch on the
445+
# ``design`` discriminator, never on a class name.
446+
if results_design(results) == "staggered":
447+
return {
448+
"name": "overall ATT (cohort-weighted staggered triple-difference)",
449+
"definition": (
450+
"A cohort-weighted aggregate of per-(g, g_c, t) triple-"
451+
"difference ATTs ``DDD(g, g_c, t)`` via the GMM optimal-"
452+
"combination weighting. Extends the DDD cancellation to "
453+
"staggered adoption timing with a third (eligibility) "
454+
"dimension."
455+
),
456+
"aggregation": "staggered_ddd",
457+
"headline_attribute": "att",
458+
"reference": "Ortiz-Villavicencio & Sant'Anna (2025); REGISTRY.md Sec. StaggeredTripleDifference",
459+
}
426460
return {
427461
"name": "ATT (triple-difference)",
428462
"definition": (
@@ -437,21 +471,6 @@ def describe_target_parameter(results: Any) -> Dict[str, Any]:
437471
"reference": "Ortiz-Villavicencio & Sant'Anna (2025); REGISTRY.md Sec. TripleDifference",
438472
}
439473

440-
if name == "StaggeredTripleDiffResults":
441-
return {
442-
"name": "overall ATT (cohort-weighted staggered triple-difference)",
443-
"definition": (
444-
"A cohort-weighted aggregate of per-(g, g_c, t) triple-"
445-
"difference ATTs ``DDD(g, g_c, t)`` via the GMM optimal-"
446-
"combination weighting. Extends the DDD cancellation to "
447-
"staggered adoption timing with a third (eligibility) "
448-
"dimension."
449-
),
450-
"aggregation": "staggered_ddd",
451-
"headline_attribute": "overall_att",
452-
"reference": "Ortiz-Villavicencio & Sant'Anna (2025); REGISTRY.md Sec. StaggeredTripleDifference",
453-
}
454-
455474
if name == "ChaisemartinDHaultfoeuilleResults":
456475
l_max = getattr(results, "L_max", None)
457476
has_controls = getattr(results, "covariate_residuals", None) is not None

0 commit comments

Comments
 (0)