You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 7cea471
Browse filesBrowse the repository at this point in the historyBrowse files
Copy file name to clipboardExpand all lines: METHODOLOGY_REVIEW.md
+4-4Lines changed: 4 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -987,7 +987,7 @@ These three are feature deferrals (paper-supported extensions that the library h
987
987
988
988
| Field | Value |
989
989
|-------|-------|
990
-
| Module |`_staggered_triple_diff_engine.py` (the engine behind `TripleDifference`'s staggered mode; the `staggered_triple_diff.py` class surface was removed at 4.0), `staggered_triple_diff_results.py`|
990
+
| Module |`_staggered_triple_diff_engine.py` (the engine behind `TripleDifference`'s staggered mode; the `staggered_triple_diff.py` class surface was removed at 4.0), `triple_diff_results.py` (the unified `TripleDifferenceResults`, both designs, since item 6)|
991
991
| Primary Reference | Ortiz-Villavicencio & Sant'Anna (2025) — same paper as TripleDifference, staggered case |
992
992
| R Reference |`triplediff::ddd(panel=TRUE)` + `agg_ddd()` (per `benchmarks/R/benchmark_staggered_triplediff.R`) |
993
993
| Status |**Complete**|
@@ -1005,7 +1005,7 @@ These three are feature deferrals (paper-supported extensions that the library h
1005
1005
-**Three-term DDD decomposition (Eq. 4.1):** post-treatment ATT(g,t) recover a known constant effect.
1006
1006
-**GMM combination (Eqs. 4.11-4.12):** optimal weights sum to one; a single comparison group reduces to `w=[1]`.
1007
1007
-**Event study (Eq. 4.13):** ES(e) equals the eligible-treated cohort-share-weighted average of ATT(g, g+e).
1008
-
-**Overall (Eq. 4.14 / Cor. 4.2):**opt-in `overall_att_es` = unweighted mean of post-treatment ES(e), cross-validated against R `agg_ddd(type="eventstudy")$overall.att`/`overall.se`.
1008
+
-**Overall (Eq. 4.14 / Cor. 4.2):**post-fit `aggregate("simple", weights="event_time")` (the 3.x fit-time `overall_att_es`) = unweighted mean of post-treatment ES(e), cross-validated against R `agg_ddd(type="eventstudy")$overall.att`/`overall.se`.
Copy file name to clipboardExpand all lines: TODO.md
+1-2Lines changed: 1 addition & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -26,9 +26,8 @@ Related tracking surfaces:
26
26
| Numeric between-period cohorts (e.g. `first_treat=4.5` with integer times) are rejected by LWDiD while CallawaySantAnna estimates them and LWDiD's own datetime/Period cohorts map to the next observed period — close the dtype asymmetry by adopting the next-observed-period mapping for numeric cohorts too (contract documented in REGISTRY cohort-encodings Note + `docs/api/lwdid.rst` Input Contract). Lands only after PR #588 merges |`diff_diff/lwdid.py`|#588| Quick | Low |
27
27
| Implement the LW 2026 eq. 7.9/7.10 unit-average cohort estimand (regress per-unit post-average transformed outcomes on `[1, D_g]` vs never-treated) as an alternative to the documented cell-mass `cohort_effects` convention (REGISTRY within-cohort aggregation Note; the two differ on unbalanced panels, where cell-mass weights units by observed post periods). Needs the 7.10 regression + its covariance on the NT path. Lands only after PR #588 merges |`diff_diff/lwdid_staggered.py`|#588| Quick | Low |
28
28
| Expose cell-mass overall ATT (Stata `Post_avg` convention; = CS-simple on balanced panels) as an aggregate extra on LWDiD results — the fit's `.att` is the paper's `tau_omega` (cohort-mean-then-treated-weight, eq. 7.18); the authors' large-N display uses cell-mass weighting instead, and both are legitimate estimands (see the REGISTRY LWDiD Aggregation note). Lands only after PR #588 merges |`diff_diff/lwdid_results.py`|#588| Quick | Low |
29
-
| Post-fit `aggregate()` for the staggered DDD container: `StaggeredTripleDiffResults` carries no `AggregationMixin`, which is why the phase-3(b) merge had to carry fit-time `aggregate=`/`balance_e=` onto the surviving `TripleDifference` (rows M-140/M-141) as the ONE documented exception to the section-6 aggregate-postfit program. Porting the container onto the M-122 aggregation contract retires both rows; note the bootstrapped-fit recompute levels will need replay or a fail-closed relay — solved for CS, DMLDiD, and EfficientDiD via the BootstrapReplaySpec state replay (the container port can adopt the same mechanism); ImputationDiD/TwoStageDiD/ContinuousDiD still track theirs. Until it lands, the DDD docs deliberately keep teaching the fit-time kwarg (the canonical route there) |`diff_diff/staggered_triple_diff_results.py`, `diff_diff/aggregation.py`, `docs/api/triple_diff.rst`, `docs/tutorials/08_triple_diff.ipynb`| 3(b) | Heavy | Medium |
30
29
| Staggered-DDD power support: `simulate_power`/`simulate_mde`/`simulate_sample_size` now REJECT a staggered-configured `TripleDifference` (both registered DDD generators emit 2x2x2 data and fit with `(group, partition, post)`, so a staggered config would be simulated under the wrong design). Support needs a staggered DDD DGP profile plus fit-kwargs builder, and a decision on whether the mode is selected by profile or by the estimator's own config |`diff_diff/power.py`| 3(b) | Mid | Low |
31
-
| Bootstrap-`seed` provenance on multiplier-bootstrap results containers: neither `StaggeredTripleDiffResults` nor `CallawaySantAnnaResults` carries the `seed` that generated its bootstrap SEs / p-values / sup-t bands, so a serialized result cannot report the random configuration behind its inference. NOT a 3(b) regression - `seed` reaches the engine and `get_params()` correctly (same seed reproduces the SE bit-exactly, a different seed moves it), the gap is results-object observability only, it predates the merge, and both containers inherit it from the shared `CallawaySantAnnaBootstrapMixin`. Add `seed` (and consider `n_bootstrap`/`bootstrap_weights`/`cband`) to BOTH containers plus `to_dict()`, with seeded and unseeded pins; sequence it with the M-014 container unification rather than schema-changing one container mid-merge. Precedent for exposing it: `ContinuousDiDResults`, `EfficientDiDResults`, `SyntheticDiDResults` already do | `diff_diff/staggered_triple_diff_results.py`, `diff_diff/staggered_results.py` | 3(b) | Quick | Low |
30
+
| Bootstrap-`seed` provenance on multiplier-bootstrap results containers: neither `TripleDifferenceResults` (staggered design) nor `CallawaySantAnnaResults` carries the `seed` that generated its bootstrap SEs / p-values / sup-t bands, so a serialized result cannot report the random configuration behind its inference. NOT a 3(b) regression - `seed` reaches the engine and `get_params()` correctly (same seed reproduces the SE bit-exactly, a different seed moves it), the gap is results-object observability only, it predates the merge, and both containers inherit it from the shared `CallawaySantAnnaBootstrapMixin`. Add `seed` (and consider `n_bootstrap`/`bootstrap_weights`/`cband`) to BOTH containers plus `to_dict()`, with seeded and unseeded pins; the M-014 container unification (4.0, item 6) landed without it - the unified container records `n_bootstrap` / `inference_method` but not `seed` - so add it to BOTH containers in one PR. Precedent for exposing it: `ContinuousDiDResults`, `EfficientDiDResults`, `SyntheticDiDResults` already do | `diff_diff/triple_diff_results.py`, `diff_diff/staggered_results.py` | 3(b) | Quick | Low |
32
31
| Staggered-mode cluster-robust ANALYTICAL SEs: `cluster=` raises in `TripleDifference`'s staggered mode, so clustered inference there is bootstrap-only. Implementing a clustered analytical path for the GMM-combined influence function would let the raise become a real lane |`diff_diff/_staggered_triple_diff_engine.py`| 3(b) | Heavy | Low |
33
32
| diagnostic_report admission for `EventStudyResults` surfaces (the TWFE event-study mode + `aggregate('event_study')` containers): DiagnosticReport/BusinessReport now REJECT the surface explicitly (Phase 3(a); previously a silent zero-check report / all-null headline) and practitioner_next_steps serves the generic fall-through - admission needs source-aware routing (the type-name-keyed `_APPLICABILITY`/`_HANDLERS` registries cannot discriminate the unified container's producers) and a scalar-vs-per-period headline design; MPD-native results received {parallel_trends, pretrends_power, sensitivity, bacon, design_effect}. PR 4c-ii (2026-10-25) DELETED the MPD-native branches that logic lived in (DiagnosticReport's `interaction_indices`+`vcov` joint-Wald covariance source, the `post_period_effects` heterogeneity source, the `pre_period_effects` Bonferroni/NaN-filter collector branch, the `avg_att` alias row, practitioner's `_handle_multi_period`) - recover them from git history (`git show 5a10b847^:diff_diff/diagnostic_report.py`) when admission lands; event-study results reach `_handle_generic` meanwhile | `diff_diff/diagnostic_report.py`, `diff_diff/business_report.py`, `diff_diff/practitioner.py` | 3(a) | Mid | Medium |
34
33
| DiagnosticReport public skip bookkeeping omits RUNNER-level skips: `applicable_checks` reflects only gate outcomes, so a check whose runner returns `status="skipped"` (heterogeneity's empty/non-finite branches, `_pt_event_study`'s empty-coefs branch - a pre-existing pattern, now also reachable via a failed post-fit event-study derivation on bootstrapped / kit-less ImputationDiD/TwoStageDiD/ContinuousDiD fits) stays listed as applicable while `skipped_checks`/`schema["skipped"]` omit it, so automation reading the public fields can misclassify the check as completed; reconcile runner-returned skipped sections into the public bookkeeping (all checks, one convention) or resolve those availabilities at the gate |`diff_diff/diagnostic_report.py`| derived-ES review | Mid | Low |
- **4.0 cut, DDD container unification (`M-014`, `M-140`, `M-141`).** `TripleDifference.fit()` returns ONE `TripleDifferenceResults` for both designs: `results.design` is `"2x2x2"` or `"staggered"`, the canonical `att` / `se` / `t_stat` / `p_value` / `conf_int` are native in both, the 2x2x2 positional layout is unchanged, and the staggered structure (`group_time_effects`, `groups`, `time_periods`, unit counts, `control_group`, `base_period`, `anticipation`, `bootstrap_results`, ...) is appended with defaults (`group_time_effects` is `{}` in the 2x2x2 design). The 3.x `StaggeredTripleDiffResults` class, its module, its `overall_*` fields and `to_dict()` keys, and its fit-time `overall_att_es` / `overall_se_es` / `overall_t_stat_es` / `overall_p_value_es` / `overall_conf_int_es` fields are REMOVED (3.x staggered pickles cannot load; 3.x 2x2x2 pickles load unchanged). The fit-time `aggregate=` / `balance_e=` keywords of `TripleDifference.fit()` are REMOVED (`TypeError`): aggregation is post-fit - `results.aggregate('event_study')` (`EventStudyResults`, `balance_e=`), `results.aggregate('group')` (`AggregationResult`) and `results.aggregate('simple')` (a one-row relay of the stored headline, both designs); bootstrapped fits REPLAY the fit-time multiplier bootstrap so sup-t bands and percentile inference match a 3.x fit-time aggregation; the 2x2x2 design fails the staggered levels closed. The paper's Eq. 4.14 overall is `results.aggregate('simple', weights='event_time')` (accepts `balance_e=`; both 3.x warnings preserved). `summary()` no longer prints the event-study / cohort / Eq. 4.14 blocks; `to_dataframe('event_study' | 'group')` raise with the post-fit pointer; `to_dataframe(level=None)` defaults to `'group_time'` in the staggered design and `'overall'` in 2x2x2 (whose one-row frame gains `design`, `pscore_trim`, `alpha`, `df_inference`). New field `df_inference` records the 2x2x2 engine's t df. `plot_group_effects` and its siblings raise `ValueError` on a 2x2x2 fit (no cells).
4
+
-**Report schemas.** For staggered DDD fits `DiagnosticReport.to_dict()['estimator']` and `BusinessReport.to_dict()['estimator']['class_name']` read `TripleDifferenceResults`, and `target_parameter.headline_attribute` reads `"att"` (the last `"overall_att"`). Both schemas gain an additive key - DR a top-level `design`, BR `estimator.design` - `"2x2x2"` / `"staggered"` for `TripleDifferenceResults`, `None` otherwise. No version bump: `DIAGNOSTIC_REPORT_SCHEMA_VERSION` 3.0 and `BUSINESS_REPORT_SCHEMA_VERSION` 4.0 had not shipped (folded into the PR 5b bump). `power.py`'s staggered-boundary guard no longer keys on `aggregate` / `balance_e` (an `estimator_kwargs` carrying them surfaces as `fit()`'s `TypeError`).
5
+
6
+
### Fixed
7
+
8
+
-**`DiagnosticReport` kept every event-study horizon on fractional calendars.** The parallel-trends and heterogeneity collectors keyed the derived `aggregate('event_study')` surface (and compared raw `event_study_effects` keys) through `int(k)`, so on a half-step calendar (`e = t - g` in `-1.5, -1.0, -0.5, ...`) `-1.5` merged into `-1` and `-0.5` into the treatment horizon `0`: pre-periods silently left the PT family and a violated horizon could read as a clean verdict. Integer-valued horizons keep their `int` keys; fractional horizons stay `float`, compare at their real value against the anticipation-aware boundary, and align with the covariance index for the joint Wald. Affected every post-fit event-study producer routed through the report (CallawaySantAnna included), not only the staggered DDD route this release enables.
9
+
10
+
### Internal
11
+
12
+
- Ledger rows `M-014`, `M-140`, `M-141` flip to `removed` with `tests/test_v4_unify_ddd.py` as their `test_ref`; `diff_diff/staggered_triple_diff_results.py` is renamed to `diff_diff/triple_diff_results.py` (hosting the unified container, re-exported from `triple_diff.py`); the staggered engine attaches the CallawaySantAnna aggregation kit (`staggered.py`'s `_build_aggregation_kit` on the eligibility-zeroed precomputed copy, plus the `ddd_df_survey_original` bookkeeping key for the Eq. 4.14 df rule); report consumers dispatch on the design through `_reporting_helpers.results_design` / `is_staggered_ddd` (DR's `_DESIGN_APPLICABILITY` / `_DESIGN_PT_METHOD`); the practitioner handler gains a staggered branch; `docs/api/_autosummary` regenerated; the spec's section 4.2 / section 6 / item-6 SHIPPED notes and the REGISTRY `## TripleDifference` post-fit Note record the mechanics and the `weights='event_time'` decision.
0 commit comments