-
Notifications
You must be signed in to change notification settings - Fork 114
Lazy experiment lifecycle: configure → optional prior checks → fit() #1175
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
drbenvincent
wants to merge
21
commits into
pymc6_and_pymcmarketing1_migration
Choose a base branch
from
issue-1092-lazy-experiment-lifecycle
base: pymc6_and_pymcmarketing1_migration
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
21 commits
Select commit
Hold shift + click to select a range
6157ce6
Lazy experiment lifecycle: configure → optional prior checks → fit()
drbenvincent 232b8b3
Migrate test suite to the lazy lifecycle API
drbenvincent b1503be
Document the lazy lifecycle in ARCHITECTURE.md, skill, release notes
drbenvincent e98c1a1
Address round-1 review findings
drbenvincent d801d33
Guard BBETS build() against post-conditioning node poisoning and pin …
drbenvincent c7c2e07
Pin the BBETS re-arm path in the refit regression test
drbenvincent 120981d
Cover the prior-phase contract; fix two latent bugs surfaced by coverage
drbenvincent e8caf65
Migrate docs notebook corpus to the lazy lifecycle API
drbenvincent 859df08
Pin the new JAX-dependent notebook skip in the consistency test
drbenvincent 021a0af
Fix doubled result-bundle prefixes in five notebooks
drbenvincent 20351ab
Rename GroupNotSampleedException to GroupNotSampledException
drbenvincent 1a5f0f5
Type the lifecycle precisely: generic result bundles and deduped assi…
drbenvincent 5753122
Honor the requested draw group in PanelRegression plots
drbenvincent bb15741
Keep stub-plot experiments reportable
drbenvincent 04a2d58
Document lifecycle forwarders under Other Parameters
drbenvincent 410dbc4
Deduplicate result-bundle fallback in maketables adapters
drbenvincent 2964544
Fix ARCHITECTURE drift and document the IV build() exception
drbenvincent af0830d
Fix second-pass review findings: IV fit kwargs, refit ppc, prior pred…
drbenvincent b83a015
Align remaining test fakes and IV forwarder fragments
drbenvincent 4b064b8
Pin prior-phase defaults, guard rebuild-with-changed-inputs, silence …
drbenvincent 7e1a63c
Address final review: dedupe PrePostNEGD counterfactual, persist IV p…
drbenvincent File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,150 @@ | ||
| # Copyright 2022 - 2026 The PyMC Labs Developers | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
| """Per-experiment result bundles for the lazy experiment lifecycle. | ||
|
|
||
| Every bundle is *fully populated* whenever it exists: an experiment either has | ||
| no bundle yet (the corresponding lifecycle verb has not run) or a complete one. | ||
| Nothing derived from model draws lives directly on the experiment object; the | ||
| two public slots ``experiment.result`` (posterior group) and | ||
| ``experiment.prior_result`` (prior group) hold these bundles and back the | ||
| ``is_fitted`` / ``has_prior_predictive`` state predicates. | ||
|
|
||
| All prediction/impact fields are typed :class:`xarray.DataArray` on the | ||
| canonical ``("chain", "draw", "obs_ind"[, ...])`` dimensions produced by | ||
| :class:`~causalpy.experiments.model_adapter.ModelAdapter.predict`. | ||
| """ | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| from dataclasses import dataclass, field | ||
|
|
||
| import pandas as pd | ||
| import xarray as xr | ||
|
|
||
| __all__ = [ | ||
| "CausalResult", | ||
| "CoefficientResult", | ||
| "DiscontinuityResult", | ||
| "GroupComparisonScenario", | ||
| "KinkResult", | ||
| "ResultBundle", | ||
| "StaggeredDifferenceInDifferencesResult", | ||
| "SyntheticDifferenceInDifferencesResult", | ||
| ] | ||
|
|
||
|
|
||
| @dataclass(frozen=True) | ||
| class CausalResult: | ||
| """Result bundle for predict-contrast experiments. | ||
|
|
||
| Used by :class:`~causalpy.experiments.interrupted_time_series.InterruptedTimeSeries`, | ||
| :class:`~causalpy.experiments.synthetic_control.SyntheticControl`, and | ||
| :class:`~causalpy.experiments.piecewise_its.PiecewiseITS`. | ||
|
|
||
| For ``PiecewiseITS`` ``predictions_pre`` carries the fitted expectation over | ||
| the full observation window and ``predictions_post`` / ``impact_post`` / | ||
| ``impact_post_cumulative`` carry the post-first-interruption slices consumed | ||
| by the reporting helpers. | ||
| """ | ||
|
|
||
| predictions_pre: xr.DataArray | ||
| predictions_post: xr.DataArray | ||
| impact_pre: xr.DataArray | ||
| impact_post: xr.DataArray | ||
| impact_post_cumulative: xr.DataArray | ||
| score: pd.Series | None = None | ||
|
|
||
|
|
||
| @dataclass(frozen=True) | ||
| class SyntheticDifferenceInDifferencesResult(CausalResult): | ||
| """Result bundle for :class:`~causalpy.experiments.synthetic_difference_in_differences.SyntheticDifferenceInDifferences`. | ||
|
|
||
| The ``CausalResult`` prediction/impact fields are reconstructed from the | ||
| synthetic-control imputation; ``tau_posterior`` carries the analytic | ||
| double-difference treatment-effect draws with dimensions | ||
| ``("chain", "draw")``. | ||
| """ | ||
|
|
||
| tau_posterior: xr.DataArray = field(kw_only=True) | ||
|
|
||
|
|
||
| @dataclass(frozen=True) | ||
| class GroupComparisonScenario: | ||
| """One scenario plotted or summarized by a group-comparison experiment.""" | ||
|
|
||
| inputs: pd.DataFrame | ||
| prediction: xr.DataArray | ||
|
|
||
|
|
||
| @dataclass(frozen=True) | ||
| class CoefficientResult: | ||
| """Result bundle for coefficient-contrast experiments (DiD, PrePostNEGD). | ||
|
|
||
| ``causal_impact`` holds draws of the treatment-effect coefficient (or its | ||
| algebraically-equivalent prediction contrast) with canonical coefficient | ||
| dimensions. | ||
| """ | ||
|
|
||
| causal_impact: xr.DataArray | ||
| scenario_control: GroupComparisonScenario | ||
| scenario_treated: GroupComparisonScenario | ||
| scenario_counterfactual: GroupComparisonScenario | ||
| score: pd.Series | None = None | ||
|
|
||
|
|
||
| @dataclass(frozen=True) | ||
| class DiscontinuityResult: | ||
| """Result bundle for :class:`~causalpy.experiments.regression_discontinuity.RegressionDiscontinuity`.""" | ||
|
|
||
| predictions: xr.DataArray | ||
| discontinuity_at_threshold: xr.DataArray | ||
| score: pd.Series | None = None | ||
|
|
||
|
|
||
| @dataclass(frozen=True) | ||
| class KinkResult: | ||
| """Result bundle for :class:`~causalpy.experiments.regression_kink.RegressionKink`.""" | ||
|
|
||
| predictions: xr.DataArray | ||
| gradient_change: xr.DataArray | ||
| score: pd.Series | None = None | ||
|
|
||
|
|
||
| @dataclass(frozen=True) | ||
| class StaggeredDifferenceInDifferencesResult: | ||
| """Result bundle for :class:`~causalpy.experiments.staggered_did.StaggeredDifferenceInDifferences`. | ||
|
|
||
| ``att_group_time`` and ``att_event_time`` are aggregated ATT tables; | ||
| ``y_pred`` retains the raw counterfactual draws so placebos and alternate | ||
| HDI levels can re-derive effects without resampling. | ||
| """ | ||
|
|
||
| att_group_time: pd.DataFrame | ||
| att_event_time: pd.DataFrame | ||
| y_pred: xr.DataArray | ||
| hdi_prob: float | ||
| score: pd.Series | None = None | ||
|
|
||
|
|
||
| #: Every bundle type the raising ``result`` / ``prior_result`` properties may | ||
| #: return, one per experiment family. Experiments without bundles (IV, IPW, | ||
| #: PanelRegression) never produce these. | ||
| ResultBundle = ( | ||
| CausalResult | ||
| | CoefficientResult | ||
| | DiscontinuityResult | ||
| | KinkResult | ||
| | StaggeredDifferenceInDifferencesResult | ||
| | SyntheticDifferenceInDifferencesResult | ||
| ) |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The
HDI_PROBrow got dropped from this table in the rewrite. Base had:That's still true, still surprising (most people assume 0.95), and this was the only place it was written down. Looks like collateral from removing the "Eager fitting" row right next to it.
Fix: put the row back.