fix(theta): keep prediction intervals centered for STM/OTM models - #1181
Open
Vedant-Agarwal wants to merge 1 commit into
Open
fix(theta): keep prediction intervals centered for STM/OTM models#1181Vedant-Agarwal wants to merge 1 commit into
Vedant-Agarwal wants to merge 1 commit into
Conversation
compute_pi_samples evolved the drift terms A and B for every model type, but the reference recursion in src/theta.cpp only updates them for the dynamic models (DSTM/DOTM) — for STM/OTM they are held constant. The simulated prediction intervals for STM/OTM therefore drifted off the analytic point forecast, with a bias that grew over the horizon. Thread the model type into compute_pi_samples, guard the A/B update accordingly, and use the pre-update mean_y for B (matching the C++ reference).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
For the STM and OTM Theta models, the simulated prediction intervals drift off the analytic point forecast, with a bias that grows over the horizon.
compute_pi_samplesre-implements the state recursion used for the point forecast, but it evolves the drift termsAandBon every step for all model types. The referenceupdate()insrc/theta.cpponly evolves them for the dynamic models (DSTM/DOTM); for STM/OTM it holdsAn/Bnconstant:compute_pi_samplesalso isn't passed the model type, so it can't apply this distinction, and it computesBusing the post-updatemean_y(the C++ uses the pre-update value).Evidence
Fitting
OptimizedTheta(OTM) and comparing the PI sample mean to the analytic point forecast (300k samples, so Monte-Carlo error ≈ 1e-3):The OTM/STM bias is systematic and grows with the horizon (DOTM/DSTM are already correct). With the fix, the OTM drift collapses to
~-0.005— the intervals stay centered on the point forecast.Honest note on magnitude: the effect is systematic but small relative to the interval width, and it's masked by Monte-Carlo noise at the default
n_samples=200(which is why it isn't obvious at a glance). It's a correctness fix — the PI simulation should match the model's own point-forecast recursion — rather than a large numerical error.Fix
Thread
modeltypeintocompute_pi_samples, guard theA/Bupdate so it only runs for DSTM/DOTM, and use the pre-updatemean_yforB(matchingsrc/theta.cpp).modeltype=Nonepreserves the previous behaviour for any external caller.Test
Added
test_theta_pi_stays_centered_for_static_models: fits STM/OTM/DSTM/DOTM and asserts the PI sample mean tracks the analytic point forecast (max drift < 0.05) — fails on the previous behaviour for STM/OTM.