Add a shared time-varying parameter interface for Rt and infections - #1451
Add a shared time-varying parameter interface for Rt and infections#1451sbfnk-bot wants to merge 60 commits into
Conversation
Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
Avoids overloaded Stan functions, which rstan::expose_stan_functions cannot handle, and exposes state.stan in the test setup. Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
Non-stationary GP reuses the shared basis via a centred cumulative sum. Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk> Co-authored-by: epiforecasts-workflows[bot] <epiforecasts-workflows[bot]@users.noreply.github.com>
States vary over the observed window and hold their last value in the horizon (the 'latest' projection), matching Rt's default. Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
A param_spec is any (constant or time-varying) parameter value specification; used wherever a value may be either. Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
rt_opts(prior=) now takes a dist_spec or a state_spec (GP/RW); a time-varying Rt prior errors explicitly until the model wiring lands. Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
Rt now flows through the generic state machinery: rt_opts(prior=) takes a constant value or a GP/RW state; the bespoke update_Rt path, R_mean and init_priors are gone. Verified identical to the previous Rt within MC error. rw is deprecated (use prior = RW()). Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
…-params # Conflicts: # inst/stan/estimate_infections.stan
Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
Retire the bespoke main Gaussian process: latent infections are now a Gaussian process on log-infections anchored at an initial value, built through the same time-varying state machinery as the reproduction number. Removes the deconvolution step, the shifted-case prior, and the associated Stan functions and data, and deprecates the backcalc_opts prior arguments. Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
…ment All Gaussian process settings (ls, alpha, kernel, basis_prop, ...) are now arguments of GP(), so a process is fully specified in one place. gp_opts() is deprecated. The back-calculation infections process is configured through backcalc_opts(prior = GP(...)), mirroring rt_opts(prior = ...), and the gp argument of estimate_infections()/epinow()/regional_epinow() is deprecated. Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
Update simulate_infections() to register the R and I state parameters, and exclude these state levels (sampled scaffolding for the time-varying trajectories) from the scalar-parameter extraction so the reproduction-number trajectory is no longer overwritten when forecasting. Update tests for the GP()/RW() interface (random walk via RW(), drop unsupported periodic-kernel and deprecated breakpoint cases). Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
Configure Gaussian processes and random walks through rt_opts(prior = ...) and backcalc_opts(prior = ...), express a fixed reproduction number as a constant prior, and drop references to the deprecated gp_opts()/gp argument and rw argument. Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
plot() on a state specification draws sample trajectories from the prior so the implied time-varying behaviour can be visualised before fitting. Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
RW(period = n) holds the value constant for n time steps between random walk steps, so the value changes every n steps (e.g. period = 7 for a weekly random walk). This restores the step interval of the deprecated rt_opts(rw = n) within the time-varying state interface. Vignettes use a weekly random walk where they previously set rw = 7. Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
Wrap reproduction-number priors that relied on the implicit default Gaussian process in GP() (a bare distribution prior is now a constant Rt), use weekly random walks via RW(period = 7) where rw = 7 was used, and update prose for the GP(init)/GP(mean) and RW interfaces. Re-rendered all affected vignettes. Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
The Gaussian process variant is now chosen through the prior: GP(mean = ...) for a mean-reverting (stationary) process or GP(init = ...) for one on first differences. Removes the now-unused stationary flag from the Rt stan data. Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
The initial value of each time-varying state (used for the init-anchor prior) now reuses the trajectory already computed for R / infections / the observation parameters instead of recomputing it, removing a redundant Gaussian process evaluation from the autodiff path. Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
…rk (#1482) Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
…-params # Conflicts: # R/create.R # inst/stan/estimate_infections.stan # inst/stan/functions/params.stan # vignettes/EpiNow2.Rmd.orig
|
This is how benchmark results would change (along with a 95% confidence interval in relative change) if 4bc10d9 is merged into main:
|
…-params # Conflicts: # NAMESPACE # inst/stan/estimate_truncation.stan
Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
|
This is how benchmark results would change (along with a 95% confidence interval in relative change) if 9416634 is merged into main:
|
…-params # Conflicts: # NAMESPACE # R/dist_spec.R # R/utilities.R # _pkgdown.yml
Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
There was a problem hiding this comment.
Automated review (fresh-context reviewers, four slices: Stan state machinery, Stan model integration, R interface, R periphery + tests). The two Stan slices verified dimensions, indexing, Jacobians and the data contract with no findings. Three findings on the R side follow. Line-anchored comments are not possible here — the diff exceeds GitHub's size limit and is truncated — so findings are listed with file:line.
1. R/create.R:419 (medium) — fully-fixed configuration crashes at initialisation.
When every registered parameter is fixed (e.g. estimate_infections(data, rt = rt_opts(prior = Fixed(1)), obs = obs_opts(family = "poisson"))), n_params_variable == 0, so out$params <- array(rtruncnorm(0L, a = numeric(0), ...)) aborts with length(a) > 0 is not TRUE. The sibling state vectors use the rtruncnorm0() zero-guard; out$params did not. Before the GP hyperparameters moved into per-state settings there were always ≥2 variable params, so this path was unreachable — a regression. Fixed by routing out$params through rtruncnorm0().
2. R/create.R:1065 (low) — GP lengthscale max truncation dropped, needs author intent.
gp_rho_params <- c(gp_rho_params, numeric_params(gp$ls, "lengthscale", name)) emits only the two distribution parameters; the upper bound of the default GP(ls = LogNormal(mean = 21, sd = 7, max = 60)) is not passed to Stan, so the lengthscale prior is effectively untruncated. Whether the max is meant to be a hard Stan truncation or only an R-side default hint is a design decision — left for @sbfnk rather than changed.
3. R/utilities.R:493 (low) — make_init_priors() is dead code.
This PR removed both call sites (estimate_infections, simulate_infections) and the init-prior Stan data blocks its roxygen references, leaving the function orphaned. Fixed by removing it and its generated man page.
|
Addressed the review:
|
|
This is how benchmark results would change (along with a 95% confidence interval in relative change) if 760caa8 is merged into main:
|
Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
|
Finding 2 ( On closer inspection this was a genuine regression, not a design question: on The fix carries each hyperparameter dist's |
|
This is how benchmark results would change (along with a 95% confidence interval in relative change) if 3300f16 is merged into main:
|
Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
|
This is how benchmark results would change (along with a 95% confidence interval in relative change) if 7e47dfd is merged into main:
|
Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
|
This is how benchmark results would change (along with a 95% confidence interval in relative change) if 08f278f is merged into main:
|
Description
This PR makes first steps towards #600; it does not close it. It introduces a general interface for time-varying model parameters and uses it to express the two existing
estimate_infections()models through a single, shared latent-state implementation. The interface is applied to the reproduction number and to the back-calculation infections, but is not yet extended to all parameters.Time-varying parameter interface
GP()andRW()constructors declare a parameter as a Gaussian process or random walk, in either a mean-reverting (mean =) or first-difference (init =) form. The anchor accepts a prior (a<dist_spec>) or a known trajectory supplied as a numeric vector.param_specsuperclass underliesdist_specand the newstate_spec, so a parameter's prior can be a constant or a time-varying state.Reproduction number
rt_opts(prior = ...)now accepts a constant value or aGP()/RW()state. The reproduction number is built through the generic state machinery, replacing the bespoke Rt Gaussian-process code path (update_Rt, the separateR0/init-prior handling). The state parameter is namedR.Back-calculation (non-mechanistic) model
backcalc_opts(prior = GP(...)), mirroringrt_opts(prior = ...).backcalcprior modes proved difficult to express within the new state interface, so they have been deliberately retired rather than carried over. The back-calculation results therefore differ from before, and this is a deliberate behaviour change to keep both models on a single shared implementation.Gaussian process configuration
ls,alpha,kernel,basis_prop,boundary_scale,matern_order,w0) are now arguments ofGP(), so a process is fully specified in one place.Deprecations
gp_opts(); supply settings toGP()instead.gpargument ofestimate_infections(),epinow(), andregional_epinow(); configure the process through the model prior.rwargument and breakpoints (use_breakpoints/ thebreakpointdata column) ofrt_opts(); usert_opts(prior = RW(...)).prior(now aGP()specification) andprior_windowarguments ofbackcalc_opts().Initial submission checklist
devtools::test()anddevtools::check()).devtools::document()).lintr::lint_package()).