Skip to content

Latest commit

 

History

History
243 lines (175 loc) · 12.6 KB

mixture.md

File metadata and controls

243 lines (175 loc) · 12.6 KB

Finite Mixture Models

Previous: Univariate Distributions | Technical Reference | Next: Competing Risks

MixtureModel represents a population in which each observation is generated by one unobserved component. This differs from competing risks, where multiple processes can occur in the same block and the recorded value is their minimum or maximum.

Mathematical Formulation

Let $K\in{1,2,3}$, let $w_k\ge 0$ be physical component weights, and let $F_k$ and $f_k$ be component CDFs and densities. The ordinary mixture is

$$ \sum_{k=1}^{K}w_k=1,\qquad f_M(x)=\sum_{k=1}^{K}w_k f_k(x),\qquad F_M(x)=\sum_{k=1}^{K}w_k F_k(x). \tag{1} $$

For exact independent observations,

$$ \ell_D=\sum_{i=1}^{n}\log\left[\sum_{k=1}^{K}w_k f_k(x_i\mid\boldsymbol\theta_k)\right]. \tag{2} $$

Positive-hurdle option

When IsZeroInflated is enabled, the implementation is a point mass at exactly zero plus continuous components conditioned on $X>0$. Write

$$ \pi_0=P(X=0),\qquad s_k=P_k(X>0)=1-F_k(0),\qquad \sum_{k=1}^{K}w_k=1-\pi_0. \tag{3} $$

Every component must have finite $s_k>0$. The mixed distribution is

$$ F_M(x)= \begin{cases} 0, & x<0,\\ \pi_0, & x=0,\\ \pi_0+\displaystyle\sum_{k=1}^{K}w_k \dfrac{F_k(x)-F_k(0)}{s_k}, & x>0. \end{cases} \tag{4} $$

$$ f_M(x)=\sum_{k=1}^{K}w_k\dfrac{f_k(x)}{s_k},\qquad x>0. \tag{5} $$

Numerics.Distributions.Mixture.PDF(0) returns $\pi_0$ under the documented Lebesgue-plus-Dirac reference measure and returns zero for $x&lt;0$. Quantiles and simulation use the same positive-conditioned law.

BestFit fixes the atom to

$$ \widehat{\pi}_0= \dfrac{#{\text{exact annual records equal to zero}}} {#{\text{all exact annual records}}}. \tag{6} $$

Uncertain, interval, and threshold records never determine the atom. Negative exact values are invalid.

Observation Likelihoods

Low-outlier, interval, and threshold contributions use the corrected mixed CDF and survival function. For measurement-error density $g_i$, retained bounds $[a_i,b_i]$, and retained mass $c_i$,

$$ L_i= \dfrac{ \pi_0 g_i(0),\mathbb I(a_i\le 0\le b_i) + \displaystyle\int_{\max(a_i,0)}^{b_i} g_i(x)\sum_{k=1}^{K}w_k\dfrac{f_k(x)}{s_k},dx }{c_i}. \tag{7} $$

BestFit retains its normalized 20-point Gauss-Legendre window. The atom is added separately and is not approximated as Lebesgue density.

Public, Sampled, and Physical Parameter Vectors

Numerics preserves its public contract: constructors, Weights, GetParameters, NumberOfParameters, every SetParameters overload, constraints, and MLE results use all $K$ physical weights. Caller-owned arrays are copied before assignment or normalization, including the legacy ref overload.

BestFit preserves its public model, prior-configuration, EM, and project-model vector with all $K$ physical weight coordinates:

$$ \boldsymbol\phi= (w_1,\ldots,w_K, \boldsymbol\theta_1^{\mathsf T},\ldots, \boldsymbol\theta_K^{\mathsf T})^{\mathsf T}. \tag{8a} $$

The Bayesian sampler is identified by omitting the last weight:

$$ \boldsymbol\eta= (w_1,\ldots,w_{K-1}, \boldsymbol\theta_1^{\mathsf T},\ldots, \boldsymbol\theta_K^{\mathsf T})^{\mathsf T}, \qquad w_K=m-\sum_{k=1}^{K-1}w_k. \tag{8} $$

$$ m= \begin{cases} 1,&\text{ordinary},\\ 1-\pi_0,&\text{positive hurdle}. \end{cases} \tag{9} $$

Every sampler-target evaluation performs this affine expansion inside BestFit before invoking the established full-$K$ likelihood and prior methods. Sampled weights must be finite and nonnegative. A residual weight below $-10^{-12}\max(1,m)$ is rejected; a negative residual within that roundoff tolerance is set to zero in the expanded vector. Other infeasible proposals have log target $-\infty$. This narrow boundary repair does not renormalize the sampled coordinates or mutate the caller's proposal. Numerics receives only the expanded full-$K$ physical vector and retains its existing copy-and-normalize boundary.

All $K$ configured weight-prior factors remain active even though only $K-1$ weights are sampled:

$$ \log p(\boldsymbol\eta) =\sum_{k=1}^{K}\log g_k(w_k) +\sum_j\log h_j(\theta_j) +\ell_{\mathrm{Jeffreys}} +\ell_{\mathrm{quantile}}. \tag{10} $$

The scalar factors $g_k$ form a joint prior kernel restricted to the simplex; they are not independent marginal distributions. The elimination map is affine, so no parameter-dependent Jacobian or additional normalization correction is introduced. PriorLogLikelihood and PointwisePriorLogLikelihood retain their full-$K$ public inputs, and the pointwise terms sum to the same scalar prior kernel.

New mixture MCMCResults store $\boldsymbol\eta$ directly. Sampler diagnostics therefore report $w_1,\ldots,w_{K-1}$ and the component parameters, but no separate trace, R-hat, ESS, or sampler covariance entry for $w_K$. Prior plots for sampled weights show the corresponding configured scalar factor, not a simplex marginal. Physical result tables, distribution reconstruction, posterior prediction, information criteria, and frequency curves derive $w_K$ locally for each retained draw without modifying MCMCResults. Legacy full-$K$ results follow the identity path and require no migration.

Component labels remain exchangeable under symmetric families and priors. Verification resolves labels by component location; applied analyses should use scientifically distinguishable components or interpret label-specific summaries cautiously 1.

Parameter Estimation

ExpectationMaximization(...) preserves the established approximate-MLE contract, iteration limit, tolerance, bounded Nelder-Mead component optimization, and convergence order. It does not include parameter, Jeffreys-scale, or quantile priors. On exact non-outlier data, BestFit delegates to Numerics, producing identical responsibilities, conditioning, simplex normalization, impossible-row behavior, iterations, and physical weights.

EM evaluates exact, censored, interval, positive-conditional and measurement-error contributions logarithmically through responsibility normalization. It centers observation log contributions before adding log weights, preserving relative weights even when a common log density has very large magnitude. Zero-weight components are skipped before singular-density and effective-support checks; active-component, atom, physical-weight and serialization rules remain unchanged.

For mixed records, BestFit adds the fixed atom to each row total and assigns component responsibilities only to continuous contributions:

$$ r_{ik}= \dfrac{\exp(a_{ik})} {\exp(a_{i0})+\sum_{j=1}^{K}\exp(a_{ij})}. \tag{12} $$

Component weights are rescaled to $m$. If any exact, uncertain, interval, or threshold row has zero or nonfinite total probability, EM throws InvalidOperationException with the row index and value.

BestFit's public EM method returns all $K$ weights followed by component parameters, preserving its original contract. Let $N_{\mathrm{eff}}$ be the sum of the final continuous-component responsibilities and let $\mathbf w=(w_1,\ldots,w_K)^{\mathsf T}$. The public weight-covariance block is

$$ \boldsymbol\Sigma_w =\frac{1}{N_{\mathrm{eff}}} \left[m,\operatorname{diag}(\mathbf w)-\mathbf w\mathbf w^{\mathsf T}\right]. \tag{12a} $$

It is symmetric, has negative off-diagonal entries, zero row sums, and rank at most $K-1$. The sampler receives the leading $(K-1)\times(K-1)$ block. The existing component covariance is retained and weight/component cross-covariances remain zero.

Priors and Posterior

BestFit adds scalar component priors to Equation (10). UseJeffreysRuleForScale adds $-\log s$ for each recognized scale. One enabled quantile prior contributes

$$ \log p_Q!\left[F_M^{-1}(1-\alpha_Q)\right]. \tag{13} $$

MixtureAnalysis uses EM as a deterministic basin finder. Its full-$K$ center and covariance are reduced to the identified coordinates by omitting $w_K$:

$$ \widehat{\boldsymbol\eta}_{EM} =(\widehat w_1,\ldots,\widehat w_{K-1}, \widehat{\boldsymbol\theta}^{\mathsf T})^{\mathsf T}. \tag{14} $$

The full initial population is sampled with the configured sampler seed from a multivariate Normal using covariance multiplier 1.5. A proposal receives at most 20 replacement draws, is accepted only with finite full-posterior fitness after the residual weight has been derived, and the best population members seed the chains. Failure of that EM population resets to randomized initialization. There is no mixture-specific MAP refinement or numerical Hessian. This changes no DEMCzs sampling default, seed, covariance multiplier, or retry limit and does not use a Dirichlet initializer.

Compile-Checked Configuration

private static MixtureAnalysis ConfigureLatentPopulationMixture(
    global::RMC.BestFit.Models.DataFrame annualPeaks)
{
    var model = new MixtureModel(
        annualPeaks,
        new List<UnivariateDistributionType>
        {
            UnivariateDistributionType.LogNormal,
            UnivariateDistributionType.GeneralizedExtremeValue
        });

    model.EnableQuantilePriors = false;
    return new MixtureAnalysis(model);
}

Assumptions and Limitations

  • Observations are conditionally independent given their latent component.
  • Weights are stationary; covariate-dependent mixing is not implemented.
  • BestFit supports one to three stationary components.
  • The hurdle option is an atom at exactly zero, not left censoring or a generic nonpositive category.
  • Negligible weights or parameters on bounds indicate weak identification.
  • Mixture information criteria remain nonregular near boundaries or with unresolved label switching.

The public model and project configuration remain full $K$, while new posterior chains deliberately persist only the identified $K-1$ sampled weights. Existing full-$K$ posterior results open and process directly. No load-time migration or result rewriting occurs.

The dependence backend and normalized 20-point measurement-error quadrature retain their existing scope. These calculations do not provide arbitrary-precision multivariate tails. Positive conditioning can lose a correction when separately returned component logs have already rounded to the same enormous magnitude, for example a Normal mean of -1e100 conditioned above zero.

Validation

Fast contracts cover the full-$K$ public boundary, $K-1$ sampled representation, derived-weight priors, residual roundoff, likelihood decomposition, covariance, diagnostics, persistence, and frequency-curve processing.

The mixture verification report separates the independent scikit-learn two-Normal mixture optimum from the retained six generation/recovery cells. A production-path comparison can protect a representation contract, but it is not an independent optimum oracle. The external comparison uses the same observations, orders components by mean, reconciles all physical weights and scales, and checks the common likelihood as well as the fitted result.

Bayesian recovery assesses identified component parameters and weights under the declared sample and prior settings, with central 95% posterior inclusion and explicit R-hat/ESS gates. Label switching and poorly separated components remain limitations: a smooth fitted mixture density can coexist with weakly identified component parameters. The evidence does not justify interpreting every fitted component as a distinct physical flood mechanism.

Implementation and Verification Traceability

Concern Implementation
Full-$K$ model, likelihood, prior, EM output, and project-model contract Models/UnivariateDistribution/MixtureModel.cs
Identified sampler initialization and physical reconstruction Analyses/Univariate/MixtureAnalysis.cs
Sampled-coordinate result storage and diagnostics Estimation/BayesianAnalysis.cs and Diagnostics/
Physical parameter-set display RMC.BestFit.App/GUI/Support/Controls/ParameterSetsControl.xaml.cs
Physical simplex and hurdle distribution sibling Numerics/Distributions/Univariate/Mixture.cs
Fast regressions RMC.BestFit.Tests/Univariate/MixtureLikelihoodContractTests.cs
Generation, recovery parity, and Bayesian recovery RMC.BestFit.Verification/Univariate/MixtureTests/MixtureRecoveryTests.cs

References

[1] G. McLachlan and D. Peel, Finite Mixture Models. New York, NY, USA: Wiley, 2000.

[2] A. P. Dempster, N. M. Laird, and D. B. Rubin, “Maximum likelihood from incomplete data via the EM algorithm,” Journal of the Royal Statistical Society: Series B, vol. 39, no. 1, pp. 1–38, 1977.


Previous: Univariate Distributions | Technical Reference | Next: Competing Risks