Skip to content

Fit cumulative multilevel submodels with one group-level intercept term by projpred's own Laplace fitter - #645

Open
avehtari wants to merge 43 commits into
masterfrom
mlvl-cumul
Open

avehtari wants to merge 43 commits into
masterfrom
mlvl-cumul

Conversation

@avehtari

Copy link
Copy Markdown
Member

Summary

This PR continues #644 and is based on its branch. If the only group-level term of a brms::cumulative() submodel is (1 | <group>), projpred now fits it with its own Laplace fitter instead of ordinal::clmm(). The fitter minimizes the Laplace objective of clmm() by nlminb() with the exact gradient. The objective and the gradient are in C++ (src/mlvlcumul.cpp). In the benchmark model with only (1 | g), varsel() is 6.7 times faster than on master. cv_varsel() with LOO validated search is 6.1 times faster. With a group-level slope term in the reference model, the speedups are 2.8 (varsel()) and 3.0 (cv_varsel()). The predictor rankings are the same as on master.

Speed

Median of 3 runs, in seconds (CSC Roihu, 1 CPU per run, ordinal 2026.7-26; both versions in one job).

model call master this PR speedup
cumulative (logit), (1 | g) varsel() 292.2 43.7 6.7x
cumulative (logit), (1 | g) cv_varsel(), LOO validated search 1782.1 294.2 6.1x
cumulative (logit), (1 + x1 | g) varsel() 433.8 155.7 2.8x
cumulative (logit), (1 + x1 | g) cv_varsel(), LOO validated search 2488.1 826.4 3.0x
  • Reference models: brms::brm() fits with 300 observations, 4 response categories and 6 predictors. varsel() used nterms_max = 6; cv_varsel() used validate_search = TRUE, nloo = 10 and nterms_max = 4.
  • The rankings were identical in all 12 comparisons. The largest mean relative difference of the submodel lppd and mu values was 1.7e-5 (rows 1 and 2) and 1.4e-5 (rows 3 and 4).
  • Slope model: 2,820 of 2,900 submodel fits in varsel() and 21,120 of 21,560 in cv_varsel() have no slope term. At most one of them used the fallback to clmm(); the others used the new fitter.
  • Rows 1 and 2: commit 5c2dfd3 against master a45cde1. Rows 3 and 4: commit 886b7e2 against master 6fef1de. The master commits after a45cde1 do not change the forward search, the fitter of this PR or the clmm() fits.

The fitter

  • Model, in the parameterization of clmm(): P(y <= j) = F(alpha_j - x'beta - b_g), with group-level intercepts b_g with standard deviation tau. With one grouping variable, the Newton iteration for the conditional modes works per group. The gradient includes the derivative of the conditional modes.
  • Links: logit, probit (also probit_approx), cloglog and cauchit, the links of brms::cumulative(). The C++ code also has loglog.
  • The fitter uses the augmented rows, with the category probabilities of the projected draw as weights. It ignores rows with weight 0.
  • Start values: thresholds from the weighted category proportions, coefficients of 0, tau of 1. tau = 0 is always a stationary point. If tau ends below 1e-3, a second run starts tau at 0.3, and projpred keeps the fit with the lower objective.
  • The fit is an object of class c("projpred_mlvl_cumul", "projpred_mlvl") without the data.
  • projpred still calls clmm() in these cases:
  • If the fitter or the conditional modes do not converge, projpred calls clmm() and converts the result to class projpred_mlvl_cumul.
  • For new group levels, the predictions use the same random numbers as the clmm() code path. Thus the same seed gives the same draws as on master. get_subparams() keeps the parameter names and order of the clmm() code path, so the column names of as.matrix.projection() do not change. as_glmerMod() gives an error for these fits.

Changes in results

  • The fits in outdmin have class projpred_mlvl_cumul instead of clmm.
  • For logit, probit and cauchit, the fitter and clmm() minimize the same function. In the unit tests, the objective of the fitter is equal to or lower than that at the clmm() estimates, and the parameters agree within 1e-3. A prototype of the fitter and clmm() fitted the 2,820 submodels of one varsel() call on the model of rows 1 and 2. The prototype objective was at most 2.5e-8 higher and at most 2.9e-5 lower than the clmm() objective. The time per fit was 100.5 ms for clmm() and 11.3 ms for the prototype.
  • cloglog: clmm() without useMatrix = TRUE fits the loglog model for a single (1 | <group>) term (clmm(): the cloglog and loglog links are swapped in the C link functions runehaubo/ordinal#76). Since Fix clmm() submodel fits with the cloglog and loglog links #613, projpred calls clmm() with useMatrix = TRUE for this link. The new fitter also fits the cloglog model.
  • The log-likelihood that clmm() reports can differ from the Laplace value at its estimates (up to 2.8e-3 in the 2,820 fits). The fitter reports the Laplace value at the conditional modes.
  • If a cumulative probability is 1 in double precision, the clmm() code path gives a linear predictor of Inf. The new fitter gives a finite value.

Tests

  • New test file test_mlvl_laplace_cumul.R (13 test blocks; no reference model fit, so it also runs on CRAN). It tests:
    • for the five links: the gradient against finite differences, a finite objective or Inf at extreme parameters, and the fitter against clmm();
    • the predictions for new group levels against the clmm() code path (all links except loglog);
    • factor predictors, a numeric grouping variable and submodels without predictors;
    • a fit without group-level variation against MASS::polr();
    • observation weights;
    • the fallback cases and the conversion after non-convergence;
    • divmin_augdat() and subprd_augdat() with mixed fit classes;
    • the methods of the new class;
    • cv_varsel() with LOO validated search for a cumulative reference model, where the weight perturbation (Smoother LOO curves with improved SE #603) changes the weights. The rankings and the elpd values equal those of the clmm() code path.
  • With the code of Fit binomial and Poisson multilevel submodels with group-level intercepts by projpred's own Laplace fitter #644 (ordinal 2025.12.29), 12 of the 13 blocks fail. The other block tests the fallback cases and passes.
  • outdmin_tester_aug() expects class projpred_mlvl_cumul. The existing clmm() tests turn the fitter off.
  • Not tested: the parallel branch of divmin_augdat().
  • Full test suite with NOT_CRAN=true on 3968e72 (with master 7b2ac74 merged), ordinal 2026.7-26, 0 failures:
    • lme4 2.1-1, R 4.6.1: 60,072 expectations, 0 skipped;
    • lme4 2.0.6, R 4.5.3: 60,422 expectations, 6 skipped (they need lme4 >= 2.1-1).
  • With ordinal 2025.12.29 (lme4 2.1-0, R 4.6.2), test_mlvl_laplace_cumul.R also passes on 3968e72.

This PR was assisted by Claude.

…cap, fallback deviance, tests against glmer()
# Conflicts:
#	NEWS.md
#	R/RcppExports.R
#	man/projpred-package.Rd
#	src/RcppExports.cpp
# Conflicts:
#	NEWS.md
#	R/RcppExports.R
#	man/projpred-package.Rd
#	src/RcppExports.cpp
# Conflicts:
#	NEWS.md
#	R/RcppExports.R
#	src/RcppExports.cpp
@avehtari avehtari added the performance Performance issues (i.e., high runtime or high memory usage). label Nov 15, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance Performance issues (i.e., high runtime or high memory usage).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants