Skip to content

Add arguments cvfun and cvrefbuilder to get_refmodel.stanreg() - #604

Open
avehtari wants to merge 2 commits into
masterfrom
stanreg-cvrefbuilder
Open

avehtari wants to merge 2 commits into
masterfrom
stanreg-cvrefbuilder

Conversation

@avehtari

@avehtari avehtari commented Nov 6, 2026

Copy link
Copy Markdown
Member

This PR was assisted by Claude.

Summary

In the review of #588, @aloctavodia suggested that get_refmodel.stanreg() and brms::get_refmodel.brmsfit() could eventually accept a cvrefbuilder argument. This PR adds the arguments cvfun = NULL and cvrefbuilder = NULL to get_refmodel.stanreg(). A function in one of these arguments replaces the internal default. A value that is not a function and not NULL gives an error.

Before this PR, get_refmodel.stanreg() built its own cvfun and cvrefbuilder and passed them to init_refmodel() together with .... A cvfun or cvrefbuilder in the call therefore stopped with an error, for example formal argument "cvrefbuilder" matched by multiple actual arguments. #588 documented a workaround for the custom latent-projection functions: replace element cvrefbuilder of the reference model object after get_refmodel(). For rstanarm reference models, argument cvrefbuilder now replaces this workaround.

The default cvrefbuilder passes ... to the fold-wise get_refmodel() calls. The new arguments are not in ..., so the fold-wise reference model objects get the defaults. The fold-wise objects do not use cvfun or cvrefbuilder.

Documentation

  • ?init_refmodel: the entries of cvfun and cvrefbuilder describe the defaults of get_refmodel.stanreg().
  • ?extend_family and the latent-projection vignette (section "K-fold CV with custom response-scale functions") now tell rstanarm users to pass the custom cvrefbuilder as an argument. brms::get_refmodel.brmsfit() has a cvfun argument but no cvrefbuilder argument yet, so for brms the text keeps the workaround. A brms PR that adds cvrefbuilder follows. After a brms release with it, a small projpred PR can update the brms text.
  • NEWS item in "Minor changes".

Tests

test_refmodel.R has 3 new test blocks:

  • With both arguments, the reference model object stores the given functions. run_cvfun() calls the given cvfun once with the given folds. cv_varsel() with cv_method = "kfold" calls the given cvrefbuilder once for each fold, in the order of the folds. The given cvfun returns the full-data fit for each fold, so the test needs no refit.
  • With only cvfun, the reference model object uses the default cvrefbuilder. The fold-wise reference model object does not get the given cvfun, and it takes sigma from the fold-wise fit. With only cvrefbuilder, the reference model object uses the default cvfun.
  • A cvfun or cvrefbuilder that is not a function gives the error "Argument cvfun must be a function or NULL." (or the same error for cvrefbuilder).

Without the change in R/refmodel.R, 2 of the new blocks stop with the "matched by multiple actual arguments" error, and the third has 2 failures.

The full test suite on master e1dd3bc with this PR (NOT_CRAN=true) gave 57 269 expectations, 0 failures, 0 errors and 5 skips. Versions: R 4.5.3, testthat 3.3.2, rstanarm 2.32.2, brms 2.23.1, lme4 2.0.6. The 19 warnings come from CHOLMOD in test_div_minimizer.R and from the generalized Pareto fit and Pareto k in the brms cvfits test of test_varsel.R. This PR does not change these two test files.

@avehtari avehtari added the feature New feature label Nov 6, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature New feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant