Skip to content

Preserve subnormal inverse-beta roots in Fisher F quantiles - #1667

Open
hector2394 wants to merge 1 commit into
boostorg:developfrom
hector2394:fix/1666-fisher-f-upper-tail
Open

hector2394 wants to merge 1 commit into
boostorg:developfrom
hector2394:fix/1666-fisher-f-upper-tail

Conversation

@hector2394

Copy link
Copy Markdown

Fixes #1666.

For two beta shape parameters below one, the initial power-series estimate can give a subnormal root that the iteration excludes from its search range. That caused a finite Fisher F upper-tail quantile to throw and a quantile beyond the range of double to return a finite value.

Return the estimate when the series correction is below type precision. The tests cover the finite result and overflow with promotion disabled.

On Windows with MSVC, both regressions fail on develop and pass with the patch. The Fisher F suite and the double-precision inverse-beta suite pass.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fisher F complemented quantile fails for two upper-tail probabilities

1 participant