MAINT: optimize.root_scalar: raise when NaN is encountered

[mdhaber]

Reference issue

Closes gh-3089
Closes gh-8394
Closes gh-14414

What does this implement/fix?

The bracketing scalar root-finders can return incorrect results when the function returns NaN. Instead, raise an error.

Additional information

The linked issues only report problems when the function is NaN at one of the initial bracketing interval endpoints, so the approach of the first commit was to only check for NaNs at the bracketing interval endpoints. However, I imagine there could also be problems if the solver does not encounter a NaN at the initial endpoints but later. One option is to also check whether the final function value is NaN, but this is not guaranteed to catch problems, and it could allow the algorithm to proceed long after the first NaN is encountered. I ended up changing the approach to check for NaN on every function evaluation using a wrapper. The performance hit seems small. In main:

import numpy as np
from scipy import stats
rng = np.random.default_rng(1638083107694713882823079058616272161)
p = rng.random(1000)
mu = 1.5
x = stats.recipinvgauss.ppf(p, mu)  # uses `brentq`
# 1.01 s ± 8.14 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

with this PR:

1.03 s ± 3.27 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

This could probably be reduced by doing the NaN check in the compiled code itself as is discussed in #8394 (comment), but since gh-3089 was reported almost a decade ago, it seems there is some hesitation to modify all those compiled functions.

[tupui]

Thanks Matt. LGTM and the performance impact seems minimal. Let see if others have more comments.

[dschmitz89]

If this change was implemented in the root finder itself, it would work for the Python interface optimize.root_scalar
and for the Cython interface in cython_optimize.

Would be a lot more work though ..

[mdhaber]

Yes, implementing it in the compiled code is something I considered, as addressed in the top post. But I decided on a less invasive solution that would be easy to implement and review.

This does fix root_scalar.
Would cython_optimize users would want to burn clock cycles on such a check?

I don't think we should continue to hold out for a compiled solution to fix this. Of course, a future compiled enhancement would be welcome, assuming there is also someone willing to review it. In the meantime, this closes three issues.

@dschmitz89 what would you suggest? Would you prefer to close this PR?

[dschmitz89]

I have never used cython_optimize myself, so I cannot really speak for those folks.

But if I were a user wanting to switch from root_scalar to cython_optimize for speed, I would expect the exact same behaviour, also for these edge cases. Also updating this in the cython wrapper code sounds like bad practice but ..

I guess there are some more opinions floating around here.. maybe the author of the root finders would like to have a word? CC @charris

[tupui]

I am merging now, additional developments in compiled code can be done later if wanted, as it is, this is a clear improvement and there is nothing preventing from merging. Thanks again Matt.