Use warnings instead of print statements

[StefanoFrazzetto]

Reference issue

Partially addresses: #1953

What does this implement/fix?

Use warnings.warn instead of print statements in scipy/optimize/optimize.py, specifically within def _minimize_bfgs and def _minimize_powell

Additional information

Related PR with partial fixes for another function: #14050

As discussed with @WarrenWeckesser, it might be worth having a look at changes in scipy/optimize/tests/test_optimize.py, to understand if those tests always generate a warning and, if that's the case, use something like with pytest.warns(OptimizeWarning):, instead of just filtering the warnings.

[StefanoFrazzetto]

I have tried the approach suggested by @mdhaber, i.e. creating a separate test with an expected failure, but passing a maxiter with a low value does not raise any warning. I run the tests a couple times, and I always get Failed: DID NOT WARN. No warnings of type (<class 'scipy.optimize.optimize.OptimizeWarning'>,) was emitted, using a maxiter = 3, or less.

[mdhaber]

Interesting. Can you tell how many iterations are occurring when you give it maxiter=3? It's possible that the function is not obeying the iteration limit, and that would be another issue!

[mdhaber]

@StefanoFrazzetto would you be interested in continuing this if I were to help fix merge conflicts?

[StefanoFrazzetto]

@mdhaber it's been a while! If it adds any value to the project, I'll try to find some time and help you getting this merged. By the way, while setting up my env, I took the chance to resolve the conflicts, so hopefully this won't take too much effort. Let me know how I can help.

[mdhaber]

Hmm oops I didn't notice that gh-1953 was already resolved by gh-15259. This may not be needed anymore.

The refactoring of the minimize_powell work is nice, but typically I wouldn't encourage refactoring unless it has some other added benefit. Perhaps we should have raised an OptimizeWarning from the start, but I'm not sure that we can now, since they are RuntimeWarnings, whereas OptimizeWarning is a subclass of UserWarning. If people are filtering them, their filters will break.

It looks like gh-15259 didn't change bfgs.

@AtsushiSakai can I get your opinion here, since you worked on this before? What should this PR do now that the main issue (gh-1953) is resolved?

[AtsushiSakai]

I realized now that #15259 fixed almost bare print statements for warnings, but the _minimize_bfgs ones are missed:

scipy/scipy/optimize/_optimize.py

Lines 1441 to 1445 in 2f36c58

	print("%s%s" % ("Warning: " if warnflag != 0 else "", msg))
	print(" Current function value: %f" % fval)
	print(" Iterations: %d" % k)
	print(" Function evaluations: %d" % sf.nfev)
	print(" Gradient evaluations: %d" % sf.ngev)

So, I think it would be great that this PR fixs these.
I reopened #1953.

[mdhaber]

@AtsushiSakai what warning class should be used? You used RuntimeWarning for the others, but OptimizeWarning would be more appropriate.

[AtsushiSakai]

but OptimizeWarning would be more appropriate.

Yeah. I didn't know the OptimizeWarning, I agree it is more appropriate than RuntimeWarning.

[mdhaber]

In that case, do you think we should replace the RuntimeWarnings with OptimizeWarnnings, then? It's slightly backward incompatible, but I think if an email were sent to the mailing list suggesting the change, it probably wouldn't bother people.
If it does, then this PR could keep using them, and afterwards we could go back and change the others.
What do you think @StefanoFrazzetto @AtsushiSakai? Send an email?

[mdhaber]

@AtsushiSakai I was hoping to get your opinion on #14069 (comment) when you have a moment.

[mdhaber]

Let's go ahead and leave the existing RuntimeWarning type alone so we can merge this. @AtsushiSakai does this look good to you?

You can test locally with:

import numpy as np
from scipy import optimize

def f(x):
    return np.sum(x**2)

method = 'newton-cg'  # PR changed bfgs, cg, newton-cg, and powell
maxiter = 1000  # if 10, warning will be raised

options={'disp': True, 'maxiter': maxiter}
optimize.minimize(lambda x: optimize.rosen(x), [0, 0], method=method,
                  jac=optimize.rosen_der, options=options)

@StefanoFrazzetto would you be willing to add unit tests that try to raise the warning and check that it occurs?

Note: CirrusCI failures unrelated.

[AtsushiSakai]

LGTM

[mdhaber]

OK, I fixed two little bugs and added tests. refguide_check failure was related before; now it isn't.
Since this was approved already, I'll go ahead and merge. Thanks, all!