TST: test_eigs_consistency() now sorts output #9456
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* test_eigs_consistency() was prone to failure because of the production of mathematically equivalent results in unsorted order; the order is now enforced to alleviate these stochastic unit test failures
|
Now, whether it is a problem for end-users or other libraries that a result can be mathematically equivalent but produced in a different sort order in an array is not something that's immediately clear to me here. I suppose one could make the argument that the sort should happen in the code proper instead of the test -- maybe thoughts on that would be useful if it is something I should worry about. |
|
Hmm, it seems the seed I added fixed it for me, but now makes it consistently fail on TravisCI on all build matrix entries (at least in the last completed CI run). This fix looks good. Let's merge as soon as things turn green. Thanks @tylerjereddy |
|
All green, merged. Thanks Tyler. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #9453
@lobpcg confirmed the mathematical equivalence of the differently sorted values (and complex or not), but the assertion does not pass in the stochastic cases where the sorting doesn't match between actual and expected values.
So, sorting is now enforced even though the mathematically correct result was already produced, to alleviate the unit test failure.
To confirm locally, due to the difficulty with reproduction, I initially looped the test 500 times, and this made it much easier to detect and confirm a likely fix.