TST: Mark check for f2py script xfail. #12033

EricSchles · 2018-09-25T16:46:57Z

Context:

When running the tests via python [ROOT DIR]/runtests.py on windows linux subsystem a test fails with respect to f2py, f2py3, f2py3.6.

This is despite the fact that f2py exists in the my path, however I don't have f2py3 or f2py3.6 which shouldn't be required for the tests to pass.

closes gh-10016

…rror out if multiple do not

charris · 2018-09-25T16:58:39Z

numpy/tests/test_scripts.py

+        if all_success == 3:
+            return
+        elif f2py_success == 1:
+            msg = "f2py fails"


This doesn't look right.

And the warning will raise an error in the development branch.

There is something special about the windows linux subsystem as this test passes on appveyor. I don't know how to check for the linux subsystem, so it would be better to simply check for f2py, or maybe for the presence of at least one of the options, which is what the test used to do.

@charris it fails on multiple other systems, see gh-10016

I think this test does need to be simplified. The logic here is that if all 3 commands are not found, it checks just for f2py and if that is found then the test does not fail. I think that's the main thing to agree on first. The rest can be tweaked as needed.

This test always fails for me on both WSL and Mac; mac also always fails test_sdot_bug_8577 in my hands. My development workflow is often to establish the baseline failure count expected for the full suite before running tests on a new feature branch.

Yep, I think just checking for f2py is the way to go. That used to fail for other systems also, but maybe it is better now that we are using entry points. @tylerjereddy Does that simplified test work for you?

On my mac with this diff on master:

--- a/numpy/tests/test_scripts.py +++ b/numpy/tests/test_scripts.py @@ -92,7 +92,7 @@ def test_f2py(): version = sys.version_info major = str(version.major) minor = str(version.minor) - f2py_cmds = ('f2py', 'f2py' + major, 'f2py' + major + '.' + minor) + f2py_cmds = ('f2py',) success = try_f2py_commands(f2py_cmds) - msg = "Warning: not all of %s, %s, and %s are found in path" % f2py_cmds - assert_(success == 3, msg) + msg = "Warning: %s not found in path" % f2py_cmds + assert_(success == 1, msg)

From python runtests.py -t "numpy/tests/test_scripts.py" -- -s I get a failure and f2py itself gives:

Traceback (most recent call last): File "/Users/treddy/miniconda3/envs/numpy_dev_py36/bin/f2py", line 11, in <module> load_entry_point('numpy==1.16.0.dev0+025bbdd', 'console_scripts', 'f2py')() File "/Users/treddy/miniconda3/envs/mda_dev_py3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/__init__.py", line 565, in load_entry_point File "/Users/treddy/miniconda3/envs/mda_dev_py3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/__init__.py", line 2598, in load_entry_point File "/Users/treddy/miniconda3/envs/mda_dev_py3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/__init__.py", line 2258, in load File "/Users/treddy/miniconda3/envs/mda_dev_py3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/__init__.py", line 2264, in resolve File "/Users/treddy/.local/lib/python3.6/site-packages/numpy-1.16.0.dev0+5f2a5ae-py3.6-macosx-10.9-x86_64.egg/numpy/f2py/__main__.py", line 4, in <module> from f2py2e import main ModuleNotFoundError: No module named 'f2py2e'

The modified test and f2py itself can be made to work if I adjust my path on mac os to check the build path first, before site-packages, with:
export PATH=/Users/treddy/github_projects/numpy_linked_worktree/build/testenv/bin:$PATH
I did something similar to get shippable builds all green.

Once the path search order was fixed, I went back and reversed the change to the unit test noted above and the original test passed on my mac, so...

I'm pretty sure @pv previously commented somewhere about the difficulty of getting the f2py path stuff working just right in a portable fashion.

EricSchles · 2018-09-25T17:54:28Z

This makes sense. I’ll refactor. Is there any case that we care about all three?

…

Sent from my iPhone

On Sep 25, 2018, at 1:38 PM, Charles Harris ***@***.***> wrote: @charris commented on this pull request. In numpy/tests/test_scripts.py: > f2py_cmds = ('f2py', 'f2py' + major, 'f2py' + major + '.' + minor) - success = try_f2py_commands(f2py_cmds) - msg = "Warning: not all of %s, %s, and %s are found in path" % f2py_cmds - assert_(success == 3, msg) + all_success = try_f2py_commands(f2py_cmds) + if all_success == 3: + return + elif f2py_success == 1: + msg = "f2py fails" Yep, I think just checking for f2py is the way to go. That used to fail for other systems also, but maybe it is better now that we are using entry points. @tylerjereddy Does that simplified test work for you? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

charris · 2018-09-25T18:12:58Z

Is there any case that we care about all three?

They are all supposed to show up somewhere, but only non-windows systems. Out of curiosity, what does sys.platform return on WSL? I wonder what is happening on the Mac? In setup.py there is

    # The f2py scripts that will be installed
    if sys.platform == 'win32':
        f2py_cmds = [
            'f2py = numpy.f2py.f2py2e:main',
            ]
    else:
        f2py_cmds = [
            'f2py = numpy.f2py.f2py2e:main',
            'f2py%s = numpy.f2py.f2py2e:main' % sys.version_info[:1],
            'f2py%s.%s = numpy.f2py.f2py2e:main' % sys.version_info[:2],
            ]

In any case, we are leaving the placement of those commands up to distutils. @tylerjereddy Does the mac detect any of those? What does which f2py report?

charris · 2018-09-25T18:22:53Z

Let's just mark the existing test as xfail. That will provide some interesting information while not causing annoying test failures. We really cannot do much more, and if someone cannot run f2py we can deal with that when they complain.

tylerjereddy · 2018-09-25T18:30:51Z

@charris Yeah, seems tricky with the PATH setting apparently platform-specific; for the Ubuntu thing in shippable I had to do this to set it dynamically to find f2py:

    - extra_directories=($SHIPPABLE_REPO_DIR/build/*scripts*)
    - extra_path=$(printf "%s:" "${extra_directories[@]}")
    - export PATH="${extra_path}${PATH}"

EricSchles · 2018-09-25T18:32:07Z

So on WSL sys.platform returns ‘linux’. Should we close out the issue then and not pull into master?

…

Sent from my iPhone

On Sep 25, 2018, at 2:23 PM, Charles Harris ***@***.***> wrote: Let's just mark the existing test as xfail. That will provide some interesting information while not causing annoying test failures. We really cannot do much more, and if someone cannot run f2py we can deal with that when they complain. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

charris · 2018-09-25T18:41:36Z

@EricSchles Let's leave the test alone, but add an @pytest.mark.xfail(reason="Test is unreliable") decorator.

EricSchles · 2018-09-25T19:07:02Z

Sounds good! Out of meeting in 30

…

Sent from my iPhone

On Sep 25, 2018, at 2:41 PM, Charles Harris ***@***.***> wrote: @EricSchles Let's leave the test alone, but add an @pytest.mark.xfail(reason="Test is unreliable") decorator. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

EricSchles · 2018-09-25T19:19:52Z

Alright, changes removed @charris should I rebase my changes so only the one code change is reflected? Or is it fine to leave as is?

charris · 2018-09-25T20:10:23Z

A rebase would be nice.

charris · 2018-09-25T21:15:41Z

Well, a rebase with squashing the commits :) But good enough. Thanks @EricSchles .

rgommers · 2018-09-25T21:37:00Z

👍 first commit, keep them coming @EricSchles!

EricSchles · 2018-09-25T21:38:58Z

Woooo. Sorry been on the train home. Was planning on squashing this.

…

Sent from my iPhone

On Sep 25, 2018, at 5:37 PM, Ralf Gommers ***@***.***> wrote: 👍 first commit, keep them coming @EricSchles! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

charris · 2018-09-25T21:51:18Z

@EricSchles No problem.

EricSchles · 2018-09-25T21:55:56Z

Thanks! I’m excited to collaborate on many more PRs. With commits squashed commits next time!

…

Sent from my iPhone

On Sep 25, 2018, at 5:51 PM, Charles Harris ***@***.***> wrote: @EricSchles No problem. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

updating f2py to ensure that at least f2py base exists, and doesn't e…

583398d

…rror out if multiple do not

rgommers added 05 - Testing 03 - Maintenance component: numpy.f2py labels Sep 25, 2018

charris requested changes Sep 25, 2018

View reviewed changes

Eric Schles added 2 commits September 25, 2018 15:18

updating test file to ignore the failure

1dde1a5

removing warnings

b13c52c

charris approved these changes Sep 25, 2018

View reviewed changes

charris merged commit 66dcb03 into numpy:master Sep 25, 2018

charris changed the title ~~TST: updating f2py to ensure that at least f2py base exists, and doesn't e…~~ TST: Mark check for f2py script xfail. Sep 25, 2018

Uh oh!

TST: Mark check for f2py script xfail. #12033

TST: Mark check for f2py script xfail. #12033

Uh oh!

Conversation

EricSchles commented Sep 25, 2018

Uh oh!

charris Sep 25, 2018

Choose a reason for hiding this comment

Uh oh!

charris Sep 25, 2018

Choose a reason for hiding this comment

Uh oh!

charris Sep 25, 2018

Choose a reason for hiding this comment

Uh oh!

rgommers Sep 25, 2018

Choose a reason for hiding this comment

Uh oh!

tylerjereddy Sep 25, 2018

Choose a reason for hiding this comment

Uh oh!

charris Sep 25, 2018

Choose a reason for hiding this comment

Uh oh!

tylerjereddy Sep 25, 2018

Choose a reason for hiding this comment

Uh oh!

EricSchles commented Sep 25, 2018 via email

Uh oh!

charris commented Sep 25, 2018

Uh oh!

charris commented Sep 25, 2018

Uh oh!

tylerjereddy commented Sep 25, 2018

Uh oh!

EricSchles commented Sep 25, 2018 via email

Uh oh!

charris commented Sep 25, 2018

Uh oh!

EricSchles commented Sep 25, 2018 via email

Uh oh!

EricSchles commented Sep 25, 2018

Uh oh!

charris commented Sep 25, 2018

Uh oh!

charris commented Sep 25, 2018

Uh oh!

rgommers commented Sep 25, 2018

Uh oh!

EricSchles commented Sep 25, 2018 via email

Uh oh!

charris commented Sep 25, 2018

Uh oh!

EricSchles commented Sep 25, 2018 via email

Uh oh!

Uh oh!