MAINT: Fix lgtm alerts

[jhelie]

This PR fixes issues flagged by lgtm.com code queries. Most are minor but several do fix unintentional code behaviour.

The standard lgtm deep code analysis highlights dependencies, vulnerabilities and currently flag 185 alerts but you can investigate further by writing custom queries to explore your code base (see (https://lgtm.com/projects/g/numpy/numpy/ and https://lgtm.com/docs/ql/about-ql).

PR integration can also help automatically check for potential issues before they find their way into the code base - for instance several issues were introduced when copying in tempita code in 581bb79 and could have been addressed in that PR (https://lgtm.com/projects/g/numpy/numpy/rev/581bb79c170748cbf2f6970cd53595b3b5f1fe3e)

Hope that helps!

[jaimefrio]

The failures here are real: Python 3 no longer supports the raise expr, expr, expr syntax of Python 2. So even though that code is only run under Python 2.x, it still raises a SyntaxError when being parsed by Python 3.

My reading of PEP 3109 is that there is no way of having code compatible with both versions, and that the best we can do would be:

if PY3:
    raise exc_info[1](e).with_traceback(exc_info[2])
else:
    raise exc_info[1](e)

It would probably be a good idea to unpack exc_info into meaningfully-named variables, e.g. exc_type, exc_value, traceback = sys.exc_info() to make the code more understandable...

[jaimefrio]

Are we certain that decl will have an __eq__ method, and that this is not intentionally checking for "is the exact same object"? I'm not familiar at all with f2py, but I wouldn't be surprised if that was the case...

[jhelie]

As I understand it decl and decl2 have to be dictionaries (so == is fine) and the following if block will override decl values with that of decl2 where relevant so if == is true that block has no effect (but does no harm, just redundant)

The only case this is not true is for setattrspec(decl, l, force) when force is true: if decl2 == decl then decl['attrspec'] would be modified by having its own values appended to itself. I thought this was not desirable behaviour but you're right that someone more familiar with f2py might want to confirm this change.

[eric-wieser]

I'd be inclined to change this line in another PR, and only if a failing test can be constructed with it in its current state.

The rest of this PR is uncontroversial, and would be good to get in

[jaimefrio]

FWIW, the other changes seem sensible and LGTM.

Thanks Jean!

[jhelie]

@jaimefrio you're welcome. re raising exceptions in Python3 and 2: yes I agree meaningful names would be useful. I just got lazy, will update :)

re f2py: any suggestion for who might be qualified to validate that change?

[jhelie]

@jaimefrio I've modified the exception code to make it 2/3 compatible. There are 2 ways to do so and to avoid the if else statements and the syntax error when parsing with the "wrong" version - see http://python-future.org/compatible_idioms.html for details:

• use future.utils
• use six

The discussion on which one to use might be best moved to another PR

[njsmith]

So far numpy has carefully avoided adding a dependency on either six or future, and instead carries around its own compatibility hacks in numpy.compat. Python packaging has improved since that decision was made so maybe there's a better chance of adding dependencies being acceptable now, but this particular example doesn't seem very compelling to me. Maybe I'm missing something – is the reraise_ version actually better in any way besides being a bit more compact?

[jhelie]

@njsmith thanks for the explanation. Bear with me as I'm not sure I understand the question: can compat help address the 2/3 exception problem here? At the moment it seems that either the 2.7 exception is not raised properly (tuple used) or there will be a syntax error when parsing.

[njsmith]

Oh, I see, raise a, b, c and raise (a, b, c) are actually not the same on py2... though weirdly from experimentation it seems like raise tuple_obj is actually interpreted as raise tuple_obj[0], so the current code isn't as badly broken as one might expect.

It looks like tools/npy_tempita has its own little blob of compat code in compat3.py, presumably because this code is only run as part of building numpy, so it can't depend on numpy. OTOH it also looks like it's only run as part of tools/cythonize.py, which already has external dependencies (cython) and is only run by developers, so I'm not sure why we're carrying around a copy of tempita in the first place.

Anyway, I'd suggest using either exec("raise a, b, c") or forgetting about the problem, these particular lines are not worth spending time thinking about :-).

[jhelie]

Oh, I see, raise a, b, c and raise (a, b, c) are actually not the same on py2... though weirdly from experimentation it seems like raise tuple_obj is actually interpreted as raise tuple_obj[0], so the current code isn't as badly broken as one might expect.

yes, this is actually how I stumbled upon them using lgtm.com (https://lgtm.com/rules/5990074/) :)

Anyway, I'd suggest using either exec("raise a, b, c") or forgetting about the problem, these particular lines are not worth spending time thinking about :-).

I believe this is what six actually does under the hood - but I agree that it seems simpler to just "manually" deal with this case. Will update.

[charris]

Feel free to hack on tempita. Cython carries its own version, but warns not to rely on it, which is why it is vendored here. Tempita itself was unmaintained for a long time, recently taken over by another, and needed compatibility fixes. The Cython version probably has fixes also. I'd rather not add a six or future dependency.

[jhelie]

ok done. Provided no one objects the change re f2py I think this should be ok to go in now.

[eric-wieser]

No parens needed here (repeated below twice)

[eric-wieser]

I'd be inclined to change this line in another PR, and only if a failing test can be constructed with it in its current state.

The rest of this PR is uncontroversial, and would be good to get in

[jhelie]

@eric-wieser done - I've reverted the change in crackfortran, it's probably not worth thinking much longer about that line

[eric-wieser]

Ideally you'd have rebased and fixed the commit message formats, but in this case it was fine just to squash down into a single commit. Thanks @jhelie!

[jhelie]

will keep it in mind for next time :)

[eric-wieser]

Do you want to open a separate issue for enabling lgtm on pull requests? I could do that right now, but it would be good to get feedback from some other maintainers first

[jhelie]

sure will do, this indeed sounds like a great idea - feedback is always welcome!

[jhelie]

opened #9297