Thread: [matplotlib-devel] scipy core patch

I just committed Daishi's patch to CVS, with Fernando's modification
to catch the new scipy versus old scipy.

Those of you with semi-pristine boxes might want to try it out to make
sure it is working.

One issue: the patch builds either numarray (if found) and
scipy|numeric (if found but not both) with precedence given to
numeric.  Shouldn't it be the other way around, with precedence to
scipy_core over Numeric since if someone has the new scipy presumably
it's an upgrade and should take precedence.

I am weakly inclined to support all three rather than one or the other
in the interim, if only because it is a nice way to profile scipy core
versus the old Numeric and because it may be easier to debug what is
going on when we hit user level snags.  But since this is only an
interim solution I'm not strongly wedded to any approach.

Please test and report.

Checking in lib/matplotlib/numerix/__init__.py;
/cvsroot/matplotlib/matplotlib/lib/matplotlib/numerix/__init__.py,v  <--  __init__.py
new revision: 1.6; previous revision: 1.5

or later.

JDH

>>>>> "John" == John Hunter <jdh...@ac...> writes:

    John> I just committed Daishi's patch to CVS, with Fernando's
    John> modification to catch the new scipy versus old scipy.

Seems there is a problem with "resize"

peds-pc311:~/python/projects/matplotlib/examples> python contour_demo.py --Numeric --verbose-helpful
matplotlib data path /usr/share/matplotlib
$HOME=/home/jdhunter
CONFIGDIR=/home/jdhunter/.matplotlib
loaded rc file /home/jdhunter/.matplotlib/matplotlibrc
matplotlib version 0.85.1.cvs
verbose.level helpful
interactive is False
platform is linux2
numerix Numeric 24.0b2
font search path ['/usr/share/matplotlib']
loaded ttfcache file /home/jdhunter/.matplotlib/ttffont.cache
backend GTKAgg version 2.6.1
Traceback (most recent call last):
  File "contour_demo.py", line 29, in ?
    clabel(CS, inline=1, fontsize=10)
  File "/home/jdhunter/debs/matplotlib/usr//lib/python2.4/site-packages/matplotlib/pylab.py", line 1783, in clabel
    ret =  gca().clabel(*args, **kwargs)
  File "/usr/lib/python2.4/site-packages/matplotlib/axes.py", line 1264, in clabel
    return CS.clabel(*args, **kwargs)
  File "/usr/lib/python2.4/site-packages/matplotlib/contour.py", line 121, in clabel
    self.labels(inline)
  File "/usr/lib/python2.4/site-packages/matplotlib/contour.py", line 338, in labels
    x,y, rotation, ind  = self.locate_label(slc, lw)
  File "/usr/lib/python2.4/site-packages/matplotlib/contour.py", line 291, in locate_label
    XX = resize(array(linecontour)[:,0],(xsize, ysize))
  File "/usr/lib/python2.4/site-packages/matplotlib/numerix/__init__.py", line 87, in resize
    return a.resize(shape)
ValueError: resize only works on contiguous arrays


and for numarray


peds-pc311:~/python/projects/matplotlib/examples> python contour_demo.py --numarray --verbose-helpful
matplotlib data path /usr/share/matplotlib
$HOME=/home/jdhunter
CONFIGDIR=/home/jdhunter/.matplotlib
loaded rc file /home/jdhunter/.matplotlib/matplotlibrc
matplotlib version 0.85.1.cvs
verbose.level helpful
interactive is False
platform is linux2
numerix numarray 1.3.3
font search path ['/usr/share/matplotlib']
loaded ttfcache file /home/jdhunter/.matplotlib/ttffont.cache
backend GTKAgg version 2.6.1
Traceback (most recent call last):
  File "contour_demo.py", line 29, in ?
    clabel(CS, inline=1, fontsize=10)
  File "/home/jdhunter/debs/matplotlib/usr//lib/python2.4/site-packages/matplotlib/pylab.py", line 1783, in clabel
    ret =  gca().clabel(*args, **kwargs)
  File "/usr/lib/python2.4/site-packages/matplotlib/axes.py", line 1264, in clabel
    return CS.clabel(*args, **kwargs)
  File "/usr/lib/python2.4/site-packages/matplotlib/contour.py", line 121, in clabel
    self.labels(inline)
  File "/usr/lib/python2.4/site-packages/matplotlib/contour.py", line 338, in labels
    x,y, rotation, ind  = self.locate_label(slc, lw)
  File "/usr/lib/python2.4/site-packages/matplotlib/contour.py", line 291, in locate_label
    XX = resize(array(linecontour)[:,0],(xsize, ysize))
  File "/usr/lib/python2.4/site-packages/matplotlib/numerix/__init__.py", line 87, in resize
    return a.resize(shape)
  File "/home/jdhunter/debs/numarray/usr/lib/python2.4/site-packages/numarray/generic.py", line 891, in resize
    self.ravel()
  File "/home/jdhunter/debs/numarray/usr/lib/python2.4/site-packages/numarray/generic.py", line 922, in ravel
    self.setshape((self.nelements(),))
  File "/home/jdhunter/debs/numarray/usr/lib/python2.4/site-packages/numarray/generic.py", line 680, in setshape
    raise TypeError("Can't reshape non-contiguous numarray")
TypeError: Can't reshape non-contiguous numarray


Ditto on finance_demo....

JDH

On Nov 30, 2005, at 2:24 PM, John Hunter wrote:
>>>>>> "John" == John Hunter <jdh...@ac...> writes:
>
>     John> I just committed Daishi's patch to CVS, with Fernando's
>     John> modification to catch the new scipy versus old scipy.
>
> Seems there is a problem with "resize"


Hi John,

Sorry the patch is causing problems.
I'm not exactly sure how the resize
problem is coming up, however - all
I did for those things is:

--- numerix/__init__.py:
if (which[0] == 'numarray' or
     which[0] == 'numeric'):
     def typecode(a):
         return a.typecode()
     def iscontiguous(a):
         return a.iscontiguous()
     def byteswapped(a):
         return a.byteswapped()
     def itemsize(a):
         return a.itemsize()
     def resize(a, shape):
         return a.resize(shape)
---

and then replace what used to be
a.resize(shape) calls to resize(a, shape).

So aside from the extra function call
overhead I don't see how the semantics
would have changed (perhaps I missed
some substitutions? I did a simple grep
on the source tree to find occurrences
and edited manually, so I may have just
oops-ed a couple of them).

I'm also happy to fix the numarray&(numeric|scipy)
thing, although it would basically just be a
cut&paste job in setupext.py. (I'm in the middle
of some other things right now so it may be
a week or so before I get to this, however).

d

I don't understand the rationale for defining these functions, and in 
particular the "resize" function that is causing problems.  It looks to 
me like all three numeric packages include both resize functions and 
resize methods, with the latter differing in that they work in place and 
require contiguous arrays.  What your patch is doing is removing access 
to the resize function, so there is no way to resize a noncontiguous array.

I suggest backing out the patch, removing the changes to "resize", and 
putting the modified version back.

Eric



da...@eg... wrote:
> On Nov 30, 2005, at 2:24 PM, John Hunter wrote:
> 
>>>>>>> "John" == John Hunter <jdh...@ac...> writes:
>>
>>
>>     John> I just committed Daishi's patch to CVS, with Fernando's
>>     John> modification to catch the new scipy versus old scipy.
>>
>> Seems there is a problem with "resize"
> 
> 
> 
> Hi John,
> 
> Sorry the patch is causing problems.
> I'm not exactly sure how the resize
> problem is coming up, however - all
> I did for those things is:
> 
> --- numerix/__init__.py:
> if (which[0] == 'numarray' or
>     which[0] == 'numeric'):
>     def typecode(a):
>         return a.typecode()
>     def iscontiguous(a):
>         return a.iscontiguous()
>     def byteswapped(a):
>         return a.byteswapped()
>     def itemsize(a):
>         return a.itemsize()
>     def resize(a, shape):
>         return a.resize(shape)
> ---
> 
> and then replace what used to be
> a.resize(shape) calls to resize(a, shape).
> 
> So aside from the extra function call
> overhead I don't see how the semantics
> would have changed (perhaps I missed
> some substitutions? I did a simple grep
> on the source tree to find occurrences
> and edited manually, so I may have just
> oops-ed a couple of them).
> 
> I'm also happy to fix the numarray&(numeric|scipy)
> thing, although it would basically just be a
> cut&paste job in setupext.py. (I'm in the middle
> of some other things right now so it may be
> a week or so before I get to this, however).
> 
> d
> 
> 
> 
> 
> 
> 
> -------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc. Do you grep through log 
> files
> for problems?  Stop!  Download the new AJAX search engine that makes
> searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
> http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

>>>>> "Eric" == Eric Firing <ef...@ha...> writes:

    Eric> I suggest backing out the patch, removing the changes to
    Eric> "resize", and putting the modified version back.

I've been working on this in my local tree and need just a few more
modifications before committing.  I already made this change, as well
as making all three packages supported.  There are additional issues
to fix, eg the scipy.fftpack import is broken if only scipy core is
installed and not the full scipy, and Robert has been helping me a bit
with these.

JDH

On Dec 2, 2005, at 3:43 PM, Eric Firing wrote:
> I don't understand the rationale for defining these functions, and in  
> particular the "resize" function that is causing problems.  It looks  
> to me like all three numeric packages include both resize functions  
> and resize methods, with the latter differing in that they work in  
> place and require contiguous arrays.  What your patch is doing is  
> removing access to the resize function, so there is no way to resize a  
> noncontiguous array.

I didn't realize such a function existed, and
was mostly following Travis' "list of necessary
changes" (in his book, Section 2.6.1). Of course
given that the right function exists from a top-
level import, that would be preferred over the
customization that I was doing.

I apologize if I gave the impression that the
patch was meant to be complete/correct/ideal.
It was mostly intended as a preliminary query
to see if there was any interest in pursuing
the specific backend numeric library approach,
since I had seen the discussion on a unified
build (which I didn't understand well enough
to implement.)

John Hunter wrote:
> A number of the symbols in numerix.mlab that Daishi originally defined
> were from various scipy (non-core) proper modules and I could not find
> these in the core.  Will scipy core not be providing the equivalent of
> MLab.py?

I did originally attempt to build vs. just
scipy_core, but failed for the same reason.
I was impatient to get things working for
my own purposes (scratching my own itch
and all that), so didn't put much effort
into pursuing this aspect further - but I
should have mentioned it in the original
message; sorry about that.

I'm happy to see that the patch was useful
enough to incorporate into the tree and
improved upon.

Eric Firing wrote:
> Here is another anomaly that I needed to work around:
>
> >>> bb = scipy.array([1.1,2,2,3.3])
> >>> 1.1 in bb
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> ValueError: The truth value of an array with more than one element is  
> ambiguous. Use a.any() or a.all()
> >>> import numarray
> >>> nn = numarray.array([1.1,2.2,3.3])
> >>> 1.1 in nn
> True
>
> x in A behaves completely differently if A is a scipy ndarray than if  
> it is a numarray array.  This seems to me like a bug in scipy; the  
> num* behavior is certainly easier to use, in that it preserves the  
> list-like behavior when used in a list-like context.

I hit this also. Another behavioral difference
that pops up is:

---
In [3]: if scipy.arange(10) == None:
    ...:     pass
    ...:
------------------------------------------------------------------------ 
---
exceptions.ValueError                                Traceback (most  
recent call last)

/usr/local/python/python_2005-11-10/<ipython console>

ValueError: The truth value of an array with more than one element is  
ambiguous. Use a.any() or a.all()
In [4]:
---

versus:

---
In [7]: if Numeric.arange(10) == None:
    ...:     pass
    ...:
In [8]:
---

This was the reason for, e.g., this part of the patch:

Index: lib/matplotlib/contour.py
===================================================================
RCS file: /cvsroot/matplotlib/matplotlib/lib/matplotlib/contour.py,v
retrieving revision 1.16
diff -r1.16 contour.py
---
659c661
<         if linewidths == None:
---
 >         if linewidths is None:
---

(In this case I believe the new scipy approach
is less ambiguous and therefore better - the
intended semantics AFAICT in contour.py is for
'is', not '==' anyways.).

Thanks,
d

John,


> I am weakly inclined to support all three rather than one or the other
> in the interim, if only because it is a nice way to profile scipy core
> versus the old Numeric and because it may be easier to debug what is
> going on when we hit user level snags.  But since this is only an
> interim solution I'm not strongly wedded to any approach.

I agree.  I think the testing and transition will be easier if all three 
are supported for now.  I don't see much downside to doing this.

Eric