matplotlib-users Mailing List for matplotlib (Page 3)

Hi,

I got curious this morning and wanted to try out the QtAgg backend. I get the 
following error when I try to import pylab from a python interpreter:

File "/usr/lib/python2.3/site-packages/matplotlib/backends/backend_qtagg.py", 
line 12, in ?
    from backend_qt import qt, FigureManagerQT, FigureCanvasQT,\
ImportError: No module named backend_qt

backend_qt.py is present in my cvs directory, but it looks like it didnt make 
it into the MPL-0.72 tarball.

-- 

Darren

On Feb 23, 2005, at 1:28 AM, Greg Novak wrote:
> Gripe #3 is related to interactive windows when Python and the X11
> server are connected by a slow link.  All of the interactive backends
> I've used for matplotlib look great, but render very slowly in this
> situation b/c they're sending all the pixel data.  Since most plots
> consist of a few lines, a bit of text, and a few dots, it'd be nice if
> the windows rendered quickly over slow connections.  For example,
> Gnuplot runs very well in this configuration.  I realize that this
> doesn't invovle matplotlib per se, I just wanted to throw it out there
> as a concern.
>
This is a consequence of how most of the interactive backends are 
implemented. Since matplotlib is using agg to render graphics for them, 
all updates to graphs mean a new image must be transmitted to the 
window. If it is a remote window, that update is going to cost. There 
may be ways of improving the backends so that not the whole window 
needs to be updated, but I have a feeling that isn't going to yield a 
big improvement in many cases (a new plot of points means changes over 
most of the image even if it can be characterized by a relatively small 
number of points and labels). I don't see any great solution to this 
(maybe John has some good ideas). I pointed this out as a consequence 
of going this way before it was done (and I recommended going this way 
:-). Implementing the backends using their built in plotting commands 
may be a way around this, but it means having much higher maintenance 
costs for backends and lots of annoying capability mismatches (some 
backends don't support rotated text, alpha blending, etc.)

Perry

Ok, I was so rattled when I wrote the previous message that I forgot
to ask my actual question:

I was trying to change backends midstream because I was only getting
b/w plots when I used pcolor--I couldn't change the colormap.  At
first I thought that this was because I was using the Postscript
backend and for some reason it assumed I wanted b/w output for
printing.  After fooling around with some of the other imaging
backends, I used one of the interactive ones (this is a pain b/c I'm
running Python remotely and I'm sitting at the end of a slow internet
link.  Therefore interactive windows take forever to pop up.  More on
this later), and the plot was still b/w.

After significant frustration, I realized that my problem was that I
was specifying the color map as 'cm=' instead of 'cmap='; a reasonable
guess since the rest of the line is 'cm.hot'  The problem was that
matplotlib silently ignored my misspecified argument.

This is gripe #1: weak typing only works when using undefined
variables bites you on the nose.  Otherwise we're back to the bad old
days of fortran, when misspelling a variable name silently defined a
new variable.  It would be nice if somewhere in the heirarchy of
function calls within matplotlib someone checked to make sure there
were no lonely, unused keyword arguments sloshing around.  I realize
that this is hard to do robustly and with any generality, but there's
a lot at stake.  Python would be unusable if use-before-definition
didn't generate an exception.

Gripe #2 is that it would be nice if the same word were abbreviated
the same way everywhere.  Ie, either cmap or cm, not both.  

Gripe #3 is related to interactive windows when Python and the X11
server are connected by a slow link.  All of the interactive backends
I've used for matplotlib look great, but render very slowly in this
situation b/c they're sending all the pixel data.  Since most plots
consist of a few lines, a bit of text, and a few dots, it'd be nice if
the windows rendered quickly over slow connections.  For example,
Gnuplot runs very well in this configuration.  I realize that this
doesn't invovle matplotlib per se, I just wanted to throw it out there
as a concern.

I apologize for the negative tone of this mail.  Don't get me wrong--I
use matplotlib and I like it b/c it's the best thing I've found.
However, I just had one of those "GAAARGHGGHHHH!" moments and it's
taking a little while to de-stress.

Cheers,
Greg

Not everyone uses matplotlib inside of scripts!  Judging from the
manual, this is the only approved way to use the library.

There seems to be no way to change the backend in the middle of an
interactive session.  Looking through the manual, I find that I can
specify it on the command line, in .matplotlibrc, or via
matplotlib.use('..'), but only "before you import matplotlib.pylab"

I would be extremely happy if there were a way to change backends
midstream from an interactive shell (I use ipython, for example)

Thanks,
Greg

John,

Thanks for the reply. Sorry for the delay, I've been off line for a few 
days.

John Hunter wrote:
>>>>>>"Chris" == Chris Barker <Chr...@no...> writes:

>     Chris> Hi all, Is there a way to get the size of a text object?

> Do you mean the width and height
> in points?

At this point, points. The other option is figure coordinates. The 
problem at hand is that we're generating a figure that has some fairly 
large labels on the Y axis, so we're having to do the layout of subplots 
ourselves. We've got it looking OK with trial and error, but if we 
change a label, it might not layout right anymore, so I thought it would 
be helpful to be able to check the sizes of things to do it automagically.

Besides, aren't there methods to convert between the various coord. 
systems anyway?

> but could be screwed up by a figure resize.

Yeah, it would , but right now I'm working with the AGG backend, so I 
can stick with one figure size.

> FYI, this is an issue that crops up a lot and is vexing.  What one
> would like to be able to do is use a layout engine and say, place
> object one above and to the right of object 2 with a pad of 2 points.

Yes, that would be nice. In fact, I've been planning to do exactly that 
for my wxFloatCanvas. It's easier there because I've only got wx as a 
back end, but still a bit tricky. One thing I've done to help is simply 
declare that I'm always assuming 72dpi, rather than trying to use the 
system's idea of the display resolution. This has helped keep results 
consistent across systems. I have had to re-scale text, however.

> The text instance can give you its bounding box in display if you pass
> it the backend renderer -- this is required because the width and
> height can be backend dependent.  I suppose you are using OO agg based
> on your previous posts.

Mostly, it's a mix as I'm working with someone else and most of the 
examples are in the pylab interface.

> One problem with the current design that is
> that the agg canvas doesn't generate it's renderer until draw time
> (with caching), but you need access to the renderer before draw time
> for layout that depends on text.

Right. So, do any of the default layout's take text into account 
(subplot, for example?)

 > If we move this logic to a
> get_renderer method, you can use it at draw time.  I'll attach a
> replacement backend_agg.FigureCanvasAgg class to support this below
> 
>     from matplotlib.figure import Figure
>     from matplotlib.backends.backend_agg import FigureCanvasAgg
> 
>     fig = Figure()
>     canvas = FigureCanvasAgg(fig)

Didn't you add canvas as a Figure() attribute to support the new 
Figure.savefig() method? (added for me, such service!)

>     renderer = canvas.get_renderer()
>     ax = fig.add_subplot(111)
>     ax.plot([1,2,3])
>     t = ax.text(1,2,'hi mom')
>     bbox =  t.get_window_extent(renderer)
>     print 'display', bbox.get_bounds()  #l,b,w,h
> 
>     # get the axes data coords bbox of this display bounding box
>     from matplotlib.transforms import inverse_transform_bbox
>     axbox = inverse_transform_bbox(ax.transData, bbox)
> 
>     print 'data coords', axbox.get_bounds()
> 
>     fig.savefig('test')

Thanks for this example. I hope I"ll get a chance to play with this 
soon. My schedule is a nightmare for a while.

thanks,

-Chris



-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

On Tue, 2005-02-22 at 16:46, Perry Greenfield wrote:
> >   - Are Numeric and numarray MAs similar enough to be used at the
> >     C-API level, which is where agg would be checking the mask?  Does
> >     anyone know whether MAs will be part of the Numeric3 core?  I
> >     haven't seen any reference to them in the PEP.
> >
> We'll have to raise it. I imagine that they should be. As I mentioned, 
> NaNs don't solve all the problems that MA does. Since MA is layered on 
> Numeric/numarray it shouldn't be hard to do so. As far as C-API, since 
> it is a Python implementation, I presume it comes down to whether or 
> not the needed attributes are the same for the numeric and Numarray 
> variants. I'm not immediately familiar with that, but Todd should be 
> able to give a fairly quick answer on that (he did the port to 
> numarray)

As Perry said,  numarray.ma is a port of MA to numarray so they're
fairly close.  Both packages are pure Python layered over ordinary
numarray or Numeric numerical arrays.   Accessing from C,  both packages
should yield data or mask information via a method callback.  The
resulting arrays are PyArrayObjects which are source compatible only: 
extensions using MA component arrays will need to be compiled for either
Numeric or numarray as we do now for _image, _transforms, and _contour.

Todd

On Feb 22, 2005, at 4:21 PM, John Hunter wrote:

>>>>>> "Perry" == Perry Greenfield <pe...@st...> writes:
>
>     Perry> As far as keyword args go, it seems to me that they would
>     Perry> be more convenient in many cases, but as Stephen mentions,
>     Perry> may be a fair amount of work (and in essence, they are an
>     Perry> attribute of the data, so that may be where they belong).
>
> In the context of plotting, it isn't clear that NaNess is an attribute
> of the data.  If the data are y = sin(2*pi*t), then NaNess enters only

I meant masks were an attribute of the data, but not NaNs (in other 
words, the way MA handles it may be the most appropriate)

Perry

On Feb 22, 2005, at 4:21 PM, John Hunter wrote:

>>>>>> "Perry" == Perry Greenfield <pe...@st...> writes:
>
>     Perry> As far as keyword args go, it seems to me that they would
>     Perry> be more convenient in many cases, but as Stephen mentions,
>     Perry> may be a fair amount of work (and in essence, they are an
>     Perry> attribute of the data, so that may be where they belong).
>
> In the context of plotting, it isn't clear that NaNess is an attribute
> of the data.  If the data are y = sin(2*pi*t), then NaNess enters only
> under certain transformations (eg log) of the data.  From my
> perspective, or the perspective of a matplotlib Line, the data are
> intact, it is just that under certain transformations the data are
> invalid.  Thus the mask is only needed under certain views
> (transformations) which the Line class is mostly unaware of.  I think
> there are two cases to be distinguished: the case Eric mentioned where
> some data points are NaN or None because the measurements are missing
> or invalid, and the case where the data are valid but are nan under
> certain transformations.
>
Right.

> For the latter, it would be maximally useful to be able to do
>
>   #x,y,xt,yt are numerix arrays; transform puts in NaN on domain error
>   xt, yt = transform(x, y)
>
> and the drawing routine drops NaN points and handles the connecting
> segments properly.  That it only works for float arrays is not a
> problem since that is what we are using, eg in the line class
>
>         self._x = asarray(x, Float)
>         self._y = asarray(y, Float)
>
> but it is my current understanding that this ain't gonna happen in a
> consistent way for Numeric, Numeric3, and numarray across platforms,
> because my brief forays into researching this issue turned up posts by
> Tim Peters saying that there was no standard way of dealing with IEEE
> 754 special values across compilers.  If that's true, and we can't fix
> it or work around it, then I think boolean masks passed as a kwarg may
> be the easiest way to handle this across various numerix
> implementations, assuming that Numeric3 comes to fruition and assuming
> that there isn't a consistent MA approach that is accessible at the
> API level, which I don't know to be true but I get the feeling that
> this is a reasonable guess.
>
I think you may be over-extending what Tim was saying. I believe the 
issue is writing C code that handles things like tests against NaN 
values correctly. In that, Tim is right, there is a great deal of 
inconsistency in how C compilers handle ieee special values. But 
generally speaking, the computations *are* handled consistently by the 
floating point processors. numarray solved this problem by not relying 
on the C compiler to test or set NaN values (it tests raw bit patterns 
instead) and so should handle this issue properly. Numeric doesn't, but 
Numeric3 is planned to. So I think NaNs can't be handled right now 
because of Numeric, but I don't think the C compiler issue that Tim 
mentions is a roadblock (it is a nuisance for implementation though)..

> I'd be interested in getting feedback from those of you who have
> informed opinions on these matters:
>
>   - Is it possible to handle NaN in Numeric/numarray arrays across the
>
>   big three platforms?  I'm pretty sure the answer here is no.
>
No for Numeric, yes for numarray/Numeric3

>   - Are Numeric and numarray MAs similar enough to be used at the
>     C-API level, which is where agg would be checking the mask?  Does
>     anyone know whether MAs will be part of the Numeric3 core?  I
>     haven't seen any reference to them in the PEP.
>
We'll have to raise it. I imagine that they should be. As I mentioned, 
NaNs don't solve all the problems that MA does. Since MA is layered on 
Numeric/numarray it shouldn't be hard to do so. As far as C-API, since 
it is a Python implementation, I presume it comes down to whether or 
not the needed attributes are the same for the numeric and Numarray 
variants. I'm not immediately familiar with that, but Todd should be 
able to give a fairly quick answer on that (he did the port to 
numarray)

Perry

>>>>> "Perry" == Perry Greenfield <pe...@st...> writes:

    Perry> As far as keyword args go, it seems to me that they would
    Perry> be more convenient in many cases, but as Stephen mentions,
    Perry> may be a fair amount of work (and in essence, they are an
    Perry> attribute of the data, so that may be where they belong).

In the context of plotting, it isn't clear that NaNess is an attribute
of the data.  If the data are y = sin(2*pi*t), then NaNess enters only
under certain transformations (eg log) of the data.  From my
perspective, or the perspective of a matplotlib Line, the data are
intact, it is just that under certain transformations the data are
invalid.  Thus the mask is only needed under certain views
(transformations) which the Line class is mostly unaware of.  I think
there are two cases to be distinguished: the case Eric mentioned where
some data points are NaN or None because the measurements are missing
or invalid, and the case where the data are valid but are nan under
certain transformations.

For the latter, it would be maximally useful to be able to do

  #x,y,xt,yt are numerix arrays; transform puts in NaN on domain error
  xt, yt = transform(x, y)  
  
and the drawing routine drops NaN points and handles the connecting
segments properly.  That it only works for float arrays is not a
problem since that is what we are using, eg in the line class

        self._x = asarray(x, Float)
        self._y = asarray(y, Float)

but it is my current understanding that this ain't gonna happen in a
consistent way for Numeric, Numeric3, and numarray across platforms,
because my brief forays into researching this issue turned up posts by
Tim Peters saying that there was no standard way of dealing with IEEE
754 special values across compilers.  If that's true, and we can't fix
it or work around it, then I think boolean masks passed as a kwarg may
be the easiest way to handle this across various numerix
implementations, assuming that Numeric3 comes to fruition and assuming
that there isn't a consistent MA approach that is accessible at the
API level, which I don't know to be true but I get the feeling that
this is a reasonable guess.

I'd be interested in getting feedback from those of you who have
informed opinions on these matters:

  - Is it possible to handle NaN in Numeric/numarray arrays across the
    big three platforms?  I'm pretty sure the answer here is no.

  - Are Numeric and numarray MAs similar enough to be used at the
    C-API level, which is where agg would be checking the mask?  Does
    anyone know whether MAs will be part of the Numeric3 core?  I
    haven't seen any reference to them in the PEP.

JDH

On Feb 21, 2005, at 7:00 PM, Eric Firing wrote:

>
> Stephen,
>
>>> Are you suggesting something like this?  Let each plotting function 
>>> have a new kwarg, perhaps called "validmask", with the same 
>>> dimensions as the dependent variable to be plotted, and with nonzero 
>>> where the variable is valid and 0 where it is missing.
>> More or less, except that the mask is an attribute (?) of a 
>> MaskedArray object.  I for one would be in favor of this capability.
>
> I agree that this is an alternative, but I am not sure that it is 
> better than what I described.  It requires all the machinery of the 
> ma/MA module, which looks cumbersome to me.  What does it gain?  max 
> and min will do the right thing on the masked array input, so one 
> would not have to duplicate this inside matplotlib.  It is not hard to 
> duplicate, however.  How much more ma/MA functionality would actually 
> be useful?
>
> When it was originally developed, the MaskedArray may have been a good 
> way to get past Numeric's lack of nan-handling.  In the long run, 
> however, it seems to me that Python needs a numeric module with good 
> nan-handling (as in Matlab and Octave), and that this will render the 
> Masked Array obsolete.  If so, then specifying a mask as a kwarg in 
> matplotlib, and not using MA internally, may be simpler, more robust, 
> and more flexible.
>
> The user would still be free to use MA/ma externally, if desired.
>
> A variation would be to support MA/ma in matplotlib only to the extent 
> of checking for a MaskedArray input, and if it is present, breaking it 
> apart and using the mask as if it had come via the kwarg. One could 
> use either the kwarg or a Masked Array.
>

When we looked at the issue of using NaNs in place of masks or masked 
arrays, we concluded (well, I did anyway) that while NaNs could be used 
to replace masks in many instances, they could not be used in all. 
There are a lot of cases where people want to retain the value being 
masked (e.g., to do statistics on the rejected values). NaNs as masks 
only work for float and complex, not ints. So both approaches are 
useful and needed as far as I can tell.

As far as keyword args go, it seems to me that they would be more 
convenient in many cases, but as Stephen mentions, may be a fair amount 
of work (and in essence, they are an attribute of the data, so that may 
be where they belong).

Perry

On Tuesday 22 February 2005 01:47 pm, John Hunter wrote:
> >>>>> "Darren" == Darren Dale <dd...@co...> writes:
>
>     Darren> Hi, I am reporting a minor bug. I noticed that if I change
>     Darren> the ticklabel size to 10, draw a semilogy plot, and save
>     Darren> an eps file, that the yticklabels in the eps are badly
>     Darren> formatted. The exponent runs into the base.
>

[...]

>
> The relevant section of code is matplotlib.text.Text._get_layout_super

[...]

> see if you can find something that works well for a variety of
> reasonable font sizes.  You might want to make the pad dependent on
> the fontsize...

The following conditions will work for eps files, however, the labels in the 
plot on the screen are adversely effected.

        if size<12: xe = xb+1.3*wb
        elif size==16: xe = xb+1.175*wb
        else: xe = xb+1.125*wb
        ye = yb+0.5*hb

Is it possible that the translation to postscript is what needs to be 
modified?

Darren

>>>>> "Darren" == Darren Dale <dd...@co...> writes:

    Darren> Hi, I am reporting a minor bug. I noticed that if I change
    Darren> the ticklabel size to 10, draw a semilogy plot, and save
    Darren> an eps file, that the yticklabels in the eps are badly
    Darren> formatted. The exponent runs into the base.

    Darren> from matplotlib import * rc('tick',labelsize=10) from
    Darren> pylab import *

    Darren> semilogy([1,2,3],[1e-5,1,1e5]) savefig('temp.eps')

The relevant section of code is matplotlib.text.Text._get_layout_super

        wb,hb = renderer.get_text_width_height(base, self._fontproperties, False)
        we,he = renderer.get_text_width_height(exponent, fpexp, False)

        w = wb+we

        xb, yb = self._transform.xy_tup((self._x, self._y))
        xe = xb+wb
        ye = yb+0.5*hb
        h = ye+he-yb

w,h is width and height and e,b is exponent and base -- eg, xb is the
x location of the base and wb is the width of the base.  Try adding a
pad to xe, eg

   xe = xb+1.05*wb

and see if you can find something that works well for a variety of
reasonable font sizes.  You might want to make the pad dependent on
the fontsize...

JDH

>>>>> "Eric" == Eric Firing <ef...@ha...> writes:

    Eric> Perry, John, Progress!  

Cool!

    Eric> I found that the problem I was having with PolyCollection
    Eric> was this: the vertices argument must be a sequence (list or
    Eric> tuple) of tuples of tuples--if one gives it a list of
    Eric> *lists* of tuples, one gets

    Eric> [first part of trace omitted] File
    Eric> "/usr/lib/python2.3/site-packages/matplotlib/collections.py",
    Eric> line 205, in draw self._offsets, self._transOffset)
    Eric> TypeError: CXX: type error

    Eric> (The line number was smaller before I put in some debugging
    Eric> print statements.)

    Eric> I think this fussiness qualifies as a bug; the docstring for
    Eric> PolyCollection says vertices can be a sequence of sequences
    Eric> of tuples.  I don't know what the right way to fix it is,
    Eric> however, so I am working around it.

Fair enough -- I just fixed all the agg collection drawing routines to
work with the sequence API and not require tuples.  Glad to see you're
making progress -- poly contouring is something I'd like to see added.

    Eric> Having solved that problem, I am getting more optimistic
    Eric> about being able to come up with a usable filled contour
    Eric> capability fairly quickly.  Still no promises, though.

Great -- be mindful of the contourf matlab docstrings.  Strict
adherence is not required, but it is nice to be compatible where
possible.

    Eric> All this brings to mind a question that has puzzled me for a
    Eric> long time: why does matplotlib internally use sequences of
    Eric> (x,y) tuples instead of numerix arrays--either a 2-D array,
    Eric> or a pair (or tuple) of 1-D arrays? I would think that
    Eric> running all plotted numbers through the conversion from
    Eric> arrays to Python tuples, and then from there into the native
    Eric> data types for each backend, would incur a big performance
    Eric> penalty when plotting large numbers of points.  Not that I
    Eric> am suggesting a redesign--I am just curious.

Historical and other reasons.  The historical part is that this part
of the code was written before Todd had solved the numeric/numarray
API compatibility problem for matplotlib.  These are now solved, 've
been slowly adding some numerix code to backend agg, most recently in
0.72.  I don't think it would make a lot of difference for
collections.  In the first place, you'd have to create all these lists
of numarray lists, since the collection is by definition a list of
disconnected lines.  In the second place, there is a fair amount going
on in the inner loop that I think would offset the gains you get from
using numeric.  In draw_lines, where the x,y access is a major part of
the inner loop, I do use numerix.

The backend API is moving to a path drawing model, which may obviate
the need for specialized collection drawing methods. The collection
interface would remain unchanged, but we might get away w/o having
special methods to draw them.

JDH

Stephen Walton wrote, in the context of using masked arrays rather than 
a keyword argument which would be the mask:

> It would be more flexible.  Instead of having to actually replace data 
> with NaN, you could create a mask which marked data to be ignored for 
> the moment:

This is, of course, incorrect.  Both approaches would allow arbitrary 
data sets to be masked as needed.

There was a mention over on an astronomy group of the progress being 
made in masked astronomical images.  Here too, the mask "comes along" 
with the data.  Perry et al., does STScI anticipate using 
numarray/Numeric masked arrays within PyFITS to handle this?

Perry,  John,

Progress!  I found that the problem I was having with PolyCollection was 
this: the vertices argument must be a sequence (list or tuple) of tuples 
of tuples--if one gives it a list of *lists* of tuples, one gets

[first part of trace omitted]
  File "/usr/lib/python2.3/site-packages/matplotlib/collections.py", 
line 205, in draw
    self._offsets,  self._transOffset)
TypeError: CXX: type error

(The line number was smaller before I put in some debugging print 
statements.)

I think this fussiness qualifies as a bug; the docstring for 
PolyCollection says vertices can be a sequence of sequences of tuples.  
I don't know what the right way to fix it is, however, so I am working 
around it.

Having solved that problem, I am getting more optimistic about being 
able to come up with a usable filled contour capability fairly quickly.  
Still no promises, though.

All this brings to mind a question that has puzzled me for a long time: 
why does matplotlib internally use sequences of (x,y) tuples instead of 
numerix arrays--either a 2-D array, or a pair (or tuple) of 1-D arrays? 
I would think that running all plotted numbers through the conversion 
from arrays to Python tuples, and then from there into the native data 
types for each backend, would incur a big performance penalty when 
plotting large numbers of points.  Not that I am suggesting a 
redesign--I am just curious.

Eric

Hi,

I am reporting a minor bug. I noticed that if I change the ticklabel size to 
10, draw a semilogy plot, and save an eps file, that the yticklabels in the 
eps are badly formatted. The exponent runs into the base.

from matplotlib import *
rc('tick',labelsize=10)
from pylab import *

semilogy([1,2,3],[1e-5,1,1e5])
savefig('temp.eps')

-- 

Darren

Hello,

>
> I agree that this is an alternative, but I am not sure that it is 
> better than what I described.  It requires all the machinery of the 
> ma/MA module, which looks cumbersome to me.  What does it gain?

It would be more flexible.  Instead of having to actually replace data 
with NaN, you could create a mask which marked data to be ignored for 
the moment:  all negative values, say, or all values with a complex part 
less than 1e-5.  Much more flexible.  Having said that, I agree that NaN 
should also be ignored wherever it occurs.

>  One could use either the kwarg or a Masked Array.

-1 on the kwarg.  It seems to me that adding it to every plot command 
uglifies the interface significantly as well as being more work for John.

Stephen

Stephen,

>> Are you suggesting something like this?  Let each plotting function 
>> have a new kwarg, perhaps called "validmask", with the same dimensions 
>> as the dependent variable to be plotted, and with nonzero where the 
>> variable is valid and 0 where it is missing.
> 
> 
> More or less, except that the mask is an attribute (?) of a MaskedArray 
> object.  I for one would be in favor of this capability.

I agree that this is an alternative, but I am not sure that it is better 
than what I described.  It requires all the machinery of the ma/MA 
module, which looks cumbersome to me.  What does it gain?  max and min 
will do the right thing on the masked array input, so one would not have 
to duplicate this inside matplotlib.  It is not hard to duplicate, 
however.  How much more ma/MA functionality would actually be useful?

When it was originally developed, the MaskedArray may have been a good 
way to get past Numeric's lack of nan-handling.  In the long run, 
however, it seems to me that Python needs a numeric module with good 
nan-handling (as in Matlab and Octave), and that this will render the 
Masked Array obsolete.  If so, then specifying a mask as a kwarg in 
matplotlib, and not using MA internally, may be simpler, more robust, 
and more flexible.

The user would still be free to use MA/ma externally, if desired.

A variation would be to support MA/ma in matplotlib only to the extent 
of checking for a MaskedArray input, and if it is present, breaking it 
apart and using the mask as if it had come via the kwarg. One could use 
either the kwarg or a Masked Array.

Eric

Eric Firing wrote:

> Are you suggesting something like this?  Let each plotting function 
> have a new kwarg, perhaps called "validmask", with the same dimensions 
> as the dependent variable to be plotted, and with nonzero where the 
> variable is valid and 0 where it is missing.

More or less, except that the mask is an attribute (?) of a MaskedArray 
object.  I for one would be in favor of this capability.

Perry,

>> I would like to phase in matplotlib to replace Matlab ASAP for 
>> plotting physical oceanographic observations, primarily current 
>> profile measurements.  I (and many other physical oceanographers) 
>> primarily use contourf to plot filled contours; I only rarely use line 
>> contours.  It looks to me like gcntr.c has the necessary 
>> functionality--the ability to output polygons enclosing regions 
>> between a pair of specified levels. Is someone already working on 
>> exposing that functionality in matplotlib, or is it planned?
>>
> No one (as far as I know :-) is working on it right now. It is in our 
> plans to add this capability. As you correctly note, the underlying C 
> code can handle this capability. I'm not sure how long it will be; right 
> now the priority is to finish contour labeling capability, and the 
> person working on that also has other work that competes with her time 
> to do this. I'm guessing that she could start looking at it in a couple 
> weeks. Of course, if someone wants to help now, that would be great.

I have started working on it.  I don't know how far I will get; the 
necessary change to the c extension code was easy, but my first attempt 
to make a PolyCollection work in place of a Line Collection is failing. 
  I will do a bit more research before asking for help, if necessary. 
(No promises--I don't have much time to work on this, and it is my first 
plunge into the innards of matplotlib.)

> 
>> It appears that gcntr.c also has the ability to handle missing data 
>> via setting elements of the reg array to zero, and that this could be 
>> exposed fairly easily in the contour method in axes.py by adding "reg" 
>> to the set of kwargs.  Correct?  If so, is this also planned?
>>
> Correct. Yes (it is planned).
> 
>> The question of missing data handling in contour plotting brings up 
>> the more general issue of how to handle data gaps in plots.  For 
>> example, the ocean current profiles that I measure using a Doppler 
>> profiler extend to varying depths, and sometimes have holes in the 
>> middle where there are not enough acoustic scatterers to give a 
>> signal.  This sort of thing--data gaps--is universal in physical 
>> oceanography.  One of Matlab's major strengths is the way it handles 
>> them, using nan as a bad value flag.  Plotting a line with the plot 
>> command, the line is broken at each nan; so if there is a hole in the 
>> data, the plot shows exactly that.  The same for contouring: nans are 
>> automatically used as a mask.
>>
>> Obviously, not everyone needs this kind of automatic handling of data 
>> gaps, but I think it would be very useful for many applications, so I 
>> hope it can be considered as a possible goal.  At the plotting level, 
>> collections may make it easier to implement than would have been the 
>> case in the early days of matplotlib.  At the array manipulation 
>> level, the implementation could involve either masked arrays or nans.  
>> I would greatly prefer the Matlab-style nan approach, but I don't know 
>> whether this would work with Numeric.  Maybe in Numeric3?  Numarray 
>> appears better equipped, with its ieeespecial.py module.
>>
> I think you touch on the key issue. I think we'd have to figure out how 
> to handle this between Numeric and numarray (and Numeric3 potentially). 
> Would a mask array be a suitable substitute as an interim solution?

Are you suggesting something like this?  Let each plotting function have 
a new kwarg, perhaps called "validmask", with the same dimensions as the 
dependent variable to be plotted, and with nonzero where the variable is 
valid and 0 where it is missing.  The mask would then be used (1) to 
limit the autoranging tests to the valid data, (2) in the case of line 
plotting, to break the line up into segments so that a LineCollection 
would be plotted, (3) in the case of contouring, to set the reg array, 
(4) for images or pcolors to similarly mask out the invalid regions with 
white, or transparent, or perhaps some settable color.

This could be implemented in matplotlib in a way that would not depend 
on any special features, or likely changes, in the 
Numeric/Numeric3/numarray set.

A numarray user could then use
def notnan(y):
     return numarray.ieeespecial.mask(y, numarray.ieeespecial.NAN)

and say
plot(x, y, validmask=notnan(y))

In any case, this "validmask kwarg" solution seems to me like a 
perfectly good one from a user's standpoint, and a good bridge to the 
happy day when Numeric/Numeric3/numarray converge or evolve to a single, 
dominant numerical module with good nan handling built in.  (I very much 
hope such convergence will occur, and the sooner the better.)

Eric

On Feb 19, 2005, at 9:23 PM, Eric Firing wrote:

> John et al.,
>
> I would like to phase in matplotlib to replace Matlab ASAP for 
> plotting physical oceanographic observations, primarily current 
> profile measurements.  I (and many other physical oceanographers) 
> primarily use contourf to plot filled contours; I only rarely use line 
> contours.  It looks to me like gcntr.c has the necessary 
> functionality--the ability to output polygons enclosing regions 
> between a pair of specified levels. Is someone already working on 
> exposing that functionality in matplotlib, or is it planned?
>
No one (as far as I know :-) is working on it right now. It is in our 
plans to add this capability. As you correctly note, the underlying C 
code can handle this capability. I'm not sure how long it will be; 
right now the priority is to finish contour labeling capability, and 
the person working on that also has other work that competes with her 
time to do this. I'm guessing that she could start looking at it in a 
couple weeks. Of course, if someone wants to help now, that would be 
great.

> It appears that gcntr.c also has the ability to handle missing data 
> via setting elements of the reg array to zero, and that this could be 
> exposed fairly easily in the contour method in axes.py by adding "reg" 
> to the set of kwargs.  Correct?  If so, is this also planned?
>
Correct. Yes (it is planned).

> The question of missing data handling in contour plotting brings up 
> the more general issue of how to handle data gaps in plots.  For 
> example, the ocean current profiles that I measure using a Doppler 
> profiler extend to varying depths, and sometimes have holes in the 
> middle where there are not enough acoustic scatterers to give a 
> signal.  This sort of thing--data gaps--is universal in physical 
> oceanography.  One of Matlab's major strengths is the way it handles 
> them, using nan as a bad value flag.  Plotting a line with the plot 
> command, the line is broken at each nan; so if there is a hole in the 
> data, the plot shows exactly that.  The same for contouring: nans are 
> automatically used as a mask.
>
> Obviously, not everyone needs this kind of automatic handling of data 
> gaps, but I think it would be very useful for many applications, so I 
> hope it can be considered as a possible goal.  At the plotting level, 
> collections may make it easier to implement than would have been the 
> case in the early days of matplotlib.  At the array manipulation 
> level, the implementation could involve either masked arrays or nans.  
> I would greatly prefer the Matlab-style nan approach, but I don't know 
> whether this would work with Numeric.  Maybe in Numeric3?  Numarray 
> appears better equipped, with its ieeespecial.py module.
>
I think you touch on the key issue. I think we'd have to figure out how 
to handle this between Numeric and numarray (and Numeric3 potentially). 
Would a mask array be a suitable substitute as an interim solution?

Perry

>>>>> "James" == James Boyle <bo...@ll...> writes:

    James>  From what I can gather, I can change the line colors and
    James> widths in the contour routine but cannot specify a line
    James> style such as dashed, dotted etc.

    James> Is this true? If so I would like to have this capability -
    James> it makes black and white plots easier to interpret.

There was a bug in the drawing routing for agg line collections that
was preventing us from using this. I fixed it this morning.  The new
default is to use dashed lines for contour levels <0.

The contour routine returns (levels, collections) where collections is
a list of LineCollection instances, one for each level.  You can now
use the collection.set_linestyle method, which *should* work across
backends.  Valid args are


'solid' | 'dashed', 'dashdot', 'dotted' |  (offset, on-off-dash-seq)

where the last option allows you to control the dash sequence
explicitly; on-off-dash-seq is an even length sequence of on/off ink
in points.  Eg, you can do 

  dashes = 0, (5,1,10,3) 
  collection.set_linestyle(dashes)

which is 5 points on, 1 off, followed by 10 points on, 3 off and then
repeated.


    James> In any case the contour in 0.72 works great - thanks

Glad to hear it -- we'll be adding more features (eg auto-labeling,
poly filling) as we get time.  

The linestyle support is in 0.72.1, which I uploaded to the sf site
this morning.

JDH

>>>>> "Darren" == Darren Dale <dd...@co...> writes:

    Darren> Wow, I was going crazy trying to reproduce this. Your
    Darren> script works fine. Try this one:

    ....

    Darren> Both the x *and* y axes have to be set_visible(False) to
    Darren> reproduce the behavior.

OK, that helps.  Thanks.

The problem can be fixed in backend_ps RendererPS.set_color, change
the line

            self.color = (r,g,b)

to 

            if store: self.color = (r,g,b)

Thanks for the report and example.

JDH

I develop a web application  under  zope and I would like installing  
matplotlib under zope.Considering that the python of zope has no the 
entry in the register, how installing matplotlib?
thanks

Wow, I was going crazy trying to reproduce this. Your script works fine. Try 
this one:

from pylab import *

a=rand(100,100)
ax=axes()
ax.imshow(a)
ax.xaxis.set_visible(False)
ax.yaxis.set_visible(False)
ax.text(10,10,'white',color='w')
ax.text(10,20,'black',color='k')
savefig('temp.ps')


Both the x *and* y axes have to be set_visible(False) to reproduce the 
behavior.

Darren


On Sunday 20 February 2005 07:47 pm, John Hunter wrote:
> >>>>> "Darren" == Darren Dale <dd...@co...> writes:
>
>     Darren> I am displaying some images and overlaying a scalebar. The
>     Darren> image on the screen displays the requested white text and
>     Darren> bar, but they are black in the eps image.
>
> Hi Darren,
>
> I'm having trouble replicating the problem.  Make sure you have an
> up-to-date matplotlib.  For example, in CVS, I get the expected
> behavior in agg and ps with
>
>   from pylab import *
>   subplot(111, axisbg='red')
>   text(0.5, 3.5,'this is black', color=0)
>   text(0.5, 2.5,'this is black', color='k')
>   text(0.5, 1.5,'this is white', color=1)
>   text(0.5, 0.5,'this is white', color='w')
>   axis([0,2,0,4])
>   savefig('test.ps')
>   show()
>
> If you are still having troubles, please submit a complete test
> script, and the output of --verbose-helpful when you run it.
>
> Is it possible you have a pre 0.72 CVS?  There were a couple of bugs I
> introduced into PS CVS before the last release, but I think I cleaned
> them up.
>
> Hope this helps,
> JDH
>
>
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real users.
> Discover which products truly live up to the hype. Start reading now.
> http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users

-- 

Darren