matplotlib-devel Mailing List for matplotlib

On Monday, November 29, 2010, Uri Laserson <las...@mi...> wrote:
> Hi everyone,
> I have recently been introduced to the protovis visualization library, and I find aspects of it to be incredibly appealing for certain visualizations that don't fit some of the canonical chart types.
>
>
> http://vis.stanford.edu/protovis/
> Has anyone ever thought of porting the API from protovis and using the MPL backends to perform the rendering?  Is anyone interested in potentially working on a port?  I was thinking we'd initially start with the static functionality, and ultimately also port the interaction if there is enough interest.
>
>
> Uri
>
> ...................................................................................Uri Laserson
> Graduate Student, Biomedical Engineering
> Harvard-MIT Division of Health Sciences and Technology
> M +1 917 742 8019
> las...@mi...
>
>

Those charts are certainly interesting.  I haven't looked at the API
(JavaScript?!), but at the very least we could probably add a few new
graphs to our gallery.  Maybe use their gallery to inspire some new
chart types in mpl?

Ben Root

On Wednesday, November 17, 2010, Benjamin Root <ben...@ou...> wrote:
> On Tue, Nov 16, 2010 at 5:20 PM, J P <jp...@gm...> wrote:
>
>
> Hi all, here's my first patch for matplotlib. Someone noticed at Stack Overflow that the plot_surface function in mplot3d wasn't especially fast for a lot of points (and small rstrides/cstrides) and using shading and a single color. I found some parts of the code that weren't vectorized. These are my changes so far.
>
> Summary of changes:
> 1. Changed from double looping over aranges to using xrange
> 2. Made the normalization of the normals and their dot product with the vector [-1,-1,0.5] to find the shading a vectorized operation.
> 3. Changed a list comprehension which calculated the colors using an iterative approach to using the already built-in vectorization of the Normalization class and using the np.outer function. The result is a numpy array rather than a list which actually speeds up things down the line.
> 4. removed the corners array from plot_surface which wasn't ever used or returned. It didn't really slow things down, but I'm thinking that it is cruft.
>
> For change number two, I made a separate function that generates the shades, but feel free to move that around if you prefer.. or maybe it should be a function that begins with a _ because it shouldn't be used externally. These changes give varying levels of speed improvement depending on the number of points and the rstrides/cstrides arguments. With larger numbers of points and small rstrides/cstrides, these changes can more than halve the running time. I have found no difference in output after my changes.
>
> I know there is more work to be done within the plot_surface function and I'll submit more changes soon.
>
> Justin
>
>
> Justin,
>
> Thank you for your efforts to improve the performance of mplot3d.  I will take a look at the patches you have submitted and test them out.  What I am probably going to do is break down the patches into smaller pieces and incorporate them piece-by-piece.
>
> I tried refactoring plot_surface once before to mixed results.  The key feature I was trying to gain was compatibility with masked arrays.  I wanted to duplicate the behavior of contourf and pcolor where masked out portions of the surface would not be created.  I managed to get it to work for that particular use-case, but I broke a bunch of other use-cases along the way.  I am curious to see if your patches make this easier/harder to do.
>
> Thank you for your efforts and thank you for using matplotlib!
>
> Ben Root
>
>

I have looked through the patches, and there are definite
improvements.  I have broken the work into four separate patches.  The
first patch is essentially code cleanup and utilization of xrange
(plot_surface_cleanup.patch).  This patch can be back-ported without
concern (although it doesn't fix any bug per se).

The second patch does the vectorization of the shades.  The original
patch that was submitted had some edge cases, but I have found that
just simply converting that for-loop that creates the shades into a
list comprehension (and then pass into np.array) yielded almost
identical speedups without changing the current code path.  (Note: I
am a minimalist when it comes to patches).  This is in
plot_surface_vectshading.patch.

The third patch improves the calculation of the normals in
plot_surface by pre-allocating the arrays for calculating the vectors
and doing a final call to np.cross rather than appending on a list.  I
deviated slightly from the original patch by calling "which" as
"which_pt", adding a couple of comments, and also added an else
condition to create a "normal" with an empty list.  This last part is
to keep the code path as similar as it was before.  It was
theoretically possible to utilize a variable called normal elsewhere
without all the same conditions holding, so this guarantees that
normal exists, which was the case before.  This patch is
plot_surface_vectnorm.patch.

Finally, the fourth patch utilizes numpy array functionality for
calculating the vertexes.  This patch also forgoes the use of
transposed arrays. I took the original patch a step further and
eliminated the array transpose line earlier in the plot_surface
function.  The array transpose was not being properly utilized here,
and I saw no speed penalty/speedup either way, so in favor of simpler
code, I eliminated its use.  This patch is
plot_surface_vectvertex.patch.

Of the four patches, the speedups are in mostly found in the second
patch (100% speedup).  The first patch does provide noticeable
improvements.  There is also a slight improvement with the third
patch.  I am up in the air regarding speed improvements with the
fourth patch, but I wonder if there might be some memory improvements
here, or if any speedup is being hidden by the for-loop that the
vectorization is done in.

I will let these patches be mulled over before applying them.  Thanks
to JP for submitting the original patch.

Ben Root

As an aside: Related to the graph/network layout algorithms (which 
appears to be only a subset of protovis) is NetworkX, which can use 
matplotlib as a rendering backend.

http://networkx.lanl.gov/index.html

Mike

On 11/30/2010 12:20 AM, Uri Laserson wrote:
> Hi everyone,
>
> I have recently been introduced to the protovis visualization library, 
> and I find aspects of it to be incredibly appealing for certain 
> visualizations that don't fit some of the canonical chart types.
>
> http://vis.stanford.edu/protovis/
>
> Has anyone ever thought of porting the API from protovis and using the 
> MPL backends to perform the rendering?  Is anyone interested in 
> potentially working on a port?  I was thinking we'd initially start 
> with the static functionality, and ultimately also port the 
> interaction if there is enough interest.
>
> Uri
>
>
>
> ...................................................................................
> Uri Laserson
> Graduate Student, Biomedical Engineering
> Harvard-MIT Division of Health Sciences and Technology
> M +1 917 742 8019
> las...@mi... <mailto:las...@mi...>
>
>
> ------------------------------------------------------------------------------
> Increase Visibility of Your 3D Game App&  Earn a Chance To Win $500!
> Tap into the largest installed PC base&  get more eyes on your game by
> optimizing for Intel(R) Graphics Technology. Get started today with the
> Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs.
> http://p.sf.net/sfu/intelisp-dev2dev
>
>
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>    


-- 
Michael Droettboom
Science Software Branch
Space Telescope Science Institute
Baltimore, Maryland, USA

Hi everyone,

I have recently been introduced to the protovis visualization library, and I
find aspects of it to be incredibly appealing for certain visualizations
that don't fit some of the canonical chart types.

http://vis.stanford.edu/protovis/

Has anyone ever thought of porting the API from protovis and using the MPL
backends to perform the rendering?  Is anyone interested in potentially
working on a port?  I was thinking we'd initially start with the static
functionality, and ultimately also port the interaction if there is enough
interest.

Uri



...................................................................................
Uri Laserson
Graduate Student, Biomedical Engineering
Harvard-MIT Division of Health Sciences and Technology
M +1 917 742 8019
las...@mi...

On 22 November 2010 19:01, Benjamin Root <ben...@ou...> wrote:
> On Thu, Nov 18, 2010 at 8:58 AM, Scott Sinclair
> <sco...@gm...> wrote:
>>
>> It seems that the autofmt_xdate helper method is broken when twinx is
>> used.
>>
>> The attached patch fixes it for me and still seems to give the
>> advertised behaviour for single and vertically stacked subplots.
>
> I am not very familiar with this part of mpl, but your diff seems to cut out
> a few things.  In particular, the original code checks to see if the figure
> has a single axes object or more.  However, your code seems to cut this
> check out.  Now, it may have been that the check could have been
> unnecessary, but I am not sure.  I am curious as to your insight on this.

I'm not very familiar with the Matplotlib classes either, but I'll
explain my reasoning.

Anytime I create a figure with a single axes object the axes object
has an is_last_row method which always returns True. I don't see much
point in special casing, unless there's significant overhead that can
be avoided by doing so?

> In addition, the original code called "self.subplots_adjust(bottom=bottom)"
> only when all subplots were on the last row.  Now, it seems that it is
> always called no matter what.

The hasattr test is only checking that each axes object in the figure
has an is_last_row method. In the current code all_subplots is always
True unless the figure contains an axes object created with twinx or
twiny (in which case the autofmt_xdate method does nothing useful -
hence my patch).

Cheers,
Scott

On 22 November 2010 19:08, Benjamin Root <ben...@ou...> wrote:

>
> Confirmed.  Ian, would you mind filing a bug report on this at mpl's
> tracker?
>

I've added a bug report:
http://sourceforge.net/tracker/?func=detail&aid=3116341&group_id=80706&atid=560720

Thanks Ben,
Ian

On Mon, Nov 22, 2010 at 3:30 AM, Ian Thomas <ian...@gm...> wrote:

> I've found a bug in mplot3d's rendering of filled contours.  Attached is a
> simple test script to reproduce the error and an example of the output: the
> 2D plot on the left shows the desired result, the 3D plot on the right shows
> the holes in the contour are not rendered correctly.
>
> Digging around in axes3d.py and art3d.py, I think the problem is that
> mplot3d ignores the 'codes' that are produced by the 2D contourf routine.  I
> am probably not using the correct terminology, but I understand that these
> codes allow a polygon to be composed of multiple line loops, some of which
> may be holes within other line loops.  By ignoring the codes, the 3D routine
> renders a single polygon by concatenating all the points together regardless
> of whether they correspond to a hole or not.
>
> In revision 8806 of lib/mpl_toolkits/mplot3d/art3d.py, lines 136-7:
>
>    for (((x, y), code), z) in zip(pathsegs, zs):
>         seg.append((x, y, z))
>
> The codes are extracted from the pathsegs (which come from the 2D contourf
> routine), but then ignored.
>
> I've taken a look at trying to fix this, but it was not obvious to me how
> to propagate the 'codes' rendering functionality from the 2D to 3D source
> code.  Perhaps someone more familiar with mplot3d could look at it.
>
> Thanks,
> Ian
>
>
Confirmed.  Ian, would you mind filing a bug report on this at mpl's
tracker?

http://sourceforge.net/tracker/?atid=560720&group_id=80706&func=browse

Thank you very much for your insightful analysis as well.  It will be
helpful in finding a fix for this.

Ben Root

On Thu, Nov 18, 2010 at 8:58 AM, Scott Sinclair <sco...@gm...
> wrote:

> Hi,
>
> It seems that the autofmt_xdate helper method is broken when twinx is
> used. Consider the script below:
>
> -----------------------------------------
> import datetime as dt
>
> import numpy as np
> import matplotlib.pyplot as plt
> from matplotlib.dates import date2num
>
> strt = dt.datetime(2000, 3, 15, 6)
> delta = dt.timedelta(hours=6)
> date_list = [(strt + i*delta) for i in range(100)]
>
> x = date2num(date_list)
> y = np.sin(x)
> z = np.cos(x)
>
> fig, ax1 = plt.subplots()
>
> ax1.plot(date_list, y, 'b-')
>
> ax2 = ax1.twinx()
> ax2.plot(date_list, z, 'r-')
>
> # using the auto format method doesn't work
> fig.autofmt_xdate()
>
> plt.show()
> -----------------------------------------
>
> This is because the 'is_last_row' attribute isn't present on ax2 and
> len(fig.axes) != 1 when the autofmt_xdate method is called on fig.
>
> The attached patch fixes it for me and still seems to give the
> advertised behaviour for single and vertically stacked subplots.
>
> Cheers,
> Scott
>
>
I am not very familiar with this part of mpl, but your diff seems to cut out
a few things.  In particular, the original code checks to see if the figure
has a single axes object or more.  However, your code seems to cut this
check out.  Now, it may have been that the check could have been
unnecessary, but I am not sure.  I am curious as to your insight on this.

In addition, the original code called "self.subplots_adjust(bottom=bottom)"
only when all subplots were on the last row.  Now, it seems that it is
always called no matter what.

Ben Root

I've found a bug in mplot3d's rendering of filled contours.  Attached is a
simple test script to reproduce the error and an example of the output: the
2D plot on the left shows the desired result, the 3D plot on the right shows
the holes in the contour are not rendered correctly.

Digging around in axes3d.py and art3d.py, I think the problem is that
mplot3d ignores the 'codes' that are produced by the 2D contourf routine.  I
am probably not using the correct terminology, but I understand that these
codes allow a polygon to be composed of multiple line loops, some of which
may be holes within other line loops.  By ignoring the codes, the 3D routine
renders a single polygon by concatenating all the points together regardless
of whether they correspond to a hole or not.

In revision 8806 of lib/mpl_toolkits/mplot3d/art3d.py, lines 136-7:

   for (((x, y), code), z) in zip(pathsegs, zs):
        seg.append((x, y, z))

The codes are extracted from the pathsegs (which come from the 2D contourf
routine), but then ignored.

I've taken a look at trying to fix this, but it was not obvious to me how to
propagate the 'codes' rendering functionality from the 2D to 3D source
code.  Perhaps someone more familiar with mplot3d could look at it.

Thanks,
Ian

On Fri, Nov 19, 2010 at 5:08 PM, Christopher Barker
<Chr...@no...> wrote:
> "interactive" means that you are typing things interactively at the
> command prompt, and thus want to see changes you make to a figure after
> each change -- exactly the opposite of what you want.
>
> "non-interactive" means that you want to be able to do a number of
> manipulations to a figure, and only show the result of those changes
> when you specifically ask to see them. This is kind of similar to what
> you want, but till really talking about behavior with a GUI back-end.

Thanks, this is a very helpful clarification.

~kj

On Fri, Nov 19, 2010 at 10:15 AM, John Hunter <jd...@gm...> wrote:
> On Fri, Nov 19, 2010 at 9:10 AM, Kynn Jones <ky...@gm...> wrote:
>> On Wed, Nov 17, 2010 at 6:30 PM, Michiel de Hoon <mjl...@ya...> wrote:
>>> In principle the non-interactive mode can also be implemented in this
>>> backend. If that would solve your problem, could you open a bug report for
>>> it?
>> How???  I feel so inept: I can't even find matplotlib's bug
>> reporting/tracking page!  Is there one?  If so, it is well hidden.  All I
>> was able to find was
>> http://matplotlib.sourceforge.net/faq/troubleshooting_faq.html?highlight=bug#reporting-problems
> Take a look at the home page http://matplotlib.sourceforge.net on the right
> panel there is a box with the title "Need help?" which says :
>
>   You can file bugs, patches and feature requests on the sourceforge tracker,
>   but it is a good idea to ping us on the mailing list too.

Thanks!  I added the bug report:

https://sourceforge.net/tracker/?func=detail&aid=3113191&group_id=80706&atid=560720

~kj

> In other words, with this simulation in place, one should be able to
> create graphical objects, translate them, scale them, shear them,
> recombine them, split them up, interrogate them, etc., and finally
> save these objects to files, without a window ever popping up.  In
> fact, this code should run perfectly well on a terminal without any
> graphical capabilities at all.

This is exactly what the non-gui back-ends are for (agg in particular).

While it would be nice to be able to do all that with a GUI back-end, 
the point of the GUI is to give you a GUI -- if you don't need one, 
don't use one.

> Incidentally, one of the reasons for my difficulties with using
> matplotlib is 100% conceptual.  I just can't wrap my head around the
> idea of needing to implement a "non-interactive" mode.  (Actually, I
> to call it "non-GUI", since it's perfectly possible to envision an
> interaction that is entirely text-based.)

non-GUI is what the Agg, etc back-ends are for. non-interactive is a 
different concept -- it's specifically a subset of GUI behavior:

"interactive" means that you are typing things interactively at the 
command prompt, and thus want to see changes you make to a figure after 
each change -- exactly the opposite of what you want.

"non-interactive" means that you want to be able to do a number of 
manipulations to a figure, and only show the result of those changes 
when you specifically ask to see them. This is kind of similar to what 
you want, but till really talking about behavior with a GUI back-end.

non-GUI back ends are inherently non-interactive.

I'm not totally sure about how MPL works, but it may need to query teh 
back-end occasinally when doing layout, etc. That may require the GUI 
toolkit to be initialized even before anyactual rendering is done. If 
that's the case, then you simpily never want to use a GUI back-end when 
you don't have/need a GUI (text terminal, web application, batch 
scripts, etc).

-Chris

On Fri, Nov 19, 2010 at 9:19 AM, John Hunter <jd...@gm...> wrote:

> On Fri, Nov 19, 2010 at 9:10 AM, Kynn Jones <ky...@gm...> wrote:
> >
> >
> > When I read your message about implementing the non-GUI mode, it turns
> this simple picture completely on its head, which tells me that matplotlib's
> architecture is, at this point, beyond my comprehension.  One of the reasons
> for looking forward to your implementation of the non-GUI mode for the
> MacOSX backend is that, by studying a diff between this enhanced version and
> the previous version I may be able to finally understand matplotlib's basic
> architecture.
> >
>
> >From the introduction: http://matplotlib.sourceforge.net/users/intro.html
> ::
>
>  The matplotlib code is conceptually divided into three parts: the
>  pylab interface is the set of functions provided by matplotlib.pylab
>  which allow the user to create plots with code quite similar to
>  MATLAB figure generating code (Pyplot tutorial). The matplotlib
>  frontend or matplotlib API is the set of classes that do the heavy
>  lifting, creating and managing figures, text, lines, plots and so on
>  (Artist tutorial). This is an abstract interface that knows nothing
>  about output. The backends are device dependent drawing devices, aka
>  renderers, that transform the frontend representation to hardcopy or
>  a display device (What is a backend?). Example backends: PS creates
>  PostScript? hardcopy, SVG creates Scalable Vector Graphics hardcopy,
>  Agg creates PNG output using the high quality Anti-Grain Geometry
>  library that ships with matplotlib, GTK embeds matplotlib in a Gtk+
>  application, GTKAgg uses the Anti-Grain renderer to create a figure
>  and embed it a Gtk+ application, and so on for PDF, WxWidgets,
>  Tkinter etc.
>
>  matplotlib is used by many people in many different contexts. Some
>  people want to automatically generate PostScript files to send to a
>  printer or publishers. Others deploy matplotlib on a web application
>  server to generate PNG output for inclusion in dynamically-generated
>  web pages. Some use matplotlib interactively from the Python shell
>  in Tkinter on Windows?. My primary use is to embed matplotlib in a
>  Gtk+ EEG application that runs on Windows, Linux and Macintosh OS X.
>
>
>
> Elaborating a little bit: the middle part of matplotlib, the artist
> hierarchy, is what I think you are looking for.  That is where
> abstractions like Line2D, Circle and Text live.  There is no concept
> of a GUI or a render at that abstraction.
>
> http://matplotlib.sourceforge.net/users/artists.html#artist-tutorial
>
>
Exactly, this is the part of matplotlib that Kynn should focus on -- as a
hierarchy of artist objects.  I do think that Kynn would have a somewhat
valid argument with regards to how matplotlib is presented, though.  For
example, the pylab interface is completely "magical" and utterly bypasses
any object-oriented-ness.  Then, the pyplot interface (which I use most of
the time) provides a more OOP-esque interface.  However, the documentation
and examples for the plotting functions do not emphasize the fact that they
return graphing objects, mostly because they get automatically attached to
the axes object and it is unnecessary to do anything with those objects.
Usually, users just let the functions return those objects without an
assignment to anywhere.

While I think pyplot strikes the right balance (for me), I often wonder if
the way this interface is presented causes a logical hinderance to fully
understanding the final layer of the graphical artist objects and
collections.  Because the underlying graphical objects are typically ignored
by the users and handled automatically, this layer doesn't receive as much
developer attention to make it more versatile and complete.

Just my 2 cents on this issue.  Actual value of the thought varies based
upon the current value of the dollar in your area...

Ben Root

On Fri, Nov 19, 2010 at 9:10 AM, Kynn Jones <ky...@gm...> wrote:
>
>
> When I read your message about implementing the non-GUI mode, it turns this simple picture completely on its head, which tells me that matplotlib's architecture is, at this point, beyond my comprehension.  One of the reasons for looking forward to your implementation of the non-GUI mode for the MacOSX backend is that, by studying a diff between this enhanced version and the previous version I may be able to finally understand matplotlib's basic architecture.
>

>From the introduction: http://matplotlib.sourceforge.net/users/intro.html::

  The matplotlib code is conceptually divided into three parts: the
  pylab interface is the set of functions provided by matplotlib.pylab
  which allow the user to create plots with code quite similar to
  MATLAB figure generating code (Pyplot tutorial). The matplotlib
  frontend or matplotlib API is the set of classes that do the heavy
  lifting, creating and managing figures, text, lines, plots and so on
  (Artist tutorial). This is an abstract interface that knows nothing
  about output. The backends are device dependent drawing devices, aka
  renderers, that transform the frontend representation to hardcopy or
  a display device (What is a backend?). Example backends: PS creates
  PostScript? hardcopy, SVG creates Scalable Vector Graphics hardcopy,
  Agg creates PNG output using the high quality Anti-Grain Geometry
  library that ships with matplotlib, GTK embeds matplotlib in a Gtk+
  application, GTKAgg uses the Anti-Grain renderer to create a figure
  and embed it a Gtk+ application, and so on for PDF, WxWidgets,
  Tkinter etc.

  matplotlib is used by many people in many different contexts. Some
  people want to automatically generate PostScript files to send to a
  printer or publishers. Others deploy matplotlib on a web application
  server to generate PNG output for inclusion in dynamically-generated
  web pages. Some use matplotlib interactively from the Python shell
  in Tkinter on Windows?. My primary use is to embed matplotlib in a
  Gtk+ EEG application that runs on Windows, Linux and Macintosh OS X.



Elaborating a little bit: the middle part of matplotlib, the artist
hierarchy, is what I think you are looking for.  That is where
abstractions like Line2D, Circle and Text live.  There is no concept
of a GUI or a render at that abstraction.

http://matplotlib.sourceforge.net/users/artists.html#artist-tutorial

On Fri, Nov 19, 2010 at 9:10 AM, Kynn Jones <ky...@gm...> wrote:

>  On Wed, Nov 17, 2010 at 6:30 PM, Michiel de Hoon <mjl...@ya...>wrote:
>
>> In principle the non-interactive mode can also be implemented in this
>> backend. If that would solve your problem, could you open a bug report for
>> it?
>>
>>
> How???  I feel so inept: I can't even find matplotlib's bug
> reporting/tracking page!  Is there one?  If so, it is well hidden.  All I
> was able to find was
>
>
> http://matplotlib.sourceforge.net/faq/troubleshooting_faq.html?highlight=bug#reporting-problems
>
>

Take a look at the home page http://matplotlib.sourceforge.net on the right
panel there is a box with the title "Need help?" which says :

  You can file bugs, patches and feature requests on the sourceforge
tracker<http://sourceforge.net/tracker2/?group_id=80706>,

  but it is a good idea to ping us on the mailing list too.

Sorry this has been so difficult and confusing.  I think you will save
yourself a lot of pain by using the Enthought Python Distribution I pointed
you too.  There is a free academic version as well as a trial version.

JDH

(The quoting got screwed in my previous post, so I'm re-posting it.
Sorry about that.)

On Wed, Nov 17, 2010 at 6:30 PM, Michiel de Hoon <mjl...@ya...> wrote:
> In principle the non-interactive mode can also be implemented in this backend. If that would solve your problem, could you open
> a bug report for it?

How???  I feel so inept: I can't even find matplotlib's bug
reporting/tracking page!  Is there one?  If so, it is well hidden.
All I was able to find was

http://matplotlib.sourceforge.net/faq/troubleshooting_faq.html?highlight=bug#reporting-problems

which basically says to send an email to the matplotlib list.  But
didn't I just do this already?  I'm confused.


On Thu, Nov 18, 2010 at 8:06 PM, Michiel de Hoon <mjl...@ya...> wrote:
> If you use the MacOSX backend, you won't need pygtk, pycairo, pyqt, pygobject, wx, tcl/tk, or anything they depend on, which
> was my main motivation for writing this backend. Other than the fact that the MacOSX backend currently does not support the
> non-interactive mode, it should work for you, doesn't it?



Actually, getting the non-interactive behavior was the reason I was
trying to install a backend different from the MacOSX one.  As I
described in an earlier post, I want to simulate a perfect separation
between the creation and manipulation of graphic objects, and their
output (either in a graphical display or by saving them to a file).
In other words, with this simulation in place, one should be able to
create graphical objects, translate them, scale them, shear them,
recombine them, split them up, interrogate them, etc., and finally
save these objects to files, without a window ever popping up.  In
fact, this code should run perfectly well on a terminal without any
graphical capabilities at all.

Incidentally, one of the reasons for my difficulties with using
matplotlib is 100% conceptual.  I just can't wrap my head around the
idea of needing to implement a "non-interactive" mode.  (Actually, I
to call it "non-GUI", since it's perfectly possible to envision an
interaction that is entirely text-based.)  In my brain, such "non-GUI"
mode is the default behavior, because, as I described above, the
creation and manipulation of graphic objects logically precedes and is
conceptually independent of their output.  Therefore, the absolutely
simplest system for dealing with such graphic objects would have no
output at all: the user would (either through a script, or
interactively, through a text-based interface) create and manipulate
graphics objects that live in memory and disappear when the
process/session is terminated.  Of course, the usefulness of such a
system would be very limited, but it would certainly be my first step.
 Only after that I would implement the various ways to output the
graphical objects.  This is the only ordering of capabilities that
makes sense to me.

When I read your message about implementing the non-GUI mode, it turns
this simple picture completely on its head, which tells me that
matplotlib's architecture is, at this point, beyond my comprehension.
One of the reasons for looking forward to your implementation of the
non-GUI mode for the MacOSX backend is that, by studying a diff
between this enhanced version and the previous version I may be able
to finally understand matplotlib's basic architecture.

~kj

 On Wed, Nov 17, 2010 at 6:30 PM, Michiel de Hoon <mjl...@ya...>wrote:

> In principle the non-interactive mode can also be implemented in this
> backend. If that would solve your problem, could you open a bug report for
> it?
>
>
How???  I feel so inept: I can't even find matplotlib's bug
reporting/tracking page!  Is there one?  If so, it is well hidden.  All I
was able to find was

http://matplotlib.sourceforge.net/faq/troubleshooting_faq.html?highlight=bug#reporting-problems

which basically says to send an email to the matplotlib list.  But didn't I
just do this already?  I'm confused.


On Thu, Nov 18, 2010 at 8:06 PM, Michiel de Hoon <mjl...@ya...>wrote:


  If you use the MacOSX backend, you won't need pygtk, pycairo, pyqt,
pygobject, wx, tcl/tk, or anything they depend on, which was my main
motivation for writing this backend. Other than the fact that the MacOSX
backend currently does not support the non-interactive mode, it should work
for you, doesn't it?



Actually, getting the non-interactive behavior was the reason I was trying
to install a backend different from the MacOSX one.  As I described in an
earlier post, I want to simulate a perfect separation between the creation
and manipulation of graphic objects, and their output (either in a graphical
display or by saving them to a file).  In other words, with this simulation
in place, one should be able to create graphical objects, translate them,
scale them, shear them, recombine them, split them up, interrogate them,
etc., and finally save these objects to files, without a window ever popping
up.  In fact, this code should run perfectly well on a terminal without
any graphical capabilities at all.

Incidentally, one of the reasons for my difficulties with using matplotlib
is 100% conceptual.  I just can't wrap my head around the idea of needing to
implement a "non-interactive" mode.  (Actually, I to call it "non-GUI",
since it's perfectly possible to envision an interaction that is entirely
text-based.)  In my brain, such "non-GUI" mode is the default behavior,
because, as I described above, the creation and manipulation of graphic
objects logically precedes and is conceptually independent of their output.
Therefore, the absolutely simplest system for dealing with such graphic
objects would have no output at all: the user would (either through a
script, or interactively, through a text-based interface) create and
manipulate graphics objects that live in memory and disappear when the
process/session is terminated.  Of course, the usefulness of such a system
would be very limited, but it would certainly be my first step.  Only after
that I would implement the various ways to output the graphical objects.
This is the only ordering of capabilities that makes sense to me.

When I read your message about implementing the non-GUI mode, it turns this
simple picture completely on its head, which tells me that matplotlib's
architecture is, at this point, beyond my comprehension.  One of the reasons
for looking forward to your implementation of the non-GUI mode for the
MacOSX backend is that, by studying a diff between this enhanced version and
the previous version I may be able to finally understand matplotlib's basic
architecture.

~kj

If you use the MacOSX backend, you won't need pygtk, pycairo, pyqt, pygobject, wx, tcl/tk, or anything they depend on, which was my main motivation for writing this backend. Other than the fact that the MacOSX backend currently does not support the non-interactive mode, it should work for you, doesn't it?

--Michiel.

--- On Thu, 11/18/10, Kynn Jones <ky...@gm...> wrote:

From: Kynn Jones <ky...@gm...>
Subject: Re: [matplotlib-devel] How to decouple non-GUI stuff from the GUI?
To: mat...@li...
Date: Thursday, November 18, 2010, 5:11 PM

On Wed, Nov 17, 2010 at 6:33 PM, Benjamin Root <ben...@ou...> wrote:


As an additional note, if you are having difficulty compiling for MacOS X, why not just ask for help with that?

Just to keep from ranting like a lunatic, basically.  The experience was traumatic enough to shake my faith in Python altogether, and made me miss good ol' CPAN.

To summarize 2-3 day's worth of frustration:
1.
GTK, GTKAgg:ImportError: Gtk* backend requires pygtk to be installed.

GTKCairo:ImportError: No module named backend_gtkcairo
FltkAgg:ImportError: No module named fltk
QtAgg:ImportError: Qt backend requires pyqt to be installed.

Qt4Agg:ImportError: Warning: formlayout requires PyQt4 >v4.3
WX, WXAgg:ImportError: Matplotlib backend_wx and backend_wxagg require wxPython >=2.8


2. configuration of pygtk fails:
checking for PYGOBJECT... configure: error: Package requirements (pygobject-2.0 >= 2.21.3) were not met:

No package 'pygobject-2.0' found

3. configuration of pygobject fails:
checking for PYCAIRO... configure: error: Package requirements (pycairo >= 1.0.2
        ) were not met:
No package 'pycairo' found

4. installation of pycairo fails (after > 6 hours of trying a variety of approaches); currently, importing cairo produces a fatal error:

% pythonPython 2.6.1 (r261:67515, Feb 11 2010, 00:51:29) [GCC 4.2.1 (Apple Inc. build 5646)] on darwinType "help", "copyright", "credits" or "license" for more information.
>>> import cairoFatal Python error: Interpreter not initialized (version mismatch?)zsh: abort      python

By this point, I just gave up on getting any more interactive backends for matplotlib.  (I reiterate that the above is a barebone's summary.  There's a lot that I'm omitting.)

As an aside, I have never in my life seen a worse installation system than pycairo's.  If you have not examined it in detail, please do so before reacting to my comments.  In particular look at the waf file, and the files it invokes.  Also, when you find the latter, do a global search for the number 1337 to get a whiff pycairo's excremental aroma.

There is no point in choosing Python as one's programming language if this means that one depends on garbage like pycairo.  I think it is unfair to put unsuspecting users of matplotlib through the ordeal of dealing with pycairo's installation.  (And pygtk's and pygobjects's are not much better.)  We can't force people to write good Python, but at the very least we should not propagate bad code.  Packages like pycairo, pygtk, and pygobjects are substandard, and they should be ostracized by the rest of the Python community until they shape up.

Even putting pycairo aside, the Python package installation system is a disaster.  The problems I've summarized on this page are par for the course with Python packages.  Just getting matplotlib to install (even without most interactive backends) was also a multi-day nightmare...

You see?  I'm ranting like a lunatic.  Sorry.  I'll stop.
~kj


-----Inline Attachment Follows-----

------------------------------------------------------------------------------
Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and DOM L2 & L3.
Spend less time writing and  rewriting code and more time creating great
experiences on the web. Be a part of the beta today
http://p.sf.net/sfu/msIE9-sfdev2dev
-----Inline Attachment Follows-----

_______________________________________________
Matplotlib-devel mailing list
Mat...@li...
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel



      

On 11/18/10 3:08 PM, John Hunter wrote:
> Yes, installing these GUIs on OSX is a mess, particularly GTK.  I only
> recommend it for the very brave and foolish.

yup -- though wxPython is trivial.

Do make sure you're running the python.org python, though -- that's the 
one most likely to be supported by various binary packages.

>  If you use the enthought
> python distribution for OSX, you should get a working tkagg, qt4agg,
> wxagg and macosx (I think)

I don't know if enthought is worth it for that -- but it sure is if you 
want VTK. Mayavi, and a few other really ugly packages!

As for the wonders of CPAN:

I've never been a perl user, but the issues with packages like 
matplotlib on OS-X are due to compiled code that depends on libraries 
that aren't delivered with the OS, on an OS that can have up to four 
diffferent hardware architectures, and a pretty rapidly changing API (or 
ABI anyway). Oh, and it depends on various GUI toolkits that have those 
same issues.

I can't imagine how a CPAN-like system would help with that at all. 
Installing pure python packages isn't a problem at all.

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

On Thu, Nov 18, 2010 at 5:57 PM, Benjamin Root <ben...@ou...> wrote:

> I think we are more asking what tutorials have you read to help install
> matplotlib (some are better than others)


This was about 6 weeks ago, and I no longer remember the details.  I do
remember I read *a lot* of advice on installing matplotlib (and scipy,
ipython, etc.), not always consistent.


> In addition, what version of MacOS X are you using and if you are using the
> built-in python interpreater or installed your own?


I'm using 64-bit Snow Leopard.  My python is /usr/bin/python, which came
with the Snow Leopard installation.


> Did you try the DMG file, or from macports, or from pypi?
>

Don't recall what I finally ended up with, but I do know that I had to
completely scrap the macports install, because my macports got totally
wedged (circular dependencies, etc.).  I had to uninstall *all* my macports,
and reinstall them from scratch.  On this re-install I did not install any
Python ports, although python2.6 and a few other python ports were pulled in
as dependencies of other ports.  Still, even though I have
/opt/local/bin/python2.6, I don't use it.

What we really need is improved documentation for the mac install process.
>

I beg to differ: what we need is a single *official* download and
installation procedure.  Of course, this in turn requires a similarly
unitary-no-two-ways-about-it download and installation procedure for
everything else that matplotlib depends on, directly or indirectly.  Without
this, adequate documentation becomes impossible, because there's no way that
the writers of the documentation can possibly test the combinatorial
explosion of possibilities.

It's bad enough that we have to deal with multiple OSs and architectures.
 We don't need to pile 31 installation flavors on top of that.

(What really boggles the mind is that of all languages out there it is
Python that finds itself in this unholy mess, while wild-eyed TIMTOWTDI Perl
hums along with CPAN.  I can count with the fingers of one hand (and have a
few left over) the number of Perl packages that I use that were not
downloaded from CPAN.  In contrast, tons of my Python modules come from
random places: SourceForge, Google Code, github, individual authors'
websites, you name it.  How is this possible?  Whatever happened to Python's
"There should be one-- and preferably only one --obvious way to do it"???)

So, let's start from the beginning and we can help you though.


Your patience puts me to shame.  Thank you for the offer.  I suppose that I
should first clean the slate, and re-install everything?  Is there clean way
to do this?

~kj

On Thu, Nov 18, 2010 at 4:11 PM, Kynn Jones <ky...@gm...> wrote:
> On Wed, Nov 17, 2010 at 6:33 PM, Benjamin Root <ben...@ou...> wrote:
>
>> As an additional note, if you are having difficulty compiling for MacOS X,
>> why not just ask for help with that?
>
> Just to keep from ranting like a lunatic, basically.  The experience was
> traumatic enough to shake my faith in Python altogether, and made me miss
> good ol' CPAN.
> To summarize 2-3 day's worth of frustration:

Yes, installing these GUIs on OSX is a mess, particularly GTK.  I only
recommend it for the very brave and foolish.  If you use the enthought
python distribution for OSX, you should get a working tkagg, qt4agg,
wxagg and macosx (I think)

http://www.enthought.com/products/epd.php

JDH

I think we are more asking what tutorials have you read to help install
matplotlib (some are better than others), as well as asking what was the
source of your matplotlib installation.  In addition, what version of MacOS
X are you using and if you are using the built-in python interpreater or
installed your own?  Did you try the DMG file, or from macports, or from
pypi?

There are a variety of issues depending on your Mac system that needs to be
sorted out to determine the best way to go about installing everything.  The
particular sticking point is that Apple supplied their own interperater
rather than the standard python interpreater.  Unfortunately, this causes
problems with numpy (and thus matplotlib).  There are other pitfalls that
makes things tricky.

Personally, I found using macports to be friendly (although it takes forever
because it builds ATLAS...) on my wife's Snow Leopard (32-bit) computer.
Friedrich Romstedt is working on a new dmg build for numpy and matplotlib, I
believe, and some of use have some school of hard-knocks experience to help
with specific questions.

What we really need is improved documentation for the mac install process.

As for the python packager... tell us something we don't know.  I have heard
of some people working on an improved system and is testing it out on numpy,
I believe (again, my memory is hazy here).  We all know the difficulties of
the packaging system, that's why we are more than willing to help you out.
So, let's start from the beginning and we can help you though.  It is
possible!

Ben Root

On Wed, Nov 17, 2010 at 6:33 PM, Benjamin Root <ben...@ou...> wrote:

As an additional note, if you are having difficulty compiling for MacOS X,
> why not just ask for help with that?
>


Just to keep from ranting like a lunatic, basically.  The experience was
traumatic enough to shake my faith in Python altogether, and made me miss
good ol' CPAN.

To summarize 2-3 day's worth of frustration:

1.

GTK, GTKAgg:
ImportError: Gtk* backend requires pygtk to be installed.

GTKCairo:
ImportError: No module named backend_gtkcairo

FltkAgg:
ImportError: No module named fltk

QtAgg:
ImportError: Qt backend requires pyqt to be installed.

Qt4Agg:
ImportError: Warning: formlayout requires PyQt4 >v4.3

WX, WXAgg:
ImportError: Matplotlib backend_wx and backend_wxagg require wxPython >=2.8


2. configuration of pygtk fails:

checking for PYGOBJECT... configure: error: Package requirements
(pygobject-2.0 >= 2.21.3) were not met:

No package 'pygobject-2.0' found


3. configuration of pygobject fails:

checking for PYCAIRO... configure: error: Package requirements (pycairo >=
1.0.2
        ) were not met:

No package 'pycairo' found


4. installation of pycairo fails (after > 6 hours of trying a variety of
approaches); currently, importing cairo produces a fatal error:

% python
Python 2.6.1 (r261:67515, Feb 11 2010, 00:51:29)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import cairo
Fatal Python error: Interpreter not initialized (version mismatch?)
zsh: abort      python


By this point, I just gave up on getting any more interactive backends for
matplotlib.  (I reiterate that the above is a barebone's summary.  There's a
lot that I'm omitting.)

As an aside, I have never in my life seen a worse installation system than
pycairo's.  If you have not examined it in detail, please do so before
reacting to my comments.  In particular look at the waf file, and the files
it invokes.  Also, when you find the latter, do a global search for the
number 1337 to get a whiff pycairo's excremental aroma.

There is no point in choosing Python as one's programming language if this
means that one depends on garbage like pycairo.  I think it is unfair to put
unsuspecting users of matplotlib through the ordeal of dealing with
pycairo's installation.  (And pygtk's and pygobjects's are not much better.)
 We can't force people to write good Python, but at the very least we should
not propagate bad code.  Packages like pycairo, pygtk, and pygobjects are
substandard, and they should be ostracized by the rest of the Python
community until they shape up.

Even putting pycairo aside, the Python package installation system is a
disaster.  The problems I've summarized on this page are par for the course
with Python packages.  Just getting matplotlib to install (even without most
interactive backends) was also a multi-day nightmare...

You see?  I'm ranting like a lunatic.  Sorry.  I'll stop.

~kj

Hi,

It seems that the autofmt_xdate helper method is broken when twinx is
used. Consider the script below:

-----------------------------------------
import datetime as dt

import numpy as np
import matplotlib.pyplot as plt
from matplotlib.dates import date2num

strt = dt.datetime(2000, 3, 15, 6)
delta = dt.timedelta(hours=6)
date_list = [(strt + i*delta) for i in range(100)]

x = date2num(date_list)
y = np.sin(x)
z = np.cos(x)

fig, ax1 = plt.subplots()

ax1.plot(date_list, y, 'b-')

ax2 = ax1.twinx()
ax2.plot(date_list, z, 'r-')

# using the auto format method doesn't work
fig.autofmt_xdate()

plt.show()
-----------------------------------------

This is because the 'is_last_row' attribute isn't present on ax2 and
len(fig.axes) != 1 when the autofmt_xdate method is called on fig.

The attached patch fixes it for me and still seems to give the
advertised behaviour for single and vertically stacked subplots.

Cheers,
Scott

> As an additional note, if you are having difficulty compiling for 
> MacOS X, why not just ask for help with that?

yup -- there are some issues with which Tk is used by tkInter, but wx 
should be easy -- how have you tried to install?

-Chris