matplotlib-devel Mailing List for matplotlib

> 
> Nicolas,
> 
> I would like to add to Mike's comments by making it clear why I am interested in having OpenGL for matplotlib.  It is not for speed.  The last time I checked, a good portion of time is actually spent on the axis tickers and automatic limits.  So, if speed is your goal, then that might be the better place to look.
> 
> No, my main interest in OpenGL is to relax matplotlib's 2D limitation.  At the core of matplotlib is a 2D layering engine that was perfectly suited for 2D graphing.  mplot3d then takes advantage of that layering feature to hack-on 3D functionality.  However, as has been noted many, many times, this does not always produce correct results.  I would love for the 2D layering core to have some sort of logic where if the OpenGL backend is in use, and if 3D art objects are detected, then bypass the dimension reduction that is done in the layering engine and send the full 3D data over to OpenGL and let it figure out the correct 2D projection.
> 
> However, I am not an OpenGL programmer and I wouldn't even know where to start.  That is why I am very happy to see someone give this another try.  Also, as a side note, in discussions with Ryan May (who did the previous attempt at using pyglet), the main reason he discontinued development at that time was because the text rendering in pyglet was very poor and not anti-aliased.  Do you have any indication if that has improved or not?
> 
> Ben Root

As I just posted, I will try to complete the GL backend (using standard GL and GLUT). For the font rendering, I think freetype-py can be used to offer decent rendering with antialiasing (see http://code.google.com/p/freetype-py/, all examples rendered without LCD filtering). But I do not know if it can compares with agg text rendering.
If not, I could cache glyphs produced by the agg renderer and just use them.


Nicolas

Hi,

I will be on vacation with limited email from July 14 to August 7, 2011.

Bonjour,

Je serai en conge du 14 juillet jusqu'au 7 aout, 2011.

I'm sorry to have used the term "useless" since it was not what I meant. In fact, after profiling the simple draw example, I realized there were a lot of necessary operations that were not quite visible from my "external" point of view. I was just thinking "why the hell is it so slow to draw some lines and ticks". I think you explained very clearly the actual situation. Also, I've been coded glumpy that mix matplotlib and OpenGL (for imshow operations only) quite efficiently and I thought something similar could be done for regular plot. Anyway, I will try to code further the GL backend without paying too much attention to speed right now.

Thanks a lot for all the explanations.


Nicolas


On Jul 29, 2011, at 6:05 PM, Michael Droettboom wrote:

> Nicolas,
> 
> I'm not sure how you've reached your conclusion.
> 
> The 21,000 calls to Line2D.draw (i.e. 21 per frame) are easily explained since each grid line (or tick) is in fact a line.
> 
> The 10,000 calls to Tick.draw (i.e. 10 per frame) are because there are 10 tick labels.
> 
> likewise for Text.draw etc. and on down the list.
> 
> I don't see how any of these objects could be drawn without calling draw. ;)  There actually already is caching that occurs rendering text, for example.  None of these calls are strictly "useless".
> 
> If animation speed is important, there are tricks in the examples/animation directory that show how to reuse the ticks from a previous draw and only update the data (i.e. eliminate many of these "useless" draw calls).  But they are tricks -- they don't work for the general case of "anything in this plot may change at any time".
> 
> But I think you are comparing apples to oranges in your speed comparison.  You say a "do-nothing" loop in PyOpenGL runs at around 2000 fps, and a "do-nothing" loop in matplotlib runs about 100fps.  But, of course, the matplotlib test case is actually doing a great deal even with the backend doing nothing -- it is doing all of the work of laying out the plot, which is the majority of time spent getting a plot to the screen.  And that all work happens in Python -- its speed is what it is and is acceptable in many contexts -- but no backend work is going to improve what's going on in the core.
> 
> I should warn you that previous attempts to speed up matplotlib using hardware acceleration have failed to produce much fruit because the backends actually do fairly little work by design in matplotlib.  The actual act of rendering paths into pixels on screen (i.e. what happens in the backend) is a small fraction of the run time, even when using a software renderer (eg. Agg).  Here's a useful benchmark that renders a plot a bunch of times to memory:
> 
>   import sys
>   import matplotlib
>   matplotlib.use(sys.argv[-1])
>   from pylab import *
>   import numpy
>   plot(numpy.arange(int(sys.argv[-2])))
>   for i in xrange(1000):
>       draw()
> 
> and the results (on my 2.33GHz Intel E6550):
> 
> > time python test_backend_speed.py 3 agg
> 
> real    0m15.211s
> user    0m15.009s
> sys     0m0.136s
> 
> # Here "backend_pyglet" is the do-nothing backend you sent to the list in a previous e-mail
> > time python test_backend_speed.py 3 module://backend_pyglet
> 
> real    0m14.038s
> user    0m13.713s
> sys     0m0.256s
> 
> > time python test_backend_speed.py 100 agg
> 
> real    0m23.038s
> user    0m22.845s
> sys     0m0.132s
> 
> > time python test_backend_speed.py 100 module://backend_pyglet
> 
> real    0m15.251s
> user    0m14.837s
> sys     0m0.304s
> 
> So you see the actual work in the backend can be a fairly small fraction of the total runtime -- that gives one an idea of the upper bound on the speed improvement that any backend could make without digging into the matplotlib core and making improvements there.
> 
> In fairness, my test is not measuring the time to (once rendering the plot) blit it to the screen.  I suspect OpenGL will have an advantage there.  It may even be possible as a mid-way solution to create an Agg/OpenGL backend that used Agg for rendering and OpenGL -- that's something that would be really useful just to have another nice cross-platform GUI backend.
> 
> The other important thing to note about this benchmark is that as the size of the data increases, the proportion of time spent in the backend increases.
> 
> I'm also worried (and I have no numbers to back this up) that a pyglet or PyOpenGL backend may actually be slower if the work to convert paths from matplotlib's path.Path format to the format understood by pyglet and/or PyOpenGL happens in Python, as your preliminary code backend_pyglet.py does in draw_path (i.e. looping over each vertex in a Python loop.)  In the Agg backend, that happens in C++ on-the-fly without copying the data -- see src/path_converters.h.  This code is exposed to Python through matplotlib._path.cleanup_path, but that does require copying memory, which for large data sets may be a limiting factor.  So you may end up needing to write the backend in C++ to really beat the current Agg backend, but I'd love to be proven wrong.
> 
> I hope this helps to better illustrate what you're seeing, and I don't mean to discourage you in implementing an OpenGL-based backend (which would be very nice to have for portability reasons among others).  But I hope this also illustrates that if the end goal is simply to "go faster", this may be somewhat like putting racing tires on a car without replacing the engine.
> 
> Cheers,
> Mike
> 
> On 07/29/2011 03:09 AM, Nicolas Rougier wrote:
>> 
>> 
>> 
>> I just did it using the regular python profiler (and a pyglet backend because glut cannot be easily profiled).
>> Here are some results for exactly 1000 frames displayed:
>> 
>> > python -m cProfile  -s cumulative test_backend_pyglet.py
>>          7723453 function calls (7596399 primitive calls) in 16.583 seconds
>> 
>>    Ordered by: cumulative time
>> 
>>    ncalls  tottime  percall  cumtime  percall filename:lineno(function)
>>         1    0.004    0.004   16.587   16.587 test_backend_pyglet.py:1(<module>)
>>         1    0.000    0.000   14.000   14.000 pyplot.py:123(show)
>>         1    0.000    0.000   14.000   14.000 backend_pyglet.py:214(show)
>>         1    0.000    0.000   13.951   13.951 __init__.py:115(run)
>>         1    0.000    0.000   13.951   13.951 base.py:117(run)
>>         1    0.009    0.009   13.949   13.949 base.py:149(_run_estimated)
>>      1000    0.018    0.000   13.639    0.014 base.py:244(idle)
>> 1119/1116    0.009    0.000   13.378    0.012 event.py:318(dispatch_event)
>>      1004    0.006    0.000   13.340    0.013 __init__.py:1148(dispatch_event)
>>      1000    0.002    0.000   13.326    0.013 backend_pyglet.py:319(on_draw)
>>      1000    0.013    0.000   13.324    0.013 backend_pyglet.py:324(draw)
>>      1000    0.004    0.000   13.001    0.013 backend_pyglet.py:342(_render_figure)
>> 60000/1000    0.204    0.000   12.977    0.013 artist.py:53(draw_wrapper)
>>      1000    0.023    0.000   12.970    0.013 figure.py:805(draw)
>>      1000    0.046    0.000   12.775    0.013 axes.py:1866(draw)
>>      2000    0.046    0.000   11.791    0.006 axis.py:1029(draw)
>>     10000    0.098    0.000    8.521    0.001 axis.py:219(draw)
>>     21000    0.710    0.000    6.467    0.000 lines.py:463(draw)
>>     20000    0.030    0.000    2.301    0.000 transforms.py:2234(get_transformed_points_and_affine)
>>     21000    0.093    0.000    2.245    0.000 transforms.py:2224(_revalidate)
>>         1    0.001    0.001    2.081    2.081 pylab.py:1(<module>)
>>         1    0.001    0.001    2.080    2.080 pylab.py:215(<module>)
>>      12/6    0.007    0.001    2.036    0.339 {__import__}
>>         1    0.001    0.001    1.937    1.937 pyplot.py:15(<module>)
>>         1    0.000    0.000    1.935    1.935 __init__.py:14(pylab_setup)
>>         1    0.001    0.001    1.931    1.931 backend_pyglet.py:55(<module>)
>>         1    0.011    0.011    1.929    1.929 __init__.py:94(<module>)
>> 68000/64000    0.450    0.000    1.824    0.000 transforms.py:1732(transform)
>>     15000    0.381    0.000    1.726    0.000 text.py:514(draw)
>>      2000    0.035    0.000    1.711    0.001 axis.py:977(_get_tick_bboxes)
>>     10000    0.091    0.000    1.668    0.000 text.py:713(get_window_extent)
>>     64605    0.694    0.000    1.520    0.000 path.py:83(__init__)
>>     24000    0.067    0.000    1.491    0.000 transforms.py:1155(transform_path_non_affine)
>>     20041    0.435    0.000    1.430    0.000 lines.py:386(recache)
>>     44000    0.118    0.000    1.385    0.000 transforms.py:1761(transform_non_affine)
>>    164452    1.340    0.000    1.340    0.000 {numpy.core.multiarray.array}
>>     20000    0.076    0.000    1.137    0.000 transforms.py:1119(transform_point)
>> 
>> 
>> It does not seem to have superfluous call to the various methods (even if the plot is a simple 3 points line, there is a lot to draw) and maybe this means an efficient OpenGL backend would require some cache system to avoid repeating "useless" operations.
>> 
>> 
>> Nicolas
>> 
>> 
>> On Jul 28, 2011, at 3:15 PM, Michael Droettboom wrote:
>> 
>>> Have you tried running it in the Python profiler?  I find this script [1] in combination with kcachegrind to be very useful in answering these kinds of questions.
>>> 
>>> [1] http://codespeak.net/pypy/dist/pypy/tool/lsprofcalltree.py
>>> 
>>> Mike
>>> 
>>> On 07/28/2011 07:16 AM, Nicolas Rougier wrote:
>>>> 
>>>> 
>>>> 
>>>> I've created a fork at: https://github.com/rougier/matplotlib/tree/gl-backend
>>>> 
>>>> The name of the backend is glut (it requires OpenGL) and does not display anything, it only measures fps.
>>>> 
>>>> It seems to be stuck at 100fps with the following test script:
>>>> 
>>>> import matplotlib
>>>> matplotlib.use('glut')
>>>> from pylab import *
>>>> plot([1,2,3])
>>>> show()
>>>> 
>>>> while the same do-nothing window directly in pyOpenGL is around 2000fps on the same machine.
>>>> 
>>>> I would like to understand why this is so slow and if it can be fixed.
>>>> 
>>>> 
>>>> 
>>>> Nicolas
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> On Jul 27, 2011, at 3:28 PM, Benjamin Root wrote:
>>>> 
>>>>> 
>>>>> 
>>>>> On Wednesday, July 27, 2011, Nicolas Rougier <Nic...@in...> wrote:
>>>>> >
>>>>> >
>>>>> > Hi all,
>>>>> >
>>>>> > I've been testing various idea around the idea of a GL backend, and I would have a few questions.
>>>>> > First, I tried to use the backend template to quickly test an empty pyglet backend and I've been quite surprised by the bad performances. Without drawing anything, I can hardly reach 100FPS and I wonder if I did something wrong ? (The backend is available backend_pyglet.py <http://www.loria.fr/~rougier/tmp/backend_pyglet.py> and the test file is at test_backend_pyglet.py <http://www.loria.fr/~rougier/tmp/test_backend_pyglet.py>)
>>>>> >
>>>>> > Second, I've been experimenting with proper anti-alias technics (using shaders) and the results are not so bad so far (IMHO) :
>>>>> > Antialiased line with thickness varying by 0.1 pixels:
>>>>> > http://www.loria.fr/~rougier/tmp/aa-line.png
>>>>> > (don't paid attention to the cap, it's not done yet)
>>>>> >
>>>>> > Antialiased circles (small circles position is increased by 0.1 pixels)
>>>>> > http://www.loria.fr/~rougier/tmp/aa-circle.png
>>>>> > (I can post source code if anyone is interested)
>>>>> > I don't know yet if all matplotlib artists can be drawing using these technics.
>>>>> >
>>>>> > My question relates to the cairo backend that now seems to support gl and shaders. Does anyone know the status of the gl-backend and how it would improve performances of matplotlib ? (I had a hard time finding any information). 
>>>>> >
>>>>> > Nicolas
>>>>> 
>>>>> Nicolas,
>>>>> 
>>>>> I want to immediately encourage you to continue on your efforts.  PLEASE make a fork on github so that we may be able to experiment better.
>>>>> 
>>>>> Cheers!
>>>>> Ben Root
>>>> 
>>>> 
>>>> ------------------------------------------------------------------------------
>>>> Got Input?   Slashdot Needs You.
>>>> Take our quick survey online.  Come on, we don't ask for help often.
>>>> Plus, you'll get a chance to win $100 to spend on ThinkGeek.
>>>> http://p.sf.net/sfu/slashdot-survey
>>>> 
>>>> _______________________________________________
>>>> Matplotlib-devel mailing list
>>>> Mat...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>> 
>>> 
>>> -- 
>>> Michael Droettboom
>>> Science Software Branch
>>> Space Telescope Science Institute
>>> Baltimore, Maryland, USA
>>> ------------------------------------------------------------------------------
>>> Got Input?   Slashdot Needs You.
>>> Take our quick survey online.  Come on, we don't ask for help often.
>>> Plus, you'll get a chance to win $100 to spend on ThinkGeek.
>>> http://p.sf.net/sfu/slashdot-survey_______________________________________________
>>> Matplotlib-devel mailing list
>>> Mat...@li...
>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>> 
>> 
>> ------------------------------------------------------------------------------
>> Got Input?   Slashdot Needs You.
>> Take our quick survey online.  Come on, we don't ask for help often.
>> Plus, you'll get a chance to win $100 to spend on ThinkGeek.
>> http://p.sf.net/sfu/slashdot-survey
>> 
>> _______________________________________________
>> Matplotlib-devel mailing list
>> Mat...@li...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
> 
> ------------------------------------------------------------------------------
> Got Input?   Slashdot Needs You.
> Take our quick survey online.  Come on, we don't ask for help often.
> Plus, you'll get a chance to win $100 to spend on ThinkGeek.
> http://p.sf.net/sfu/slashdot-survey_______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

On Fri, Jul 29, 2011 at 11:05 AM, Michael Droettboom <md...@st...>wrote:

> **
> Nicolas,
>
> I'm not sure how you've reached your conclusion.
>
> The 21,000 calls to Line2D.draw (i.e. 21 per frame) are easily explained
> since each grid line (or tick) is in fact a line.
>
> The 10,000 calls to Tick.draw (i.e. 10 per frame) are because there are 10
> tick labels.
>
> likewise for Text.draw etc. and on down the list.
>
> I don't see how any of these objects could be drawn without calling draw.
> ;)  There actually already is caching that occurs rendering text, for
> example.  None of these calls are strictly "useless".
>
> If animation speed is important, there are tricks in the examples/animation
> directory that show how to reuse the ticks from a previous draw and only
> update the data (i.e. eliminate many of these "useless" draw calls).  But
> they are tricks -- they don't work for the general case of "anything in this
> plot may change at any time".
>
> But I think you are comparing apples to oranges in your speed comparison.
> You say a "do-nothing" loop in PyOpenGL runs at around 2000 fps, and a
> "do-nothing" loop in matplotlib runs about 100fps.  But, of course, the
> matplotlib test case is actually doing a great deal even with the backend
> doing nothing -- it is doing all of the work of laying out the plot, which
> is the majority of time spent getting a plot to the screen.  And that all
> work happens in Python -- its speed is what it is and is acceptable in many
> contexts -- but no backend work is going to improve what's going on in the
> core.
>
> I should warn you that previous attempts to speed up matplotlib using
> hardware acceleration have failed to produce much fruit because the backends
> actually do fairly little work by design in matplotlib.  The actual act of
> rendering paths into pixels on screen (i.e. what happens in the backend) is
> a small fraction of the run time, even when using a software renderer (eg.
> Agg).  Here's a useful benchmark that renders a plot a bunch of times to
> memory:
>
>   import sys
>   import matplotlib
>   matplotlib.use(sys.argv[-1])
>   from pylab import *
>   import numpy
>   plot(numpy.arange(int(sys.argv[-2])))
>   for i in xrange(1000):
>       draw()
>
> and the results (on my 2.33GHz Intel E6550):
>
> > time python test_backend_speed.py 3 agg
>
> real    0m15.211s
> user    0m15.009s
> sys     0m0.136s
>
> # Here "backend_pyglet" is the do-nothing backend you sent to the list in a
> previous e-mail
> > time python test_backend_speed.py 3 module://backend_pyglet
>
> real    0m14.038s
> user    0m13.713s
> sys     0m0.256s
>
> > time python test_backend_speed.py 100 agg
>
> real    0m23.038s
> user    0m22.845s
> sys     0m0.132s
>
> > time python test_backend_speed.py 100 module://backend_pyglet
>
> real    0m15.251s
> user    0m14.837s
> sys     0m0.304s
>
> So you see the actual work in the backend can be a fairly small fraction of
> the total runtime -- that gives one an idea of the upper bound on the speed
> improvement that any backend could make without digging into the matplotlib
> core and making improvements there.
>
> In fairness, my test is not measuring the time to (once rendering the plot)
> blit it to the screen.  I suspect OpenGL will have an advantage there.  It
> may even be possible as a mid-way solution to create an Agg/OpenGL backend
> that used Agg for rendering and OpenGL -- that's something that would be
> really useful just to have another nice cross-platform GUI backend.
>
> The other important thing to note about this benchmark is that as the size
> of the data increases, the proportion of time spent in the backend
> increases.
>
> I'm also worried (and I have no numbers to back this up) that a pyglet or
> PyOpenGL backend may actually be slower if the work to convert paths from
> matplotlib's path.Path format to the format understood by pyglet and/or
> PyOpenGL happens in Python, as your preliminary code backend_pyglet.py does
> in draw_path (i.e. looping over each vertex in a Python loop.)  In the Agg
> backend, that happens in C++ on-the-fly without copying the data -- see
> src/path_converters.h.  This code is exposed to Python through
> matplotlib._path.cleanup_path, but that does require copying memory, which
> for large data sets may be a limiting factor.  So you may end up needing to
> write the backend in C++ to really beat the current Agg backend, but I'd
> love to be proven wrong.
>
> I hope this helps to better illustrate what you're seeing, and I don't mean
> to discourage you in implementing an OpenGL-based backend (which would be
> very nice to have for portability reasons among others).  But I hope this
> also illustrates that if the end goal is simply to "go faster", this may be
> somewhat like putting racing tires on a car without replacing the engine.
>
> Cheers,
> Mike
>
>
Nicolas,

I would like to add to Mike's comments by making it clear why I am
interested in having OpenGL for matplotlib.  It is not for speed.  The last
time I checked, a good portion of time is actually spent on the axis tickers
and automatic limits.  So, if speed is your goal, then that might be the
better place to look.

No, my main interest in OpenGL is to relax matplotlib's 2D limitation.  At
the core of matplotlib is a 2D layering engine that was perfectly suited for
2D graphing.  mplot3d then takes advantage of that layering feature to
hack-on 3D functionality.  However, as has been noted many, many times, this
does not always produce correct results.  I would love for the 2D layering
core to have some sort of logic where if the OpenGL backend is in use, and
if 3D art objects are detected, then bypass the dimension reduction that is
done in the layering engine and send the full 3D data over to OpenGL and let
it figure out the correct 2D projection.

However, I am not an OpenGL programmer and I wouldn't even know where to
start.  That is why I am very happy to see someone give this another try.
Also, as a side note, in discussions with Ryan May (who did the previous
attempt at using pyglet), the main reason he discontinued development at
that time was because the text rendering in pyglet was very poor and not
anti-aliased.  Do you have any indication if that has improved or not?

Ben Root

Hi,

I will be on vacation with limited email from July 14 to August 7, 2011.

Bonjour,

Je serai en conge du 14 juillet jusqu'au 7 aout, 2011.

Nicolas,

I'm not sure how you've reached your conclusion.

The 21,000 calls to Line2D.draw (i.e. 21 per frame) are easily explained 
since each grid line (or tick) is in fact a line.

The 10,000 calls to Tick.draw (i.e. 10 per frame) are because there are 
10 tick labels.

likewise for Text.draw etc. and on down the list.

I don't see how any of these objects could be drawn without calling 
draw. ;)  There actually already is caching that occurs rendering text, 
for example.  None of these calls are strictly "useless".

If animation speed is important, there are tricks in the 
examples/animation directory that show how to reuse the ticks from a 
previous draw and only update the data (i.e. eliminate many of these 
"useless" draw calls).  But they are tricks -- they don't work for the 
general case of "anything in this plot may change at any time".

But I think you are comparing apples to oranges in your speed 
comparison.  You say a "do-nothing" loop in PyOpenGL runs at around 2000 
fps, and a "do-nothing" loop in matplotlib runs about 100fps.  But, of 
course, the matplotlib test case is actually doing a great deal even 
with the backend doing nothing -- it is doing all of the work of laying 
out the plot, which is the majority of time spent getting a plot to the 
screen.  And that all work happens in Python -- its speed is what it is 
and is acceptable in many contexts -- but no backend work is going to 
improve what's going on in the core.

I should warn you that previous attempts to speed up matplotlib using 
hardware acceleration have failed to produce much fruit because the 
backends actually do fairly little work by design in matplotlib.  The 
actual act of rendering paths into pixels on screen (i.e. what happens 
in the backend) is a small fraction of the run time, even when using a 
software renderer (eg. Agg).  Here's a useful benchmark that renders a 
plot a bunch of times to memory:

   import sys
   import matplotlib
   matplotlib.use(sys.argv[-1])
   from pylab import *
   import numpy
   plot(numpy.arange(int(sys.argv[-2])))
   for i in xrange(1000):
       draw()

and the results (on my 2.33GHz Intel E6550):

 > time python test_backend_speed.py 3 agg

real    0m15.211s
user    0m15.009s
sys     0m0.136s

# Here "backend_pyglet" is the do-nothing backend you sent to the list 
in a previous e-mail
 > time python test_backend_speed.py 3 module://backend_pyglet

real    0m14.038s
user    0m13.713s
sys     0m0.256s

 > time python test_backend_speed.py 100 agg

real    0m23.038s
user    0m22.845s
sys     0m0.132s

 > time python test_backend_speed.py 100 module://backend_pyglet

real    0m15.251s
user    0m14.837s
sys     0m0.304s

So you see the actual work in the backend can be a fairly small fraction 
of the total runtime -- that gives one an idea of the upper bound on the 
speed improvement that any backend could make without digging into the 
matplotlib core and making improvements there.

In fairness, my test is not measuring the time to (once rendering the 
plot) blit it to the screen.  I suspect OpenGL will have an advantage 
there.  It may even be possible as a mid-way solution to create an 
Agg/OpenGL backend that used Agg for rendering and OpenGL -- that's 
something that would be really useful just to have another nice 
cross-platform GUI backend.

The other important thing to note about this benchmark is that as the 
size of the data increases, the proportion of time spent in the backend 
increases.

I'm also worried (and I have no numbers to back this up) that a pyglet 
or PyOpenGL backend may actually be slower if the work to convert paths 
from matplotlib's path.Path format to the format understood by pyglet 
and/or PyOpenGL happens in Python, as your preliminary code 
backend_pyglet.py does in draw_path (i.e. looping over each vertex in a 
Python loop.)  In the Agg backend, that happens in C++ on-the-fly 
without copying the data -- see src/path_converters.h.  This code is 
exposed to Python through matplotlib._path.cleanup_path, but that does 
require copying memory, which for large data sets may be a limiting 
factor.  So you may end up needing to write the backend in C++ to really 
beat the current Agg backend, but I'd love to be proven wrong.

I hope this helps to better illustrate what you're seeing, and I don't 
mean to discourage you in implementing an OpenGL-based backend (which 
would be very nice to have for portability reasons among others).  But I 
hope this also illustrates that if the end goal is simply to "go 
faster", this may be somewhat like putting racing tires on a car without 
replacing the engine.

Cheers,
Mike

On 07/29/2011 03:09 AM, Nicolas Rougier wrote:
>
>
> I just did it using the regular python profiler (and a pyglet backend 
> because glut cannot be easily profiled).
> Here are some results for exactly 1000 frames displayed:
>
> > python -m cProfile  -s cumulative test_backend_pyglet.py
>          7723453 function calls (7596399 primitive calls) in 16.583 
> seconds
>
>    Ordered by: cumulative time
>
>    ncalls  tottime  percall  cumtime  percall filename:lineno(function)
>         1    0.004    0.004   16.587   16.587 
> test_backend_pyglet.py:1(<module>)
>         1    0.000    0.000   14.000   14.000 pyplot.py:123(show)
>         1    0.000    0.000   14.000   14.000 backend_pyglet.py:214(show)
>         1    0.000    0.000   13.951   13.951 __init__.py:115(run)
>         1    0.000    0.000   13.951   13.951 base.py:117(run)
>         1    0.009    0.009   13.949   13.949 base.py:149(_run_estimated)
>      1000    0.018    0.000   13.639    0.014 base.py:244(idle)
> 1119/1116    0.009    0.000   13.378    0.012 event.py:318(dispatch_event)
>      1004    0.006    0.000   13.340    0.013 
> __init__.py:1148(dispatch_event)
>      1000    0.002    0.000   13.326    0.013 
> backend_pyglet.py:319(on_draw)
>      1000    0.013    0.000   13.324    0.013 backend_pyglet.py:324(draw)
>      1000    0.004    0.000   13.001    0.013 
> backend_pyglet.py:342(_render_figure)
> 60000/1000    0.204    0.000   12.977    0.013 artist.py:53(draw_wrapper)
>      1000    0.023    0.000   12.970    0.013 figure.py:805(draw)
>      1000    0.046    0.000   12.775    0.013 axes.py:1866(draw)
>      2000    0.046    0.000   11.791    0.006 axis.py:1029(draw)
>     10000    0.098    0.000    8.521    0.001 axis.py:219(draw)
>     21000    0.710    0.000    6.467    0.000 lines.py:463(draw)
>     20000    0.030    0.000    2.301    0.000 
> transforms.py:2234(get_transformed_points_and_affine)
>     21000    0.093    0.000    2.245    0.000 
> transforms.py:2224(_revalidate)
>         1    0.001    0.001    2.081    2.081 pylab.py:1(<module>)
>         1    0.001    0.001    2.080    2.080 pylab.py:215(<module>)
>      12/6    0.007    0.001    2.036    0.339 {__import__}
>         1    0.001    0.001    1.937    1.937 pyplot.py:15(<module>)
>         1    0.000    0.000    1.935    1.935 __init__.py:14(pylab_setup)
>         1    0.001    0.001    1.931    1.931 
> backend_pyglet.py:55(<module>)
>         1    0.011    0.011    1.929    1.929 __init__.py:94(<module>)
> 68000/64000    0.450    0.000    1.824    0.000 
> transforms.py:1732(transform)
>     15000    0.381    0.000    1.726    0.000 text.py:514(draw)
>      2000    0.035    0.000    1.711    0.001 
> axis.py:977(_get_tick_bboxes)
>     10000    0.091    0.000    1.668    0.000 
> text.py:713(get_window_extent)
>     64605    0.694    0.000    1.520    0.000 path.py:83(__init__)
>     24000    0.067    0.000    1.491    0.000 
> transforms.py:1155(transform_path_non_affine)
>     20041    0.435    0.000    1.430    0.000 lines.py:386(recache)
>     44000    0.118    0.000    1.385    0.000 
> transforms.py:1761(transform_non_affine)
>    164452    1.340    0.000    1.340    0.000 
> {numpy.core.multiarray.array}
>     20000    0.076    0.000    1.137    0.000 
> transforms.py:1119(transform_point)
>
>
> It does not seem to have superfluous call to the various methods (even 
> if the plot is a simple 3 points line, there is a lot to draw) and 
> maybe this means an efficient OpenGL backend would require some cache 
> system to avoid repeating "useless" operations.
>
>
> Nicolas
>
>
> On Jul 28, 2011, at 3:15 PM, Michael Droettboom wrote:
>
>> Have you tried running it in the Python profiler?  I find this script 
>> [1] in combination with kcachegrind to be very useful in answering 
>> these kinds of questions.
>>
>> [1] http://codespeak.net/pypy/dist/pypy/tool/lsprofcalltree.py
>>
>> Mike
>>
>> On 07/28/2011 07:16 AM, Nicolas Rougier wrote:
>>>
>>>
>>> I've created a fork at: 
>>> https://github.com/rougier/matplotlib/tree/gl-backend
>>>
>>> The name of the backend is glut (it requires OpenGL) and does not 
>>> display anything, it only measures fps.
>>>
>>> It seems to be stuck at 100fps with the following test script:
>>>
>>> import matplotlib
>>> matplotlib.use('glut')
>>> from pylab import *
>>> plot([1,2,3])
>>> show()
>>>
>>> while the same do-nothing window directly in pyOpenGL is around 
>>> 2000fps on the same machine.
>>>
>>> I would like to understand why this is so slow and if it can be fixed.
>>>
>>>
>>>
>>> Nicolas
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Jul 27, 2011, at 3:28 PM, Benjamin Root wrote:
>>>
>>>>
>>>>
>>>> On Wednesday, July 27, 2011, Nicolas Rougier 
>>>> <Nic...@in... <mailto:Nic...@in...>> wrote:
>>>> >
>>>> >
>>>> > Hi all,
>>>> >
>>>> > I've been testing various idea around the idea of a GL backend, 
>>>> and I would have a few questions.
>>>> > First, I tried to use the backend template to quickly test an 
>>>> empty pyglet backend and I've been quite surprised by the bad 
>>>> performances. Without drawing anything, I can hardly reach 100FPS 
>>>> and I wonder if I did something wrong ? (The backend is 
>>>> available backend_pyglet.py 
>>>> <http://www.loria.fr/~rougier/tmp/backend_pyglet.py 
>>>> <http://www.loria.fr/%7Erougier/tmp/backend_pyglet.py>> and the 
>>>> test file is at test_backend_pyglet.py 
>>>> <http://www.loria.fr/~rougier/tmp/test_backend_pyglet.py 
>>>> <http://www.loria.fr/%7Erougier/tmp/test_backend_pyglet.py>>)
>>>> >
>>>> > Second, I've been experimenting with proper anti-alias technics 
>>>> (using shaders) and the results are not so bad so far (IMHO) :
>>>> > Antialiased line with thickness varying by 0.1 pixels:
>>>> > http://www.loria.fr/~rougier/tmp/aa-line.png 
>>>> <http://www.loria.fr/%7Erougier/tmp/aa-line.png>
>>>> > (don't paid attention to the cap, it's not done yet)
>>>> >
>>>> > Antialiased circles (small circles position is increased by 0.1 
>>>> pixels)
>>>> > http://www.loria.fr/~rougier/tmp/aa-circle.png 
>>>> <http://www.loria.fr/%7Erougier/tmp/aa-circle.png>
>>>> > (I can post source code if anyone is interested)
>>>> > I don't know yet if all matplotlib artists can be drawing using 
>>>> these technics.
>>>> >
>>>> > My question relates to the cairo backend that now seems to 
>>>> support gl and shaders. Does anyone know the status of the 
>>>> gl-backend and how it would improve performances of matplotlib ? (I 
>>>> had a hard time finding any information).
>>>> >
>>>> > Nicolas
>>>>
>>>> Nicolas,
>>>>
>>>> I want to immediately encourage you to continue on your efforts. 
>>>>  PLEASE make a fork on github so that we may be able to experiment 
>>>> better.
>>>>
>>>> Cheers!
>>>> Ben Root 
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Got Input?   Slashdot Needs You.
>>> Take our quick survey online.  Come on, we don't ask for help often.
>>> Plus, you'll get a chance to win $100 to spend on ThinkGeek.
>>> http://p.sf.net/sfu/slashdot-survey
>>>
>>>
>>> _______________________________________________
>>> Matplotlib-devel mailing list
>>> Mat...@li...
>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>
>>
>> -- 
>> Michael Droettboom
>> Science Software Branch
>> Space Telescope Science Institute
>> Baltimore, Maryland, USA
>> ------------------------------------------------------------------------------
>> Got Input?   Slashdot Needs You.
>> Take our quick survey online.  Come on, we don't ask for help often.
>> Plus, you'll get a chance to win $100 to spend on ThinkGeek.
>> http://p.sf.net/sfu/slashdot-survey_______________________________________________
>> Matplotlib-devel mailing list
>> Mat...@li...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>
>
> ------------------------------------------------------------------------------
> Got Input?   Slashdot Needs You.
> Take our quick survey online.  Come on, we don't ask for help often.
> Plus, you'll get a chance to win $100 to spend on ThinkGeek.
> http://p.sf.net/sfu/slashdot-survey
>
>
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Hi,

I will be on vacation with limited email from July 14 to August 7, 2011.

Bonjour,

Je serai en conge du 14 juillet jusqu'au 7 aout, 2011.

I just did it using the regular python profiler (and a pyglet backend because glut cannot be easily profiled).
Here are some results for exactly 1000 frames displayed:

> python -m cProfile  -s cumulative test_backend_pyglet.py
         7723453 function calls (7596399 primitive calls) in 16.583 seconds

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.004    0.004   16.587   16.587 test_backend_pyglet.py:1(<module>)
        1    0.000    0.000   14.000   14.000 pyplot.py:123(show)
        1    0.000    0.000   14.000   14.000 backend_pyglet.py:214(show)
        1    0.000    0.000   13.951   13.951 __init__.py:115(run)
        1    0.000    0.000   13.951   13.951 base.py:117(run)
        1    0.009    0.009   13.949   13.949 base.py:149(_run_estimated)
     1000    0.018    0.000   13.639    0.014 base.py:244(idle)
1119/1116    0.009    0.000   13.378    0.012 event.py:318(dispatch_event)
     1004    0.006    0.000   13.340    0.013 __init__.py:1148(dispatch_event)
     1000    0.002    0.000   13.326    0.013 backend_pyglet.py:319(on_draw)
     1000    0.013    0.000   13.324    0.013 backend_pyglet.py:324(draw)
     1000    0.004    0.000   13.001    0.013 backend_pyglet.py:342(_render_figure)
60000/1000    0.204    0.000   12.977    0.013 artist.py:53(draw_wrapper)
     1000    0.023    0.000   12.970    0.013 figure.py:805(draw)
     1000    0.046    0.000   12.775    0.013 axes.py:1866(draw)
     2000    0.046    0.000   11.791    0.006 axis.py:1029(draw)
    10000    0.098    0.000    8.521    0.001 axis.py:219(draw)
    21000    0.710    0.000    6.467    0.000 lines.py:463(draw)
    20000    0.030    0.000    2.301    0.000 transforms.py:2234(get_transformed_points_and_affine)
    21000    0.093    0.000    2.245    0.000 transforms.py:2224(_revalidate)
        1    0.001    0.001    2.081    2.081 pylab.py:1(<module>)
        1    0.001    0.001    2.080    2.080 pylab.py:215(<module>)
     12/6    0.007    0.001    2.036    0.339 {__import__}
        1    0.001    0.001    1.937    1.937 pyplot.py:15(<module>)
        1    0.000    0.000    1.935    1.935 __init__.py:14(pylab_setup)
        1    0.001    0.001    1.931    1.931 backend_pyglet.py:55(<module>)
        1    0.011    0.011    1.929    1.929 __init__.py:94(<module>)
68000/64000    0.450    0.000    1.824    0.000 transforms.py:1732(transform)
    15000    0.381    0.000    1.726    0.000 text.py:514(draw)
     2000    0.035    0.000    1.711    0.001 axis.py:977(_get_tick_bboxes)
    10000    0.091    0.000    1.668    0.000 text.py:713(get_window_extent)
    64605    0.694    0.000    1.520    0.000 path.py:83(__init__)
    24000    0.067    0.000    1.491    0.000 transforms.py:1155(transform_path_non_affine)
    20041    0.435    0.000    1.430    0.000 lines.py:386(recache)
    44000    0.118    0.000    1.385    0.000 transforms.py:1761(transform_non_affine)
   164452    1.340    0.000    1.340    0.000 {numpy.core.multiarray.array}
    20000    0.076    0.000    1.137    0.000 transforms.py:1119(transform_point)


It does not seem to have superfluous call to the various methods (even if the plot is a simple 3 points line, there is a lot to draw) and maybe this means an efficient OpenGL backend would require some cache system to avoid repeating "useless" operations.


Nicolas


On Jul 28, 2011, at 3:15 PM, Michael Droettboom wrote:

> Have you tried running it in the Python profiler?  I find this script [1] in combination with kcachegrind to be very useful in answering these kinds of questions.
> 
> [1] http://codespeak.net/pypy/dist/pypy/tool/lsprofcalltree.py
> 
> Mike
> 
> On 07/28/2011 07:16 AM, Nicolas Rougier wrote:
>> 
>> 
>> 
>> I've created a fork at: https://github.com/rougier/matplotlib/tree/gl-backend
>> 
>> The name of the backend is glut (it requires OpenGL) and does not display anything, it only measures fps.
>> 
>> It seems to be stuck at 100fps with the following test script:
>> 
>> import matplotlib
>> matplotlib.use('glut')
>> from pylab import *
>> plot([1,2,3])
>> show()
>> 
>> while the same do-nothing window directly in pyOpenGL is around 2000fps on the same machine.
>> 
>> I would like to understand why this is so slow and if it can be fixed.
>> 
>> 
>> 
>> Nicolas
>> 
>> 
>> 
>> 
>> 
>> 
>> On Jul 27, 2011, at 3:28 PM, Benjamin Root wrote:
>> 
>>> 
>>> 
>>> On Wednesday, July 27, 2011, Nicolas Rougier <Nic...@in...> wrote:
>>> >
>>> >
>>> > Hi all,
>>> >
>>> > I've been testing various idea around the idea of a GL backend, and I would have a few questions.
>>> > First, I tried to use the backend template to quickly test an empty pyglet backend and I've been quite surprised by the bad performances. Without drawing anything, I can hardly reach 100FPS and I wonder if I did something wrong ? (The backend is available backend_pyglet.py <http://www.loria.fr/~rougier/tmp/backend_pyglet.py> and the test file is at test_backend_pyglet.py <http://www.loria.fr/~rougier/tmp/test_backend_pyglet.py>)
>>> >
>>> > Second, I've been experimenting with proper anti-alias technics (using shaders) and the results are not so bad so far (IMHO) :
>>> > Antialiased line with thickness varying by 0.1 pixels:
>>> > http://www.loria.fr/~rougier/tmp/aa-line.png
>>> > (don't paid attention to the cap, it's not done yet)
>>> >
>>> > Antialiased circles (small circles position is increased by 0.1 pixels)
>>> > http://www.loria.fr/~rougier/tmp/aa-circle.png
>>> > (I can post source code if anyone is interested)
>>> > I don't know yet if all matplotlib artists can be drawing using these technics.
>>> >
>>> > My question relates to the cairo backend that now seems to support gl and shaders. Does anyone know the status of the gl-backend and how it would improve performances of matplotlib ? (I had a hard time finding any information). 
>>> >
>>> > Nicolas
>>> 
>>> Nicolas,
>>> 
>>> I want to immediately encourage you to continue on your efforts.  PLEASE make a fork on github so that we may be able to experiment better.
>>> 
>>> Cheers!
>>> Ben Root
>> 
>> 
>> ------------------------------------------------------------------------------
>> Got Input?   Slashdot Needs You.
>> Take our quick survey online.  Come on, we don't ask for help often.
>> Plus, you'll get a chance to win $100 to spend on ThinkGeek.
>> http://p.sf.net/sfu/slashdot-survey
>> 
>> _______________________________________________
>> Matplotlib-devel mailing list
>> Mat...@li...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
> 
> 
> -- 
> Michael Droettboom
> Science Software Branch
> Space Telescope Science Institute
> Baltimore, Maryland, USA
> ------------------------------------------------------------------------------
> Got Input?   Slashdot Needs You.
> Take our quick survey online.  Come on, we don't ask for help often.
> Plus, you'll get a chance to win $100 to spend on ThinkGeek.
> http://p.sf.net/sfu/slashdot-survey_______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

On Wed, Jul 27, 2011 at 10:57 AM, Benjamin Root <ben...@ou...> wrote:

> I have come across an odd layering issue with errorbars.  The errorbar caps
> are not drawn at the same layer as the errorbar itself.  Also, the line
> connecting multiple errorbars are plotted on a completely different layer
> than the errorbars.  I am attaching an example demonstrating this.  Note the
> green caps are showing up on top of the red errorbar, meanwhile, the red
> errorbar is drawn on top of the green errorbar.  Also notice that the thin
> red line connecting the red errorbars is showing up on top of the green cap
> for lambda = 1.
>
> I am not exactly sure how this should be fixed.  It is most likely an issue
> with the fact that the caps are drawn as markers instead of a line (I am
> guessing so that the marker would then be sized based on width in points
> instead of data).  When an axes' child objects are z-sorted, the objects
> that do not declare a z-order are loaded in the order of their creation (or
> is it in some arbitrary order based on the type of collection?).
>
> Cheers,
> Ben Root
>


Just for the record, I filed issue #409 here:
https://github.com/matplotlib/matplotlib/issues/409

Ben Root

Have you tried running it in the Python profiler?  I find this script 
[1] in combination with kcachegrind to be very useful in answering these 
kinds of questions.

[1] http://codespeak.net/pypy/dist/pypy/tool/lsprofcalltree.py

Mike

On 07/28/2011 07:16 AM, Nicolas Rougier wrote:
>
>
> I've created a fork at: 
> https://github.com/rougier/matplotlib/tree/gl-backend
>
> The name of the backend is glut (it requires OpenGL) and does not 
> display anything, it only measures fps.
>
> It seems to be stuck at 100fps with the following test script:
>
> import matplotlib
> matplotlib.use('glut')
> from pylab import *
> plot([1,2,3])
> show()
>
> while the same do-nothing window directly in pyOpenGL is around 
> 2000fps on the same machine.
>
> I would like to understand why this is so slow and if it can be fixed.
>
>
>
> Nicolas
>
>
>
>
>
>
> On Jul 27, 2011, at 3:28 PM, Benjamin Root wrote:
>
>>
>>
>> On Wednesday, July 27, 2011, Nicolas Rougier 
>> <Nic...@in... <mailto:Nic...@in...>> wrote:
>> >
>> >
>> > Hi all,
>> >
>> > I've been testing various idea around the idea of a GL backend, and 
>> I would have a few questions.
>> > First, I tried to use the backend template to quickly test an empty 
>> pyglet backend and I've been quite surprised by the bad 
>> performances. Without drawing anything, I can hardly reach 100FPS and 
>> I wonder if I did something wrong ? (The backend is 
>> available backend_pyglet.py 
>> <http://www.loria.fr/~rougier/tmp/backend_pyglet.py 
>> <http://www.loria.fr/%7Erougier/tmp/backend_pyglet.py>> and the test 
>> file is at test_backend_pyglet.py 
>> <http://www.loria.fr/~rougier/tmp/test_backend_pyglet.py 
>> <http://www.loria.fr/%7Erougier/tmp/test_backend_pyglet.py>>)
>> >
>> > Second, I've been experimenting with proper anti-alias technics 
>> (using shaders) and the results are not so bad so far (IMHO) :
>> > Antialiased line with thickness varying by 0.1 pixels:
>> > http://www.loria.fr/~rougier/tmp/aa-line.png 
>> <http://www.loria.fr/%7Erougier/tmp/aa-line.png>
>> > (don't paid attention to the cap, it's not done yet)
>> >
>> > Antialiased circles (small circles position is increased by 0.1 pixels)
>> > http://www.loria.fr/~rougier/tmp/aa-circle.png 
>> <http://www.loria.fr/%7Erougier/tmp/aa-circle.png>
>> > (I can post source code if anyone is interested)
>> > I don't know yet if all matplotlib artists can be drawing using 
>> these technics.
>> >
>> > My question relates to the cairo backend that now seems to support 
>> gl and shaders. Does anyone know the status of the gl-backend and how 
>> it would improve performances of matplotlib ? (I had a hard time 
>> finding any information).
>> >
>> > Nicolas
>>
>> Nicolas,
>>
>> I want to immediately encourage you to continue on your efforts. 
>>  PLEASE make a fork on github so that we may be able to experiment 
>> better.
>>
>> Cheers!
>> Ben Root 
>
>
> ------------------------------------------------------------------------------
> Got Input?   Slashdot Needs You.
> Take our quick survey online.  Come on, we don't ask for help often.
> Plus, you'll get a chance to win $100 to spend on ThinkGeek.
> http://p.sf.net/sfu/slashdot-survey
>
>
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel


-- 
Michael Droettboom
Science Software Branch
Space Telescope Science Institute
Baltimore, Maryland, USA

  Hi,

I would like to report some issues in python basemap package and 
easy-fixes for
some of them. We would really appreciate if there was somebody who could 
look
on this and consider important bugs to be fixed.

These bugs was found by Coverity scan and we have ran it on Fedora 15
packages (srpm). There was some findings in python basemap package also. 
Coverity
is proprietary software but we can give its result to community (if 
interrested),
possibly we can re-run some tests on srpms on demand.

Patch for next three obvious bugs (plaintext cov. output) is attached:

Error: OVERRUN_STATIC:
basemap-0.99.4/src/pj_gridlist.c:252: overrun-local: Overrunning static 
array "name", with 128 elements, at position 128 with index variable 
"end_char".

Error: UNINIT:
basemap-0.99.4/src/mk_cheby.c:42: var_decl: Declaring variable "T" 
without initializer.
basemap-0.99.4/src/mk_cheby.c:150: uninit_use: Using uninitialized value 
"T".
basemap-0.99.4/src/mk_cheby.c:151: uninit_use: Using uninitialized value 
"T->mu".
basemap-0.99.4/src/mk_cheby.c:152: uninit_use: Using uninitialized value 
"T->cu".
basemap-0.99.4/src/mk_cheby.c:154: uninit_use: Using uninitialized value 
"T->mv".
basemap-0.99.4/src/mk_cheby.c:155: uninit_use: Using uninitialized value 
"T->cv".
basemap-0.99.4/src/mk_cheby.c:163: uninit_use: Using uninitialized value 
"T".

Error: NO_EFFECT:
basemap-0.99.4/src/PJ_sconics.c:52: self_assign: Assignment operation 
"*del = *del" has no effect.

__________________________

But there is more defects (or coding style issues) and some of them are not
so obvious. There could be potential problems -- need to be consulted, e.g.:

Error: EVALUATION_ORDER:
basemap-0.99.4/src/PJ_stere.c:232: write_write_order: In "P->phits = 
(pj_param(P->params, "tlat_ts").i ? P->phits = pj_param(P->params, 
"rlat_ts").f : 1.5708)", "P->phits" is written in "P->phits" (the 
assignment left-hand side) and written in "pj_param(P->params, 
"tlat_ts").i ? P->phits = pj_param(P->params, "rlat_ts").f : 1.5708" but 
the order in which the side effects take place is undefined because 
there is no intervening sequence point.

Error: FORWARD_NULL:
basemap-0.99.4/src/emess.c:29: var_compare_op: Comparing "fmt" to null 
implies that "fmt" might be null.
basemap-0.99.4/src/emess.c:51: var_deref_model: Passing null variable 
"fmt" to function "vfprintf", which dereferences it.

Error: FORWARD_NULL:
basemap-0.99.4/src/pj_gridinfo.c:505: var_compare_op: Comparing "gp" to 
null implies that "gp" might be null.
basemap-0.99.4/src/pj_gridinfo.c:512: alias_transfer: Assigning null: 
"lnk" = "gp".
basemap-0.99.4/src/pj_gridinfo.c:512: var_deref_op: Dereferencing null 
variable "lnk".

Error: FORWARD_NULL:
basemap-0.99.4/src/pj_ell_set.c:30: var_compare_op: Comparing 
"start->next" to null implies that "start->next" might be null.
basemap-0.99.4/src/pj_ell_set.c:92: var_deref_op: Dereferencing null 
variable "start->next".

Coverity test was done on:
http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/basemap-0.99.4/basemap-0.99.4.tar.gz

..so svn version is little different (line numbers) but it can be handy for
finding hidden bugs. I can send you full plain-text log if you want.

Pavel

I've created a fork at: https://github.com/rougier/matplotlib/tree/gl-backend

The name of the backend is glut (it requires OpenGL) and does not display anything, it only measures fps.

It seems to be stuck at 100fps with the following test script:

import matplotlib
matplotlib.use('glut')
from pylab import *
plot([1,2,3])
show()

while the same do-nothing window directly in pyOpenGL is around 2000fps on the same machine.

I would like to understand why this is so slow and if it can be fixed.



Nicolas






On Jul 27, 2011, at 3:28 PM, Benjamin Root wrote:

> 
> 
> On Wednesday, July 27, 2011, Nicolas Rougier <Nic...@in...> wrote:
> >
> >
> > Hi all,
> >
> > I've been testing various idea around the idea of a GL backend, and I would have a few questions.
> > First, I tried to use the backend template to quickly test an empty pyglet backend and I've been quite surprised by the bad performances. Without drawing anything, I can hardly reach 100FPS and I wonder if I did something wrong ? (The backend is available backend_pyglet.py <http://www.loria.fr/~rougier/tmp/backend_pyglet.py> and the test file is at test_backend_pyglet.py <http://www.loria.fr/~rougier/tmp/test_backend_pyglet.py>)
> >
> > Second, I've been experimenting with proper anti-alias technics (using shaders) and the results are not so bad so far (IMHO) :
> > Antialiased line with thickness varying by 0.1 pixels:
> > http://www.loria.fr/~rougier/tmp/aa-line.png
> > (don't paid attention to the cap, it's not done yet)
> >
> > Antialiased circles (small circles position is increased by 0.1 pixels)
> > http://www.loria.fr/~rougier/tmp/aa-circle.png
> > (I can post source code if anyone is interested)
> > I don't know yet if all matplotlib artists can be drawing using these technics.
> >
> > My question relates to the cairo backend that now seems to support gl and shaders. Does anyone know the status of the gl-backend and how it would improve performances of matplotlib ? (I had a hard time finding any information). 
> >
> > Nicolas
> 
> Nicolas,
> 
> I want to immediately encourage you to continue on your efforts.  PLEASE make a fork on github so that we may be able to experiment better.
> 
> Cheers!
> Ben Root

Hi,

I will be on vacation with limited email from July 14 to August 7, 2011.

Bonjour,

Je serai en conge du 14 juillet jusqu'au 7 aout, 2011.

On Wednesday, July 27, 2011, Nicolas Rougier <Nic...@in...>
wrote:
>
>
> Hi all,
>
> I've been testing various idea around the idea of a GL backend, and I
would have a few questions.
> First, I tried to use the backend template to quickly test an empty pyglet
backend and I've been quite surprised by the bad performances. Without
drawing anything, I can hardly reach 100FPS and I wonder if I did something
wrong ? (The backend is available backend_pyglet.py <
http://www.loria.fr/~rougier/tmp/backend_pyglet.py> and the test file is
at test_backend_pyglet.py <
http://www.loria.fr/~rougier/tmp/test_backend_pyglet.py>)
>
> Second, I've been experimenting with proper anti-alias technics (using
shaders) and the results are not so bad so far (IMHO) :
> Antialiased line with thickness varying by 0.1 pixels:
> http://www.loria.fr/~rougier/tmp/aa-line.png
> (don't paid attention to the cap, it's not done yet)
>
> Antialiased circles (small circles position is increased by 0.1 pixels)
> http://www.loria.fr/~rougier/tmp/aa-circle.png
> (I can post source code if anyone is interested)
> I don't know yet if all matplotlib artists can be drawing using these
technics.
>
> My question relates to the cairo backend that now seems to support gl and
shaders. Does anyone know the status of the gl-backend and how it would
improve performances of matplotlib ? (I had a hard time finding any
information).
>
> Nicolas

Nicolas,

I want to immediately encourage you to continue on your efforts.  PLEASE
make a fork on github so that we may be able to experiment better.

Cheers!
Ben Root

Hi all,


I've been testing various idea around the idea of a GL backend, and I would have a few questions.

First, I tried to use the backend template to quickly test an empty pyglet backend and I've been quite surprised by the bad performances. Without drawing anything, I can hardly reach 100FPS and I wonder if I did something wrong ? (The backend is available backend_pyglet.py and the test file is at test_backend_pyglet.py)


Second, I've been experimenting with proper anti-alias technics (using shaders) and the results are not so bad so far (IMHO) :

Antialiased line with thickness varying by 0.1 pixels:
http://www.loria.fr/~rougier/tmp/aa-line.png
(don't paid attention to the cap, it's not done yet)


Antialiased circles (small circles position is increased by 0.1 pixels)
http://www.loria.fr/~rougier/tmp/aa-circle.png

(I can post source code if anyone is interested)

I don't know yet if all matplotlib artists can be drawing using these technics.


My question relates to the cairo backend that now seems to support gl and shaders. Does anyone know the status of the gl-backend and how it would improve performances of matplotlib ? (I had a hard time finding any information). 


Nicolas

Thanks for the report.  This issue was fixed some time ago on the 
maintenance branch, but we have not been very good at getting 
maintenance releases out lately.  Hopefully one will drop shortly.

Mike

On 07/22/2011 03:57 PM, Andy Somogyi wrote:
> Hi All,
>
> Apparently, some time ago, libpng made the png_info structure opaque. OSX has a newer version on libpng, so matplotlib would not compile. Attached is the _png.cpp that fixes this issue, I changed it to use the new libpng access routines.
>
> This issue will certainly come up on any platform with newer libpng versions.
>
>
>
> ------------------------------------------------------------------------------
> 10 Tips for Better Web Security
> Learn 10 ways to better secure your business today. Topics covered include:
> Web security, SSL, hacker attacks&  Denial of Service (DoS), private keys,
> security Microsoft Exchange, secure Instant Messaging, and much more.
> http://www.accelacomm.com/jaw/sfnl/114/51426210/
>
>
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Hi,

I will be on vacation with limited email from July 14 to August 7, 2011.

Bonjour,

Je serai en conge du 14 juillet jusqu'au 7 aout, 2011.

Hi All,

Apparently, some time ago, libpng made the png_info structure opaque. OSX has a newer version on libpng, so matplotlib would not compile. Attached is the _png.cpp that fixes this issue, I changed it to use the new libpng access routines. 

This issue will certainly come up on any platform with newer libpng versions. 

This turned up when I generated blowups of portions of a thumbnail image by
clicking on various locations. The blowups contained a button, but the
button was inoperative. My guess is that when the click handler that created
the blowup exited it took the button object with it. My workaround was to
create the various button handlers in the Button __init__ method so that
they hung onto a reference to self. This may not be the best approach so
I've attached a file for discussion.

Chuck

On Friday, July 15, 2011, Fernando Perez <fpe...@gm...> wrote:
> On Wed, Jul 13, 2011 at 1:58 PM, Benjamin Root <ben...@ou...> wrote:
>> I would love to find out if there is some way to embed a video using sphinx
>
> have a look at the sources for:
>
> http://fperez.org/talks
>
> Cheers,
>
> f
>

That is pretty straight-forward and easy to use.  My main concern
though will be that YouTube demo videos have to be considered as
static resources and not created on the fly.  This has to be the case
because we would need the YouTube URL for the link.  Might not be
terrible.

On a related note, I was reading through the docs for la (labeled
arrays) when I noticed that the docs had functions organized.  I
haven't looked at the source code yet, but I wonder if it was done
with some sort of sphinx magic that we could use?

Ben

Sorry for this very very reply.
The error message is now fixed in the git master branch.

Regards,

-JJ


On Fri, May 20, 2011 at 12:31 PM, Steve Ward <pla...@gm...> wrote:
> In legend.__init__ the exception message doesn't match the condition
> that raises it.  I think it should say "numpoints must be > 0".  I'm
> using matplotlib version 0.99.3.
>
> Example:
> #----------------------------
> import matplotlib.pyplot as plt
> ax = plt.subplot(1,1,1)
> ax.plot(1.1, 2.2, label='foo')
> ax.legend(numpoints=0)
> #----------------------------
> # ValueError: numpoints must be >= 0; it was 0
>
>
> Steve
>
> ------------------------------------------------------------------------------
> What Every C/C++ and Fortran developer Should Know!
> Read this article and learn how Intel has extended the reach of its
> next-generation tools to help Windows* and Linux* C/C++ and Fortran
> developers boost performance applications - including clusters.
> http://p.sf.net/sfu/intel-dev2devmay
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>

Hi,

I will be on vacation with limited email from July 14 to August 7, 2011.

Bonjour,

Je serai en conge du 14 juillet jusqu'au 7 aout, 2011.

I came across this website where different colormaps have been compared 
and the author has come up with an optimal colormap for data 
visualization called the "cool-warm colormap".

http://www.cs.unm.edu/~kmorel/documents/ColorMaps/index.html

It is somewhat similar to the cool colormap already included in 
matplotlib, but I've added the new colormap to matplotlib in the patch 
attached in case it is deemed fit to be included in the matplotlib source.

Regards,
Sameer

Came across an oddity here in one of the demos:
http://matplotlib.sourceforge.net/mpl_toolkits/mplot3d/tutorial.html#subplotting

In the second subplotting example, there is supposed to be a y-axis label,
but it is being cut off.  In this example, I used plt.figaspect() to set an
appropriate figure aspect for a 2x1 subplot grid.  However, it looks like it
sets the figure too narrow.  The aspect ratio has to be set to around 1.05
before the entire y-axis label appears.  I don't know if this is a bug or
not, but I thought I ought to raise this issue.

Ben Root

Hi,
 
Using matplotlib.path.Path.arc to create a wedge (e.g. via matplotlib.path.Path.wedge) creates a Path with three uninitialized vertex positions. (The first one and the last two.)
 
As you would expect - sometimes this leads to nasty drawing artifacts!
 
I've tried zeroing the entire array and a couple of ways to just zero the offending vertices, but they all have equal performance, so I've gone with zeroing the entire array. It has the simplest, cleanest code, and will be more robust to future changes.
 
Regards
Richard Hattersley
 
 
--------------- patch follows ----------------------- 
 
>From 6f5d9f71d1d56d80c0c1764268a6ddfc3f653f44 Mon Sep 17 00:00:00 2001
From: Richard Hattersley <hat...@bt...>
Date: Mon, 18 Jul 2011 14:20:16 +0100
Subject: [PATCH] Fix uninitialized vertices in Path.arc() for wedges.
---
 lib/matplotlib/path.py |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/lib/matplotlib/path.py b/lib/matplotlib/path.py
index da37d7d..d5461b0 100644
--- a/lib/matplotlib/path.py
+++ b/lib/matplotlib/path.py
@@ -621,7 +621,7 @@ class Path(object):
         if is_wedge:
             length = n * 3 + 4
-            vertices = np.empty((length, 2), np.float_)
+            vertices = np.zeros((length, 2), np.float_)
             codes = cls.CURVE4 * np.ones((length, ), cls.code_type)
             vertices[1] = [xA[0], yA[0]]
             codes[0:2] = [cls.MOVETO, cls.LINETO]
--
1.7.6