matplotlib-devel Mailing List for matplotlib (Page 3)

On Wed, Oct 20, 2010 at 8:43 AM, Giovanni Plantageneto
<g.p...@ru...> wrote:
> Hi,
>
> I think I found a bug in the command fill_between().
> If I try to make a fill_between plot on an x-axis that spans both negative and positive values (and only in that case), I have some funny results.
> I attach the code that gives the wrong result, the workaround I use, and their output.


Can you post a complete free-standing example that replicates the problem?

Hi,

I think I found a bug in the command fill_between().
If I try to make a fill_between plot on an x-axis that spans both negative and positive values (and only in that case), I have some funny results.
I attach the code that gives the wrong result, the workaround I use, and their output.

Regards



figure()
fill_between(totime,vartotal-varstddv,vartotal+varstddv,color='blue',alpha=0.3)
savefig('wrong.png',format='png',dpi=150)

figure()
fill(np.concatenate((totime[::-1],totime)),np.concatenate((vartotal[::-1]-varstddv[::-1],vartotal+varstddv)),color='blue',alpha=0.25)
savefig('right.png',format='png',dpi=150)

Hi

As discussed recently I have a patch, see
http://github.com/friedrichromstedt/matplotlib/commit/1ac794b362d143ac6a634b70993d950d8fe4adeb
.

I'd like to provide a patch in .patch format for this.  But I don't
know how to do this.  Is it simply $git format-patch -1?

The test suite passes except for some failures on my 10.6 Mac OS X
with yesterday checked-out astraw github repo.  I'll send them in
another thread.  Let's keep tidy :-)  There's more than the symlog
error mentioned by LittleBigBrain on my sys.

The failures are identical with and without the patch.

Opinions?

I'm still hesitating to send this to -devel, because this list shall
be better low-traffic I believe, but I think this one definitely is a
devel email :-)

Friedrich

On Sat, Oct 16, 2010 at 12:54 PM, Benjamin Root <ben...@ou...> wrote:
> Note that blitting isn't always a time-saver.  If most of your image is
> animated and there is very little static background information, the
> overhead of blitting will impact your rendering performance.  However, for
> most purposes of matplotlib, blitting will improve animation performance.

Just to note here: a lot of time in matplotlib is spent rendering
axes, ticks, and tick labels. So even if the plot elements can't be
cached, animation performance can be *vastly* improved just by
eliminating the drawing of those things.

Ryan

-- 
Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma

Bug fixed in repo github.com/friedrichromstedt/matplotlib:trunk

A pull by astraw if possible would be great, I don't want to merge in
the trunk from astraw all the time, according to numpy-discussion this
will generate messy structure once we switched to GitHub :-)

UNTESTED, LittleBigBrain (how's your real name?), can you please fetch
from that repo and run the test suite, I have to work for numpy 10.5
Installers.

Your tests provided by you pass now.  But it might break other things
if I had bad luck.

Alternatively I can provide the diff, if someone tells me how to do
that.  First awaiting LittleBigBrain's test results.

Friedrich

2010/10/18 LittleBigBrain <bra...@gm...>:
> 2010/10/18 Friedrich Romstedt <fri...@gm...>:
>> 2010/10/18 脑关(BrainGateway)生命科学仪器 <bra...@gm...>:
>>> On Mon, Oct 18, 2010 at 12:54 AM, Benjamin Root <ben...@ou...> wrote:
>>>> On Sun, Oct 17, 2010 at 5:35 PM, braingateway <bra...@gm...>
>>>> wrote:
>>>>> I tried all possible colormaps and found out: gist_rainbow, terrain,
>>>>> bwr, brg, and seismic will generate the same error, all other colormaps
>>>>> are OK. I wonder is this a bug or expected behavior?
>>> Hi Ben,
>>> Thanks a lot for answering my question! I am a newbie to matplotlib,
>>> so please forgive me, if the question is stupid ;p
>>
>> Questions are never stupid ...
>>
>>> In my program I wanna know the exact RGBA value of a data point in the
>>> figure, in order to plot a corresponding line with the same color in
>>> another figure. That is why I need to call a(z) to get RGBA value of a
>>> point at (x,y,z) (z is represented by color). If I understood it
>>> correctly, you said I do not need to specify the 'lut' in
>>> cm.get_cmap(name,lut)? I thought the colormap object is actually a
>>> lookup table with a length specified by lut.
>>
>> Kind of, but it's initialised from a linear segmentation dictionary,
>> so you can get cmaps with a precision you want.
>>
>>> It turns out I do not
>>> need to specify anything here.
>>
>> Default param is 256.
>>
>>> But if I do not specify anything the
>>> colormap.N is always 256. What will happen then, if I need more color
>>> steps?
>>
>> cm = get_cmap('...', 1024)
>>
>>> I do notice a(2) and a(2.2) returns different values. So I am
>>> very confused about the principle how the RGBA value is generated by
>>> the colormaps.
>>
>> It's the "magic" to distinguish between integer (in the LUT range) and
>> float (in [0.0, 1.0]).  2 gives the LUT entry 2, 2.0 will give the
>> upper value since it's > 1, as will 2.2 do.  Try 0.5, and 0.6, or 0.0
>> and 0.1.
>>
>> Actually I cannot reproduce your error on a recently (some weeks ago)
>> checked-out GitHub repo version of mpl 1.0.0.  Please provide
>> mpl.__version__ so that we check if that's the reason - as simple as
>> it might be.
>>
>> MacBook-Pro-Friedrich:Report Friedrich$ python
>> Python 2.6.5 (r265:79063, Jul 18 2010, 12:14:53)
>> [GCC 4.2.1 (Apple Inc. build 5659)] on darwin
>> Type "help", "copyright", "credits" or "license" for more information.
>>>>> import matplotlib.cm
>>>>> cm = matplotlib.cm.get_cmap('gist_rainbow')
>>>>> cm(range(256))
>> array([[ 1.        ,  0.        ,  0.16      ,  1.        ],
>>       [ 1.        ,  0.        ,  0.13908497,  1.        ],
>>       [ 1.        ,  0.        ,  0.11816993,  1.        ],
>>       ...,
>>       [ 1.        ,  0.        ,  0.79262575,  1.        ],
>>       [ 1.        ,  0.        ,  0.77131287,  1.        ],
>>       [ 1.        ,  0.        ,  0.75      ,  1.        ]])
>>>>> print matplotlib.__version__
>> 1.0.0
>>
>> The attribute of the cm where the error occurs on your machine is set
>> to a valid value for me.  I also cannot find any code path leading to
>> a wrong initialisation of the attribute.  Might be that the data the
>> cm is init'ed from changed.  There is a code path initialising the cm
>> from a tuple (your attribute was a tuple used like a dict), but this
>> works too:  (terrain is such an example):
>>
>>>>> cm = matplotlib.cm.get_cmap('terrain')
>>>>> cm(range(256))
>> array([[ 0.2       ,  0.2       ,  0.6       ,  1.        ],
>>       [ 0.19477124,  0.21045752,  0.61045752,  1.        ],
>>       [ 0.18954248,  0.22091503,  0.62091503,  1.        ],
>>       ...,
>>       [ 0.98431373,  0.97992157,  0.97898039,  1.        ],
>>       [ 0.99215686,  0.98996078,  0.9894902 ,  1.        ],
>>       [ 1.        ,  1.        ,  1.        ,  1.        ]])
>>
>> Might be that there was 'red' misspelled in gist_rainbow in you mpl
>> version, this may explain the behaviour if we track it down.  Can you
>> do the following to verify this:
>>
>> import matplotlib._cm
>> print matplotlib._cm._gist_stern_data
>>
>> ? Thx,
>> Friedrich
>>
>
> Thanks a lot for your detailed response!
> In previous posts I did give the version: '1.0.svn'. Then I installed
> the latest stable version. Here is the version info:
>>>> print matplotlib.__version__
> 1.0.0
>
> However, the error is still there.
>
> You could not reproduce the error in your code, because you did not
> specify 'lut' when you call get_cmap
> try this:
> maps=[m for m in matplotlib.cm.datad if not m.endswith("_r")]
> for i in maps:
>        try:
>                a=matplotlib.cm.get_cmap(i,256)(range(256))
>        except:
>                (type, value, traceback) = sys.exc_info()
>                print "Problems to create %s" % (i,)
>                print "The error was --> %s: %s" % (type, value)
> u will reproduce the error.
>
> Thanks very much to point out I could actually print the colormap
> definition-data.
> I printed all of them, and found out the colormaps invoking errors are
> all tuples but not dictionaries and all other colormaps are actually
> dictionaries.
> ###Examples without Errors###
> ###They are all dictionaries###
> print matplotlib._cm._gist_stern_data
> {'blue': ((0.0, 0.0, 0.0), (0.5, 1.0, 1.0), (0.73499999999999999, 0.0,
> 0.0), (1.0, 1.0, 1.0)), 'green': ((0, 0, 0), (1, 1, 1)), 'red': ((0.0,
> 0.0, 0.0), (0.054699999999999999, 1.0, 1.0), (0.25, 0.027, 0.25),
> (1.0, 1.0, 1.0))}
> print matplotlib._cm._jet_data
> {'blue': ((0.0, 0.5, 0.5), (0.11, 1, 1), (0.34000000000000002, 1, 1),
> (0.65000000000000002, 0, 0), (1, 0, 0)), 'green': ((0.0, 0, 0),
> (0.125, 0, 0), (0.375, 1, 1), (0.64000000000000001, 1, 1),
> (0.91000000000000003, 0, 0), (1, 0, 0)), 'red': ((0.0, 0, 0),
> (0.34999999999999998, 0, 0), (0.66000000000000003, 1, 1),
> (0.89000000000000001, 1, 1), (1, 0.5, 0.5))}
> ###Colormaps invoking Errors###
> ###They are all Tuples###
> Problems to create gist_rainbow
> ((0.0, (1.0, 0.0, 0.16)), (0.029999999999999999, (1.0, 0.0, 0.0)),
> (0.215, (1.0, 1.0, 0.0)), (0.40000000000000002, (0.0, 1.0, 0.0)),
> (0.58599999999999997, (0.0, 1.0, 1.0)), (0.77000000000000002, (0.0,
> 0.0, 1.0)), (0.95399999999999996, (1.0, 0.0, 1.0)), (1.0, (1.0, 0.0,
> 0.75)))
> The error was --> <type 'exceptions.TypeError'>: tuple indices must be
> integers, not str
> Problems to create terrain
> ((0.0, (0.20000000000000001, 0.20000000000000001,
> 0.59999999999999998)), (0.14999999999999999, (0.0,
> 0.59999999999999998, 1.0)), (0.25, (0.0, 0.80000000000000004,
> 0.40000000000000002)), (0.5, (1.0, 1.0, 0.59999999999999998)), (0.75,
> (0.5, 0.35999999999999999, 0.33000000000000002)), (1.0, (1.0, 1.0,
> 1.0)))
> The error was --> <type 'exceptions.TypeError'>: tuple indices must be
> integers, not str
> Problems to create bwr
> ((0.0, 0.0, 1.0), (1.0, 1.0, 1.0), (1.0, 0.0, 0.0))
> The error was --> <type 'exceptions.TypeError'>: tuple indices must be
> integers, not str
> Problems to create brg
> ((0.0, 0.0, 1.0), (1.0, 0.0, 0.0), (0.0, 1.0, 0.0))
> The error was --> <type 'exceptions.TypeError'>: tuple indices must be
> integers, not str
> Problems to create seismic
> ((0.0, 0.0, 0.29999999999999999), (0.0, 0.0, 1.0), (1.0, 1.0, 1.0),
> (1.0, 0.0, 0.0), (0.5, 0.0, 0.0))
> The error was --> <type 'exceptions.TypeError'>: tuple indices must be
> integers, not str

On Fri, Oct 15, 2010 at 11:46 PM, Eric Firing <ef...@ha...> wrote:

> On 10/15/2010 04:37 PM, Michiel de Hoon wrote:
> > Hello,
> >
> > Recently I tried to get animations to work for the Mac OS X backend with
> the new timer framework by implementing a TimerMac class that inherits from
> backend_bases.TimerBase. This seems to work fine; the code is uploaded to
> SVN if you'd like to try it.
> >
> > I am not sure what to do about blitting though. I can imagine two uses
> for blitting: Either to make sure that the next figure appears
> instantaneously as a whole after it has been drawn off-screen, or to
> precalculate a background figure that is used repeatedly, and upon which the
> changing parts of the figure are drawn.
> >
> > The latter would make sense for the Mac OS X backend also, but the former
> would not. Effectively, because of double buffering in Quartz all figures
> are automatically blitted already.
> >
> > So my question is: What is the purpose of blitting for animations in
> matplotlib?
>
> As far as I know, it is only the second--speeding up the animation by
> not having to recalculate and redraw so much.
>
> Eric
>
>
Just to add information here.  In an animation, you have to draw each
frame.  The actual process of displaying that draw (between the graphics
card and the monitor) is very fast, however, the process of calculating the
draw by the graphics card slow (relative to the display part).

So, imagine your mouse cursor moving across your screen, technically that is
an animation of sorts.  If everything else on the screen is static, the
graphics card can keep the draw information for the static background in
memory and then only bother to recalculate the draw for the mouse cursor and
put that data into the appropriate place in the data of the static
background.

Note, it is important to use the 'animated=True' kwarg for the collections
and plots that will be non-static in order to take advantage of blitting.
Any plot object that is tagged as animated will have their rendering
deferred until explicitly told to draw.

Note that blitting isn't always a time-saver.  If most of your image is
animated and there is very little static background information, the
overhead of blitting will impact your rendering performance.  However, for
most purposes of matplotlib, blitting will improve animation performance.

I hope this is informative,
Ben Root

OK, thanks. Then it makes sense to implement blitting for the Mac OS X backend as well.

Unfortunately I have hit one snag:
Drawing in Mac OS X / Quartz should be done from inside the event loop. You cannot take a graphics context and just start drawing to it; the graphics context won't be properly initialized.
In lib/matplotlib/animation.py, the actual drawing occurs here:

def _post_draw(self, framedata, blit):
    # After the frame is rendered, this handles the actual flushing of
    # the draw, which can be a direct draw_idle() or make use of the
    # blitting.
    if blit and self._drawn_artists:
        self._blit_draw(self._drawn_artists, self._blit_cache)
    else:
        self._fig.canvas.draw_idle()

Without blitting, this function calls draw_idle on the canvas, which triggers a call to figure.draw(renderer) from inside the event loop, which does the actual drawing, and all is well.

With blitting, it appears that self._blit_draw starts to draw directly to the graphics context; this won't work on Mac OS X.

Can this code be reorganized such that the blitting occurs from inside figure.draw(renderer)? In my opinion, there is no essential difference between blitting and other drawing functions, so to me it would make sense to have blitting and drawing together in figure.draw(renderer).

Best,
--Michiel.



      

On 10/15/2010 04:37 PM, Michiel de Hoon wrote:
> Hello,
>
> Recently I tried to get animations to work for the Mac OS X backend with the new timer framework by implementing a TimerMac class that inherits from backend_bases.TimerBase. This seems to work fine; the code is uploaded to SVN if you'd like to try it.
>
> I am not sure what to do about blitting though. I can imagine two uses for blitting: Either to make sure that the next figure appears instantaneously as a whole after it has been drawn off-screen, or to precalculate a background figure that is used repeatedly, and upon which the changing parts of the figure are drawn.
>
> The latter would make sense for the Mac OS X backend also, but the former would not. Effectively, because of double buffering in Quartz all figures are automatically blitted already.
>
> So my question is: What is the purpose of blitting for animations in matplotlib?

As far as I know, it is only the second--speeding up the animation by 
not having to recalculate and redraw so much.

Eric

>
> Best,
> --Michiel.
>

Hello,

Recently I tried to get animations to work for the Mac OS X backend with the new timer framework by implementing a TimerMac class that inherits from backend_bases.TimerBase. This seems to work fine; the code is uploaded to SVN if you'd like to try it.

I am not sure what to do about blitting though. I can imagine two uses for blitting: Either to make sure that the next figure appears instantaneously as a whole after it has been drawn off-screen, or to precalculate a background figure that is used repeatedly, and upon which the changing parts of the figure are drawn.

The latter would make sense for the Mac OS X backend also, but the former would not. Effectively, because of double buffering in Quartz all figures are automatically blitted already.

So my question is: What is the purpose of blitting for animations in matplotlib?

Best,
--Michiel.


      

On Thu, Oct 14, 2010 at 11:55 AM, Michael Droettboom <md...@st...> wrote:
>
> I would suggest:
>
>  canvas.print_figure(string_io, format='svg')
>

Fantastic!  You now have your first commit in the IPython repo:

http://github.com/ipython/ipython/commit/4abff4a2c1b60b9cf342ce66bfe5113123e29294

Many thanks!  Between this and a better understanding of the png/svg
issues, we're converging on a very solid system.

And all problems I had reported as being from the mpl side are now
resolved, so your debt is cleared ;)

Regards,

f

Yeah -- the print_* format methods should probably be made private.  
They don't properly save/restore state -- that is all done in one place 
in the higher-level function "print_figure".

I would suggest:

   canvas.print_figure(string_io, format='svg')

I'll go ahead and make these functions private to avoid confusion -- 
probably on the trunk only, though, to avoid breaking things for cases 
where it's currently working "by accident" (i.e. in a non-interactive 
setting).

Mike

On 10/14/2010 02:43 PM, Fernando Perez wrote:
> from cStringIO import StringIO
> plot(rand(100))
> canvas = gcf().canvas
> string_io = StringIO()
> canvas.print_svg(string_io)
>    


-- 
Michael Droettboom
Science Software Branch
Space Telescope Science Institute
Baltimore, Maryland, USA

On Thu, Oct 14, 2010 at 8:15 AM, Michael Droettboom <md...@st...> wrote:
> I'm not sure what's causing this.  I don't have a Python >= 2.6
> environment with all the Qt bells and whistles to test ipython HEAD with
> (our house standard here is still 2.5)...  Once I find the time for
> that, hopefully I can see what's going on.  But I suspect "paste()" is
> doing something fishy here.  Saving an SVG from the standard matplotlib
> window does not cause the "dpi changing without the window size
> changing" problem shown here.  I would suggest looking at the code
> triggered from the save dialog in matplotlib and compare it to what
> "paste()" is doing.

Here's how to reproduce it in plain python/ipyhon:

from cStringIO import StringIO
plot(rand(100))
canvas = gcf().canvas
string_io = StringIO()
canvas.print_svg(string_io)

Now try panning the figure.  Once you pan it and it redraws, the
problem shows up.

This is not seen via a savefig() call, but calling directly print_svg
seems to be the issue.  Are we not supposed to use print_svg
ourselves?  Should we only use savefig()?

Cheers,

f

On Thu, Oct 14, 2010 at 7:51 AM, Fernando Perez <fpe...@gm...> wrote:
> Well, the client can save a session to html, svgs and all:
>
> http://fperez.org/tmp/ipython-svg.xml
>
> If the svg has extra metadata embedded, this will preserve it.  The
> author of the html saving works in genomics at UCSF, and he embeds
> extra info in his MPL svgs so that he can later annotate and
> manipulate the generated SVGs.
>
> So I think there's a good justification for wanting SVG.  However,
> this limitation in Qt means we may want to make it at least optional,
> so that people can use png when they want accurate rendering but
> without access to the original vector figure, and SVG when they need
> the original vector info (and are OK with Qt's limitations).

I suppose you could send both, and the client could display the png
and save the svg.

-- Nathaniel

I'm not sure what's causing this.  I don't have a Python >= 2.6 
environment with all the Qt bells and whistles to test ipython HEAD with 
(our house standard here is still 2.5)...  Once I find the time for 
that, hopefully I can see what's going on.  But I suspect "paste()" is 
doing something fishy here.  Saving an SVG from the standard matplotlib 
window does not cause the "dpi changing without the window size 
changing" problem shown here.  I would suggest looking at the code 
triggered from the save dialog in matplotlib and compare it to what 
"paste()" is doing.

Mike

On 09/16/2010 04:54 PM, Fernando Perez wrote:
> Howdy,
>
> I just wanted to bump this again: given the speed of Michael's recent
> SVG fixes, maybe fixing these two is also quite easy.  They are the
> only problem coming from matplotlib right now regarding the use of the
> new qt console for ipython.
>
> If not let me know and I'll just file reports on the tracker for
> long-term storage :)
>
> Cheers,
>
> f
>
> On Tue, Sep 7, 2010 at 3:39 AM, Fernando Perez<fpe...@gm...>  wrote:
>    
>> Hi folks,
>>
>> I've just implemented support in ipython for simultaneous use of the
>> interactive mpl gui backends along with inlined figures, as I had
>> suggested to Eric things could work.
>>
>> But I'm seeing two little glitches, illustrated here:
>>
>> http://fperez.org/tmp/mpl_svg_bug.png
>>
>> The white console on the right  is IPython, the mpl window was my only
>> open figure.
>>
>> The code I ran was:
>>
>> x=rand(1000)
>> plot (x) # this pops up the normal gui, I tested Qt4Agg and GTKAgg
>>
>> paste() # this pasted the open figure into the IPython console.
>>
>> At this point, the plot in the window got that funny size, with the
>> x-labels double-drawn.  It seems as if the figure got re-drawn over
>> the previous canvas, at a different size.  If I resize the window, the
>> problem goes away, but if I don't resize it, it persists through new
>> plot/draw operations.
>>
>> The second problem... I then zoomed the interactive window and issued
>> paste again, getting the plot in the bottom right of the figure:
>> paste()
>>
>> And here the bug seems to be related to clipping: while the window
>> clips OK, the SVG seems not to.
>>
>> Is this a fundamental limitation of the SVG backend?
>>
>> For IPython we can also switch to pngs if that turns out to work
>> better, but I figured I'd report these...
>>
>> All this was done with current mpl from trunk.
>>      
> ------------------------------------------------------------------------------
> Start uncovering the many advantages of virtual appliances
> and start using them to simplify application deployment and
> accelerate your shift to cloud computing.
> http://p.sf.net/sfu/novell-sfdev2dev
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>    


-- 
Michael Droettboom
Science Software Branch
Space Telescope Science Institute
Baltimore, Maryland, USA

On Thu, Oct 14, 2010 at 10:05 AM, Robert Kern <rob...@gm...> wrote:

> On 10/14/10 8:57 AM, Benjamin Root wrote:
>
> > My only concern about that approach -- if I understand it correctly -- is
> that
> > this path clipping would have to be redone at every change in view
> limits.
> > Wouldn't that mean that in order to support panning and zooming, we would
> need
> > to keep an original copy of the SVG data and then create a clipped
> version of
> > the SVG data to post into the QT widget?
>
> Do you currently support panning and zooming just given the SVG file?
> That's
> quite a trick. ipythonqt certainly doesn't attempt that.
>
>
Sorry, you are right, I had a brain fart and wasn't thinking straight...
disregard my previous email.

Ben Root

On 10/14/10 8:57 AM, Benjamin Root wrote:

> My only concern about that approach -- if I understand it correctly -- is that
> this path clipping would have to be redone at every change in view limits.
> Wouldn't that mean that in order to support panning and zooming, we would need
> to keep an original copy of the SVG data and then create a clipped version of
> the SVG data to post into the QT widget?

Do you currently support panning and zooming just given the SVG file? That's 
quite a trick. ipythonqt certainly doesn't attempt that.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco

On Thu, Oct 14, 2010 at 6:04 AM, John Hunter <jd...@gm...> wrote:
> I believe the motivation here is in the separation of the ipython
> kernel process from the shell process, eg so the kernel can be running
> remotely and they can ship the image files across the wire.   Fernando
> can clarify though.

Indeed.

> Also, what was the motivation to go with svg by default rather than
> png -- that it is vector graphics?  I think the benefits of vector
> graphics over raster in an environment where you are not planning on
> interacting with the data (eg arbitrary zoom) are limited.  At least
> with PNG, you can rely on 100% correctness vis-a-vis mpl w/o worrying

Well, the client can save a session to html, svgs and all:

http://fperez.org/tmp/ipython-svg.xml

If the svg has extra metadata embedded, this will preserve it.  The
author of the html saving works in genomics at UCSF, and he embeds
extra info in his MPL svgs so that he can later annotate and
manipulate the generated SVGs.

So I think there's a good justification for wanting SVG.  However,
this limitation in Qt means we may want to make it at least optional,
so that people can use png when they want accurate rendering but
without access to the original vector figure, and SVG when they need
the original vector info (and are OK with Qt's limitations).

Does that make sense?

> about 3rd party rendering.  I also wonder if some of the ideas in the
> html5 canvas for embedding an interactive graphic into a remote client
> with socket communication can be applied to the ipython qt shell .

Certainly!  In fact a UC Berkeley student has already started working
on an html5 ipython frontend (that can work simultaneously with the Qt
one!), and his plan is to look into the html5 MPL back end for the
plotting.  Stay tuned.

Cheers,

f

On Thu, Oct 14, 2010 at 1:55 AM, Hans Meine <han...@gm...> wrote:
> Inkscape is an incredible piece of software, but maybe you should better test
> with e.g. Firefox, since there are *many* SVGs out there which only display
> fine in the almighty Inkscape.  (Could you attach the example SVG?  It looks
> simple enough to serve as a testcase.)

Attached.  Firefox shows the same as inkscape.

>
>> However, the problem seems to be that the Qt SVG renderer simply
>> doesn't support clipping paths, period.
>
> That's an unfortunate limitation, indeed. :-(

[...]

Thanks for the feedback.  As MD points out, perhaps bug isn't the
right word, just that Qt chose to target a simpler svg spec level than
what mpl uses.

Cheers,

f

On Thu, Oct 14, 2010 at 7:49 AM, Michael Droettboom <md...@st...> wrote:

> It looks like the QtSvg module is targeting SVG Tiny 1.2, which does not
> support clipping:
>
> http://doc.qt.nokia.com/4.3/qtsvg.html
>
> SVG Tiny also limits a lot of the CSS styling things that matplotlib
> makes use of -- though those things are probably easier to workaround if
> necessary.
>
> So, it's not broken so much as scoped below the threshold of usefulness ;)
>
> A possible workaround would be to add vector clipping to matplotlib and
> perform it before output.  This has the nice side benefit of reducing
> file sizes (in many cases).  However, that's a pretty major project to
> get right.
>
>
My only concern about that approach -- if I understand it correctly -- is
that this path clipping would have to be redone at every change in view
limits.  Wouldn't that mean that in order to support panning and zooming, we
would need to keep an original copy of the SVG data and then create a
clipped version of the SVG data to post into the QT widget?

Ben Root

On Thu, Oct 14, 2010 at 7:49 AM, Michael Droettboom <md...@st...> wrote:

> I haven't had a chance to look at the new Qt shell in ipython yet -- but
> is it necessary to go through a file?  Perhaps writing a native Qt4

I believe the motivation here is in the separation of the ipython
kernel process from the shell process, eg so the kernel can be running
remotely and they can ship the image files across the wire.   Fernando
can clarify though.

Also, what was the motivation to go with svg by default rather than
png -- that it is vector graphics?  I think the benefits of vector
graphics over raster in an environment where you are not planning on
interacting with the data (eg arbitrary zoom) are limited.  At least
with PNG, you can rely on 100% correctness vis-a-vis mpl w/o worrying
about 3rd party rendering.  I also wonder if some of the ideas in the
html5 canvas for embedding an interactive graphic into a remote client
with socket communication can be applied to the ipython qt shell .

JDH

It looks like the QtSvg module is targeting SVG Tiny 1.2, which does not 
support clipping:

http://doc.qt.nokia.com/4.3/qtsvg.html

SVG Tiny also limits a lot of the CSS styling things that matplotlib 
makes use of -- though those things are probably easier to workaround if 
necessary.

So, it's not broken so much as scoped below the threshold of usefulness ;)

A possible workaround would be to add vector clipping to matplotlib and 
perform it before output.  This has the nice side benefit of reducing 
file sizes (in many cases).  However, that's a pretty major project to 
get right.

I haven't had a chance to look at the new Qt shell in ipython yet -- but 
is it necessary to go through a file?  Perhaps writing a native Qt4 
backend (rather than the Qt4Agg one we currently have) would allow this 
to work better.  The Qt4 rendering infrastructure itself does seem to 
support clipping.

Mike

On 10/13/2010 07:36 PM, Fernando Perez wrote:
> Hi folks,
>
> [CC'ing mpl-dev just for reference so they know we're taking care of
> this on our side]
>
> I've been investigating further the bug where clipped paths in SVG
> render wrong in our console.  It turns out the mpl team already fixed
> some things on their side, and I can confirm that their SVGs now
> render fine in inkscape.  Here's an annotated example:
>
> http://i.imgur.com/NCSEJ.png
>
> However, the problem seems to be that the Qt SVG renderer simply
> doesn't support clipping paths, period.  Furthermore, they seem to
> have decided they won't fix this, as the bug was closed with "won't
> fix":
>
> http://bugreports.qt.nokia.com/browse/QTBUG-1865
>
> > From the Qt experts: am I misreading this?  Because if we're indeed
> stuck with a half-broken SVG renderer from qt, then we'll need to
> reconsider our implementation of pastefig(), to perhaps support an
> optional format flag so that users can send png if they prefer...
> Bummer.
>
> Cheers,
>
> f
>
> ------------------------------------------------------------------------------
> Beautiful is writing same markup. Internet Explorer 9 supports
> standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and DOM L2&  L3.
> Spend less time writing and  rewriting code and more time creating great
> experiences on the web. Be a part of the beta today.
> http://p.sf.net/sfu/beautyoftheweb
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>    


-- 
Michael Droettboom
Science Software Branch
Space Telescope Science Institute
Baltimore, Maryland, USA

Hi,

Op den Dunnersdag 14 Oktober 2010 Klock 01:36:52 hett Fernando Perez schreven:
> I've been investigating further the bug where clipped paths in SVG
> render wrong in our console.  It turns out the mpl team already fixed
> some things on their side, and I can confirm that their SVGs now
> render fine in inkscape.

Inkscape is an incredible piece of software, but maybe you should better test 
with e.g. Firefox, since there are *many* SVGs out there which only display 
fine in the almighty Inkscape.  (Could you attach the example SVG?  It looks 
simple enough to serve as a testcase.)

> However, the problem seems to be that the Qt SVG renderer simply
> doesn't support clipping paths, period.

That's an unfortunate limitation, indeed. :-(

> Furthermore, they seem to
> have decided they won't fix this, as the bug was closed with "won't
> fix":
> 
> http://bugreports.qt.nokia.com/browse/QTBUG-1865

In this older tracker item (before migration), you can see that the "Won't 
Fix" was later changed to "Deferred", now "Expired":

http://qt.nokia.com/developer/task-tracker/index_html?method=entry&id=204966

That at least does not look like Nokia's official policy is that this won't 
get fixed.

Anyhow, I am not so sure I fully understand the situation.  AFAICS, there are 
two separate issues here:
a) generation of the SVG
b) SVG rendering

Even if Qt's renderer does not support clipping, what does that have to do 
with a) (which is relevant for ipython export)?  Does MPL use Qt for SVG 
export?  ('Cause the tracker item is about QSvgGenerator, a QPaintDevice for 
*generating* SVGs.)

I wonder why QSvgGenerator is not fixed even without clipping support in Qt's 
SVG renderer.. I can understand if they say "we want to be able to render what 
we produce", but in this case this introduces seemingly unnecessary 
limitations.

(BTW: The viewbox will not help, this only sets a global SVG 'viewbox' 
property, which is not related to path clipping at all.)

Have a nice day,
  Hans

PS: Here are the current docs; I could not find a word about clipping in them:
  http://doc.qt.nokia.com/4.7/qtsvg.html
  http://doc.qt.nokia.com/4.7/qsvggenerator.html

Hi folks,

[CC'ing mpl-dev just for reference so they know we're taking care of
this on our side]

I've been investigating further the bug where clipped paths in SVG
render wrong in our console.  It turns out the mpl team already fixed
some things on their side, and I can confirm that their SVGs now
render fine in inkscape.  Here's an annotated example:

http://i.imgur.com/NCSEJ.png

However, the problem seems to be that the Qt SVG renderer simply
doesn't support clipping paths, period.  Furthermore, they seem to
have decided they won't fix this, as the bug was closed with "won't
fix":

http://bugreports.qt.nokia.com/browse/QTBUG-1865

>From the Qt experts: am I misreading this?  Because if we're indeed
stuck with a half-broken SVG renderer from qt, then we'll need to
reconsider our implementation of pastefig(), to perhaps support an
optional format flag so that users can send png if they prefer...
Bummer.

Cheers,

f

On Tue, Oct 12, 2010 at 10:11 AM, John Hunter <jd...@gm...> wrote:
> On Tue, Oct 12, 2010 at 11:47 AM, Keith Goodman <kwg...@gm...> wrote:
>
>> Well, then perhaps the doc string needs tweaking?
>>
>>    *adjusted*
>>      If True (default) replace open, close, high, low, and volume with
>>      their adjusted values.
>>      The adjustment is by a scale factor, S = adjusted_close/close.
>>      Adjusted volume is actual volume divided by S;
>>      Adjusted prices are actual prices multiplied by S.  Hence,
>>      the product of price and volume is unchanged by the adjustment.
>>
>> It does state that volume is adjusted by S (where S contains dividend
>> info). I take it that volume is the same whether adjusted is True or
>> False, which means that dollar volume is not unchanged.
>
>
>
> How does this look?
>
>    Parse the historical data in file handle fh from yahoo finance.
>
>    *adjusted*
>      If True (default) replace open, close, high, low, and volume with
>      their adjusted values.
>      The adjustment is by a scale factor, S = adjusted_close/close.
>      Adjusted prices are actual prices multiplied by S.  Hence,
>
>      Note that volume is already backward split adjusted by Yahoo, so
>      if you want to compute dollars traded, multiply volume by the
>      adjusted close, regardless of whether you choose adjusted =
>      True|False

Here are some suggested tweaks:

   *adjusted*
     If True (default) replace open, close, high, and low prices with
     their adjusted values. The adjustment is by a scale factor,
     S = adjusted_close/close. Adjusted prices are actual prices
     multiplied by S.

     Volume is not adjusted as it is already backward split adjusted
     by Yahoo. If you want to compute dollars traded, multiply
     volume by the adjusted close, regardless of whether you choose
     adjusted = True|False.

On Tue, Oct 12, 2010 at 11:47 AM, Keith Goodman <kwg...@gm...> wrote:

> Well, then perhaps the doc string needs tweaking?
>
>    *adjusted*
>      If True (default) replace open, close, high, low, and volume with
>      their adjusted values.
>      The adjustment is by a scale factor, S = adjusted_close/close.
>      Adjusted volume is actual volume divided by S;
>      Adjusted prices are actual prices multiplied by S.  Hence,
>      the product of price and volume is unchanged by the adjustment.
>
> It does state that volume is adjusted by S (where S contains dividend
> info). I take it that volume is the same whether adjusted is True or
> False, which means that dollar volume is not unchanged.



How does this look?

    Parse the historical data in file handle fh from yahoo finance.

    *adjusted*
      If True (default) replace open, close, high, low, and volume with
      their adjusted values.
      The adjustment is by a scale factor, S = adjusted_close/close.
      Adjusted prices are actual prices multiplied by S.  Hence,

      Note that volume is already backward split adjusted by Yahoo, so
      if you want to compute dollars traded, multiply volume by the
      adjusted close, regardless of whether you choose adjusted =
      True|False