matplotlib-devel Mailing List for matplotlib

On Thu, Oct 14, 2010 at 11:55 AM, Michael Droettboom <md...@st...> wrote:
>
> I would suggest:
>
>  canvas.print_figure(string_io, format='svg')
>

Fantastic!  You now have your first commit in the IPython repo:

http://github.com/ipython/ipython/commit/4abff4a2c1b60b9cf342ce66bfe5113123e29294

Many thanks!  Between this and a better understanding of the png/svg
issues, we're converging on a very solid system.

And all problems I had reported as being from the mpl side are now
resolved, so your debt is cleared ;)

Regards,

f

Yeah -- the print_* format methods should probably be made private.  
They don't properly save/restore state -- that is all done in one place 
in the higher-level function "print_figure".

I would suggest:

   canvas.print_figure(string_io, format='svg')

I'll go ahead and make these functions private to avoid confusion -- 
probably on the trunk only, though, to avoid breaking things for cases 
where it's currently working "by accident" (i.e. in a non-interactive 
setting).

Mike

On 10/14/2010 02:43 PM, Fernando Perez wrote:
> from cStringIO import StringIO
> plot(rand(100))
> canvas = gcf().canvas
> string_io = StringIO()
> canvas.print_svg(string_io)
>    


-- 
Michael Droettboom
Science Software Branch
Space Telescope Science Institute
Baltimore, Maryland, USA

On Thu, Oct 14, 2010 at 8:15 AM, Michael Droettboom <md...@st...> wrote:
> I'm not sure what's causing this.  I don't have a Python >= 2.6
> environment with all the Qt bells and whistles to test ipython HEAD with
> (our house standard here is still 2.5)...  Once I find the time for
> that, hopefully I can see what's going on.  But I suspect "paste()" is
> doing something fishy here.  Saving an SVG from the standard matplotlib
> window does not cause the "dpi changing without the window size
> changing" problem shown here.  I would suggest looking at the code
> triggered from the save dialog in matplotlib and compare it to what
> "paste()" is doing.

Here's how to reproduce it in plain python/ipyhon:

from cStringIO import StringIO
plot(rand(100))
canvas = gcf().canvas
string_io = StringIO()
canvas.print_svg(string_io)

Now try panning the figure.  Once you pan it and it redraws, the
problem shows up.

This is not seen via a savefig() call, but calling directly print_svg
seems to be the issue.  Are we not supposed to use print_svg
ourselves?  Should we only use savefig()?

Cheers,

f

On Thu, Oct 14, 2010 at 7:51 AM, Fernando Perez <fpe...@gm...> wrote:
> Well, the client can save a session to html, svgs and all:
>
> http://fperez.org/tmp/ipython-svg.xml
>
> If the svg has extra metadata embedded, this will preserve it.  The
> author of the html saving works in genomics at UCSF, and he embeds
> extra info in his MPL svgs so that he can later annotate and
> manipulate the generated SVGs.
>
> So I think there's a good justification for wanting SVG.  However,
> this limitation in Qt means we may want to make it at least optional,
> so that people can use png when they want accurate rendering but
> without access to the original vector figure, and SVG when they need
> the original vector info (and are OK with Qt's limitations).

I suppose you could send both, and the client could display the png
and save the svg.

-- Nathaniel

I'm not sure what's causing this.  I don't have a Python >= 2.6 
environment with all the Qt bells and whistles to test ipython HEAD with 
(our house standard here is still 2.5)...  Once I find the time for 
that, hopefully I can see what's going on.  But I suspect "paste()" is 
doing something fishy here.  Saving an SVG from the standard matplotlib 
window does not cause the "dpi changing without the window size 
changing" problem shown here.  I would suggest looking at the code 
triggered from the save dialog in matplotlib and compare it to what 
"paste()" is doing.

Mike

On 09/16/2010 04:54 PM, Fernando Perez wrote:
> Howdy,
>
> I just wanted to bump this again: given the speed of Michael's recent
> SVG fixes, maybe fixing these two is also quite easy.  They are the
> only problem coming from matplotlib right now regarding the use of the
> new qt console for ipython.
>
> If not let me know and I'll just file reports on the tracker for
> long-term storage :)
>
> Cheers,
>
> f
>
> On Tue, Sep 7, 2010 at 3:39 AM, Fernando Perez<fpe...@gm...>  wrote:
>    
>> Hi folks,
>>
>> I've just implemented support in ipython for simultaneous use of the
>> interactive mpl gui backends along with inlined figures, as I had
>> suggested to Eric things could work.
>>
>> But I'm seeing two little glitches, illustrated here:
>>
>> http://fperez.org/tmp/mpl_svg_bug.png
>>
>> The white console on the right  is IPython, the mpl window was my only
>> open figure.
>>
>> The code I ran was:
>>
>> x=rand(1000)
>> plot (x) # this pops up the normal gui, I tested Qt4Agg and GTKAgg
>>
>> paste() # this pasted the open figure into the IPython console.
>>
>> At this point, the plot in the window got that funny size, with the
>> x-labels double-drawn.  It seems as if the figure got re-drawn over
>> the previous canvas, at a different size.  If I resize the window, the
>> problem goes away, but if I don't resize it, it persists through new
>> plot/draw operations.
>>
>> The second problem... I then zoomed the interactive window and issued
>> paste again, getting the plot in the bottom right of the figure:
>> paste()
>>
>> And here the bug seems to be related to clipping: while the window
>> clips OK, the SVG seems not to.
>>
>> Is this a fundamental limitation of the SVG backend?
>>
>> For IPython we can also switch to pngs if that turns out to work
>> better, but I figured I'd report these...
>>
>> All this was done with current mpl from trunk.
>>      
> ------------------------------------------------------------------------------
> Start uncovering the many advantages of virtual appliances
> and start using them to simplify application deployment and
> accelerate your shift to cloud computing.
> http://p.sf.net/sfu/novell-sfdev2dev
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>    


-- 
Michael Droettboom
Science Software Branch
Space Telescope Science Institute
Baltimore, Maryland, USA

On Thu, Oct 14, 2010 at 10:05 AM, Robert Kern <rob...@gm...> wrote:

> On 10/14/10 8:57 AM, Benjamin Root wrote:
>
> > My only concern about that approach -- if I understand it correctly -- is
> that
> > this path clipping would have to be redone at every change in view
> limits.
> > Wouldn't that mean that in order to support panning and zooming, we would
> need
> > to keep an original copy of the SVG data and then create a clipped
> version of
> > the SVG data to post into the QT widget?
>
> Do you currently support panning and zooming just given the SVG file?
> That's
> quite a trick. ipythonqt certainly doesn't attempt that.
>
>
Sorry, you are right, I had a brain fart and wasn't thinking straight...
disregard my previous email.

Ben Root

On 10/14/10 8:57 AM, Benjamin Root wrote:

> My only concern about that approach -- if I understand it correctly -- is that
> this path clipping would have to be redone at every change in view limits.
> Wouldn't that mean that in order to support panning and zooming, we would need
> to keep an original copy of the SVG data and then create a clipped version of
> the SVG data to post into the QT widget?

Do you currently support panning and zooming just given the SVG file? That's 
quite a trick. ipythonqt certainly doesn't attempt that.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco

On Thu, Oct 14, 2010 at 6:04 AM, John Hunter <jd...@gm...> wrote:
> I believe the motivation here is in the separation of the ipython
> kernel process from the shell process, eg so the kernel can be running
> remotely and they can ship the image files across the wire.   Fernando
> can clarify though.

Indeed.

> Also, what was the motivation to go with svg by default rather than
> png -- that it is vector graphics?  I think the benefits of vector
> graphics over raster in an environment where you are not planning on
> interacting with the data (eg arbitrary zoom) are limited.  At least
> with PNG, you can rely on 100% correctness vis-a-vis mpl w/o worrying

Well, the client can save a session to html, svgs and all:

http://fperez.org/tmp/ipython-svg.xml

If the svg has extra metadata embedded, this will preserve it.  The
author of the html saving works in genomics at UCSF, and he embeds
extra info in his MPL svgs so that he can later annotate and
manipulate the generated SVGs.

So I think there's a good justification for wanting SVG.  However,
this limitation in Qt means we may want to make it at least optional,
so that people can use png when they want accurate rendering but
without access to the original vector figure, and SVG when they need
the original vector info (and are OK with Qt's limitations).

Does that make sense?

> about 3rd party rendering.  I also wonder if some of the ideas in the
> html5 canvas for embedding an interactive graphic into a remote client
> with socket communication can be applied to the ipython qt shell .

Certainly!  In fact a UC Berkeley student has already started working
on an html5 ipython frontend (that can work simultaneously with the Qt
one!), and his plan is to look into the html5 MPL back end for the
plotting.  Stay tuned.

Cheers,

f

On Thu, Oct 14, 2010 at 1:55 AM, Hans Meine <han...@gm...> wrote:
> Inkscape is an incredible piece of software, but maybe you should better test
> with e.g. Firefox, since there are *many* SVGs out there which only display
> fine in the almighty Inkscape.  (Could you attach the example SVG?  It looks
> simple enough to serve as a testcase.)

Attached.  Firefox shows the same as inkscape.

>
>> However, the problem seems to be that the Qt SVG renderer simply
>> doesn't support clipping paths, period.
>
> That's an unfortunate limitation, indeed. :-(

[...]

Thanks for the feedback.  As MD points out, perhaps bug isn't the
right word, just that Qt chose to target a simpler svg spec level than
what mpl uses.

Cheers,

f

On Thu, Oct 14, 2010 at 7:49 AM, Michael Droettboom <md...@st...> wrote:

> It looks like the QtSvg module is targeting SVG Tiny 1.2, which does not
> support clipping:
>
> http://doc.qt.nokia.com/4.3/qtsvg.html
>
> SVG Tiny also limits a lot of the CSS styling things that matplotlib
> makes use of -- though those things are probably easier to workaround if
> necessary.
>
> So, it's not broken so much as scoped below the threshold of usefulness ;)
>
> A possible workaround would be to add vector clipping to matplotlib and
> perform it before output.  This has the nice side benefit of reducing
> file sizes (in many cases).  However, that's a pretty major project to
> get right.
>
>
My only concern about that approach -- if I understand it correctly -- is
that this path clipping would have to be redone at every change in view
limits.  Wouldn't that mean that in order to support panning and zooming, we
would need to keep an original copy of the SVG data and then create a
clipped version of the SVG data to post into the QT widget?

Ben Root

On Thu, Oct 14, 2010 at 7:49 AM, Michael Droettboom <md...@st...> wrote:

> I haven't had a chance to look at the new Qt shell in ipython yet -- but
> is it necessary to go through a file?  Perhaps writing a native Qt4

I believe the motivation here is in the separation of the ipython
kernel process from the shell process, eg so the kernel can be running
remotely and they can ship the image files across the wire.   Fernando
can clarify though.

Also, what was the motivation to go with svg by default rather than
png -- that it is vector graphics?  I think the benefits of vector
graphics over raster in an environment where you are not planning on
interacting with the data (eg arbitrary zoom) are limited.  At least
with PNG, you can rely on 100% correctness vis-a-vis mpl w/o worrying
about 3rd party rendering.  I also wonder if some of the ideas in the
html5 canvas for embedding an interactive graphic into a remote client
with socket communication can be applied to the ipython qt shell .

JDH

It looks like the QtSvg module is targeting SVG Tiny 1.2, which does not 
support clipping:

http://doc.qt.nokia.com/4.3/qtsvg.html

SVG Tiny also limits a lot of the CSS styling things that matplotlib 
makes use of -- though those things are probably easier to workaround if 
necessary.

So, it's not broken so much as scoped below the threshold of usefulness ;)

A possible workaround would be to add vector clipping to matplotlib and 
perform it before output.  This has the nice side benefit of reducing 
file sizes (in many cases).  However, that's a pretty major project to 
get right.

I haven't had a chance to look at the new Qt shell in ipython yet -- but 
is it necessary to go through a file?  Perhaps writing a native Qt4 
backend (rather than the Qt4Agg one we currently have) would allow this 
to work better.  The Qt4 rendering infrastructure itself does seem to 
support clipping.

Mike

On 10/13/2010 07:36 PM, Fernando Perez wrote:
> Hi folks,
>
> [CC'ing mpl-dev just for reference so they know we're taking care of
> this on our side]
>
> I've been investigating further the bug where clipped paths in SVG
> render wrong in our console.  It turns out the mpl team already fixed
> some things on their side, and I can confirm that their SVGs now
> render fine in inkscape.  Here's an annotated example:
>
> http://i.imgur.com/NCSEJ.png
>
> However, the problem seems to be that the Qt SVG renderer simply
> doesn't support clipping paths, period.  Furthermore, they seem to
> have decided they won't fix this, as the bug was closed with "won't
> fix":
>
> http://bugreports.qt.nokia.com/browse/QTBUG-1865
>
> > From the Qt experts: am I misreading this?  Because if we're indeed
> stuck with a half-broken SVG renderer from qt, then we'll need to
> reconsider our implementation of pastefig(), to perhaps support an
> optional format flag so that users can send png if they prefer...
> Bummer.
>
> Cheers,
>
> f
>
> ------------------------------------------------------------------------------
> Beautiful is writing same markup. Internet Explorer 9 supports
> standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and DOM L2&  L3.
> Spend less time writing and  rewriting code and more time creating great
> experiences on the web. Be a part of the beta today.
> http://p.sf.net/sfu/beautyoftheweb
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>    


-- 
Michael Droettboom
Science Software Branch
Space Telescope Science Institute
Baltimore, Maryland, USA

Hi,

Op den Dunnersdag 14 Oktober 2010 Klock 01:36:52 hett Fernando Perez schreven:
> I've been investigating further the bug where clipped paths in SVG
> render wrong in our console.  It turns out the mpl team already fixed
> some things on their side, and I can confirm that their SVGs now
> render fine in inkscape.

Inkscape is an incredible piece of software, but maybe you should better test 
with e.g. Firefox, since there are *many* SVGs out there which only display 
fine in the almighty Inkscape.  (Could you attach the example SVG?  It looks 
simple enough to serve as a testcase.)

> However, the problem seems to be that the Qt SVG renderer simply
> doesn't support clipping paths, period.

That's an unfortunate limitation, indeed. :-(

> Furthermore, they seem to
> have decided they won't fix this, as the bug was closed with "won't
> fix":
> 
> http://bugreports.qt.nokia.com/browse/QTBUG-1865

In this older tracker item (before migration), you can see that the "Won't 
Fix" was later changed to "Deferred", now "Expired":

http://qt.nokia.com/developer/task-tracker/index_html?method=entry&id=204966

That at least does not look like Nokia's official policy is that this won't 
get fixed.

Anyhow, I am not so sure I fully understand the situation.  AFAICS, there are 
two separate issues here:
a) generation of the SVG
b) SVG rendering

Even if Qt's renderer does not support clipping, what does that have to do 
with a) (which is relevant for ipython export)?  Does MPL use Qt for SVG 
export?  ('Cause the tracker item is about QSvgGenerator, a QPaintDevice for 
*generating* SVGs.)

I wonder why QSvgGenerator is not fixed even without clipping support in Qt's 
SVG renderer.. I can understand if they say "we want to be able to render what 
we produce", but in this case this introduces seemingly unnecessary 
limitations.

(BTW: The viewbox will not help, this only sets a global SVG 'viewbox' 
property, which is not related to path clipping at all.)

Have a nice day,
  Hans

PS: Here are the current docs; I could not find a word about clipping in them:
  http://doc.qt.nokia.com/4.7/qtsvg.html
  http://doc.qt.nokia.com/4.7/qsvggenerator.html