Thread: [matplotlib-devel] backends

John,

All this brings to mind something I wanted to bring up anyway: we have a 
proliferation of backends, and occasional requests for more--are there 
any we can simply drop now, or soon?  For example, gd? And what are the 
prospects that several backends could be consolidated via cairo?  And/or 
via more extensive subclassing?

I am worried that we are increasingly going to get into trouble with 
lack of up-to-date support for all these backends.  Major improvements 
sometimes depend on backend changes, and the more backends we have that 
lack active maintainers, the harder this gets; and the worse for users, 
as features work on one backend but break on another.

Eric

John Hunter wrote:
>>>>>>"Eric" == Eric Firing <ef...@ha...> writes:
> 
> 
>     Eric> Martin, When I try your example with svn matplotlib, I get a
>     Eric> 34 MB eps file, and looking at it, I don't see much room for
>     Eric> making it smaller--there is one obvious optimization,
>     Eric> abbreviating "marker", but that's it.  (The svg file is 456
>     Eric> MB!)  So, maybe some major optimization has already been
>     Eric> done between mpl 0.87.2 and svn.
> 
> Yep, Darren got "draw_markers" properly implemented for backend PS.
> This function is much better in time and space; I believe only *Agg
> and PS implement it, but it could be ported over to SVG fairly easily
> by modifying the PS implementation.
> 
>     Eric> The bigger problem is that each file format has basic
>     Eric> characteristics and limitations.  If you draw a million
>     Eric> markers and line segments, you are inevitably going to have
>     Eric> a big postscript file, unless the postscript backend somehow
>     Eric> detects the fact that almost all of your points are
>     Eric> indistinguishable and therefore deletes most of them--and
>     Eric> this is really asking too much of a plotting backend, I
> 
> Agg does this for draw_lines -- it drops points in the path that are
> less one pixel away from the previous point.  
> 
> JDH

>>>>> "Eric" == Eric Firing <ef...@ha...> writes:

    Eric> John, All this brings to mind something I wanted to bring up
    Eric> anyway: we have a proliferation of backends, and occasional
    Eric> requests for more--are there any we can simply drop now, or
    Eric> soon?  For example, gd? And what are the prospects that
    Eric> several backends could be consolidated via cairo?  And/or
    Eric> via more extensive subclassing?

    Eric> I am worried that we are increasingly going to get into
    Eric> trouble with lack of up-to-date support for all these
    Eric> backends.  Major improvements sometimes depend on backend
    Eric> changes, and the more backends we have that lack active
    Eric> maintainers, the harder this gets; and the worse for users,
    Eric> as features work on one backend but break on another.

It's a concern, and I have no problem dropping some (gd is a good
example, it was really just a proof-of-concept backend to prove that
we could mix and match GUIs with pure image backends as I was
developing agg).  A number of backends are incomplete and get little
attention but do little harm either -- in my opinion they are
unsupported.  paint and emf is another example of a backend that noone uses
except for the authors.  I think almost noone uses gdk, gtk, cairo or
gtkcairo, but I hesitate to pull these because Steve Chaplin, the GTK
maintainer, wrote them, uses them, and has done a great job with GTK.

Ideally, I would like to see PS, SVG, Agg and [Tk|GTK|WX|Qt|FLTK]Agg
and no more.  But I know that other people feel differently.

Taking the long view, enthought's kiva/chaco uses the pdf drawing
model, which is a nice model and superior to the matplotlib GTK
model.  I've talked with Eric Jones many times about the desirability
of sharing a single low-level drawing model, probably based on
PDF/KIVA, that have backends for PS, SVG, PDF and one raster format
(eg Agg or Cairo).  But nothing has ever happened here because we both
have lots to do and a lot of code that depends on what is already in
place.

JDH

On Fri, Jul 07, 2006 at 03:46:01PM -0500, John Hunter wrote:
> Ideally, I would like to see PS, SVG, Agg and [Tk|GTK|WX|Qt|FLTK]Agg
> and no more.  But I know that other people feel differently.

    pdf seems very important to me.

	Just my two cents,

    Ga=EBl

>>>>> "Ga=EBl" =3D=3D Ga=EBl Varoquaux <gae...@no...> wr=
ites:

    Ga=EBl> On Fri, Jul 07, 2006 at 03:46:01PM -0500, John Hunter wrote:
    >> Ideally, I would like to see PS, SVG, Agg and
    >> [Tk|GTK|WX|Qt|FLTK]Agg and no more.  But I know that other
    >> people feel differently.

    Ga=EBl>     pdf seems very important to me.

PDF is certainly an important document format, but it doesn't seem to
be widely used for figures.  I often make pdf documents but
incorporate PNG figures into them (eg PDF latex).  My backend list
above is not a list of backends that I think matplotlib should support
as much as it is a list of backends that I personally find useful and
would personally support even if noone else did.  Any backend that is
supported by someone should remain in, even if they are the only one
using it, in my opinion.  But most backends are not supported.

Only PS and Agg (and by extension all of the GTK* backends) support
all of matplotlib's features.  This should give some indication of how
much work it is to develop a fully compliant backend, and why we have
an incentive to minimize the number of them.

JDH=20

On 7/7/06, John Hunter <jdh...@ac...> wrote:
> >>>>> "Ga=EBl" =3D=3D Ga=EBl Varoquaux <gae...@no...> wr=
ites:
>
>     Ga=EBl> On Fri, Jul 07, 2006 at 03:46:01PM -0500, John Hunter wrote:
>     >> Ideally, I would like to see PS, SVG, Agg and
>     >> [Tk|GTK|WX|Qt|FLTK]Agg and no more.  But I know that other
>     >> people feel differently.
>
>     Ga=EBl>     pdf seems very important to me.
>
> PDF is certainly an important document format, but it doesn't seem to
> be widely used for figures.  I often make pdf documents but
> incorporate PNG figures into them (eg PDF latex).  My backend list
> above is not a list of backends that I think matplotlib should support
> as much as it is a list of backends that I personally find useful and
> would personally support even if noone else did.  Any backend that is
> supported by someone should remain in, even if they are the only one
> using it, in my opinion.  But most backends are not supported.
>
> Only PS and Agg (and by extension all of the GTK* backends) support
> all of matplotlib's features.  This should give some indication of how
> much work it is to develop a fully compliant backend, and why we have
> an incentive to minimize the number of them.

PDF has one important combination that neither PS nor png provide:
vector (hence, resolution independent) images with proper
transparency.  I know SVG has that, but I don't know how well SVGs
embed in PDFs (last I tried, I couldn't make it work satisfactorily).

Cheers,

f

On Friday 07 July 2006 17:16, John Hunter wrote:
> >>>>> "Ga=EBl" =3D=3D Ga=EBl Varoquaux <gae...@no...> wr=
ites:
>
>     Ga=EBl> On Fri, Jul 07, 2006 at 03:46:01PM -0500, John Hunter wrote:
>     >> Ideally, I would like to see PS, SVG, Agg and
>     >> [Tk|GTK|WX|Qt|FLTK]Agg and no more.  But I know that other
>     >> people feel differently.
>
>     Ga=EBl>     pdf seems very important to me.
>
> PDF is certainly an important document format, but it doesn't seem to
> be widely used for figures.  I often make pdf documents but
> incorporate PNG figures into them (eg PDF latex).

I would prefer to use pdf figures in my latex documents. They are smaller,=
=20
they support alpha blending, and unlike png they are scalable. Unfortunatel=
y,=20
most academic journals still request postscript files, but this may change=
=20
some day. Printers may eventually use pdf as their native format instead of=
=20
postscript.

Not that I am advocating adding more backends to mpl, just more=20
contributors. :)

On Jul 7, 2006, at 5:16 PM, John Hunter wrote:

> PDF is certainly an important document format, but it doesn't seem to
> be widely used for figures.

I only use PDF figures.  They are useful both in pdflatex (how I  
write), and play nice with all of the other mac utilities, in  
particular Keynote (how I present).  They are generally small, and  
good looking at any size because of the vector graphics.

When converting MPL figures to pdf I usually just export to eps, then  
convert using ghostscript.  In fact, if you open an eps file, OS X  
will do the conversion automagically, and open your new pdf in  
Preview.  For publication quality (with annotations, etc), I export  
to eps, edit in Illustrator, and eventually export to pdf.

I'm not suggesting immediate development for a PDF backend (although  
if it came automatically with kiva, I would use it), I think eps 
+ghostscript is good enough for now.  However, general pdf support is  
an important issue for me.

-Rob

>>>>> "Robert" == Robert Hetland <rhe...@ma...> writes:

    Robert> I'm not suggesting immediate development for a PDF backend
    Robert> (although if it came automatically with kiva, I would use
    Robert> it), I think eps +ghostscript is good enough for now.

As you may know, we have a prototype of a PDF backend, though it is
missing many features.

JDH

John Hunter wrote:
> I've talked with Eric Jones many times about the desirability
> of sharing a single low-level drawing model, probably based on
> PDF/KIVA, that have backends for PS, SVG, PDF and one raster format
> (eg Agg or Cairo). 

Just as a note, if you have Cairo, you have PS, PDF, SVG, raster, and, 
indeed, native draw-to-the-screen functionality.

They're not all robust and complete, but it's seeing a lot of 
development. A robust, numpy-aware PyCairo would be really nice.

In short, Cairo has essentially the same goals (plus a few more) as 
Kiva, except it's written in C. There's some discussion of using it in 
wxWidgets, and, of course, it's already used in newer GTKs.

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

>>>>> "Christopher" == Christopher Barker <Chr...@no...> writes:
    Christopher> Just as a note, if you have Cairo, you have PS, PDF,
    Christopher> SVG, raster, and, indeed, native draw-to-the-screen
    Christopher> functionality.

Last I checked if I recall correctly, about a year ago, the PS backend
merely dumped a raster image and didn't do real vector graphics.  Has
this changed?

JDH

>>>>> "John" == John Hunter <jdh...@ac...> writes:

    John> Last I checked if I recall correctly, about a year ago, the
    John> PS backend merely dumped a raster image and didn't do real
    John> vector graphics.  Has this changed?

Looks like it has:

  http://lwn.net/Articles/189905

John Hunter <jdh...@ac...> writes:

> As you may know, we have a prototype of a PDF backend, though it is
> missing many features.

I've been offline for a while (and will be for at least a week more),
but I hacked a bit more on the PDF backend and just did a commit. Now
it tries to support images and mathtext, but both are incomplete. 
Here's what I think is currently missing:

 * the alpha channel of images is not used (I think this is easy to do)
 * image compression could be improved (PDF supports png-like compression)
 * there are encoding problems in text, especially in mathtext 
   (e.g. \alpha doesn't show up)
 * mathtext positioning seems a bit off
 * no unicode support in text (or mathtext)
 * no Type 1 or Base-14 font support (i.e., "pdf.use_afm")
 * TTF support has lots of small TODOs, e.g. how do you know if a font 
   is serif/sans-serif, or symbolic/non-symbolic?
 * draw_markers, draw_line_collection, etc.
 * use_tex

Perhaps now would be a good time for anyone interested to give the PDF
backend a spin around the block and tell me about any other bugs or
missing features, so when I next do a bout of backend_pdf hacking,
I'll know what people's priorities are.

-- 
Jouni

    This question triggers another one from myself (that was raised by
colleagues).

    I know that there is some traits lying in mpl. Will there be one day
some traitsUI code too, to generate GUI to modify properties of objects
one the display ? This is fully related to backends.
    I find that this is one of the most requested features for mpl by my
colleagues (mind you, not by myself).

        Ga=EBl