Thread: [Matplotlib-users] Pgf Backend with Xelatex support

Hi,

When creating figures to be included in Latex documents I encountered a few
problems. In the end the text rendering just doesn't blend in well, one way
or another. I found that the problems can be fixed by using Xelatex, which
provides full unicode support and is able to use the installed fonts of your
operating system.

I wrote a new backend that uses the "pgf" latex package for drawing
matplotlib figures. It is compatible with pdflatex, xelatex and lualatex.
The pgf pictures can be included in latex documents or can be directly
compiled to PDF by the backend, utilizing the benefits of Xelatex.

The code for the backend and a script creating a test figure is on github:
https://github.com/pwuertz/matplotlib-backend-pgf/

A document that demonstrates the benefits of using pgf/xelatex is also
there:
https://github.com/pwuertz/matplotlib-backend-pgf/raw/master/demo/demo.pdf

Although I think the pgf backend is very useful already and produces figures
in publication quality (an overused expression ;) ), there are still some
loose ends. Basically, everything I need works but I don't have the time
anymore to figure out all the rest. Maybe someone is interested in improving
this backend, possibly making it a real option for the masses? I wrote down
all open questions I had in TODO comments within the code. To summarize
them:

* The default font for the backend is the unicode variant of Computer Modern
(CMU Serif), which might not be present on most users' systems. If you don't
want to install/use it, you can just specify another (see test script). I
could as well check for the fonts specified in the rc parameters but these
just do fit in Latex documents.

* When printing pgf commands, the actual font depends on the latex
environment you are embedding the figure in. Matplotlib only needs a font
for calculating the text positions and for direct PDF output.

* I'm not sure how certain draw methods of the renderer should behave due to
lack of documentation.

* Some text properties like switching font families or making the text
italic/bold are ignored since I did not need them.

* Backends like svg or pdf are able to display the document upon show(). I
don't know how this is achieved without creating a graphical user interface
myself. The other backends don't implement it.

* The method of obtaining the metrics of text elements is pretty cool I
think (XelatexManager), but it breaks easily since there is no way of
defining a timeout for reading the output of a subprocesses that keeps
running during the figure creation process. Right now, if Latex doesn't
understand a text-element the process stalls. An alternative is to run a new
latex process for every single text element or start using threads.

-- 
View this message in context: http://old.nabble.com/Pgf-Backend-with-Xelatex-support-tp34072290p34072290.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

On 06/26/2012 03:14 AM, Peter Würtz wrote:
> * Backends like svg or pdf are able to display the document upon show(). I
> don't know how this is achieved without creating a graphical user interface
> myself. The other backends don't implement it.

Not so--the non-interactive backends like svg and pdf do not do anything 
upon show(), so it sounds like nothing needs to be added to your pgf 
backend on this score.

Eric

efiring wrote:
> 
> Not so--the non-interactive backends like svg and pdf do not do anything 
> upon show(), so it sounds like nothing needs to be added to your pgf 
> backend on this score.
> 

Hmm right, when explicitly changing to non gui backends like "ps" or "pdf"
the show command is ignored too. I somehow thought that there must be an
implementation somewhere because I'll get a preview of a latex rendered
figure when using the text.usetex option.. but this option seems to trigger
a completely different rendering path anyways.

The funny thing is that when enabling text.usetex, which I'm not using in my
backend in any way, my PDF ends up with additional text elements I did not
create. I have no idea how matplotlib manages to tap into my PDF creation
process :D.

Anyway.. one less problem to worry about then :)
Thanks
-- 
View this message in context: http://old.nabble.com/Pgf-Backend-with-Xelatex-support-tp34072290p34074505.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

>
> Hi,
>
> When creating figures to be included in Latex documents I encountered a
> few
> problems. In the end the text rendering just doesn't blend in well, one
> way
> or another. I found that the problems can be fixed by using Xelatex, which
> provides full unicode support and is able to use the installed fonts of
> your
> operating system.
>
> I wrote a new backend that uses the "pgf" latex package for drawing
> matplotlib figures. It is compatible with pdflatex, xelatex and lualatex.
> The pgf pictures can be included in latex documents or can be directly
> compiled to PDF by the backend, utilizing the benefits of Xelatex.
>
> The code for the backend and a script creating a test figure is on github:
> https://github.com/pwuertz/matplotlib-backend-pgf/
>
> A document that demonstrates the benefits of using pgf/xelatex is also
> there:
> https://github.com/pwuertz/matplotlib-backend-pgf/raw/master/demo/demo.pdf
>
> Although I think the pgf backend is very useful already and produces
> figures
> in publication quality (an overused expression ;) ), there are still some
> loose ends. Basically, everything I need works but I don't have the time
> anymore to figure out all the rest. Maybe someone is interested in
> improving
> this backend, possibly making it a real option for the masses? I wrote
> down
> all open questions I had in TODO comments within the code. To summarize
> them:
>
> * The default font for the backend is the unicode variant of Computer
> Modern
> (CMU Serif), which might not be present on most users' systems. If you
> don't
> want to install/use it, you can just specify another (see test script). I
> could as well check for the fonts specified in the rc parameters but these
> just do fit in Latex documents.
>
> * When printing pgf commands, the actual font depends on the latex
> environment you are embedding the figure in. Matplotlib only needs a font
> for calculating the text positions and for direct PDF output.
>
> * I'm not sure how certain draw methods of the renderer should behave due
> to
> lack of documentation.
>
> * Some text properties like switching font families or making the text
> italic/bold are ignored since I did not need them.
>
> * Backends like svg or pdf are able to display the document upon show(). I
> don't know how this is achieved without creating a graphical user
> interface
> myself. The other backends don't implement it.
>
> * The method of obtaining the metrics of text elements is pretty cool I
> think (XelatexManager), but it breaks easily since there is no way of
> defining a timeout for reading the output of a subprocesses that keeps
> running during the figure creation process. Right now, if Latex doesn't
> understand a text-element the process stalls. An alternative is to run a
> new
> latex process for every single text element or start using threads.
>

Looks great! How do I use this with my currently installed matplotlib 1.1.0?

Cheers,
A.

Andreas Hilboll wrote:
> 
>> I wrote a new backend that uses the "pgf" latex package for drawing
>> matplotlib figures. It is compatible with pdflatex, xelatex and lualatex.
>> The pgf pictures can be included in latex documents or can be directly
>> compiled to PDF by the backend, utilizing the benefits of Xelatex.
>>
>> The code for the backend and a script creating a test figure is on
>> github:
>> https://github.com/pwuertz/matplotlib-backend-pgf/
> 
> Looks great! How do I use this with my currently installed matplotlib
> 1.1.0?
> 

The easiest way without touching your matplotlib installation is to fetch
"backend_pgf.py" from github and put it in the directory where you are
running your plotting script (or in any directory in python's search path).
Then have a look at "test_pgf_backend.py". It demonstrates how to select and
configure the pgf backend. Basically, you just call

matplotlib.use('module://backend_pgf')

to use it. With the backend selected, the show() command won't work anymore
since this is not a GUI backend, but you can now save the figure as ".pgf"
file (just the pgf commands for latex) or ".pdf" file (already compiled with
xelatex). Also make sure that you select a font that exists on your system

matplotlib.rcParams.update({"pgf.font": "CMU Serif"})

or you'll get an exception when saving the figure. As already said, you'll
get the best results when you install the Computer Modern Unicode Fonts
(Ubuntu Package: fonts-cmu, Manual Install:
http://sourceforge.net/projects/cm-unicode/). Be aware that if there is any
text element that produces an error in Latex the savefig call will stall
until I have found a better way to deal with such errors.


Incorporating the backend into matplotlib requires a little more work. You
have to copy the file to the folder where all the other modules reside (for
me it's /usr/lib/pymodules/python2.7/matplotlib/backends/) and then include
the "pgf" type in "backend_bases.py" one folder above. I'm not sure how to
handle the case when the user wants to create a pdf from backend_pgf instead
of backend_pdf since the only 'switch' is the extension of the file. But I'd
suggest to use the matplotlib.use method for now..

Good luck ;)
-- 
View this message in context: http://old.nabble.com/Pgf-Backend-with-Xelatex-support-tp34072290p34082447.html
Sent from the matplotlib - users mailing list archive at Nabble.com.