matplotlib-devel Mailing List for matplotlib (Page 7)

I just reworked the examples directory to make it more approachable
and hopefully more usable.  The README is included below.  As you have
time, please add more examples ot the "api" directory either by
translating existing pylab examples or writing new ones.  This has
been a long standing complaint among users that it is difficult to get
going with the API, so more examples will help.  backend_driver has
been reloaded to a tests dir.


matplotlib examples
===================

There are a variety of ways to use matplotlib, and most of them are
illustrated in the examples in this directory.

Probably the most common way people use matplotlib is with the
procedural interface, which follows the matlab/IDL/mathematica
approach of using simple proceedures like "plot" or "title" to modify
the current figure.  These examples are included in the "pylab"
directory.  If you want to write more robust scripts, eg for
production use or in a web application server, you will probably want
to use the matplotlib API for full control.  These examples are found
in the "api" directory.  Below is a brief description of the different
directories found here:

  * animation - dynamic plots, see the tutorial at
    http://www.scipy.org/Cookbook/Matplotlib/Animations

  * api - working with the matplotlib API directory.  See the
    doc/artist_api_tut.txt in the matplotlib src directory ((PDF at
    http://matplotlib.sf.net/pycon)

  * data - some data files we use in the examples

  * event_handling - how to interact with your figure, mouse presses,
    key presses, object picking, etc.  See the event handling tutorial
    in the "doc" directory of the source distribution (PDF at
    http://matplotlib.sf.net/pycon)

  * misc - some miscellaneous examples.  some demos for loading and
    working with record arrays

  * pylab - the proceedural interface to matplotlib similar to matlab

  * tests - tests used by matplotlib developers to check functionality

  * units - working with unit data an custom types in matplotlib

  * user_interfaces - using matplotlib in a GUI application, eg
    TkInter, WXPython, pygtk, pyqt or FLTK widgets

Erik Tollerud wrote:
> It'd be nice to get this into the new release unless it has been
> somehow fixed by some other code path in the latest svn... the line
> numbers on the diff are no longer accurate, but the idea is still the
> same - just change all lines of the form
> 
> caplines.extend( self.plot(leftlo, ylo,ls=None,
> marker=mlines.CARETLEFT, **plot_kw) )
> to
> caplines.extend( self.plot(leftlo, ylo, 'k|', marker=mlines.CARETLEFT,
> **plot_kw) )
> 
> e.g. change the ls=None to 'k|' and the color cycle is not
> interrupted.  I don't have the 1.1 numpy yet, so I haven't explicitly
> tested it, but I don't see why anything would have changed...
> 
> On Tue, Feb 12, 2008 at 10:46 PM, Erik Tollerud <eri...@gm...> wrote:
>> After a little testing on the subversion repository, the attached diff
>> seems to work.
>>
>> On Feb 10, 2008 12:12 AM, Erik Tollerud <eri...@gm...> wrote:
>>> I noticed while making some plots with upper bound error bars that
>>> whenever Axes.errorbars is called with any of the errorbars chosen as
>>> upper or lower bounds, that the color cycle was off, skipping over 2
>>> colors each time another errorbar plot was made (e.g. the first would
>>> be green and the second would be cyan instead of blue,green,red,cyan).
>>> Looking into the axes class, it appears that if you don't specify a
>>> color and want the markers to be drawn over the errorbars, the color
>>> cycle has already been skipped over one because of calls to draw the
>>> markers into the plot.  The solution is to change all the lines that
>>> say " ls='None' " in the errorbar method to instead say " ls='k' " -
>>> this will prevent those calls from cycling the colors, and hence only
>>> the call to actually draw the markers will do so.  Can this patch be
>>> committed to svn?
>>>
>>
>>
>> --
>> Erik Tollerud
>> Graduate Student
>> Center For Cosmology
>> Department of Physics and Astronomy
>> 4155B Frederick Reines Hall
>> University of California, Irvine
>> Office Phone: (949)824-2996
>> Cell: (651)307-9409
>> eto...@uc...
>>
> 
> 
> 

Hi Erik,

   I don't see the color cycle broken in the latest svn version (but it 
might be broken in the 0.91 maintenance branch).

Manuel

On Thu, May 15, 2008 at 2:22 PM, Michael Droettboom <md...@st...> wrote:

>> OK, I just committed a change using scanf with type %lu.  tkagg svn
>> trunk users should test on any platform they have access to.
>
> Once we're confident in the fix, I'll backport this to 0.91.x, as it exists
> there also.
>>
>> I can
>> try OS X tonight.

So I have three successful tests to date

OSX: 10.5 x86 w/  gcc i686-apple-darwin9-gcc-4.0.1

Linux:x86_64 x86_64 x86_64 GNU/Linux w/ gcc (GCC) 4.1.2 20070925

Solaris x86: SunOS flag 5.10 Generic_118855-15 i86pc i386 i86pc gcc (GCC) 3.4.1

Still a limited sample since they are all gcc and all x86.  I'm having
some trouble compiling on my OSX 10.4 ppc but I don't think it is
related to these changes.  If I get some time I'll try that too, but
in the meantime if anyone has a mpl svn build on a ppc and can give
the tkagg backend a test, that would be helpful.

JDH

Johann Cohen-Tanugi wrote:
> hello,
> is there an example in the distribution that shows these new features?

I just added an example to the trunk, see examples/histogram_demo_step.py

> How about the idea to allow for an option to get cumulative histograms, 
> that sounded a very nice idea....

   I also added the keyword 'cumulative' to the axes hist() method. 
Actually, in the current version, if cumulative=True AND normed=True the 
cumulative histogram is normed to 1, which seemed to be most convenient 
to me (rather than 1/binwidths which is what numpy.hist actually does).

Manuel

> thanks,
> Johann
> 
> Manuel Metz wrote:
>> Eric Firing wrote:
>>   
>>> John Hunter wrote:
>>>     
>>>> On Wed, Apr 2, 2008 at 6:39 PM, Erik Tollerud <eri...@gm...> wrote:
>>>>
>>>>       
>>>>>  are slightly different).  There's a slight compatibility issue in that
>>>>>  as it stands in that the returned tuple now has 4 values (I added a
>>>>>  list of the lines that are generated if the steps command is used),
>>>>>  but I can't really imagine how that could break anything but the
>>>>>  poorest-written code...
>>>>>         
>>>> I'm not sure I understand this: won't it break all code written like:
>>>>
>>>>   n, bins, patches = ax.hist(x, 50, normed=1)
>>>>
>>>> which is the code presented in the histogram example and a fairly
>>>> common approach.  I don't see this as an example of the "poorest
>>>> written code".    I am inclined to not break this call signature
>>>> unless the lines are actually used, ie 'step' in histtype.  On a quick
>>>> read of the code, you either get lines or patches but not both, so how
>>>> about
>>>>
>>>>     n, bins, patches = ax.hist(x, 50, normed=1)
>>>>
>>>> or
>>>>
>>>>    n, bins, lines = ax.hist(x, 50, normed=1, histtype='lines')
>>>>       
>>> That was my first reaction also, but the proposed "stepfill" option 
>>> yields a bunch of bar patches *and* a line.  The solution may be to 
>>> accomplish "stepfill" with two separate calls, or to have 4 outputs only 
>>> in the "stepfill" case.  Or, with sufficient rewriting I think the 
>>> "stepfill" case could yield a single patch and a single line, and the 
>>> third output in this case could be a single corresponding 2-element 
>>> tuple or list.  That is, the third output is considered simply a list of 
>>> artists.  Now I will stop speculating and leave it to Manuel to sort out.
>>>
>>> Eric
>>>     
>> I have just committed a patch to add the histogram step functionality. I 
>>   took Erik Tollerud's patch as basis, but basically re-wrote it 
>> completely in the end ...
>>
>>    The advantages are: (i) considerably smaller code and (ii) return 
>> values are unchanged, ie.
>>
>>    n, bins, patches = ax.hist(x, 50)
>>    n, bins, patches = ax.hist(x, 50, histtype='step')
>>
>> In contrast to the original patch, histtype='step' is filled and to 
>> produce a non-filled histogram, one has to use facecolor='None'.
>>
>> Hope I haven't overlooked anything or broken other code ;-)
>>
>> Manuel
>>
>> -------------------------------------------------------------------------
>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
>> Don't miss this year's exciting event. There's still time to save $100. 
>> Use priority code J8TL2D2. 
>> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
>> _______________________________________________
>> Matplotlib-devel mailing list
>> Mat...@li...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>   
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft 
> Defy all challenges. Microsoft(R) Visual Studio 2008. 
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

On Fri, May 16, 2008 at 7:18 AM, Michael Droettboom <md...@st...> wrote:
> Yeah.  Those all use the SWIG wrapper to Agg that we ultimately decided not to use.  I think clearing out the examples is probably a
> good idea -- though moving them to some part of SVN (outside of the main trunk) might be a good idea so we don't "lose" them.
> The SWIG wrapper represents a lot of work and may be useful in the future.

I removed these from the trunk since they are already in the maintenance branch.

On the subject of examples, one thing I'd like to do is organize them a bit

  - "pylab" - all the current examples that do "from pylab import *"
with a README indicating that while this idiom is fine for interactive
use, for scripts or anything worth keepking, we recommend pyplot

  - "pyplot" - as we get time, rewrite the examples from pylab using
the pyplot and numpy namespaces, and emphasizing the API, eg

      import numpy as np
      import matplotlib.pyplot as plt\
      fig = plt.figure()
      ax = fig.add_subplot(111)
      ax.plot(np.random.rand(10))
      ax.set_title('a simple plot')
      plt.show()

  - "applications" or "user_interfaces" or something like that.  All
the UI examples

  - animations -  example code doing dynamic/animated plotting.  this
overlaps a bit wtih user interfaces since the recommended way    to do
animations is with the GUI event loop

  - event handling -  interacting with the plot, mouse and key events, picking

  - widgets - the mpl widget examples

Does anyone object to me doing this, or would like to see it done otherwise?

JDH

Yeah.  Those all use the SWIG wrapper to Agg that we ultimately decided not to use.  I think clearing out the examples is probably a good idea -- though moving them to some part of SVN (outside of the main trunk) might be a good idea so we don't "lose" them.  The SWIG wrapper represents a lot of work and may be useful in the future.

Cheers,
Mike

Four examples fail because they depend on matplotlib.agg, which cannot 
be imported:

In [1]:import matplotlib.agg
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)

/home/efiring/programs/py/mpl/mpl_trunk/examples/<ipython console> in 
<module>()

/usr/local/lib/python2.5/site-packages/matplotlib/agg.py in <module>()
       5 # This file is compatible with both classic and new-style classes.
       6
----> 7 import _agg
       8 import new
       9 new_instancemethod = new.instancemethod

ImportError: No module named _agg

The examples are:

agg_resize.py
agg_test.py
clippath_test.py
glyph_to_path.py

I suspect this is all indicative of some obsolete components that it 
might be worthwhile to clean out.

Eric

John Hunter wrote:
> On Thu, May 15, 2008 at 2:09 PM, Michael Droettboom <md...@st...> wrote:
>   
>> Oh joy!  Maybe we should try (as Eric suggested) sscanf instead.
>>     
>
> OK, I just committed a change using scanf with type %lu.  tkagg svn
> trunk users should test on any platform they have access to.  
Once we're confident in the fix, I'll backport this to 0.91.x, as it 
exists there also.
> I can
> try OS X tonight.
>   
Thanks.

Mike

-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

On Thu, May 15, 2008 at 2:09 PM, Michael Droettboom <md...@st...> wrote:
> Oh joy!  Maybe we should try (as Eric suggested) sscanf instead.

OK, I just committed a change using scanf with type %lu.  tkagg svn
trunk users should test on any platform they have access to.  I can
try OS X tonight.

JDH

Oh joy!  Maybe we should try (as Eric suggested) sscanf instead.

Cheers,
Mike

John Hunter wrote:
> On Thu, May 15, 2008 at 8:08 AM, Michael Droettboom <md...@st...> wrote:
>   
>> I recently committed a fix (courtesy of Malte Marquarding) for segfaults
>> in the Tk backend.  It seems it was converting a string of digits to a
>> "signed long" and then casting to a pointer.  Unfortunately, it really
>> needs to be an "unsigned long" or overflow may occur.  Since the C
>> standard library doesn't provide a version of "atol" for "unsigned
>> long", Malte's solution was to use C++ stringstreams.  There's a good
>> chance that is portable, but it should probably be tested on Visual
>> Studio for good measure.  Would any of you kind Windows folks be willing
>> to compile SVN trunk and open a plot with the TkAgg backend to verify this?
>>     
>
> Unfortunately, this change in segfaulting tkagg on solaris
>
> johnh@flag:~> uname -a
> SunOS flag 5.10 Generic_118855-15 i86pc i386 i86pc
> johnh@flag:~> gcc --version
> gcc (GCC) 3.4.1
>
>
> BUILDING MATPLOTLIB
>             matplotlib: 0.98pre
>                 python: 2.4.5 (#4, Apr 12 2008, 09:09:16)  [GCC 3.4.1]
>               platform: sunos5
>
> REQUIRED DEPENDENCIES
>                  numpy: 1.2.0.dev5136
>              freetype2: found, but unknown version (no pkg-config)
>                         * WARNING: Could not find 'freetype2' headers in any
>                         * of '/usr/local/include', '.',
>                         * '/usr/local/include/freetype2', './freetype2'.
>
> OPTIONAL BACKEND DEPENDENCIES
>                 libpng: found, but unknown version (no pkg-config)
>                         * Could not find 'libpng' headers in any of
>                         * '/usr/local/include', '.'
>                Tkinter: Tkinter: 39220, Tk: 8.4, Tcl: 8.4
>               wxPython: no
>                         * wxPython not found
>                   Gtk+: gtk+: 2.10.11, glib: 2.12.11, pygtk: 2.10.4,
>                         pygobject: 2.12.3
>                     Qt: no
>                    Qt4: no
>                  Cairo: 1.4.0
>   

-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

On Thu, May 15, 2008 at 8:08 AM, Michael Droettboom <md...@st...> wrote:
> I recently committed a fix (courtesy of Malte Marquarding) for segfaults
> in the Tk backend.  It seems it was converting a string of digits to a
> "signed long" and then casting to a pointer.  Unfortunately, it really
> needs to be an "unsigned long" or overflow may occur.  Since the C
> standard library doesn't provide a version of "atol" for "unsigned
> long", Malte's solution was to use C++ stringstreams.  There's a good
> chance that is portable, but it should probably be tested on Visual
> Studio for good measure.  Would any of you kind Windows folks be willing
> to compile SVN trunk and open a plot with the TkAgg backend to verify this?

Unfortunately, this change in segfaulting tkagg on solaris

johnh@flag:~> uname -a
SunOS flag 5.10 Generic_118855-15 i86pc i386 i86pc
johnh@flag:~> gcc --version
gcc (GCC) 3.4.1


BUILDING MATPLOTLIB
            matplotlib: 0.98pre
                python: 2.4.5 (#4, Apr 12 2008, 09:09:16)  [GCC 3.4.1]
              platform: sunos5

REQUIRED DEPENDENCIES
                 numpy: 1.2.0.dev5136
             freetype2: found, but unknown version (no pkg-config)
                        * WARNING: Could not find 'freetype2' headers in any
                        * of '/usr/local/include', '.',
                        * '/usr/local/include/freetype2', './freetype2'.

OPTIONAL BACKEND DEPENDENCIES
                libpng: found, but unknown version (no pkg-config)
                        * Could not find 'libpng' headers in any of
                        * '/usr/local/include', '.'
               Tkinter: Tkinter: 39220, Tk: 8.4, Tcl: 8.4
              wxPython: no
                        * wxPython not found
                  Gtk+: gtk+: 2.10.11, glib: 2.12.11, pygtk: 2.10.4,
                        pygobject: 2.12.3
                    Qt: no
                   Qt4: no
                 Cairo: 1.4.0

On Thu, May 15, 2008 at 8:08 AM, Michael Droettboom <md...@st...> wrote:
> I recently committed a fix (courtesy of Malte Marquarding) for segfaults
> in the Tk backend.  It seems it was converting a string of digits to a
> "signed long" and then casting to a pointer.  Unfortunately, it really
> needs to be an "unsigned long" or overflow may occur.  Since the C
> standard library doesn't provide a version of "atol" for "unsigned
> long", Malte's solution was to use C++ stringstreams.  There's a good
> chance that is portable, but it should probably be tested on Visual
> Studio for good measure.  Would any of you kind Windows folks be willing
> to compile SVN trunk and open a plot with the TkAgg backend to verify this?

Charlie -- as far as I know you are the only developer who has win32
setup to build from svn.  Can you test this when you get a chance?

Thanks,
JDH

I recently committed a fix (courtesy of Malte Marquarding) for segfaults 
in the Tk backend.  It seems it was converting a string of digits to a 
"signed long" and then casting to a pointer.  Unfortunately, it really 
needs to be an "unsigned long" or overflow may occur.  Since the C 
standard library doesn't provide a version of "atol" for "unsigned 
long", Malte's solution was to use C++ stringstreams.  There's a good 
chance that is portable, but it should probably be tested on Visual 
Studio for good measure.  Would any of you kind Windows folks be willing 
to compile SVN trunk and open a plot with the TkAgg backend to verify this?

Cheers,
Mike

-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

hello,
is there an example in the distribution that shows these new features? 
How about the idea to allow for an option to get cumulative histograms, 
that sounded a very nice idea....
thanks,
Johann

Manuel Metz wrote:
> Eric Firing wrote:
>   
>> John Hunter wrote:
>>     
>>> On Wed, Apr 2, 2008 at 6:39 PM, Erik Tollerud <eri...@gm...> wrote:
>>>
>>>       
>>>>  are slightly different).  There's a slight compatibility issue in that
>>>>  as it stands in that the returned tuple now has 4 values (I added a
>>>>  list of the lines that are generated if the steps command is used),
>>>>  but I can't really imagine how that could break anything but the
>>>>  poorest-written code...
>>>>         
>>> I'm not sure I understand this: won't it break all code written like:
>>>
>>>   n, bins, patches = ax.hist(x, 50, normed=1)
>>>
>>> which is the code presented in the histogram example and a fairly
>>> common approach.  I don't see this as an example of the "poorest
>>> written code".    I am inclined to not break this call signature
>>> unless the lines are actually used, ie 'step' in histtype.  On a quick
>>> read of the code, you either get lines or patches but not both, so how
>>> about
>>>
>>>     n, bins, patches = ax.hist(x, 50, normed=1)
>>>
>>> or
>>>
>>>    n, bins, lines = ax.hist(x, 50, normed=1, histtype='lines')
>>>       
>> That was my first reaction also, but the proposed "stepfill" option 
>> yields a bunch of bar patches *and* a line.  The solution may be to 
>> accomplish "stepfill" with two separate calls, or to have 4 outputs only 
>> in the "stepfill" case.  Or, with sufficient rewriting I think the 
>> "stepfill" case could yield a single patch and a single line, and the 
>> third output in this case could be a single corresponding 2-element 
>> tuple or list.  That is, the third output is considered simply a list of 
>> artists.  Now I will stop speculating and leave it to Manuel to sort out.
>>
>> Eric
>>     
>
> I have just committed a patch to add the histogram step functionality. I 
>   took Erik Tollerud's patch as basis, but basically re-wrote it 
> completely in the end ...
>
>    The advantages are: (i) considerably smaller code and (ii) return 
> values are unchanged, ie.
>
>    n, bins, patches = ax.hist(x, 50)
>    n, bins, patches = ax.hist(x, 50, histtype='step')
>
> In contrast to the original patch, histtype='step' is filled and to 
> produce a non-filled histogram, one has to use facecolor='None'.
>
> Hope I haven't overlooked anything or broken other code ;-)
>
> Manuel
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
> Don't miss this year's exciting event. There's still time to save $100. 
> Use priority code J8TL2D2. 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>   

It'd be nice to get this into the new release unless it has been
somehow fixed by some other code path in the latest svn... the line
numbers on the diff are no longer accurate, but the idea is still the
same - just change all lines of the form

caplines.extend( self.plot(leftlo, ylo,ls=None,
marker=mlines.CARETLEFT, **plot_kw) )
to
caplines.extend( self.plot(leftlo, ylo, 'k|', marker=mlines.CARETLEFT,
**plot_kw) )

e.g. change the ls=None to 'k|' and the color cycle is not
interrupted.  I don't have the 1.1 numpy yet, so I haven't explicitly
tested it, but I don't see why anything would have changed...

On Tue, Feb 12, 2008 at 10:46 PM, Erik Tollerud <eri...@gm...> wrote:
> After a little testing on the subversion repository, the attached diff
> seems to work.
>
> On Feb 10, 2008 12:12 AM, Erik Tollerud <eri...@gm...> wrote:
>> I noticed while making some plots with upper bound error bars that
>> whenever Axes.errorbars is called with any of the errorbars chosen as
>> upper or lower bounds, that the color cycle was off, skipping over 2
>> colors each time another errorbar plot was made (e.g. the first would
>> be green and the second would be cyan instead of blue,green,red,cyan).
>> Looking into the axes class, it appears that if you don't specify a
>> color and want the markers to be drawn over the errorbars, the color
>> cycle has already been skipped over one because of calls to draw the
>> markers into the plot.  The solution is to change all the lines that
>> say " ls='None' " in the errorbar method to instead say " ls='k' " -
>> this will prevent those calls from cycling the colors, and hence only
>> the call to actually draw the markers will do so.  Can this patch be
>> committed to svn?
>>
>
>
>
> --
> Erik Tollerud
> Graduate Student
> Center For Cosmology
> Department of Physics and Astronomy
> 4155B Frederick Reines Hall
> University of California, Irvine
> Office Phone: (949)824-2996
> Cell: (651)307-9409
> eto...@uc...
>



-- 
Erik Tollerud
Graduate Student
Center For Cosmology
Department of Physics and Astronomy
2142 Frederick Reines Hall
University of California, Irvine
Office Phone: (949)824-2587
Cell: (651)307-9409
eto...@uc...

You're right.  Just a typo on my part.  I'll commit these changes to SVN 
so they'll make it into the next release.

Cheers,
Mike

Mark Bakker wrote:
> That fixes it on my machine.
> Except that the function GetRealpathAndStat is in cbook.py (I don't 
> even have a mplutil.py file).
> Thanks,
> Mark
>
> On Tue, May 13, 2008 at 7:21 PM, Michael Droettboom <md...@st... 
> <mailto:md...@st...>> wrote:
>
>     I suspect on Windows it is safe enough (in most cases) to use the
>     realpath as the key, in lieu of anything better.  But that would
>     require testing on Windows to make sure.  Can you try replacing
>     the GetRealpathAndStat in mplutil.py with the following?
>
>
>     class GetRealpathAndStat:
>      def __init__(self):
>          self._cache = {}
>
>      def __call__(self, path):
>          result = self._cache.get(path)
>          if result is None:
>              realpath = os.path.realpath(path)
>              if sys.platform == "win32":
>                  stat_key = realpath
>              else:
>
>                  stat = os.stat(realpath)
>                  stat_key = (stat.st_ino, stat.st_dev)
>              result = realpath, stat_key
>              self._cache[path] = result
>          return result
>     get_realpath_and_stat = GetRealpathAndStat()
>
>
>     Cheers,
>     Mike
>
>     Mark Bakker wrote:
>
>         John, Michael -
>
>         Now that we are talking about a new release, did you guys ever
>         manage to fix the bug described below. It had to do with greek
>         symbols not showing up in postscript files on windows. John
>         seemed to have tracked down the source of the problem, but I
>         never heard of a solution.
>
>         Thanks,
>
>         Mark
>
>         On Tue, Mar 25, 2008 at 7:50 PM, John Hunter
>         <jd...@gm... <mailto:jd...@gm...>
>         <mailto:jd...@gm... <mailto:jd...@gm...>>> wrote:
>
>            On Tue, Mar 25, 2008 at 12:02 PM, Michael Droettboom
>            <md...@st... <mailto:md...@st...>
>         <mailto:md...@st... <mailto:md...@st...>>> wrote:
>
>            > The *intention* is that the fonts *should* be included
>         (with the
>            >  exception of ps.useafm == True).  That was definitely not a
>            deliberate
>            >  change.
>            >
>            >  However, as one of the ones who hasn't been able to
>         reproduce this
>            >  problem, I'm afraid I'm not of much help.  From reading the
>            code, I'm
>            >  still completely stumped as to why the font is not embedded.
>             Someone
>            >  will have to step through with a debugger on one of the
>         broken
>            systems
>            >  to figure this out, I'm afraid.
>
>            I was able to replicate the bug and find the source of the
>         problem.  I
>            am not 100% sure how to fix it, but someone who knows
>         os.stat better
>            might.  The problem is that
>         matplotlib.cbook.get_realpath_and_stat
>
>            class GetRealpathAndStat:
>               def __init__(self):
>                   self._cache = {}
>
>               def __call__(self, path):
>                   result = self._cache.get(path)
>                   if result is None:
>                       realpath = os.path.realpath(path)
>                       stat = os.stat(realpath)
>                       stat_key = (stat.st_ino, stat.st_dev)
>                       result = realpath, stat_key
>                       self._cache[path] = result
>                   return result
>            get_realpath_and_stat = GetRealpathAndStat()
>
>
>            is returning the same stat ino and dev for all the font
>         files, and
>            thus the renderer.used_characters dictionary is getting
>         improper keys
>            -- always (0,0).  So the first font in the gate, in this
>         case Vera, is
>            getting a place in the dict and subsequent fonts (the cm*
>         ones) are
>            not.  The basic problem is that the inode and dev appear to
>         be unix
>            only.
>
>            Michael: if you let me know better what this key is
>         supposed to be
>            doing (can we not simply use the filename for windows?)
>         then I can
>            attempt or test some fixes.
>
>            JDH
>
>
>
>     -- 
>     Michael Droettboom
>     Science Software Branch
>     Operations and Engineering Division
>     Space Telescope Science Institute
>     Operated by AURA for NASA
>
>

-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

That fixes it on my machine.
Except that the function GetRealpathAndStat is in cbook.py (I don't even
have a mplutil.py file).
Thanks,
Mark

On Tue, May 13, 2008 at 7:21 PM, Michael Droettboom <md...@st...> wrote:

> I suspect on Windows it is safe enough (in most cases) to use the realpath
> as the key, in lieu of anything better.  But that would require testing on
> Windows to make sure.  Can you try replacing the GetRealpathAndStat in
> mplutil.py with the following?
>
> class GetRealpathAndStat:
>  def __init__(self):
>      self._cache = {}
>
>  def __call__(self, path):
>      result = self._cache.get(path)
>      if result is None:
>          realpath = os.path.realpath(path)
>          if sys.platform == "win32":
>              stat_key = realpath
>          else:
>              stat = os.stat(realpath)
>              stat_key = (stat.st_ino, stat.st_dev)
>          result = realpath, stat_key
>          self._cache[path] = result
>      return result
> get_realpath_and_stat = GetRealpathAndStat()
>
>
> Cheers,
> Mike
>
> Mark Bakker wrote:
>
> > John, Michael -
> >
> > Now that we are talking about a new release, did you guys ever manage to
> > fix the bug described below. It had to do with greek symbols not showing up
> > in postscript files on windows. John seemed to have tracked down the source
> > of the problem, but I never heard of a solution.
> >
> > Thanks,
> >
> > Mark
> >
> > On Tue, Mar 25, 2008 at 7:50 PM, John Hunter <jd...@gm... <mailto:
> > jd...@gm...>> wrote:
> >
> >    On Tue, Mar 25, 2008 at 12:02 PM, Michael Droettboom
> >    <md...@st... <mailto:md...@st...>> wrote:
> >
> >    > The *intention* is that the fonts *should* be included (with the
> >    >  exception of ps.useafm == True).  That was definitely not a
> >    deliberate
> >    >  change.
> >    >
> >    >  However, as one of the ones who hasn't been able to reproduce this
> >    >  problem, I'm afraid I'm not of much help.  From reading the
> >    code, I'm
> >    >  still completely stumped as to why the font is not embedded.
> >     Someone
> >    >  will have to step through with a debugger on one of the broken
> >    systems
> >    >  to figure this out, I'm afraid.
> >
> >    I was able to replicate the bug and find the source of the problem.
> >  I
> >    am not 100% sure how to fix it, but someone who knows os.stat better
> >    might.  The problem is that matplotlib.cbook.get_realpath_and_stat
> >
> >    class GetRealpathAndStat:
> >       def __init__(self):
> >           self._cache = {}
> >
> >       def __call__(self, path):
> >           result = self._cache.get(path)
> >           if result is None:
> >               realpath = os.path.realpath(path)
> >               stat = os.stat(realpath)
> >               stat_key = (stat.st_ino, stat.st_dev)
> >               result = realpath, stat_key
> >               self._cache[path] = result
> >           return result
> >    get_realpath_and_stat = GetRealpathAndStat()
> >
> >
> >    is returning the same stat ino and dev for all the font files, and
> >    thus the renderer.used_characters dictionary is getting improper keys
> >    -- always (0,0).  So the first font in the gate, in this case Vera,
> > is
> >    getting a place in the dict and subsequent fonts (the cm* ones) are
> >    not.  The basic problem is that the inode and dev appear to be unix
> >    only.
> >
> >    Michael: if you let me know better what this key is supposed to be
> >    doing (can we not simply use the filename for windows?) then I can
> >    attempt or test some fixes.
> >
> >    JDH
> >
> >
> >
> --
> Michael Droettboom
> Science Software Branch
> Operations and Engineering Division
> Space Telescope Science Institute
> Operated by AURA for NASA
>
>

I suspect on Windows it is safe enough (in most cases) to use the 
realpath as the key, in lieu of anything better.  But that would require 
testing on Windows to make sure.  Can you try replacing the 
GetRealpathAndStat in mplutil.py with the following?

class GetRealpathAndStat:
   def __init__(self):
       self._cache = {}

  def __call__(self, path):
       result = self._cache.get(path)
       if result is None:
           realpath = os.path.realpath(path)
           if sys.platform == "win32":
               stat_key = realpath
           else:
               stat = os.stat(realpath)
               stat_key = (stat.st_ino, stat.st_dev)
           result = realpath, stat_key
           self._cache[path] = result
       return result
get_realpath_and_stat = GetRealpathAndStat()


Cheers,
Mike

Mark Bakker wrote:
> John, Michael -
>
> Now that we are talking about a new release, did you guys ever manage 
> to fix the bug described below. It had to do with greek symbols not 
> showing up in postscript files on windows. John seemed to have tracked 
> down the source of the problem, but I never heard of a solution.
>
> Thanks,
>
> Mark
>
> On Tue, Mar 25, 2008 at 7:50 PM, John Hunter <jd...@gm... 
> <mailto:jd...@gm...>> wrote:
>
>     On Tue, Mar 25, 2008 at 12:02 PM, Michael Droettboom
>     <md...@st... <mailto:md...@st...>> wrote:
>
>     > The *intention* is that the fonts *should* be included (with the
>     >  exception of ps.useafm == True).  That was definitely not a
>     deliberate
>     >  change.
>     >
>     >  However, as one of the ones who hasn't been able to reproduce this
>     >  problem, I'm afraid I'm not of much help.  From reading the
>     code, I'm
>     >  still completely stumped as to why the font is not embedded.
>      Someone
>     >  will have to step through with a debugger on one of the broken
>     systems
>     >  to figure this out, I'm afraid.
>
>     I was able to replicate the bug and find the source of the problem.  I
>     am not 100% sure how to fix it, but someone who knows os.stat better
>     might.  The problem is that matplotlib.cbook.get_realpath_and_stat
>
>     class GetRealpathAndStat:
>        def __init__(self):
>            self._cache = {}
>
>        def __call__(self, path):
>            result = self._cache.get(path)
>            if result is None:
>                realpath = os.path.realpath(path)
>                stat = os.stat(realpath)
>                stat_key = (stat.st_ino, stat.st_dev)
>                result = realpath, stat_key
>                self._cache[path] = result
>            return result
>     get_realpath_and_stat = GetRealpathAndStat()
>
>
>     is returning the same stat ino and dev for all the font files, and
>     thus the renderer.used_characters dictionary is getting improper keys
>     -- always (0,0).  So the first font in the gate, in this case Vera, is
>     getting a place in the dict and subsequent fonts (the cm* ones) are
>     not.  The basic problem is that the inode and dev appear to be unix
>     only.
>
>     Michael: if you let me know better what this key is supposed to be
>     doing (can we not simply use the filename for windows?) then I can
>     attempt or test some fixes.
>
>     JDH
>
>

-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

John, Michael -

Now that we are talking about a new release, did you guys ever manage to fix
the bug described below. It had to do with greek symbols not showing up in
postscript files on windows. John seemed to have tracked down the source of
the problem, but I never heard of a solution.

Thanks,

Mark

On Tue, Mar 25, 2008 at 7:50 PM, John Hunter <jd...@gm...> wrote:

> On Tue, Mar 25, 2008 at 12:02 PM, Michael Droettboom <md...@st...>
> wrote:
>
> > The *intention* is that the fonts *should* be included (with the
> >  exception of ps.useafm == True).  That was definitely not a deliberate
> >  change.
> >
> >  However, as one of the ones who hasn't been able to reproduce this
> >  problem, I'm afraid I'm not of much help.  From reading the code, I'm
> >  still completely stumped as to why the font is not embedded.  Someone
> >  will have to step through with a debugger on one of the broken systems
> >  to figure this out, I'm afraid.
>
> I was able to replicate the bug and find the source of the problem.  I
> am not 100% sure how to fix it, but someone who knows os.stat better
> might.  The problem is that matplotlib.cbook.get_realpath_and_stat
>
> class GetRealpathAndStat:
>    def __init__(self):
>        self._cache = {}
>
>    def __call__(self, path):
>        result = self._cache.get(path)
>        if result is None:
>            realpath = os.path.realpath(path)
>            stat = os.stat(realpath)
>            stat_key = (stat.st_ino, stat.st_dev)
>            result = realpath, stat_key
>            self._cache[path] = result
>        return result
> get_realpath_and_stat = GetRealpathAndStat()
>
>
> is returning the same stat ino and dev for all the font files, and
> thus the renderer.used_characters dictionary is getting improper keys
> -- always (0,0).  So the first font in the gate, in this case Vera, is
> getting a place in the dict and subsequent fonts (the cm* ones) are
> not.  The basic problem is that the inode and dev appear to be unix
> only.
>
> Michael: if you let me know better what this key is supposed to be
> doing (can we not simply use the filename for windows?) then I can
> attempt or test some fixes.
>
> JDH
>

Jeff,

Travis fixed the problem on Saturday (r5155).  Trac doesn't seem to have 
the nice feature of sending an email to the originator of a ticket when 
it is changed or closed, and I didn't check until this morning.  I 
don't know whether this fix, or any other recent fixes, or fixes yet to 
be made (addressing memory leaks that seem to have cropped up--see 
ticket 774) will get into the release.

The problem that I reported was triggered by an array created by 
scipy.io.loadmat, reading an old-format mat file created by custom code. 
  I suspect most users would never trip over it. I never did figure out 
what was different about the dtype of that array.

Eric

Jeff Whitaker wrote:
> Eric Firing wrote:
>> Jeff,
>>
>> The problem is not in basemap, it is numpy.  Basemap is doing the 
>> right thing, using the astype method to ensure a copy--but that is not 
>> what the method is actually doing in this case.  I filed a numpy 
>> ticket (788).  I took a quick look at the numpy code but did not find 
>> anything obviously amiss.
>>
>> If this does not get solved in numpy right away, the workaround in 
>> basemap/pyproj _copytobuffer is to use npy.array(x, dtype=npy.float64, 
>> copy=True) instead of the astype method.
>>
>> Eric
> 
> Eric:  Thanks a lot for tracking this down - great detective work!   
> I'll keep an eye on the ticket and if it doesn't get fixed in 1.1 I'll 
> use your suggested workaround.
> 
> -Jeff
>>
>> Eric Firing wrote:
>>> Jeff,
>>>
>>> I sent that last message a little too quickly. Now I see that pyproj 
>>> really is trying to ensure a copy--that is what the comments and 
>>> docstring say--so I will simply try to find and fix the bug, the 
>>> particular case in which a copy is not being made.
>>>
>>> Eric
>>>
>>> Eric Firing wrote:
>>>> Jeff,
>>>>
>>>> I ran into a major bug that I think results from a fairly 
>>>> deep-seated gotcha in basemap.  The problem lies with the in-place 
>>>> transformation of 
>> [...]
> 
> 

I will be mostly out of touch for the next few days, but feel free to
proceed with the 0.91.3 and 0.98.beta pre-release as you all see fit
as you clear up the remaining hurdles.


Thanks,
JDH

Eric Firing wrote:
> Jeff,
>
> The problem is not in basemap, it is numpy.  Basemap is doing the 
> right thing, using the astype method to ensure a copy--but that is not 
> what the method is actually doing in this case.  I filed a numpy 
> ticket (788).  I took a quick look at the numpy code but did not find 
> anything obviously amiss.
>
> If this does not get solved in numpy right away, the workaround in 
> basemap/pyproj _copytobuffer is to use npy.array(x, dtype=npy.float64, 
> copy=True) instead of the astype method.
>
> Eric

Eric:  Thanks a lot for tracking this down - great detective work!   
I'll keep an eye on the ticket and if it doesn't get fixed in 1.1 I'll 
use your suggested workaround.

-Jeff
>
> Eric Firing wrote:
>> Jeff,
>>
>> I sent that last message a little too quickly. Now I see that pyproj 
>> really is trying to ensure a copy--that is what the comments and 
>> docstring say--so I will simply try to find and fix the bug, the 
>> particular case in which a copy is not being made.
>>
>> Eric
>>
>> Eric Firing wrote:
>>> Jeff,
>>>
>>> I ran into a major bug that I think results from a fairly 
>>> deep-seated gotcha in basemap.  The problem lies with the in-place 
>>> transformation of 
> [...]


-- 
Jeffrey S. Whitaker         Phone : (303)497-6313
NOAA/OAR/CDC  R/PSD1        FAX   : (303)497-6449
325 Broadway                Boulder, CO, USA 80305-3328

Jeff,

The problem is not in basemap, it is numpy.  Basemap is doing the right 
thing, using the astype method to ensure a copy--but that is not what 
the method is actually doing in this case.  I filed a numpy ticket 
(788).  I took a quick look at the numpy code but did not find anything 
obviously amiss.

If this does not get solved in numpy right away, the workaround in 
basemap/pyproj _copytobuffer is to use npy.array(x, dtype=npy.float64, 
copy=True) instead of the astype method.

Eric

Eric Firing wrote:
> Jeff,
> 
> I sent that last message a little too quickly. Now I see that pyproj 
> really is trying to ensure a copy--that is what the comments and 
> docstring say--so I will simply try to find and fix the bug, the 
> particular case in which a copy is not being made.
> 
> Eric
> 
> Eric Firing wrote:
>> Jeff,
>>
>> I ran into a major bug that I think results from a fairly deep-seated 
>> gotcha in basemap.  The problem lies with the in-place transformation of 
[...]

Jeff,

I sent that last message a little too quickly. Now I see that pyproj 
really is trying to ensure a copy--that is what the comments and 
docstring say--so I will simply try to find and fix the bug, the 
particular case in which a copy is not being made.

Eric

Eric Firing wrote:
> Jeff,
> 
> I ran into a major bug that I think results from a fairly deep-seated 
> gotcha in basemap.  The problem lies with the in-place transformation of 
> lon,lat in the proj library.  This is not an inherent problem in itself; 
> the trouble comes because sometimes the data get copied along the way 
> from the basemap interface into proj, and sometimes they don't.  When 
> they don't get copied, rotate_vector breaks, because the input lon,lat 
> variables are transformed in place, and then they are no longer lon and 
> lat when they need to be reused as such.
> 
> This could be fixed with a copy in rotate_vector, but I recommend that 
> instead it be fixed by guaranteeing that there is a copy at a much lower 
> level, probably in the pyproj Proj.__call__ method, and if not there, 
> then in the basemap.__call__ method.  I am not sure what all the effects 
> might be, though--is anything relying on in-place calculation at that level?
> 
> The reason for recommending the low-level copy is that if 
> basemap.__call__ continues to behave as it does, sometimes obliterating 
> its input arguments via the in-place operation and sometimes not, it 
> will continue to cause horrible surprises.  I think that the overhead of 
> a single copy would be a small price to pay to eliminate such surprises.
> 
> If there is a need to preserve the in-place option, then perhaps it 
> could be accessed through a different interface.  Let __call__ guarantee 
> that its input arguments are preserved, and have a "transform" method 
> that minimizes copying.
> 
> Eric
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
> Don't miss this year's exciting event. There's still time to save $100. 
> Use priority code J8TL2D2. 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Jeff,

I ran into a major bug that I think results from a fairly deep-seated 
gotcha in basemap.  The problem lies with the in-place transformation of 
lon,lat in the proj library.  This is not an inherent problem in itself; 
the trouble comes because sometimes the data get copied along the way 
from the basemap interface into proj, and sometimes they don't.  When 
they don't get copied, rotate_vector breaks, because the input lon,lat 
variables are transformed in place, and then they are no longer lon and 
lat when they need to be reused as such.

This could be fixed with a copy in rotate_vector, but I recommend that 
instead it be fixed by guaranteeing that there is a copy at a much lower 
level, probably in the pyproj Proj.__call__ method, and if not there, 
then in the basemap.__call__ method.  I am not sure what all the effects 
might be, though--is anything relying on in-place calculation at that level?

The reason for recommending the low-level copy is that if 
basemap.__call__ continues to behave as it does, sometimes obliterating 
its input arguments via the in-place operation and sometimes not, it 
will continue to cause horrible surprises.  I think that the overhead of 
a single copy would be a small price to pay to eliminate such surprises.

If there is a need to preserve the in-place option, then perhaps it 
could be accessed through a different interface.  Let __call__ guarantee 
that its input arguments are preserved, and have a "transform" method 
that minimizes copying.

Eric