matplotlib-users Mailing List for matplotlib

Thanks Ben, that solved my issue. I guess I got thrown off because the
plot stayed open, as you described. It's hard to troubleshoot when you
don't get any errors. In the future I will attach all of my objects.
-Mark

On Fri, Sep 7, 2012 at 12:03 PM, Benjamin Root <ben...@ou...> wrote:
> I think I see your problem... see below:
>
>
> class start_lasso():
>     def __init__(self):
>         data = [Datum(*xy) for xy in rand(100, 2)]
>
>         fig = figure()
>         ax = fig.add_subplot(111, xlim=(0,1), ylim=(0,1),
> autoscale_on=False)
>         lman = LassoManager(ax, data)
>         show()
>
> You aren't saving any of the objects created in the "start_lasso" class to
> your start_lasso object.  Luckily, with the way pyplot works, the figure
> object your create gets implicitly saved to the "pyplot state manager" (a
> sort of smart global location for the figure objects), and the axes object
> gets implicitly attached to the figure object.  Therefore, when the python
> execution goes out of this scope, the figure object and the axes do not get
> garbage-collected.  However, the lasso widget that gets created and all the
> callbacks that were attached are all done with weak references to the figure
> and axes.  So when you leave this scope, the LassoManager no longer exists
> and the callback fails to execute (as designed).
>
> So, make sure that at least lman (and possibly fig and ax) gets saved to the
> start_lasso object to solve that part of the problem.  Next, you don't save
> the start_lasso object your create anywhere, so even if you saved lman to
> start_lasso, the start_lasso object gets garbage-collected anyway as a
> temporary.
>
> I hope this is clear.  Let me know if you still have more issues.
>
> Cheers!
> Ben Root
>

I think I see your problem... see below:


class start_lasso():
    def __init__(self):
        data = [Datum(*xy) for xy in rand(100, 2)]

        fig = figure()
        ax = fig.add_subplot(111, xlim=(0,1), ylim=(0,1),
autoscale_on=False)
        lman = LassoManager(ax, data)
        show()

You aren't saving any of the objects created in the "start_lasso" class to
your start_lasso object.  Luckily, with the way pyplot works, the figure
object your create gets implicitly saved to the "pyplot state manager" (a
sort of smart global location for the figure objects), and the axes object
gets implicitly attached to the figure object.  Therefore, when the python
execution goes out of this scope, the figure object and the axes do not get
garbage-collected.  However, the lasso widget that gets created and all the
callbacks that were attached are all done with weak references to the
figure and axes.  So when you leave this scope, the LassoManager no longer
exists and the callback fails to execute (as designed).

So, make sure that at least lman (and possibly fig and ax) gets saved to
the start_lasso object to solve that part of the problem.  Next, you don't
save the start_lasso object your create anywhere, so even if you saved lman
to start_lasso, the start_lasso object gets garbage-collected anyway as a
temporary.

I hope this is clear.  Let me know if you still have more issues.

Cheers!
Ben Root

On Sep 7, 2012, at 11:04 AM, Eric Firing wrote:

> On 2012/09/07 4:00 AM, Benjamin Root wrote:
>> 
>> 
>> On Fri, Sep 7, 2012 at 9:49 AM, Shahar Shani-Kadmiel
>> <ka...@po... <mailto:ka...@po...>> wrote:
>> 
>>    On Sep 7, 2012, at 4:25 PM, Benjamin Root wrote:
>> 
>>> 
>>> 
>>>    On Fri, Sep 7, 2012 at 8:44 AM, Shahar Shani-Kadmiel
>>>    <ka...@po... <mailto:ka...@po...>> wrote:
>>> 
> 
<snip>

> Normalization has to handle all sorts of inputs--masked or not, all 
> sorts of numbers, scalar or array--and it is much easier to do this 
> efficiently if all these possibilities are reduced to a very few at the 
> start.  Specifically, it needs to supply a copy of the input (so that 
> normalization doesn't change the original) in a floating point masked 
> array, using float32 if possible for space efficiency.  It needs to keep 
> track of whether the input was a scalar, so that normalization can 
> return a scalar when given a scalar input.
> 
> Eric

Another option as I understand it is to pass in a 1D (greyscale) or 3d (color) array (where the 3rd dimension is RGB and optionally A) of type uint8(?). This array does not need to get normalized, it will be displayed as raw pixel values.  I don't remember if you also have to specifically tell it not to normalize the data. 

But the easier answer for your case would probably be imshow(data[::10,::10]) which will take every 10th element in x and y thus reducing the size by a factor of 100 (depending on the size of your data you could use ::2 or ::50, etc)

Ethan

On 2012/09/07 4:00 AM, Benjamin Root wrote:
>
>
> On Fri, Sep 7, 2012 at 9:49 AM, Shahar Shani-Kadmiel
> <ka...@po... <mailto:ka...@po...>> wrote:
>
>     On Sep 7, 2012, at 4:25 PM, Benjamin Root wrote:
>
>>
>>
>>     On Fri, Sep 7, 2012 at 8:44 AM, Shahar Shani-Kadmiel
>>     <ka...@po... <mailto:ka...@po...>> wrote:
>>
>>         1. an ipython session is invoked with qtconsole --pylab
>>         2. I load a large NetCDF grid (Grid file format: nf (# 18) GMT
>>         netCDF format (float)  (COARDS-compliant) [DEFAULT]), approx.
>>         1.15 GB
>>         3. I then try to plot with imshow the data
>>
>>         added below are the lines leading up to the error and the
>>         error itself.
>>
>>
>>         This is running on OS X 10.7.4 with a recently installed EPD 7.3.
>>
>>
>>         {code}
>>         from scipy.io <http://scipy.io/> import netcdf_file as netcdf
>>         data =
>>         netcdf('srtm_43_44_05_06_07_08.grd','r').variables['z'][::-1]
>>
>>         fig, ax = subplots()
>>
>>         data.shape
>>         Out[5]: (24004, 12002)
>>
>>         im = ax.imshow(data,
>>         aspect=((data.shape[1])/float(data.shape[0])),
>>         interpolation='none')
>>         ---------------------------------------------------------------------------
>>         MemoryError                               Traceback (most
>>         recent call last)
>>         <ipython-input-6-f92e4c4c63b5> in <module>()
>>         ----> 1 im = ax.imshow(data,
>>         aspect=((data.shape[1])/float(data.shape[0])),
>>         interpolation='none')
>>
>>         /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/axes.py
>>         in imshow(self, X, cmap, norm, aspect, interpolation, alpha,
>>         vmin, vmax, origin, extent, shape, filternorm, filterrad,
>>         imlim, resample, url, **kwargs)
>>            6743             im.set_clim(vmin, vmax)
>>            6744         else:
>>         -> 6745             im.autoscale_None()
>>            6746         im.set_url(url)
>>            6747
>>
>>         /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/cm.py
>>         in autoscale_None(self)
>>             281         if self._A is None:
>>             282             raise TypeError('You must first set_array
>>         for mappable')
>>         --> 283         self.norm.autoscale_None(self._A)
>>             284         self.changed()
>>             285
>>
>>         /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/colors.py
>>         in autoscale_None(self, A)
>>             889         ' autoscale only None-valued vmin or vmax'
>>             890         if self.vmin is None:
>>         --> 891             self.vmin = ma.min(A)
>>             892         if self.vmax is None:
>>             893             self.vmax = ma.max(A)
>>
>>         /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/numpy/ma/core.pyc
>>         in min(obj, axis, out, fill_value)
>>            5873 def min(obj, axis=None, out=None, fill_value=None):
>>            5874     try:
>>         -> 5875         return obj.min(axis=axis,
>>         fill_value=fill_value, out=out)
>>            5876     except (AttributeError, TypeError):
>>            5877         # If obj doesn't have a max method,
>>
>>         /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/numpy/ma/core.pyc
>>         in min(self, axis, out, fill_value)
>>            5054         # No explicit output
>>            5055         if out is None:
>>         -> 5056             result =
>>         self.filled(fill_value).min(axis=axis, out=out).view(type(self))
>>            5057             if result.ndim:
>>            5058                 # Set the mask
>>
>>         /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/numpy/ma/core.pyc
>>         in filled(self, fill_value)
>>            3388             return self._data
>>            3389         else:
>>         -> 3390             result = self._data.copy()
>>            3391             try:
>>            3392                 np.putmask(result, m, fill_value)
>>
>>         MemoryError:
>>         {/code}
>>
>>
>>     This is more a NumPy issue than anything else.  We need to know
>>     the min and the max of the array in order to automatically scale
>>     the colormap for display.  Therefore, we query the array object
>>     for its min/max.  Because we support masked arrays, the array is
>>     first cast as a masked array, and then these queries are done.
>>
>>     It appears that in numpy's masked array module, it calculates the
>>     array's min by making a copy of itself first.  I would have
>>     figured that it would have done its task differently.  In the
>>     meantime, I suspect you can work around this problem by explicitly
>>     setting the vmin/vmax keyword arguments to imshow if you know
>>     them.  Therefore, there should be no need to determine the array's
>>     min/max in this inefficient manner.
>>
>>     Ben Root
>>
>
>     Hi Ben,
>     I tried adding vmin & vmax the the imshow call but I still get a
>     MemoryError.
>     The grid file is 1.15 GB and I have ~4.5 out of 8 GB of memory
>     available when I launch ipython, 3.5 when I execute imshow and 2
>     when I execute plt.draw().

mpl simply is not designed for image-type operations on huge arrays, 
vastly larger than what can be displayed.  It is up to the user to 
down-sample or otherwise reduce the size of the array fed to imshow.

>
>
> Well, it looks like setting vmin/vmax helped, because your traceback

Did a new traceback get posted in a message that was not sent to the 
list?  I found only the original traceback.

> shows that the code made significant progress.  The issue here is that
> the process_value() method doesn't make a lot of sense.  I am not sure
> what is the rationale behind its logic.  Hopefully, someone else can
> chime in with an explanation of what is going on.

Normalization has to handle all sorts of inputs--masked or not, all 
sorts of numbers, scalar or array--and it is much easier to do this 
efficiently if all these possibilities are reduced to a very few at the 
start.  Specifically, it needs to supply a copy of the input (so that 
normalization doesn't change the original) in a floating point masked 
array, using float32 if possible for space efficiency.  It needs to keep 
track of whether the input was a scalar, so that normalization can 
return a scalar when given a scalar input.

Eric

>
> In the meantime, are you using a 32 or 64-bit machine?
>
> Ben Root
>
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>
>
>
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>

On Sep 7, 2012, at 5:00 PM, Benjamin Root <ben...@ou...> wrote:

> 
> 
> On Fri, Sep 7, 2012 at 9:49 AM, Shahar Shani-Kadmiel <ka...@po...> wrote:
> On Sep 7, 2012, at 4:25 PM, Benjamin Root wrote:
> 
>> 
>> 
>> On Fri, Sep 7, 2012 at 8:44 AM, Shahar Shani-Kadmiel <ka...@po...> wrote:
>> 1. an ipython session is invoked with qtconsole --pylab
>> 2. I load a large NetCDF grid (Grid file format: nf (# 18) GMT netCDF format (float)  (COARDS-compliant) [DEFAULT]), approx. 1.15 GB
>> 3. I then try to plot with imshow the data
>> 
>> added below are the lines leading up to the error and the error itself.
>> 
>> 
>> This is running on OS X 10.7.4 with a recently installed EPD 7.3.
>> 
>> 
>> {code}
>> from scipy.io import netcdf_file as netcdf
>> data = netcdf('srtm_43_44_05_06_07_08.grd','r').variables['z'][::-1]
>> 
>> fig, ax = subplots()
>> 
>> data.shape
>> Out[5]: (24004, 12002)
>> 
>> im = ax.imshow(data, aspect=((data.shape[1])/float(data.shape[0])), interpolation='none')
>> ---------------------------------------------------------------------------
>> MemoryError                               Traceback (most recent call last)
>> <ipython-input-6-f92e4c4c63b5> in <module>()
>> ----> 1 im = ax.imshow(data, aspect=((data.shape[1])/float(data.shape[0])), interpolation='none')
>> 
>> /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/axes.py in imshow(self, X, cmap, norm, aspect, interpolation, alpha, vmin, vmax, origin, extent, shape, filternorm, filterrad, imlim, resample, url, **kwargs)
>>    6743             im.set_clim(vmin, vmax)
>>    6744         else:
>> -> 6745             im.autoscale_None()
>>    6746         im.set_url(url)
>>    6747
>> 
>> /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/cm.py in autoscale_None(self)
>>     281         if self._A is None:
>>     282             raise TypeError('You must first set_array for mappable')
>> --> 283         self.norm.autoscale_None(self._A)
>>     284         self.changed()
>>     285
>> 
>> /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/colors.py in autoscale_None(self, A)
>>     889         ' autoscale only None-valued vmin or vmax'
>>     890         if self.vmin is None:
>> --> 891             self.vmin = ma.min(A)
>>     892         if self.vmax is None:
>>     893             self.vmax = ma.max(A)
>> 
>> /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/numpy/ma/core.pyc in min(obj, axis, out, fill_value)
>>    5873 def min(obj, axis=None, out=None, fill_value=None):
>>    5874     try:
>> -> 5875         return obj.min(axis=axis, fill_value=fill_value, out=out)
>>    5876     except (AttributeError, TypeError):
>>    5877         # If obj doesn't have a max method,
>> 
>> /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/numpy/ma/core.pyc in min(self, axis, out, fill_value)
>>    5054         # No explicit output
>>    5055         if out is None:
>> -> 5056             result = self.filled(fill_value).min(axis=axis, out=out).view(type(self))
>>    5057             if result.ndim:
>>    5058                 # Set the mask
>> 
>> /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/numpy/ma/core.pyc in filled(self, fill_value)
>>    3388             return self._data
>>    3389         else:
>> -> 3390             result = self._data.copy()
>>    3391             try:
>>    3392                 np.putmask(result, m, fill_value)
>> 
>> MemoryError:
>> {/code}
>> 
>> This is more a NumPy issue than anything else.  We need to know the min and the max of the array in order to automatically scale the colormap for display.  Therefore, we query the array object for its min/max.  Because we support masked arrays, the array is first cast as a masked array, and then these queries are done.
>> 
>> It appears that in numpy's masked array module, it calculates the array's min by making a copy of itself first.  I would have figured that it would have done its task differently.  In the meantime, I suspect you can work around this problem by explicitly setting the vmin/vmax keyword arguments to imshow if you know them.  Therefore, there should be no need to determine the array's min/max in this inefficient manner.
>> 
>> Ben Root
>> 
> 
> Hi Ben,
> I tried adding vmin & vmax the the imshow call but I still get a MemoryError.
> The grid file is 1.15 GB and I have ~4.5 out of 8 GB of memory available when I launch ipython, 3.5 when I execute imshow and 2 when I execute plt.draw().
> 
> Well, it looks like setting vmin/vmax helped, because your traceback shows that the code made significant progress.  The issue here is that the process_value() method doesn't make a lot of sense.  I am not sure what is the rationale behind its logic.  Hopefully, someone else can chime in with an explanation of what is going on.
> 
> In the meantime, are you using a 32 or 64-bit machine?
> 
> Ben Root
> 

I am on a 64 bit machine but 32 bit distro of Enthought. 

On Fri, Sep 7, 2012 at 9:49 AM, Shahar Shani-Kadmiel <ka...@po...
> wrote:

> On Sep 7, 2012, at 4:25 PM, Benjamin Root wrote:
>
>
>
> On Fri, Sep 7, 2012 at 8:44 AM, Shahar Shani-Kadmiel <
> ka...@po...> wrote:
>
>> 1. an ipython session is invoked with qtconsole --pylab
>> 2. I load a large NetCDF grid (Grid file format: nf (# 18) GMT netCDF
>> format (float)  (COARDS-compliant) [DEFAULT]), approx. 1.15 GB
>> 3. I then try to plot with imshow the data
>>
>> added below are the lines leading up to the error and the error itself.
>>
>>
>> This is running on OS X 10.7.4 with a recently installed EPD 7.3.
>>
>>
>> {code}
>> from scipy.io import netcdf_file as netcdf
>> data = netcdf('srtm_43_44_05_06_07_08.grd','r').variables['z'][::-1]
>>
>> fig, ax = subplots()
>>
>> data.shape
>> Out[5]: (24004, 12002)
>>
>> im = ax.imshow(data, aspect=((data.shape[1])/float(data.shape[0])),
>> interpolation='none')
>>
>> ---------------------------------------------------------------------------
>> MemoryError                               Traceback (most recent call
>> last)
>> <ipython-input-6-f92e4c4c63b5> in <module>()
>> ----> 1 im = ax.imshow(data,
>> aspect=((data.shape[1])/float(data.shape[0])), interpolation='none')
>>
>> /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/axes.py
>> in imshow(self, X, cmap, norm, aspect, interpolation, alpha, vmin, vmax,
>> origin, extent, shape, filternorm, filterrad, imlim, resample, url,
>> **kwargs)
>>    6743             im.set_clim(vmin, vmax)
>>    6744         else:
>> -> 6745             im.autoscale_None()
>>    6746         im.set_url(url)
>>    6747
>>
>> /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/cm.py
>> in autoscale_None(self)
>>     281         if self._A is None:
>>     282             raise TypeError('You must first set_array for
>> mappable')
>> --> 283         self.norm.autoscale_None(self._A)
>>     284         self.changed()
>>     285
>>
>> /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/colors.py
>> in autoscale_None(self, A)
>>     889         ' autoscale only None-valued vmin or vmax'
>>     890         if self.vmin is None:
>> --> 891             self.vmin = ma.min(A)
>>     892         if self.vmax is None:
>>     893             self.vmax = ma.max(A)
>>
>> /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/numpy/ma/core.pyc
>> in min(obj, axis, out, fill_value)
>>    5873 def min(obj, axis=None, out=None, fill_value=None):
>>    5874     try:
>> -> 5875         return obj.min(axis=axis, fill_value=fill_value, out=out)
>>    5876     except (AttributeError, TypeError):
>>    5877         # If obj doesn't have a max method,
>>
>> /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/numpy/ma/core.pyc
>> in min(self, axis, out, fill_value)
>>    5054         # No explicit output
>>    5055         if out is None:
>> -> 5056             result = self.filled(fill_value).min(axis=axis,
>> out=out).view(type(self))
>>    5057             if result.ndim:
>>    5058                 # Set the mask
>>
>> /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/numpy/ma/core.pyc
>> in filled(self, fill_value)
>>    3388             return self._data
>>    3389         else:
>> -> 3390             result = self._data.copy()
>>    3391             try:
>>    3392                 np.putmask(result, m, fill_value)
>>
>> MemoryError:
>> {/code}
>>
>
> This is more a NumPy issue than anything else.  We need to know the min
> and the max of the array in order to automatically scale the colormap for
> display.  Therefore, we query the array object for its min/max.  Because we
> support masked arrays, the array is first cast as a masked array, and then
> these queries are done.
>
> It appears that in numpy's masked array module, it calculates the array's
> min by making a copy of itself first.  I would have figured that it would
> have done its task differently.  In the meantime, I suspect you can work
> around this problem by explicitly setting the vmin/vmax keyword arguments
> to imshow if you know them.  Therefore, there should be no need to
> determine the array's min/max in this inefficient manner.
>
> Ben Root
>
>
> Hi Ben,
> I tried adding vmin & vmax the the imshow call but I still get a
> MemoryError.
> The grid file is 1.15 GB and I have ~4.5 out of 8 GB of memory available
> when I launch ipython, 3.5 when I execute imshow and 2 when I execute
> plt.draw().
>

Well, it looks like setting vmin/vmax helped, because your traceback shows
that the code made significant progress.  The issue here is that the
process_value() method doesn't make a lot of sense.  I am not sure what is
the rationale behind its logic.  Hopefully, someone else can chime in with
an explanation of what is going on.

In the meantime, are you using a 32 or 64-bit machine?

Ben Root

On Sep 7, 2012, at 4:25 PM, Benjamin Root wrote:

> 
> 
> On Fri, Sep 7, 2012 at 8:44 AM, Shahar Shani-Kadmiel <ka...@po...> wrote:
> 1. an ipython session is invoked with qtconsole --pylab
> 2. I load a large NetCDF grid (Grid file format: nf (# 18) GMT netCDF format (float)  (COARDS-compliant) [DEFAULT]), approx. 1.15 GB
> 3. I then try to plot with imshow the data
> 
> added below are the lines leading up to the error and the error itself.
> 
> 
> This is running on OS X 10.7.4 with a recently installed EPD 7.3.
> 
> 
> {code}
> from scipy.io import netcdf_file as netcdf
> data = netcdf('srtm_43_44_05_06_07_08.grd','r').variables['z'][::-1]
> 
> fig, ax = subplots()
> 
> data.shape
> Out[5]: (24004, 12002)
> 
> im = ax.imshow(data, aspect=((data.shape[1])/float(data.shape[0])), interpolation='none')
> ---------------------------------------------------------------------------
> MemoryError                               Traceback (most recent call last)
> <ipython-input-6-f92e4c4c63b5> in <module>()
> ----> 1 im = ax.imshow(data, aspect=((data.shape[1])/float(data.shape[0])), interpolation='none')
> 
> /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/axes.py in imshow(self, X, cmap, norm, aspect, interpolation, alpha, vmin, vmax, origin, extent, shape, filternorm, filterrad, imlim, resample, url, **kwargs)
>    6743             im.set_clim(vmin, vmax)
>    6744         else:
> -> 6745             im.autoscale_None()
>    6746         im.set_url(url)
>    6747
> 
> /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/cm.py in autoscale_None(self)
>     281         if self._A is None:
>     282             raise TypeError('You must first set_array for mappable')
> --> 283         self.norm.autoscale_None(self._A)
>     284         self.changed()
>     285
> 
> /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/colors.py in autoscale_None(self, A)
>     889         ' autoscale only None-valued vmin or vmax'
>     890         if self.vmin is None:
> --> 891             self.vmin = ma.min(A)
>     892         if self.vmax is None:
>     893             self.vmax = ma.max(A)
> 
> /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/numpy/ma/core.pyc in min(obj, axis, out, fill_value)
>    5873 def min(obj, axis=None, out=None, fill_value=None):
>    5874     try:
> -> 5875         return obj.min(axis=axis, fill_value=fill_value, out=out)
>    5876     except (AttributeError, TypeError):
>    5877         # If obj doesn't have a max method,
> 
> /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/numpy/ma/core.pyc in min(self, axis, out, fill_value)
>    5054         # No explicit output
>    5055         if out is None:
> -> 5056             result = self.filled(fill_value).min(axis=axis, out=out).view(type(self))
>    5057             if result.ndim:
>    5058                 # Set the mask
> 
> /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/numpy/ma/core.pyc in filled(self, fill_value)
>    3388             return self._data
>    3389         else:
> -> 3390             result = self._data.copy()
>    3391             try:
>    3392                 np.putmask(result, m, fill_value)
> 
> MemoryError:
> {/code}
> 
> This is more a NumPy issue than anything else.  We need to know the min and the max of the array in order to automatically scale the colormap for display.  Therefore, we query the array object for its min/max.  Because we support masked arrays, the array is first cast as a masked array, and then these queries are done.
> 
> It appears that in numpy's masked array module, it calculates the array's min by making a copy of itself first.  I would have figured that it would have done its task differently.  In the meantime, I suspect you can work around this problem by explicitly setting the vmin/vmax keyword arguments to imshow if you know them.  Therefore, there should be no need to determine the array's min/max in this inefficient manner.
> 
> Ben Root
> 

Hi Ben,
I tried adding vmin & vmax the the imshow call but I still get a MemoryError.
The grid file is 1.15 GB and I have ~4.5 out of 8 GB of memory available when I launch ipython, 3.5 when I execute imshow and 2 when I execute plt.draw().

im = ax.imshow(data, aspect=((data.shape[1])/float(data.shape[0])), interpolation=None, vmin=-400., vmax=3000.)

plt.draw()
---------------------------------------------------------------------------
MemoryError                               Traceback (most recent call last)
/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/artist.py in draw_wrapper(artist, renderer, *args, **kwargs)
     53     def draw_wrapper(artist, renderer, *args, **kwargs):
     54         before(artist, renderer)
---> 55         draw(artist, renderer, *args, **kwargs)
     56         after(artist, renderer)
     57 

/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/figure.py in draw(self, renderer)
    882         dsu.sort(key=itemgetter(0))
    883         for zorder, func, args in dsu:
--> 884             func(*args)
    885 
    886         renderer.close_group('figure')

/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/artist.py in draw_wrapper(artist, renderer, *args, **kwargs)
     53     def draw_wrapper(artist, renderer, *args, **kwargs):
     54         before(artist, renderer)
---> 55         draw(artist, renderer, *args, **kwargs)
     56         after(artist, renderer)
     57 

/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/axes.py in draw(self, renderer, inframe)
   1981 
   1982         for zorder, a in dsu:
-> 1983             a.draw(renderer)
   1984 
   1985         renderer.close_group('axes')

/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/artist.py in draw_wrapper(artist, renderer, *args, **kwargs)
     53     def draw_wrapper(artist, renderer, *args, **kwargs):
     54         before(artist, renderer)
---> 55         draw(artist, renderer, *args, **kwargs)
     56         after(artist, renderer)
     57 

/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/image.py in draw(self, renderer, *args, **kwargs)
    353                 warnings.warn("Image will not be shown correctly with this backend.")
    354 
--> 355             im = self.make_image(renderer.get_image_magnification())
    356             if im is None:
    357                 return

/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/image.py in make_image(self, magnification)
    573         im, xmin, ymin, dxintv, dyintv, sx, sy = \
    574             self._get_unsampled_image(self._A, [_x1, _x2, _y1, _y2],
--> 575                                       transformed_viewLim)
    576 
    577         fc = self.axes.patch.get_facecolor()

/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/image.py in _get_unsampled_image(self, A, image_extents, viewlim)
    200             else:
    201                 if self._rgbacache is None:
--> 202                     x = self.to_rgba(self._A, self._alpha, bytes=True)
    203                     self._rgbacache = x
    204                 else:

/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/cm.py in to_rgba(self, x, alpha, bytes)
    211             pass
    212         x = ma.asarray(x)
--> 213         x = self.norm(x)
    214         x = self.cmap(x, alpha=alpha, bytes=bytes)
    215         return x

/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/colors.py in __call__(self, value, clip)
    843             clip = self.clip
    844 
--> 845         result, is_scalar = self.process_value(value)
    846 
    847         self.autoscale_None(result)

/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/colors.py in process_value(value)
    829             if result.dtype.kind == 'f':
    830                 if isinstance(value, np.ndarray):
--> 831                     result = result.copy()
    832             elif result.dtype.itemsize > 2:
    833                 result = result.astype(np.float)

/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/numpy/ma/core.pyc in __call__(self, *args, **params)
   2449         mask = instance._mask
   2450         cls = type(instance)
-> 2451         result = getattr(data, methodname)(*args, **params).view(cls)
   2452         result._update_from(instance)
   2453         if result.ndim:

MemoryError: 

On Fri, Sep 7, 2012 at 8:44 AM, Shahar Shani-Kadmiel <ka...@po...
> wrote:

> 1. an ipython session is invoked with qtconsole --pylab
> 2. I load a large NetCDF grid (Grid file format: nf (# 18) GMT netCDF
> format (float)  (COARDS-compliant) [DEFAULT]), approx. 1.15 GB
> 3. I then try to plot with imshow the data
>
> added below are the lines leading up to the error and the error itself.
>
>
> This is running on OS X 10.7.4 with a recently installed EPD 7.3.
>
>
> {code}
> from scipy.io import netcdf_file as netcdf
> data = netcdf('srtm_43_44_05_06_07_08.grd','r').variables['z'][::-1]
>
> fig, ax = subplots()
>
> data.shape
> Out[5]: (24004, 12002)
>
> im = ax.imshow(data, aspect=((data.shape[1])/float(data.shape[0])),
> interpolation='none')
> ---------------------------------------------------------------------------
> MemoryError                               Traceback (most recent call last)
> <ipython-input-6-f92e4c4c63b5> in <module>()
> ----> 1 im = ax.imshow(data,
> aspect=((data.shape[1])/float(data.shape[0])), interpolation='none')
>
> /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/axes.py
> in imshow(self, X, cmap, norm, aspect, interpolation, alpha, vmin, vmax,
> origin, extent, shape, filternorm, filterrad, imlim, resample, url,
> **kwargs)
>    6743             im.set_clim(vmin, vmax)
>    6744         else:
> -> 6745             im.autoscale_None()
>    6746         im.set_url(url)
>    6747
>
> /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/cm.py
> in autoscale_None(self)
>     281         if self._A is None:
>     282             raise TypeError('You must first set_array for
> mappable')
> --> 283         self.norm.autoscale_None(self._A)
>     284         self.changed()
>     285
>
> /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/colors.py
> in autoscale_None(self, A)
>     889         ' autoscale only None-valued vmin or vmax'
>     890         if self.vmin is None:
> --> 891             self.vmin = ma.min(A)
>     892         if self.vmax is None:
>     893             self.vmax = ma.max(A)
>
> /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/numpy/ma/core.pyc
> in min(obj, axis, out, fill_value)
>    5873 def min(obj, axis=None, out=None, fill_value=None):
>    5874     try:
> -> 5875         return obj.min(axis=axis, fill_value=fill_value, out=out)
>    5876     except (AttributeError, TypeError):
>    5877         # If obj doesn't have a max method,
>
> /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/numpy/ma/core.pyc
> in min(self, axis, out, fill_value)
>    5054         # No explicit output
>    5055         if out is None:
> -> 5056             result = self.filled(fill_value).min(axis=axis,
> out=out).view(type(self))
>    5057             if result.ndim:
>    5058                 # Set the mask
>
> /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/numpy/ma/core.pyc
> in filled(self, fill_value)
>    3388             return self._data
>    3389         else:
> -> 3390             result = self._data.copy()
>    3391             try:
>    3392                 np.putmask(result, m, fill_value)
>
> MemoryError:
> {/code}
>

This is more a NumPy issue than anything else.  We need to know the min and
the max of the array in order to automatically scale the colormap for
display.  Therefore, we query the array object for its min/max.  Because we
support masked arrays, the array is first cast as a masked array, and then
these queries are done.

It appears that in numpy's masked array module, it calculates the array's
min by making a copy of itself first.  I would have figured that it would
have done its task differently.  In the meantime, I suspect you can work
around this problem by explicitly setting the vmin/vmax keyword arguments
to imshow if you know them.  Therefore, there should be no need to
determine the array's min/max in this inefficient manner.

Ben Root

1. an ipython session is invoked with qtconsole --pylab 
2. I load a large NetCDF grid (Grid file format: nf (# 18) GMT netCDF format (float)  (COARDS-compliant) [DEFAULT]), approx. 1.15 GB
3. I then try to plot with imshow the data

added below are the lines leading up to the error and the error itself.


This is running on OS X 10.7.4 with a recently installed EPD 7.3.


{code}
from scipy.io import netcdf_file as netcdf
data = netcdf('srtm_43_44_05_06_07_08.grd','r').variables['z'][::-1]

fig, ax = subplots()

data.shape
Out[5]: (24004, 12002)

im = ax.imshow(data, aspect=((data.shape[1])/float(data.shape[0])), interpolation='none')
---------------------------------------------------------------------------
MemoryError                               Traceback (most recent call last)
<ipython-input-6-f92e4c4c63b5> in <module>()
----> 1 im = ax.imshow(data, aspect=((data.shape[1])/float(data.shape[0])), interpolation='none')

/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/axes.py in imshow(self, X, cmap, norm, aspect, interpolation, alpha, vmin, vmax, origin, extent, shape, filternorm, filterrad, imlim, resample, url, **kwargs)
   6743             im.set_clim(vmin, vmax)
   6744         else:
-> 6745             im.autoscale_None()
   6746         im.set_url(url)
   6747 

/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/cm.py in autoscale_None(self)
    281         if self._A is None:
    282             raise TypeError('You must first set_array for mappable')
--> 283         self.norm.autoscale_None(self._A)
    284         self.changed()
    285 

/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/colors.py in autoscale_None(self, A)
    889         ' autoscale only None-valued vmin or vmax'
    890         if self.vmin is None:
--> 891             self.vmin = ma.min(A)
    892         if self.vmax is None:
    893             self.vmax = ma.max(A)

/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/numpy/ma/core.pyc in min(obj, axis, out, fill_value)
   5873 def min(obj, axis=None, out=None, fill_value=None):
   5874     try:
-> 5875         return obj.min(axis=axis, fill_value=fill_value, out=out)
   5876     except (AttributeError, TypeError):
   5877         # If obj doesn't have a max method,

/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/numpy/ma/core.pyc in min(self, axis, out, fill_value)
   5054         # No explicit output
   5055         if out is None:
-> 5056             result = self.filled(fill_value).min(axis=axis, out=out).view(type(self))
   5057             if result.ndim:
   5058                 # Set the mask

/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/numpy/ma/core.pyc in filled(self, fill_value)
   3388             return self._data
   3389         else:
-> 3390             result = self._data.copy()
   3391             try:
   3392                 np.putmask(result, m, fill_value)

MemoryError: 
{/code}

Hi all,

I have just received the following information from John's family
regarding the memorial service:

John's memorial service will be held on Monday, October 1, 2012, at
11.a.m. at Rockefeller Chapel at the University of Chicago.  The exact
address is 5850 S. Woodlawn Ave, Chicago, IL 60615.  The service is
open to the public.  The service will be fully planned and scripted
with no room for people to eulogize, however, we will have a reception
after the service, hosted by Tradelink, where people can talk.

Regards,

f

On 2012/09/06 9:03 PM, Phil Elson wrote:
> This seems to a be common misconception...
>
> I guess in future, we could add a check to the add_patch method to see
> if the given artist already has an associated Axes, and if it does, emit
> a warning.

It's not just patches; but I think a single warning in Artist.add_axes 
would do it, perhaps at the cost of generating an unnecessary warning 
for some legitimate use case.

Eric

>
>
>
> On 7 September 2012 07:42, Eric Firing <ef...@ha...
> <mailto:ef...@ha...>> wrote:
>
>     On 2012/09/06 8:35 PM, jonasr wrote:
>      > That seems to work, thank you.
>      >
>      > I would have thought that the add_patch function creates two seperate
>      > objects independent of the defined Rectangle.
>
>     add_patch doesn't create any objects; it just attaches the axes to the
>     patch in both directions: a reference to the patch object is appended to
>     a list of axes artists, and the patch object gets a reference to the
>     axes.  Fortunately, python has garbage collection to handle such
>     circular references.
>
>     Eric
>
>      >
>      > greets jonas
>
>
>     ------------------------------------------------------------------------------
>     Live Security Virtual Conference
>     Exclusive live event will cover all the ways today's security and
>     threat landscape has changed and how IT managers can respond.
>     Discussions
>     will include endpoint security, mobile security and the latest in
>     malware
>     threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>     _______________________________________________
>     Matplotlib-users mailing list
>     Mat...@li...
>     <mailto:Mat...@li...>
>     https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>
>

This seems to a be common misconception...

I guess in future, we could add a check to the add_patch method to see if
the given artist already has an associated Axes, and if it does, emit a
warning.



On 7 September 2012 07:42, Eric Firing <ef...@ha...> wrote:

> On 2012/09/06 8:35 PM, jonasr wrote:
> > That seems to work, thank you.
> >
> > I would have thought that the add_patch function creates two seperate
> > objects independent of the defined Rectangle.
>
> add_patch doesn't create any objects; it just attaches the axes to the
> patch in both directions: a reference to the patch object is appended to
> a list of axes artists, and the patch object gets a reference to the
> axes.  Fortunately, python has garbage collection to handle such
> circular references.
>
> Eric
>
> >
> > greets jonas
>
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>

On 2012/09/06 8:35 PM, jonasr wrote:
> That seems to work, thank you.
>
> I would have thought that the add_patch function creates two seperate
> objects independent of the defined Rectangle.

add_patch doesn't create any objects; it just attaches the axes to the 
patch in both directions: a reference to the patch object is appended to 
a list of axes artists, and the patch object gets a reference to the 
axes.  Fortunately, python has garbage collection to handle such 
circular references.

Eric

>
> greets jonas

That seems to work, thank you.

I would have thought that the add_patch function creates two seperate
objects independent of the defined Rectangle.

greets jonas



--
View this message in context: http://matplotlib.1069221.n5.nabble.com/Rectangle-Bug-tp38825p38828.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

On Thursday, September 6, 2012, jonasr wrote:

> Hello,
>
> i spotted a bug in the Rectangle function when plotting with multiple
> subplot, here is a source code example:
>
> #!/usr/bin/env python
> # -*- coding: utf-8 -*-
>
> from matplotlib.pyplot import *
> from numpy import *
> import sys, os
>
> def main():
>         f, axs = subplots(1,2)
>
>         x=arange(0,10,0.001)
>         y=sin(x)
>         axs[0].plot(x,y,"blue")#,alpha=1)
>         axs[1].plot(x,y,"blue",alpha=1)
>
>         rect = Rectangle((0,0), 1, 1, facecolor="blue",alpha=1)
>         axs[0].add_patch(rect)
>         axs[1].add_patch(rect)
>
>         show()
>         return 0
>
> if __name__ == '__main__':
>         main()
>
> this script should plot two sin functions and a rectangle  in both subplots
> .
> the rectangle doesn't seem to appear until i move the area from the left
> subplot (where the rectangle should appear)
> over to the second subplot.
> here is an example how it looks like:
> http://www.imagebanana.com/view/hzm8bjro/example.png
>
> if i remove the command
> axs[1].add_patch(rect)
> the problem doesn't seem to appear
>
> my current matplotlib version is: '1.1.1rc'
> os is ubuntu 12.04
>
> greetz jonas
>
>
>
>
Not a bug.  An artist, such as the rectangle patch, can only reliably work
in one axes object at a time.  It can not be in two places at once.  Make
two rectangles and attach each to an axes, and it will work.

Cheers!
Ben Root