matplotlib-users Mailing List for matplotlib

Hi Everyone,

I'm having problems when rasterizing many lines in a plot using the
rasterized=True keyword using the pdf output.
Some version info:
matplotlib version 1.1.1rc
ubuntu 12.04
python 2.7.3


Here's a basic example that demonstrates my problem:
# Import matplotlib to create a pdf document
import matplotlib
matplotlib.use('Agg')
from matplotlib.backends.backend_pdf import PdfPages
pdf = PdfPages('rasterized_test.pdf')

import matplotlib.pylab as plt

# some test data
import numpy as np
ts = np.linspace(0,2*np.pi,100) * np.ones((200,100))
ts += (np.linspace(0, np.pi, 200)[np.newaxis] * np.ones((100,200))).T
ys = np.sin(ts)

fig = plt.figure()
ax = fig.add_subplot(111)
ax.plot(ts[0], ys.T, color='r', lw=0.5, alpha=0.5, rasterized=True)
pdf.savefig()

pdf.close()



Essentially, I have a lot (200 in this case) of closely overlapping lines
which makes the resulting figure (not rasterized) overly difficult to load.
I would like to rasterize these lines, such that the axis labels (and other
elements of the plot, not shown) remain vectors while the solution
trajectories are flattened to a single raster background. However, using
the code above, the image still takes a long time to load since each
trajectory is independently rasterized, resulting in multiple layers. (If I
open the resulting pdf with a program like inkscape, I can manipulate each
trajectory independently.)

Is it possible to flatten all of the rasterized elements into a single
layer, so the pdf size would be greatly reduced?

Thanks,
--Peter

On Tue, Aug 21, 2012 at 8:56 AM, Virgil Stokes <vs...@it...> wrote:
> On 21-Aug-2012 17:50, Paul Hobson wrote:
>>
>> On Tue, Aug 21, 2012 at 7:58 AM, Virgil Stokes <vs...@it...> wrote:
>>>
>>> In reference to my previous email.
>>>
>>> How can I find the outliers (samples points beyond the whiskers) in the
>>> data
>>> used for the boxplot?
>>>
>>> Here is a code snippet that shows how it was used for the timings data (a
>>> list
>>> of 4 sublists (y1,y2,y3,y4), each containing 400,000 real data values),
>>>     ...
>>>     ...
>>>     ...
>>>     # Box Plots
>>>     plt.subplot(2,1,2)
>>>     timings = [y1,y2,y3,y4]
>>>     pos = np.array(range(len(timings)))+1
>>>     bp = plt.boxplot( timings, sym='k+', patch_artist=True,
>>>                      positions=pos, notch=1, bootstrap=5000 )
>>>
>>>     plt.xlabel('Algorithm')
>>>     plt.ylabel('Exection time (sec)')
>>>     plt.ylim(0.9*ymin,1.1*ymax)
>>>
>>>     plt.setp(bp['whiskers'], color='k',  linestyle='-' )
>>>     plt.setp(bp['fliers'], markersize=3.0)
>>>     plt.title('Box plots (%4d trials)' %(n))
>>>     plt.show()
>>>     ...
>>>     ...
>>>     ...
>>>
>>> Again my questions:
>>> 1) How to get the value of the median?
>>> 2) How to find the outliers (outside the whiskers)?
>>> 3) How to find the width of the notch?
>>
>> Virgil, the objects stuffed inside the `bp` dictionary should have
>> methods to retrieve their values. Let's see:
>>
>> In [35]: x = np.random.lognormal(mean=1.25, sigma=1.35, size=(37,3))
>>
>> In [36]: bp = plt.boxplot(x, bootstrap=5000, notch=True)
>>
>> In [37]: # Question 1
>>      ...: print('medians')
>>      ...: for n, median in enumerate(bp['medians']):
>>      ...:     print('%d: %f' % (n, median.get_ydata()[0]))
>>      ...:
>> medians
>> 0: 6.339692
>> 1: 3.449320
>> 2: 4.503706
>>
>> In [38]: # Question 2
>>      ...: print('fliers')
>>      ...: for n in range(0, len(bp['fliers']), 2):
>>      ...:     print('%d: upper outliers = \t' % (n/2,))
>>      ...:     print(bp['fliers'][n].get_ydata())
>>      ...:     print('\n%d: lower outliers = \t' % (n/2,))
>>      ...:     print(bp['fliers'][n+1].get_ydata())
>>      ...:     print('\n')
>>      ...:
>
> You had no outliers!
>
>>
>> In [39]: # Question 3
>>      ...: print('Confidence Intervals')
>>      ...: for n, box in enumerate(bp['boxes']):
>>      ...:     print('%d: lower CI: %f' % (n, box.get_ydata()[2]))
>>      ...:     print('%d: upper CI: %f' % (n, box.get_ydata()[4]))
>>      ...:
>> Confidence Intervals
>> 0: lower CI: 1.760701
>> 0: upper CI: 10.102221
>> 1: lower CI: 1.626386
>> 1: upper CI: 5.601927
>> 2: lower CI: 2.173173
>>
>> Hope that helps,
>> -paul
>
> Just what I was looking for Paul! Thanks very much.
>
> One final question --- Where can I find the documentation that answers my
> questions and gives more details about the equations used for the width of
> notch. etc.?
>
> Thanks again :-)

That should all be in the boxplot docstring. Do you use ipython? If
not, you should :)

if so, just do `plt.boxplot?` at the ipython terminal and it'll show up.
-paul

On Tue, Aug 21, 2012 at 7:58 AM, Virgil Stokes <vs...@it...> wrote:
> In reference to my previous email.
>
> How can I find the outliers (samples points beyond the whiskers) in the data
> used for the boxplot?
>
> Here is a code snippet that shows how it was used for the timings data (a list
> of 4 sublists (y1,y2,y3,y4), each containing 400,000 real data values),
>    ...
>    ...
>    ...
>    # Box Plots
>    plt.subplot(2,1,2)
>    timings = [y1,y2,y3,y4]
>    pos = np.array(range(len(timings)))+1
>    bp = plt.boxplot( timings, sym='k+', patch_artist=True,
>                     positions=pos, notch=1, bootstrap=5000 )
>
>    plt.xlabel('Algorithm')
>    plt.ylabel('Exection time (sec)')
>    plt.ylim(0.9*ymin,1.1*ymax)
>
>    plt.setp(bp['whiskers'], color='k',  linestyle='-' )
>    plt.setp(bp['fliers'], markersize=3.0)
>    plt.title('Box plots (%4d trials)' %(n))
>    plt.show()
>    ...
>    ...
>    ...
>
> Again my questions:
> 1) How to get the value of the median?
> 2) How to find the outliers (outside the whiskers)?
> 3) How to find the width of the notch?

Ooops. Here's my reply -- this time to whole list
Virgil, the objects stuffed inside the `bp` dictionary should have
methods to retrieve their values. Let's see:

In [35]: x = np.random.lognormal(mean=1.25, sigma=1.35, size=(37,3))

In [36]: bp = plt.boxplot(x, bootstrap=5000, notch=True)

In [37]: # Question 1
    ...: print('medians')
    ...: for n, median in enumerate(bp['medians']):
    ...:     print('%d: %f' % (n, median.get_ydata()[0]))
    ...:
medians
0: 6.339692
1: 3.449320
2: 4.503706

In [38]: # Question 2
    ...: print('fliers')
    ...: for n in range(0, len(bp['fliers']), 2):
    ...:     print('%d: upper outliers = \t' % (n/2,))
    ...:     print(bp['fliers'][n].get_ydata())
    ...:     print('\n%d: lower outliers = \t' % (n/2,))
    ...:     print(bp['fliers'][n+1].get_ydata())
    ...:     print('\n')
    ...:

In [39]: # Question 3
    ...: print('Confidence Intervals')
    ...: for n, box in enumerate(bp['boxes']):
    ...:     print('%d: lower CI: %f' % (n, box.get_ydata()[2]))
    ...:     print('%d: upper CI: %f' % (n, box.get_ydata()[4]))
    ...:
Confidence Intervals
0: lower CI: 1.760701
0: upper CI: 10.102221
1: lower CI: 1.626386
1: upper CI: 5.601927
2: lower CI: 2.173173

Hope that helps,
-paul

On Aug 21, 2012, at 10:58 AM, Virgil Stokes wrote:

> In reference to my previous email.
>
> How can I find the outliers (samples points beyond the whiskers) in  
> the data
> used for the boxplot?
>
> Here is a code snippet that shows how it was used for the timings  
> data (a list
> of 4 sublists (y1,y2,y3,y4), each containing 400,000 real data  
> values),
>    ...
>    ...
>    ...
>    # Box Plots
>    plt.subplot(2,1,2)
>    timings = [y1,y2,y3,y4]
>    pos = np.array(range(len(timings)))+1
>    bp = plt.boxplot( timings, sym='k+', patch_artist=True,
>                     positions=pos, notch=1, bootstrap=5000 )
>
>    plt.xlabel('Algorithm')
>    plt.ylabel('Exection time (sec)')
>    plt.ylim(0.9*ymin,1.1*ymax)
>
>    plt.setp(bp['whiskers'], color='k',  linestyle='-' )
>    plt.setp(bp['fliers'], markersize=3.0)
>    plt.title('Box plots (%4d trials)' %(n))
>    plt.show()
>    ...
>    ...
>    ...
>
> Again my questions:
> 1) How to get the value of the median?

This is easily calculated from your data. Numpy will even do it for  
you: np.median(timings)

> 2) How to find the outliers (outside the whiskers)?

 From the boxplot documentation: the whiskers extend to the most  
extreme data point within distance X of the bottom or top of the box,  
where X is 1.5 times the extent of the box. Any points more extreme  
than that are the outliers. The box itself of course extends from the  
25th percentile to the 75th percentile of your data. Again, you can  
easily calculate these values from your data.

> 3) How to find the width of the notch?

Again, from the docs: with bootstrap=5000, it calculates the width of  
the notch by bootstrap resampling your data (the timings array) 5000  
times and finding the 95% confidence interval of the median, and uses  
that as the notch width. You can redo that yourself pretty easily.  
Here is some bootstrap code for you to adapt:
http://mail.scipy.org/pipermail/scipy-user/2009-July/021704.html

I encourage you to read the documentation! This page is very useful  
for reference:
http://matplotlib.sourceforge.net/api/pyplot_api.html

-Jeff

In reference to my previous email.

How can I find the outliers (samples points beyond the whiskers) in the data 
used for the boxplot?

Here is a code snippet that shows how it was used for the timings data (a list 
of 4 sublists (y1,y2,y3,y4), each containing 400,000 real data values),
   ...
   ...
   ...
   # Box Plots
   plt.subplot(2,1,2)
   timings = [y1,y2,y3,y4]
   pos = np.array(range(len(timings)))+1
   bp = plt.boxplot( timings, sym='k+', patch_artist=True,
                    positions=pos, notch=1, bootstrap=5000 )

   plt.xlabel('Algorithm')
   plt.ylabel('Exection time (sec)')
   plt.ylim(0.9*ymin,1.1*ymax)

   plt.setp(bp['whiskers'], color='k',  linestyle='-' )
   plt.setp(bp['fliers'], markersize=3.0)
   plt.title('Box plots (%4d trials)' %(n))
   plt.show()
   ...
   ...
   ...

Again my questions:
1) How to get the value of the median?
2) How to find the outliers (outside the whiskers)?
3) How to find the width of the notch?

http://housesforsells.com/wp-admin/makoler.html?tebv=vzbeba

On 8/20/12 10:26 PM, Scott Henderson wrote:
> I'm trying to efficiently get the distances of all points on a map to a
> specified point. If the map is in projected coordinates, what is the
> best way of going about this? Is there is a 'standard' way to get the
> distance between points through internal basemap functions? After some
> scrounging around what I have below works for individual points, but is
> there a way to avoid the big for loop?
>
> Any advice would be appreciated.
> Thanks.

Scott:  The pyproj Geod methods don't take numpy arrays, as you 
discovered. It's very accurate and robust, but slow since you have to 
loop over the points.  Since you are assuming the earth is a perfect 
sphere you could use the less accurate Haversine formula:

# function to compute great circle distance between point lat1 and lon1 
and arrays of points
# given by lons, lats
def get_dist(lon1,lons,lat1,lats):
     # great circle distance.
     arg = 
np.sin(lat1)*np.sin(lats)+np.cos(lat1)*np.cos(lats)*np.cos(lon1-lons)
     arg = np.where(np.fabs(arg) < 1., arg, 0.999999)
     return np.arccos(arg)

-Jeff

>
>
> # --------------------------------------
> from mpl_toolkits.basemap import Basemap
> from mpl_toolkits.basemap import pyproj
> import numpy as np
> import matplotlib.pyplot as plt
>
> fig = plt.figure()
> ax = fig.add_subplot(111)
>
> LL = (-69, -26)
> UR = (-66, -21)
> map = Basemap(projection='merc',
>                  llcrnrlat=LL[1],
>                  urcrnrlat=UR[1],
>                  llcrnrlon=LL[0],
>                  urcrnrlon=UR[0],
>                  resolution='i',
>                  suppress_ticks=False,
>                  ax=ax)
>
> lons, lats, xs, ys = map.makegrid(200, 200, returnxy=True)
> lon1, lat1 = (-67.28, -23.78)
>
> gc = pyproj.Geod(a=map.rmajor, b=map.rminor)
> #azis12, azis21, distances = gc.inv(lon1,lat1,lons,lats) #doesn't work
> with vectors or matrices?
> distances = np.zeros(lons.size)
> for i, (lon, lat) in enumerate(zip(lons.flatten(),lats.flatten())):
>       azi12, azi21, distances[i] = gc.inv(lon1,lat1,lon,lat)
> distances = distances.reshape(200,200) / 1000.0 #in km
>
> # Plot perimeters of equal distance
> levels = [50] #[50,100,150]
> map.drawcountries()
> x1,y1 = map(lon1,lat1)
> map.plot(x1,y1,'m*')
> cs = map.contour(xs, ys, distances, levels)
>

I'm trying to efficiently get the distances of all points on a map to a 
specified point. If the map is in projected coordinates, what is the 
best way of going about this? Is there is a 'standard' way to get the 
distance between points through internal basemap functions? After some 
scrounging around what I have below works for individual points, but is 
there a way to avoid the big for loop?

Any advice would be appreciated.
Thanks.


# --------------------------------------
from mpl_toolkits.basemap import Basemap
from mpl_toolkits.basemap import pyproj
import numpy as np
import matplotlib.pyplot as plt

fig = plt.figure()
ax = fig.add_subplot(111)

LL = (-69, -26)
UR = (-66, -21)
map = Basemap(projection='merc',
                llcrnrlat=LL[1],
                urcrnrlat=UR[1],
                llcrnrlon=LL[0],
                urcrnrlon=UR[0],
                resolution='i',
                suppress_ticks=False,
                ax=ax)

lons, lats, xs, ys = map.makegrid(200, 200, returnxy=True)
lon1, lat1 = (-67.28, -23.78)

gc = pyproj.Geod(a=map.rmajor, b=map.rminor)
#azis12, azis21, distances = gc.inv(lon1,lat1,lons,lats) #doesn't work 
with vectors or matrices?
distances = np.zeros(lons.size)
for i, (lon, lat) in enumerate(zip(lons.flatten(),lats.flatten())):
     azi12, azi21, distances[i] = gc.inv(lon1,lat1,lon,lat)
distances = distances.reshape(200,200) / 1000.0 #in km

# Plot perimeters of equal distance
levels = [50] #[50,100,150]
map.drawcountries()
x1,y1 = map(lon1,lat1)
map.plot(x1,y1,'m*')
cs = map.contour(xs, ys, distances, levels)

-- 
---------------
Scott T. Henderson
http://www.geo.cornell.edu/eas/gstudent/sth54/contact.html

On 8/20/12 8:21 PM, Scott Henderson wrote:
> On Mon 20 Aug 2012 06:29:01 PM EDT, Jeff Whitaker wrote:
>> On 8/20/12 4:41 PM, Scott Henderson wrote:
>>> I'm having trouble with transform_scalar() and imshow() with basemap.
>>> Essential I have data from satellite tracks that are either smaller
>>> or larger than the map extent, so I don't want to use
>>> Basemap.imshow() which sets the 'extent' keyword automatically.
>>>
>>> I tried following the following suggestion, but I can't get things to
>>> line up..
>>> http://comments.gmane.org/gmane.comp.python.matplotlib.general/25085
>>>
>>> I suspect I've done something wrong in the code below?:
>>
>> Scott:  If you use the pcolormesh or contourf Basemap methods (instead
>> of imshow), you can specify the map projection coordinates of your
>> data and it will be plotted correctly (whether or not it covers the
>> whole map projection region).
>>
>> -Jeff
>>>
>>>
>>> # -------------------------------
>>> from mpl_toolkits.basemap import Basemap
>>> import matplotlib.pyplot as plt
>>>
>>> LL = (-68, -26)
>>> UR = (-66, -21)
>>> bmap = Basemap(projection='merc',
>>>                llcrnrlat=LL[1],
>>>                urcrnrlat=UR[1],
>>>                llcrnrlon=LL[0],
>>>                urcrnrlon=UR[0],
>>>                resolution='i',
>>>                suppress_ticks=False,
>>>                ax=ax)
>>> bmap.drawcountries(linewidth=1)
>>>
>>> # Overlay image with extents that don't match map boundaries
>>> nLon, nLat = image.shape
>>> LL = (-69.915, -31.161)
>>> UR = (-65.075, -17.334)
>>> dy = -0.006666664
>>> lats = UR[1] + dy * np.arange(nLat)
>>> extent = (LL[0], UR[0], LL[1], UR[1])
>>> nx = nLon; ny = nLat
>>> image_xy = bmap.transform_scalar(image, lons, lats[::-1], nx, ny)
>>> x, y = bmap(extent[:2],extent[2:])
>>> extent_xy = (x[0], x[1], y[0], y[1])
>>> im = plt.imshow(image_xy, extent=extent_xy)
>>>
>>>
>>>
>>> -- 
>>> ---------------
>>> Scott T. Henderson
>>> http://www.geo.cornell.edu/eas/gstudent/sth54/contact.html
>>>
>>>
>>> ------------------------------------------------------------------------------ 
>>>
>>> Live Security Virtual Conference
>>> Exclusive live event will cover all the ways today's security and
>>> threat landscape has changed and how IT managers can respond. 
>>> Discussions
>>> will include endpoint security, mobile security and the latest in 
>>> malware
>>> threats.http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>>>
>>>
>>> _______________________________________________
>>> Matplotlib-users mailing list
>>> Mat...@li...
>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>>
>>
>> -- 
>> Jeffrey S. Whitaker         Phone  : (303)497-6313
>> Meteorologist               FAX    : (303)497-6449
>> NOAA/OAR/PSD  R/PSD1        Email  :Jef...@no...
>> 325 Broadway                Office : Skaggs Research Cntr 1D-113
>> Boulder, CO, USA 80303-3328 Web    :http://tinyurl.com/5telg
>
> Thanks Jeff,
>
> pcolormesh seems to get the georeferencing correct. Why is it that 
> imshow doesn't work? The trouble with pcolormesh is that it's much 
> slower, the file sizes get bigger and when zooming into detailed 
> regions the pixels edges show up even when I set edgecolors='None'. 
> I've attached a few screenshots for comparison.

Scott: That's weird - I don't know why the edges show.  The only thing 
you're getting with imshow is interpolation.  To use imshow, you need to 
have a grid that exactly fits the map projection region.  You could do 
this with transform_scalar using the keyword masked=True, which will 
assign missing values to grid points outside the range of the input 
grid.  When you plot with imshow, these areas should just show up as 
transparent.

-Jeff
>
> # New version:
> # -------------------
> data_xy, x, y = bmap.transform_scalar(data[::-1], lons, lats[::-1], 
> nx, ny, returnxy=True)
> data_xym =  np.ma.array(data_xy, mask=np.isnan(data_xy))
> im = bmap.pcolormesh(x,y,data_xym,
>                     shading='flat', #'gouraud'
>                     edgecolors='None',
>                     alpha=alpha,
>                     vmin=cmin,
>                     vmax=cmax)
> plt.colorbar(im)
>
>
> # Different figures attached
>
>
>
> -- 
> ---------------
> Scott T. Henderson
> http://www.geo.cornell.edu/eas/gstudent/sth54/contact.html
>

Hmm, I just found out that if I change path.Path.contains_point to use
"point_on_path" instead of "point_in_path", the containment tests work
properly.  I'm not that familiar with the path code...is the difference
that one is testing for polygonal insideness, and one is testing for
literally being on the "stroke"?  If so, do we have to make sure that the
proper one is called if there are no polygons involved in the path?

On Mon, Aug 20, 2012 at 9:28 PM, Daniel Hyams <dh...@gm...> wrote:

> I've run into a strange problem with contains() on an arrow; there is a
> large area to the left of the arrow that insists that it is contained
> within the arrow.  Small runnable sample attached.
>
> I've looked at the path for the arrow, and it looks fine to me.  I even
> went so far as to hack a STOP onto the end of the path, but that resulted
> in the same behavior.
>
> Can anyone else confirm this behavior?  matplotlib 1.1.1 is what I'm
> using.  Seen on both Windows, Linux, and OSX.
>
> --
> Daniel Hyams
> dh...@gm...
>



-- 
Daniel Hyams
dh...@gm...

I've run into a strange problem with contains() on an arrow; there is a
large area to the left of the arrow that insists that it is contained
within the arrow.  Small runnable sample attached.

I've looked at the path for the arrow, and it looks fine to me.  I even
went so far as to hack a STOP onto the end of the path, but that resulted
in the same behavior.

Can anyone else confirm this behavior?  matplotlib 1.1.1 is what I'm using.
 Seen on both Windows, Linux, and OSX.

-- 
Daniel Hyams
dh...@gm...