Python Programming

for
Data Processing and Climate Analysis

Jules Kouatchou and Hamid Oloso

[email protected] and [email protected]

Goddard Space Flight Center

Software System Support Office
Code 610.3

March 25, 2013

 Background Information

Training Objectives

We want to introduce:
Basic concepts of Python programming
Array manipulations
Handling of files
2D visualization
EOFs

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 2 / 94
 Background Information

Special Topics

Based on the feedback we have received so far, we plan to have a hand-on

presentation on the following topic(s):
F2Py: Python interface to Fortran
Tentative Date: April 29, 2013 at 1:30pm

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 3 / 94
 Background Information

Obtaining the Material

Slides for this session of the training are available from:

https://modelingguru.nasa.gov/docs/DOC-2322

You can obtain materials presented here on discover at

/discover/nobackup/jkouatch/pythonTrainingGSFC.tar.gz

After you untar the above file, you will obtain the directory
pythonTrainingGSFC/ that contains:

Examples/
Slides/

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 4 / 94
 Background Information

Settings on discover

We installed a Python distribution. To use it, you need to load the

modules:

module load other/comp/gcc-4.5-sp1

module load lib/mkl-10.1.2.024
module load other/SIVO-PyD/spd_1.7.0_gcc-4.5-sp1

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 5 / 94
 Background Information

What Have We Learned So Far?

Strings ’spam’, ”guido’s”

Lists [1, [2,’tree’], 4]
Dictionaries ’food’:’spam’, ’taste’:’yum’
Tuples (1,’spam’, 4, ’U’)
NumPy Arrays arange(a, b, m)
linspace(a, b, n)
array(list)

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 6 / 94
 Background Information

What Will be Covered Today

1 Matplotlib 2 netCDF4
2D Plot 3 H5Py
3D Plot 4 Visualization
Basemap toolkit Session

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 7 / 94
 Matplotlib

Matplolib

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 8 / 94
 Matplotlib

Useful Links for Matplotlib

Video Presentation
http://videolectures.net/mloss08_hunter_mat

User’s Guide
http://mural.uv.es/parmur/matplotlib.pdf

Image Galery
http://matplotlib.sourceforge.net/gallery.html

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 9 / 94
 Matplotlib

What is Matplotlib?

Library for making 2D plots of arrays in Python

Makes heavy use of Numpy and other extension code to provide good
performance
Can be used to create plots with few commands

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 10 / 94
 Matplotlib

What Can we Do with Matplotlib?

You can generate plots, histograms, power spectra, bar charts, error
charts, scatter plots, etc., with just a few lines of code.

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 11 / 94
 Matplotlib

Two Main Interfaces of Matplotlib

pyplot
Provides a Matlab-style state-machine interface to the
underlying object-oriented plotting library in matplotlib.
Preferred method of access for interactive plotting.
pylab
Combines the pyplot functionality (for plotting) with the
Numpy functionality (for mathematics and for working
with arrays) in a single namespace, making that
namespace (or environment) even more Matlab-like.
Formerly preferred method of access for interactive
plotting, but still available.

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 12 / 94
 Matplotlib

pyplot vs. pylab

pyplot: pylab:

import matplotlib.pyplot from pylab import *

import numpy as np

x = np.arrange(0, 10, 0.2) x = arange(0, 10, 0.2)

y = np.sin(x) y = sin(x)

pyplot.plot(x, y) plot(x, y)

pyplot.show() show()

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 13 / 94
 Matplotlib 2D Plot

Syntax for Plotting

1 # !/ usr / bin / env python

2 import matplotlib . pyplot as plt
3
4 x = [...] # define the points on the x - axis
5 y = [...] # define the points on the y - axis
6
7 plt . plot (x , y )
8 plt . show () # display the plot on the screen

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 14 / 94
 Matplotlib 2D Plot

Creating a Basic Graph

1 # !/ usr / bin / env python

2 import matplotlib . pyplot as plt
3
4 x = [2 , 3 , 5 , 7 , 11]
5 y = [4 , 9 , 5 , 9 , 1]
6 plt . plot (x , y )
7 plt . show ()

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 15 / 94
 Matplotlib 2D Plot

Basic Graph

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 16 / 94
 Matplotlib 2D Plot

Some pyplot Functions

plot(x,y)
xlabel(’string’) # label the x-axis
ylabel(’string’) # label the y-axis
title(’string’) # write the title of the plot
grid(true/false) # adds grid boxes
savefig(’fileName.type’) # type can be png, ps, pdf, etc
show() # display the graph on the screen
xlim(xmin,xmax) # set/get the xlimits
ylim(ymin,ymax) # set/get the ylimits
hold(True/False) # to overlay figures on the same grap

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 17 / 94
 Matplotlib 2D Plot

Code for Plotting the Cosine Function

1 # !/ usr / bin / env python

2 import math
3 import numpy as np
4 import matplotlib . pyplot as plt
5
6 t = np . arange (0.0 , 1.0+0.01 , 0.01)
7 s = np . cos (2*2* math . pi * t )
8 plt . plot (t , s )
9
10 plt . xlabel ( ’ time ( s ) ’)
11 plt . ylabel ( ’ voltage ( mV ) ’)
12 plt . title ( ’ About as simple as it gets , folks ’)
13 plt . grid ( True )
14 plt . savefig ( ’ simple_plot ’)

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 18 / 94
 Matplotlib 2D Plot

Simple Cosine Plot

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 19 / 94
 Matplotlib 2D Plot

Two Figures on the Same Plot

1 import numpy as np
2 import matplotlib . pyplot as plt
3

4 def f ( t ):
5 return np . exp ( - t ) * np . cos (2* np . pi * t )
6
7 t1 = np . arange (0.0 , 5.0 , 0.1)
8 t2 = np . arange (0.0 , 5.0 , 0.02)
9

10 plt . figure (1)

11 plt . subplot (211)
12 plt . plot ( t1 , f ( t1 ) , ’ bo ’ , t2 , f ( t2 ) , ’k ’)
13
14 plt . subplot (212)
15 plt . plot ( t2 , np . cos (2* np . pi * t2 ) , ’r - - ’)
16 plt . show ()

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 20 / 94
 Matplotlib 2D Plot

Graph of Two Figures on the Same Plot

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 21 / 94
 Matplotlib 2D Plot

Syntax for Plotting Multiples Figures and Axes

figure(num)
# allows to plot multiple figures at the same time
# can be called several times
# num: reference number to keep tract of the figure object

subplot(numrows, numcols, fignum)

# fignum range from numrows*numcols
# subplot(211) is identical to subplot(2,1,1)

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 22 / 94
 Matplotlib 2D Plot

Sample Code for Plotting Four Figures and Axes

1 plt . subplot (2 ,2 ,1)
2 plt . plot (x , y01 , linewidth =3); plt . hold ( True )
3 plt . plot (x , y02 , ’r ’ , linewidth =3)
4
5 plt . subplot (2 ,2 ,2)
6 plt . plot ( y03 , linewidth =2)
7
8 plt . subplot (2 ,2 ,3)
9 plt . plot (x , y04 , ’k ’ , linewidth =3); plt . hold ( True )
10 plt . subplot (2 ,2 ,3)
11 plt . plot (x , y05 , ’ -- ’ , linewidth =3)
12 plt . subplot (2 ,2 ,3)
13 plt . plot (x , y06 , ’r ’ , linewidth =2)
14
15 plt . subplot (2 ,2 ,4)
16 plt . plot ( Y04 , linewidth =2.5)

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 23 / 94
 Matplotlib 2D Plot

Example of Graph with Four Figures on the Same Plot

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 24 / 94
 Matplotlib 2D Plot

Sample Pie Chart

1 figure (1 , figsize =(6 ,6))

2 ax = axes ([0.1 , 0.1 , 0.8 , 0.8])
3
4 labels = ’ Frogs ’ , ’ Hogs ’ , ’ Dogs ’ , ’ Logs ’
5 fracs = [15 , 30 , 45 , 10]
6
7 explode =(0 , 0.05 , 0 , 0)
8

9 pie ( fracs , explode = explode , labels = labels )

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 25 / 94
 Matplotlib 2D Plot

Graph for a Pie Chart

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 26 / 94
 Matplotlib 2D Plot

Sample Histogram
1 import numpy as np
2 import matplotlib . pyplot as plt
3

4 mu , sigma = 100 , 15
5 x = mu + sigma * np . random . randn (10000)
6
7 # the histogram of the data
8 n , bins , patches = plt . hist (x , 50 , normed =1 , \
9 facecolor = ’g ’ , alpha =0.75)
10
11 plt . xlabel ( ’ Smarts ’)
12 plt . ylabel ( ’ Probability ’)
13 plt . title ( ’ Histogram of IQ ’)
14 plt . text (60 , .025 , r ’$ \ mu =100 ,\ \ sigma =15 $ ’)
15 plt . axis ([40 , 160 , 0 , 0.03])
16 plt . grid ( True )

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 27 / 94
 Matplotlib 2D Plot

Graph for an Histogram

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 28 / 94
 Matplotlib 2D Plot

Using Mathematical Expressions in Text

Matplotlib accepts TeX equation expressions in any text.

Matplotlib has a built-in TeX parser
To write the expression σi = 15 in the title, you can write:
plt.title(r’$\sigma_i=15$’)

where r signifies that the string is a raw string and not to treat backslashes
and python escapes.

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 29 / 94
 Matplotlib 2D Plot

Sample Code for Annotating Text

1 import numpy as np
2 import matplotlib . pyplot as plt
3
4 ax = plt . subplot (111)
5 t = np . arange (0.0 , 5.0 , 0.01)
6 s = np . cos (2* np . pi * t )
7 line , = plt . plot (t , s , lw =2)
8
9 plt . annotate ( ’ local max ’ , xy =(2 , 1) , \
10 xytext =(3 , 1.5) , \
11 arrowprops = dict ( facecolor = ’ black ’ , \
12 shrink =0.05) , )
13
14 plt . ylim ( -2 ,2)
15 plt . show ()

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 30 / 94
 Matplotlib 2D Plot

Graph for Annotating Text

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 31 / 94
 Matplotlib 2D Plot

Log Plots

Use the following pyplot functions:

semilogx() # make a plot with log scaling on the x axis

semilogy() # make a plot with log scaling on the y axis
loglog() # make a plot with log scaling on the x and y axis

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 32 / 94
 Matplotlib 2D Plot

Graph with Log Plots

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 33 / 94
 Matplotlib 2D Plot

Sample Code for Plot with Fill

1 import numpy as np
2 import matplotlib . pyplot as plt
3
4 t = np . arange (0.0 , 1.01 , 0.01)
5 s = np . sin (2*2* np . pi * t )
6
7 plt . fill (t , s * np . exp ( -5* t ) , ’r ’)
8 plt . grid ( True )
9 plt . show ()

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 34 / 94
 Matplotlib 2D Plot

Graph for Plot with Fill

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 35 / 94
 Matplotlib 2D Plot

Legend

Call signatute:

legend(*args, **kwargs)

Place a legend on the current axes at location loc

Labels are a sequence of strings
loc can be a string or an integer

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 36 / 94
 Matplotlib 2D Plot

Sample Legend Commands

# make a legend with existing lines

legend()

# automatically generate the legend from labels

legend( (’label1’, ’label2’, ’label3’) )

# Make a legend for a list of lines and labels

legend( (line1, line2, line3), (’label1’, ’label2’, ’label3’) )

# make a legend at a given location, using a location argument

legend( (’label1’, ’label2’, ’label3’), loc=’upper left’)
legend( (line1, line2, line3), (’label1’, ’label2’, ’label3’), loc=

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 37 / 94
 Matplotlib 2D Plot

A Graph with Legend

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 38 / 94
 Matplotlib 2D Plot

Another Graph with Legend

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 39 / 94
 Matplotlib 2D Plot

Colorbar
You need to include the call: colorbar()

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 40 / 94
 Matplotlib 2D Plot

Contour Plot

# Make a contour plot of an array Z.

# The level values are chosen automatically
contour(Z)

# X, Y specify the (x, y) coordinates of the surface

contour(X, Y, Z)

# contour N automatically-chosen levels

contour(Z,N)
contour(X,Y,Z,N)

# draw contour lines at the values specified in sequence V

contour(Z,V)
contour(X,Y,Z,V)

# fill the (len(V)-1) regions between the values in V

contourf(..., V)

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 41 / 94
 Matplotlib 2D Plot

A Graph with Contour Plot

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 42 / 94
 Matplotlib 2D Plot

Another Graph with Contour Plot

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 43 / 94
 Matplotlib 3D Plots

The mplot3d Module

The mplot3d toolkit adds simple 3d plotting capabilities to Matplotlib

by supplying an axis object that can create a 2d projection of a 3d
scene.
It produces a list of 2d lines and patches that are drawn by the
normal Matplotlib code.
The resulting graph will have the same look and feel as regular 2d
plots.
Provide the ability to rotate and zoom the 3d scene.

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 44 / 94
 Matplotlib 3D Plots

3d Graphs

Matplotlib’s 3D capabilities were added by incorporating:

1 from mpl_toolkits . mplot3d import Axes3D
2 import matplotlib . pyplot as plt
3

4 fig = plt . figure ()

5 ax = Axes3D ( fig )
6 x = ...
7 y = ...
8 z = ...
9 ax . TYPE_of_Plot (x ,y ,z , ...)

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 45 / 94
 Matplotlib 3D Plots

Example of 2D Function

Assume that we want to plot the function:

p
z = sin ( x 2 + y 2 )
−5 ≤ x, y ≤ 5

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 46 / 94
 Matplotlib 3D Plots

Code for Plotting 2D Function

1 # !/ usr / bin / env python
2
3 from mpl_toolkits . mplot3d import Axes3D
4 from matplotlib import cm
5 import matplotlib . pyplot as plt
6 import numpy as np
7
8 fig = plt . figure ()
9 ax = Axes3D ( fig )
10 X = np . arange ( -5 , 5 , 0.25)
11 Y = np . arange ( -5 , 5 , 0.25)
12 X , Y = np . meshgrid (X , Y )
13 R = np . sqrt ( X **2 + Y **2)
14 Z = np . sin ( R )
15 ax . plot_surface (X , Y , Z , rstride =1 , cstride =1 , cmap = cm . j
16
17 plt . show ()

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 47 / 94
 Matplotlib 3D Plots

3D Surface Demo

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 48 / 94
 Matplotlib 3D Plots

Manipulating Images

1 from pylab import imread , imshow

2
3 a = imread ( ’ myImage . png ’)
4
5 imshow ( a )

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 49 / 94
 Matplotlib Visualizing Geographical Data

Plotting Geographical Data Using Basemap

Matplotlib toolkit
Collection of application-specific functions that extends Matplotlib
functionalities
Provides an efficient way to draw Matplotlib plots over real world
maps
Useful for scientists such as oceanographers and meteorologists.

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 50 / 94
 Matplotlib Visualizing Geographical Data

Defining a Basemap Object

1 import matplotlib . pyplot as plt
2 from mpl_toolkits . basemap import Basemap
3 import numpy as np
4
5 # Lambert Conformal map of USA lower 48 states
6 m = Basemap ( llcrnrlon = -119 ,
7 llcrnrlat =22 ,
8 urcrnrlon = -64 ,
9 urcrnrlat =49 ,
10 projection = ’ lcc ’ ,
11 lat_1 =33 ,
12 lat_2 =45 ,
13 lon_0 = -95 ,
14 resolution = ’h ’ ,
15 area_thresh =10000)

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 51 / 94
 Matplotlib Visualizing Geographical Data

Arguments for Defining a Basemap Object

projection: Type of map projection used

lat_1: First standard parallel for lambert conformal, albers equal area
and equidistant conic
lat_2: Second standard parallel for lambert conformal, albers equal area
and equidistant conic.
lon_0: Central meridian (x-axis origin) - used by all projections
llcrnrlon: Longitude of lower-left corner of the desired map domain
llcrnrlat: Latitude of lower-left corner of the desired map domain
urcrnrlon: Longitude of upper-right corner of the desired map domain
urcrnrlat: Latitude of upper-right corner of the desired map domain
resolution: Specifies what the resolution is of the features added to the map
(such as coast lines, borders, and so on), here we have chosen high
resolution (h), but crude, low, and intermediate are also available.
area_thresh: Specifies what the minimum size is for a feature to be plotted.
In this case, only features bigger than 10,000 square kilometer

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 52 / 94
 Matplotlib Visualizing Geographical Data

Defining Borders

1 # draw the coastlines of continental area

2 m . drawcoastlines ()
3
4 # draw country boundaries
5 m . drawcountries ( linewidth =2)
6
7 # draw states boundaries ( America only )
8 m . drawstates ()

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 53 / 94
 Matplotlib Visualizing Geographical Data

Coloring the Map

1 # fill the background ( the oceans )

2 m . drawmapboundary ( fill_color = ’ aqua ’)
3
4 # fill the continental area
5 # we color the lakes like the oceans
6 m . fillcontinents ( color = ’ coral ’ , lake_color = ’ aqua ’)

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 54 / 94
 Matplotlib Visualizing Geographical Data

Drawing Parallels and Meridians

1 # We draw a 20 degrees graticule of parallels and

2 # meridians for the map .
3 # Note how the labels argument controls the
4 # positions where the graticules are labeled
5 # labels =[ left , right , top , bottom ]
6
7 m . drawparallels ( np . arange (25 ,65 ,20) , labels =[1 ,0 ,0 ,0])
8 m . drawmeridians ( np . arange ( -120 , -40 ,20) , labels =[0 ,0 ,0 ,1])

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 55 / 94
 Matplotlib Visualizing Geographical Data

US Map

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 56 / 94
 Matplotlib Visualizing Geographical Data

Using Satellite Background

1 # display blue marble image ( from NASA )

2 # as map background
3 m . bluemarble ()

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 57 / 94
 Matplotlib Visualizing Geographical Data

US Map with Satellite Background

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 58 / 94
 Matplotlib Visualizing Geographical Data

Cities over a Map

1 cities = [ ’ London ’ , ’ New York ’ , ’ Madrid ’ , ’ Cairo ’ ,
2 ’ Moscow ’ , ’ Delhi ’ , ’ Dakar ’]
3 lat = [51.507778 , 40.716667 , 40.4 , 30.058 , 55.751667 ,
4 28.61 , 14.692778]
5 lon = [ -0.128056 , -74 , -3.683333 , 31.229 , 37.617778 ,
6 77.23 , -17.446667]
7
8 m = Basemap ( projection = ’ ortho ’ , lat_0 =45 , lon_0 =10)
9 m . drawmapboundary ()
10 m . drawcoastlines ()
11 m . fillcontinents ()
12
13 x , y = m ( lon , lat )
14 plt . plot (x , y , ’ ro ’)
15 for city , xc , yc in zip ( cities , x , y ):
16 plt . text ( xc +250000 , yc -150000 , city , bbox = dict (
17 facecolor = ’ yellow ’ , alpha =0.5)

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 59 / 94
 Matplotlib Visualizing Geographical Data

Graph of Cities over a Map

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 60 / 94
 Matplotlib Visualizing Geographical Data

Plotting Data over a Map

1 # make up some data on a regular lat / lon grid .

2 nlats = 73; nlons = 145; delta = 2.* np . pi /( nlons -1)
3 lats = (0.5* np . pi - delta * np . indices (( nlats , nlons ))[0 ,: ,:])
4 lons = ( delta * np . indices (( nlats , nlons ))[1 ,: ,:])
5 wave = 0.75*( np . sin (2.* lats )**8* np . cos (4.* lons ))
6 mean = 0.5* np . cos (2.* lats )*(( np . sin (2.* lats ))**2 + 2.)
7
8 # compute native map projection coordinates of
9 # lat / lon grid .
10 x , y = m ( lons *180./ np . pi , lats *180./ np . pi )
11
12 # contour data over the map .
13 CS = m . contour (x ,y , wave + mean ,15 , linewidths =1.5)

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 61 / 94
 Matplotlib Visualizing Geographical Data

Graph of Data over a Map

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 62 / 94
 Matplotlib Visualizing Geographical Data

Graph of Data over a Map with Satellite Background

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 63 / 94
 netCDF4

netCDF4

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 64 / 94
 netCDF4

Useful Links for netCDF4

Introduction
http://netcdf4-python.googlecode.com/svn/trunk/docs/
netCDF4-module.html

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 65 / 94
 netCDF4

What is netCDF4?

Python interface to the netCDF version 4 library.

Can read and write files in both the new netCDF 4 and the netCDF 3
format.
Can create files that are readable by HDF5 utilities.
Relies on NumPy arrays.

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 66 / 94
 netCDF4

Opening a netCDF File

from netCDF4 import Dataset

ncFid = Dataset(ncFileName, mode=modeType, format=fileFormat)
ncFid.close()

modeType can be: ’w’, ’r+’, ’r’, or ’a’

fileFormat can be: ’NETCDF3 CLASSIC’, ’NETCDF3 64BIT’,
’NETCDF4 CLASSIC’, ’NETCDF4’

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 67 / 94
 netCDF4

Creating Dimensions in a netCDF File

1 time = ncFid . createDimension ( ’ time ’ , None )

2 lev = ncFid . createDimension ( ’ lev ’ , 72)
3 lat = ncFid . createDimension ( ’ lat ’ , 91)
4 lon = ncFid . createDimension ( ’ lon ’ , 144)
5
6 print ncFid . dimensions

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 68 / 94
 netCDF4

Creating Variables in a netCDF File

1 times = ncFid . createVariable ( ’ time ’ , ’ f8 ’ ,( ’ time ’ ,))

2 levels = ncFid . createVariable ( ’ lev ’ , ’ i4 ’ ,( ’ lev ’ ,))
3 latitudes = ncFid . createVariable ( ’ lat ’ , ’ f4 ’ ,( ’ lat ’ ,))
4 longitudes = ncFid . createVariable ( ’ lon ’ , ’ f4 ’ ,( ’ lon ’ ,))
5
6 temp = ncFid . createVariable ( ’ temp ’ , ’ f4 ’ , \
7 ( ’ time ’ , ’ lev ’ , ’ lat ’ , ’ lon ’ ,))

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 69 / 94
 netCDF4

Adding Variable Attributes in a netCDF File

1 ncFid . description = ’ Sample netCDF file ’

2 ncFid . history = ’ Created for GSFC on March 25 , 2013 ’
3 ncFid . source = ’ netCDF4 python tutorial ’
4 latitudes . units = ’ degrees north ’
5 longitudes . units = ’ degrees east ’
6 levels . units = ’ hPa ’
7 temp . units = ’K ’
8 times . units = ’ hours since 0001 -01 -01 00:00:00.0 ’
9 times . calendar = ’ gregorian ’

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 70 / 94
 netCDF4

Writing Data in a netCDF File

1 import numpy
2 latitudes [:] = numpy . arange ( -90 ,91 ,2.0)
3 longitudes [:] = numpy . arange ( -180 ,180 ,2.5)
4 levels [:] = numpy . arange (0 ,72 ,1)
5
6 from numpy . random import uniform
7 temp [0:5 ,: ,: ,:] = uniform (
8 size =(5 , levels . size , latitudes . size ,
9 longitudes . size ))

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 71 / 94
 netCDF4

Reading Data from a netCDF File

1 ncFid = Dataset ( ’ myFile . nc4 ’ , mode = ’r ’)

2 time = ncFid . variables [ ’ time ’ ][:]
3 lev = ncFid . variables [ ’ lev ’ ][:]
4 lat = ncFid . variables [ ’ lat ’ ][:]
5 lon = ncFid . variables [ ’ lon ’ ][:]
6
7 temp = ncFid . variables [ ’ temp ’ ][:]

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 72 / 94
 H5py

H5py

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 73 / 94
 H5py

Useful Links for H5py

Quick Start Guide

http://www.h5py.org/docs/intro/quick.html

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 74 / 94
 H5py

What is H5py?

A Python-HDF5 interface
Allows interaction with with files, groups and datasets using
traditional Python and NumPy syntax.
No need to know anything about HDF5 library.
The files it manipulates are ”plain-vanilla” HDF5 files.

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 75 / 94
 H5py

Opening a HDF5 File

import h5py
hFid = h5py.File(’myfile.h5’, modeType) ’
hFid.close()

modeType can be: ’w’, ’w-’, ’r+’, ’r’, or ’a’

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 76 / 94
 H5py

Creating Dimensions in a HDF5 File

1 lat = numpy . arange ( -90 ,91 ,2.0)
2 dset = hFid . require_dataset ( ’ lat ’ , shape = lat . shape )
3 dset [...] = lat
4 dset . attrs [ ’ name ’] = ’ latitude ’
5 dset . attrs [ ’ units ’] = ’ degrees north ’
6
7 lon = numpy . arange ( -180 ,180 ,2.5)
8 dset = hFid . require_dataset ( ’ lon ’ , shape = lon . shape )
9 dset [...] = lon
10 dset . attrs [ ’ name ’] = ’ longitude ’
11 dset . attrs [ ’ units ’] = ’ degrees east ’
12
13 lev = numpy . arange (0 ,72 ,1)
14 dset = hFid . require_dataset ( ’ lev ’ , shape = lev . shape )
15 dset [...] = lev
16 dset . attrs [ ’ name ’] = ’ vertical levels ’
17 dset . attrs [ ’ units ’] = ’ hPa ’

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 77 / 94
 H5py

Creating Variables in a HDF5 File

1 from numpy . random import uniform

2 arr = np . zeros ((5 , lev . size , lat . size , lon . size ))
3 arr [0:5 ,: ,: ,:] = uniform (
4 size =(5 , lev . size , lat . size , lon . size ))
5 dset = hFid . require_dataset ( ’ temp ’ , shape = arr . shape )
6 dset [...] = arr
7 dset . attrs [ ’ name ’] = ’ temperature ’
8 dset . attrs [ ’ units ’] = ’K ’

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 78 / 94
 H5py

Creating Groups in a HDF5 File

1 gpData2D = hFid . create_group ( ’2 D_Data ’)

2 sgpLand = gpData2D . create_group ( ’2 D_Land ’)
3 sgpSea = gpData2D . create_group ( ’2 D_Sea ’)
4
5 gpData3D = hFid . create_group ( ’3 D_Data ’)

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 79 / 94
 H5py

Writing Data in a Group

1 temp = gpData3D . create_dataset ( ’ temp ’ , data = arr )

2 temp . attrs [ ’ name ’] = ’ temperature ’
3 temp . attrs [ ’ units ’] = ’K ’

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 80 / 94
 H5py

Reading Data from a HDF5 File

1 hFid = h5py . File ( ’ myFile . h5 ’ , ’r ’)

2 lev = hFid [ ’ lev ’ ]. value
3 lat = hFid [ ’ lat ’ ]. value
4 lon = hFid [ ’ lon ’ ]. value
5 time = hFid [ ’ time ’ ]. value
6
7 temp1 = hFid [ ’ temp ’ ]. value
8

9 temp2 = hFid [ ’3 D_Data ’ ][ ’ temp ’ ]. value

10
11 hFid . close ()

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 81 / 94
 Visualizing Gridded Data

Visualizing Gridded Data

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 82 / 94
 Visualizing Gridded Data

Goals

Access a netCDF file

Retrieve data from the netCDF file
Manipulate and plot the data

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 83 / 94
 Visualizing Gridded Data

Code for Reading SLP

1 ncFid = Dataset ( fileName , mode = ’r ’)

2
3 lat = ncFid . variables [ ’ lat ’ ][:]
4 lon = ncFid . variables [ ’ lon ’ ][:]
5
6 slp = 0.01* ncFid . variables [ ’ SLP ’ ][:]
7
8 ncFid . close ()
9

10 nlat = lat . size - 1

11 nlon = lon . size - 1
12
13 mySLP = slp [0 ,: ,:]

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 84 / 94
 Visualizing Gridded Data

Code for Plotting

1 fig = plt . figure (1 , figsize =(15 ,8) , dpi =75)
2 ax = fig . add_axes ([0.05 ,0.05 ,0.9 ,0.85])
3 m = Basemap ( projection = ’ mill ’ ,
4 llcrnrlat = lat [0] , urcrnrlat = lat [ nlat ] ,
5 llcrnrlon = lon [0] , urcrnrlon = lon [ nlon ] )
6 m . drawcoastlines ( linewidth =1.25)
7 m . fillcontinents ( color = ’ 0.8 ’)
8 m . drawparallels ( np . arange ( -80 ,81 ,20) , labels =[1 ,1 ,0 ,0])
9 m . drawmeridians ( np . arange ( -180 ,180 ,60) , labels =[0 ,0 ,0 ,1])
10 im = m . imshow ( mySLP ,
11 interpolation = ’ nearest ’ ,
12 extent =[ lon [0] , lon [ nlon ] , lat [0] , lat [ nlat ]] ,
13 cmap = plt . cm . jet )
14 plt . colorbar ( orientation = ’ hoirzontal ’ , shrink =.8)
15 plt . title ( ’ Sea Level Pressure ’)
16 plt . savefig ( ’ fig_slp . png ’)
17 plt . show ()

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 85 / 94
 Visualizing Gridded Data

Plot of SLP

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 86 / 94
 Visualizing Gridded Data

Define a Generic Function for Contour Plots

1 def bmContourPlot ( var , lats , lons , figName , figTitle ):
2 plt . figure ()
3 latLow = lats [0]; latHigh = lats [ -1]
4 lonLow = lons [0]; lonHigh = lons [ -1]
5 m = Basemap ( projection = ’ mill ’ ,
6 llcrnrlat = latLow , urcrnrlat = latHigh ,
7 llcrnrlon = lonLow , urcrnrlon = lonHigh ,
8 resolution = ’c ’)
9 m . drawcoastlines ()
10 m . drawparallels ( np . arange ( latLow , latHigh +1 ,30.))
11 m . drawmeridians ( np . arange ( lonLow , lonHigh +1 ,60.))
12 longrid , latgrid = np . meshgrid ( lons , lats )
13 x , y = m ( longrid , latgrid )
14 m . contour (x ,y , var ); m . contourf (x ,y , var )
15 plt . title ( figTitle )
16 plt . colorbar ( shrink =.8)
17 plt . savefig ( figName + ’. png ’)
18 plt . show ()
J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 87 / 94
 Visualizing Gridded Data

Code for Plotting Mean and Variance of Temperature at

500mb
1 time = ncFid . variables [ ’ time ’ ][:]
2 lev = ncFid . variables [ ’ lev ’ ][:]
3 lat = ncFid . variables [ ’ lat ’ ][:]
4 lon = ncFid . variables [ ’ lon ’ ][:]
5 T = ncFid . variables [ ’T ’ ][:]
6

7 level500 = 29 # level of interest

8 T500 = T [: , level500 ,: ,:] # time , lat , lon
9 T500mean = np . mean ( T500 ,0)
10 T500var = np . var ( T500 ,0)
11
12 bmContourPlot ( T500mean , lat , lon , ’ fig_TempMean ’ ,
13 ’ Spatial Temperature Mean ’)
14 bmContourPlot ( T500var , lat , lon , ’ fig_TempVariance ’ ,
15 ’ Spatial Temperature Variance ’)

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 88 / 94
 Visualizing Gridded Data

Plot of the Mean of Temperature

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 89 / 94
 Visualizing Gridded Data

Plot of the Variance of Temperature

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 90 / 94
 Visualizing Gridded Data

Slicing the Data

Assume that we want to plot the data in prescibed latitude and longitude
ranges.
1 # !/ usr / bin / env python
2
3 import numpy as np
4
5 def sliceLatLon ( lat , lon , ( minLat , maxLat ) , \
6 ( minLon , maxLon )):
7 indexLat = np . nonzero (( lat [:]>= minLat ) &
8 ( lat [:]<= maxLat ))[0]
9 indexLon = np . nonzero (( lon [:]>= minLon ) &
10 ( lon [:]<= maxLon ))[0]
11 return indexLat , indexLon

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 91 / 94
 Visualizing Gridded Data

Plot of the Mean of Temperature (Slice)

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 92 / 94
 Visualizing Gridded Data

Plot of the Variance of Temperature (Slice)

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 93 / 94
 Visualizing Gridded Data

References I

Hans Petter Langtangen.

A Primer on Scientific Programming with Python.
Springer, 2009.
Johnny Wei-Bing Lin.
A Hands-On Introduction to Using Python in the Atmospheric and
Oceanic Sciences.
http://www.johnny-lin.com/pyintro, 2012.
Drew McCormack.
Scientific Scripting with Python.
2009.
Sandro Tosi.
Matplotlib for Python Developers.
2009.

J. Kouatchou and H. Oloso (SSSO) Maplotlib and netCDF4 March 25, 2013 94 / 94