Pandas,scipy,numpy cheatsheet
1 like•980 views
The document provides a comprehensive cheat sheet for using numpy, scipy, and pandas in data manipulation and analysis, covering various commands for selecting, reshaping, and summarizing data. It includes guidance on handling time series, grouping data, and exporting/importing data to/from CSV and Excel formats. Additionally, it discusses plotting techniques using matplotlib, along with specific examples for visualizing data from Quandl's database.
![1 www.quandl.com
NumPy / SciPy / Pandas Cheat Sheet
Select column.
Select row by label.
Return DataFrame index.
Delete given row or column. Pass axis=1 for columns.
Reindex df1 with index of df2.
Reset index, putting old index in column named index.
Change DataFrame index, new indecies set to NaN.
Show first n rows.
Show last n rows.
Sort index.
Sort columns.
Pivot DataFrame, using new conditions.
Transpose DataFrame.
Change lowest level of column labels into innermost row index.
Change innermost row index into lowest level of column labels.
NumPy / SciPy
arr = array([])
arr.shape
convolve(a,b)
arr.reshape()
sum(arr)
mean(arr)
std(arr)
dot(arr1,arr2)
vectorize()
Create a Series.
Create a Dataframe.
Create a Panel.
Pandas
Create Structures
s = Series (data, index)
df = DataFrame (data, index, columns)
p = Panel (data, items, major_axis, minor_axis)
df.stack()
df.unstack()
df.pivot(index,column,values)
df.T
DataFrame commands
df[col]
df.iloc[label]
df.index
df.drop()
df1 = df1.reindex_like(df1,df2)
df.reset_index()
df.reindex()
df.head(n)
df.tail(n)
df.sort()
df.sort(axis=1)
Create numpy array.
Shape of an array.
Linear convolution of two sequences.
Reshape array.
Sum all elements of array.
Compute mean of array.
Compute standard deviation of array.
Compute inner product of two arrays.
Turn a scalar function into one which
accepts & returns vectors.](https://image.slidesharecdn.com/pandasscipynumpycheatsheet-161214201553/85/Pandas-scipy-numpy-cheatsheet-1-320.jpg)
![2 www.quandl.com
Create a time series index.date_range(start, end, freq)
Pandas Time Series
Business Day
Calender day
Weekly
Monthly
Quarterly
Annual
Hourly
B
D
W
M
Q
A
H
Freq has many options including:
Any Structure with a datetime index
Split DataFrame by columns. Creates a GroupBy object (gb).
Apply function (single or list) to a GroupBy object.
Applies function and returns object with same index as one
being grouped.
Filter GroupBy object by a given function.
Return dict whose keys are the unique groups, and values
are axis labels belonging to each group.
Groupby
groupby()
gb.agg()
gb.transform()
gb.filter()
gb.groups
Save to CSV.
Read CSV into DataFrame.
Save to Excel.
Read exel into DataFrame.
I/O
df.to_csv(‘foo.csv’)
read_csv(‘foo.csv’)
to_excel(‘foo.xlsx’, sheet_name)
read_excel(‘foo.xlsx’,’sheet1’, index_col =
None, na_values = [‘NA’])
df.dropna()
df.count()
df.min()
df.max()
df.describe()
concat()
Drops rows where any data is missing.
Returns Series of row counts for every column.
Return minimum of every column.
Return maximum of every column.
Generate various summary statistics for every column.
Merge DataFrame or Series objects.
Apply function to every element in DataFrame.
Apply function along a given axis.
df.applymap()
df.apply()
Resample data with new frequency.ts.resample()](https://image.slidesharecdn.com/pandasscipynumpycheatsheet-161214201553/85/Pandas-scipy-numpy-cheatsheet-2-320.jpg)
![3 www.quandl.com
Select current axis.
Change axis color, none to remove.
Change axis position. Can change coordinate space.
Create legend. Set to ‘best’ for auto placement.
Save plot.
ax=gca()
ax.spines[].set_color()
ax.spines[].set_position()
Label the x-axis.
Label the y-axis.
Title the graph.
Set tick values for x-axis. First array for values, second
for labels.
Set tick values for y-axis. First array for values, second
for labels.
yticks([],[])
Plot data or plot a function against a range.
Create multiple plots; n- number of plots, x - number
horizontally displayed, y- number vertically displayed.
plot()
subplot(n,x,y)
xlabel()
ylabel()
title()
xticks([],[])
Plotting
Matplotlib is an extremely powerful module.
See www.matplotlib.org for complete documentation.
Quandl is a search engine for numerical data, allowing
easy access to financial, social, and demographic data
from hundreds of sources.
Quandl
The Quandl package enables Quandl API access from
within Python, which makes acquiring and manipulating
numerical data as quick and easy as possible.
In your first Quandl function call you should specifiy your
authtoken (found on Quandl’s website after signing up) to
avoid certain API call limits.
See www.quandl.com/help/packages/python for more.
savefig(‘foo.png’)
legend(loc=’ ‘)
Return data for nearest time interval.
Return data for specific time.
Return data between specific interval.
Convert Pandas DatetimeIndex to datetime.datetime object.
Conver a list of date-like objects (strings, epochs, etc.) to a
DatetimeIndex.
ts.ix[start:end]
ts[]
ts.between_time()
to_pydatetime()
to_datetime()](https://image.slidesharecdn.com/pandasscipynumpycheatsheet-161214201553/85/Pandas-scipy-numpy-cheatsheet-3-320.jpg)
![4 www.quandl.com
Plotting Example
import Quandl as q
import matplotlib.pyplot as plt
rural = q.get(‘WORLDBANK/USA_SP_RUR_TOTL_ZS’)
urban = q.get(‘WORLDBANK/USA_SP_URB_TOTL_IN_ZS’)
plt.subplot(2, 1, 1)
plt.plot(rural.index,rural)
plt.xticks(rural.index[0::3],[])
plt.title(‘American Population’)
plt.ylabel(‘% Rural’)
plt.subplot(2, 1, 2)
plt.plot(urban.index,urban)
plt.xlabel(‘year’)
plt.ylabel(‘% Urban’)
plt.show()
Add the following to any function call.
Download Quandl data for a certain Quandl code
as a Dataframe.
Search Quandl. Outputs first 4 results.
Upload a Pandas DataFrame (with a time series index) to
Quandl. Code must be all capital alphanumeric.
authtoken = ‘YOURTOKENHERE’
get(‘QUANDL/CODE’)
search(‘searchterm’)
push(data, code, name)](https://image.slidesharecdn.com/pandasscipynumpycheatsheet-161214201553/85/Pandas-scipy-numpy-cheatsheet-4-320.jpg)
Pandas,scipy,numpy cheatsheet
- 1. 1 www.quandl.com NumPy / SciPy / Pandas Cheat Sheet Select column. Select row by label. Return DataFrame index. Delete given row or column. Pass axis=1 for columns. Reindex df1 with index of df2. Reset index, putting old index in column named index. Change DataFrame index, new indecies set to NaN. Show first n rows. Show last n rows. Sort index. Sort columns. Pivot DataFrame, using new conditions. Transpose DataFrame. Change lowest level of column labels into innermost row index. Change innermost row index into lowest level of column labels. NumPy / SciPy arr = array([]) arr.shape convolve(a,b) arr.reshape() sum(arr) mean(arr) std(arr) dot(arr1,arr2) vectorize() Create a Series. Create a Dataframe. Create a Panel. Pandas Create Structures s = Series (data, index) df = DataFrame (data, index, columns) p = Panel (data, items, major_axis, minor_axis) df.stack() df.unstack() df.pivot(index,column,values) df.T DataFrame commands df[col] df.iloc[label] df.index df.drop() df1 = df1.reindex_like(df1,df2) df.reset_index() df.reindex() df.head(n) df.tail(n) df.sort() df.sort(axis=1) Create numpy array. Shape of an array. Linear convolution of two sequences. Reshape array. Sum all elements of array. Compute mean of array. Compute standard deviation of array. Compute inner product of two arrays. Turn a scalar function into one which accepts & returns vectors.
- 2. 2 www.quandl.com Create a time series index.date_range(start, end, freq) Pandas Time Series Business Day Calender day Weekly Monthly Quarterly Annual Hourly B D W M Q A H Freq has many options including: Any Structure with a datetime index Split DataFrame by columns. Creates a GroupBy object (gb). Apply function (single or list) to a GroupBy object. Applies function and returns object with same index as one being grouped. Filter GroupBy object by a given function. Return dict whose keys are the unique groups, and values are axis labels belonging to each group. Groupby groupby() gb.agg() gb.transform() gb.filter() gb.groups Save to CSV. Read CSV into DataFrame. Save to Excel. Read exel into DataFrame. I/O df.to_csv(‘foo.csv’) read_csv(‘foo.csv’) to_excel(‘foo.xlsx’, sheet_name) read_excel(‘foo.xlsx’,’sheet1’, index_col = None, na_values = [‘NA’]) df.dropna() df.count() df.min() df.max() df.describe() concat() Drops rows where any data is missing. Returns Series of row counts for every column. Return minimum of every column. Return maximum of every column. Generate various summary statistics for every column. Merge DataFrame or Series objects. Apply function to every element in DataFrame. Apply function along a given axis. df.applymap() df.apply() Resample data with new frequency.ts.resample()
- 3. 3 www.quandl.com Select current axis. Change axis color, none to remove. Change axis position. Can change coordinate space. Create legend. Set to ‘best’ for auto placement. Save plot. ax=gca() ax.spines[].set_color() ax.spines[].set_position() Label the x-axis. Label the y-axis. Title the graph. Set tick values for x-axis. First array for values, second for labels. Set tick values for y-axis. First array for values, second for labels. yticks([],[]) Plot data or plot a function against a range. Create multiple plots; n- number of plots, x - number horizontally displayed, y- number vertically displayed. plot() subplot(n,x,y) xlabel() ylabel() title() xticks([],[]) Plotting Matplotlib is an extremely powerful module. See www.matplotlib.org for complete documentation. Quandl is a search engine for numerical data, allowing easy access to financial, social, and demographic data from hundreds of sources. Quandl The Quandl package enables Quandl API access from within Python, which makes acquiring and manipulating numerical data as quick and easy as possible. In your first Quandl function call you should specifiy your authtoken (found on Quandl’s website after signing up) to avoid certain API call limits. See www.quandl.com/help/packages/python for more. savefig(‘foo.png’) legend(loc=’ ‘) Return data for nearest time interval. Return data for specific time. Return data between specific interval. Convert Pandas DatetimeIndex to datetime.datetime object. Conver a list of date-like objects (strings, epochs, etc.) to a DatetimeIndex. ts.ix[start:end] ts[] ts.between_time() to_pydatetime() to_datetime()
- 4. 4 www.quandl.com Plotting Example import Quandl as q import matplotlib.pyplot as plt rural = q.get(‘WORLDBANK/USA_SP_RUR_TOTL_ZS’) urban = q.get(‘WORLDBANK/USA_SP_URB_TOTL_IN_ZS’) plt.subplot(2, 1, 1) plt.plot(rural.index,rural) plt.xticks(rural.index[0::3],[]) plt.title(‘American Population’) plt.ylabel(‘% Rural’) plt.subplot(2, 1, 2) plt.plot(urban.index,urban) plt.xlabel(‘year’) plt.ylabel(‘% Urban’) plt.show() Add the following to any function call. Download Quandl data for a certain Quandl code as a Dataframe. Search Quandl. Outputs first 4 results. Upload a Pandas DataFrame (with a time series index) to Quandl. Code must be all capital alphanumeric. authtoken = ‘YOURTOKENHERE’ get(‘QUANDL/CODE’) search(‘searchterm’) push(data, code, name)