Modules (sometimes called packages or libraries) help group together related
sets of tools in Python. In this exercise, we will examine two modules that are
frequently used by Data Scientists:
1. statsmodels: used in machine learning; usually aliased as sm
2. seaborn: a visualization library; usually aliased as sns
Pandas
?What can pandas do for you
Loading tabular data from different sources
Search for particular rows or columns
Calculate aggregate statistics
Combining data from multiple sources
CSV files:
Import pandas as pd
Df = pd.read_csv('random.csv')
Print(df)
Methods:
Print(Df.head()) (method to display the first five lines in data frame)
Df.info() (disply information about data frame)
Selecting columns ways:
1. selecting with brackets and string: strings with spaces or special characters
Dataframe name['columnname']
2. selecting with a dot: if the name contains only letters or numbers
Dataframe name.columnname
Logical statements:
Question == solution (test if the two nu,ber are the same
>, >=, <, <=, !=
Booleans: true or false
In data frame we can compare more than one variable with one variable
Ex: credit_records[credit_records.price > 20.00]
matplotlib
Creating plot line:
From matplotlib import pyplot as plt;
Plt.plot( Xvalues, Yvalues )
Plt.show() to display the plot
Adding labels and legends:
Axes and title labels:
Plt.xlabel(" ")
Plt.ylabel(" ")
Plt.title(" ")
Plt.xticks() to change the numbers on axis
Plt.yticks()
Legends:
Add keyword argument label:
Plt.plot(x, y, label = " ")
Plt.legend() to show legends
Arbitrary text: (floating text)
Plt.text(x, y, ' ')
Modifying text:
Change font size:
Plt.title(' ', fontsize=20)
Change font color:
Plt.legend(color='green')
Adding style:
Changing line color:
Plt.plot(x, y, color=" ")
Changing line width:
Linewidth=1 (from 1 to 7)
Changing line style:
Linestyle='-' or '—' or '-.' Or ':'
Adding markers:
Marker='x' or 's' or 'o' or 'd' or '*' or 'h'
Setting a style:
Plt.style.use(' ')
'fivethirtyeight' - Based on the color scheme of the popular website
'grayscale' - Great for when you don't have a color printer!
'seaborn' - Based on another Python visualization library
'classic' - The default color scheme for Matplotlib
Making a Scatter Plot:
Plt.scatter(x,y)
Plt.show()
Changing marker transparency:
Alpha= 0 to 1
logarithmic scale:
plt.xscale('log');
Making a bar chart:
Plt.bar(x, y)
Plt.barh(x,y) to make horizontal bar chart
Adding error bars:
Yerr=dataframename.error
Stacked bar carts:
Display two diff. sets of bars
Bottom = df.column
Making a histogram:
Plt.hist(x,y)
Changing bins:
Bins=num of bins
Changing range:
Range = (xmin, xmax)
Normalizing:
Reduces the height of each bar by a const. factor so that the area of each bar adda
to one.
Density=True
Dictionary:
varName = {key: value}
name[key] >> to print value
name.keys() >> print all keys
key in name >> true/ false
del(name[key]);
differences between list and dictionariy:
list is indexed by range of numbers, order matters
dictionary is indexed by unique keys, order doesn't matter
PANDAS:
Pandas is an open source library, providing high-performance, easy-to-use
data structures and data analysis tools for Python.
The DataFrame is one of Pandas' most important data structures. way to store
tabular data where you can label the rows and the columns. One way to build a
DataFrame is from a dictionary.
Import pandas as pd
To turn dictionary into dataframe
DataFrameName= pd.dataframe(dictionaryName);
dataframeName.index = …;
CSV files:
Name= pd.read_csv('', index_col = 0);
Index and select data:
Square brackets: selecting columns: dataframeName['colName'],
dataframeName[['colName']]
Selecting rows: dataframeName['1:4'],
Advanced methods:
1- loc: select parts of your data based on labels: loc['dataname'] or [['name']]
Square brackets
● Column access
● Row access: only through slicing
● loc (label-based)
● Row access
● Column access
● Row & Column access brics[["country", "capital"]] brics[1:4] brics.loc[["RU", "IN", "CH"]]
brics.loc[:, ["country", "capital"]] brics.loc[["RU", "IN", "CH"], ["country", "capital"]]
2- iloc: position-based
Like loc but instead of col names you put numbers from 0 to …
Boolean operators: and, or , not
True and true >> true
Numpy operators:
Np.Logical_and(), np.logical_or(), np.logical_not()
Conditional statements:
If, elif, else
If z % 2 == 0 :
Print('z is even')
Else :
Print('z is odd')
Elif ( it is like else if)
Loops:
1- While loop = repeated if statement
Syntax:
While condition :
expression
2- For loop:
Syntax:
For var in seq :
Expression
Ex:
Fam = [1, 2, 4, 5]
For height in fam :
Print(height)
Using enumerate:
For index, height in enumerate (fam) :
Print( height + index)
For c in "family" :
The loop will run for a number of times equal to each char in the string
Loop data structure:
1- Dictionary: use a method
For key, value in dicName .items() :
Print (key + value)
2- Numpy Arrays: 2D array: use a function
For val in np.nditer(npArr) :
Print(val)
3- pandas Data Frame: use a method
For lab, row in dataframeName.iterrows() :
Print(lab)
Print(row)
Using apply func to apply a function for every row in data:
dataframeName["colName'].apply(function)
str(integer) >> convert a number to a string
Random Number:
Random generators:
Np.random.rand()
Np. Random. Seed()
Np.random.randint(0,2) generate 0 or 1
Arr.append() >> add item to array
For x in range(number) : >> make for loop run for a number of times
Np.mean()
Python Data Science Toolbox (Part 1)
1-Defining a function:
Def funcName(): (function header)
Function body
Calling the function>> funcName()
2-Function Parameters:
Def funcName(value):
……
funcName(---)
3-return values from functions:
Return ---
4-Docstrings:
-describe what your function does, placed in the immediate line after the function header
"""….."""
5- Multiple Function Param.
6- return multiple values (tuples)
Tuples is like a list and immutable and constructed using ()
Even_nums = (2, 4, 6)
A, b, c = even_nums
Scope and user-defined functions:
Scopes are three: Global, local, built-in (like print())
Python first looks in local, then enclosing functions if there any, then global then built-in
scope
Using Global key word alter the value of a variable defined in the global scope.
Import builtins to access the python built-in scope????
Nested Functions:
Def outer(..):
Code
Def inner(…):
code
outer function is called enclosing function
Returning Functions
Closure: anything defined locally in the enclosing scope is available to the inner
function even when the outer function has finished execution.
Using nonlocal to alter the value of a variable defined in the enclosing scope.
Default and flexible arguments
Add default argument: funcName(arg = default value)
Functions with variable-length arguments: (*args)
Functions with variable-length arguments: (**Kwargs)
Lambda Functions:
Quicker way to write functions:
Varname = lambda x, y: x ** y
Varname(1,2)
Anonymous functions:
Map(func, seq)
We can write this function into map using lambda and without defining it.
To convert a variable into a list: list(variable)
Errors and exceptions:
Exceptions - caught during execution
Catch exceptions with try-except clause
Runs the code following try
If there’s an exception, run the code following except
Try:
Except Type Error:
Print()
If x < 0:
Raise ValueError('')
Try:
….
Except:
Print()
