FDS Lab Manual
FDS Lab Manual
NAME : …………………………………………........................
REG. NO : …………………………………………........................
YEAR/SEM : …………………………………………........................
BRANCH : …………………………………………........................
PGP COLLEGE OF ENGINEERING AND TECHNOLOGY
NAMAKKAL – 637 207
CERTIFICATE
S. PAGE
DATE EXPERIMENTS MARK SIGN
NO NO.
1
Install Anaconda
2
3
4
This finishes the installation of Anaconda distribution, now let’s see how
to create an environment and install Jupyter Notebook.
5
Create Anaconda Environment from Navigator
6
select + Create icon at the bottom of the screen to create an Anaconda
environment.
7
Install and Run Jupyter Notebook
Once you create the anaconda environment, go back to the Home page on
Anaconda Navigator and install Jupyter Notebook from an application on
the right panel.
It will take a few seconds to install Jupyter to your environment, once the
install completes, you can open Jupyter from the same screen or by
accessing Anaconda Navigator -> Environments -> your
environment (mine pandas-tutorial) -> select Open With Jupyter
Notebook.
8
Now select New -> PythonX and enter the below lines and select Run. On
Jupyter, each cell is a statement, so you can run each cell independently
when there are no dependencies on previous cells.
9
10
RESULT:
Thus Jupyter Notebook environment has been successfully installed with all the
necessary packages using Anaconda distribution.
11
Ex. No 2 Working with Numpy arrays
Aim
import numpy as np
arr = np.array([1, 2, 3, 4, 5])
print(arr)
print(type(arr))
Output
[1 2 3 4 5]
<class 'numpy.ndarray'>
(ii) Dimensions in Arrays
0-D Arrays
Program
import numpy as np
arr = np.array(42)
print(arr)
Output
42
12
1-D Arrays
Program
import numpy as np
arr = np.array([1, 2, 3, 4, 5])
print(arr)
Output
[1 2 3 4 5]
2-D Arrays
Program
import numpy as np
arr = np.array([[1, 2, 3], [4, 5, 6]])
print(arr)
Output
[[1 2 3]
[4 5 6]]
3-D arrays
Program
import numpy as np
arr = np.array([[[1, 2, 3], [4, 5, 6]], [[1, 2, 3], [4, 5, 6]]])
print(arr)
Output
[[[1 2 3]
[4 5 6]]
[[1 2 3]
[4 5 6]]]
(iii) Check Number of Dimensions?
Program
import numpy as np
a = np.array(42)
13
b = np.array([1, 2, 3, 4, 5])
c = np.array([[1, 2, 3], [4, 5, 6]])
d = np.array([[[1, 2, 3], [4, 5, 6]], [[1, 2, 3], [4, 5, 6]]])
print(a.ndim)
print(b.ndim)
print(c.ndim)
print(d.ndim)
Output
0
1
2
3
import numpy as np
arr = np.array([1, 2, 3, 4])
print(arr[0])
Output
Program
import numpy as np
arr = np.array([1, 2, 3, 4])
print(arr[2] + arr[3])
Output
14
import numpy as np
arr = np.array([1, 2, 3, 4, 5, 6, 7])
print(arr[1:5])
Output
[2 3 4 5]
import numpy as np
newarr = arr.reshape(4, 3)
print(newarr)
Output
[[ 1 2 3]
[ 4 5 6]
[ 7 8 9]
[10 11 12]]
import numpy as np
arr = np.array([1, 2, 3])
for x in arr:
print(x)
15
Output
3
(ix) Joining NumPy Arrays
Program
import numpy as np
arr1 = np.array([1, 2, 3])
arr2 = np.array([4, 5, 6])
arr = np.concatenate((arr1, arr2))
print(arr)
Output
[1 2 3 4 5 6]
import numpy as np
newarr = np.array_split(arr, 3)
print(newarr)
Output
import numpy as np
arr = np.array([1, 2, 3, 4, 5, 4, 4])
x = np.where(arr == 4)
print(x)
16
Output
(array([3, 5, 6]),)
import numpy as np
arr = np.array([3, 2, 0, 1])
print(np.sort(arr))
Output
[0 1 2 3]
RESULT:
Thus Array object has been explored using Numpy module in Python programming
successfully.
17
Exp. No. 3. Working with Pandas data frames
Aim:
Algorithm:
Step 4: Load data into a DataFrame object otherwise Load Files(excel/csv) into a
DataFrame
Step 5: Display the rows and describe the data set using built in method.
import pandas as pd
data = {
df = pd.DataFrame(data)
print(df)
Output
calories duration
0 420 50
18
1 380 40
2 390 45
print(df.loc[0])
Output
calories 420
duration 50
Program
print(df.loc[[0, 1]])
Output
calories duration
0 420 50
1 380 40
import pandas as pd
data = {
19
"duration": [50, 40, 45]
print(df)
Output
calories duration
day1 420 50
day2 380 40
day3 390 45
print(df.loc["day2"])
Output
calories 380
duration 40
import pandas as pd
df = pd.read_csv('data.csv')
print(df)
Output
20
2 60 103 135 340.0
import pandas as pd
print(pd.options.display.max_rows)
In my system the number is 60, which means that if the DataFrame contains more
than 60 rows, the print(df) statement will return only the headers and the first and
last 5 rows.
import pandas as pd
pd.options.display.max_rows = 9999
df = pd.read_csv('data.csv')
print(df)
import pandas as pd
df = pd.read_csv('data.csv')
print(df.head(4))
21
Output
print(df.tail())
print(df.info())
Output
<class 'pandas.core.frame.DataFrame'>
None
RESULT:
Thus DataFrame object using Pandas module in Python Programming has been
successfully explored
22
Exp. No. 4. Reading data from text files, Excel and the web and exploring
various commands for doing descriptive analytics on the Iris
data set.
Aim:
Algorithm
Step 3: Load Files(excel/csv/ text) into a DataFrame from Iris data set
Step 4: Display the rows and describe the data set using built in methods
Step 6: Visualize the data set using histogram with distplot, heatmaps
Program
import pandas as pd
df = pd.read_csv("Iris.csv")
df.head()
Output:
23
Getting Information about the Dataset
df.shape
Output:
(150, 6)
df.info()
Output
df.describe()
df.isnull().sum()
24
Checking Duplicates
data
Output
df.value_counts("Species")
Data Visualization
# importing packages
sns.countplot(x='Species', data=df, )
plt.show()
25
Comparing Sepal Length and Sepal Width
# importing packages
sns.scatterplot(x='SepalLengthCm', y='SepalWidthCm',
hue='Species', data=df, )
plt.show()
26
# importing packages
import seaborn as sns
import matplotlib.pyplot as plt
sns.pairplot(df.drop(['Id'], axis = 1),
hue='Species', height=2)
Output:
27
Histograms
Program
# importing packages
import seaborn as sns
import matplotlib.pyplot as plt
fig, axes = plt.subplots(2, 2, figsize=(10,10))
axes[0,0].set_title("Sepal Length")
axes[0,0].hist(df['SepalLengthCm'], bins=7)
axes[0,1].set_title("Sepal Width")
axes[0,1].hist(df['SepalWidthCm'], bins=5)
axes[1,0].set_title("Petal Length")
axes[1,0].hist(df['PetalLengthCm'], bins=6)
axes[1,1].set_title("Petal Width")
axes[1,1].hist(df['PetalWidthCm'], bins=6)
Output:
28
Histograms with Distplot Plot
Program
# importing packages
import seaborn as sns
import matplotlib.pyplot as plt
Output:
29
30
Handling Correlation
data.corr(method='pearson')
Output:
Heatmaps
Program
# importing packages
import seaborn as sns
import matplotlib.pyplot as plt
sns.heatmap(df.corr(method='pearson').drop(
['Id'], axis=1).drop(['Id'], axis=0),
annot = True);
plt.show()
Output:
31
Box Plots
Program
# importing packages
import seaborn as sns
import matplotlib.pyplot as plt
def graph(y):
sns.boxplot(x="Species", y=y, data=df)
plt.figure(figsize=(10,10))
# Adding the subplot at the specified
# grid position
plt.subplot(221)
graph('SepalLengthCm')
plt.subplot(222)
graph('SepalWidthCm')
plt.subplot(223)
graph('PetalLengthCm')
plt.subplot(224)
graph('PetalWidthCm')
plt.show()
Output:
32
Program
# importing packages
import seaborn as sns
import matplotlib.pyplot as plt
# Load the dataset
df = pd.read_csv('Iris.csv')
sns.boxplot(x='SepalWidthCm', data=df)
Output:
33
Removing Outliers
Program
# Importing
import sklearn
from sklearn.datasets import load_boston
import pandas as pd
import seaborn as sns
# Load the dataset
df = pd.read_csv('Iris.csv')
# IQR
Q1 = np.percentile(df['SepalWidthCm'], 25,
interpolation = 'midpoint')
Q3 = np.percentile(df['SepalWidthCm'], 75,
interpolation = 'midpoint')
IQR = Q3 - Q1
print("Old Shape: ", df.shape)
# Upper bound
upper = np.where(df['SepalWidthCm'] >= (Q3+1.5*IQR))
# Lower bound
lower = np.where(df['SepalWidthCm'] <= (Q1-1.5*IQR))
34
# Removing the Outliers
df.drop(upper[0], inplace = True)
df.drop(lower[0], inplace = True)
print("New Shape: ", df.shape)
sns.boxplot(x='SepalWidthCm', data=df)
Output:
RESULT:
Thus Iris dataset has been explored and descriptively analysed using Python
programming
35
Exp. No. 5. Use the diabetes data set from UCI and Pima Indians Diabetes
data set for performing the following:
Aim:
Step 3: Load Files (excel/csv/ text) into a Data Frame from UCI and Pima Indians
Diabetes data set
Step 4: Display the rows and describe the data set using built in methods
Step 6: Visualize the data set using histogram with distplot, heatmaps
Step 7: Check Missing Values, Duplicates and remove outliers using built in method
import pandas as pd
%matplotlib inline
36
count = df['Glucose'].value_counts()
display(count)
df.head()
df.describe()
df.mean()
df.mode()
37
df.var()
df.std()
df.skew()
Pregnancies 0.901674
Glucose 0.173754
BloodPressure -1.843608
SkinThickness 0.109372
Insulin 2.272251
BMI -0.428982
DiabetesPedigreeFunction 1.919911
Age 1.129597
Outcome 0.635017
dtype: float64
df.kurtosis()
Pregnancies 0.159220
Glucose 0.640780
BloodPressure 5.180157
SkinThickness -0.520072
38
Insulin 7.214260
BMI 3.290443
DiabetesPedigreeFunction 5.594954
Age 0.643159
Outcome -1.600930
dtype: float64
corr = df.corr()
sns.heatmap(corr,
xticklabels=corr.columns,
yticklabels=corr.columns)
sns.countplot('Outcome', data=df)
plt.show()
39
# Computing the %age of diabetic and non-diabetic in the sample
Out0=len([df.Outcome==1])
Out1=len([df.Outcome==0])
Total=Out0+Out1
PC_of_1 = Out1*100/Total
PC_of_0 = Out0*100/Total
PC_of_1, PC_of_0
(50.0, 50.0)
plt.yticks(rotation = 0)
plt.xticks(rotation = 90)
plt.title('Correlation Heatmap')
plt.show()
40
RESULT:
Thus various exploratory data analysis has been performed on Pima Indians
Diabetes dataset using Python Programming successfully.
41
Exp. No. 6. Apply and explore various plotting functions on UCI data sets.
a. Normal curves
b. Density and contour plots
c. Correlation and scatter plots
d. Histograms
e. Three dimensional plotting
Aim:
To apply various plotting functions on UCI data set using Python Programming
Algorithm
Step 6: Visualize the data set using Explore various plotting functions on UCI data
sets for the following
a. Normal curves
d. Histograms
e. Three-dimensional plotting
import pandas as pd
import numpy as np
df=pd.read_csv("C:/Users/praveen/Downloads/dataset_diabetes/diabetic_data.cs
v")
42
df.head()
mean =df['time_in_hospital'].mean()
std =df['time_in_hospital'].std()
plt.show()
Output
df.time_in_hospital.plot.density(color='green')
plt.show()
43
df.num_lab_procedures.plot.density(color='green')
plt.show()
df.num_medications.plot.density(color='green')
44
Output
Program
# using plot.kde()
df.number_emergency.plot.kde(color='green')
plt.show()
45
Output
Program
mean =df['time_in_hospital'].mean()
std =df['time_in_hospital'].std()
x = np.linspace(0, mean)
y = np.linspace(0, std)
X, Y = np.meshgrid(x, y)
Z = func(X, Y)
plt.contour(X, Y, Z, cmap='gist_rainbow_r');
46
Output
mp.figure(figsize=(20,10))
47
d. Histograms
Program
df.hist(figsize=(12,12),layout=(5,3))
Output
sb.distplot(a=df.num_lab_procedures, kde=False)
plt.show()
48
Output
fig = plt.figure()
ax = plt.axes(projection = '3d')
x = df['number_emergency']
y = df['number_inpatient']
z = df['number_outpatient']
ax.plot3D(x, y, z, 'green')
plt.show()
49
Output
RESULT:
Thus apply various plotting functions on UCI data set using Python Programming
50
Exp. No. 7. Visualizing Geographic Data with Basemap
Aim:
Step 4: Display the Base map using built in method like basemap along with latitude
Step 5: Display the Coastal lines meters and Country boundaries using built in
methods
Step 6: Fill the Coastal lines meters and Country boundaries with suitable colours
%matplotlib inline
import numpy as np
plt.figure(figsize=(8, 8))
m.bluemarble(scale=0.5);
51
Output
Program
m = Basemap(projection='lcc', resolution=None,
width=8E6, height=8E6,
lat_0=45, lon_0=-100,)
m.etopo(scale=0.5, alpha=0.5)
x, y = m(-122.3, 47.6)
52
Output
-
Create a global map with a Coastlines
Program
m = Basemap()
m.drawcoastlines()
plt.title("Coastlines", fontsize=20)
plt.show()
53
Program
m = Basemap()
plt.title("Coastlines", fontsize=20)
plt.show()
54
Create a global map with a Country boundaries
Program
m = Basemap()
m.drawcountries()
x, y = m(-122.3, 47.6)
plt.show()
Output
m = Basemap(projection='merc',llcrnrlat=-80,urcrnrlat=80,llcrnrlon=-
180,urcrnrlon=180)
m.drawcoastlines()
m.fillcontinents(color='tan',lake_color='lightblue')
55
m.drawcountries(linewidth=1, linestyle='solid', color='k' )
m.drawmapboundary(fill_color='lightblue')
Output
56
Output
57
Output
58
Output
RESULT
Thus Geographic Data has been visualized using BaseMap module in Python
Programming successfully.
59