Pandas plots

Pandas also provides us options to visualize the data. Here are some of the examples:

Syntax:

df.plot(X, y, marker, kind)

X = value on X axis

y = value on y axis

marker = shape in case of specific plots like a scatter plot

kind = type of plot

A scatter plot to visualize the trend of acceleration in different years.

1. df.plot(x = 'model_year', y = 'acceleration', marker = 'o',

kind = 'scatter');

2.
A bar plot to visualize mean acceleration in different years.

1. df.groupby('model_year').mean()[['acceleration']].plot(kind
= 'bar');

2.

A histogram to visualize the frequency distribution of cylinders

1. df['cylinders'].plot(kind = 'hist')

2.
A scatter plot to visualize the relationship between weight and mpg.

1. df.plot(x = 'weight', y = 'mpg', kind = 'scatter')

2.
A bar plot to visualize the sorted mean values of acceleration with respect to number of
cylinders.

1. df.groupby('cylinders').mean().sort_values('acceleration')
[['acceleration']].plot(kind = 'bar')

2.

Pandas Exercise

Problem Statement:
to find the rainfall dataset that is to be considered for the exercise.. This data contains
region(district) wise rainfall across India.

1. Import the data into Python environment as a Pandas DataFrame.

2. Check for missing values, if any and drop the corresponding rows.
3. Find the district that gets the highest annual rainfall.
4. Display the top 5 states that get the highest annual rainfall.
5. Drop the columns 'Jan-Feb', 'Mar-May', 'Jun-Sep', 'Oct-Dec'.
6. Display the state-wise mean rainfall for all the months using a pivot table.
7. Display the count of districts in each state.
8. For each state, display the district that gets the highest rainfall in May. Also display the
recorded rainfall.
Exercise
Problem Statement:
Given below is a DataFrame df:

1. import pandas as pd

2.

3. df = pd.DataFrame([[0.23,'f1'],[5.36,'f2']],

4. index = list('pq'),

5. columns = list('ab'))

6.

Do the following:

1. Change the column name from ' a' to ' A'.

2. Add a new column ' c' filled with random values.

3. Change the datatype of column ' A' values to complex.

4. Display rows whose any of the element matches with any element of the given list:

1. lst = ['f30','f50','f2','f0']

Merging
There are few more commonly used methods involved in Pandas as listed:

 Merging
 Reshaping
 Pivot Tables
 Grouping

Let us understand their importance by some illustrations starting with merging of datasets.
Given two datasets from a conducted experiment with each dataset having its own feature. Our
task is to form a single dataset combining all the features particular to each observation. To do so
we can take help of concat() function.

1. import pandas as pd

2.

3. data1 = pd.DataFrame([[15, 12, -3],

4. [33, 54, 21],

5. [10, 32, 22]],

6. columns = list('ABC'))

7. data2 = pd.DataFrame([[10, 1, 3],

8. [33, -54, 2],

9. [10, 0.32, 2]],

10. columns = list('DEF'))

11. print(data1)

12. # A B C

13. # 0 15 12 -3

14. # 1 33 54 21

15. # 2 10 32 22

16.

17. print(data2)

18. # D E F

19. # 0 10 1.00 3

20. # 1 33 -54.00 2

21. # 2 10 0.32 2
 22.

23. print(pd.concat( [data1,data2], axis = 1 ))

24. # A B C D E F

25. # 0 15 12 -3 10 1.00 3

26. # 1 33 54 21 33 -54.00 2

27. # 2 10 32 22 10 0.32 2

28.

29.

Suppose an individual took observation of 3 different features in two instances. Next, she wants
to combine all these samples to form a single dataset. We can achieve this task using the
same concat() function, but this time with the change of axis.

1. import numpy as np

2. import pandas as pd

3.

4. data1 = pd.DataFrame(np.random.randn(9).reshape(3,3),

5. columns = list('ABC'))

6. data2 = pd.DataFrame(np.arange(9).reshape(3,3),

7. columns = list('ABC'))

8. print(data1) # Random values

9. # A B C

10. # 0 1.957218 0.433266 1.214950

11. # 1 -0.143500 -0.092030 -0.823898

12. # 2 0.481486 -0.024111 -0.769195

13.

14. print(data2)

15. # A B C

16. # 0 0 1 2

17. # 1 3 4 5

18. # 2 6 7 8

19.

20. print(pd.concat( [data1,data2], axis = 0 ))

21. # A B C

22. # 0 1.957218 0.433266 1.214950

23. # 1 -0.143500 -0.092030 -0.823898

24. # 2 0.481486 -0.024111 -0.769195

25. # 0 0.000000 1.000000 2.000000

26. # 1 3.000000 4.000000 5.000000

27. # 2 6.000000 7.000000 8.000000

Pivot Tables
Reshaping a dataset is a perfect seed for quick text visualization. However, similar to reshaping we have
another term named as pivot tables which are more efficient in delivering a better visualization.

To understand pivot tables we take the same last dataframe except adding a new feature 'score'.

1. import pandas as pd

2.

3. df = pd.DataFrame([

4. ['IND', 'Gold', 'Game1', '9.9'],

 5. ['IND', 'Bronze', 'Game2', '8'],

6. ['USA', 'Silver', 'Game1', '9.5'],

7. ['USA', 'Gold', 'Game2', '8.6'],

8. ], columns = ['Country', 'Medal',

9. 'Game', 'Score'],

10. index = ['Year1', 'Year2','Year1', 'Year2'])

11.

12. print(df)

13. # Country Medal Game Score

14. # Year1 IND Gold Game1 9.9

15. # Year2 IND Bronze Game2 8

16. # Year1 USA Silver Game1 9.5

17. # Year2 USA Gold Game2 8.6

18.

Pivot tables come handy when we have to break down a large dataset (in terms of features) into fewer features
for quick visualization. For example, finding which medal is common to both IND and USA, listing game(s) in
which India won Silver, etc.

We implement pivot table on the given dataset:

1. # Listing all the features

2. print(df.pivot(index = 'Country', columns = 'Medal'))

3. # Game Score

4. # Medal Bronze Gold Silver Bronze Gold Silver

5. # Country

6. # IND Game2 Game1 None 8 9.9 None

 7. # USA None Game2 Game1 None 8.6 9.5

8.

9. # Listing only Score feature

10. print(df.pivot(index = 'Country', columns = 'Medal',

11. values = 'Score'))

12. # Medal Bronze Gold Silver

13. # Country

14. # IND 8 9.9 None

15. # USA None 8.6 9.5

16.

Pivot table limitations

The previous dataset doesn't include any redundant sample, for instance, let's say 2 people win gold as well as
silver medals in Year1 of Game1 from India. In that case, the above function named pivot() breaks due to
ambiguity in Game column. Therefore, to overcome this limitation, we have another function named
as pivot_table() which requires an argument named as aggfun through which we can clear the ambiguity. Let's
understand it through an example.

1. import numpy as np

2. import pandas as pd

3.

4. df = pd.DataFrame([

5. ['IND', 'Gold', 'Game1', '9.9'],

6. ['IND', 'Silver', 'Game1', '9.5'],

7. ['IND', 'Bronze', 'Game2', '8'],

8. ['USA', 'Bronze', 'Game1', '9.0'],

 9. ['USA', 'Silver', 'Game2', '8.6'],

10. ], columns = ['Country', 'Medal',

11. 'Game', 'Score'],

12. index = ['Year1', 'Year1', 'Year2','Year1',

'Year2'])

13.

14. print(df)

15. # Country Medal Game Score

16. # Year1 IND Gold Game1 9.9

17. # Year1 IND Silver Game1 9.5

18. # Year2 IND Bronze Game2 8

19. # Year1 USA Bronze Game1 9.0

20. # Year2 USA Silver Game2 8.6

21.

22. print(df.pivot_table(index = 'Country',

23. columns = 'Game',

24. values = 'Score',

25. aggfunc = np.max))

26. # Game Game1 Game2

27. # Country

28. # IND 9.9 8

29. # USA 9.0 8.6

Grouping
To understand grouping, a concept similar to databases, let us consider that we have been given a dataset for
the sales of laptop and desktop systems. The observations for a particular type can be repeated for the different
sales price. In this case, if we need to calculate the total sales of each category then we can group similar data
and apply a certain function.

To do so we create a sample dummy dataset and sum up the total sales particular to each category.

1. import pandas as pd

2.

3. df = pd.DataFrame([["Laptop", 1000],

4. ["Laptop", 2520],

5. ["Desktop", 3000],

6. ["Desktop", 400]], columns =

['Category','Sales'])

7.

8. print(df)

9. # Category Sales

10. # 0 Laptop 1000

11. # 1 Laptop 2520

12. # 2 Desktop 3000

13. # 3 Desktop 400

14.

15. print(df.groupby(['Category'], sort = False).sum())

16. # Sales

17. # Category

18. # Laptop 3520

19. # Desktop 3400

Exercise

Problem Statement:
Given a dataframe df which has three attributes defining: set_name: system names, spd_per_day:
Speed per day, speed: Network speed in MBps

1. sys = ['s1','s1','s1','s1',

2. 's2','s2','s2','s2']

3. net_day = ['d1','d1','d2','d2',

4. 'd1','d1','d2','d2']

5. spd = [1.3, 11.4, 5.6, 12.3,

6. 6.2, 1.1, 20.0, 8.8]

7. df = pd.DataFrame({'set_name':sys,

8. 'spd_per_day':net_day,

9. 'speed':spd})

10.

Do the following:

1. Construct a dataframe new_df where the given dataset is grouped based on each system
(s1 and s2) and speed per day (d1 and d2) with the median speed each day per system.
Also, provide a secondary name ' Median' for the speed attribute.
2. Sort the dataframe new_df in the ascending order of the median speed.

Hints/Answers:

1. Answer:

1. set_name spd_per_day speed

2. Median
 3. 0 s1 d1 6.35

4. 1 s1 d2 8.95

5. 2 s2 d1 3.65

6. 3 s2 d2 14.40

7.

2. Answer:

1. set_name spd_per_day speed

2. Median

3. 2 s2 d1 3.65

4. 0 s1 d1 6.35

5. 1 s1 d2 8.95

6. 3 s2 d2 14.40