Remove Duplicate Values from a Pandas DataFrame in Python



To remove duplicate values from a Pandas DataFrame, use the drop_duplicates() method. At first, create a DataFrame with 3 columns −

dataFrame = pd.DataFrame({'Car': ['BMW', 'Mercedes', 'Lamborghini', 'BMW', 'Mercedes', 'Porsche'],'Place': ['Delhi', 'Hyderabad', 'Chandigarh', 'Delhi', 'Hyderabad', 'Mumbai'],'UnitsSold': [95, 70, 80, 95, 70, 90]})

Remove duplicate values −

dataFrame = dataFrame.drop_duplicates()

Example

Following is the complete code −

Open Compiler
import pandas as pd # Create DataFrame dataFrame = pd.DataFrame({'Car': ['BMW', 'Mercedes', 'Lamborghini', 'BMW', 'Mercedes', 'Porsche'],'Place': ['Delhi', 'Hyderabad', 'Chandigarh', 'Delhi', 'Hyderabad', 'Mumbai'], 'UnitsSold': [95, 70, 80, 95, 70, 90]}) print"Dataframe...\n", dataFrame # counting frequency of column Car count = dataFrame['Car'].value_counts() print"\nCount in column Car" print(count) # removing duplicates dataFrame = dataFrame.drop_duplicates() print"\nUpdated DataFrame after removing duplicates...\n",dataFrame # counting frequency of column Car after removing duplicates count = dataFrame['Car'].value_counts() print"\nCount in column Car" print(count)

Output

This will produce the following output −

Dataframe...
           Car        Place   UnitsSold
0          BMW        Delhi         95
1     Mercedes    Hyderabad         70
2  Lamborghini   Chandigarh         80
3          BMW        Delhi         95
4     Mercedes    Hyderabad         70
5      Porsche       Mumbai         90

Count in column Car
BMW            2
Mercedes       2
Porsche        1
Lamborghini    1
Name: Car, dtype: int64

Updated DataFrame after removing duplicates...
           Car         Place   UnitsSold
0          BMW        Delhi         95
1     Mercedes    Hyderabad         70
2  Lamborghini   Chandigarh         80
5      Porsche       Mumbai         90

Count in column Car
BMW           1
Porsche       1
Lamborghini   1
Mercedes      1
Name: Car, dtype: int64
Updated on: 2021-09-16T07:28:05+05:30

718 Views

Kickstart Your Career

Get certified by completing the course

Get Started
Advertisements