Python Pandas – Create a subset and display only the last entry from duplicate values

To create a subset and display only the last entry from duplicate values, use the “keep” parameter with the ‘last” value in drop_duplicates() method. The drop_duplicates() method removed duplicates.

Let us first create a DataFrame with 3 columns −

dataFrame = pd.DataFrame({'Car': ['BMW', 'Mercedes', 'Lamborghini', 'BMW', 'Mercedes', 'Porsche'],'Place': ['Delhi', 'Hyderabad', 'Chandigarh', 'Delhi', 'Hyderabad', 'Mumbai'],'UnitsSold': [85, 70, 80, 95, 55, 90]})

Removing duplicates and displaying last entry. Using keep parameter, we have set "last". Duplicate rows except the last entry will get deleted. We have considered a subset using the “subset” parameter −

dataFrame2 = dataFrame.drop_duplicates(subset = ['Car', 'Place'], keep ='last').reset_index(drop = True)

Example

Following is the code −

import pandas as pd

# Create DataFrame
dataFrame = pd.DataFrame({'Car': ['BMW', 'Mercedes', 'Lamborghini', 'BMW', 'Mercedes', 'Porsche'],'Place': ['Delhi', 'Hyderabad', 'Chandigarh', 'Delhi', 'Hyderabad', 'Mumbai'],'UnitsSold': [85, 70, 80, 95, 55, 90]})

print"Dataframe...\n", dataFrame

# removing duplicates and displaying last entry
# using keep parameter, we have set "last"
# duplicate rows except the last entry will get deleted
# considered a subset using the subset parameter
dataFrame2 = dataFrame.drop_duplicates(subset = ['Car', 'Place'], keep ='last').reset_index(drop = True)
print"\nUpdated DataFrame after removing duplicates...\n",dataFrame2

Output

This will produce the following output −

Dataframe...
           Car       Place   UnitsSold
0          BMW       Delhi          85
1     Mercedes   Hyderabad          70
2  Lamborghini  Chandigarh          80
3          BMW       Delhi          95
4     Mercedes   Hyderabad          55
5      Porsche      Mumbai          90

Updated DataFrame after removing duplicates...
           Car       Place   UnitsSold
0  Lamborghini  Chandigarh          80
1          BMW       Delhi          95
2     Mercedes   Hyderabad          55
3      Porsche      Mumbai          90