How to Drop Unnamed Column in Pandas DataFrame

Pandas is an open-source data analysis and manipulation tool widely used for handling structured data. In some cases, when importing data from CSV files, unnamed columns (often labeled as Unnamed: X) may appear. These columns usually contain unnecessary data, such as row indices from previous exports.

This article explores various methods to remove unnamed columns from a Pandas DataFrame efficiently.

For the link to the CSV file Used in the Code, click student_data.csv file

Using the drop() function

drop() function removes specified columns or rows by providing label names. It can be used to drop unnamed columns from a DataFrame.

Syntax:

df.drop(df.columns[df.columns.str.contains('unnamed', case=False)], axis=1, inplace=True)

Parameters:

• df.columns.str.contains('unnamed', case=False): Identifies columns containing the text "unnamed" (case-insensitive).
• axis=1: Specifies that the operation should be applied to columns (instead of rows).
• inplace=True: Modifies the DataFrame in place without creating a copy.

Example:

import pandas as pd

# Read the CSV file
df = pd.read_csv(
    'https://media.geeksforgeeks.org/wp-content/uploads/20240208164753/student_data3.csv')

# Removing unnamed columns using drop function
df.drop(df.columns[df.columns.str.contains(
    'unnamed', case=False)], axis=1, inplace=True)

print(df)

Output:

Actual dataframe:
      name         subject  Unnamed: 2   fees  fine
0     Arun           Maths           9   9000   400
1   Aniket  Social Science          10  12000   600
2   Ishita         English          11  15000     0
3  Pranjal         Science          12  18000  1000
4  Vinayak        Computer          12  18000   500
Dataframe after removing unnamed columns:
      name         subject   fees  fine
0     Arun           Maths   9000   400
1   Aniket  Social Science  12000   600
2   Ishita         English  15000     0
3  Pranjal         Science  18000  1000
4  Vinayak        Computer  18000   500

Explanation: drop() identifies columns containing "Unnamed" using df.columns.str.contains('unnamed', case=False). The axis=1 parameter ensures column-wise deletion, while inplace=True modifies the DataFrame directly without creating a copy.

Using the loc[] function

loc[] function is used to access groups of rows and columns by labels. It can be used to filter out unnamed columns.

Syntax:

df = df.loc[:, ~df.columns.str.contains('^Unnamed')]

Parameters:

• df.columns.str.contains('^Unnamed'): Finds all columns whose name starts with "Unnamed".
• ~: Negates the condition, selecting only the columns that do not match.
• loc[:, ...]: Selects all rows (:) and only the columns that do not contain "Unnamed".

Example:

import pandas as pd

# Read the CSV file
df = pd.read_csv(
    'https://media.geeksforgeeks.org/wp-content/uploads/20240208164753/student_data3.csv')


# Removing unnamed columns using drop function
df = df.loc[:, ~df.columns.str.contains('^Unnamed')]


print(df)

Output:

Actual dataframe:
      name         subject  Unnamed: 2   fees  fine
0     Arun           Maths           9   9000   400
1   Aniket  Social Science          10  12000   600
2   Ishita         English          11  15000     0
3  Pranjal         Science          12  18000  1000
4  Vinayak        Computer          12  18000   500
Dataframe after removing unnamed columns:
      name         subject   fees  fine
0     Arun           Maths   9000   400
1   Aniket  Social Science  12000   600
2   Ishita         English  15000     0
3  Pranjal         Science  18000  1000
4  Vinayak        Computer  18000   500

Explanation: This method filters out unnamed columns by selecting only those that do not match the pattern '^Unnamed'. The ~ operator negates the condition, ensuring that columns with "Unnamed" are excluded while keeping all rows intact.

Handling unnamed coloums while importing data

If unnamed columns appear due to exporting data with an index, we can specify the index_col=0 parameter while reading the CSV file. This method prevents Pandas from reading the index column as a separate unnamed column.

Syntax:

df=pd.read_csv(csv_file ,index_col=0)

Parameters:

• csv_file: The path or URL of the CSV file.
• index_col=0: Specifies that the first column should be treated as the index, preventing Pandas from interpreting it as an unnamed column.

Example:

import pandas as pd

df1 = pd.DataFrame({'name': ['Arun', 'Aniket', 'Ishits', 'Pranjal', 'Vinayak'],
                    'subject': ['Maths', 'Social Science', 'English', 'Science', 'Computer'],
                    'fees': [9000, 12000, 15000, 18000, 18000]})

# Store the data frame in a CSV file
df1.to_csv('student_data.csv')

# Read the CSV file
df = pd.read_csv('student_data.csv')

# Read the CSV file removing unnamed columns
df = pd.read_csv('student_data.csv', index_col=0)


print(df)

Output:

Actual dataframe:
   Unnamed: 0     name         subject   fees
0           0     Arun           Maths   9000
1           1   Aniket  Social Science  12000
2           2   Ishits         English  15000
3           3  Pranjal         Science  18000
4           4  Vinayak        Computer  18000
Dataframe after removing unnamed columns:
      name         subject   fees
0     Arun           Maths   9000
1   Aniket  Social Science  12000
2   Ishits         English  15000
3  Pranjal         Science  18000
4  Vinayak        Computer  18000

Explanation: If unnamed columns appear due to index inclusion during CSV exports, specifying index_col=0 in pd.read_csv() prevents Pandas from treating the first column as a separate unnamed column. This ensures a cleaner DataFrame without unnecessary columns.