Drop Empty Columns in Pandas

Cleaning data is an essential step in data analysis. In this guide we will explore different ways to drop empty, null and zero-value columns in a Pandas DataFrame using Python. By the end you'll know how to efficiently clean your dataset using the dropna() and replace() methods.

Understanding dropna()

The dropna() function is a powerful method in Pandas that allows us to remove rows or columns containing missing values (NaN). Depending on the parameters used it can remove rows or columns where at least one value is missing or only those where all values are missing.

Syntax: DataFrameName.dropna(axis=0, how='any', inplace=False)

Parameters:

• axis: axis takes int or string value for rows/columns. Input can be 0 or 1 for Integer and ‘index’ or ‘columns’ for String.
• how: how takes string value of two kinds only (‘any’ or ‘all’). ‘any’ drops the row/column if ANY value is Null and ‘all’ drops only if ALL values are null.
• inplace: It is a boolean which makes the changes in the data frame itself if True.

Create a Sample DataFrame:

This is the sample data frame on which we will use to perform different operations.

import numpy as np
import pandas as pd

df = pd.DataFrame({'FirstName': ['Vipul', 'Ashish', 'Milan'],
                            "Gender": ["", "", ""],
                            "Age": [0, 0, 0]})
df['Department'] = np.nan

print(df)

Output:

Example 1: Remove All Null Value Columns

This method removes columns where all values are NaN. If a column is completely empty (contains only NaN values) it is unnecessary for analysis and can be removed using dropna(how='all', axis=1).

import numpy as np
import pandas as pd

df = pd.DataFrame({'FirstName': ['Vipul', 'Ashish', 'Milan'],
                            "Gender": ["", "", ""],
                            "Age": [0, 0, 0]})

df['Department'] = np.nan

display(df)

df.dropna(how='all', axis=1, inplace=True)

display(df)

Output:

Example 2: Replace Empty Strings with Null and Drop Null Columns

If a column contains empty strings we need to replace them with NaN before dropping the column. Empty strings are not automatically recognized as missing values in Pandas so converting them to NaN ensures they can be handled correctly. After conversion we use dropna(how='all', axis=1) to remove columns that are entirely empty.

import numpy as np
import pandas as pd

df = pd.DataFrame({'FirstName': ['Vipul', 'Ashish', 'Milan'],
                            "Gender": ["", "", ""],
                            "Age": [0, 0, 0]})

df['Department'] = np.nan
display(df)

nan_value = float("NaN")
df.replace("", nan_value, inplace=True)

df.dropna(how='all', axis=1, inplace=True)

display(df)

Output:

Example 3: Replace Zeros with Null and Drop Null Columns

If columns contain only zero values, we convert them to NaN before dropping them.

import numpy as np
import pandas as pd

df = pd.DataFrame({'FirstName': ['Vipul', 'Ashish', 'Milan'],
                            "Gender": ["", "", ""],
                            "Age": [0, 0, 0]})

df['Department'] = np.nan
display(df)

nan_value = float("NaN")
df.replace(0, nan_value, inplace=True)

df.dropna(how='all', axis=1, inplace=True)

display(df)

Output:

Example 4: Replace Both Zeros and Empty Strings with Null and Drop Null Columns

To clean a dataset fully we may need to replace both zeros and empty strings.

import numpy as np
import pandas as pd

df = pd.DataFrame({'FirstName': ['Vipul', 'Ashish', 'Milan'],
                            "Gender": ["", "", ""],
                            "Age": [0, 0, 0]})

df['Department'] = np.nan
display(df)

nan_value = float("NaN")
df.replace(0, nan_value, inplace=True)
df.replace("", nan_value, inplace=True)

df.dropna(how='all', axis=1, inplace=True)

display(df)

Output:

skrg141

Follow

Improve

Article Tags :

• Python
• Python-pandas
• Python pandas-dataFrame

Practice Tags :

• python