How to Drop Columns with NaN Values in Pandas DataFrame?

Last Updated : 24 Oct, 2020

Nan(Not a number) is a floating-point value which can't be converted into other data type expect to float. In data analysis, Nan is the unnecessary value which must be removed in order to analyze the data set properly. In this article, we will discuss how to remove/drop columns having Nan values in the pandas Dataframe. We have a function known as Pandas.DataFrame.dropna() to drop columns having Nan values.

Syntax: DataFrame.dropna(axis=0, how='any', thresh=None, subset=None, inplace=False)

Example 1: Dropping all Columns with any NaN/NaT Values.

Python3

# Importing libraries
import pandas as pd
import numpy as np

# Creating a dictionary
dit = {'August': [pd.NaT, 25, 34, np.nan, 1.1, 10],
       'September': [4.8, pd.NaT, 68, 9.25, np.nan, 0.9],
       'October': [78, 5.8, 8.52, 12, 1.6, 11], }

# Converting it to data frame
df = pd.DataFrame(data=dit)

# DataFrame
df

Output:

Python3

# Dropping the columns having NaN/NaT values
df = df.dropna(axis=1)

df

Output:

In the above example, we drop the columns 'August' and 'September' as they hold Nan and NaT values.

Example 2: Dropping all Columns with any NaN/NaT Values and then reset the indices using the df.reset_index() function.

Python3

# Importing libraries
import pandas as pd
import numpy as np

# Initializing the nested list with Data set
player_list = [['M.S.Dhoni', 36, 75, 5428000], 
               [np.nan, 36, 74, np.nan],
               ['V.Kholi', 31, 70, 8428000],
               ['S.Smith', 34, 80, 4428000], 
               [pd.NaT, 39, 100, np.nan],
               [np.nan, 33, 90.5, 7028000],
               ['K.Peterson', 42, 85, pd.NaT]]

# creating a pandas dataframe
df = pd.DataFrame(player_list, columns=['Name', 'Age', 
                                        'Weight', 'Salary'])

df

Output:

Python3

# Dropping the columns having NaN/NaT values
df = df.dropna(axis=1)

# Resetting the indices using df.reset_index()
df = df.reset_index(drop=True)

df

Output:

In the above example, we drop the columns 'Name' and 'Salary' and then reset the indices.

Example 3:

Python3

# Importing libraries
import pandas as pd
import numpy as np

# creating and initializing a nested list
age_list = [[np.nan, 1952, 8425333, np.nan, 28.35], 
            ['Australia', 1957, 9712569, 'Oceania', 24.26],
            ['Brazil', 1962, 76039390, np.nan, 30.24],
            [pd.NaT, 1957, 637408000, 'Asia', 28.32], 
            ['France', 1957, 44310863, pd.NaT, 25.21],
            ['India', 1952, 3.72e+08, pd.NaT, 27.36], 
            ['United States', 1957, 171984000, 'Americas', 28.98]]

# creating a pandas dataframe
df = pd.DataFrame(age_list, columns=[
                  'Country', 'Year', 'Population', 'Continent', 'lifeExp'])

df

Output:

Python3

# Dropping the columns having NaN/NaT values
df = df.dropna(axis=1)

# Resetting the indices using df.reset_index()
df = df.reset_index(drop=True)

df

Output:

In the above example, we drop the columns 'Country' and 'Continent' as they hold Nan and NaT values.

Example 4: Dropping all Columns with any NaN/NaT Values under a certain label index using 'subset' attribute.

Python3

# Importing libraries 
import pandas as pd 
import numpy as np 

# Creating a dictionary 
dit = {'August': [10, np.nan, 34, 4.85, 71.2, 1.1], 
       'September': [np.nan, 54, 68, 9.25, pd.NaT, 0.9], 
        'October': [np.nan, 5.8, 8.52, np.nan, 1.6, 11],
       'November': [pd.NaT, 5.8, 50, 8.9, 77, pd.NaT] }

# Converting it to data frame
df = pd.DataFrame(data=dit)

# data frame
df

Output:

Python3

# Dropping the columns having NaN/NaT values
# under certain label index using  'subset' attribute
df = df.dropna(subset=[3], axis=1)

# Resetting the indices using df.reset_index()
df = df.reset_index(drop=True)

df