How to transform features into Normal/Gaussian Distribution

I NT E RM E D I AT E M A C HI NE LE A RNI NG M AT HS PYT HO N S T AT I S T I C S

This article was published as a part of the Data Science Blogathon

Introduction

In Machine learning or Deep Learning, some of the models such as Linear Regression, Logistic Regression,
Artificial Neural Networks assume that features are normally distributed and can perform much better if
the features provided to them during modeling are normally distributed.

But the data provided to us does not necessarily follow a normal distribution. So what should we do in
such cases? That’s what we are going to cover in this article.

Table of Content

1. A brief introduction to Gaussian Distribution

2. How to check if a variable is following Gaussian Distribution
3. Various Transformations to change the distribution of features

Log Transformation
Square root Transformation
Reciprocal Transformation
Exponential Transformation
Box-Cox Transformation

Introduction to Gaussian Distribution

In probability theory, a normal (or Gaussian) distribution is a type of continuous probability distribution for a real-valued random
variable. The general form of its probability density function is

Samples of the Gaussian Distribution follow a bell-shaped curve and lies around the mean. The mean,
median, and mode of Gaussian Distribution are the same.

To read more about Normal/Gaussian Distribution in detail, refer to the attached article.

How to check if a variable is following Normal Distribution

There are various ways in which we can check the distribution of the variables. Some of them are:

Histogram
Q-Q plot
KDE plot
Skewness and Kurtosis
Five Number Summary (This one just gives the glimpse and not the entire detail. Check out my blog if
you want to know more about this concept)

We will see all the above-mentioned methods with the help of a dataset. For this, we will use a subset of
Car price dataset where categorical variables will be excluded for now and will check the distributions on
numerical variables.

#importing library import pandas as pd #reading the dataset cp = pd.read_csv('CarPrice_Assignment.csv')

#taking the subset of the data
cp=cp[['symboling','wheelbase','carlength','carwidth','carheight','curbweight','boreratio','stroke','compressionratio
#printing first five rows of the data cp.head()

Output :

Checking the distribution with Skewness

#skewness in the data cp.skew()

Output:
From the above result, we can check which variable is normally distributed and which is not.

The variables with skewness > 1 such as wheelbase, compressionratio, horsepower, price are highly
positively skewed.
The variables with skewness < -1 are highly negatively skewed.
The variables with 0.5 < skewness < 1 such as carwidth, curbweight, citympg are moderately positively
skewed.
The variables with -0.5 < skewness < -1 such as stroke are moderately negatively skewed.
And, the variables with -0.5 < skewness < 0.5 are symmetric i.e normally distributed such as
symboling, carheight, boreration, peakrpm, highwaympg.

Checking the distribution of some variables using Histogram

cp.horsepower.hist() plt.title('Distribution of horsepower');

Output: Highly positive Skewed i.e does not follow a normal distribution

cp.stroke.hist() plt.title('Distribution of stroke');

Output: Moderately negatively Skewed i.e does not follow a normal distribution
 

cp.highwaympg.hist() plt.title('Distribution of highway mpg');

Output: Symmetric i.e does follow a normal distribution

Checking the distribution of variables using KDE plot

sns.kdeplot(cp.compressionratio);

sns.kdeplot(cp.price);
The distribution of both the variables compression ratio and the price is highly positively skewed and is
not normally distributed.

Checking the distribution of variables using a Q-Q plot

A Q-Q plot is a scatterplot created by plotting two sets of quantiles against one another. If both sets of quantiles came from the same
distribution, we should see the points forming a roughly straight line.

That is, if the data falls in a straight line then the variable follows normal distribution otherwise not.

#importing necessary libraries import scipy.stats as stats import pylab

From the Skewness and KDE plots, we have seen that the price variable is highly positively skewed. Let’s
plot the Q-Q plot for the Price variable and check.

stats.probplot(cp.price,plot=pylab);

The X-axis of the above plot has Quantiles values and Y-axis has the price values. From the plot, we can
analyze that the data points of the price feature are not falling on a straight line. This implies that it does
not follow a normal distribution.

Let’s check for the city mpg feature now. The graph shows that the majority of the data points of city mpg
fall on the straight line so the distribution is normal.

stats.probplot(cp.citympg,plot=pylab);
 

Transformations to change the distribution of features

Till now we have seen how to check the distribution of the feature. This section will learn how to convert
the variables into the normal distribution if they are not following it.
For this purpose, we will work on a single feature first and will see which transformation out of all works
the best to convert the feature into a normal distribution.

First of all, defining a function in python which will take data and feature name as inputs and return the
KDE plot and Q-Q plot of the feature.

#function to return plots for the feature def normality(data,feature): plt.figure(figsize=(10,5))

plt.subplot(1,2,1) sns.kdeplot(data[feature]) plt.subplot(1,2,2) stats.probplot(data[feature],plot=pylab)

plt.show()

Plotting for the Price feature using the above-defined function:

n orm a lity(cp,’price’)

Performing the transformations on the Price variable.

Logarithmic Transformation – This will conver t the Price value to its log value i.e log(Price)

#performing logarithmic transformation on the feature cp['price_log']=np.log(cp['price'])

#plotting to check the transformation normality(cp,'price_log')

The distribution changed slightly and looks moderately skewed now. More data points are also falling on
the line in the Q-Q plot.

Reciprocal Transformation – This will inverse values of Price i.e1/Price

cp['price_reciprocal']=1/cp.price normality(cp,'price_reciprocal')

The Reciprocal Transformation has kind of conver ted the feature to normal distribution. Most of the data
points are also falling on the line in the Q-Q plot. This one is better than the log transformation for the
Price feature.

Square Root Transformation – This transformation will take the square root of the Price column i.e
sqr t(Price).

cp['price_sqroot']=np.sqrt(cp.price) normality(cp,'price_sqroot')
This one has reduced the skewness of the feature but has not transformed it into a normal distribution.

Exponential Transformation: The exponential value of the Price variable will be taken.

cp['price_exponential']=cp.price**(1/1.2) normality(cp,'price_exponential')

Box-Cox Transformation – The Box-Cox transformation is defined as:

where y is the response variable and λ is the transformation parameter. λ value varies from -5 to 5. During
the transformation, all values of λ are considered and the optimal/best value for the variable is selected.
log(y) is only applied when λ=0.

cp['price_Boxcox'],parameters=stats.boxcox(cp['price']) normality(cp,'price_Boxcox')

Box cox transformation technique also gives a good result and normally distributes the Price feature.
The data points on the Q-Q plot are almost on the line as well.

From all the transformations discussed above, we can conclude that the Box cox and Reciprocal
transformation perform the best on the Price variable and transform it to normal distribution. Any one of
the two can be used but as Box cox is more logic-based and involves the λ variable which is chosen as per
the best skewness for the data so Box cox will be a better transformation to go with.

About the Author

I am Deepanshi Dhingra currently working as a Data Science Researcher, and possess knowledge of
Analytics, Exploratory Data Analysis, Machine Learning, and Deep Learning. Feel free to content with me on
LinkedIn for any feedback and suggestions.

The media shown in this ar ticle are not owned by Analytics Vidhya and is used at the Author’s discretion. 

Article Url - https://www.analyticsvidhya.com/blog/2021/05/how-to-transform-features-into-normal-

gaussian-distribution/

deepanshi6