Stock Market Prediction using Procedural And

Exploratory Data Analysis

Sachin Sudam Ugale Shaikh Mohammad Bilal N Assistant Professor,
KJ Somaiya College of Science and Commerce, Dept of CS KJ Somaiya College Mumbai,
Mumbai India [email protected] India [email protected]

Abstract— Investing in stock market is equivalent to average (ARMA) model. Then, presented the
gambling as the mathematicians believe it’s a stochastic process sentiment analysis details of the financial news and
which cannot be predicted by only modeled method using introduced how to use them to improve prediction
equations. Data scientist and AI experts on the other hand believe performance. At last, introduced the differential
predicting stocks behavior is possible. According to them it’s a
privacy framework and the loss function
series of pattern and increase the profit of investors by reducing
factors such as human impulsivity. A method can be made that
performs tasks like analyzing previous data, predict stock • R. Goonatilake and S. Herath, in their research
behavior using models and algorithms and study the efficiency of monitored the opening and the closing of the three
the market prominent market indicators for ten weeks. They also
categorized the news into four types Global Stability,
Keywords— Algorithms, Equations, Text Mining, Sentiment Economic Data, world business and social cultural
analysis. changes later segregating it into good news, bad news
and neutral news. Accordingly observed the impact
I. Introduction on stock market if the number of good news is more
or bad news is more.
In the investment field, stock market and its trends are
awfully unstable in character. Investors and market analysts All proved to be effective to a certain extent. But they relied
learn the market actions and map their buy or sell strategies on a single source of platform for news or only the previous
accordingly. As stock market generates huge amount of data recorded data.
per minute, it is nearly impossible for a person to judge on the
basis of all the modern and precedent information for
predicting future trend of a stock. [1] This made researchers
III. Proposed methodology
and analysts eager to find a technique that captures the
capricious nature and predict the next trend of the market.
Primarily there are two techniques for forecasting market
trends. One is procedural analysis and other is elemental
analysis. Procedural analysis reflects on past price and volume
[2] where as elemental analysis on the other hand, involves
analyzing its monetary data to get some insights such as news
and future plans. The value of both procedural and elemental
analysis is unclear by the efficient-market theory which states
that “stock market prices are essentially unpredictable”.

II. Literature survey

• Kalyani Joshi, Prof. Bharathi H. N. and Prof. Jyothi
Rao In their paper they checked the impact of news
articles on stock prices. They used supervised
machine learning as classification and other text
mining techniques to check news polarity Figure 1: Flow of the proposed method

• Xinyi Li, Yinchuan Li, Hongyang Yang, Liuqing

Yang and Xiao-Yang Liu In their research first
introduced the background of the stock price model,
which was based on the autoregressive moving
Step 1:- ARIMA Model

The ARIMA model, one of the most extensively worn linear

models in time series prediction, [6][7] be used for assuming
the future behavior of the stocks based on the prior filed
records and the trends. Autoregressive Integrated Moving
Average (ARIMA) furthermore recognized as Box-Jenkins
approach. According to Box and Jenkins, non-stationary data
can be made stationary [8] by differencing the series Yt. [2]
The general model for Yt is written as.

Yt =ϕ1Yt−1 +ϕ2Yt−2…ϕpYt−p +ϵt + θ1ϵt−1+ θ2ϵt−2

+…θqϵt−q [2]

Where Yt is the differenced time series value, ϕ and θ are

unknown parameters and ϵ are independent identically
distributed error terms with zero mean. Here, Yt is expressed Figure 2 Actual vs Forecasted :Source: Google
in terms of its past values and the current and past values of
error terms. Step 2:- News Collection
The ARIMA model combines three basic methods:
• AR: < Auto Regressive > means that the The data set used for sentiment analysis should also be
model utilizes the reliant relationship involving an considered from as much as social media platforms as possible
such as twitter, facebook, yahoo finance and many more. The
observation and some predefined number of holdup
prospect strategy should also be taken into account for
observations (also known as “time lag” or “lag”).[5] performing the following Algorithm [1]:
• I:< Integrated > means that the model 1. Tokenize the document into word vector.
utilize differencing of raw observations (e.g. it 2. Organize the vocabulary which contains words with its
subtracts an observation from an observation at the polarity (positive or negative)
previous time step) in order to make the time- 3. Verify alongside every word weather it is equivalent with
series stationary.MA: [5] one of the word from positive word dictionary or negative
words dictionary.
• MA: < Moving Average > means that the model 4. Tally amount of words that belongs to positive and negative
polarity.
utilizes the relationship between the residual error
5. Calculate Score of document = count (pos.matches) – count
and the observations.[5] (neg.matches)
6. If the Score is 0 or more, we consider the document is
Model parameters positive or else, negative.
The standard ARIMA models expect as input parameters 3
arguments i.e. p,d,q. [9] As the majority of the study illustrate that SVM, Random
• p is the number of lag observations. Forest and Naïve Bayes [1][2] classification algorithms
achieve excellent in text classification. So take into account of
• d is the degree of differencing. all three algorithms to categorize the text and check each
algorithm’s precision.
• q is the size/width of the moving average window. Step 3:- involving the association group

Even if administration policies and communal authority

problem have a big impact on stock prices they are not the
lone manipulator [4]. Commodity cost can also crash a
manufacturing company's stock value.
 IV. Conclusion
In this paper, I proposed the integrated model of ARIMA and
sentiment analysis to predict the stock trend with precision
based on documented data in the past and current textual as
well as numerical data so that the investment risk can be
minimum.
In the potential development, the data set can be improved by
modifying and adding a few entities. Since this model uses
two basic analyses, in the future, more complex analysis can
be added. That can foretell the value array by which stock will
go up or down

Acknowledgment
I would like to thank my guide, teachers and friends who
supported in the completion of this research paper.
Appreciating everyone who helped me knowingly or
unknowingly for this paper.

References
[1] Kalyani Joshi, Prof. Bharathi H. N., Prof. Jyothi Rao International
Figure 3commodity vs stock price:Source: Google Journal of Computer Science & Information Technology (IJCSIT) Vol
8, No 3, June 2016 DOI:10.5121/ijcsit.2016.8306 67 STOCK TREND
PREDICTION USING NEWS SENTIMENT ANALYSIS
Above is the commodity cost vs stock price example
[2] Xinyi Li1, Yinchuan Li, Hongyang Yang, Liuqing Yang, Xiao -Yang Liu
DP-LSTM: Differential Privacy-inspired LSTM for stock Pred iction
Using Financial News, Columbia University, Beij ing Institute of
Technology
[3] J. Bean, R by example: Mining Twitter for consumer attitudes towards
airlines, In Boston Predictive Analytics Meetup Presentation, 2011
[4] Rohita Goonatilake and Susantha Herath: The Volatility of the market
and news, International Research Journal of Finance and Economics
[5] ARIMA Model https://towardsdatascience.com/time-series-forecasting-
predicting-stock-prices-using-an-arima-model-2e3b3080bd70
[6] ARIMA Model https://blog.quantinsti.com/forecasting-
stock-returns-using-arima-model/
[7] Investigating the Pattern of Inflation Rates in Nigeria usin gSeasonal
Auto-Regressive Integrated Moving Average(SARIMA) Model
Figure 4 stock price affected by crude oil [4] IJIRMPS | Volume 8, Issue 3, 2020
[8] Flood Predict ion using ARIMA Model in Sungai Melaka, Malaysia
The above is the observed data from the R. Goonatilake and S. Volume 9, No.4, July August 2020 International Journal of Advanced
Herath research. As you can see there is a strong association Trends in Computer Science and Engineering
between DJIA and crude oil prices. They are directly [9] Price analysis and f orecasting f or decision making: Insights
proportional to each other [4] f romwheat markets in India Indian Journal of Agricultural
Sciences May 2020/Article
[10] Property Price Prediction Application Developed Using Machine
Learning Algorithm, Shaikh Mohammad Bilal N Assistant
Prof essor, Dept of CS KJ Somaiya College