Use Machine Learning to

Forecast Future Earnings

Edward, XU Zhaoyu
Clair, CUI Xinyue
Ashlley, ZHOU Yue

The Hong Kong Polytechnic University

1
Contents

I. Project Overview ………………………................................ p. 3

II. Model Construction ………………………............................ p. 11

III. Data and Experiment ………………………......................... p. 19

IV. Results and Analysis ……………………….......................... p. 23

V. Conclusion ……………………….......................................... p. 32

2
I. Project Overview

3
Project Overview
Overview

Objective
• We are trying to select, adjust and integrate a series of machine learning or deep learning models to
comprehensively assess their feasibility and suitability on predicting the company fundamentals (i.e.
earnings).

Highlights
• Large Samples: select from Top 3000 Market
Capitalization Companies in US
• Multi-Class Prediction: 2 to 9 groups compared
to peers
• Rolling Window validation: 40 overlapping sets
• Relative Earnings Change: Year-over-Year 𝑁𝑒𝑡 𝐼𝑛𝑐𝑜𝑚𝑒!"# − 𝑁𝑒𝑡 𝐼𝑛𝑐𝑜𝑚𝑒!
(YoY) or Quarter-over-Quarter (QoQ) 𝐸! = 𝐶𝑙𝑎𝑠𝑠𝑖𝑓𝑖𝑐𝑎𝑡𝑖𝑜𝑛|
𝑁𝑒𝑡 𝐼𝑛𝑐𝑜𝑚𝑒!
4
Project Overview
Techniques

• Principal Component Analysis (PCA)

• Dimension reduction of input variables for
easier training

• LightGBM
• Gradient Boost Decision Tree (GBDT)
Main Classifier

• Hyperopt
• Tree of Parzen Estimators (TPE)
Hyperparameter Optimization
5
Project Overview
Variables

• Fundamental Data on Company Level

Financial Variables • 73 most relevant variables based on past literature
WRDS

• Gross Domestic Product (GDP)

• Inflation:
• Personal Consumption Expenditure Price Index (PCE)
Macroeconomic • S&P 500 Index
Variables • Interest Rate:
• Central bank market rate
• Short-term market rate (rM3)
• Long-term interest rate (rY10)

Market Variables • Abnormal returns of each company against the S&P500 benchmark

Time-series Variables • 19 Quarterly Lagging variables over the past 5 years

6
Project Overview
Literature Review

• Fundamental Data is highly supportive.

-----by Kothari (2001)

• Most frequently cited research papers of earnings forecast

• Ordinary least squares (OSL) regression (Lewellen, 2004)
• Time-series model (Myers, Myers, & Skinner, 2007)
• Random walk model (Watts & Leftwich, 1977)
• Statistical distribution of earnings

• Highest cited papers regarding machine learning models in Accounting & Finance
• Support vector machines (SVMs) (Kim, 2003; Pai and Lin, 2005)
• Neural networks (Campbell, 1987; Chang, Liu, Lin, Fan, & Ng, 2009; Chen, Leung, & Daouk, 2003)
• Autoregressive conditional heteroskedasticity (ARCH/GARCH) (Engle, 1982; Bollerslev, 1986)
• Gradient Boosting Decision Tree (GBDT) (Jones, Johnstone, & Wilson, 2015)

7
II. Model Construction

8
Model Construction
Dimension Reduction – Principal Components Analysis

Why bother to reduce the dimensionality of our features?

• In our project, initially there are 3091 different features (dimensions)
after adding the artificial lagging. Thus, the degree of freedom
would be so high that no way we could find sufficient number of
samples to ascertain every aspect of them.

• Moreover, there are also a lot of machine learning algorithm that

involves the sequencing of “distance” (norm). When #dimension
goes up, the comparison of distance will be much more difficult.
• The concept of “far” and “near” would become ambiguous in high-
dimensional spaces.

• The problems caused by sparse data samples and difficult

distance ordering in high-dimensional situations are serious
obstacles common to all machine learning methods, which are
called the curse of dimensionality (Bellman, 1957).

9
Model Construction
Dimension Reduction– Principal Components Analysis

Then, how do we deal with such issue?

Naturally, we want to reduce the number of dimensions.
Or more rigorously, that is to find a low-dimensional subspace (like a “simplified version”) of the original
high-dimensional dataset, in which the sample density will be greatly increased, and the distance ranking will
also become much easier.

Well, however, though such thought is quite simple and plain, but one has to realize that a “subspace” would
certainly mean that, it only contains “part” of the original information, while some of the other information
are correspondingly lost in the “reduced dimensions”.
10
Model Construction
Dimension Reduction – Principal Components Analysis

Trade-off between
Number of
Dimensions
and
Number of
Information

11
Model Construction
Gradient Boost Decision Tree - LightGBM

What is Machine Learning?

• Artificial intelligence is a science like mathematics or

biology. It studies ways to build intelligent programs and
machines that can creatively solve problems (which has
always been considered a human prerogative).

• Machine learning is a subset of artificial intelligence (AI)

that provides systems the ability to automatically learn
and improve from experience without being explicitly
programmed.

• Deep learning (or deep neural learning), is a subset of

machine learning, which uses the neural networks to
analyze different factors with a structure that is similar to
the human neural system.

12
Model Construction
Gradient Boost Decision Tree - LightGBM

What is Decision Tree? Revenue

Growth
A decision tree is actually an imitation of the decision-making process of us …
human-beings – You make a decision to an ultimate question by > 10%
progressively considering several ‘sub-questions’.
…….
D/A Ratio

Intuitively, we shall expect that with the development of such < 50% …
division (i.e. with the tree growing deeper), the ‘purity’ of samples
contained in each node will also increase gradually. Stock …….
Return
Ultimately, in the perfect case, only the real outperform stocks …
can be classified into the final category – “Outperform(2)”, the > 5%
purity of this class will then be 100% (i.e. 100% correct).
…….
Purity ↑ Outper
form
13
Model Construction
Gradient Boost Decision Tree - LightGBM

What is Decision Tree? Revenue

Growth
After each split, we shall assume that the ‘Information Entropy’ …
> 10%
for the subsets obtained would gradually decrease (purity ↑).
…….
D/A Ratio
For each split, we will then choose the feature that could bring
the largest improvements (“Information Gain”) as the criteria of < 50% …
classification.
Stock …….
Return
And similar procedures will go on and on, until you reached to
the pre-set limits (If the trees goes too deep, i.e. the > 5% …
classification is too specified, the result would become
useless -- Overfitted). …….
Purity ↑ Outper
form
14
Model Construction
Gradient Boost Decision Tree - LightGBM

What is LightGBM?
Accuracy

Introduced by Microsoft in 2017, LightGBM is a L

“ridiculously” fast toolkit specifically designed for
modeling extremely large data sets of high
dimensionality, often being more accurate and
many times faster than XGBoost or several other
previous tree-based algorithms.
Time
In virtue of its satisfactory and consistent
performance on multiple tasks, more and more
competitors on Kaggle (one of the largest data L
science contest platform) started to integrate such
model into their tasks.

15
Model Construction
Gradient Boost Decision Tree - LightGBM

What is LightGBM?
LightGBM grows tree vertically (Leaf-wise) while other algorithms grow trees horizontally (Level-wise).

LightGBM will only choose the leaf with max delta loss (another measurement of ‘purity’ ) to grow.

When growing the same leaf, Leaf-wise algorithm can reduce more loss (increase more purity) than a
level-wise algorithm.

LightGBM is prefixed as ‘Light’ because of its high speed in virtue of this ground-breaking algorithm.
Therefore, LightGBM can handle the large size of data and takes lower memory to run.
16
III. Data and Experiment

17
Data and Experiment
Sample Selection

How to Select Samples?

Universe Exclusion Final

• Quarterly Data in US • Remove ETF (i.e. without

• 333325 samples
Equity Market Total Asset)
from 4592
• 3000 companies with • Remove samples with companies
the highest market abnormal trading histories (i.e.
capitalization share price below $1)
• 30-year period from • Remove Utility (GICS:55) and
1988 to 2017 Finance (GICS:40) companies
• Remove Fiscal year-end not
on Mar, Jun, Sep, or Dec
18
Data and Experiment
Preprocessing

How to Preprocess Samples? 20 Quarters

• Add time-series with lagging data point NI Q-19 NI Q-1 NI Q0 NI Q1

• Standardize samples for time-series / cross-section analysis

• Year-over-year growth rate (YoY): Balance Sheet Items
• Quarter-over-quarter growth rate (QoQ): Income Statement / Cash Flow Statement Items
• Percentage of total asset (%Asset): Balance Sheet Items
• Percentage of total revenue (%Revenue): Income Statement / Cash Flow Statement Items
• Nominal value: Total Asset and Total Revenue as Company Size Indicator

19
Data and Experiment
Preprocessing

How to Preprocess Samples?

Outliers Missing Values

• (Before conversion) Replace all • Sample deletion: delete samples with

negative value to zero to avoid missing on Total Assets, Total Liabilities,
misleading conversion; Total Equity, Revenue, Net Incomes, and
Cash & Cash Equivalent;
• Use 95% Percentile as maximum
value to avoid infinity values • Variable deletion: remove variables with over
70% missing rate;
• Relevant fill-in: fill in with the rolling average
over the optimal fill-in period
• Constant fill-in: fill-in remaining missing with
constant value -1 to imply the missing
20
Data and Experiment
Process Illustration

< 348 Dimensions

Lowest 1/3 -> 0 (Underperform)

3 Bins Middle 1/3 -> 1 (Medium)
Top 1/3 -> 2 (Outperform)
21
Data and Experiment
Dependent Variables

𝑁𝑒𝑡 𝐼𝑛𝑐𝑜𝑚𝑒!"# − 𝑁𝑒𝑡 𝐼𝑛𝑐𝑜𝑚𝑒!

𝐸! = 𝐶𝑙𝑎𝑠𝑠𝑖𝑓𝑖𝑐𝑎𝑡𝑖𝑜𝑛|
𝑁𝑒𝑡 𝐼𝑛𝑐𝑜𝑚𝑒!

YoY Prediction:
Growth of Annual Net
Income

QoQ Prediction:
Growth of Quarterly Net
Income

22
IV. Results and Analysis

23
Results and Analysis
Benchmark Comparison – Logistic Regression

LightGBM vs Logistic Regression

Comparison of Prediction Performance of Sign of Earnings Changes

Methods Our Paper Ou and Penman (1989)* Hunt, Myers & Myers (2019)**
LightGBM 64.2%
Stepwise Logistic Regression 63.1% 62.1%
Elastic Net Logistic Regression 62.0%
* This paper predicts probablity of earnings increase in the subsequent year. Here shows the average accuracy.
** Here presents the maximum accuracy in both case.

• Our model outperform the accuracy of Both Logistic Regression Model proposed by Ou and
Penman (1989) and Hunt Myers & Myers (2019).

• All models here deal with Sign of Earnings Change problem (increase / decrease).
24
Results and Analysis
Benchmark Comparison – Consensus

LightGBM vs I/B/E/S Analysts’ Consensus Prediction

𝑁𝑒𝑡 𝐼𝑛𝑐𝑜𝑚𝑒!"# − 𝑁𝑒𝑡 𝐼𝑛𝑐𝑜𝑚𝑒!

𝐸! = 𝐶𝑙𝑎𝑠𝑠𝑖𝑓𝑖𝑐𝑎𝑡𝑖𝑜𝑛|
• Because I/B/E/S predict Non-GAAP 𝑁𝑒𝑡 𝐼𝑛𝑐𝑜𝑚𝑒!
Earnings, we compared the classified
I/B/E/S prediction with the classified I/B/E/S GAAP
Street Earnings. Earnings
• Mean
• For equivalent comparison, we cut • Medium • LightGBM
I/B/E/S consensus with the same
threshold for GAAP earnings Analysts’ LightGBM
conversion as the dependent variable of Consensus Accuracy Prediction
our model. Score
Non-GAAP Compare GAAP
Earnings Earnings
25
Results and Analysis
Benchmark Comparison – Consensus

LightGBM vs I/B/E/S Analysts’ Consensus Prediction

Comparison of Multi-Class Prediction Performance on Average

Types of Depentent Number of
Variables Class LightGBM Conesnsus (Mean) Conesnsus (Medium)
QoQ 3 52.7% 74.3% 74.2%
6 32.3% 56.3% 56.3%
9 23.4% 46.2% 46.3%
YoY 3 52.1% 71.6% 72.0%
6 32.4% 50.8% 51.4%
9 23.5% 40.4% 40.7%

• Our model underperform I/B/E/S Analysts’ Consensus Prediction.

• LightGBM performs relatively better for Year-over-Year (YoY) prediction and when the number
of class is less (i.e. 3-Class) .
26
Results and Analysis
Benchmark Comparison – Consensus

LightGBM vs I/B/E/S Analysts’ Consensus Prediction

Comparison of Converge Accuracy and Total Accuracy
Converge Total
Types of Depentent Number of
Variables Class LightGBM Conesnsus (Mean) LightGBM Conesnsus (Mean)
QoQ 3 68.4% 81.3% 52.7% 74.3%
6 52.0% 67.3% 32.3% 56.5%
9 43.3% 58.4% 23.4% 46.4%
YoY 3 63.0% 78.4% 52.1% 71.6%
6 43.7% 61.2% 32.4% 50.8%
9 33.5% 52.1% 23.5% 40.4%

• Our model improves the accuracy of I/B/E/S Analysts’ Consensus Prediction by evaluating the
converging case of LightGBM models and Consensus Prediction (i.e. same class).
• 3-class Quarter-over-Quarter problem achieved highest accuracy to 81.3%. However,
comparatively 9-class YoY problem improves the most by 29%. 27
Results and Analysis
Benchmark Comparison – Consensus

Why LightGBM fails Consensus?

QoQ (qcut = 3)
Analyst’s Superiority1 1
0.9

• Non-accounting information 0.8

• News 0.7
0.6
• Language & sentiment data 0.5
0.4
• Information released after the prior fiscal 0.3
year/quarter 0.2
0.1

Correlation 0

2008-03-31
2008-06-30
2008-09-30
2008-12-31
2009-03-31
2009-06-30
2009-09-30
2009-12-31
2010-03-31
2010-06-30
2010-09-30
2010-12-31
2011-03-31
2011-06-30
2011-09-30
2011-12-31
2012-03-31
2012-06-30
2012-09-30
2012-12-31
2013-03-31
2013-06-30
2013-09-30
2013-12-31
2014-03-31
2014-06-30
2014-09-30
2014-12-31
2015-03-31
2015-06-30
2015-09-30
2015-12-31
2016-03-31
2016-06-30
2016-09-30
2016-12-31
2017-03-31
2017-06-30
2017-09-30
2017-12-31
• Our results has 0.44 correlation with the Conesnsus (Mean) Conesnsus (Medium) LightGBM Random Walk

consensus prediction.

1 Fried 28
& Givoly 1982, Das, Levine, & Sivaramakrishnan 1998
V. Conclusion

29
Conclusion
Summary and Suggestions

Summary
• LightGBM is an innovative machine learning techniques that has great potential in
accounting and finance research.
• Our paper proves its ability to predict future earnings and generate relative accurate
results compared to many other statistical models.

Suggestions
• Unfortunately, due to the constrains of time and data access, our paper failed to
outperform consensus.
• Future research may implement Natural Language Processing (NPL) to include non-
financial data into the analysis and improves the results.

30
Thanks for Listening!
Edward, XU Zhaoyu
[email protected]

Clair, CUI Xinyue

[email protected]

Ashlley, ZHOU Yue

[email protected]
31