Week 4

AI applications in accounting and auditing:

Deep learning and sentiment analysis

Instructor: Ting Sun

 Deep Learning (Deep Neural Network)
Deep Learning is an AI technique that mimic the way a human
brain learns. It uses deep neural networks to extract high-level and
abstract features from raw data by building multiple layers of
representations that are expressed in terms of other, simpler
representations (Goodfellow et al. 2016).

2
Deep learning
Deep learning mimics how a human brain thinks. It aims to make a
machine think like humans.

“The general idea of deep learning is to use neural networks to build

multiple layers of abstraction to solve a complex semantic problem.”
-- Aaron Chavez, formerly chief scientist at Alchemy API
Artificial Neural Networks

Autoencoders: feature learning, dimension reduction

Convolutional NN: image recognition
4
Recurrent NN: speech recognition, sentiment analysis, NLP
Biological Neurons

Terminal Branches
Dendrites
of Axon

Electrical
impulse
soma

Axon

5
 Model Neuron

Terminal Branches
Dendrites
of Axon
x1
w1
x2
w2
x3 w3

S
Axon

wn
xn

6
 Artificial Neural Network

Input nodes Output nodes

Hidden nodes
Connections
Each layer’s output is simultaneously the
subsequent layer’s input, starting from an
initial input layer receiving your data Output : yi = f ( wi1 x1 + wi2 x 2 + wi3 x 3 + ! + wim x m )
= f ( å wij x j )
j
-0.06

W1

W2
-2.5 f(x)
W3

1.4
-0.06

2.7

-8.6
-2.5 f(x)
0.002 F(x) = -0.06×2.7 + 2.5×8.6 + 1.4×0.002 = 21.34

1.4
A dataset
Fields class
1.4 2.7 1.9 0
3.8 3.4 3.2 0
6.4 2.8 1.7 1
4.1 0.1 0.2 0
etc …
Training the neural network
Fields class
1.4 2.7 1.9 0
3.8 3.4 3.2 0
6.4 2.8 1.7 1
4.1 0.1 0.2 0
etc …
Training data
Fields class Initialise with random weights
1.4 2.7 1.9 0
3.8 3.4 3.2 0
6.4 2.8 1.7 1
4.1 0.1 0.2 0
etc …
Training data
Fields class Present a training pattern
1.4 2.7 1.9 0
3.8 3.4 3.2 0
6.4 2.8 1.7 1
4.1 0.1 0.2 0 1.4
etc …
2.7

1.9
Training data
Fields class Feed it through to get output
1.4 2.7 1.9 0
3.8 3.4 3.2 0
6.4 2.8 1.7 1
4.1 0.1 0.2 0 1.4
etc …
2.7 0.8

1.9
Training data
Fields class Compare with target output
1.4 2.7 1.9 0
3.8 3.4 3.2 0
6.4 2.8 1.7 1
4.1 0.1 0.2 0 1.4
etc …
2.7 0.8
0
1.9 error 0.8
Training data
Fields class Adjust weights based on error
1.4 2.7 1.9 0
3.8 3.4 3.2 0
6.4 2.8 1.7 1
4.1 0.1 0.2 0 1.4
etc …
2.7 0.8
0
1.9 error 0.8
Training data
Fields class Present a training pattern
1.4 2.7 1.9 0
3.8 3.4 3.2 0
6.4 2.8 1.7 1
4.1 0.1 0.2 0 6.4
etc …
2.8

1.7
Training data
Fields class Feed it through to get output
1.4 2.7 1.9 0
3.8 3.4 3.2 0
6.4 2.8 1.7 1
4.1 0.1 0.2 0 6.4
etc …
2.8 0.9

1.7
Training data
Fields class Compare with target output
1.4 2.7 1.9 0
3.8 3.4 3.2 0
6.4 2.8 1.7 1
4.1 0.1 0.2 0 6.4
etc …
2.8 0.9
1
1.7 error -0.1
Training data
Fields class Adjust weights based on error
1.4 2.7 1.9 0
3.8 3.4 3.2 0
6.4 2.8 1.7 1
4.1 0.1 0.2 0 6.4
etc …
2.8 0.9
1
1.7 error -0.1
Training data
Fields class And so on ….
1.4 2.7 1.9 0
3.8 3.4 3.2 0
6.4 2.8 1.7 1
4.1 0.1 0.2 0 6.4
etc …
2.8 0.9
1
1.7 error -0.1

Repeat this thousands, maybe millions of times – each time

taking a random training observation, and making slight
weight adjustments
Algorithms for weight adjustment are designed to make
changes that will reduce the error
Deep neural network ANN vs. DNN: The depth of the hidden layers

Extract features from unstructured data like

image, audio, video and text

As layers go further, it recognizes more

advanced and more abstract features of data

Each successive layer uses features in the

previous layer to learn more complex
features

Each hidden layer going further into the

network is a weighted non-linear
combinations of the lower level layers

The entire deep learning process is about

refining the weights
object models

object parts
(combination
of edges)

edges

pixels
 The Incremental Informativeness of
Sentiment in Conference Calls for the
Prediction of Internal Control Material
Weaknesses (ICMW)
An application of deep learning technology

Ting Sun
Published in Journal of Emerging Technologies in Accounting: Spring
2018, Vol. 15, No. 1, pp. 11-27.
 Objectives
• Demonstrate the effectiveness and efficiency of Deep Learning in the context
of auditing
• Examine the relationship between sentiment features of management in
conference calls and the likelihood of ICMW
• Investigate whether the sentiment features contain incremental information for
the prediction of ICMW

25
 Background and motivation
• An internal control material weakness is one or more
deficiencies in the design or operation of internal control that
create a reasonable possibility of a material misstatement in
the financial statements of the company (PCAOB, 2004).

26
SOX404

SOX 404 requires the disclosure of any material weaknesses in the

annual report and, more specifically, the report of management on
the company’s internal control over financial reporting (ICFR)
should be concluded ineffective by the auditor.

27
 IC Audit

• In order to form a basis for the opinion on the effectiveness of the ICFR,
auditors plan and perform the audit to obtain appropriate evidences
regarding the existence of material weakness in internal control (PCAOB,
2007).
• IC audit is integrated with FS audit
• IC audit planning, especially determining what evidence that auditors
will collect and how to use them, plays a key role in helping auditors
form a fair opinion
• The quality of IC audit is unsatisfactory due to information asymmetry

28
SEC’s annual reviews of registrants’ disclosures of
material weaknesses

29
Conference calls
• large-scale telephone conference calls
• management makes presentations on earnings etc.
• answers participants’ questions
• contains incremental information beyond mandated disclosures for
the situation of the company (Allee and Deangelis, 2015; Sedor, 2002)
• less formal and more flexible and spontaneous(Frankel, Johnson, and
Skinner, 1999).
• The language used by the managers may provide clues for their
feeling and cognitive process (i.e., sentiment, emotion)

30
 Hypotheses
H 1: The sentiment features of conference calls are significantly
associated with the likelihood of internal control material
weaknesses.
H2: The explanatory ability of the model that incorporates
sentiment features of conference calls along with major financial
determinants is superior to that of the model that merely uses the
financial determinants.

31
 Research Design

Data
Initial conference call transcript samples from 6379 The size of the final conference call (CC) transcripts
Seek iNF is 1758 corresponding to fiscal year from 2004 to
Less: Missing fiscal year information (1595) 2014, among which, 201 observations are related
to ICMW regulated by SOX 404.
Calculate the lowest score of sentiment features
for firm-years with multiple conference calls

Remaining: 2408
Less:
No internal control information (20)
Missing Compustat data (619)
Missing Audit Analytics data (11)
Final sample 1758

32
Sentiment Analysis Tool
• Watson Natural language understanding API, a deep learning
based text analysis cloud services
• A collection of text analysis functions that derive semantic
information from the content
• The deep learning model is trained with 200 billions of webpages
• Serves 40,000 developers and handles three billion API calls per
month

33
Sentiment features

• The sentiment features acquired from Watson NLU API include the
overall sentiment score (attitude) and the joy score.
• The returned sentiment score measures the sentiment strength of the
document, ranged from -1 to 1
• The score of joy values ranges from 0 to 1, which represents the
confidence level indicating the probability that the emotion of joy is
implied by the sample text.

34
Logistic regression
The Baseline Model
𝑰𝑪𝑾
= 𝜷𝟎 + 𝜷𝟏 𝑴𝒂𝒓𝒌𝒆𝒕𝒗𝒂𝒍𝒖𝒆 + 𝜷𝟐 𝑨𝒈𝒈𝒓𝒆𝒈𝒂𝒕𝒆𝒍𝒐𝒔𝒔 + 𝜷𝟑 𝑫𝒊𝒔𝒕𝒓𝒆𝒔𝒔 + 𝜷𝟒 𝑺𝒆𝒈𝒎𝒆𝒏𝒕𝒔 + 𝜷𝟓 𝑭𝒐𝒓𝒆𝒊𝒈𝒏
+ 𝜷𝟔 𝑰𝒏𝒗𝒆𝒏𝒕𝒓𝒐𝒚 + 𝜷𝟕 𝑹𝒆𝒔𝒕𝒓𝒖𝒄𝒕𝒖𝒓𝒆 + 𝜷𝟖 𝑨𝒄𝒒𝒖𝒊𝒔𝒊𝒕𝒊𝒐𝒏 + 𝜷𝟗 𝑹𝒆𝒔𝒊𝒈𝒏 + 𝜷𝟏𝟎 𝑩𝒊𝒈𝟒 + 𝜷𝟏𝟏 𝑳𝒊𝒕𝒊𝒈𝒂𝒕𝒊𝒐𝒏
+ ∑𝑰𝒏𝒅𝒖𝒔𝒕𝒓𝒚𝑭𝑬 + ℇ

35
The Sentiment Model
𝑰𝑪𝑾
= 𝜷𝟎 + 𝜶𝟏 𝑺𝒆𝒏𝒕𝒊𝒎𝒆𝒏𝒕 + 𝜶𝟐 𝑱𝒐𝒚 + 𝜷𝟏 𝑴𝒂𝒓𝒌𝒆𝒕𝒗𝒂𝒍𝒖𝒆
+ 𝜷𝟐 𝑨𝒈𝒈𝒓𝒆𝒈𝒂𝒕𝒆𝒍𝒐𝒔𝒔 + 𝜷𝟑 𝑫𝒊𝒔𝒕𝒓𝒆𝒔𝒔 + 𝜷𝟒 𝑺𝒆𝒈𝒎𝒆𝒏𝒕𝒔
+ 𝜷𝟓 𝑭𝒐𝒓𝒆𝒊𝒈𝒏 + 𝜷𝟔 𝑰𝒏𝒗𝒆𝒏𝒕𝒓𝒐𝒚 + 𝜷𝟕 𝑹𝒆𝒔𝒕𝒓𝒖𝒄𝒕𝒖𝒓𝒆
+ 𝜷𝟖 𝑨𝒄𝒒𝒖𝒊𝒔𝒊𝒕𝒊𝒐𝒏 + 𝜷𝟗 𝑹𝒆𝒔𝒊𝒈𝒏 + 𝜷𝟏𝟎 𝑩𝒊𝒈𝟒 + 𝜷𝟏𝟏 𝑳𝒊𝒕𝒊𝒈𝒂𝒕𝒊𝒐𝒏
+ ∑𝑰𝒏𝒅𝒖𝒔𝒕𝒓𝒚𝑭𝑬 + ℇ

36
 Results
primary analysis
• Logistic regression of the determinants of ICMW

37
 Predicted Estimate coefficients of group A Estimate coefficients of group B
sign Baseline model A Sentiment model A Baseline model B Sentiment model B

Intercept -14.5503 -14.5190 -13.9141 -13.7929

𝑀𝑎𝑟𝑘𝑒𝑡𝑣𝑎𝑙𝑢𝑒 - -0.2557*** -0.2486*** -0.3587*** -0.3343***
𝐴𝑔𝑔𝑟𝑒𝑔𝑎𝑡𝑒𝑙𝑜𝑠𝑠 + -0.2914 -0.2919 -0.4308 -0.4719
𝐷𝑖𝑠𝑡𝑟𝑒𝑠𝑠 + -3.64𝐸 !" -0.0009 -0.0058 -0.0046
𝑆𝑒𝑔𝑚𝑒𝑛𝑡𝑠 + 0.2755** 0.2762** -.0149 -0.0904
𝐹𝑜𝑟𝑒𝑖𝑔𝑛 + 0.3584 0.3820 Variable dropped Variable dropped

𝐼𝑛𝑣𝑒𝑛𝑡𝑟𝑜𝑦 + -0.1793 -0.1753 -1.4041 -1.0135

Growth + -0.8943** -0.8975**
𝑅𝑒𝑠𝑡𝑟𝑢𝑐𝑡𝑢𝑟𝑒 + -0.0581 -0.0656 -0.0708 -0.0550
Acquisition + 0.1934 0.2109 0.7645*** 0.7591***
𝑅𝑒𝑠𝑖𝑔𝑛 + 2.3138*** 2.3031*** 3.0643 *** 3.1091***
𝐵𝑖𝑔4 - -0.1997 -0.2042 0.2613 0.3247
𝐿𝑖𝑡𝑖𝑔𝑎𝑡𝑖𝑜𝑛 + 0.1045 0.1255 0.0438 0.0297
𝑆𝑒𝑛𝑡𝑖𝑚𝑒𝑛𝑡 ? 0.1079 1.0278
𝐽𝑜𝑦 - -1.2631** -2.5623**
Industry indicator Included Included Included Included
variables
Number of total 1758 1758 749 749
observations
Likelihood ratio, 94.35 101.81 78.94 86.03
𝜒#
(0.0001) (0.0001) (0.0001) (0.0001)
(p-value)
Pseudo 𝑅# 0.0755 0.0815 0.1577 0.1719
Likelihood-ratio 7.45** 7.09**
test: Likelihood
ratio (p-value) (0.0241) (0.0289)

38
 Conclusions

• With the incorporation of the sentiment features, especially the score of joy, the
explanatory ability of the model improves significantly, compared to the baseline
model that merely utilizes the major ICW determinants suggested by prior literature.

• Deep Learning is a promising technology that can effectively and efficiently help
auditors make decisions.

39
Thank You

40
Appendix: variables definition
where:
𝐼𝐶𝑊 = indicator equals 1 if there is a material internal control weakness in the company;
𝑀𝑎𝑟𝑘𝑒𝑡𝑣𝑎𝑙𝑢𝑒 = logarithm of share price multiplied by number of shares outstanding;
𝐴𝑔𝑔𝑟𝑒𝑔𝑎𝑡𝑒𝑙𝑜𝑠𝑠 = indicator equals 1 if earnings before extraordinary items in year t and t–1 sum to less than zero, and 0 otherwise;
𝐷𝑖𝑠𝑡𝑟𝑒𝑠𝑠 = Z-score (Altman, 1968), which measures financial distress of the company;
𝑆𝑒𝑔𝑚𝑒𝑛𝑡𝑠 = logarithm of the sum of the number of operating and geographic segments reported by the Compustat Segments database for the firm in
year t;
𝐹𝑜𝑟𝑒𝑖𝑔𝑛 = indicator equals 1 if the company has a non-zero foreign currency transaction in year t, and 0 otherwise. This variable is reported by Compustat
Segment database;
𝐼𝑛𝑣𝑒𝑛𝑡𝑟𝑜𝑦= inventory scaled by total assets;
𝑅𝑒𝑠𝑡𝑟𝑢𝑐𝑡𝑢𝑟𝑒 = indicator equals 1 if the company was involved in a restructuring in the last three years, and o otherwise;
𝐴𝑐𝑞𝑢𝑖𝑠𝑖𝑡𝑖𝑜𝑛=indicator equals 1 if the company engages in acquisitions in the last three years, and 0 otherwise.
𝑅𝑒𝑠𝑖𝑔𝑛 = indicator equals 1 if the auditor resigned in the year prior to an ICW disclosure; and 0 otherwise;
𝐵𝑖𝑔4 = indicator equals 1 if the firm is audited by a Big 4 audit firm, and 0 otherwise;
𝐿𝑖𝑡𝑖𝑔𝑎𝑡𝑖𝑜𝑛 = indicator equals 1 if the company is in a litigious industry, and 0 otherwise.
𝐼𝑛𝑑𝑢𝑠𝑡𝑟𝑦𝐹𝐸 = industry fixed effects
The number of shares outstanding is presented in millions
The definition of litigious industry follows Francis et al. (1994). Companies in litigious industries are with SIC codes of 2833-2836 (biotechnology);3570-
3577 (computer equipment);3600-3674 (electronics);5200-5961 (retailing); and 7370-7374 (computer services).

41