01-06-0107 Infocus_Sun.

qxp_Layout 1 6/2/17 2:20 PM Page 24

Deep Learning and the

In
FOCUS

Future of Auditing
How an Evolving Technology Could Transform Analysis and Improve Judgment

By Ting Sun and Miklos A. Vasarhelyi

Some tasks that individuals accomplish effortlessly are actu-

Introduction
IN BRIEF ally extremely complex computational problems. Imagine how
This article introduces deep learning technology—an emerging easily a human can distinguish cats from dogs, read traffic
signs, or identify handwriting. In order for a machine to per-
form these tasks, the object must be converted to a machine-
form of artificial intelligence that can be trained to recognize pat-

readable format and analyzed pixel-by-pixel. Due to variations

terns in vast volumes of data that would be impossible for humans

in the object’s position, viewpoint, pose, lighting, and back-

to process. This still evolving technology represents a way to
utilize big data to create supplementary audit evidence that ground, the complexities of the image—and thus, the object
itself—can be represented in a variety of ways (Nicolas Pinto,
David D. Cox, and James J. DiCarlo, “Why is Real-World
improves the effectiveness and efficiency of audit automation and

Visual Object Recondition Hard?” PLOS Computational

decision making. The authors also discuss the application of these

Biology, Jan. 25, 2008, http://bit.ly/2qMUuG4). Tasks like

techniques to audit procedures.

object recognition require the construction of deep artificial

neural networks that simulate the multiple layers of neurons
n the current business environment, the development of in a human brain and the ways that sensory data streams are
data-intensive technologies (e.g., ERP systems, sensors,

I cloud storage, remote communication tools) facilitates the

production and maintenance of large amounts of data,
which necessitates a new data environment and serves as
a motivator for audit automation. Leading accounting firms
have leveraged deep learning, a cutting-edge use of artificial
intelligence, to conduct audit tasks. For example, KPMG applies
processed through those layers (Robert D. Hof, “10
Breakthrough Technologies—Deep Learning,” MIT
Technology Review, 2013, http://bit.ly/2rIzj5Y).
Human brains contain about 100 billion interconnected neu-
rons. Each neuron receives input signals (e.g., the visual signal
of a cat) from other connected neurons. If the combination of
IBM Watson’s deep learning–powered systems to analyze these signals exceeds a certain threshold (or activation level),
banks’ credit files for commercial mortgage loan portfolios, and the neuron transmits an output signal to other neurons. Humans’
Deloitte has allied with Kira Systems to review contracts, leases, deep layers of neurons and their complex interconnections result
invoices, and tweets. The adoption of deep learning within the in a “thinking machine,” the human brain (“Neuralyst Users
accounting profession is still, admittedly, at an early stage. To Guide,” Cheshire Engineering Corporation, 1994,
accelerate the wider use of this technology, it is necessary to http://bit.ly/2qOM3bk).
create economies of scale by integrating its cognitive capabilities Although the idea of artificial neural networks dates back to
in the areas of textual analysis, voice recognition, image and the 1950s, such networks could not be called real artificial intel-
video parsing, and judgment support into the audit process. This ligence until recent advances in computational power and data
article discusses how the cognitive capabilities of deep learning storage enabled the development of deep neural networks that
could be applied to various audit procedures to enable audit model the structure and thinking process of the brain. The hid-
automation and improve decision making. den layers of a deep neural network automatically “learn” from

24 JUNE 2017 / THE CPA JOURNAL

01-06-0107 Infocus_Sun.qxp_Layout 1 6/2/17 2:20 PM Page 25

massive amounts of data (especially semi- essential data element of the input images, layer identifies the face. The output will
structured or unstructured data) received the intensity of the pixels. Second, the be compared with the actual observation
by the input layer (e.g., millions of input neuron transforms the data with a to check for errors, and the system will
images, years’ worth of speeches, tera- nonlinear algorithm. During this process, adjust the output weights and other param-
bytes of text files), recognize data patterns the output of each neuron in the input layer eters in the next round of training. The
in more and more abstract representations is assigned a random weight, then com- entire “learning” process will be repeated
as the data is processed and transmitted bined and sent to the neuron of the next millions, or even billions, of times until
from one hidden layer to the next, and layer (the first hidden layer); now the first error is minimized. The more examples
classify the data into predefined cate- hidden layer can extract more abstract data the model is trained with, the higher accu-
gories in the output layer. representation, the edge (or the simple racy the system can achieve. The perfor-
shape) comprising the pixels. Third, the mance of such a deep learning–based
neurons in the first hidden layer perform model (e.g., for financial misstatement
another round of complex nonlinear trans- detection) improves the sample size. More
The Effectiveness and Efficiency of

In order to ensure predictive effective- formations to the received data and assign importantly, out-of-sample tests can be
Deep Learning

ness, a deep neural network is trained with them new weights. The successor hidden conducted with new samples to validate
a relatively large amount of data (e.g., mil- layers then receive, transform, and send the effectiveness of the model using a set
lions of images of human faces) to allow the data to the next layer in their turn; the of metrics (e.g., F score, precision, recall,
the system to fine tune the parameters to system detects more and more complex AUC).
minimize prediction errors. First, the input data features (e.g., parts of the face) with The efficiency of deep learning can be
layer receives and identifies the most each successive layer. Finally, the output demonstrated by the fact that it is able

JUNE 2017 / THE CPA JOURNAL 25

01-06-0107 Infocus_Sun.qxp_Layout 1 6/2/17 2:21 PM Page 26

In
FOCUS

to “learn” patterns in data without human tional text mining approach for predicting reports that are issued quarterly or annu-
intervention and requires fewer data pre- financial misreporting using the senti- ally. Thus, the use of big data analysis
processing steps compared to traditional ment of management's discussion and in auditing has been hampered due to the
data mining approaches. For example, in analysis (MD&A). lack of effective and efficient technolo-
the case of textual analysis, a person is gies to solve the issues in data extraction,
usually needed to remove HTML tags, transformation, and validation. While the
translate HTML characters into text char- Semi-structured or unstructured big challenges of big data analysis require a
The Challenge of Big Data Analysis

acters, drop stop words and non-linguis- data contains a variety of information willingness to adopt more advanced data
tic marks, and develop word lists. With that allows auditors to freely explore the analytical technologies, such as deep
a deep learning model, however, this status of their clients’ products, services, learning, the availability of massive
time-consuming process is unnecessary. and operations and reduces auditors’ amounts of financial data facilitates the
IBM Watson develops deep learning– dependence on their clients for data implementation and improvement of this
based textual analysis that can directly (Kyunghee Yun, Lucas Hoogduin, and technology in auditing.
read text files or even URLs; it automat- Li Zhang, “Big Data as Complementary
ically removes advertisements, naviga- Audit Evidence,” Accounting Horizons,
tion links, and other irrelevant content June 2015, http://bit.ly/2rIFRl0). As a
The Need for Automated Audit

The automation of some tedious and

Procedures

repetitive audit processes would signifi-

Given a sufficiently large sample of how auditors make cantly enhance audit effectiveness and
efficiency (Jon Raphael, “How Artificial
decisions under various circumstances, a deep learning Intelligence Can Boost Audit Quality,”
CFO, June 2015, http://bit.ly/2q301ZM).
Given a sufficiently large sample of how
auditors make decisions under various
system allows auditors to automate many tasks that
circumstances, a deep learning system
have traditionally been conducted manually. allows auditors to automate many tasks
that have traditionally been conducted
and creates a list of data features such as result, mining and extracting meaningful manually, such as checking inventories,
authors, keywords, concepts, relation- patterns from big data has great value for processing paperwork, reviewing con-
ships among those concepts, and embed- audit decision-making, especially in the tracts, and even drafting audit reports.
ded sentiments or emotions (“Overview area of risk assessment. Analyzing big
of the IBM Watson Natural Language data is not easy, however; big data anal-
Understanding Service,” IBM, ysis was cited as one of the top chal- The application of deep learning to
Possible Applications

https://ibm.co/2rIsSQh). The tool pro- lenges of the future by 25% of improve audit efficiency and effective-
vides a high level of accuracy, and it is respondents to a 2014 AICPA survey ness is especially relevant to facilitating
continuously improving as more and (“2014 AICPA Survey on International repetitive audit procedures and support-
more data is used to train the model; as Trends in Forensic and Valuation ing audit judgments. Deep learning can
of February 2014, it claims to have pro- Services,” http://bit.ly/2q3avIT). add value for routine tasks that involve
cessed over 3 billion documents a month The vast majority of big data is semi- massive amounts of data and require sig-
for 40,000 users across 36 countries structured or unstructured, and requires nificant effort for auditors to solve, such
(Janet Wagner, “Deep Learning: 6 Real labeling and categorizing; however, audi- as text analysis, speech recognition, and
World Use Cases,” AlchemyAPI, 2014, tors cannot do this manually, since the parsing images and videos. It can also
http://bit.ly/2qczSTI). A recent study data contains a variety of types and be used to reduce manual work by
(Ting Sun, Yue Liu, and Miklos A. sources and is too voluminous for automating some substantive procedures,
Vasarhelyi, “The Performance of humans to process it all. In addition, por- such as confirmation and examination.
Sentiment Feature of MD&As for tions of big data are usually generated Furthermore, these competencies will
Financial Misstatements Prediction,” on a real-time basis, so it requires timely allow auditors to perform tasks, such as
working paper, 2017) found that this responses. Furthermore, automated trad- examining all corporate contracts, that
technique, without text preprocessing ing, which accounts for the majority of are currently cost prohibitive or too com-
work, generally outperforms the tradi- stock trades, cannot work well with plex for unaided human minds.

26 JUNE 2017 / THE CPA JOURNAL

01-06-0107 Infocus_Sun.qxp_Layout 1 6/2/17 2:21 PM Page 27

Text analysis. A large amount of tex- may be filled with slang, idioms, and cation. Although public accounting firms
tual data is generated and disseminated other linguistic clues reflecting the cog- offer deception detection training to help
during a company’s operating processes, nitive process of the speaker (Marina their auditors identify verbal red flags, the
such as regulatory filings, transcripts of Druz, Alexander F. Wagner, and Richard information processing of interviews is a
conference calls, press releases, earnings J. Zeckhauser, “Tips and Tells from daunting task because interviewees exhibit
announcements, MD&As, business con- Managers: How Analysts and the Market myriad verbal behaviors. It is difficult and
tracts, news articles, and social media Read between the Lines of Conference inefficient for auditors to analyze all oral
messages. Textual data provides infor-
mation on multiple aspects of a business
from various perspectives. For example,
MD&A contains management’s perspec-
Exhibit 1

tive on the company’s current financial

Input, Output, and Applicable Audit Procedures for Deep Learning–Based

situation and future prospects, analysts’

Textual Analysis

reports include retrospective analysis of

past events and forecasts of future earn-
Examples

ings and cash flows, and social media

posts can include advertisements, product
Input Data Regulatory filings, press releases, earnings announcements,

reviews, and news announcements.

MD&As, business contracts, comment letters from SEC,

Textual analysis can be automated by

news articles, analysts’ reports, e-mails, disclosures on
company’s website, social media messages

deep learning; specifically, text data can

be classified based on features of interest. Output Features Sentiment, emotion, entity, topic, concept, keywords, authors

Furthermore, a deep learning model can

be trained using transcripts of Q&A sec- Applicable Audit Procedures Inspection, analytical procedure, confirmation

tions of conference calls—labeled as pos-

itive, negative, or neutral in terms of
sentiment—to predict future calls. The
MD&A=management’s discussion and analysis

entire procedure can be performed auto-

matically, and the result is machine-read- Calls,” National Bureau of Economic responses or transcribe them into text
able. In this way, a deep learning model Research, 2015, http://bit.ly/2rKXiAH), manually, and even transcribed text doc-
transforms qualitative information that which may provide clues to potential uments can still be a chore for auditors to
used to require great human effort to ana- risks. Exhibit 1 provides examples of tex- analyze.
lyze into quantitative data that can easily tual documents, possible output features, The speech recognition function of
be integrated with other data for further and audit tasks for which deep learning deep learning can now transcribe and
audit analysis. is applicable. translate speech on a real-time basis
Deep learning algorithms further Speech recognition. To acquire back- regardless of noise or the various accents
enrich audit evidence by identifying relat- ground information about a client’s busi- of speakers, enabling analysis of the text
ed concepts or topics, recognizing entities ness and industry environment and to and extraction of emotion, risk factors,
(e.g., people, place, events, companies), collect audit evidence, auditors interview and other insights directly. Researchers
extracting emotions (e.g., anger, joy, sad- management, internal auditors, employees, are now considering using embodied
ness, disgust), and understanding subject- predecessor auditors, bankers, legal coun- conversational agents (ECA), which are
action-object relationships. In addition, sel, underwriters, analysts, or other stake- “autonomous computer interfaces capa-
they can link concepts to a document and holders. The language that subjects use ble of human-like interactions” (Matthew
tag them accordingly. and how they respond to questions over D. Pickard, Mary B. Burns, and Kevin
This textual analysis is better suited to the course of the interview can be just as C. Moffit, “A Theoretical Justification
unprepared content (the Q&A section of important as the answers themselves, for Using Embodied Conversational
conference calls, Facebook status because they may indicate deception. For Agents to Augment Accounting-Related
updates, blog posts) than prepared con- example, the use of terms that suggest Interviews,” Journal of Information
tent (e.g., press releases, the presentation uncertainty, such as “kind of,” “maybe,” Systems, Fall 2013, http://bit.ly/2qcBf4Z),
section of conference calls). Compared or “sort of,” as well as response latency, to conduct interviews automatically. Due
to prepared content, unprepared content could be signs of concealment or falsifi- to technical limitations, current ECAs ask

JUNE 2017 / THE CPA JOURNAL 27

01-06-0107 Infocus_Sun.qxp_Layout 1 6/2/17 2:22 PM Page 28

In
FOCUS

data fields to develop a new deep learn-

ing prediction model. Deep learning per-
Exhibit 2
forms as an appropriate prediction
Input, Output, and Applicable Audit Procedures for Deep Learning–Based

algorithm in this case because, by intro-

Speech Analysis

ducing the extracted attributes, the num-

ber of predictors is much larger than
Examples

what a traditional machine-learning algo-

rithm could process. For each assertion,
the output of the model could be the pre-
Input Data Interviews, conference calls, phone calls,
project meetings, presentations

dicted risk level or suggested follow-up

Output Features Risk of deception, sentiment, emotion, entity, topic, concept, tests, depending upon the nature and the
label of the training data.
keyword, speaker, suggested follow-up actions

Applicable Audit Procedures Inquiry, inspection

Auditors can leverage deep learning

Implementation

technologies without being experts by

using cloud-based services. But to
only pre-designed questions and record ics are capable of recognizing human improve the predictive performance of
the responses (Matthew D. Pickard, Ryan faces, detecting objects, and identifying deep learning algorithms for audit automa-
Schuetzler, Joseph Valacich, and David concepts and types of scenes on a nearly tion and judgment support, it is important
A. Wood, “Next Generation Accounting real-time basis, and the video processing that auditors and machine learning spe-
Interviewing: A Comparison of Human speed is extraordinarily high; Clarifai, for cialists make a cooperative effort to devel-
and Embodied Conversational Agents as instance, offers deep learning software op auditing-specific training datasets (e.g.,
Interviewers,” working paper, 2017). that can analyze a 3.5-minute video clip the text dataset of 10-Ks) to establish deep
Once integrated with deep learning, within 10 seconds. The internal control learning models that are designed specif-
ECAs may be able to read verbal signs observation procedure could eventually ically for audit tasks. It is worth noting
of deception and ask follow-up questions be automated using this technology to that retrieving the auditing-specific data is
based on the interviewee’s responses. analyze videos recorded by drones in easier today than it was several years ago.
Similarly, other audio documents such offices or at large worksites. Exhibit 3 Various data services, such as SeekiNF
as conference calls, phone calls, and pro- shows the input video/image data, output (http://www.seekedgar.com) allow users
ject meetings could be automatically pro- features, and audit procedures to which to search for financial and nonfinancial
cessed by deep learning technology. the image and video analysis function of information in databases containing mil-
Exhibit 2 depicts possible input and out- deep learning could apply. lions of records in seconds.
put data, as well as applicable areas for Judgment support. In addition to con- The data analytical process must be well
deep learning–based speech recognition. ducting repetitive and mechanical tasks, planned to balance human resources and
Image and video parsing. Certain rou- deep learning provides a new way to technology as well as to ensure that audi-
tine, manual audit procedures can now support audit judgement and improve tors strengthen their professional skills and
be automated with the visual recognition audit quality. Financial statements can be judgment by concentrating on high-risk
function of deep learning. For example, scanned and the financial statement items items identified through deep learning. In
deep learning algorithms can identify the automatically linked to related supporting addition, regulators should start considering
content (e.g., model, quantity, condition evidence such as video clips, press guidance for the use of data analytics tech-
of inventory) of an image from a video release, news, tweets, and interviews nology. In fact, the Data Analytics
filmed by a drone in a company’s ware- (Raphael 2015), as well as corresponding Working Group of the IAASB released a
house. Deep learning systems are able to data attributes (e.g., the overall sentiment Request for Input, “Exploring the Growing
extract a series of predefined numerical and the topic of a news) that have been Use of Technology in the Audit,” in
attributes describing the content of the extracted by deep learning models. For September 2016, and is seeking advice on
video, attach searchable tags accordingly, example, auditors can select data determining whether new or revised inter-
and save both the attributes and the attributes in order to predict fraud risk; national standards or guidance are neces-
images to the auditor’s data warehouse. the selected attributes are then combined sary. Furthermore, the PCAOB has set up
Furthermore, deep learning video analyt- with traditional financial or nonfinancial a research project (Changes in the Use of

28 JUNE 2017 / THE CPA JOURNAL

01-06-0107 Infocus_Sun.qxp_Layout 1 6/2/17 2:23 PM Page 29

2017, http://bit.ly/2rtLtCD).
How can smaller firms and sole prac-
Exhibit 3
titioners acquire this now very experimen-
Input, Output, and Applicable Audit Procedures for Deep Learning–Based Image

tal and expensive technology? An entire

and Video Analysis

industry is already evolving, with pay-by-

Examples
usage options available. CPAs, however,
will need to beef up their competencies in
Input Data Inventory counting and other control activity; interviews; statistics and IT. q
video taken in office, warehouse, or store

Ting Sun is a PhD student in the depart-

Output Features Object, human face, concept and type of scene
ment of accounting and information sys-
tems at Rutgers Business School (Rutgers
University), Newark, N.J. Miklos A.
Vasarhelyi, PhD, is the KPMG
Applicable Audit Procedures Observation, inquiry, inspection

Distinguished Professor of Accounting

Information Systems and Director of
Rutgers Accounting Research Center and
Data and Technology in the Conduct of for disruptive change in auditing and the Continuous Auditing & Reporting Lab
Audits, http://bit.ly/2qPdpOF) on analytics need for analytics (“Technology and the in the department of accounting and
in audits, and Steven Harris, a PCAOB Audit of Today and Tomorrow,” speech information systems at Rutgers Business
board member, issued a passionate plea at PCAOB/AAA annual meeting, Apr. 20, School, Newark, N.J.

JUNE 2017 / THE CPA JOURNAL 29

Reproduced with permission of copyright owner. Further reproduction prohibited
without permission.