Experience Seamless Full Ebook Downloads for Every Genre at textbookfull.

com

Machine Learning in Finance: From Theory to

Practice Matthew F. Dixon

https://textbookfull.com/product/machine-learning-in-
finance-from-theory-to-practice-matthew-f-dixon/

OR CLICK BUTTON

DOWNLOAD NOW

Explore and download more ebook at https://textbookfull.com

 Recommended digital products (PDF, EPUB, MOBI) that
you can download immediately if you are interested.

Extramural English in Teaching and Learning: From Theory

and Research to Practice 1st Edition Pia Sundqvist

https://textbookfull.com/product/extramural-english-in-teaching-and-
learning-from-theory-and-research-to-practice-1st-edition-pia-
sundqvist/
textboxfull.com

Programming Machine Learning From Coding to Deep Learning

1st Edition Paolo Perrotta

https://textbookfull.com/product/programming-machine-learning-from-
coding-to-deep-learning-1st-edition-paolo-perrotta/

textboxfull.com

Science Education in Theory and Practice An Introductory

Guide to Learning Theory Ben Akpan

https://textbookfull.com/product/science-education-in-theory-and-
practice-an-introductory-guide-to-learning-theory-ben-akpan/

textboxfull.com

Fundamentals of optimization theory with applications to

machine learning Gallier J.

https://textbookfull.com/product/fundamentals-of-optimization-theory-
with-applications-to-machine-learning-gallier-j/

textboxfull.com
Fundamentals of optimization theory with applications to
machine learning Gallier J.

https://textbookfull.com/product/fundamentals-of-optimization-theory-
with-applications-to-machine-learning-gallier-j-2/

textboxfull.com

Financial Management: Theory & Practice Eugene F. Brigham

https://textbookfull.com/product/financial-management-theory-practice-
eugene-f-brigham/

textboxfull.com

Corporate Finance theory and practice Fourth Edition

Dallocchio

https://textbookfull.com/product/corporate-finance-theory-and-
practice-fourth-edition-dallocchio/

textboxfull.com

Machine Learning for Economics and Finance in TensorFlow

2: Deep Learning Models for Research and Industry Isaiah
Hull
https://textbookfull.com/product/machine-learning-for-economics-and-
finance-in-tensorflow-2-deep-learning-models-for-research-and-
industry-isaiah-hull/
textboxfull.com

C from Theory to Practice George S. Tselikis

https://textbookfull.com/product/c-from-theory-to-practice-george-s-
tselikis/

textboxfull.com
Matthew F. Dixon
Igor Halperin
Paul Bilokon

Machine
Learning in
Finance
From Theory to Practice
Machine Learning in Finance
Matthew F. Dixon • Igor Halperin • Paul Bilokon

Machine Learning in Finance

From Theory to Practice
Matthew F. Dixon Igor Halperin
Department of Applied Mathematics Tandon School of Engineering
Illinois Institute of Technology New York University
Chicago, IL, USA Brooklyn, NY, USA

Paul Bilokon
Department of Mathematics
Imperial College London
London, UK

Additional material to this book can be downloaded from http://mypages.iit.edu/~mdixon7/

book/ML_Finance_Codes-Book.zip

ISBN 978-3-030-41067-4 ISBN 978-3-030-41068-1 (eBook)

https://doi.org/10.1007/978-3-030-41068-1

© Springer Nature Switzerland AG 2020

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of
the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology
now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this book
are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or
the editors give a warranty, expressed or implied, with respect to the material contained herein or for any
errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional
claims in published maps and institutional affiliations.

This Springer imprint is published by the registered company Springer Nature Switzerland AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
Once you eliminate the impossible, whatever
remains, no matter how improbable, must be
the truth.
—Arthur Conan Doyle
Introduction

Machine learning in finance sits at the intersection of a number of emergent

and established disciplines including pattern recognition, financial econometrics,
statistical computing, probabilistic programming, and dynamic programming. With
the trend towards increasing computational resources and larger datasets, machine
learning has grown into a central computational engineering field, with an emphasis
placed on plug-and-play algorithms made available through open-source machine
learning toolkits. Algorithm focused areas of finance, such as algorithmic trading
have been the primary adopters of this technology. But outside of engineering-based
research groups and business activities, much of the field remains a mystery.
A key barrier to understanding machine learning for non-engineering students
and practitioners is the absence of the well-established theories and concepts that
financial time series analysis equips us with. These serve as the basis for the
development of financial modeling intuition and scientific reasoning. Moreover,
machine learning is heavily entrenched in engineering ontology, which makes devel-
opments in the field somewhat intellectually inaccessible for students, academics,
and finance practitioners from the quantitative disciplines such as mathematics,
statistics, physics, and economics. Consequently, there is a great deal of miscon-
ception and limited understanding of the capacity of this field. While machine
learning techniques are often effective, they remain poorly understood and are
often mathematically indefensible. How do we place key concepts in the field of
machine learning in the context of more foundational theory in time series analysis,
econometrics, and mathematical statistics? Under which simplifying conditions are
advanced machine learning techniques such as deep neural networks mathematically
equivalent to well-known statistical models such as linear regression? How should
we reason about the perceived benefits of using advanced machine learning methods
over more traditional econometrics methods, for different financial applications?
What theory supports the application of machine learning to problems in financial
modeling? How does reinforcement learning provide a model-free approach to
the Black–Scholes–Merton model for derivative pricing? How does Q-learning
generalize discrete-time stochastic control problems in finance?

vii
viii Introduction

This book is written for advanced graduate students and academics in financial
econometrics, management science, and applied statistics, in addition to quants and
data scientists in the field of quantitative finance. We present machine learning
as a non-linear extension of various topics in quantitative economics such as
financial econometrics and dynamic programming, with an emphasis on novel
algorithmic representations of data, regularization, and techniques for controlling
the bias-variance tradeoff leading to improved out-of-sample forecasting. The book
is presented in three parts, each part covering theory and applications. The first
part presents supervised learning for cross-sectional data from both a Bayesian
and frequentist perspective. The more advanced material places a firm emphasis
on neural networks, including deep learning, as well as Gaussian processes, with
examples in investment management and derivatives. The second part covers
supervised learning for time series data, arguably the most common data type
used in finance with examples in trading, stochastic volatility, and fixed income
modeling. Finally, the third part covers reinforcement learning and its applications
in trading, investment, and wealth management. We provide Python code examples
to support the readers’ understanding of the methodologies and applications. As
a bridge to research in this emergent field, we present the frontiers of machine
learning in finance from a researcher’s perspective, highlighting how many well-
known concepts in statistical physics are likely to emerge as research topics for
machine learning in finance.

Prerequisites

This book is targeted at graduate students in data science, mathematical finance,

financial engineering, and operations research seeking a career in quantitative
finance, data science, analytics, and fintech. Students are expected to have com-
pleted upper section undergraduate courses in linear algebra, multivariate calculus,
advanced probability theory and stochastic processes, statistics for time series
(econometrics), and gained some basic introduction to numerical optimization and
computational mathematics. Students shall find the later chapters of this book,
on reinforcement learning, more accessible with some background in investment
science. Students should also have prior experience with Python programming and,
ideally, taken a course in computational finance and introductory machine learning.
The material in this book is more mathematical and less engineering focused than
most courses on machine learning, and for this reason we recommend reviewing
the recent book, Linear Algebra and Learning from Data by Gilbert Strang as
background reading.
Introduction ix

Advantages of the Book

Readers will find this book useful as a bridge from well-established foundational
topics in financial econometrics to applications of machine learning in finance.
Statistical machine learning is presented as a non-parametric extension of financial
econometrics and quantitative finance, with an emphasis on novel algorithmic rep-
resentations of data, regularization, and model averaging to improve out-of-sample
forecasting. The key distinguishing feature from classical financial econometrics
and dynamic programming is the absence of an assumption on the data generation
process. This has important implications for modeling and performance assessment
which are emphasized with examples throughout the book. Some of the main
contributions of the book are as follows:
• The textbook market is saturated with excellent books on machine learning.
However, few present the topic from the prospective of financial econometrics
and cast fundamental concepts in machine learning into canonical modeling and
decision frameworks already well established in finance such as financial time
series analysis, investment science, and financial risk management. Only through
the integration of these disciplines can we develop an intuition into how machine
learning theory informs the practice of financial modeling.
• Machine learning is entrenched in engineering ontology, which makes develop-
ments in the field somewhat intellectually inaccessible for students, academics,
and finance practitioners from quantitative disciplines such as mathematics,
statistics, physics, and economics. Moreover, financial econometrics has not kept
pace with this transformative field, and there is a need to reconcile various
modeling concepts between these disciplines. This textbook is built around
powerful mathematical ideas that shall serve as the basis for a graduate course for
students with prior training in probability and advanced statistics, linear algebra,
times series analysis, and Python programming.
• This book provides financial market motivated and compact theoretical treatment
of financial modeling with machine learning for the benefit of regulators, wealth
managers, federal research agencies, and professionals in other heavily regulated
business functions in finance who seek a more theoretical exposition to allay
concerns about the “black-box” nature of machine learning.
• Reinforcement learning is presented as a model-free framework for stochastic
control problems in finance, covering portfolio optimization, derivative pricing,
and wealth management applications without assuming a data generation
process. We also provide a model-free approach to problems in market
microstructure, such as optimal execution, with Q-learning. Furthermore,
our book is the first to present on methods of inverse reinforcement
learning.
• Multiple-choice questions, numerical examples, and more than 80 end-of-
chapter exercises are used throughout the book to reinforce key technical
concepts.
x Introduction

• This book provides Python codes demonstrating the application of machine

learning to algorithmic trading and financial modeling in risk management
and equity research. These codes make use of powerful open-source software
toolkits such as Google’s TensorFlow and Pandas, a data processing environment
for Python.

Overview of the Book

Chapter 1

Chapter 1 provides the industry context for machine learning in finance, discussing
the critical events that have shaped the finance industry’s need for machine learning
and the unique barriers to adoption. The finance industry has adopted machine
learning to varying degrees of sophistication. How it has been adopted is heavily
fragmented by the academic disciplines underpinning the applications. We view
some key mathematical examples that demonstrate the nature of machine learning
and how it is used in practice, with the focus on building intuition for more technical
expositions in later chapters. In particular, we begin to address many finance
practitioner’s concerns that neural networks are a “black-box” by showing how they
are related to existing well-established techniques such as linear regression, logistic
regression, and autoregressive time series models. Such arguments are developed
further in later chapters.

Chapter 2

Chapter 2 introduces probabilistic modeling and reviews foundational concepts

in Bayesian econometrics such as Bayesian inference, model selection, online
learning, and Bayesian model averaging. We develop more versatile representations
of complex data with probabilistic graphical models such as mixture models.

Chapter 3

Chapter 3 introduces Bayesian regression and shows how it extends many of

the concepts in the previous chapter. We develop kernel-based machine learning
methods—specifically Gaussian process regression, an important class of Bayesian
machine learning methods—and demonstrate their application to “surrogate” mod-
els of derivative prices. This chapter also provides a natural point from which to
Introduction xi

develop intuition for the role and functional form of regularization in a frequentist
setting—the subject of subsequent chapters.

Chapter 4

Chapter 4 provides a more in-depth description of supervised learning, deep

learning, and neural networks—presenting the foundational mathematical and sta-
tistical learning concepts and explaining how they relate to real-world examples in
trading, risk management, and investment management. These applications present
challenges for forecasting and model design and are presented as a reoccurring
theme throughout the book. This chapter moves towards a more engineering
style exposition of neural networks, applying concepts in the previous chapters to
elucidate various model design choices.

Chapter 5

Chapter 5 presents a method for interpreting neural networks which imposes mini-
mal restrictions on the neural network design. The chapter demonstrates techniques
for interpreting a feedforward network, including how to rank the importance of
the features. In particular, an example demonstrating how to apply interpretability
analysis to deep learning models for factor modeling is also presented.

Chapter 6

Chapter 6 provides an overview of the most important modeling concepts in

financial econometrics. Such methods form the conceptual basis and performance
baseline for more advanced neural network architectures presented in the next
chapter. In fact, each type of architecture is a generalization of many of the models
presented here. This chapter is especially useful for students from an engineering or
science background, with little exposure to econometrics and time series analysis.

Chapter 7

Chapter 7 presents a powerful class of probabilistic models for financial data.

Many of these models overcome some of the severe stationarity limitations of the
frequentist models in the previous chapters. The fitting procedure demonstrated is
also different—the use of Kalman filtering algorithms for state-space models rather
xii Introduction

than maximum likelihood estimation or Bayesian inference. Simple examples of

hidden Markov models and particle filters in finance and various algorithms are
presented.

Chapter 8

Chapter 8 presents various neural network models for financial time series analysis,
providing examples of how they relate to well-known techniques in financial econo-
metrics. Recurrent neural networks (RNNs) are presented as non-linear time series
models and generalize classical linear time series models such as AR(p). They
provide a powerful approach for prediction in financial time series and generalize
to non-stationary data. The chapter also presents convolution neural networks for
filtering time series data and exploiting different scales in the data. Finally, this
chapter demonstrates how autoencoders are used to compress information and
generalize principal component analysis.

Chapter 9

Chapter 9 introduces Markov decision processes and the classical methods of

dynamic programming, before building familiarity with the ideas of reinforcement
learning and other approximate methods for solving MDPs. After describing Bell-
man optimality and iterative value and policy updates before moving to Q-learning,
the chapter quickly advances towards a more engineering style exposition of the
topic, covering key computational concepts such as greediness, batch learning, and
Q-learning. Through a number of mini-case studies, the chapter provides insight
into how RL is applied to optimization problems in asset management and trading.
These examples are each supported with Python notebooks.

Chapter 10

Chapter 10 considers real-world applications of reinforcement learning in finance,

as well as further advances the theory presented in the previous chapter. We start
with one of the most common problems of quantitative finance, which is the problem
of optimal portfolio trading in discrete time. Many practical problems of trading or
risk management amount to different forms of dynamic portfolio optimization, with
different optimization criteria, portfolio composition, and constraints. The chapter
introduces a reinforcement learning approach to option pricing that generalizes the
classical Black–Scholes model to a data-driven approach using Q-learning. It then
presents a probabilistic extension of Q-learning called G-learning and shows how it
Introduction xiii

can be used for dynamic portfolio optimization. For certain specifications of reward
functions, G-learning is semi-analytically tractable and amounts to a probabilistic
version of linear quadratic regulators (LQRs). Detailed analyses of such cases are
presented and we show their solutions with examples from problems of dynamic
portfolio optimization and wealth management.

Chapter 11

Chapter 11 provides an overview of the most popular methods of inverse reinforce-

ment learning (IRL) and imitation learning (IL). These methods solve the problem
of optimal control in a data-driven way, similarly to reinforcement learning, however
with the critical difference that now rewards are not observed. The problem is rather
to learn the reward function from the observed behavior of an agent. As behavioral
data without rewards are widely available, the problem of learning from such data
is certainly very interesting. The chapter provides a moderate-level description of
the most promising IRL methods, equips the reader with sufficient knowledge to
understand and follow the current literature on IRL, and presents examples that use
simple simulated environments to see how these methods perform when we know
the “ground truth" rewards. We then present use cases for IRL in quantitative finance
that include applications to trading strategy identification, sentiment-based trading,
option pricing, inference of portfolio investors, and market modeling.

Chapter 12

Chapter 12 takes us forward to emerging research topics in quantitative finance

and machine learning. Among many interesting emerging topics, we focus here
on two broad themes. The first one deals with unification of supervised learning
and reinforcement learning as two tasks of perception-action cycles of agents. We
outline some recent research ideas in the literature including in particular informa-
tion theory-based versions of reinforcement learning and discuss their relevance for
financial applications. We explain why these ideas might have interesting practical
implications for RL financial models, where feature selection could be done within
the general task of optimization of a long-term objective, rather than outside of it,
as is usually performed in “alpha-research.”
The second topic presented in this chapter deals with using methods of reinforce-
ment learning to construct models of market dynamics. We also introduce some
advanced physics-based approaches for computations for such RL-inspired market
models.
xiv Introduction

Source Code

Many of the chapters are accompanied by Python notebooks to illustrate some

of the main concepts and demonstrate application of machine learning methods.
Each notebook is lightly annotated. Many of these notebooks use TensorFlow.
We recommend loading these notebooks, together with any accompanying Python
source files and data, in Google Colab. Please see the appendices of each chapter
accompanied by notebooks, and the README.md in the subfolder of each chapter,
for further instructions and details.

Scope

We recognize that the field of machine learning is developing rapidly and to keep
abreast of the research in this field is a challenging pursuit. Machine learning is an
umbrella term for a number of methodology classes, including supervised learning,
unsupervised learning, and reinforcement learning. This book focuses on supervised
learning and reinforcement learning because these are the areas with the most
overlap with econometrics, predictive modeling, and optimal control in finance.
Supervised machine learning can be categorized as generative and discriminative.
Our focus is on discriminative learners which attempt to partition the input
space, either directly, through affine transformations or through projections onto
a manifold. Neural networks have been shown to provide a universal approximation
to a wide class of functions. Moreover, they can be shown to reduce to other well-
known statistical techniques and are adaptable to time series data.
Extending time series models, a number of chapters in this book are devoted to
an introduction to reinforcement learning (RL) and inverse reinforcement learning
(IRL) that deal with problems of optimal control of such time series and show how
many classical financial problems such as portfolio optimization, option pricing, and
wealth management can naturally be posed as problems for RL and IRL. We present
simple RL methods that can be applied for these problems, as well as explain how
neural networks can be used in these applications.
There are already several excellent textbooks covering other classical machine
learning methods, and we instead choose to focus on how to cast machine learning
into various financial modeling and decision frameworks. We emphasize that much
of this material is not unique to neural networks, but comparisons of alternative
supervised learning approaches, such as random forests, are beyond the scope of
this book.
Introduction xv

Multiple-Choice Questions

Multiple-choice questions are included after introducing a key concept. The correct
answers to all questions are provided at the end of each chapter with selected, partial,
explanations to some of the more challenging material.

Exercises

The exercises that appear at the end of every chapter form an important component
of the book. Each exercise has been chosen to reinforce concepts explained in the
text, to stimulate the application of machine learning in finance, and to gently bridge
material in other chapters. It is graded according to difficulty ranging from (*),
which denotes a simple exercise which might take a few minutes to complete,
through to (***), which denotes a significantly more complex exercise. Unless
specified otherwise, all equations referenced in each exercise correspond to those
in the corresponding chapter.

Instructor Materials

The book is supplemented by a separate Instructor’s Manual which provides worked

solutions to the end of chapter questions. Full explanations for the solutions to the
multiple-choice questions are also provided. The manual provides additional notes
and example code solutions for some of the programming exercises in the later
chapters.

Acknowledgements

This book is dedicated to the late Mark Davis (Imperial College) who was an
inspiration in the field of mathematical finance and engineering, and formative in
our careers. Peter Carr, Chair of the Department of Financial Engineering at NYU
Tandon, has been instrumental in supporting the growth of the field of machine
learning in finance. Through providing speaker engagements and machine learning
instructorship positions in the MS in Algorithmic Finance Program, the authors have
been able to write research papers and identify the key areas required by a text
book. Miquel Alonso (AIFI), Agostino Capponi (Columbia), Rama Cont (Oxford),
Kay Giesecke (Stanford), Ali Hirsa (Columbia), Sebastian Jaimungal (University
of Toronto), Gary Kazantsev (Bloomberg), Morton Lane (UIUC), Jörg Osterrieder
(ZHAW) have established various academic and joint academic-industry workshops
xvi Introduction

and community meetings to proliferate the field and serve as input for this book.
At the same time, there has been growing support for the development of a book
in London, where several SIAM/LMS workshops and practitioner special interest
groups, such as the Thalesians, have identified a number of compelling financial
applications. The material has grown from courses and invited lectures at NYU,
UIUC, Illinois Tech, Imperial College and the 2019 Bootcamp on Machine Learning
in Finance at the Fields Institute, Toronto.
Along the way, we have been fortunate to receive the support of Tomasz Bielecki
(Illinois Tech), Igor Cialenco (Illinois Tech), Ali Hirsa (Columbia University),
and Brian Peterson (DV Trading). Special thanks to research collaborators and
colleagues Kay Giesecke (Stanford University), Diego Klabjan (NWU), Nick
Polson (Chicago Booth), and Harvey Stein (Bloomberg), all of whom have shaped
our understanding of the emerging field of machine learning in finance and the many
practical challenges. We are indebted to Sri Krishnamurthy (QuantUniversity),
Saeed Amen (Cuemacro), Tyler Ward (Google), and Nicole Königstein for their
valuable input on this book. We acknowledge the support of a number of Illinois
Tech graduate students who have contributed to the source code examples and
exercises: Xiwen Jing, Bo Wang, and Siliang Xong. Special thanks to Swaminathan
Sethuraman for his support of the code development, to Volod Chernat and George
Gvishiani who provided support and code development for the course taught at
NYU and Coursera. Finally, we would like to thank the students and especially the
organisers of the MSc Finance and Mathematics course at Imperial College, where
many of the ideas presented in this book have been tested: Damiano Brigo, Antoine
(Jack) Jacquier, Mikko Pakkanen, and Rula Murtada. We would also like to thank
Blanka Horvath for many useful suggestions.

Chicago, IL, USA Matthew F. Dixon

Brooklyn, NY, USA Igor Halperin
London, UK Paul Bilokon
December 2019
Contents

Part I Machine Learning with Cross-Sectional Data

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1 Big Data—Big Compute in Finance . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2 Fintech . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2 Machine Learning and Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.1 Entropy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.2 Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3 Statistical Modeling vs. Machine Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.1 Modeling Paradigms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.2 Financial Econometrics and Machine Learning . . . . . . . . . . . . . . . 18
3.3 Over-fitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
4 Reinforcement Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
5 Examples of Supervised Machine Learning in Practice . . . . . . . . . . . . . . 28
5.1 Algorithmic Trading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
5.2 High-Frequency Trade Execution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
5.3 Mortgage Modeling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
2 Probabilistic Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
2 Bayesian vs. Frequentist Estimation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3 Frequentist Inference from Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
4 Assessing the Quality of Our Estimator: Bias and Variance . . . . . . . . . 53
5 The Bias–Variance Tradeoff (Dilemma) for Estimators . . . . . . . . . . . . . . 55
6 Bayesian Inference from Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
6.1 A More Informative Prior: The Beta Distribution . . . . . . . . . . . . . 60
6.2 Sequential Bayesian updates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

xvii
xviii Contents

6.3
Practical Implications of Choosing a Classical
or Bayesian Estimation Framework. . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
7 Model Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
7.1 Bayesian Inference. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
7.2 Model Selection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
7.3 Model Selection When There Are Many Models . . . . . . . . . . . . . 66
7.4 Occam’s Razor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
7.5 Model Averaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
8 Probabilistic Graphical Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
8.1 Mixture Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
10 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
3 Bayesian Regression and Gaussian Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
2 Bayesian Inference with Linear Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
2.1 Maximum Likelihood Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
2.2 Bayesian Prediction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
2.3 Schur Identity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
3 Gaussian Process Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
3.1 Gaussian Processes in Finance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
3.2 Gaussian Processes Regression and Prediction . . . . . . . . . . . . . . . 93
3.3 Hyperparameter Tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
3.4 Computational Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
4 Massively Scalable Gaussian Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
4.1 Structured Kernel Interpolation (SKI) . . . . . . . . . . . . . . . . . . . . . . . . . 97
4.2 Kernel Approximations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
5 Example: Pricing and Greeking with Single-GPs. . . . . . . . . . . . . . . . . . . . . 98
5.1 Greeking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
5.2 Mesh-Free GPs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
5.3 Massively Scalable GPs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
6 Multi-response Gaussian Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
6.1 Multi-Output Gaussian Process Regression
and Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
8.1 Programming Related Questions* . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
4 Feedforward Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
2 Feedforward Architectures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
2.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
2.2 Geometric Interpretation of Feedforward Networks . . . . . . . . . . 114
2.3 Probabilistic Reasoning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
Contents xix

2.4 Function Approximation with Deep Learning* . . . . . . . . . . . . . . . 119

2.5 VC Dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
2.6 When Is a Neural Network a Spline?* . . . . . . . . . . . . . . . . . . . . . . . . . 124
2.7 Why Deep Networks? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
3 Convexity and Inequality Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
3.1 Similarity of MLPs with Other Supervised Learners . . . . . . . . . 138
4 Training, Validation, and Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
5 Stochastic Gradient Descent (SGD) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
5.1 Back-Propagation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
5.2 Momentum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
6 Bayesian Neural Networks* . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
8.1 Programming Related Questions* . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
5 Interpretability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
2 Background on Interpretability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
2.1 Sensitivities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
3 Explanatory Power of Neural Networks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
3.1 Multiple Hidden Layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
3.2 Example: Step Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
4 Interaction Effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
4.1 Example: Friedman Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
5 Bounds on the Variance of the Jacobian. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
5.1 Chernoff Bounds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
5.2 Simulated Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
6 Factor Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
6.1 Non-linear Factor Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
6.2 Fundamental Factor Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
8.1 Programming Related Questions* . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188

Part II Sequential Learning

6 Sequence Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
2 Autoregressive Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
2.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
2.2 Autoregressive Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
2.3 Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
2.4 Stationarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
2.5 Partial Autocorrelations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
xx Contents

2.6 Maximum Likelihood Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199

2.7 Heteroscedasticity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200
2.8 Moving Average Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
2.9 GARCH . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
2.10 Exponential Smoothing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204
3 Fitting Time Series Models: The Box–Jenkins Approach . . . . . . . . . . . . 205
3.1 Stationarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
3.2 Transformation to Ensure Stationarity . . . . . . . . . . . . . . . . . . . . . . . . . 206
3.3 Identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
3.4 Model Diagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
4 Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
4.1 Predicting Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
4.2 Time Series Cross-Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
5 Principal Component Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
5.1 Principal Component Projection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
5.2 Dimensionality Reduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216
6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
7 Probabilistic Sequence Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
2 Hidden Markov Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
2.1 The Viterbi Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224
2.2 State-Space Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
3 Particle Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
3.1 Sequential Importance Resampling (SIR) . . . . . . . . . . . . . . . . . . . . . 228
3.2 Multinomial Resampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
3.3 Application: Stochastic Volatility Models . . . . . . . . . . . . . . . . . . . . . 230
4 Point Calibration of Stochastic Filters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
5 Bayesian Calibration of Stochastic Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
8 Advanced Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239
2 Recurrent Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240
2.1 RNN Memory: Partial Autocovariance . . . . . . . . . . . . . . . . . . . . . . . . 244
2.2 Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245
2.3 Stationarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246
2.4 Generalized Recurrent Neural Networks (GRNNs) . . . . . . . . . . . 248
3 Gated Recurrent Units. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
3.1 α-RNNs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
3.2 Neural Network Exponential Smoothing . . . . . . . . . . . . . . . . . . . . . . 251
3.3 Long Short-Term Memory (LSTM) . . . . . . . . . . . . . . . . . . . . . . . . . . . 254
Contents xxi

4 Python Notebook Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255

4.1 Bitcoin Prediction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256
4.2 Predicting from the Limit Order Book. . . . . . . . . . . . . . . . . . . . . . . . . 256
5 Convolutional Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
5.1 Weighted Moving Average Smoothers . . . . . . . . . . . . . . . . . . . . . . . . 258
5.2 2D Convolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
5.3 Pooling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263
5.4 Dilated Convolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264
5.5 Python Notebooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265
6 Autoencoders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266
6.1 Linear Autoencoders. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
6.2 Equivalence of Linear Autoencoders and PCA . . . . . . . . . . . . . . . 268
6.3 Deep Autoencoders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270
7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272
8.1 Programming Related Questions* . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275

Part III Sequential Data with Decision-Making

9 Introduction to Reinforcement Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
2 Elements of Reinforcement Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284
2.1 Rewards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284
2.2 Value and Policy Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286
2.3 Observable Versus Partially Observable Environments . . . . . . . 286
3 Markov Decision Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
3.1 Decision Policies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
3.2 Value Functions and Bellman Equations . . . . . . . . . . . . . . . . . . . . . . 293
3.3 Optimal Policy and Bellman Optimality. . . . . . . . . . . . . . . . . . . . . . . 296
4 Dynamic Programming Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299
4.1 Policy Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300
4.2 Policy Iteration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302
4.3 Value Iteration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303
5 Reinforcement Learning Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306
5.1 Monte Carlo Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307
5.2 Policy-Based Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309
5.3 Temporal Difference Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
5.4 SARSA and Q-Learning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313
5.5 Stochastic Approximations and Batch-Mode Q-learning . . . . . 316
5.6 Q-learning in a Continuous Space: Function
Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323
5.7 Batch-Mode Q-Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327
5.8 Least Squares Policy Iteration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331
5.9 Deep Reinforcement Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335
xxii Contents

6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337
7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345
10 Applications of Reinforcement Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347
2 The QLBS Model for Option Pricing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349
3 Discrete-Time Black–Scholes–Merton Model . . . . . . . . . . . . . . . . . . . . . . . . 352
3.1 Hedge Portfolio Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352
3.2 Optimal Hedging Strategy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354
3.3 Option Pricing in Discrete Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356
3.4 Hedging and Pricing in the BS Limit . . . . . . . . . . . . . . . . . . . . . . . . . . 359
4 The QLBS Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360
4.1 State Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361
4.2 Bellman Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362
4.3 Optimal Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365
4.4 DP Solution: Monte Carlo Implementation . . . . . . . . . . . . . . . . . . . 368
4.5 RL Solution for QLBS: Fitted Q Iteration . . . . . . . . . . . . . . . . . . . . . 370
4.6 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373
4.7 Option Portfolios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375
4.8 Possible Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379
5 G-Learning for Stock Portfolios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 380
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 380
5.2 Investment Portfolio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381
5.3 Terminal Condition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382
5.4 Asset Returns Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383
5.5 Signal Dynamics and State Space. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383
5.6 One-Period Rewards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384
5.7 Multi-period Portfolio Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . 386
5.8 Stochastic Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386
5.9 Reference Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 388
5.10 Bellman Optimality Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 388
5.11 Entropy-Regularized Bellman Optimality Equation . . . . . . . . . . 389
5.12 G-Function: An Entropy-Regularized Q-Function . . . . . . . . . . . . 391
5.13 G-Learning and F-Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393
5.14 Portfolio Dynamics with Market Impact . . . . . . . . . . . . . . . . . . . . . . 395
5.15 Zero Friction Limit: LQR with Entropy Regularization . . . . . . 396
5.16 Non-zero Market Impact: Non-linear Dynamics . . . . . . . . . . . . . . 400
6 RL for Wealth Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401
6.1 The Merton Consumption Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401
6.2 Portfolio Optimization for a Defined Contribution
Retirement Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405
6.3 G-Learning for Retirement Plan Optimization . . . . . . . . . . . . . . . . 408
6.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413
7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413
Contents xxiii

8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 414
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 416
11 Inverse Reinforcement Learning and Imitation Learning . . . . . . . . . . . . . 419
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419
2 Inverse Reinforcement Learning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423
2.1 RL Versus IRL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425
2.2 What Are the Criteria for Success in IRL? . . . . . . . . . . . . . . . . . . . . 426
2.3 Can a Truly Portable Reward Function Be Learned
with IRL?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427
3 Maximum Entropy Inverse Reinforcement Learning . . . . . . . . . . . . . . . . . 428
3.1 Maximum Entropy Principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 430
3.2 Maximum Causal Entropy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433
3.3 G-Learning and Soft Q-Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 436
3.4 Maximum Entropy IRL. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 438
3.5 Estimating the Partition Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 442
4 Example: MaxEnt IRL for Inference of Customer Preferences . . . . . . 443
4.1 IRL and the Problem of Customer Choice. . . . . . . . . . . . . . . . . . . . . 444
4.2 Customer Utility Function. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445
4.3 Maximum Entropy IRL for Customer Utility . . . . . . . . . . . . . . . . . 446
4.4 How Much Data Is Needed? IRL and Observational
Noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 450
4.5 Counterfactual Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452
4.6 Finite-Sample Properties of MLE Estimators . . . . . . . . . . . . . . . . . 454
4.7 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455
5 Adversarial Imitation Learning and IRL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 457
5.1 Imitation Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 457
5.2 GAIL: Generative Adversarial Imitation Learning. . . . . . . . . . . . 459
5.3 GAIL as an Art of Bypassing RL in IRL . . . . . . . . . . . . . . . . . . . . . . 461
5.4 Practical Regularization in GAIL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 464
5.5 Adversarial Training in GAIL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 466
5.6 Other Adversarial Approaches* . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 468
5.7 f-Divergence Training* . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 468
5.8 Wasserstein GAN*. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 469
5.9 Least Squares GAN* . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 471
6 Beyond GAIL: AIRL, f-MAX, FAIRL, RS-GAIL, etc.* . . . . . . . . . . . . . 471
6.1 AIRL: Adversarial Inverse Reinforcement Learning . . . . . . . . . 472
6.2 Forward KL or Backward KL?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474
6.3 f-MAX. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476
6.4 Forward KL: FAIRL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 477
6.5 Risk-Sensitive GAIL (RS-GAIL) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 479
6.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481
7 Gaussian Process Inverse Reinforcement Learning. . . . . . . . . . . . . . . . . . . 481
7.1 Bayesian IRL. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 482
7.2 Gaussian Process IRL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 483
xxiv Contents

8 Can IRL Surpass the Teacher? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 484

8.1 IRL from Failure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485
8.2 Learning Preferences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 487
8.3 T-REX: Trajectory-Ranked Reward EXtrapolation . . . . . . . . . . . 488
8.4 D-REX: Disturbance-Based Reward EXtrapolation . . . . . . . . . . 490
9 Let Us Try It Out: IRL for Financial Cliff Walking . . . . . . . . . . . . . . . . . . 490
9.1 Max-Causal Entropy IRL. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 491
9.2 IRL from Failure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 492
9.3 T-REX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 493
9.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 494
10 Financial Applications of IRL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 495
10.1 Algorithmic Trading Strategy Identification. . . . . . . . . . . . . . . . . . . 495
10.2 Inverse Reinforcement Learning for Option Pricing . . . . . . . . . . 497
10.3 IRL of a Portfolio Investor with G-Learning . . . . . . . . . . . . . . . . . . 499
10.4 IRL and Reward Learning for Sentiment-Based
Trading Strategies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 504
10.5 IRL and the “Invisible Hand” Inference . . . . . . . . . . . . . . . . . . . . . . . 505
11 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 512
12 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 513
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 515
12 Frontiers of Machine Learning and Finance . . . . . . . . . . . . . . . . . . . . . . . . . . . . 519
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 519
2 Market Dynamics, IRL, and Physics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 521
2.1 “Quantum Equilibrium–Disequilibrium” (QED) Model . . . . . . 522
2.2 The Langevin Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523
2.3 The GBM Model as the Langevin Equation . . . . . . . . . . . . . . . . . . . 524
2.4 The QED Model as the Langevin Equation . . . . . . . . . . . . . . . . . . . 525
2.5 Insights for Financial Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 527
2.6 Insights for Machine Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 528
3 Physics and Machine Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 529
3.1 Hierarchical Representations in Deep Learning
and Physics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 529
3.2 Tensor Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 530
3.3 Bounded-Rational Agents in a Non-equilibrium
Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 534
4 A “Grand Unification” of Machine Learning? . . . . . . . . . . . . . . . . . . . . . . . . 535
4.1 Perception-Action Cycles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 537
4.2 Information Theory Meets Reinforcement Learning. . . . . . . . . . 538
4.3 Reinforcement Learning Meets Supervised Learning:
Predictron, MuZero, and Other New Ideas . . . . . . . . . . . . . . . . . . . . 539
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 540

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 543
About the Authors

Matthew F. Dixon is an Assistant Professor of Applied Math at the Illinois Institute

of Technology. His research in computational methods for finance is funded by
Intel. Matthew began his career in structured credit trading at Lehman Brothers
in London before pursuing academics and consulting for financial institutions in
quantitative trading and risk modeling. He holds a Ph.D. in Applied Mathematics
from Imperial College (2007) and has held postdoctoral and visiting professor
appointments at Stanford University and UC Davis, respectively. He has published
over 20 peer-reviewed publications on machine learning and financial modeling,
has been cited in Bloomberg Markets and the Financial Times as an AI in fintech
expert, and is a frequently invited speaker in Silicon Valley and on Wall Street. He
has published R packages, served as a Google Summer of Code mentor, and is the
co-founder of the Thalesians Ltd.

Igor Halperin is a Research Professor in Financial Engineering at NYU and an

AI Research Associate at Fidelity Investments. He was previously an Executive
Director of Quantitative Research at JPMorgan for nearly 15 years. Igor holds a
Ph.D. in Theoretical Physics from Tel Aviv University (1994). Prior to joining
the financial industry, he held postdoctoral positions in theoretical physics at the
Technion and the University of British Columbia.

Paul Bilokon is CEO and Founder of Thalesians Ltd. and an expert in electronic
and algorithmic trading across multiple asset classes, having helped build such
businesses at Deutsche Bank and Citigroup. Before focusing on electronic trading,
Paul worked on derivatives and has served in quantitative roles at Nomura, Lehman
Brothers, and Morgan Stanley. Paul has been educated at Christ Church College,
Oxford, and Imperial College. Apart from mathematical and computational finance,
his academic interests include machine learning and mathematical logic.

xxv
 Part I
Machine Learning with Cross-Sectional
Data
Chapter 1
Introduction

This chapter introduces the industry context for machine learning in finance, dis-
cussing the critical events that have shaped the finance industry’s need for machine
learning and the unique barriers to adoption. The finance industry has adopted
machine learning to varying degrees of sophistication. How it has been adopted
is heavily fragmented by the academic disciplines underpinning the applications.
We view some key mathematical examples that demonstrate the nature of machine
learning and how it is used in practice, with the focus on building intuition for
more technical expositions in later chapters. In particular, we begin to address
many finance practitioner’s concerns that neural networks are a “black-box” by
showing how they are related to existing well-established techniques such as
linear regression, logistic regression, and autoregressive time series models. Such
arguments are developed further in later chapters. This chapter also introduces
reinforcement learning for finance and is followed by more in-depth case studies
highlighting the design concepts and practical challenges of applying machine
learning in practice.

1 Background

In 1955, John McCarthy, then a young Assistant Professor of Mathematics, at

Dartmouth College in Hanover, New Hampshire, submitted a proposal with Marvin
Minsky, Nathaniel Rochester, and Claude Shannon for the Dartmouth Summer
Research Project on Artificial Intelligence (McCarthy et al. 1955). These organizers
were joined in the summer of 1956 by Trenchard More, Oliver Selfridge, Herbert
Simon, Ray Solomonoff, among others. The stated goal was ambitious:
“The study is to proceed on the basis of the conjecture that every aspect
of learning or any other feature of intelligence can in principle be so precisely
described that a machine can be made to simulate it. An attempt will be made to

© Springer Nature Switzerland AG 2020 3

M. F. Dixon et al., Machine Learning in Finance,
https://doi.org/10.1007/978-3-030-41068-1_1
4 1 Introduction

find how to make machines use language, form abstractions and concepts, solve
kinds of problems now reserved for humans, and improve themselves.” Thus the
field of artificial intelligence, or AI, was born.
Since this time, AI has perpetually strived to outperform humans on various judg-
ment tasks (Pinar Saygin et al. 2000). The most fundamental metric for this success
is the Turing test—a test of a machine’s ability to exhibit intelligent behavior equiv-
alent to, or indistinguishable from, that of a human (Turing 1995). In recent years,
a pattern of success in AI has emerged—one in which machines outperform in the
presence of a large number of decision variables, usually with the best solution being
found through evaluating an exponential number of candidates in a constrained
high-dimensional space. Deep learning models, in particular, have proven remark-
ably successful in a wide field of applications (DeepMind 2016; Kubota 2017;
Esteva et al. 2017) including image processing (Simonyan and Zisserman 2014),
learning in games (DeepMind 2017), neuroscience (Poggio 2016), energy conser-
vation (DeepMind 2016), skin cancer diagnostics (Kubota 2017; Esteva et al. 2017).
One popular account of this reasoning points to humans’ perceived inability
to process large amounts of information and make decisions beyond a few key
variables. But this view, even if fractionally representative of the field, does no
justice to AI or human learning. Humans are not being replaced any time soon.
The median estimate for human intelligence in terms of gigaflops is about 104 times
more than the machine that ran alpha-go. Of course, this figure is caveated on the
important question of whether the human mind is even a Turing machine.

1.1 Big Data—Big Compute in Finance

The growth of machine-readable data to record and communicate activities through-

out the financial system combined with persistent growth in computing power and
storage capacity has significant implications for every corner of financial modeling.
Since the financial crises of 2007–2008, regulatory supervisors have reoriented
towards “data-driven” regulation, a prominent example of which is the collection
and analysis of detailed contractual terms for the bank loan and trading book stress-
testing programs in the USA and Europe, instigated by the crisis (Flood et al. 2016).
“Alternative data”—which refers to data and information outside of the usual
scope of securities pricing, company fundamentals, or macroeconomic indicators—
is playing an increasingly important role for asset managers, traders, and decision
makers. Social media is now ranked as one of the top categories of alternative data
currently used by hedge funds. Trading firms are hiring experts in machine learning
with the ability to apply natural language processing (NLP) to financial news and
other unstructured documents such as earnings announcement reports and SEC 10K
reports. Data vendors such as Bloomberg, Thomson Reuters, and RavenPack are
providing processed news sentiment data tailored for systematic trading models.
1 Background 5

In de Prado (2019), some of the properties of these new, alternative datasets are
explored: (a) many of these datasets are unstructured, non-numerical, and/or non-
categorical, like news articles, voice recordings, or satellite images; (b) they tend
to be high-dimensional (e.g., credit card transactions) and the number of variables
may greatly exceed the number of observations; (c) such datasets are often sparse,
containing NaNs (not-a-numbers); (d) they may implicitly contain information
about networks of agents.
Furthermore, de Prado (2019) explains why classical econometric methods fail
on such datasets. These methods are often based on linear algebra, which fail when
the number of variables exceeds the number of observations. Geometric objects,
such as covariance matrices, fail to recognize the topological relationships that
characterize networks. On the other hand, machine learning techniques offer the
numerical power and functional flexibility needed to identify complex patterns
in a high-dimensional space offering a significant improvement over econometric
methods.
The “black-box” view of ML is dismissed in de Prado (2019) as a misconception.
Recent advances in ML make it applicable to the evaluation of plausibility of
scientific theories; determination of the relative informational variables (usually
referred to as features in ML) for explanatory and/or predictive purposes; causal
inference; and visualization of large, high-dimensional, complex datasets.
Advances in ML remedy the shortcomings of econometric methods in goal
setting, outlier detection, feature extraction, regression, and classification when it
comes to modern, complex alternative datasets. For example, in the presence of p
features there may be up to 2p − p − 1 multiplicative interaction effects. For two
features there is only one such interaction effect, x1 x2 . For three features, there are
x1 x2 , x1 x3 , x2 x3 , x1 x2 x3 . For as few as ten features, there are 1,013 multiplicative
interaction effects. Unlike ML algorithms, econometric models do not “learn”
the structure of the data. The model specification may easily miss some of the
interaction effects. The consequences of missing an interaction effect, e.g. fitting
yt = x1,t + x2,t + t instead of yt = x1,t + x2,t + x1,t x2,t + t , can be dramatic.
A machine learning algorithm, such as a decision tree, will recursively partition
a dataset with complex patterns into subsets with simple patterns, which can then
be fit independently with simple linear specifications. Unlike the classical linear
regression, this algorithm “learns” about the existence of the x1,t x2,t effect, yielding
much better out-of-sample results.
There is a draw towards more empirically driven modeling in asset pricing
research—using ever richer sets of firm characteristics and “factors” to describe and
understand differences in expected returns across assets and model the dynamics
of the aggregate market equity risk premium (Gu et al. 2018). For example,
Harvey et al. (2016) study 316 “factors,” which include firm characteristics and
common factors, for describing stock return behavior. Measurement of an asset’s
risk premium is fundamentally a problem of prediction—the risk premium is the
conditional expectation of a future realized excess return. Methodologies that can
reliably attribute excess returns to tradable anomalies are highly prized. Machine
learning provides a non-linear empirical approach for modeling realized security
6 1 Introduction

returns from firm characteristics. Dixon and Polson (2019) review the formulation
of asset pricing models for measuring asset risk premia and cast neural networks in
canonical asset pricing frameworks.

1.2 Fintech

The rise of data and machine learning has led to a “fintech” industry, covering
digital innovations and technology-enabled business model innovations in the
financial sector (Philippon 2016). Examples of innovations that are central to
fintech today include cryptocurrencies and the blockchain, new digital advisory and
trading systems, peer-to-peer lending, equity crowdfunding, and mobile payment
systems. Behavioral prediction is often a critical aspect of product design and risk
management needed for consumer-facing business models; consumers or economic
agents are presented with well-defined choices but have unknown economic needs
and limitations, and in many cases do not behave in a strictly economically rational
fashion. Therefore it is necessary to treat parts of the system as a black-box that
operates under rules that cannot be known in advance.

1.2.1 Robo-Advisors

Robo-advisors are financial advisors that provide financial advice or portfolio

management services with minimal human intervention. The focus has been on
portfolio management rather than on estate and retirement planning, although there
are exceptions, such as Blooom. Some limit investors to the ETFs selected by the
service, others are more flexible. Examples include Betterment, Wealthfront, Wise-
Banyan, FutureAdvisor (working with Fidelity and TD Ameritrade), Blooom, Motif
Investing, and Personal Capital. The degree of sophistication and the utilization of
machine learning are on the rise among robo-advisors.

1.2.2 Fraud Detection

In 2011 fraud cost the financial industry approximately $80 billion annually
(Consumer Reports, June 2011). According to PwC’s Global Economic Crime
Survey 2016, 46% of respondents in the Financial Services industry reported being
victims of economic crime in the last 24 months—a small increase from 45%
reported in 2014. 16% of those that reported experiencing economic crime had
suffered more than 100 incidents, with 6% suffering more than 1,000. According
to the survey, the top 5 types of economic crime are asset misappropriation (60%,
down from 67% in 2014), cybercrime (49%, up from 39% in 2014), bribery and
corruption (18%, down from 20% in 2014), money laundering (24%, as in 2014),
and accounting fraud (18%, down from 21% in 2014). Detecting economic crimes is
Another Random Scribd Document
with Unrelated Content
Boulouf, 407
Bouncer, 407
Bourgeois, M., var. orig. by, 174
Bower, 407
Bower, var. orig. by, 407
Bower’s Gage (syn. of Bower), 407
Bowle, 407
Brackett, 408
Bradford Gage, Green Gage or Queen Gage (syns. of Reine
Claude), 327
Bradley’s King or King of Damsons (syns. of King Damson), 259
Bradshaw, 166
Bradshaw (syn. of Bradshaw), 166
Bradshaw (syn. of Gueii), 234
Bradshaw (syn. of Nectarine), 291
Brahy, 408
Brahy’s Green Gage (syn. of Brahy), 408
Brainerd, 408
Brainerd’s Best (syn. of Brainerd), 408
Brandon Ruby, 408
Brandy Gage, 408
Brant of Naples, 408
Brauman, 408
Braunauer Aprikosenartige; Braunauer Aprikosenartige
Damascene or Neue Kernfrucht (syns. of Abricotée de
Braunau), 391
Braunauer Damascenenartige Pflaume, 408
Braunauer Königs Pflaume or Violetter Perdrigon (syns. of
Buchner Königspflaume), 410
Breck, 408
Breck, Joseph, var. orig. by, 454
Breece, J. S., var. orig. by, 414, 421, 425, 458, 496, 498, 511,
512, 535, 539, 541, 544
Breitgedrückte Kaiser Zwetsche or Zwetsche (syn. of Quetsche
Aplatie), 523
Bremen Prune (syn. of Austrian Quetsche), 397
Brevoort, Henry, var. orig. by, 408
Brevoort Purple, 408
Brevoort’s Purple, Purple Bolmar or Purple Washington;
Brevorts; Brevort’s Purple; Brevorts Purple Bolmar;
Brevorts’ Purple Bolmar (syns. of Brevoort Purple), 408
Briançon, 409
Bricet (syn. of Late Mirabelle), 263
Bricet (syn. of Saint Catherine), 334
Bricetta or Bricette (syns. of Late Mirabelle), 263
Bricette (syn. of Saint Catherine), 334
Brignole, 409
Brignole (syn. of Brignole Violette), 409
Brignole (syn. of Orleans), 302
Brignole or Brignolle (syns. of White Perdrigon), 375
Brignole Jaune (syn. of Brignole), 409
Brignole Plum (syn. of Brignole), 409
Brignole Violette, 409
Brignole Violette (syn. of Blue Perdrigon), 164
Brignole Violette (syn. of Orleans), 302
Brill, 409
Brisette (syn. of Late Mirabelle, 263; of Saint Catherine, 334)
Bristol, 409
Briton Seedling, 409
Brittlewood, 409
Brittlewood No. 1 (syn. of Brittlewood), 409
Brittlewood No. 2 (syn. of U. S.), 557
Brittlewood No. 3, 410
Britzer Egg, 410
Brock, 410
Brodie, R., var. orig. by, 410
Brodie, 410
Brompton, 410
Brooklyn, 410
Brown, var. orig. by, 461
Brown, John, var. orig. by, 445
Brown-rot, discussion of, 127
Brown’s Crimson Drop (syn. of Crimson Drop), 424.
Bruce, A. L., life of, 527-528;
var. orig. by, 402, 423, 425, 434, 447, 455, 490, 509, 519,
526, 527, 554, 560
Bruce, Philip Alexander, quoted, 84, 85
Brugnole (syn. of Brignole Violette), 409
Brugnole (syn. of Orleans), 302
Brugnon de Neerveteren (syn. of Rademakers), 525
Brugnon Gage (syn. of Reine Claude), 327
Brugnon Green Gage (syn. of Reine Claude), 327
Brünner Zwetsche, 410
Brunswick, 410
Brussels, 410
Brustwarzenpflaume, Die (syn. of Mamelonnée), 489
Bruyn Gage (syn. of Reine Claude), 327
Bryan, 410
Bryanston, 168
Bryanston Gage; Bryanstone; Bryanstone Gage; Bryanston’s
Gage (syns. of Bryanston), 168
Buchanan, 410
Buchner Königspflaume, 410
Buckman, Benjamin, var. orig. by, 476
Budd, 410
Budd, Prof. J. L., life of, 145;
quoted, 145, 269;
var. orig. by, 145, 362, 471
Buel, 411
Buel, Judge, var. orig. by, 256
Buel’s Favorite; Buel’s Liebling’s Zwetsche (syns. of Buel), 411
Buffalo Bill, 411
Buhl-Eltershofen, 411
Buhl-Eltershofen Zwetsche (syn. of Buhl-Eltershofen), 411
Buhler, 411
Buhler’s Early Prune (syn. of Buhler), 411
Bulah No. 4, 411
Bulgaria, 411
Bulgarian, 411
Bullace (syn. of White Bullace), 373
Bullace (syn. of Damson), 186
Bullace group, 40
Bullesse (syn. of Black Bullace), 161
Bullman, 411
Bullock, Jesse, var. orig. by, 175
Bullock, 411
Bullock’s Heart (syn. of Bullock), 411
Bull Plum, 411
Bunker Hill, 411
Bunte Früh Pflaume, 411
Bunte Herzformige Pflaume (syn. of Red Diaper), 323
Bunter Perdrigon, 412
Buntfarbige Violette Pflaume (syn. of Violet Diaper), 365
Buon-Boccone (syn. of Catelano Giallo), 415
Burbank, 169
Burbank First, 412
Burbank, Luther, life of, 170-171;
quoted, 76, 77, 95, 96, 146, 147, 333, 571;
var. orig. by, 141, 142, 146, 147, 154, 170, 171, 179, 222,
225, 237, 257, 275, 285, 296, 297, 333, 337, 343, 354,
377, 393, 402, 416, 420, 421, 439, 446, 447, 450, 452,
 461, 465, 467, 471, 473, 480, 483, 485, 491, 504, 518,
521, 539, 543, 547, 548, 559, 560, 571
Burbank’s First (syn. of Burbank First), 412
Burbank No. 1, 412
Burbank No. 2 (syn. of Abundance), 136
Burbank No. 3 (syn. of Late Blood), 480
Burbank No. 4 (syn. of Heikes), 460
Burbank No. 7, 412
Burbank No. 11, 412
Burbank × Redick, 412
Burbank’s Sultan (syn. of Occident), 295
Burchardt Gelbe Früh Zwetsche, 412
Burettes, 412
Burford, 412
Burgnon Gage (syn. of Reine Claude), 327
Burgunder Zwetsche (syn. of Burgundy Prune), 412
Burgundy Prune, 412
Burgundy Prune (syn. of Burgundy Prune), 412
Burlington Gage, 413
Burnet, 413
Burrettes (syn. of Burettes), 412
Bursoto, 413
Burwood (syn. of Emerald), 442
Bury Seedling (syn. of Golden Drop), 228
Bush; Bush Plum of Kent (syns. of Kent), 476
Byefield, 413
Caddo Chief, 413
Caldwell Golden Drop, 413
Caldwell’s Golden Drop (syn. of Caldwell Golden Drop), 413
Caldwell’s White Gage (syn. of Caldwell White Gage), 413
Caldwell White Gage, 413
Caledonian (syn. of Dove Bank, 435; of Goliath, 231; of
Nectarine, 231; of Peach, 309)
California (syn. of Agen), 138
California, 413
California Seedling; Cal. Seedling (syns. of California), 413
Calvels Pfirschenpflaume (syn. of Peach), 309
Cambell, 414
Cambell’s Seedling (syn. of Cambell), 414
Camp, Charles B., var. orig. by, 478
Campbell, 414
Canada Blue, 414
Canada Egg (syn. of Canada Orleans), 414
Canada Orleans, 414
Canadian Apricot, 414
Canawa (syn. of Kanawha), 474
Candelaria, 414
Candelaria Prune (syn. of Candelaria), 414
Cant’s Late Green Gage (syn. of Reine Claude), 327
Caper, 414
Capitaine Kirchhof, 414
Capitaine Kirckhof or Kirckkof (syns. of Capitaine Kirchhof), 414
Captain, 414
Capt. Bacon, 415
Capt. Watrous, 415
Caradeuc (syn. of De Caradeuc), 188
Caro, 415
Caroline, 415
Carpenter, 415
Carrière, quoted, 336
Carson, 415
Carstesen, H. P., var. orig. by, 415
Carstesen, 415
Cartier, Jacques, quoted, 70
Carver, 415
Castelane; Castellan (syns. of Early Yellow), 203
Catalana Propria, 415
Catalana-Susina (syn. of Catalano), 415
Catalana Toscana (syn. of Catalana Propria), 415
Catalane; Catalonia; Catalonian; Catalonische Kricke or Pflaume;
Catalonischer Spilling (syns. of Early Yellow), 203
Catalano, 415
Catelana Morella (syn. of Catelano Violaceo), 416
Catelane (syn. of Catalano), 415
Catelane Giallo, 415
Catelano Violaceo, 416
Catharinenpflaume (syn. of Saint Catherine), 334
Catharinenpflaume von Schenectady (syn. of Schenectady), 540
Catherine, 416
Catherine (Prune de Ste.) (syn. of Saint Catherine), 334
Catherine violette (syn. of Saint Martin), 336
Cavel’s Pfirschenpflaume (syn. of Nectarine), 291
Cel, 416
Centennial, 416
Centralia, 416
Cerasus nigra, 69
Cerasus umbellata, 78
Cerasus pubescens, 92
Cerise; Cerisette; Cerisette Blanche; Cerizette (syns. of
Myrobalan), 290
Cerisette Blanche (syn. of Gelbe Kirschpflaume, 450; of Early
Yellow, 203)
Cernay Perdrigon (syn. of Perdrigon of Cernay), 514
Cernay Perdrigon (syn. of Cerney Perdrigon), 416
Cerney Perdrigon, 416
Ceur de Beuf, 416
Chabardin, M., var. orig. by, 456
Chabot (syn. of Chabot), 172
Chabot, 172
Chabot Blood, 416
Chalco, 174
Chambourcy, 174
Champion, 175
Champion, 416
Champion Prune (syn. of Champion), 175
Chancellor Gage, 416
Chapin, Aaron, var. orig. by, 416
Chapin, 416
Chapin’s Early (syn. of Howell), 465
Chapman, var. orig. by, 367
Chapman’s Prince of Wales; Chapman’s Prince of Wales’ Plum
(syns. of Wales), 366
Chariot, 417
Charity Clark, 417
Charles Downing (syn. of Downing), 193
Charlotte (syn. of Charlotte), 417
Charlotte, 417
Charmer, 417
Chase (syn. of Abundance), 136
Chautauqua, 417
Chauviere, M., var. orig. by, 417
Chauviere, 417
Cheney, 176
Cheresoto, 417
Cherokee, 417
Cherry, 417
Cherry (syn. of Cheney, 176; of Myrobalan, 290; of Queen
Mother, 522)
Cherry Plum (syn. of Myrobalan), 290
Chester, 417
Chester County Prune, 418
Chestnut, 418
Cheston; Cheston Matchless; Cheston’s Plumb (syns. of Violet
Diaper), 365
Chicasaw Chief (syn. of Miner), 281
Chickasaw (syn. of Scioto), 541
Chicrigland, 418
Chili (syn. of Pond), 314
Chinook, 418
Chippewa, 418
Chippeway (syn. of Chippewa), 418
Choptank, 418
Christian, 418
Christie, 418
Christ’s Damascene (syn. of Musk Damson), 501
Churchill, 419
Chypre, 418
Cinnamon, 419
Cire (syn. of Wax), 562
Ciriselle (syn. of Myrobalan), 290
Cistena, 419
City, 419
Clairac Mammoth (syn. of Imperial Epineuse), 250
Clara, 419
Clarendon, 419
Clark, Thomas, var. orig. by, 522
Clark, 419
Claudia (syn. of Reine Claude), 327
Cleavinger, 419
Cleveland, 419
Clifford, Mrs., var. orig. by, 420
Clifford, 419
Climate, effects of, 101-109
Climax, 178
Climax’s Brother, 420
Clingstone Wolf (syn. of Wolf Clingstone), 567
Cling Stem, 179
Clinton, 420
Cloth of Gold; Cloth of Gold Plum (syn. of Drap d’Or), 194
Cloth of Gold; Cloth of Gold Esperen (syn. of Esperen), 206
Cluck, 420
Cluck, George, var. orig. by, 420
Cluster (syn. of Crittenden), 184
Cluster, 420
Cluster Damson (syn. of Crittenden), 184
Clyman, 180
Clyman, Mrs. Hannah, var. orig. by, 184
Cochet, 420
Cochet Pére; Cochets Pflaume (syns. of Cochet), 420
Coe (syn. of Golden Drop), 228
Coe, Jervaise, var. orig. by, 229.
Coe à Fruit Violet (syn. of Coe Violet), 420
Coe Golden Drop (syn. of Golden Drop), 228
Coe (Pr. de); Coe’s; Coe’s Golden Drop; Coe’s Golden Drop
Plum; Coe’s Imperial; Coe’s Plum; Coe’s Rotgefleckte
Pflaume; Coe’s Rothgefleckte Pflaume; Coe’s Seedling
(syns. of Golden Drop), 228
Coe’s Fine Late Red (syn. of Saint Martin), 336
Coe’s Golden Drop Violette (syn. of Coe Violet), 420
Coe’s Late Red (syn. of Saint Martin Quetsche), 538
Coe’s Late Red (syn. of Saint Martin), 336
Coë’s sehr späte rothe Pflaume (syn. of Saint Martin), 336
Coe Violette; Coe’s Violet; Coe’s Violette (syns. of Coe Violet),
420
Coe Violet, 420
Coeur de Boeuf, 420
Coeur de Pigeon (syn. of Queen Mother), 522
Coferer, 421
Coferers (syn. of Coferer), 421
Coinage, 421
Col. Wetherell (syn. of Wetherell), 563
Col. Young’s Seedling (syn. of Yellow Egg), 386
Coletta, 421
Coleus, 421
Collman (syn. of Colman), 421
Collo-Torto (syn. of Basaricatta), 399
Colman, 421
Colonel Bryan (syn. of Bryan), 410
Colonel or Col. Wilder (syns. of Wilder), 565
Colorado (syn. of Colorado Queen), 421
Colorado Queen, 421
Columbia, 181
Columbia (syn. of Captain), 414
Columbia Gage; Columbia Pflaume; Columbian Gage (syns. of
Columbia), 181
Columella, quoted, 28, 36
Combination, 421
Comfort, 421
Common Blue Damson (syn. of Winter Damson), 567
Common Damson (syn. of Damson, 186; of Winter Damson,
567)
Common Orleans (syn. of Orleans), 302
Common Quetsche (syn. of German Prune), 219
Common Saint Julien (syn. of Saint Julien), 335
Common Sloe (syn. of Sloe), 544
Communia, 421
Compass, 422
Compass, 182
Compass Cherry (syn. of Compass), 182
Compass Cherry (syn. of Compass), 422
Compote d’Automne (syn. of Autumn Compote), 153
Comptine, 422
Comte Gustavo d’Egger, 422
Consul, 422
Cope, John, var. orig. by, 423
Cooch, 422
Cooch, var. orig. by, 422
Cook, 422
Cook (syn. of Cook Choice), 422
Cook Choice, 422
Cook’s Choice; Cook’s Favorite (syns. of Cook Choice), 422
Cook’s Early (syn. of Cook), 422
Cooper, 422
Cooper, Joseph, var. orig. by, 423
Cooper’s (syn. of Smith Orleans), 348
Cooper’s (syn. of Cooper), 423
Cooper’s Blue Gage (syn. of Early Blue), 438
Cooper’s Grosse Pflaume (syn. of Cooper), 423
Cooper’s Grosse Rothe Zwetsche (syn. of Cooper), 423
Cooper’s Large (syn. of Cooper, 423; of Golden Drop, 228; of
Smith Orleans, 348)
Cooper’s Large American (syn. of Cooper), 423
Cooper’s Large Red (syn. of Cooper, 423; of Smith Orleans, 348)
Cooper’s Large Red American (syn. of Cooper), 423
Coopers Plum (syn. of Cooper), 423
Cooper’s Plum (syn. of Cooper), 423
Cooper’s Red (syn. of Cooper, 423; of Smith Orleans, 348)
Cope, 423
Cope’s Seedling (syn. of Cope), 423
Copper (syn. of Copper, 183; of Richland, 531)
Copper, 183
Copper Plum (syn. of Red Magnum Bonum), 325
Cornell, J. R., quoted, 244
Cornemuse, 423
Corse, Henry, var. orig. by, 391, 432, 445, 453, 455, 507, 532,
540, 557, 563
Corse’s Admiral (syn. of Admiral), 391
Corse’s Dictator (syn. of Dictator), 432
Corse’s Field Marshal; Corse’s Field Marshall (syns. of Field
Marshall), 445
Corse’s Great Bearer (syn. of Great Bearer), 455
Corse’s Nota Bena or Nota Bene (syn. of Nota Bene), 507
Corse’s Rising Sun (syn. of Rising Sun), 532
Corse’s Sauvageon (syn. of Sauvageon), 539
Corse’s Twins (syn. of Twins), 557
Corymbus, 423
Cottrell, 423
Cottrell, R. T., var. orig. by, 423
Couetch; Couetche; Couetsche (syns. of German Prune), 219
Couetsche d’Italie; Couetsche Fellenberg (syns. of Italian
Prune), 253
Couetsche Ordinaire (syn. of German Prune), 219
Couetsche Précoce; Couetsche Précoce La Vraie (syns. of Wahre
Frühzwetsche), 560
Couler, 423
Couler, William, var. orig. by, 423
Coulommiers, 423
Coulommiers Pflaume (syn. of Coulommiers), 423
Coulon Reine Claude, 423
Coulon’s Reine Claude (syn. of Coulon Reine Claude), 423
Coul Orleans, 423
Count Althann’s Gage (syn. of Altham), 141
Court Royal, 424
Cover-crops, value of, 121
Covetche (syn. of German Prune), 219
Covetsche (syn. of German Prune), 219
Cowles, E. D., var. orig. by, 516
Cowperthwait Green Gage, 424
Cox, Mrs., var. orig. by, 424
Cox, 424
Coxe, William, quoted, 25
Cox’s Emperor (syn. of Denbigh), 430
Cox’s Seedling (syn. of Cox), 424
Crable, 424
Craig, 424
Crescent, 424
Crescent City (syn. of Crescent), 424
Crimson, 424
Crimson Beauty, 424
Crimson Drop, 424
Crittenden, 184
Crittenden, James, var. orig. by, 184
Crittenden’s Damson (syn. of Crittenden), 184
Crittenden’s Prolific; Crittenden’s Prolific Damson (syns. of
Crittenden), 184
Croft Early, 424
Croft’s Early (syn. of Croft Early), 424
Cross, var. orig. by, 438
Cross-bred Prune A.P.-318 (syn. of Splendor), 547
Cruger, Henry, var. orig. by, 424
Cruger Scarlet, 424
Cruger’s; Cruger’s Früh Pflaume, Plum, Rote Pflaume, Scarlet,
Scarlet Gage, or Seedling (syns. of Cruger Scarlet), 424
Csaszar Sziloa, 425
Culberson, 425
Cumberland, 425
Curlew, 425
Curry, 425
Curry, S. L., var. orig. by, 425
Cyca Mono, 425
Cyclone, 425
Cydemarine, 425
Cyprian (syn. of Chypre), 418
Cyprische Pflaume (syn. of Red Diaper), 323
Czar, 184
D’Abricot of Streets of Paris (syn. of Peach), 309
D’Agen (syn. of Agen), 138
D’Agen Ameliorée (syn. of Ameliorée), 394
D’Agen Dorée (syn. of Dorée), 434
Dahlgreen, 425
Daisy, 425
Dakota, 425
D’Allemagne (syn. of German Prune), 219
Dalrymple, 426
Dalrymple Damson (syn. of Dalrymple), 426
D’Altesse Blanche (syn. of Yellow Impératrice), 569
Damas Ambre, 426
Damas Aubert (syn. of Yellow Egg), 386
Damas-Ballon Jaune (syn. of Ballonartige Gelbe Zwetsche), 398
Damas-Ballon Rouge (syn. of Ballonartige Rote Damascene),
398
Damas-Ballon Panachée Variete (syn. of Ballonartige Gelbe
Zwetsche), 398
Damas Blanc, Blanc Gros, Blanc Hâtif Gros, Blanc Tres Hâtif
(syns. of Large White Damson), 480
Damas blanc petit (syn. of Small White Damson), 544
Damas Commun (syn. of Damson), 186
Damas de Dieffenbach (syn. of Damas de Diffenbach), 426
Damas de Diffenbach, 426
Damas de Leipsick (syn. of Frühe Leipziger Damascene), 448
Damas de Mangeron; Damas de Mangeron, de Maugeron, de
Maugerou; or de Maugiron, Damascene Maugeron, von
Mangeron or von Maugeron (syns. of Maugeron), 492
Damas de Provence, 426
Damas de Provence or de Provence Hâtif (syns. of Damas de
Provence), 426
Damas de Septembre (syn. of Perdrigon Tardif, 515; of
September Damask, 542)
Damas d’Espagne (syn. of Spanish Damask), 546
Damas d’Ete, 426
Damas de Tours (syn. of Royal Tours), 332
Damas de Tours (syn. of Early Tours), 202
Damas de Tours or de Tours Gros (syns. of Gros Damas de
Tours), 456
Damas de Valence (syn. of Valence), 557
Damas d’Hiver (syn. of Winter Damson), 567
Damas d’Italie (syn. of Italian Damask), 470
Damas d’Onderka (syn. of Onderka Damascene), 509
Damas Dronet, 426
Damas Dronet (syn. of Damas Dronet), 426
Damas Dronet de Merlet, 427
Damas Fin (syn. of Musk Damson), 501
Damas Gris (syn. of Reine Claude), 327
Damas Gros (syn. of German Prune), 219
Damas gros de Tours (syn. of Gros Damas de Tours), 456
Damas Hâtif (syn. of Morocco), 288
Damas Jaune (syn. of Drap d’Or), 194
Damas Jaune Musque, 427
Damas Jaune Tardif de Koch (syn. of Koch späte Damascene),
477
Damas Lawson (syn. of Lawson), 482
Damas Long (syn. of German Prune), 219
Damas Musque (syn. of Musk Damson), 501
Damas Noir (syn. of Morocco), 288
Damas noir (syn. of Late Black Damson), 480
Damas Noir (syn. of Damson), 186
Damas Noir de Dieffenbach (syn. of Damas de Diffenbach), 426
Damas Noir de Tours, 427
Damas Noir Hâtif (syn. of Noire de Montreuil), 504
Damas Noir Hatif (syn. of Morocco), 288
Damas Noir Tardif (syn. of Late Black Damson), 480
Damas Précoce de Rivers (syn. of Rivers Early), 532
Damas Rouge (syn. of Orleans), 302
Damas Rouge de Biondeck, 427
Damas Rouge de Friedheim (syn. of Friedheim Damascene), 448
Damas Rouge de Mayer or de Moyer (syns. of Mayers Rothe
Damascene), 493
Damas Rouge Hâtif, 427
Damas Rouge Hâtif (syn. of Damas Rouge Hâtif), 427
Damas rouge tardif (syn. of Late Red Damask), 481
Damas Tardif de Kock (syn. of Koch Späte Damascene), 477
Damas Verd or Vert (syns. of Reine Claude), 327
Damas Vert (syn. of Mirabelle), 284
Damas Violet, 427
Damas Violet (syn. of Damas Violet, 427; of German Prune,
219; of Maugeron, 492; of Orleans, 302; of Queen Mother,
522)
Damas Violet Allonge (syn. of Lange Violette Damascene), 479
Damas Violet Gros (syn. of German Prune), 219
Damas Violet Longuet (syn. of Lange Violette Damascene), 479
Damas Violet of Tours (syn. of Gros Damas de Tours), 456
Damas Violet Tardif (syn. of Damas Violet), 427
Damascena Armeniaca or Armeniacea (syns. of Mirabelle), 284
Damascena Dominicalis Praecox (syn. of Early Orleans), 198
Damascena Schamali (syn. of Schamal), 540
Damascene (syn. of Damson), 186
Damascene (syn. of Shropshire), 344
Damascene Maugeron, von Mangeron or von Maugeron (syns.
of Maugeron), 492
Damascenen Mirabelle (syn. of Small White Damson), 544
Damaschino d’Estate (syn. of Damaschino Estivo), 426
Damaschino Estivo, 426
Damaschino Settembrino, 426
Damascus (syn. of Damson), 186
Damasine (syn. of Myrobalan), 290
Damasine (syn. of Rote Mirabelle), 533
Damask (syn. of German Prune), 219
Damask Violet (syn. of Queen Mother), 522
Damask of Provence (syn. of Damas de Provence), 426
Damasquinée (syn. of Musk Damson), 501
Damatie Rouge, 427
Dame Ambert, Ambert Blanche or Ambert Jaune, Aubert, Aubert
Blanche, Aubert Grosse Luisante, or Aubert Jaune (syns. of
Yellow Egg), 386
Dame Aubert, Aubert Rouge or Aubert Violette (syns. of Red
Magnum Bonum), 325
Dame Aubert Jaune (syn. of Aubert), 397
Dame Aubert Rouge (syn. of Red Diaper), 323
Dame-Aubert Rouge (syn. of Blue Egg), 405
Dame Aubert Violet (syn. of Duane), 196
D’Amerique Rouge (syn. of Myrobalan), 290
Damson (syn. of Damson, 186; of Winter Damson, 567)
Damson, 185
Damson Cluster (syn. of Crittenden), 184
Damson group, description of, 39-40
Damson Plum (syn. of Shropshire), 344
Damson Riley, 427
Damson Royal, 427
Damson Winter (syn. of Winter Damson), 567
Dana, Rev., var. orig. by, 428
Dana’s Gage; Dana’s Yellow Gage (syns. of Dana Yellow Gage),
428
Dana Yellow Gage, 427
Daniel, Dr., var. orig. by, 428
Daniel Weeping, 428
Danish Damson, 428
Dark Blue Egg (syn. of Kaiser Wilhelm), 474
Darst, 428
Darst, var. orig. by, 428
Darwin Peach (syn. of Yellow Egg), 386
Das Blaue Auge (syn. of Lange Violette Damascene), 479
D’Ast (syn. of Agen), 138
Date, 428
Date (syn. of Agen), 138
Date de Hongrie; Date Plum; Datte de Hongrie or Hungrie;
Datte Violette; Dattelzwetsche (syns. of Hungarian), 246
Date Plum (syn. of Date), 428
Datilles, 428
Datte Hongroise Jaune, 428
Datte Jaune (syn. of Weisse Kaiserin), 563
Dattel Pflaume or Zwetsche (syns. of Red Date), 322
Dattelpflaumen; Dattelzwetsche (syns. of Date), 428
Datte Verte (syn. of Grüne Dattel Zwetsche), 456
Datte Violette (syn. of Agen), 138
Dauphin (syn. of Victoria), 363
Dauphin (syn. of Sharp), 340
Dauphin à Fleurs doubles; Dauphin à Fleurs semi-doubles (syns.
of Double Flowering Gage), 192
Dauphine, 428
Dauphine; Dauphiny (syns. of Reine Claude), 327
D’Automne de Schamal (syn. of Schamal), 540
D’Autriche (syn. of Hungarian), 246
D’Autriche (syn. of Date), 428
Davenport, 429
Davies Seedling, 429
Davies’ Seedling (syn. of Davies Seedling), 429
Davis, 429
D’Avoine (syn. of Early Yellow), 203
Dawson, 186, 429
Dawson, P. P., var. orig. by, 187
Dawson City, 429
Dawson Seedling (syn. of Dawson), 186
De Bavay (syn. of Bavay), 155
De Besançon (syn. of Yellow Egg), 386
De Bordeaux (syn. of Jerusalem), 472
De Briançon (syn. of Briançon), 409
De Briancon (syn. of Red Diaper), 323
De Brignole (syn. of Agen), 138
DeCaisne, 187
Decaisnes Pflaume (syn. of DeCaisne), 187
De Caradeuc, 188
De Caradeuc, A., var. orig. by, 188
De Catalogue (syn. of Early Yellow), 203
De Chypre (syn. of Chypre, 418; of Red Diaper, 323; of Musk
Damson, 501)
De Coe (syn. of Golden Drop), 228
De Délice, 429
De Deux Saison (syn. of Twice Bearing), 556
De Gisborne (syn. of Gisborne), 451
De Gondin, 430
De Jerusalem (syn. of Jerusalem), 472
De Kirke (syn. of Kirke), 260
De la Toussaint, 430
De la Saint-Martin (syn. of Saint Martin), 336
Delaware, 430
De l’Inde, 430
De Mirabelle (syn. of Mirabelle), 284
De Mitchelson (syn. of Mitchelson), 498
De Monsieur (syn. of Yellow Egg, 386; Early Tours, 202;
Orleans, 302; Early Orleans, 198)
De Monsieur Hâtive (syn. of Early Orleans), 198
De Monsieur Jaune (syn. of Yellow Impératrice), 569
De Montfort, 430
De Montmirail, 430
De Pologne (syn. of Quetsche Dr. Létricourt), 524
De Pontbriant (syn. of Pontbriant), 517
De Prince (syn. of Norbert), 505
De Prince (en Lorraine) (syn. of De Seigneur), 431
De Reizeinstein (syn. of Reizenstein Yellow Prune), 531
De Sainte-Catherine (syn. of Saint Catherine), 334
De Seigneur, 431
De Soto, 189
De Soto × Oregon No. 3 (syn. of Ames), 144
De Virginie (syn. of Myrobalan), 290
De Wangenheim (syn. of Wangenheim), 368
De Wolf, M. J., var. orig. by, 498
Dean, 429
Deane, Samuel, D. D., quoted, 21, 38
Dean’s Jedburgh Seedling (syn. of Dean), 429
Deaton, 429
Deck, 429
Decks Damson (syn. of Deck), 429
Decker, 429
Decker, H. C., var. orig. by, 429
Decker’s Late Seedling (syn. of Decker), 429
Decker’s Seedling (syn. of Decker), 429
Deep Creek, 429
Deepcreek (syn. of Deep Creek), 429
Defresne, 430
Denbigh, 430
Denbigh-Pflaume; Denbigh Seedling; Denbigh (syns. of
Denbigh), 430
Dennie (syn. of Violet Diaper), 365
Dennis, 431
Dennison’s Red; Denniston’s Red; Denniston’s Rote Pflaume
(syns. of Denniston Red), 431
Dennison’s Superb (syn. of Denniston Superb), 431
Dennis Seedling No. 13, 431
Denniston, Isaac, var. orig. by, 393, 411, 431, 478, 500
Denniston Red, 431
Denniston’s Albany or Albany Beauty (syns. of Albany Beauty),
392
Denniston Superb, 431
Denniston’s Superb or Superb Gage (syns. of Denniston
Superb), 431
D’Ente (syn. of Lot d’Ente), 486
D’Ente; D’Ente d’Agen (syns. of Agen), 138
D’Ente Impériale, 431
Denton, 431
Denyer’s Victoria (syn. of Sharp, 340; of Victoria, 363)
Der blaue Rebhuhn aus der Normandie (syn. of Normand
Perdrigon), 505
Der Bunter Perdrigon (syn. of Bunter Perdrigon), 412
Derbyshire Green Gage, 431
Deron, 431
Deron’s (syn. of Deron), 431
Des Béjonnières (syn. of Béjonnières), 157
Des Burettes (syn. of Burettes), 412
Des Moines, 431
Deutsche Blaue Herbstzwetsche (syn. of German Prune), 219
Deux fois l’an (syn. of Twice Bearing), 556
Dewey, 432
Diademe; Diademe Imperial or Imperiale (syns. of Red Diaper),
323
Diademe Imperial-Isabelle, 432
Diamant; Diamantpflaume (syns. of Diamond), 191
Diamant; Diamantpflaume (syns. of Pond), 314
Diamond, 191, 432
Diamond, var. orig. by, 191
Diana, 432
Diaper; Diaper Rouge; Diapre Rouge, Diaprea rubra; Diaprée de
Roche Corbon; Diaprée Rouge (syns. of Red Diaper), 323
Diaper’d Plumb (syn. of Diaprée Blanche), 432
Diaphane (syn. of Transparent), 360
Diaphane Laffay (syn. of Transparent), 360
Diapre Blanc (syn. of White Diaper), 564
Diaprée Blanche; Diaprée Weisse (syns. of Diaprée Blanche),
432
Diaprée Blanche (syn. of White Diaper, 564; of White Perdrigon,
375)
Diaprée Blanche, 432
Diaprée noire; Diapre Violet; Diaprée Violette (syns. of Violet
Diaper), 365
Diapree Nouvelle De Kook, 432
Dictator, 432
Die Abrikosenartige Pflaume (syn. of Apricot), 148
Die Albertus Damenpflaume (syn. of Yellow Egg), 386
Die Aprikosenpflaume (syn. of Red Apricot), 321
Die Bischofsmütze (syn. of Bonnet d’Eveque), 407
Die Blaue Dattelpflaume (syn. of Agen), 138
Die Blutfarbege Pflaume (syn. of Red Diaper), 323
Die Brisette (syn. of Late Mirabelle), 263
Die Brustwarzenpflaume (syn. of Mamelonnée), 489
Die Damascenerpflaume von Maugeron (syn. of Maugeron), 492
Die Damaskpflaume aus Spanien (syn. of Spanish Damask), 546
Die Durchsichtige (syn. of Transparente), 555
Die frühe Pflaume von Tours (syn. of Early Tours), 202
Die Gartenzwetsche (syn. of Quetsche Maraichère), 524
Die gelbe frühzeitige Pflaume (syn. of Early Yellow), 203
Die grosse indianische braunrothe Pflaume (syn. of Indian), 468
Die grosse Königin Claudiapflaume, die grüne Abrikose (syn. of
Reine Claude), 327
Die grosse Königin Klaudia Pflaume mit halbgefullter Bluthe
(syn. of Double Flowering Gage), 192
Die grosse rothe Feigenpflaume (syn. of Red Date), 322
Die grosse Weisse Glanzende (syn. of Yellow Egg), 386
Die grüne herzförmige Pflaume (syn. of Grüne Herzformige),
456
Die Hauszwetsche (syn. of German Prune), 219
Die Herrnpflaume (syn. of Orleans), 302
Die Hyacinthenpflaume (syn. of Jacinthe), 471
Die Isabelle (syn. of Isabella), 469
Die Kaiserliche veilchenfarbige Pflaume (syn. of Red Magnum
Bonum), 325
Die Kaiserliche Weisse Pflaume (syn. of Yellow Egg), 386
Die kleine Königin Claudia (syn. of Small Reine Claude), 347
Die kleine langlichte Damaskuspflaume (syn. of Damas Dronet),
426
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

textbookfull.com