Download Full Version ebookmass - Visit ebookmass.

com

Numerical Methods Using Kotlin: For Data Science,

Analysis, and Engineering 1st Edition Haksun Li

https://ebookmass.com/product/numerical-methods-using-
kotlin-for-data-science-analysis-and-engineering-1st-
edition-haksun-li-2/

OR CLICK HERE

DOWLOAD NOW

Discover More Ebook - Explore Now at ebookmass.com

 Instant digital products (PDF, ePub, MOBI) ready for you
Download now and discover formats that fit your needs...

Numerical Methods Using Kotlin: For Data Science,

Analysis, and Engineering 1st Edition Haksun Li

https://ebookmass.com/product/numerical-methods-using-kotlin-for-data-
science-analysis-and-engineering-1st-edition-haksun-li/

ebookmass.com

Numerical Methods in Environmental Data Analysis Moses

Eterigho Emetere

https://ebookmass.com/product/numerical-methods-in-environmental-data-
analysis-moses-eterigho-emetere/

ebookmass.com

Spatial analysis using big data: methods and urban

applications Yamagata

https://ebookmass.com/product/spatial-analysis-using-big-data-methods-
and-urban-applications-yamagata/

ebookmass.com

The Crimson Queen (Dark Halos Book 3) Amanda Aggie

https://ebookmass.com/product/the-crimson-queen-dark-halos-
book-3-amanda-aggie/

ebookmass.com
Genetic and Metabolic Engineering for Improved Biofuel
Production from Lignocellulosic Biomass Arindam Kuila

https://ebookmass.com/product/genetic-and-metabolic-engineering-for-
improved-biofuel-production-from-lignocellulosic-biomass-arindam-
kuila/
ebookmass.com

The Wiley Blackwell Companion to Hinduism 2nd Edition

Gavin Flood

https://ebookmass.com/product/the-wiley-blackwell-companion-to-
hinduism-2nd-edition-gavin-flood/

ebookmass.com

Wisdom by Anthony De Mello - author of Awareness -

includes talks not published in Anthony De Mello's books
Anthony De Mello
https://ebookmass.com/product/wisdom-by-anthony-de-mello-author-of-
awareness-includes-talks-not-published-in-anthony-de-mellos-books-
anthony-de-mello/
ebookmass.com

Her Ruthless Duke: A Guardian Ward Regency Romance

(Rogue's Guild Book 1) Scarlett Scott

https://ebookmass.com/product/her-ruthless-duke-a-guardian-ward-
regency-romance-rogues-guild-book-1-scarlett-scott/

ebookmass.com

Ruthless Kings: A Dark, Why Choose Mafia Romance (Black

Hollow Isle Book 2) Dani René

https://ebookmass.com/product/ruthless-kings-a-dark-why-choose-mafia-
romance-black-hollow-isle-book-2-dani-rene/

ebookmass.com
Nutrition, Sustainable Agriculture and Climate Change in
Africa: Issues and Innovative Strategies 1st ed. Edition
Gbadebo Odularu
https://ebookmass.com/product/nutrition-sustainable-agriculture-and-
climate-change-in-africa-issues-and-innovative-strategies-1st-ed-
edition-gbadebo-odularu/
ebookmass.com
Haksun Li, PhD

Numerical Methods Using Kotlin

For Data Science, Analysis, and Engineering
Haksun Li, PhD
Hong Kong, China

ISBN 978-1-4842-8825-2 e-ISBN 978-1-4842-8826-9

https://doi.org/10.1007/978-1-4842-8826-9

© Haksun Li, PhD 2023

This work is subject to copyright. All rights are solely and exclusively
licensed by the Publisher, whether the whole or part of the material is
concerned, specifically the rights of translation, reprinting, reuse of
illustrations, recitation, broadcasting, reproduction on microfilms or in
any other physical way, and transmission or information storage and
retrieval, electronic adaptation, computer software, or by similar or
dissimilar methodology now known or hereafter developed.

The use of general descriptive names, registered names, trademarks,

service marks, etc. in this publication does not imply, even in the
absence of a specific statement, that such names are exempt from the
relevant protective laws and regulations and therefore free for general
use.

The publisher, the authors, and the editors are safe to assume that the
advice and information in this book are believed to be true and accurate
at the date of publication. Neither the publisher nor the authors or the
editors give a warranty, expressed or implied, with respect to the
material contained herein or for any errors or omissions that may have
been made. The publisher remains neutral with regard to jurisdictional
claims in published maps and institutional affiliations.

This Apress imprint is published by the registered company APress

Media, LLC, part of Springer Nature.
The registered company address is: 1 New York Plaza, New York, NY
10004, U.S.A.
Dedicated to my beloved children, Tintin, Yiyi and Abigail, who will one
day find the one sentence in this book about how to solve any problems.
Preface
When I was a boy, I liked to organize my math notes. I did not really like
math (or any studies) at all until I met my first math teacher, who
enlightened me. It was Professor Bīn Hé (何斌) who showed me how
interesting math could be and taught me math at the Sun Yat-sen
University in the summers when I was in junior high school. To him, I
am forever grateful. Mathematical training shaped the way I think and
opened the door to learning other science subjects like physics. Those
summers changed my life from being a mediocre student to a top one.
I found mathematics interesting because there are structures in
numbers. For example, prime numbers build all numbers; there are no
“holes” in the real number line. There are so many ways to prove the
same thing, e.g., the Pythagorean theorem. There are again structures
and relationships among geometric shapes, as well as between shapes
and numbers. It is fascinating why. I thought I was pretty good at
mathematics until my first year taking an honor’s class in real analysis
at the university. Some of my classmates were way beyond my league. I
did not see myself becoming a mathematician, so I switched to
computer science in graduate school, thinking that it might be easier.
Studying computer science is never about learning to program. It
turned out that I was a pretty good coder, or so I thought, after coding
C++ almost a decade at universities. I had the good aptitude of spending
hours and even days doing nothing other than debugging a
segmentation fault, fixing a memory leak, or comparing numbers down
to the last decimal place. (I still remember when it took me three days
to figure out a very obscure bug at UBS.) It turned out that I was
ignorantly over-confident when my first mentor introduced design
patterns to me. Dr. Anil Kumar asked me to design a singleton during a
job interview and guided me through designing the double-checked
locking pattern. Luckily, he still hired me onto his team for my first job. I
am thankful for his training, guidance, and mentorship, which jump-
started my career of becoming a professional quantitative trader.
My training in mathematics and computer science, with a
specialization in artificial intelligence, has been powering my career as
a quantitative trader and applying mathematics to solve business
problems. It is a rare combination of skills. Most mathematicians
cannot code (Excel/MATLIB/R is not really professional coding), and
most programmers do not read mathematics. We often needed to
translate R code into Java code and back and forth between the
mathematicians who created the trading models and the programmers
who implemented them in an order-execution system. So, one day I
decided to make myself useful by creating a programming library of
mathematics, using both of my skills. This is kind of a continuation of
my childhood hobby of organizing my math notes, because now I am
organizing math algorithms in a more useful form. The objective is that
quantitative programmers can code mathematics in the same language
that they use to build applications. There will be no more “lost in
translation” issues.
I was lucky to hire Dr. Ken Yiu in 2007, starting out as an intern and
then going full-time, who is a cofounder of the startup. Ken is the best
software engineer I have ever met. He is smart, quick-witted, careful,
and above all responsible. I cannot thank him enough for traveling with
me through this adventure of learning and building the math library,
trading strategies, and systems. He has taught me a lot about computer
science and programming.
In 2008, Dr. Kevin Sun joined the team as an intern. I was fortunate
to have Kevin later join the startup as another cofounder. He is now the
brains behind our team and is the key person creating many
mathematical models in trading and gambling. He has excellent
mathematical-thinking and problem-solving skills. I am still impressed
by how he could back then (and still can) solve in a nice and elegant
way just about every problem we threw at him. He always saves the
team before deadlines with his incredible math and R skills. He has
taught me a lot about mathematics and statistics.
Also, I am forever grateful to my brother, Kevin, for all his moral
support, teamwork, contributions, encouragement, advice, connections,
precious time, and financial assistance through all the ups and downs,
laughs and tears, and hopes on this journey with me. Nothing would
have been possible without his tremendous effort and resources. I
thank God for having him in life.
This is the story of how NM Dev, the math library, started. Today, the
library has been adopted by big-name commercial organizations as well
as academic institutions. NM Dev is the cornerstone of all the work that
we do. We build trading strategies, financial analytics (AlgoQuant),
fixed-income analytics (SuperCurve), data-driven decision-making
systems (NM Optim), and high-performance solvers and optimizers
(NM Solver) based on this Java numerical programming library.

Source Code
All source code used in this book can be downloaded from
github.com/apress/numerical-methods-kotlin.
Any source code or other supplementary material referenced by the
author in this book is available to readers on GitHub
(github.com/apress). For more detailed information, please visit
www.apress.com/source-code.
Acknowledgments
I never knew that writing a book would be such a tremendous effort,
especially for this book with such wide coverage. It started out as a one-
year project and then was extended to become an almost two-year
project. For eight months in 2021, I basically did nothing other than
write this book, pretty much every day. What would be a better thing to
do during the indefinite COVID-19 lockdown?
This book covers a lot of topics in basic college applied
mathematics, supplemented with a lot of sample code and more than
300 images for illustration. It was a tremendous effort to put the
materials together. I want to thank my team for their incredible support
throughout the writing of this manuscript. They are amazing and this
book would not have happened without them: Kehao Feng, Jingqi Guo,
Shihao Liu, Yang Lu, Hengyu Luo, and Dr. Xueying Ni.
Although we have never met in person, I thank Steve Anglin for
inviting me to write this book, something that I have wanted to do for a
long time but never found sufficient motivation to do. I thank you Mark
Wickham for carefully reviewing my book and making a lot of good
comments and suggestions. Last but not least, I must thank Mark
Powers for pushing me to submit the manuscript. He provided a huge
push for me to sit down at my computer and start typing and meet the
always-missed-and-always-late deadlines. I thank you all for your
patience!
Table of Contents
Chapter 1:​Introduction to Numerical Methods in Kotlin/​S2
1.​1.​Library Design
1.​1.​1.​Class Parsimony
1.​1.​2.​Java vs C++ Performance
1.​2.​Java Scripting
1.​2.​1.​Kotlin Programming Language
1.​3.​S2
1.​3.​1.​R Replacement
1.​3.​2.​Python Replacement
1.​3.​3.​S2 Online IDE
1.​4.​About This Book
1.​4.​1.​Sample Code
Chapter 2:​Linear Algebra
2.​1.​Vector
2.​1.​1.​Element-Wise Operations
2.​1.​2.​Norm
2.​1.​3.​Inner Product and Angle
2.​2.​Matrix
2.​2.​1.​Matrix Operations
2.​2.​2.​Element-Wise Operations
2.​2.​3.​Transpose
2.​2.​4.​Matrix Multiplication
2.​2.​5.​Rank
2.​2.​6.​Determinant
2.​2.​7.​Inverse and Pseudo-Inverse
 2.​2.​8.​Kronecker Product
2.​3.​Matrix Decomposition
2.​3.​1.​LU Decomposition
2.​3.​2.​Cholesky Decomposition
2.​3.​3.​Hessenberg Decomposition and Tridiagonalizati​on
2.​3.​4.​QR Decomposition
2.​3.​5.​Eigen Decomposition
2.​3.​6.​Singular Value Decomposition
2.​4.​System of Linear Equations
2.​4.​1.​Row Echelon Form and Reduced Row Echelon Form
2.​4.​2.​Back Substitution
2.​4.​3.​Forward Substitution
2.​4.​4.​Elementary Operations
2.​4.​5.​Gauss Elimination and Gauss-Jordan Elimination
2.​4.​6.​Homogeneous and Nonhomogeneous Systems
2.​4.​7.​Overdetermined Linear System
2.​5.​Sparse Matrix
2.​5.​1.​Dictionary of Keys
2.​5.​2.​List of Lists
2.​5.​3.​Compressed Sparse Row
2.​5.​4.​Sparse Matrix/​Vector Operations
2.​5.​5.​Solving Sparse Matrix Equations
Chapter 3:​Finding Roots of Equations
3.​1.​An Equation of One Variable
3.​2.​Jenkins-Traub Algorithm
3.​3.​The Bisection Method
 3.​4.​Brent’s Method
3.​4.​1.​Linear Interpolation Method, False Position Method,
Secant Method
3.​4.​2.​Inverse Quadratic Interpolation
3.​4.​3.​Brent’s Method Implementation
3.​5.​The Newton-Raphson Method
3.​5.​1.​Halley’s Method
Chapter 4:​Finding Roots of System of Equations
4.​1.​System of Equations
4.​2.​Finding Roots of Systems of Two Nonlinear Equations
4.​3.​Finding Roots of Systems of Three or More Equations
Chapter 5:​Curve Fitting and Interpolation
5.​1.​Least Squares Curve Fitting
5.​2.​Interpolation
5.​2.​1.​Linear Interpolation
5.​2.​2.​Cubic Hermite Spline Interpolation
5.​2.​3.​Cubic Spline Interpolation
5.​2.​4.​Newton Polynomial Interpolation
5.​3.​Multivariate Interpolation
5.​3.​1.​Bivariate Interpolation
5.​3.​2.​Multivariate Interpolation
Chapter 6:​Numerical Differentiation and Integration
6.​1.​Numerical Differentiation
6.​2.​Finite Difference
6.​2.​1.​Forward Difference
6.​2.​2.​Backward Difference
 6.​2.​3.​Central Difference
6.​2.​4.​Higher-Order Derivatives
6.​3.​Multivariate Finite Difference
6.​3.​1.​Gradient
6.​3.​2.​Jacobian
6.​3.​3.​Hessian
6.​4.​Ridders’ Method
6.​5.​Derivative Functions of Special Functions
6.​5.​1.​Gaussian Derivative Function
6.​5.​2.​Error Derivative Function
6.​5.​3.​Beta Derivative Function
6.​5.​4.​Regularized Incomplete Beta Derivative Function
6.​5.​5.​Gamma Derivative Function
6.​5.​6.​Polynomial Derivative Function
6.​6.​Numerical Integration
6.​7.​The Newton-Cotes Family
6.​7.​1.​The Trapezoidal Quadrature Formula
6.​7.​2.​The Simpson Quadrature Formula
6.​7.​3.​The Newton-Cotes Quadrature Formulas
6.​8.​Romberg Integration
6.​9.​Gauss Quadrature
6.​9.​1.​Gauss-Legendre Quadrature Formula
6.​9.​2.​Gauss-Laguerre Quadrature Formula
6.​9.​3.​Gauss-Hermite Quadrature Formula
6.​9.​4.​Gauss-Chebyshev Quadrature Formula
6.​10.​Integration by Substitution
 6.​10.​1.​Standard Interval
6.​10.​2.​Inverting Variable
6.​10.​3.​Exponential
6.​10.​4.​Mixed Rule
6.​10.​5.​Double Exponential
6.​10.​6.​Double Exponential for Real Line
6.​10.​7.​Double Exponential for Half Real Line
6.​10.​8.​Power Law Singularity
Chapter 7:​Ordinary Differential Equations
7.​1.​Single-Step Method
7.​1.​1.​Euler’s Method (Polygon Method)
7.​1.​2.​Runge-Kutta Family
7.​1.​3.​Convergence
7.​1.​4.​Stability
7.​2.​Linear Multistep Method
7.​2.​1.​Adams-Bashforth Method
7.​3.​Comparison of Different Methods
7.​4.​System of ODEs and Higher-Order ODEs
Chapter 8:​Partial Differential Equations
8.​1.​Second-Order Linear PDE
8.​1.​1.​Parabolic Equation
8.​1.​2.​Hyperbolic Equation
8.​1.​3.​Elliptic Equation
8.​2.​Finite Difference Method
8.​2.​1.​Numerical Solution for Hyperbolic Equation
8.​2.​2.​Numerical Solution for Elliptic Equation
 8.​2.​3.​Numerical Solution for Parabolic Equation
Chapter 9:​Unconstrained Optimization
9.​1.​Brute-Force Search
9.2. C2OptimProblem
9.​3.​Bracketing Methods
9.​3.​1.​Fibonacci Search Method
9.​3.​2.​Golden-Section Search
9.​3.​3.​Brent’s Search
9.​4.​Steepest Descent Methods
9.​4.​1.​Newton-Raphson Method
9.​4.​2.​Gauss-Newton Method
9.​5.​Conjugate Direction Methods
9.​5.​1.​Conjugate Directions
9.​5.​2.​Conjugate Gradient Method
9.​5.​3.​Fletcher-Reeves Method
9.​5.​4.​Powell Method
9.​5.​5.​Zangwill Method
9.​6.​Quasi-Newton Methods
9.​6.​1.​Rank-One Method
9.​6.​2.​Davidon-Fletcher-Powell Method
9.​6.​3.​Broyden-Fletcher-Goldfarb-Shanno Method
9.​6.​4.​Huang Family (Rank One, DFP, BFGS, Pearson,
McCormick)
Chapter 10:​Constrained Optimization
10.​1.​The Optimization Problem
10.​1.​1.​General Optimization Algorithm
 10.​1.​2.​Constraints
10.​2.​Linear Programming
10.​2.​1.​Linear Programming Problems
10.​2.​2.​First-Order Necessary Conditions
10.​2.​3.​Simplex Method
10.​2.​4.​The Algebra of the Simplex Method
10.​3.​Quadratic Programming
10.​3.​1.​Convex QP Problems with Only Equality Constraints
10.​3.​2.​Active-Set Methods for Strictly Convex QP Problems
10.​4.​Semidefinite Programming
10.​4.​1.​Primal and Dual SDP Problems
10.​4.​2.​Central Path
10.​4.​3.​Primal-Dual Path-Following Method
10.​5.​Second-Order Cone Programming
10.​5.​1.​SOCP Problems
10.​5.​2.​Primal-Dual Method for SOCP Problems
10.​6.​General Nonlinear Optimization Problems
10.​6.​1.​SQP Problems with Only Equality Constraints
10.​6.​2.​SQP Problems with Inequality Constraints
Chapter 11:​Heuristics
11.​1.​Penalty Function Method
11.​2.​Genetic Algorithm
11.​2.​1.​Encoding
11.​2.​2.​Fitness Function
11.​2.​3.​Initial Population
11.​2.​4.​The Operation Process
 11.​2.​5.​Selection
11.​2.​6.​Crossover
11.​2.​7.​Mutation
11.​2.​8.​Differential Evolution
11.​3.​Simulated Annealing
Chapter 12:​Basic Statistics
12.​1.​Random Variables
12.​2.​Sample Statistics
12.​2.​1.​Mean
12.​2.​2.​Weighted Mean
12.​2.​3.​Variance
12.​2.​4.​Weighted Variance
12.​2.​5.​Skewness
12.​2.​6.​Kurtosis
12.​2.​7.​Moments
12.​2.​8.​Rank
12.​2.​9.​Covariance
12.​3.​Probability Distribution
12.​3.​1.​Moments
12.​3.​2.​Normal Distribution
12.​3.​3.​Log-Normal Distribution
12.​3.​4.​Exponential Distribution
12.​3.​5.​Poisson Distribution
12.​3.​6.​Binomial Distribution
12.​3.​7.​T-Distribution
12.​3.​8.​Chi-Square Distribution
 12.​3.​9.​F-Distribution
12.​3.​10.​Rayleigh Distribution
12.​3.​11.​Gamma Distribution
12.​3.​12.​Beta Distribution
12.​3.​13.​Weibull Distribution
12.​3.​14.​Empirical Distribution
12.​4.​Multivariate Probability Distributions
12.​4.​1.​Multivariate Normal Distribution
12.​4.​2.​Multivariate T-Distribution
12.​4.​3.​Multivariate Beta Distribution
12.​4.​4.​Multinomial Distribution
12.​5.​Hypothesis Testing
12.​5.​1.​Distribution Tests
12.​5.​2.​Rank Test
12.​6.​Markov Models
12.​6.​1.​Discrete-Time Markov Chain
12.​6.​2.​Hidden Markov Model
12.​7.​Principal Component Analysis
12.​8.​Factor Analysis
12.​9.​Covariance Selection
Chapter 13:​Random Numbers and Simulation
13.​1.​Uniform Random Number Generators
13.​1.​1.​Linear Congruential Methods
13.​1.​2.​Mersenne Twister
13.​2.​Sampling from Probability Distributions
13.​2.​1.​Inverse Transform Sampling
 13.​2.​2.​Acceptance-Rejection Sampling
13.​2.​3.​Sampling from Univariate Distributions
13.​2.​4.​Sampling from Multivariate Distributions
13.​2.​5.​Resampling Method
13.​3.​Variance Reduction
13.​3.​1.​Common Random Numbers
13.​3.​2.​Antithetic Variates
13.​3.​3.​Control Variates
13.​3.​4.​Importance Sampling
Chapter 14:​Linear Regression
14.​1.​Ordinary Least Squares
14.​1.​1.​Assumptions
14.​1.​2.​Model Properties
14.​1.​3.​Residual Analysis
14.​1.​4.​Influential Point
14.​1.​5.​Information Criteria
14.​1.​6.​NM Dev Linear Regression Package
14.​2.​Weighted Least Squares
14.​3.​Logistic Regression
14.​4.​Generalized Linear Model
14.​4.​1.​Quasi-family
14.​5.​Stepwise Regression
14.​6.​LASSO
Chapter 15:​Time-Series Analysis
15.​1.​Univariate Time Series
15.​1.​1.​Stationarity
 15.​1.​2.​Autocovariance
15.​1.​3.​Autocorrelation
15.​1.​4.​Partial Autocorrelation
15.​1.​5.​White Noise Process and Random Walk
15.​1.​6.​Ljung-Box Test for White Noise
15.​1.​7.​Model Decomposition
15.​2.​Time-Series Models
15.​2.​1.​AR Models
15.​2.​2.​MA Model
15.​2.​3.​ARMA Model
15.​2.​4.​ARIMA Model
15.​2.​5.​GARCH Model
15.​3.​Multivariate Time Series
15.​3.​1.​VAR Model
15.​3.​2.​VMA Model
15.​3.​3.​VARMA Model
15.​3.​4.​VARIMA Model
15.​4.​Cointegration
15.​4.​1.​VEC Model
15.​4.​2.​Johansen Cointegration Test
Index
About the Author
Haksun Li
, PhD, is the founder and CEO of NM DEV
PTE. LTD, NM Education LTD, NM Optim
LTD, and NM FinTech LTD. The NM group
of companies has the single mission of
“Making the World Better Using
Mathematics.” Under his leadership, the
company group serves security houses,
hedge funds, high-net-worth individuals,
multinational corporations, factories,
and plants all over the world. See
https://nm.sg.
NM DEV PTE. LTD is dedicated to the
research and development of innovative
computing technologies for business
operations as well as quantitative wealth
management. Haksun and the team have
developed a suite of programming libraries for mathematics. They
created the S2 platform, a next-generation data science toolbox that
offers high performance and easy deployment. See https://nm.dev.
NM Education LTD is an online tutoring service to teach pre-
university kids mathematics using an innovative learn-by-coding
approach. Traditional math teaching often asks students to repeatedly
do exercises mechanically. Traditional programming classes mainly
teach language constructs and syntax and ask students to code toy
problems. Haksun and his team have designed a new teaching
methodology to teach math using programming. Kids are asked to code
the math concepts to solve real engineering problems. Consequently,
not only do students gain hands-on experience applying new skills, but
they also gain deeper understanding of how math actually works. See
https://nm.education.
NM Optim LTD helps factories and plants streamline operations,
improve productivity, and increase revenue by applying operations,
research, and optimization theories. It is the bridge between industries
and academia. Haksun leads a team that is revolutionizing the
workflow of factories by making more efficient use of resources and
offering better scheduling, better management, better pricing, and so
on, all based on mathematical models, hence upgrading companies to
Industry 4.0. See https://nmoptim.com.
NM FinTech LTD develops financial analytic systems for quantitative
portfolio managers worldwide. For the U.S. equity market, Haksun
created a risk-management system, together with a factor library and
tools for portfolio construction. For the China bond market, Haksun
created SuperCurve, which is to date the only accurate yield curve
publicly available for pricing Chinese bonds, together with a suite of
fixed-income analytics. See https://nmfin.tech.
Haksun is the associate dean and a professor of the Big Data Finance
and Investment Institute of Fudan University in Shanghai, China. He
was an adjunct professor with multiple universities. He has taught at
the National University of Singapore (mathematics), the Nanyang
Technological University (business), the Fudan University (economics),
and the Hong Kong University of Science and Technology
(mathematics). Prior to that, Haksun was a quantitative analyst and a
quantitative trader at UBS and BNP Paribas New York, Tokyo, London,
and Singapore.
Haksun holds a bachelor’s degree in mathematics and a master’s
degree in financial mathematics from the University of Chicago, as well
as a master’s and a doctoral degree in computer science and
engineering (with specialization in artificial intelligence) from the
University of Michigan – Ann Arbor.
About the Technical Reviewer
Mark Wickham
is a software developer based in Dallas. Prior to returning to Texas in
2020, Mark lived in Beijing for nearly 20 years, where he worked for
Motorola, Inc., as a software engineering manager, and in various
program management and product management roles for the Asia-
Pacific region. Mark got his start in Java and Android when Motorola
produced the first Java-based mobile phone and he wanted to write his
own apps.
Mark was a regular speaker at some of the well-known Android
developer conferences and has written two books for Apress, Practical
Java Machine Learning and Practical Android.
Before embarking on his professional career, Mark obtained a
degree in physics from Creighton University in Omaha, with a major in
computer science and a minor in mathematics. Mark later obtained an
MBA from the University of Washington and also studied at the Hong
Kong University of Science and Technology.
An Iowa native, in his free time, Mark enjoys gravel bike riding,
sports photography, and his latest new passion, disc golf. For more
information about Mark, connect with him at
linkedin.com/in/mark-j-wickham.
© The Author(s), under exclusive license to APress Media, LLC, part of Springer
Nature 2023
H. Li, PhD, Numerical Methods Using Kotlin
https://doi.org/10.1007/978-1-4842-8826-9_1

1. Introduction to Numerical Methods

in Kotlin/S2
Haksun Li, PhD1

(1) Hong Kong, China

When I coded the first line of a matrix multiplication algorithm, the

classical O(n3) version, in the attic of my London apartment during an
evening back in 2009, it was nothing more than a hobby. I was coding a
trading strategy in Java but could not find a good Java math library to
use (namely, one that was professionally maintained, well documented
with examples, solidly object-oriented, and had an extensive open data
structure). A large part of the strategy code was developed in
R/MATLAB and the trade/order execution system was in Java. However,
making Java talk to R/MATLAB and vice versa was a nightmare, let
alone the many other IT nuisances (e.g., multi-thread safety, multi-
instances, debugging, R/MATLAB was very slow, and the mysterious
who-knows-why-not-working issues).
I thought that the world would benefit from having a Java math
library much like the then-ubiquitous “numerical recipes in C”. One easy
way to create one was to (auto) translate the C code into Java. Yet, this
gave me a bunch of functions (procedures) that did only what the
functions did. There was no data structure or module or
(sub-)algorithm that I could reuse to build my new data structures or
algorithms. A vector should not just be a bunch of numbers or an array.
A vector is a mathematical concept that is well-defined with properties
and operations. Any algorithm that works with vectors should
understand them, and not have to re-code the properties and
operations.
While there were a few Java numerical libraries at the time, most
notably Apache Commons Math, none of them was designed to be high
performance. Modern computing has come a long way since the
FORTRAN/C era. We now have concurrent programming, more efficient
algorithms (for matrix multiplication, etc.), new hardware like GPUs,
and so on. Simple translation of FORTRAN/C code or textbook
implementation of algorithms would not leverage these advances.
Performance would be very slow. We needed a multi-threading, highly
optimized data structure as well as efficient algorithms. For example,
instead of using the simple O(n3) algorithm for matrix multiplication,
we implemented a parallelized version of the Strassen algorithm. Due
to a lot of team effort twisting, experimenting, and testing things during
these ten years, the end result is that we now have arguably the fastest
linear algebra implementation in Java.
Another problem that I had with the existing Java math libraries
was that they were very limited in scope. A quantitative trading
strategy can be as simple as a moving average crossover, but it can also
be very complicated, involving solving stochastic differential equations.
It certainly involves a lot of statistical analysis. Last but not least, it also
takes a suite of solvers to optimize things like strategy parameters.
Pretty much all statistical procedures involve some optimization, for
example, regression and maximum likelihood. There is no free or cheap
optimizer that is good and is in Java. Professional ones such as Gurobi
and Mosek are expensive.
Something might have hit my head and made me do the
unthinkable. I resigned from my trader job with the investment bank
and embarked on a great adventure. I set out to create a Java math
library that was well designed with data structure and software
engineering, easy-to-read for humans (not machines), leveraged
modern computing technologies, enjoyed high performance, and was
extensive, including linear algebra, Calculus, statistics, time series
analysis, optimization, stochastic differential equations, and so on. I
didn’t realize how difficult this would be when I quit my job. I didn’t
know how many Herculean tasks there were to overcome. I just thought
that it would be fun to create something to (hopefully) share with the
world. When I was a teen, I used to write my own math books by
organizing my mathematics notes. I supposed then that organizing
algorithms in code would be much more useful, a sort of continuation
of my childhood hobby.
It turned out that building a math library, especially a good one, was
very challenging. It was much more than translating math formulas into
code. First, we needed to understand every detail in the algorithms. For
many of them, we read the original publications. Some old publications
were written in an ancient and arcane language that was very difficult
to understand. Many papers had details missing. Many papers had
errors and typos. For each algorithm that we implemented, we had to
re-derive every step in the mathematics to make sure that we
understood all the details and that the algorithm was correct. This was
nothing like learning from textbooks, which are usually well written
and illustrated with plenty of examples.
There are many concepts that we are used to using without even
thinking about the details. For example, we use eigenvalues and
eigenvectors of a matrix all the time, but few of us care how
decomposing a matrix actually works. To code a linear algebra package,
we needed to understand the many different decompositions and
factorizations of a matrix, their pros and cons, precision, numerical
stability, debugging and testing, API design, and user friendliness — all
little details. This process was non-trivial.
Second, we had to verify that our code was correct. If the paper
came with examples, we reproduced them to make sure that our results
matched theirs. They were usually very few simple illustrative
examples. We needed to try for a wide range of test cases, corner cases,
stress tests, and performance tests. Our benchmark was R. We would
always match our results with Rs. When there were differences, we had
to explain every one of them. We were very serious about the accuracy
of our code and were constantly fixing bugs in each release. In some
cases, our code was even more accurate than the Rs. For example, we
didn’t know that R has for Johansen tests only a few critical points —
e.g., 10%, 5%, 1% — and did a linear interpolation between them for
the other values (from the R’s source code). On the other hand, we
computed the asymptotic distribution for Johansen tests using the exact
distribution functions. That was many hours of debugging and reading
the R source code.
Third, we designed the software so that it was user friendly and in
line with good software engineering principles. Most popular math
libraries, such as Netlib, were written decades ago in FORTRAN/C. They
were neither user friendly nor used good programming (because there
was not much computer science at that time). One most annoying thing
to me was that a function often took dozens of parameters with
impossible-to-decipher-and-remember names (because the concept of
“object” and data structure did not exist back then; names were limited
to eight characters). I often could not remember what the variables
meant, missed putting in some arguments, and messed up their orders.
They were not user friendly by modern standards.
The code was often very difficult to read. Good code should read like
English and should be easy to maintain by other professional
programmers without the author sitting next to them. When we coded
our math library, we made every effort to have the lines mirror the
paper so that another person can follow our logic by comparison. This
was so that others could fix our bugs, extend our work, translate it to
another language (e.g., C#), and port it to other hardware (e.g., GPU).
They needed to understand our code to do so. My opinion was that it
was next-to-impossible to do this with the decades’ old FORTRAN/C
code. I bet that once those authors were gone, no one could do anything
with that code other than run it, having faith that it had no bugs. I coded
up ALGORITHM AS 288 in Java from reading the original paper. I tried
to understand the AS 288 FORTRAN code (see Figure 1-1) for
comparison, but have still failed to do so to this date. The FORTRAN
code just seems to move variables around for no obvious reason and
somehow gets it right magically.
Figure 1-1 It is very hard to remember the meanings and order of A, B, C, K, N, X, Y
(AS108)
Our first attempt in building such a library was SuanShu, named
after one of the earliest known Chinese mathematical treatises, ​算数书,
which was written around 200 B.C. Since 2009, it was a decade of effort.
It started as a basic linear algebra package and grew to be an extensive
library covering Calculus, optimization algorithms, statistical analysis,
differential equations, and so on. It started being my personal hobby
and then dozens of experts, professors, PhDs, students, and
practitioners around the world contributed to the library. It was
adopted by some big-name commercial organizations as well as
academic institutions. In 2018, we decided to discontinue the sale and
support for SuanShu. We made the codebase open-source as of June
2012.
Leveraging our decade of experience with numerical programming,
NM Dev is a fresh new numerical library that succeeds SuanShu. We
have collected a large amount of feedback from users and bug reports
in the last ten years. We want to have a new software foundation and
architecture to account for all these good suggestions.

1.1 Library Design

The objective of SuanShu and hence NM Dev is to enable very easy
programming of engineering applications. Programmers can program
mathematics in such a way that the source code is solidly object-
oriented and individually testable. NM Dev source code adheres to the
strictest coding standards so that it is readable, maintainable, and can
be easily modified and extended.
NM Dev revolutionizes how numerical computing is traditionally
done, for example, Netlib and gsl. The repositories of these popular and
“standard” libraries are collections of ad hoc source code in old
languages, such as FORTRAN and C. One problem with such code is that
it is not readable (for most modern programmers), hence it’s
unmaintainable. For example, it is quite a challenge to understand AS
288, let alone improve on it. Other problems include, but are not
limited to, the lack of data structure, duplicated code, being entirely
procedural, very bad variable naming, abuse of GOTO, the lack of test
cases, insufficient documentation, the lack of IDE support, inconvenient
linking to modern languages such as Java, unfriendly to parallel
computing, and so on.
To address these problems, NM Dev designs a framework of
reusable math components (not procedures) so that programmers can
put components together like Legos to build more complex algorithms.
NM Dev is written anew so that it conforms to the modern
programming paradigms, such as variable naming, code structuring,
reusability, readability, maintainability, as well as software engineering
procedures. To ensure very high quality of the code and very few bugs,
NM Dev has thousands of unit test cases that run daily.

1.1.1 Class Parsimony

We decided to make the class library as parsimonious as possible to
avoid method pollution. This is inspired by the jMatrices’ whitepaper.1
The challenge is to organize the methods by minimal and correct
packages. We illustrate this with the NM Dev matrix package.
The Matrix class has only 26 methods, nine of which are
constructors and the related methods, three are overrides for the
AbstractMatrix interfaces, and eight are overrides for the
MatrixSpace interfaces. Only six of them are class-specific, to make
calling these methods convenient for the user. The other dozens of
matrix operations — such as the different factorizations, properties like
rank, and transformations like inverse — are grouped into multiple
classes and packages. In most cases, each of these operations is a class
on its own. For instance, the inverse operation is a class inheriting from
Matrix. The constructor takes as input the matrix to invert. For
example, to find the inverse for the following:

The NM Dev code in Kotlin is as follows:

val A = DenseMatrix(arrayOf(
doubleArrayOf(1.0, 2.0, 3.0),
doubleArrayOf(6.0, 5.0, 4.0),
doubleArrayOf(8.0, 7.0, 9.0)))
val Ainv = Inverse(A)
Ainv

The output is as follows:

3x3
[,1] [,2] [,3]
[1,] -0.809524, -0.142857, 0.333333,
[2,] 1.047619, 0.714286, -0.666667,
[3,] -0.095238, -0.428571, 0.333333,

It is important to note that the Ainv object is of the Matrix class;

it’s created as a new instance of the Inverse class. In summary, we
choose to have hundreds of classes, rather than to have one class with
hundreds of methods. Each class is kept deliberately short. This class
parsimony principle is a key design decision that guides the whole
library development.

1.1.2 Java vs C++ Performance

FORTRAN/C/C++ have traditionally been the languages for
mathematics because of their bare metal performance. Modern Java
performance is comparable to that of C++. There was the wrong
perception about Java slowness. The bad impression was by-and-large
because Java 1 in 1995 was indeed slower than C++. Java has improved
a lot since then, for example, hotspot. It is now version 17 LTS (or 18).
Java is now a competitive technology compared to C/C++. In fact, in
order to realistically optimize for C/C++, you need to find the “right”
programmer to code it. This programmer needs to be aware of all the
performance issues of C/C++, including profiling and code optimization
such as loop unfolding, and may even need to write code snippets in
assembly. An average programmer coding in C/C++ is probably not any
faster than coding in Java. (I am in general against code-optimization
techniques because they make the code unreadable to humans, hence
unmaintainable, such as a lot of the FORTRAN/C/C++ code found in
Netlib and Statlib.)
More importantly, most modern software runs on multiple cores.
Code-optimization techniques are dwarfed by parallel-computing
technologies. It is significantly easier and more efficient (and more
enjoyable) to write concurrent programming code in Java than in C++.
Therefore, to code high-performance software, I personally prefer to
code multi-core, multi-CPU, and cloud in Java rather than doing code
optimization in C/C++.
In general, among the general-purpose programming languages, I
like to use Java instead of C/C++, FORTRAN, Assembly, and so on,
whenever possible because Java is a very easy programming language
to learn and work with, without a sacrifice in performance. It has been
the most popular language for building systems and applications.
(As For FORTRAN, I am not sure why it survives in 2022. How are
you supposed to read thousands of lines of code all in upper- and
lowercase with a bunch of Cs and GOTOs everywhere?)
1.2 Java Scripting
To do data science and analysis, the main problem with using Java is
that you cannot work on the data in interactive mode. That is, I would
like to type a command to work on the data, such as to extract the first
six rows from a table, take a look at the result, think about what to do
next, and then type another command. The ability to work in
interactive mode makes MATLAB/R the most popular tool in data
analysis. See Figure 1-2.

Figure 1-2 Print out the first six rows in a data set in R

By contrast, when using Java, you always have to write, compile, and
then run the whole program. There are many problems with this
approach. First, it is very annoying to always have to repeatedly
compile, deploy, and execute the code. Second, Java is verbose. It has too
much boilerplate code. There are sections of unnecessary code that
have to be included in many places with little or no alteration. You need
to write a lot of code even for simple things like printing out a number.
Third, it is very time consuming to run the whole program from the
beginning each time. Fourth, you often do not know what program you
want to write without examining, understanding, and experimenting
with the data. Java does not allow programmers to try different things
with the data, such as look the first six rows versus the first hundred
rows. Fifth, you cannot pause a Java program in the middle of a run to
examine the intermediate results. In a nutshell, Java is not the right tool
for prototyping and developing data science code. You need a Java-
based scripting language that lets you interact with data.
There are a few Java-based scripting languages that fully
interoperate with existing Java code and libraries, most notably Clojure,
Scala, Groovy, and Kotlin. (I do not count JRuby and Jython because they
are simply Java implementation of Ruby and Python, respectively. Their
goals are to make Ruby and Python code run on JVM.) I do not like
Clojure, a dynamic and functional dialect of the LISP programming
language on the Java platform, because there are too many parentheses
to read and they hurt my eyes and brain.2 I can read English-like
sentences much faster than LISPs. Scala is a strong, statically typed
general-purpose programming language that supports both object-
oriented programming and functional programming. Designed to be
concise, many of Scala’s design decisions are aimed to address the
criticisms of Java. My problem with Scala is its learning curve. I cannot
figure out what the lines mean at the first glance, especially with all the
symbols like :, ::, :::. I want a language that I can pick up and use right
away without spending time learning it.
Groovy is easy to learn and read. I did not spend any time learning
it. I simply copied, pasted, and modified the code and was able to get
things working. Groovy is a Java-syntax-compatible object-oriented
programming language for the Java platform. It is both a static and
dynamic language with features similar to Python. It can be used as a
programming language and as a scripting language, is compiled to Java
Virtual Machine (JVM) bytecode, and interoperates seamlessly with
other Java code and libraries. Groovy uses a curly-bracket syntax
similar to Java’s. SuanShu used to support Groovy. For example, the
following two-line Groovy script evaluates a definite integral. See
Figure 1-3. More examples can be found in Figure 1-4.
I = new Riemann();//create an integrator
v = I.integrate(
['evaluate' : {x -> return x * x}] as
nivariateRealFunction,//integrate the function y =
x^2
0, 1);//limit: [0, 1]

Figure 1-3 Evaluate a definite integral using Groovy and SuanShu

Figure 1-4 Matrix operations using Groovy and SuanShu
Unfortunately, the company Pivotal Software ceased sponsoring
Groovy in April 2015. I was looking for an alternative that was
motivated by commercial interest with better resources and support.

1.2.1 Kotlin Programming Language

Kotlin was invented by the company JetBrains. It is a cross-platform,
statically typed, general-purpose programming language with type
inference. Kotlin is designed to interoperate fully with Java and existing
Java libraries. Kotlin mainly targets JVM, but it also compiles to
JavaScript (e.g., for frontend web applications using React) and native
code (e.g., for native iOS apps sharing business logic with Android
apps).
On May 7, 2019, Google announced that Kotlin was its preferred
language for Android app developers. It was the third language fully
supported for Android, in addition to Java and C++. In 2020, Kotlin was
the most widely used on Android, with Google estimating that 70% of
the top 1000 apps on the Play Store were written in Kotlin. Google itself
has 60 apps written in Kotlin, including Maps and Drive. Many Android
apps, such as Google’s Home, are in the process of being migrated to
Kotlin.
In addition to its prominent use on Android, Kotlin is gaining
traction in server-side development. The Spring Framework officially
added Kotlin support with version 5 on January 4, 2017. To further
support Kotlin, Spring has translated all its documentation to Kotlin
and added built-in support for many Kotlin-specific features, such as
coroutines. In addition to Spring, JetBrains has produced a Kotlin-first
framework called Ktor for building web applications.
With all the commercial interests and community momentum,
Kotlin is likely to remain a mainstream language and be well supported.
Kotlin will likely (and hopefully) stay for a very long time. Therefore, I
chose Kotlin as the scripting language to do math computing and data
analysis.
One major advantage of Kotlin over Java is that Kotlin code is
concise. It has little boilerplate code, is shorter and more readable. Also,
Kotlin is very easy to learn. I have never read any tutorials on the
Internet or books about Kotlin. I simply copy, paste, and modify existing
snippets. This approach seems to work for me and proves how easy
Kotlin is to use and read. In fact, all sample code in this book was coded
in this fashion.
For example, to add two numbers, you do the following:

1 + 1

The output is as follows:

To multiply two numbers, you do this:

2 * 2

The output is as follows:

4
To define a constant, you do the following:

val A = 1.0

To print out a variable, you simply type its name. For example:

The output is as follows:

1.0

To define a variable, you do the following:

var x : Double = 1.0

This statement says that x is a variable whose value can change. It is

of type Double.
To change x, you can add 1 to it.

X = x + 1
x

The output is as follows:

2.0

A for loop is, for example:

for (i in 1..3) {
println(i)
}

The output is as follows:

1
2
3
 The following snippet defines an array and prints it out.

Val arr1 = arrayOf(1.0, 2.0, 3.0, 4.0, 5.0)

for (I in arr1) {
println(i)
}

The output is as follows:

1.0
2.0
3.0
4.0
5.0

The following snippet defines a function:

fun add1(x : Double) : Double {

return x + 1.0
}

To call the function, you do the following:

add1(2.0)

The output is as follows:

3.0

There are many good books and tutorials on the Internet to learn
how to code Kotlin. The official Kotlin Foundation website has a lot of
information about Kotlin, its development, news, and community.
https://kotlinlang.org
They also have an online course to get beginners started on coding
Kotlin.
https://kotlinlang.org/docs/getting-
started.xhtml
1.3 S2
S2 is an online integrated development environment of data and
analytics for data science and numerical computing. It is developed by
the company NM DEV PTE LTD. The official S2 website is:
http://nm.dev/s2
S2 contains all numerical algorithms in the NM Dev library and
some mathematics, science, machine learning, graphing, plotting,
financial, and other commercial libraries. S2 lets users create their
math algorithms by calling all these built-in libraries in a simple
manner using Kotlin.

1.3.1 R Replacement
S2 is designed to be a modern-day replacement for R. You can do a lot
of things in S2 that you can do in R and S2 does them faster. Moreover, R
code runs inside only the R environment. It is very difficult to deploy
the code anywhere else such as in embedded devices like microwaves,
automobiles, or space rockets. S2 code runs in any JVM environment.
There are now 15 billion devices that run JVM!
S2 is an online IDE for numerical computing and coding numerical
algorithms. It is like an online calculator. Let’s start with 1+1, as shown
in Figure 1-5.

Figure 1-5 S2 1+1

With 1+1 working, you can do pretty much anything in numerical

programming, which is just a more complicated series of many 1+1
computations. For example, integration is shown in Figure 1-6.
Random documents with unrelated
content Scribd suggests to you:
accessible by the widest array of equipment including outdated
equipment. Many small donations ($1 to $5,000) are particularly
important to maintaining tax exempt status with the IRS.

The Foundation is committed to complying with the laws

regulating charities and charitable donations in all 50 states of
the United States. Compliance requirements are not uniform
and it takes a considerable effort, much paperwork and many
fees to meet and keep up with these requirements. We do not
solicit donations in locations where we have not received written
confirmation of compliance. To SEND DONATIONS or
determine the status of compliance for any particular state visit
www.gutenberg.org/donate.

While we cannot and do not solicit contributions from states

where we have not met the solicitation requirements, we know
of no prohibition against accepting unsolicited donations from
donors in such states who approach us with offers to donate.

International donations are gratefully accepted, but we cannot

make any statements concerning tax treatment of donations
received from outside the United States. U.S. laws alone swamp
our small staff.

Please check the Project Gutenberg web pages for current

donation methods and addresses. Donations are accepted in a
number of other ways including checks, online payments and
credit card donations. To donate, please visit:
www.gutenberg.org/donate.

Section 5. General Information About Project

Gutenberg™ electronic works
Professor Michael S. Hart was the originator of the Project
Gutenberg™ concept of a library of electronic works that could
be freely shared with anyone. For forty years, he produced and
distributed Project Gutenberg™ eBooks with only a loose
network of volunteer support.

Project Gutenberg™ eBooks are often created from several

printed editions, all of which are confirmed as not protected by
copyright in the U.S. unless a copyright notice is included. Thus,
we do not necessarily keep eBooks in compliance with any
particular paper edition.

Most people start at our website which has the main PG search
facility: www.gutenberg.org.

This website includes information about Project Gutenberg™,

including how to make donations to the Project Gutenberg
Literary Archive Foundation, how to help produce our new
eBooks, and how to subscribe to our email newsletter to hear
about new eBooks.
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

ebookmass.com