Applied Functional Data Analysis Methods and Case Studies

Visit the link below to download the full version of this book:

https://medidownload.com/product/applied-functional-data-analysis-methods-and-ca
se-studies/

Click Download Now

Applied Functional Data Analysis:
Methods and Case Studies
James O. Ramsay and Bernard W. Silverman
This page intentionally left blank
Preface

Almost as soon as we had completed our previous book Functional Data

Analysis in 1997, it became clear that potential interest in the field was
far wider than the audience for the thematic presentation we had given
there. At the same time, both of us rapidly became involved in relevant
new research involving many colleagues in fields outside statistics.
This book treats the field in a different way, by considering case stud-
ies arising from our own collaborative research to illustrate how functional
data analysis ideas work out in practice in a diverse range of subject areas.
These include criminology, economics, archaeology, rheumatology, psychol-
ogy, neurophysiology, auxology (the study of human growth), meteorology,
biomechanics, and education—and also a study of a juggling statistician.
Obviously such an approach will not cover the field exhaustively, and
in any case functional data analysis is not a hard-edged closed system of
thought. Nevertheless we have tried to give a flavor of the range of method-
ology we ourselves have considered. We hope that our personal experience,
including the fun we had working on these projects, will inspire others to
extend “functional” thinking to many other statistical contexts. Of course,
many of our case studies required development of existing methodology, and
readers should gain the ability to adapt methods to their own problems too.
No previous knowledge of functional data analysis is needed to read this
book, and although it complements our previous book in some ways, neither
is a prerequisite for the other. We hope it will be of interest, and accessi-
ble, both to statisticians and to those working in other fields. Similarly, it
should appeal both to established researchers and to students coming to
the subject for the first time.
vi Preface

Functional data analysis is very much involved with computational

statistics, but we have deliberately not written a computer manual
or cookbook. Instead, there is an associated Web site accessible from
www.springer-ny.com giving annotated analyses of many of the data sets,
as well as some of the data themselves. The languages of these analyses are
MATLAB, R, or S-PLUS, but the aim of the analyses is to explain the
computational thinking rather than to provide a package, so they should
be useful for those who use other languages too. We have, however, freely
used a library of functions that we developed in these languages, and these
may be downloaded from the Web site.
In both our books, we have deliberately set out to present a personal
account of this rapidly developing field. Some specialists will, no doubt,
notice omissions of the kind that are inevitable in this kind of presenta-
tion, or may disagree with us about the aspects to which we have given
most emphasis. Nevertheless, we hope that they will find our treatment in-
teresting and stimulating. One of our reasons for making the data, and the
analyses, available on the Web site is our wish that others may do better.
Indeed, may others write better books!
There are many people to whom we are deeply indebted. Particular ac-
knowledgment is due to the distinguished paleopathologist Juliet Rogers,
who died just before the completion of this book. Among much other re-
search, Juliet’s long-term collaboration with BWS gave rise to the studies
in Chapters 4 and 8 on the shapes of the bones of arthritis sufferers of many
centuries ago. Michael Newton not only helped intellectually, but also gave
us some real data by allowing his juggling to be recorded for analysis in
Chapter 12. Others whom we particularly wish to thank include Darrell
Bock, Virginia Douglas, Zmira Elbaz-King, Theo Gasser, Vince Gracco,
Paul Gribble, Michael Hermanussen, John Kimmel, Craig Leth-Steenson,
Xiaochun Li, Nicole Malfait, David Ostry, Tim Ramsay, James Ramsey,
Natasha Rossi, Lee Shepstone, Matthew Silverman, and Xiaohui Wang.
Each of them made a contribution essential to some aspect of the work
we report, and we apologize to others we have neglected to mention by
name. We are very grateful to the Stanford Center for Advanced Study
in the Behavioral Sciences, the American College Testing Program, and to
the McGill students in the Psychology 747A seminar on functional data
analysis. We also thank all those who provided comments on our software
and pointed out problems.

Montreal, Quebec, Canada Jim Ramsay

Bristol, United Kingdom Bernard Silverman
January 2002
Contents

Preface v

1 Introduction 1
1.1 Why consider functional data at all? . . . . . . . . . . . 1
1.2 The Web site . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 The case studies . . . . . . . . . . . . . . . . . . . . . . . 2
1.4 How is functional data analysis distinctive? . . . . . . . . 14
1.5 Conclusion and bibliography . . . . . . . . . . . . . . . . 15

2 Life Course Data in Criminology 17

2.1 Criminology life course studies . . . . . . . . . . . . . . . 17
2.1.1 Background . . . . . . . . . . . . . . . . . . . . . 17
2.1.2 The life course data . . . . . . . . . . . . . . . . . 18
2.2 First steps in a functional approach . . . . . . . . . . . . 19
2.2.1 Turning discrete values into a functional datum . 19
2.2.2 Estimating the mean . . . . . . . . . . . . . . . . 21
2.3 Functional principal component analyses . . . . . . . . . 23
2.3.1 The basic methodology . . . . . . . . . . . . . . . 23
2.3.2 Smoothing the PCA . . . . . . . . . . . . . . . . 26
2.3.3 Smoothed PCA of the criminology data . . . . . 26
2.3.4 Detailed examination of the scores . . . . . . . . 28
2.4 What have we seen? . . . . . . . . . . . . . . . . . . . . . 31
viii Contents

2.5 How are functions stored and processed? . . . . . . . . . 33

2.5.1 Basis expansions . . . . . . . . . . . . . . . . . . 33
2.5.2 Fitting basis coefficients to the observed data . . 35
2.5.3 Smoothing the sample mean function . . . . . . . 36
2.5.4 Calculations for smoothed functional PCA . . . . 37
2.6 Cross-validation for estimating the mean . . . . . . . . . 38
2.7 Notes and bibliography . . . . . . . . . . . . . . . . . . . 40

3 The Nondurable Goods Index 41

3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.2 Transformation and smoothing . . . . . . . . . . . . . . . 43
3.3 Phase-plane plots . . . . . . . . . . . . . . . . . . . . . . 44
3.4 The nondurable goods cycles . . . . . . . . . . . . . . . . 47
3.5 What have we seen? . . . . . . . . . . . . . . . . . . . . . 54
3.6 Smoothing data for phase-plane plots . . . . . . . . . . . 55
3.6.1 Fourth derivative roughness penalties . . . . . . . 55
3.6.2 Choosing the smoothing parameter . . . . . . . . 55

4 Bone Shapes from a Paleopathology Study 57

4.1 Archaeology and arthritis . . . . . . . . . . . . . . . . . . 57
4.2 Data capture . . . . . . . . . . . . . . . . . . . . . . . . . 58
4.3 How are the shapes parameterized? . . . . . . . . . . . . 59
4.4 A functional principal components analysis . . . . . . . . 61
4.4.1 Procrustes rotation and PCA calculation . . . . . 61
4.4.2 Visualizing the components of shape variability . 61
4.5 Varimax rotation of the principal components . . . . . . 63
4.6 Bone shapes and arthritis: Clinical relationship? . . . . . 65
4.7 What have we seen? . . . . . . . . . . . . . . . . . . . . . 66
4.8 Notes and bibliography . . . . . . . . . . . . . . . . . . . 66

5 Modeling Reaction-Time Distributions 69

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 69
5.2 Nonparametric modeling of density functions . . . . . . . 71
5.3 Estimating density and individual differences . . . . . . . 73
5.4 Exploring variation across subjects with PCA . . . . . . 76
5.5 What have we seen? . . . . . . . . . . . . . . . . . . . . . 79
5.6 Technical details . . . . . . . . . . . . . . . . . . . . . . . 80

6 Zooming in on Human Growth 83

6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 83
6.2 Height measurements at three scales . . . . . . . . . . . 84
6.3 Velocity and acceleration . . . . . . . . . . . . . . . . . . 86
6.4 An equation for growth . . . . . . . . . . . . . . . . . . . 89
6.5 Timing or phase variation in growth . . . . . . . . . . . . 91
6.6 Amplitude and phase variation in growth . . . . . . . . . 93
 Contents ix

6.7 What we have seen? . . . . . . . . . . . . . . . . . . . . . 96

6.8 Notes and further issues . . . . . . . . . . . . . . . . . . 97
6.8.1 Bibliography . . . . . . . . . . . . . . . . . . . . . 97
6.8.2 The growth data . . . . . . . . . . . . . . . . . . 98
6.8.3 Estimating a smooth monotone curve to fit data . 98

7 Time Warping Handwriting and Weather Records 101

7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 101
7.2 Formulating the registration problem . . . . . . . . . . . 102
7.3 Registering the printing data . . . . . . . . . . . . . . . . 104
7.4 Registering the weather data . . . . . . . . . . . . . . . . 105
7.5 What have we seen? . . . . . . . . . . . . . . . . . . . . . 110
7.6 Notes and references . . . . . . . . . . . . . . . . . . . . 110
7.6.1 Continuous registration . . . . . . . . . . . . . . . 110
7.6.2 Estimation of the warping function . . . . . . . . 113

8 How Do Bone Shapes Indicate Arthritis? 115

8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 115
8.2 Analyzing shapes without landmarks . . . . . . . . . . . 116
8.3 Investigating shape variation . . . . . . . . . . . . . . . . 120
8.3.1 Looking at means alone . . . . . . . . . . . . . . 120
8.3.2 Principal components analysis . . . . . . . . . . . 120
8.4 The shape of arthritic bones . . . . . . . . . . . . . . . . 123
8.4.1 Linear discriminant analysis . . . . . . . . . . . . 123
8.4.2 Regularizing the discriminant analysis . . . . . . 125
8.4.3 Why not just look at the group means? . . . . . . 127
8.5 What have we seen? . . . . . . . . . . . . . . . . . . . . . 128
8.6 Notes and further issues . . . . . . . . . . . . . . . . . . 128
8.6.1 Bibliography . . . . . . . . . . . . . . . . . . . . . 128
8.6.2 Why is regularization necessary? . . . . . . . . . 129
8.6.3 Cross-validation in classification problems . . . . 130

9 Functional Models for Test Items 131

9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 131
9.2 The ability space curve . . . . . . . . . . . . . . . . . . . 132
9.3 Estimating item response functions . . . . . . . . . . . . 135
9.4 PCA of log odds-ratio functions . . . . . . . . . . . . . . 136
9.5 Do women and men perform differently on this test? . . 138
9.6 A nonlatent trait: Arc length . . . . . . . . . . . . . . . . 140
9.7 What have we seen? . . . . . . . . . . . . . . . . . . . . . 143
9.8 Notes and bibliography . . . . . . . . . . . . . . . . . . . 143

10 Predicting Lip Acceleration from Electromyography 145

10.1 The neural control of speech . . . . . . . . . . . . . . . . 145
10.2 The lip and EMG curves . . . . . . . . . . . . . . . . . . 147
x Contents

10.3 The linear model for the data . . . . . . . . . . . . . . . 148

10.4 The estimated regression function . . . . . . . . . . . . . 150
10.5 How far back should the historical model go? . . . . . . 152
10.6 What have we seen? . . . . . . . . . . . . . . . . . . . . . 155
10.7 Notes and bibliography . . . . . . . . . . . . . . . . . . . 155

11 The Dynamics of Handwriting Printed Characters 157

11.1 Recording handwriting in real time . . . . . . . . . . . . 157
11.2 An introduction to dynamic models . . . . . . . . . . . . 158
11.3 One subject’s printing data . . . . . . . . . . . . . . . . . 160
11.4 A differential equation for handwriting . . . . . . . . . . 162
11.5 Assessing the fit of the equation . . . . . . . . . . . . . . 165
11.6 Classifying writers by using their dynamic equations . . 166
11.7 What have we seen? . . . . . . . . . . . . . . . . . . . . . 170

12 A Differential Equation for Juggling 171

12.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 171
12.2 The data and preliminary analyses . . . . . . . . . . . . 172
12.3 Features in the average cycle . . . . . . . . . . . . . . . . 173
12.4 The linear differential equation . . . . . . . . . . . . . . 176
12.5 What have we seen? . . . . . . . . . . . . . . . . . . . . . 180
12.6 Notes and references . . . . . . . . . . . . . . . . . . . . 181

References 183

Index 187
1
Introduction

1.1 Why consider functional data at all?

Functional data come in many forms, but their defining quality is that
they consist of functions—often, but not always, smooth curves. In this
book, we consider functional data arising in many different fields, ranging
from the shapes of bones excavated by archaeologists, to economic data
collected over many years, to the path traced out by a juggler’s finger. The
fundamental aims of the analysis of functional data are the same as those
of more conventional statistics: to formulate the problem at hand in a way
amenable to statistical thinking and analysis; to develop ways of presenting
the data that highlight interesting and important features; to investigate
variability as well as mean characteristics; to build models for the data
observed, including those that allow for dependence of one observation or
variable on another, and so on.
We have chosen case studies to cover a wide range of fields of application,
and one of our aims is to demonstrate how large is the potential scope
of functional data analysis. If you work through all the case studies you
will have covered a broad sweep of existing methods in functional data
analysis and, in some cases, you will study new methodology developed for
the particular problem in hand. But more importantly, we hope that the
readers will gain an insight into functional ways of thinking.
What sort of data come under the general umbrella of functional data?
In some cases, the original observations are interpolated from longitudi-
nal data, quantities observed as they evolve through time. However, there
2 1. Introduction

are many other ways that functional data can arise. For instance, in our
study of children with attention deficit hyperactivity disorder, we take a
large number of independent numerical observations for each child, and
the functional datum for that child is the estimated probability density of
these observations. Sometimes our data are curves traced out on a surface
or in space. The juggler’s finger directly traces out the data we analyze in
that case, but in another example, on the characteristics of examination
questions, the functional data arise as part of the modeling process. In the
archaeological example, the shape of a two-dimensional image of each bone
is the functional datum in question. And of course images as well as curves
can appear as functional data or as functional parameters in models, as we
show in our study of electromyography recordings and speech articulation.
The field of functional data analysis is still in its infancy, and the bound-
aries between functional data analysis and other aspects of statistics are
definitely fuzzy. Part of our aim in writing this book is to encourage read-
ers to develop further the insights—both statistically and in the various
subject areas from which the data come—that can be gained by thinking
about appropriate data from a functional point of view. Our own view
about what is distinctive about functional data analysis should be gained
primarily from the case studies we discuss, as summarized in Section 1.3,
but some specific remarks are made in Section 1.4 below.

1.2 The Web site

Working through examples for oneself leads to deeper insight, and is an
excellent way into applying and adapting methods to one’s own data. To
help this process, there is a Web site associated with the text. The Web
site contains many of the data sets and analyses discussed in the book.
These analyses are not intended as a package or as a “cookbook”, but our
hope is that they will help readers follow the steps that we went through
in carrying out the analyses presented in the case studies. Some of the
analyses were carried out in MATLAB and some in S-PLUS.
At the time of printing the Web site is linked to the Springer Web site
at www.springer-ny.com.

1.3 The case studies

In this section, the case studies are briefly reviewed. Further details of
the context of the data sets, and appropriate bibliographic references, are
given in the individual chapters where the case studies are considered in
full. In most of them, in addition to the topics explicitly mentioned below,
there is some discussion of computational issues and other fine points of
 1.3. The case studies 3

Root annual number of offenses

3
2
1
0

10 15 20 25 30 35

Age

Figure 1.1. The functional datum corresponding to a particular individual in the

criminology sample; it shows the way that the annual square root number of
crimes varies over the life course.

methodology. In some chapters, we develop or explain some material that

will be mainly of interest to statistical experts. These topics are set out in
sections towards the end of the relevant chapter, and can be safely skipped
by the more general reader.

Chapter 2: Life course data in criminology

We study data on the criminal careers of over 400 individuals followed
over several decades of their lifespan. For each individual a function is
constructed over the interval [11, 35], representing that person’s level of
criminal activity between ages 11 and 35. For reasons that are explained, it
is appropriate to track the square root of the number of crimes committed
each year, and a typical record is given in Figure 1.1. Altogether we consider
413 records like this one, and the records are all plotted in Figure 1.2.
This figure demonstrates little more than the need for careful methods of
summarizing and analyzing collections of functional data.
Data of this kind are the simplest kind of functional data: we have a
number of independent individuals, for each of whom we observe a sin-
gle function. In standard statistics, we are accustomed to the notion of
a sequence of independent numerical observations. This is the functional
equivalent: a sequence of independent functional observations.
4 1. Introduction

Root annual number of offenses

3
2
1
0

10 15 20 25 30 35

Age

Figure 1.2. The functional data for all 413 subjects in the criminology study.

The questions we address in Chapter 2 include the following.

• What are the steps involved in making raw data on an individual’s
criminal record into a continuous functional observation?
• How should we estimate the mean of a population such as that in
Figure 1.2, and how can we investigate its variability?
• Are there distinct groups of offenders, or do criminals reside on more
of a continuum?
• How does our analysis point to salient features of particular data? Of
particular interest to criminologists are those individuals who are ju-
venile offenders who subsequently mature into reasonably law-abiding
citizens.
The answers to the third and fourth questions address controversial issues
in criminology; it is of obvious importance if there is a “criminal frater-
nity” with a distinct pattern of offending, and it is also important to know
whether reform of young offenders is possible. Quantifying reform is a key
step towards this goal.

Chapter 3: The nondurable goods index

In Chapter 3 we turn to a single economic series observed over a long
period of time, the U.S. index of nondurable goods production, as plotted
 1.3. The case studies 5

120
100
80
Index

60
40
20

1920 1940 1960 1980 2000

Year

Figure 1.3. The nondurable goods index over the period 1919 to 2000.

1923 1996
j
6

D
J
4

4
Acceleration

Acceleration

J M
A
2

D
A N j
A j
-2 0

-2 0

O M
j O mF
F A
N S
m
J J
S
-6

-6

-0.3 -0.1 0.1 0.3 -0.3 -0.1 0.1 0.3

Velocity Velocity

Figure 1.4. Phase-plane plots for two contrasting years: left 1923, right 1996.