Earth Observation Using Python A Practical

Programming Guide Rebekah B Esmaili download

https://ebookbell.com/product/earth-observation-using-python-a-
practical-programming-guide-rebekah-b-esmaili-46457136

Explore and download more ebooks at ebookbell.com

Here are some recommended products that we believe you will be
interested in. You can click the link to download.

Earth Observation Data Analytics Using Machine And Deep Learning

Modern Tools Applications And Challenges 1st Edition Sanjay Garg

https://ebookbell.com/product/earth-observation-data-analytics-using-
machine-and-deep-learning-modern-tools-applications-and-
challenges-1st-edition-sanjay-garg-50558476

The Value Of Using Hydrological Datasets For Water Allocation

Decisions Earth Observations Hydrological Models And Seasonal
Forecasts 1st Edition Alexander Jos Kaune Schmidt Author

https://ebookbell.com/product/the-value-of-using-hydrological-
datasets-for-water-allocation-decisions-earth-observations-
hydrological-models-and-seasonal-forecasts-1st-edition-alexander-jos-
kaune-schmidt-author-11911748

Earth Observation Science And Applications For Risk Reduction And

Enhanced Resilience In Hindu Kush Himalaya Region A Decade Of
Experience From Servir 1st Edition Birendra Bajracharya

https://ebookbell.com/product/earth-observation-science-and-
applications-for-risk-reduction-and-enhanced-resilience-in-hindu-kush-
himalaya-region-a-decade-of-experience-from-servir-1st-edition-
birendra-bajracharya-51700146

Earth Observation Of Wildland Fires In Mediterranean Ecosystems 1st

Edition Emilio Chuvieco Auth

https://ebookbell.com/product/earth-observation-of-wildland-fires-in-
mediterranean-ecosystems-1st-edition-emilio-chuvieco-auth-2171122
Earth Observation With Champ Results From Three Years In Orbit 1st
Edition Christoph Reigber

https://ebookbell.com/product/earth-observation-with-champ-results-
from-three-years-in-orbit-1st-edition-christoph-reigber-2202872

Earth Observation Science And Applications For Risk Reduction And

Enhanced Resilience In Hindu Kush Himalaya Region Birendra Bajracharya

https://ebookbell.com/product/earth-observation-science-and-
applications-for-risk-reduction-and-enhanced-resilience-in-hindu-kush-
himalaya-region-birendra-bajracharya-38403946

Earth Observation Of Global Change The Role Of Satellite Remote

Sensing In Monitoring The Global Environment 1st Edition Beatriz
Alonso

https://ebookbell.com/product/earth-observation-of-global-change-the-
role-of-satellite-remote-sensing-in-monitoring-the-global-
environment-1st-edition-beatriz-alonso-4390370

Earth Observation Of Global Changes Eogc 1st Edition Roland Pail Auth

https://ebookbell.com/product/earth-observation-of-global-changes-
eogc-1st-edition-roland-pail-auth-4404798

Earth Observation Applications And Global Policy Frameworks Argyro

Kavvada

https://ebookbell.com/product/earth-observation-applications-and-
global-policy-frameworks-argyro-kavvada-44746210
 Earth Observation Using Python

0005078634.3D 1 12/7/2021 12:32:26 PM

 Special Publications 75

EARTH OBSERVATION
USING PYTHON
A Practical Programming Guide
Rebekah B. Esmaili

This Work is a co-publication of

the American Geophysical Union and John Wiley and Sons, Inc.

0005078634.3D 3 12/7/2021 12:32:26 PM

 This edition first published 2021
© 2021 American Geophysical Union

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted,
in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as
permitted by law. Advice on how to obtain permission to reuse material from this title is available at
http://www.wiley.com/go/permissions.

Published under the aegis of the AGU Publications Committee

Brooks Hanson, Executive Vice President, Science
Carol Frost, Chair, Publications Committee
For details about the American Geophysical Union visit us at www.agu.org.

The right of Rebekah B. Esmaili to be identified as the author of this work has been asserted in accordance with law.

Registered Office
John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, USA

Editorial Office
111 River Street, Hoboken, NJ 07030, USA

For details of our global editorial offices, customer services, and more information about Wiley products visit us at
www.wiley.com.

Wiley also publishes its books in a variety of electronic formats and by print-on-demand. Some content that appears
in standard print versions of this book may not be available in other formats.

Limit of Liability/Disclaimer of Warranty

While the publisher and authors have used their best efforts in preparing this work, they make no representations or
warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all
warranties, including without limitation any implied warranties of merchantability or fitness for a particular purpose.
No warranty may be created or extended by sales representatives, written sales materials or promotional statements
for this work. The fact that an organization, website, or product is referred to in this work as a citation and/or
potential source of further information does not mean that the publisher and authors endorse the information or
services the organization, website, or product may provide or recommendations it may make. This work is sold with
the understanding that the publisher is not engaged in rendering professional services. The advice and strategies
contained herein may not be suitable for your situation. You should consult with a specialist where appropriate.
Further, readers should be aware that websites listed in this work may have changed or disappeared between when this
work was written and when it is read. Neither the publisher nor authors shall be liable for any loss of profit or any
other commercial damages, including but not limited to special, incidental, consequential, or other damages.

Library of Congress Cataloging-in-Publication Data

Name: Esmaili, Rebekah Bradley, author.
Title: Earth observation using Python : a practical programming guide /
Rebekah B. Esmaili.
Description: Hoboken, NJ : Wiley, [2021] | Includes bibliographical
references and index.
Identifiers: LCCN 2021001631 (print) | LCCN 2021001632 (ebook) | ISBN
9781119606888 (hardback) | ISBN 9781119606895 (adobe pdf) | ISBN
9781119606918 (epub)
Subjects: LCSH: Earth sciences–Data processing. | Remote sensing–Data
processing. | Python (Computer program language) | Information
visualization. | Artificial satellites in earth sciences. | Earth
sciences–Methodology.
Classification: LCC QE48.8 .E85 2021 (print) | LCC QE48.8 (ebook) | DDC
550.285/5133–dc23
LC record available at https://lccn.loc.gov/2021001631
LC ebook record available at https://lccn.loc.gov/2021001632

Cover Design: Wiley

Cover Image: © NASA

Set in 10/12pt Times New Roman by Straive, Pondicherry, India

10 9 8 7 6 5 4 3 2 1

0005078634.3D 4 12/7/2021 12:32:26 PM

 CONTENTS

Foreword ......................................................................................................... vii

Acknowledgments ............................................................................................ ix

Introduction ...................................................................................................... 1
Part I: Overview of Satellite Datasets ............................................................... 5
1 A Tour of Current Satellite Missions and Products .................................. 7
2 Overview of Python................................................................................ 17
3 A Deep Dive into Scientific Data Sets.................................................... 25

Part II: Practical Python Tutorials for Remote Sensing................................... 45

4 Practical Python Syntax .......................................................................... 47
5 Importing Standard Earth Science Datasets ........................................... 67
6 Plotting and Graphs for All.....................................................................95
7 Creating Effective and Functional Maps...............................................125
8 Gridding Operations.............................................................................155
9 Meaningful Visuals through Data Combination ................................... 177
10 Exporting with Ease ..............................................................................207

Part III: Effective Coding Practices ...............................................................219

11 Developing a Workflow .......................................................................221
12 Reproducible and Shareable Science ...................................................239

Conclusion..................................................................................................... 253

Appendix A: Installing Python.......................................................................255

Appendix B: Jupyter Notebook .....................................................................259
Appendix C: Additional Learning Resources.................................................267
Appendix D: Tools......................................................................................... 269
Appendix E: Finding, Accessing, and Downloading Satellite Datasets .........271
Appendix F: Acronyms ..................................................................................279

Index.............................................................................................................. 283

0005078635.3D 5 12/7/2021 12:35:39 PM

 FOREWORD

When I first met the author a few years ago, she was eager to become more
involved in the Joint Polar Satellite System’s Proving Ground. The Proving
Ground by definition assesses the impact of a product in the user’s environment;
this intrigued Rebekah because as a product developer, she wanted to understand
the user’s perspective. Rebekah worked with the National Weather Service to
demonstrate how satellite-derived atmospheric temperature and water vapor
soundings can be used to describe the atmosphere’s instability to support severe
weather warnings. Rebekah spent considerable time with users at the Storm Pre-
diction Center in Norman, Oklahoma, to understand their needs, and she found
their thirst for data and the need for data to be easily visualized and understand-
able. This is where Rebekah leveraged her expert skills in Python to provide NWS
with the information they found to be most useful. Little did I know at the time she
was writing a book.
As noted in this book, a myriad of Earth-observing satellites collect critical
information of the Earth’s complex and ever-changing environment and land-
scape. However, today, unfortunately, all that information is not effectively being
used for various reasons: issues with data access, different data formats, and the
need for better tools for data fusion and visualization. If we were able to solve these
problems, then suddenly there would be vast improvements in providing societies
with the information needed to support decisions related to weather and climate
and their impacts, including high-impact weather events, droughts, flooding, wild-
fires, ocean/coastal ecosystems, air quality, and more. Python is becoming the uni-
versal language to bridge these various data sources and translate them into useful
information. Open and free attributes, and the data and code sharing mindset of
the Python communities, make Python very appealing.
Being involved in a number of international collaborations to improve the
integration of Earth observations, I can certainly emphasize the importance
of working together, data sharing, and demonstrating the value of data
fusion. I am very honored to write this Foreword, since this book focuses on these

vii

0005078636.3D 7 14/6/2021 5:41:00 PM

 viii Foreword

issues and provides an excellent guide with relevant examples for the reader to
follow and relate to.

Dr. Mitch Goldberg

Chief Program Scientist
NOAA-National Environmental Satellite, Data,
and Information Service
June 22, 2020

0005078636.3D 8 14/6/2021 5:41:00 PM

 ACKNOWLEDGMENTS

This book evolved from a series of Python workshops that I developed with
the help of Eviatar Bach and Kriti Bhargava from the Department of Atmospheric
and Oceanic Science at the University of Maryland. I am very grateful for their
assistance providing feedback for the examples in this book and for leading several
of these workshops with me.
This book would not exist without their support and contributions from
others, including:
The many reviewers who took the time to read versions of this book, several of
whom I have never met in person. Thanks to modern communication systems,
I was able to draw from their expertise. Their constructive feedback and insights
not only helped to improve this quality and breadth of the book but also helped me
hone my technical writing skills.
Rituparna Bose, Jenny Lunn, Layla Harden, and the rest of the team at AGU
and Wiley for keeping me informed, organized, and on track throughout this
process. They were truly a pleasure to work with.
Nadia Smith and Chris Barnet, and my other colleagues at Science and Tech-
nology Corp., who provided both feedback and conversations that helped shape
some of the ideas and content in this book.
Catherine Thomas, Clare Flynn, Erin Lynch, and Amy Ho for their endless
encouragement and support.
Tracie and Farid Esmaili, my parents, who encouraged me to aim high even if
they were initially confused when their atmospheric scientist daughter became
interested in “snakes.”

ix

0005078636.3D 9 14/6/2021 5:41:00 PM

 INTRODUCTION

Python is a programming language that is rapidly growing in popularity. The

number of users is large, although difficult to quantify; in fact, Python is currently
the most tagged language on stackoverflow.com, a coding Q&A website with
approximately 3 million questions a year. Some view this interest as hype, but
there are many reasons to join the movement. Scientists are embracing Python
because it is free, open source, easy to learn, and has thousands of add-on
packages. Many routine tasks in the Earth sciences have already been coded
and stored in off-the-shelf Python libraries. Users can download these libraries
and apply them to their research rather than simply using older, more primitive
functions. The widespread adoption of Python means scientists are moving toward
a common programming language and set of tools that will improve code share-
ability and research reproducibility.
Among the wealth of remote sensing data available, satellite datasets are par-
ticularly voluminous and tend to be stored in a variety of binary formats. Some
datasets conform to a “standard” structure, such as netCDF4. However, because
of uncoordinated efforts across different agencies and countries, such standard
formats bear their own inconsistencies in how data are handled and intended to
be displayed. To address this, many agencies and companies have developed
numerous “quick look” methods. For instance, data can be searched for and
viewed online as Jpeg images, or individual files can be displayed with free,
open-source software tools like Panoply (www.giss.nasa.gov/tools/panoply/)
and HDFView (www.hdfgroup.org/downloads/hdfview/).
Still, scientists who wish to execute more sophisticated visualization techni-
ques will have to learn to code. Coding knowledge is not the only limitation for
users. Not all data are “analysis ready,” i.e., in the proper input format for visu-
alization tools. As such, many pre-processing steps are required to make the data
usable for scientific analysis. This is particularly evident for data fusion, where two
datasets with different resolutions must first be mapped to the same grid before
they are compared. Many data users are not satellite scientists or professional

Earth Observation Using Python: A Practical Programming Guide, Special Publications 75,
First Edition. Rebekah B. Esmaili.
© 2021 American Geophysical Union. Published 2021 by John Wiley & Sons, Inc.
DOI: 10.1002/9781119606925.introduction

cintro.3d 1 13/7/2021 9:13:41 AM

 2 Earth Observation Using Python

programmers but rather members of other research and professional communities,

these barriers can be too great to overcome. Even to a technical user, the nuances
can be frustrating. At worst, obstacles in coding and data visualization can poten-
tially lead to data misuse, which can tarnish the work of an entire community.
The purpose of this text is to provide an overview of the common preparatory
work and visualization techniques that are applied to environmental satellite data
using the Python language. This book is highly example-driven, and all the exam-
ples are available online. The exercises are primarily based on hands-on tutorial
workshops that I have developed. The motivation for producing this book is to
make the contents of the workshops accessible to more Earth scientists, as very
few Python books currently available target the Earth science community.
This book is written to be a practical workbook and not a theoretical text-
book. For example, readers will be able to interactively run prewritten code inter-
actively alongside the text to guide them through the code examples. Exercises in
each section build on one another, with incremental steps folded in. Readers with
minimal coding experience can follow each “baby step” to get them up to become
“spun up” quickly, while more experienced coders have the option of working with
the code directly and spending more time on building a workflow as described in
Section III.
The exercises and solutions provided in this book use Jupyter Notebook, a
highly interactive, web-based development environment. Using Jupyter Note-
book, code can be run in a single line or short blocks, and the results are generated
within an interactive documented format. This allows the student to view both the
Python commands and comments alongside the expected results. Jupyter Note-
book can also be easily converted to programs or scripts than can be executed
on Linux Machines for high-performance computing. This provides a friendly
work environment to new Python users. Students are also welcome to develop
code in any environment they wish, such as the Spyder IDE or using iPython.
While the material builds on concepts learned in other chapters, the book
references the location of earlier discussions of the material. Within each chapter,
the examples are progressive. This design allows students to build on their under-
standing knowledge (and learn where to find answers when they need guidance)
rather than memorizing syntax or a “recipe.” Professionally, I have worked with
many datasets and I have found that the skills and strategies that I apply on sat-
ellite data are fairly universal. The examples in this book are intended to help read-
ers become familiar with some of the characteristic quirks that they may encounter
when analyzing various satellite datasets in their careers. In this regard, students
are also strongly encouraged to submit requests for improvements in future
editions.
Like many technological texts, there is a risk that the solutions presented will
become outdated as new tools and techniques are developed. The sizable user com-
munity already contributing to Python implies it is actively advancing; it is a living
language in contrast to compiled, more slowly evolving legacy languages like

cintro.3d 2 13/7/2021 9:13:51 AM

 Introduction 3

Fortran and C/C++. A drawback of printed media is that it tends to be static and
Python is evolving more rapidly than the typical production schedule of a book.
To mitigate this, this book intends to teach fluency in a few, well-established
packages by detailing the steps and thought processes needed for a user needs
to carry out more advanced studies. The text focuses discipline-agnostic packages
that are widely used, such as NumPy, Pandas, and xarray, as well as plotting
packages such as Matplotlib and Cartopy.
I have chosen to highlight Python primarily because it is a general-purpose
language, rather than being discipline or task-specific. Python programmers
can script, process, analyze, and visualize data. Python’s popularity does not
diminish the usefulness and value of other languages and techniques. As with
all interpreted programming languages, Python may run more slowly compared
to compiled languages like Fortran and C++, the traditional tools of the trade.
For instance, some steps in data analysis could be done more succinctly and with
greater computational efficiency in other languages. Also, underlying packages in
Python often rely on compiled languages, so an advanced Python programmer can
develop very computationally efficient programs with popular packages that are
built with speed-optimized algorithms. While not explicitly covered in this book,
emerging packages such as Dask can be helpful to process data in parallel, so more
advanced scientific programmers can learn to optimize the speed performance of
their code. Python interfaces with a variety of languages, so advanced scientific
programmers can compile computationally expensive processing components
and run them using Python. Then, simpler parts of the code can be written in
Python, which is easier to use and debug.
This book encourages readers to share their final code online with the broader
community, a practice more common among software developers than scientists.
However, it is also good practice to write code and software in a thoughtful and
carefully documented manner so that it is usable for others. For instance, well-
written code is general purpose, lacks redundancy, and is intuitively organized
so that it may be revised or updated if necessary. Many scientific programmers
are self-learners with a background in procedural programming, and thus their
Python code will tend to resemble the flow of a Fortran or IDL program. This
text uses Jupyter Notebook, which is designed to promote good programming
habits in establishing a “digestible code” mindset; this approach organizes code
into short chunks. This book focuses on clear documentation in science algorithms
and code. This is handled through version control, using virtual environments,
how to structure a usable README file, and what to include in inline
commenting.
For most environmental science endeavors, data and code sharing are part of
the research-to-operations feedback loop. “Operations” refers to continuous data
collection for scientific research and hazard monitoring. By sharing these tools
with other researchers, datasets are more fully and effectively utilized. Satellite
data providers can upgrade existing datasets if there is a demand. Globally,

cintro.3d 3 13/7/2021 9:13:51 AM

 4 Earth Observation Using Python

satellite data are provided through data portals by NASA, NOAA, EUMETSAT,
ESA, JAXA, and other international agencies. However, the value of these data-
sets is often only visible through scientific journal articles, which only represent a
small subset of potential users. For instance, if the applications of satellite obser-
vations used for routine disaster mitigation and planning in a disadvantaged
nation are not published in a scientific journal, improvements for disaster-
mitigation specific needs may never be met.
Further, there may be unexpected or novel uses of datasets that can drive sci-
entific inquiry, but if the code that brings those uses to life is hastily written and not
easily understood, it is effectively a waste of time for colleagues to attempt to
employ such applications. By sharing clearly written code and corresponding doc-
umentation for satellite data applications, users can alert colleagues in their com-
munity of the existence of scientific breakthrough efforts and expand the potential
value of satellite datasets within and beyond their community. Moreover, public
knowledge of those efforts can help justify the versatility and value of satellite mis-
sions and provide a return on investment for organizations that fund them. In the
end, the dissemination of code and data analysis tools will only benefit the scien-
tific community as a whole.

cintro.3d 4 13/7/2021 9:13:51 AM

 Part I

Overview of Satellite
Datasets

0005078637.3D 5 14/6/2021 3:17:17 PM

 1

A TOUR OF CURRENT SATELLITE MISSIONS

AND PRODUCTS

There are thousands of datasets containing observations of the

Earth. This chapter describes some satellite types, orbits, and
missions, which benefit a variety of fields within Earth sciences,
including atmospheric science, oceanography, and hydrology.
Data are received on the ground through receiver stations and
processed for use using retrieval algorithms. But the raw data
requires further manipulation to be useful, and Python is a good
choice for analysis and visualization of these datasets.

At present, there are over 13,000 satellite-based Earth observations freely and
openly listed on www.data.gov. Not only is the quantity of available data notable,
its quality is equally impressive; for example, infrared sounders can estimate
brightness temperatures within 0.1 K from surface observations (Tobin et al.,
2013), imagers can detect ocean currents with an accuracy of 1.0 km/hr
(NOAA, 2020), and satellite-based lidar can measure the ice-sheet elevation
change with a 10 cm sensitivity (Garner, 2015). Previously remote parts of our
planet are now observable, including the open oceans and sparsely populated
areas. Furthermore, many datasets are available in near real time with image
latencies ranging from less than an hour down to minutes – the latter being crit-
ically important for natural disaster prediction. Having data rapidly available
enables science applications and weather prediction as well as to emergency man-
agement and disaster relief. Research-grade data take longer to process (hours to
months) but has a higher accuracy and precision, making it suitable for long-term
consistency. Thus, we live in the “golden age” of satellite Earth observation. While
the data are accessible, the tools and skills necessary to display and analyze this
information require practice and training.

Earth Observation Using Python: A Practical Programming Guide, Special Publications 75,
First Edition. Rebekah B. Esmaili.
© 2021 American Geophysical Union. Published 2021 by John Wiley & Sons, Inc.
DOI: 10.1002/9781119606925.ch1

0005078640.3D 7 12/7/2021 1:07:30 PM

 8 Earth Observation Using Python

Python is a modern programming language that has exploded in popularity,

both within and beyond the Earth science community. Part of its appeal is its easy-
to-learn syntax and the thousands of available libraries that can be synthesized
with the core Python package to do nearly any computing task imaginable. Python
is useful for reading Earth-observing satellite datasets, which can be difficult to use
due to the volume of information that results from the multitude of sensors, plat-
forms, and spatio-temporal spacing. Python facilitates reading a variety of self-
describing binary datasets in which these observations are often encoded. Using
the same software, one can complete the entirety of a research project and produce
plots. Within a notebook environment, a scientist can document and distribute the
code to other users, which can improve efficiency and transparency within the
Earth sciences community.
Satellite data often require some pre-processing to make it usable, but which
steps to take and why are not always clear. Data users often misinterpret concepts
such as data quality, how to perform an atmospheric correction, or how to imple-
ment the complex gridding schemes necessary to compare data at different resolu-
tions. Even to a technical user, the nuances can be frustrating and difficult to
overcome. This book walks you through some of the considerations a user should
make when working with satellite data.
The primary goal of this text is to get the reader up to speed on the Python
coding techniques needed to perform research and analysis using satellite datasets.
This is done by adopting an example-driven approach. It is light on theory but will
briefly cover relevant background in a nontechnical manner. Rather than getting
lost in the weeds, this book purposefully uses realistic examples to explain con-
cepts. I encourage you to run the interactive code alongside reading the text. In
this chapter, I will discuss a few of the satellites, sensors, and datasets covered
in this book and explain why Python is a great tool for visualizing the data.

1.1. History of Computational Scientific Visualization

Scientific data visualizing used to be a very tedious process. Prior to the 1970s,
data points were plotted by hand using devices such as slide rules, French curls,
and graph paper. During the 1970s, IBM mainframes became increasingly avail-
able at universities and facilitated data analysis on the computer. For analysis,
IBM mainframes required that a researcher write Fortran-IV code, which was
then printed to cards using a keypunch machine (Figure 1.1). The punch cards
then were manually fed into a shared university computer to perform calculations.
Each card is roughly one line of code. To make plots, the researcher could create a
Fortran program to make an ASCII plot, which creates a plot by combining lines,
text, and symbols. The plot could then be printed to a line-printer or a teleprinter.
Some institutions had computerized graphic devices, such as Calcomp plotters.
Rather than create ASCII plots, the researcher could use a Calcomp plotting

0005078640.3D 8 12/7/2021 1:07:30 PM

 A Tour of Current Satellite Missions and Products 9

Figure 1.1 (a) An example of a Fortran punch card. Each vertical column represents a
character and one card roughly one line of Fortran code. (b) 1979 photo of an IMSAI
8080 computer that could store up to 32 kB of the data, which could then be
transferred to a keypunch machine to create punch cards. (c) an image created from the
Hubble Space Telescope using a Calcomp printer, which was made from running
punch cards and plotting commands through a card reader.

command library to control how data were visualized and store the code on com-
puter tape. The scientist would then take the tape to a plotter, which was not nec-
essarily (or usually) in the same area as the computer or keypunch machine. Any
errors – such as bugs in the code, damaged punch cards, or damaged tape – meant
the whole process would have to be repeated from scratch.
In the mid-1980s, universities provided remote terminals that would eventu-
ally replace the keypunch and card reader machine system. This substantially
improved data visualization processes, as scientists no longer had to share limited
resources such as keypunch machines, card readers, or terminals. By the late
1980s, personal computers became more affordable for scientists. A typical PC,
such as the IBM XT 286, had 640 Kb of random access memory, a 32 MB hard
drive, and 5.25 inch floppy disks with 1.2 MB of disk storage (IBM, 1989). At this

0005078640.3D 9 12/7/2021 1:07:30 PM

 10 Earth Observation Using Python

time, pen plotters became increasingly common for scientific visualization, fol-
lowed later by the prevalence of ink-jet printers in the 1990s. These technologies
allowed researchers to process and visualize data conveniently from their offices.
With the proliferation of user-friendly person computers, printers eventually made
their way into all homes and offices.
Now with advances in computing and internet access, researchers no longer
need to print their visualizations at all, but often keep data in digital form only. Plots
can be created in various data formats that easily embed into digital presentations
and documents. Scientists often do not ever print visualizations because computers
and cloud storage can store many gigabytes of data. Information is created and con-
sumed entirely in digital form. Programming languages, such as Python, can tap
into high-level plotting programs and can minimize the axis calculation and labeling
requirements within a plot. Thus, the expanded access to computing tools and sim-
plified processes have advanced scientific data visualization opportunities.

1.2. Brief Catalog of Current Satellite Products

In Figure 1.2, you can see that the international community has developed
and launched a plethora of Earth-observing satellites, each with several onboard
sensors that have a range of capabilities. I am not able to discuss every sensor,
dataset, and mission (a term coined by NASA to describe projects involving

Figure 1.2 Illustration of current Earth, space weather, and environmental monitoring
satellites from the World Meteorological Organization (WMO). Source: U.S.
Department of Commerce / NOAA / Public Domain.

0005078640.3D 10 12/7/2021 1:07:30 PM

 A Tour of Current Satellite Missions and Products 11

spacecraft). However, I will describe some that are relevant to this text, organized
by subject area.

1.2.1. Meteorological and Atmospheric Science

Most Earth-observing satellites orbit our planet either in either geostationary

or low-Earth orbiting patterns. These types of satellites tend to be managed and
operated by large international government agencies, and the data are often freely
accessible online:
• Geosynchronous equatorial orbit (GEO) satellites. Geostationary platforms
orbit the Earth at 35,700 km above the Earth’s surface. GEO satellites are
designed to continuously monitor the same region on Earth, and thus can pro-
vide many images over a short period of time to monitor change. National
Oceanic and Atmospheric Administration (NOAA) operates the Geostation-
ary Environmental Satellite System (GOES) satellites for monitoring North
and South America. GOES-16 and -17 have an advanced baseline instrument
(ABI) onboard to create high-resolution imagery in visible and infrared (IR)
wavelengths. The GOES-16 and -17 satellites are also equipped with the Geo-
stationary Lightning Mapper (GLM) to detect lightning. Instruments
designed for space weather include the Solar Ultraviolet Imager (SUVI)
and X-ray Irradiance Sensors (EXIS). The European Organization for the
Exploitation of Meteorological Satellites (EUMETSAT) operates and main-
tains the Meteosat series GEO satellites that monitor Europe and Africa. The
Japan Aerospace Exploration Agency (JAXA) operates and maintains the
Himawari satellite that monitors Asia and Oceania.
• Low-Earth orbit (LEO) satellites. Polar orbiting satellites provide approxi-
mately twice daily global observations at the equator (with more observations
per day at the poles). Figure 1.3 displays the equatorial crossing time for his-
toric and existing LEO satellites, which refers to the local time at the equator
when observations are made. Overpasses from some LEO satellites shift dur-
ing a mission, while others are periodically adjusted back to maintain a con-
sistent overpass time throughout the duration of a mission. Polar orbiting
satellites are called low-Earth orbit satellites because they are much closer
to the Earth’s surface (at 400–900 km) than GEO satellites, which are approx-
imately 40 times further away from the earth or at ~35,000 km. The lower alti-
tude of LEO satellites facilitates their higher spatial resolution relative to
GEO, although the temporal resolution tends to be lower for LEO satellites.
The Suomi-NPP and NOAA-20 are two satellites that were developed and
maintained by NASA and NOAA, respectively. They are each equipped with
an imager, the Visible Infrared Imaging Radiometer Suite (VIIRS), and infra-
red and microwave sounders, the Cross-track Infrared Sounder (CrIS) and an
Advanced Technology Microwave Sounder (ATMS). The MetOp series of
LEO satellites (named MetOp-A, -B, and -C) were developed by the Euro-
pean Space Agency (ESA) and are operated by EUMETSAT.

0005078640.3D 11 12/7/2021 1:07:31 PM

 12 Earth Observation Using Python

AQUA
23 METOP-A
METOP-B
22
Local Equator Crossing Time (hour)

NOAA-06
NOAA-07
21 NOAA-08
NOAA-09
20
NOAA-10

19 NOAA-11
NOAA-12
18 NOAA-14
NOAA-15
17 NOAA-16
NOAA-18
16 NOAA-19
NOAA-20
15 SNPP
TERRA
14
TIROS-N

13
12
80 85 90 95 00 05 10 15
19 19 19 19 20 20 20 20
Date (UTC)

Figure 1.3 Equatorial crossing times for various LEO satellites displayed using Python.

1.2.2. Hydrology

Because water is sensitive to microwave frequencies, microwave instruments

and sounders are useful for detecting water vapor, precipitation, and ground mois-
ture. The Global Precipitation Mission (GPM) uses the core GPM satellite along
with a constellation of microwave imagers and sounders to estimate global precip-
itation. The SMAP satellite mission uses active and passive microwave sensors to
observe surface soil moisture every two to three days. The GRACE-FO satellite
measures gravitational anomalies, that can be used to infer changes in global sea
levels and soil moisture. All three hydrology missions were developed and oper-
ated by NASA.

1.2.3. Oceanography and Biogeosciences

Both GEO and LEO satellites can provide sea surface temperature (SST)
observations. The GOES series of GEO satellites provides continuous sampling
of SSTs over the Atlantic and Pacific Ocean basins. The MODIS instrument on
the Aqua satellite has been providing daily, global SST observations continuously
since the year 2000. Visible wavelengths are useful for detecting ocean color, par-
ticularly from LEO satellites, which are often observed at very high resolutions.

0005078640.3D 12 12/7/2021 1:07:31 PM

 A Tour of Current Satellite Missions and Products 13

Additionally, LEO satellites can detect global sea-surface anomaly para-

meters. Jason-3 is a low-Earth satellite developed as a partnership between
EUMETSAT, NOAA, NASA, and CNES. The radar altimeter instrument on
Jason-3 is sensitive to height changes less than 4 cm and completes a full Earth
scan every 10 days (Vaze et al., 2010).

1.2.4. Cryosphere

ICESat-2 (Ice, Cloud, and land Elevation Satellite 2) is a LEO satellite mis-
sion designed to measure ice sheet elevation and sea ice thickness. The GRACE-
FO satellite mission can also monitor changes in glaciers and ice sheets.

1.3. The Flow of Data from Satellites to Computer

The missions mentioned in the previous section provide open and free data to
all users. However, data delivery, the process of downloading sensor data from the
satellite and converting it into a usable form, is not trivial. Raw sensor data are
first acquired on the satellite, then the data must be relayed to the Earth’s ground
system, often at speeds around 30 Mbits/second. For example, GOES satellite
data are acquired by NASA’s Wallops Flight Facility in Virginia; data from
the Suomi NPP satellite is downloaded to the ground receiving station in Svalbard,
Norway (Figure 1.4). Once downloaded, the observations are calibrated and sev-
eral corrections are applied, such as an atmospheric correction to reduce haze in
the image or topographical corrections to adjust changes in pixel brightness on
complex terrain. The corrected data are then incorporated into physical products
using satellite retrieval algorithms. Altogether, the speed of data download and
processing can impact the data latency, or the difference between the time the
physical observation is made and the time it becomes available to the data user.
Data can be accessed in several ways. The timeliest data can be downloaded
using a direct broadcast (DB) antenna, which can immediately receive data when
the satellite is in range. This equipment is expensive to purchase and maintain, so
usually only weather and hazard forecasting offices install them. Most users will
access data via the internet. FTP websites post data in near real time, providing the
data within a few hours of the observation. Not all data must be timely – research-
grade data can take months to calibrate to ensure accuracy. In this case, ordering
through an online data portal will grant users access to long records of data.
While data can be easily accessed online, they are rarely analysis ready. Soft-
ware and web-based tools allow for quick visualization, but to create custom ana-
lyses and visualizations, coding is necessary. To combine multiple datasets, each
must be gridded to the same resolution for an apples-to-apples comparison. Fur-
ther, data providers use quality flags to indicate the likelihood of a suitable
retrieval. However, the meaning and appropriateness of these flags are not always

0005078640.3D 13 12/7/2021 1:07:31 PM

 14 Earth Observation Using Python

Stored Mission
Data Antenna
NOAA-20

Svalbard, Norway
300 Mbps
High-Rate Data
Antenna

Direct
Broadcast
Network
15 Mbps

McMurdo, Antarctica
300 Mbps

Figure 1.4 NOAA-20 satellite downlink.

well communicated to data users. Moreover, understanding how such datasets are
organized can be cumbersome to new users. This text thus aims to identify specific
Python routines that enable custom preparation, analysis, and visualization of sat-
ellite datasets.

1.4. Learning Using Real Data and Case Studies

I have structured this book so that you can learn Python through a series of
examples featuring real phenomena and public datasets. Some of the datasets and
visualizations are useful for studying wildfires and smoke, dust plumes, and hur-
ricanes. I will not cover all scenarios encountered in Earth science, but the skills
you learn should be transferrable to your field. Some of these case studies include:
• California Camp Fire (2018). California Camp Fire was a forest fire that
began on November 8, 2018, and burned for 17 days over a 621 km2 area.
It was primarily caused by very low regional humidity due to strong gusting
wind events and a very dry surface. The smoke from the fire also affected
regional air quality. In this case study, I will examine satellite observations
to show the location and intensity as well as the impact that the smoke had
on regional CO, ozone, and aerosol optical depth (AOD). Combined satellite
channels also provide useful imagery for tracking smoke, such as the dust

0005078640.3D 14 12/7/2021 1:07:31 PM

 A Tour of Current Satellite Missions and Products 15

RGB product. Land datasets such as the Normalized Difference Vegetation

Index (NDVI) are useful for highlighting burn scars from before and after the
fire events.
• Hurricane Michael (2018). Michael was a major hurricane that affect the Flor-
ida Panhandle of the United States. Michael developed as a tropical wave on
October 7 in the southwest Caribbean Sea and grew into a Category 5 storm
by October 10. Throughout its life cycle, Michael caused extensive flooding,
leading to 74 deaths and $25 billion in damage. Several examples in this text
utilize visible and infrared imagery of Hurricane Michael.
• Louisiana Flood Event (2016). Thousands of homes were flooded in Louisiana
when over 20 inches of rain fell between August 12 and August 21, 2016. The
event began after a mesoscale convective system stalled over the area near
Baton Rouge and Lafayette, Louisiana. I will use the IMERG global rainfall
dataset to examine this event.

1.5. Summary

I have provided a brief overview of the many satellite missions and datasets
that are available. This book has two main objectives: (1) to make satellite data
and analysis accessible to the Earth science community through practical Python
examples using real-world datasets; and (2) to promote a reproducible and trans-
parent scientific code philosophy. In the following chapters, I will focus on
describing data conventions, common methods, and problem-solving skills
required to work with satellite datasets.

References

Tobin, D., Revercomb, H., Knuteson, R., Taylor, J., Best, F., Borg, L., et al. (2013).
Suomi-NPP CrIS radiometric calibration uncertainty. Journal of Geophysical Research:
Atmospheres, 118(18), 10,589–10,600. https://doi.org/10.1002/jgrd.50809
Garner, R. (2015, July 10). ICESat-2 Technical Requirements. www.nasa.gov/content/god-
dard/icesat-2-technical-requirements
IBM (1989, January). Personal Computer Family Service Information Manual. IBM doc-
ument SA38-0037-00. http://bitsavers.trailing-edge.com/pdf/ibm/pc/SA38-0037-00_Per-
sonal_Computer_Family_Service_Information_Manual_Jul89.pdf
National Oceanic and Atmospheric Administration (2020, June 12). GOES-R Series Level
I Requirements (LIRD). www.goes-r.gov/syseng/docs/LIRD.pdf.

0005078640.3D 15 12/7/2021 1:07:31 PM

 2

OVERVIEW OF PYTHON

Python is a free and open-source programming language. There

are over 200,000 packages registered online that expand Python’s
capabilities. This chapter provides a description of some useful
packages for the Earth sciences. Some of these useful packages
include NumPy, Pandas, Matplotlib, netCDF4, h5py, Cartopy,
and xarray. These packages have a strong development base
and a large community of support, making them appropriate for
scientific investigation.

In this chapter, I discuss some reasons why Python is a valuable tool for Earth
scientists. I will also provide an overview of some of the commonly used Python
packages for remote sensing applications that I will use later in this book. Python
evolves rapidly, so I expect these tools to improve and new ones to become avail-
able. However, these will provide a solid foundation for you to begin your
learning.

2.1. Why Python?

Chances are, you may already know a little about what Python is and have
some motivation to learn it. Below, I outline common reasons to use Python rel-
evant to the Earth sciences:
• Python is open-source and free. Some of the legacy languages are for profit,
and licenses can be prohibitively expensive for individuals. If your career
plans include remaining at your current institution or company that supplies
you with language licenses, then open source may not be of concern to you.
But often, career growth means working for different organizations. Python is

Earth Observation Using Python: A Practical Programming Guide, Special Publications 75,
First Edition. Rebekah B. Esmaili.
© 2021 American Geophysical Union. Published 2021 by John Wiley & Sons, Inc.
DOI: 10.1002/9781119606925.ch2

17

0005078641.3D 17 12/7/2021 1:10:54 PM

 18 Earth Observation Using Python

portable, which frees up your skillset from being exclusively reliant on propri-
etary software.
• Python can increase productivity. There are thousands of supported libraries
to download and install. For instance, if you want to open multiple netCDF
files at once, the package called xarray can do that. If you want to re-grid an
irregular dataset, there is a package called pyresample that will do this quickly.
Even more subject-specific plots, like Skew-T diagrams, have a prebuilt pack-
age called MetPy. For some datasets, you can download data directly into
Python using OPenDAP. Overall, you spend less time developing routines
and more time analyzing results.
• Python is easy to learn, upgrade, and share. Python code is very “readable” and
easy to modularize, so that functions can be easily expanded or improved.
Further, low- or no-cost languages like Python increase the shareability of
the code. When the code works, it should be distributed online for other users’
benefit. In fact, some grants and journals require online dissemination.

You may already have knowledge of other computer languages such as IDL,
MATLAB, Fortran, C++, or R. Learning Python does not mean you will stop
using other languages or rewrite all your existing code. In many cases, other lan-
guages will continue to be a valuable part of your daily work. For example, there
are a few drawbacks to using Python:
• Python may be slower than compiled languages. While many of the core scien-
tific packages compile code on the back end, Python itself is not a compiled
language. For a novice user, Python will run more slowly, especially if loops
are present in the code. For a typical user, this speed penalty may not be
noticeable, and advanced users can tap into other runtime frameworks like
Dask or Cython or even run compiled Fortran subroutines to enhance perfor-
mance. However, new users might not feel comfortable learning these work-
arounds, and even with runtime frameworks and subroutines, performance
might not improve. If speed is a concern, then Python could be used as a pro-
totype code tool prior to converting into a compiled language.
• New users often run packages “as-is” and the contents are not inspected. There
are thousands of libraries available, but many are open-source, community
projects and bugs and errors will exist. For example, irregular syntax can
result whenever there is a large community of developers. Thus, scientists
and researchers should be extra vigilant and only use vetted packages.
• Python packages may change function syntax or discontinued. Python changes
rapidly. While most developers refrain from abruptly changing syntax, this
practice is not always followed. In contrast, because much of the work in
developing these packages is on a volunteer basis, the communities supporting
them could move on to other projects and those who take over could begin a
completely new syntax structure. While this is unlikely to be the case for highly

0005078641.3D 18 12/7/2021 1:10:54 PM

 Overview of Python 19

used packages, anything is possible. For example, a popular map plotting

package called Basemap was discontinued in the module Matplotlib and
replaced with Cartopy, which is an older code package. I recommend using
packages that have backing from Earth science research institutions (e.g.,
the UK Met Office, NASA, Lamont, etc.) to raise confidence that the
packages you choose to use will be relatively stable.

Unlike legacy languages such as Fortran and C++, there is no guarantee that
code written in Python will remain stable for 30+ years. However, the packages
presented in this book are “mature” and are likely to continue to be supported
for many years. Additionally, you can reproduce the exact packages and versions
using virtual environments (Section 11.3). This text highlights newer packages that
save significant amounts of development time and streamline certain processes,
including how to open and read netCDF files and gridding operations.

2.2. Useful Packages for Remote Sensing Visualization

Python contains intrinsic structure and mathematical commands, but its cap-
abilities can be dramatically increased using modules. Modules are written in Python
or a compiled language like C to help simplify common, general, or redundant tasks.
For instance, the datetime module helps programmers manipulate calendar dates
and times using a variety of units. Packages contain one or more modules, which
are often designed to facilitate tasks that follow a central theme. Some other terms
used interchangeably for packages are libraries and distributions.
At the time of writing, there are over 200,000 Python packages registered on
pipy.org and more that live on the internet in code repositories such as GitHub
(https://github.com/). Many of the most popular packages are often developed
and maintained by large online communities. This massive effort benefits you
as a scientist because many common tasks have already been developed in Python
by someone else. This can also create a dilemma for scientists and researchers – the
trade-off between using existing code to save time against time spent researching
and vetting so many code options. Additionally, because many of these packages
do not have full-time staff support, the projects can be abandoned by their devel-
opment teams, and your code could eventually become obsolete.
In your research, I suggest you use three rules when choosing packages to
learn and work with:
1. Use established packages.
2. Use packages that have a large community of support.
3. Use code that is efficient with respect to reduced coding time and increased
speed of performance.
Following is a list of the main Python packages that I will cover in this text.

0005078641.3D 19 12/7/2021 1:10:54 PM

 20 Earth Observation Using Python

2.2.1. NumPy

NumPy is the fundamental package for scientific computing with Python. It

can work with multidimensional arrays, contains many advanced mathematical
functions, and is useful for linear algebra, Fourier transforms, and for generating
random numbers. NumPy also allows users to encapsulate data efficiently. If you
are familiar with MATLAB, you will feel very comfortable using this package.

2.2.2. Pandas

Pandas is a library that permits using data frame (stylized DataFrame) struc-
tures and includes a suite of I/O and data manipulation tools. Unlike NumPy,
Pandas allows you to reference named columns instead of using indices. With
Pandas, you can perform the same kinds of essential tasks that are available in
spreadsheet programs (but now automated and with fewer mouse clicks!). For
those who are familiar with R programming language, Pandas mimics the
R data.frame function.
A limitation of Pandas is that it can only operate with 2D data structures.
More recently, the xarray package has been developed to handle higher-
dimensional datasets. In addition, Pandas can be somewhat inefficient because
the library is technically a wrapper for NumPy, so it can consume up to three times
as much memory, particularly in Jupyter Notebook. For larger row operations
(500K rows or greater), the differences can even out. (Goutham, 2017).

2.2.3. Matplotlib

Matplotlib is a plotting library, arguably the most popular one. Matplotlib

can generate histograms, power spectra, bar charts, error charts, and scatterplots
with a few lines of code. The plots can be completely customized to suit your aes-
thetics. Due to their similarities, this is another package where MATLAB experi-
ence may come in handy.

2.2.4. netCDF4 and h5py

I will discuss two common self-describing data formats, netCDF and HDF, in
Section 3.2.3. Two major packages for importing these formats are the netCDF4
and h5py packages. These tools are advantageous because the user does not have
to have any knowledge of how to parse the input files, so long as the files follow
standard formatting. These two packages import the data, which can then be con-
verted to NumPy to perform more rigorous data operations.

0005078641.3D 20 12/7/2021 1:10:54 PM

 Overview of Python 21

2.2.5. Cartopy

Cartopy is a package for projecting geospatial data to maps. It can also be

used to access a wealth of features, including land/ocean masks and topography.
Many projections are available, and you can easily transform the data
between them.
Previously, Basemap was the primary package for creating maps. You may
come across examples that use it online. However, the package is now deprecated
and Cartopy has become the primary package that interfaces with Matplotlib.
Cartopy is a package available from the SciTools organization, which was
originally developed by the UK Met Office. It has now expanded into a commu-
nity collaboration.

2.3. Maturing Packages

The packages detailed in this section are worth mentioning because they may
apply to your specific project. Further, some features are too good to ignore, so
they are highlighted below. However, if your code requires a long-term shelf life,
it may be best to find alternative solutions, as the following packages may change
more rapidly than those listed in Section 2.3.

2.3.1. xarray

xarray is a package that borrows heavily from Pandas to organize multidi-

mensional data. Mathematical operations are lightning fast thanks to dimensional
and coordinate indexing. Visualization is also easy. xarray is valuable to Earth
scientists because it permits opening multiple netCDF files with ease. Interpola-
tion and group operations is also possible.
The xarray syntax can be challenging to newcomers. It can be difficult to
wrangle the data into the format needed. Nevertheless, this tool is worth the time
investment due to the many features of interest to Earth science.

2.3.2. Dask

Dask interfaces with Pandas, Scikit-Learn, and NumPy to perform parallel

processing and out-of-memory operations that can read data in chunks without
ever being totally in the computer’s RAM. This is very useful for working with
large datasets. If speed needs to be prioritized, it would be worth learning this
package.

0005078641.3D 21 12/7/2021 1:10:54 PM

 22 Earth Observation Using Python

2.3.3. Iris

Iris is a format-agnostic Python library for analyzing and visualizing Earth

science data. If datasets follow the standard CF formatting conventions, Iris
can easily load the data. The Iris package has a steep learning curve but can be
useful for performing meteorological computations. Like Cartopy, Iris is a pack-
age available from the SciTools organization.

2.3.4. MetPy

MetPy is a collection of tools in Python for reading, visualizing, and perform-

ing calculations with weather data. MetPy enables downloading a curated
collection of remote sensing datasets. Unit conversions are easy to perform,
which is helpful when making calculations of meteorological variables. MetPy
is maintained by Unidata in Boulder, Colorado.

2.3.5. cfgrib and eccodes

Cfgrib is a useful package for reading GRIB1 and GRIB2 data, which is a
common format for reanalysis and model data, particularly for the ECMWF.
Cfgrib decodes GRIB data in a way that it mimics the structure of NetCDF files
using the ecCodes python package. ecCodes was developed by ECMWF to decod-
ing and encoding standard WMO GRIB and BUFR files.

2.4. Summary

I hope you are excited to begin your Python journey. Since it is free and open-
source, Python is a valuable tool that you can carry with you for the rest of your
career. Furthermore, there are many existing packages to perform common tasks
in the Earth Sciences, such as importing common datasets, organizing data, per-
forming mathematical analysis, and displaying results. In the next chapter, I will
describe some of the common satellite data formats you may encounter.

References

Dask: Scalable analytics in Python. (n.d.). Retrieved November 25, 2020, from https://
dask.org/
ecmwf/cfgrib. (2020). Python, European Centre for Medium-Range Weather Forecasts.
Retrieved from https://github.com/ecmwf/cfgrib (Original work published July 16, 2018).
Matplotlib: Python plotting — Matplotlib 3.3.3 documentation. (n.d.). Retrieved November
25, 2020, from https://matplotlib.org/.
MetPy — MetPy 0.12. (n.d.). Retrieved November 25, 2020, from https://unidata.github.io/
MetPy/latest/index.html.

0005078641.3D 22 12/7/2021 1:10:55 PM

 Overview of Python 23

NumPy. (n.d.). Retrieved November 25, 2020, from https://numpy.org/

Overview: Why xarray? — xarray 0.16.2.dev3+g18a59a6d.d20200920 documentation. (n.
d.). Retrieved November 25, 2020, from http://xarray.pydata.org/en/stable/why-xar-
ray.html.
pandas documentation — pandas 1.1.4 documentation. (n.d.). Retrieved November 25,
2020, from https://pandas.pydata.org/pandas-docs/stable/index.html.
Vaze, P., Neeck, S., Bannoura, W., Green, J., Wade, A., Mignogno, M., et al. (2010). The
Jason-3 Mission: completing the transition of ocean altimetry from research to opera-
tions. In R. Meynart, S. P. Neeck, & H. Shimoda (Eds.) (p. 78260Y). Presented at the
Remote Sensing, Toulouse, France. https://doi.org/10.1117/12.868543.

0005078641.3D 23 12/7/2021 1:10:55 PM

 3

A DEEP DIVE INTO SCIENTIFIC DATA SETS

Satellite data is voluminous, so data must be compressed for

storage while also documenting the contents. Scientific data can
be stored in text formats that are human readable but lack a high
degree of compression. Binary data are highly compressed but not
human readable. Furthermore, unless supporting documentation is
included in the data archive, it may be impossible to know how to
read the files or what kind of data are stored in them. Self-describing
formats like NetCDF4 and HDF5 contain compressed data but also
store metadata inside the data file, which helps users understand
the contents, such as the description of the data and data quality.

NASA’s Earth Observing System Data and Information System (EOSDIS) has
accumulated 27 PBs of data since the 1990s with the purpose of furthering scientific
research. EOSDIS continues to add data from missions prior to the 1990s, which are
stored as hard disk media (Figure 3.1). Many of these older datasets need to be
“rescued,” which is challenging because such legacy media are often disorganized,
lack documentation, are physically damaged, or lack appropriate readers (Meier et
al., 2013; James et al., 2018). As the satellite era began in the 1960s, it is unlikely that
the planners considered how voluminous the data would become and how rapidly it
would be produced.
Nowadays, data archiving at agencies is carefully planned and organized, and
it uses scientific data formats. This infrastructure allows EOSDIS to freely distrib-
ute hundreds of millions of data files a year to the public (I describe how to obtain
data in Appendix E). In addition to improving access and storage, scientific data
formats provide consistency between files, which reduces the burden on research-
ers who can more easily write code using tools like Python to read, combine, and
analyze data from different sources. This philosophy is encompassed by the term

Earth Observation Using Python: A Practical Programming Guide, Special Publications 75,
First Edition. Rebekah B. Esmaili.
© 2021 American Geophysical Union. Published 2021 by John Wiley & Sons, Inc.
DOI: 10.1002/9781119606925.ch3

25

0005078642.3D 25 12/7/2021 1:16:00 PM

 26 Earth Observation Using Python

(a) (b)

(c) (d)

Figure 3.1 (a) Canisters of 35mm film that contain imagery recovered from Nimbus 1 of
polar sea ice extent collected in 1964. From roughly 40,000 recovered images, scientists
were able to reconstruct scenes such as (b) the ice edge north of Russia (78 N, 54 E) and
composites of the (c) north and (d) south poles. Photos courtesy of the National Snow and
Ice Data Center, University of Colorado, Boulder.

analysis-ready data (Dwyer et al., 2018). Common formats and labeling also
ensure that data are well understood and used appropriately. In this chapter,
I go over some of the ways data are stored, provide an overview of major satellite
data formats, as well as common ways to classify and label satellite data.

3.1. Storage

3.1.1. Single Values

Satellite data are voluminous, so data need to be stored as computationally

efficiently as possible. While detailed discussion is beyond the scope of this text,
computers store data in bits for short, and there are 8 bits in a byte. Table 3.1

0005078642.3D 26 12/7/2021 1:16:27 PM

 A Deep Dive into Scientific Data Sets 27

Table 3.1 Data Types, Typical Ranges, and Decimal Precision and Size in Computer
Memory

Numeric Type Range of Values Decimal Precision

1-byte integer –128 to 127 (0–255) –

2-byte integer –32,768 to 32,767 (0– to 65,535) –
4-byte integer –2,147,483,648 to 2,147,483,647 (0–4,294,967,295) –
4-byte float –3.4 × 10-38 to 3.4 × 1038 6
8-byte float –1.7 × 10-308 to 1.7 × 10308 15

Note: Numbers in parentheses are the unsigned integers (positive only). Unsigned floats are
not supported in Python.

has some computer number formats that satellite data are commonly stored in,
along with their respective numeric ranges and storage requirements. Integers
are numbers that have no decimal precision (e.g., 4, 8, 15, 16, –5,250, 8,642,
…). Floats are real numbers that have decimal precision (e.g., 3.14, 5.0,
1.2E23). Integers are typically advantageous for storage because they are smaller
and will not have rounding errors like float values can. To keep data small, values
are stored in the smallest numerical type necessary.
Even if an observed value is large, values can be linearly scaled (using an offset
and a scale factor) to fit within integer ranges to keep file sizes small. This is
because many Earth observations naturally only scale across a small range of
numbers. The scale and offset can be related to the measured value as follows:
Measured Value = Offset + Stored_Value ∗ Scale_Factor
For instance, the coldest and hottest surface temperatures on Earth are on the
order of 185 K (–88 C) and 331 K (58 C), respectively. The difference between the
two extremes is 146 K, so only 146 numbers are needed if you are not using any
decimal precision. For example, if I observe a surface temperature of 300 K,
I would store this in a two-byte unsigned integer if I do not rescale the data.
I can offset our data by 185 K, which is the lowest realistic temperature. Then,
I can store this measurement as 115 K, which fits in the one-byte integer ranges.
If later I want to read this value, I would then add the 185 K offset back to the
value. While reading and writing the data is more complex, the file is now 50%
smaller.
I may want further precision, but I can also account for this. For example, at one-
digit decimal place (e.g., 300.1 K), the number can be scale multiplied times 10 (which
is called the scale factor) and still saved as an integer (Table 3.2). However, our stored
value is now 1,515, so this data would be stored as a two-byte integer, which can con-
tain unsigned values up to 65,535. This time, if I am reading the data from the file,
value will need to be divided by 10. Again, this conversion saves two bytes of memory
over storing 300.15 K as a floating-point value, which is four bytes.

0005078642.3D 27 12/7/2021 1:16:28 PM

 28 Earth Observation Using Python

Table 3.2 Examples of How Data Can Be Rescaled to Fit in Integer Ranges

Measured Value Offset Scale Factor Stored Value Numeric Type

300 K – – 300 2-byte integer

300 K 185 K 1.0 115 1-byte integer
300.1 K 185 K 10.0 1,151 2-byte integer
300.15 K 185 K 100.0 11,515 2-byte integer
300.15 K – – 300.15 K 4-byte float

Note: Storing data in integers can provide significant storage savings over real values.

If a dataset does rescale the data to improve data storage (and many do not),
linear scaling is the most common. Conveniently, some Python packages can auto-
matically detect and apply scale and offset factors when importing certain data
types, such as netCDF and HDF formats, which we will discuss in Section 3.2.1.

3.1.2. Arrays

Nearly all satellite data are in three dimensions (3D), which are a value and
two corresponding spatial coordinates. The value can, for example, be a physical
observation, a numeric code that tells us something about the data, or some ancil-
lary characteristics. The spatial coordinate can be an x, y or a latitude, longitude
pair. Some datasets may also have a vertical coordinate or a time element, further
increasing the dimensionality of the data. As an example, I will discuss some dif-
ferent ways you can structure (or, how one organizes and manages) three-
dimensional data: latitude, longitude, and surface albedo (which shows the reflec-
tivity of the Earth to solar radiation). A simple way to structure the data is in a list,
an ordered sequence of values. So, I could store the following values:

Longitude = [42.7, 42.6, 42.5]

Latitude = [17.5, 17.6, 17.7]
Albedo = [0.30, 0.35, 0.32]

Where longitude ranges from –180 to 180 (East is positive), latitude from –
90 to 90 (North is positive), and surface albedo from 0 to 1 (which shows low to
high sunlight reflection, respectively).
Instead of three separate values, I could organize the data into a table, which
stores data in rows and columns:

Longitude Latitude Albedo

42.7 17.5 0.30

42.6 17.6 0.35
42.5 17.7 0.32

0005078642.3D 28 12/7/2021 1:16:28 PM

 A Deep Dive into Scientific Data Sets 29

This is helpful because it is human readable, and in a programming sense, I can

access the data across the rows rather than work with three different lists. If you are
sorting your data, it may also be easier to keep track of the items order in the table
form. You may have worked with data in comma separated variables (.csv), Google
sheets, or Excel format (.xls, .xlsx), which will resemble this structure.
The above two methods are useful if you have a few points, or if you want to
plot point observations. However, even the finest satellite data represents an exact
point, its value covers an area (or a volume, if there is a vertical component). The
total area that the satellite “sees” is known as the satellite footprint or field of
view (FOV).
Rather than use a list, it might then be often easier to store the latitude, lon-
gitude values as a multidimensional array (or matrix). For example, values of lon-
gitude, latitude, and albedo could be stored in three 2D arrays:
Longitude

42.70 42.68 42.66

42.69 42.68 42.66
42.69 42.68 42.66

Latitude

17.46 17.46 17.46

17.48 17.48 17.48
17.49 17.49 17.49

Albedo

0.30 0.32 0.30

0.35 0.30 0.31
0.34 0.33 0.32

Within Python, this organization is called a meshgrid, which stores the spatial
and value coordinated in a 2D rectangular grid. Coordinates stored in a meshgrid
are either regularly spaced or irregularly spaced data (Figure 3.2). If every coordi-
nate inside the meshgrid has a consistent distance between their neighboring coor-
dinate, then they are regularly spaced. On the other hand, the irregularly spaced
data will have varying distance between the x and y coordinates, or both. In gen-
eral, if grid coordinates are regularly spaced, the coordinates may be stored in
datasets as lists (e.g., GPM IMERG L3). If the data are irregularly spaced, the
data will likely be stored as a 2D meshgrid (e.g., in both cases, the data are very
commonly stored in a 2D grid).
In Figure 3.3, I show a plot of a single granule that contains several swaths of
surface albedo from 25 June 2018 at 11:21 UTC. Swaths are what a satellite can see

0005078642.3D 29 12/7/2021 1:16:28 PM

 30 Earth Observation Using Python

Regularly Spaced grid

0.1 0.1 0.1 0.1 0.1

0.1 0.1 0.1 0.1

Irregularly Spaced Grid

0.14 0.78 0.51 0.23

0.18 0.15 0.13 0.12 0.10

0.55 0.25
0.20 0.82

Figure 3.2 Spacing and distance between (x, y) points for an example regularly and
irregularly spaced rectangular grid.

Columns (3200)
Rows (768)

Figure 3.3 An illustration of how a granule of satellite data taken from a polar-orbiting
satellite would be stored in a meshgrid. Regions outside of where the satellite can see
or that are not stored in the file are indicated using fill values. These values are
excluded from analysis.

in longitude and latitude during a scan of the Earth (also called a scanline).
A granule is a collection of swaths, usually taken over a time period or for a fixed
number of scanlines. Data are often stored in chunks because global scans of the
Earth are voluminous. Additionally, the smaller data files improve the latency or
timeliness of the data because we do not have to wait for the full global image to
be assembled.
On the right of Figure 3.3, I illustrate what happens when the data are flat-
tened and projected into a 2D array. Due to the curvature of the Earth and the

0005078642.3D 30 12/7/2021 1:16:28 PM

 A Deep Dive into Scientific Data Sets 31

satellite viewing angle, the coordinates are spatially irregular. At the scan edges,
the footprints are larger than they are when the satellite is directly overhead. So,
when the data are stored in a rectangular grid, there will be places there is no data.
These empty values are called missing, gaps, or fill values. They will contain a spe-
cial number that is not used elsewhere in the data, which for continuous data, com-
mon values are NaN (not a number), –9999.9, –999.9, or –999. These numbers are
not particularly common in nature and thus won't usually be confused with mean-
ingful observations, 0 is usually not used as a fill number because it becomes dif-
ficult to distinguish low values of observed data with values that are outside of the
satellite scan. Data can also contain integers, for example data flags, which are
integer codes that categorize the data, which may indicate the data quality, or
identify if the scene is over land or ocean. For integer data, some common fill
values may include 0, 128, –127, 255, and 32,766. These numbers are endpoint
values for the numeric data types shown in Table 3.1.

3.2. Data Formats

Data must be combined to provide a complete description of an observed

value. Most satellite datasets have a primary empirical variable of interest (e.g.,
precipitation). Then, to display and understand these values, we need abstract sup-
porting data such as latitude, longitude, and data quality flags. Metadata, such as
fill values, helps us understand information about the empirical variable, such
where the satellite is taking observations in a granule. In a more general sense, sci-
entific data can be broadly classified into three forms:
• Empirical data are observed measurements, including air temperature, precip-
itation, lightning flash area, or any other measured (or estimated) physical or
countable quantity.
• Abstract data describe how the empirical data are indexed or stored. For
example, latitude, longitude, time, and height are abstract data that describe
empirical data like air temperature. Other, nonphysical coordinates are also
classified as abstract.
• Metadata describes information, formatting, and constraints of empirical
data. Metadata includes attributes of the data, such as the valid number
ranges and if measurements are missing. Metadata can apply to the
entire dataset by describing the agency source, the sensor it came from,
the algorithm version that created it, and the conventions it uses for
storage.
Without well-documented supporting data, even highly accurate empirical
data are not useful. As mentioned in the introduction, analysis-ready data must
be well-understood and thus would include all the above to ensure data are
self-explanatory, efficiently stored, and easily readable. In the following sections,
I will describe several data storage methods that are found in the Earth sciences,
their strengths and weaknesses, and suitability for analysis and sharing.

0005078642.3D 31 12/7/2021 1:16:29 PM

 32 Earth Observation Using Python

3.2.1. Binary

In our daily lives, we use the base-10 numerical. Our counting numbers are
0–9, and then digits increase in length every tenth number. Alternatively, com-
puter data are natively stored in base-2 or binary, which takes take on two discrete
states, 1 or 0 and can be thought of as “on” or “off.” Each state is called a bit. Each
additional bit doubles the previous amount of information: one bit allows you to
count 0 or 1 (two possible integers), two bits allowed you to count to 3 (four inte-
gers), three bits allows you to count to 7 (8 possible integers), and so on (Table 3.3).
Note that 8 bits = 1 byte.
Binary data are aggregated into a dataset by structuring bits into sequential
arrays. Reading a single-dimension sequence of numbers is relatively simple. For
instance, if you know the data type, you can calculate the dimensions of the data
by dividing the byte size by the file size. For example, if you have a 320-bit file and
you know it contains 32-bit floats, then there it is an array with length 10. How-
ever, if you have a multidimensional array, the read order of the programming
language will matter. If the 32-but file in the previous example is a 5 × 2 array,
then it’s not obvious if you are reading the row or the column. Row-major order
(used by Python’s NumPy, C/C++, and most object-oriented languages) and col-
umn-major order (Fortran and many procedural programming languages) are
methods for storing multidimensional arrays in memory (Figure 3.4). By conven-
tion, most languages that are zero indexed tend to be row major, while languages
where the index starts with 1 are column major. If you use the wrong index, the
program will read the data in the incorrect order.
The default byte read order (the endianness) varies from one operating system
or software tool to another; little endian data are read from right to left (most sig-
nificant data byte last) and big endian is read from left to right (most significant
data byte first). You can often tell if you are reading data using the wrong system if
you have unexpectedly large or small values.

Table 3.3 Comparing Increasing Numbers in

Base-10 to Base-2 (Binary)

Base-10 Binary

0 0000
1 0001
2 0010
3 0011
4 0100
5 0101
6 0110
7 0111
8 1000

0005078642.3D 32 12/7/2021 1:16:29 PM

 A Deep Dive into Scientific Data Sets 33

Row-major order

Column-major order

Figure 3.4 Read order for row and column major.

In summary, binary files can be efficiently organized to compactly store

empirical data and even abstract data, such as geolocation information. Metadata
can also in theory be stored in binary files. However, there is no standard organ-
ization or formatting in binary data, so binary data cannot be read unless it
includes written documentation describing the internal structure.

3.2.2. Text

Most of us have probably already seen or used text data. Some advantages of
text-based data are that they are easily readable, so you can visually check if the
imported data matches the input file formatting. However, text data also has dif-
ferent encoding methods and organization. American Standard Code for Informa-
tion Interchange (ASCII) was released in 1963 and is one of the earliest text
encodings for computers. When viewed, text files look like alphanumeric charac-
ters, but under the hood ASCII character encoding consists of 128 characters
which are represented by 7-bit integers.
Originally, ASCII could only display Latin characters and limited punctua-
tion. ASCII was followed by the Unicode Transformation Format – 8-bit
(UTF-8) which produces 1,112,064 valid code points in Unicode using one to four
8-bit bytes. Since 2009, UTF-8 has become the primary encoding of the World
Wide Web, with over 60% of webpages encoded in UTF-8 in 2012. Under the
hood, UTCF-8 is backward compatible with ASCII data; the first 127 characters
in Unicode map to the integers as their ASCII predecessor.
While all languages can be displayed in UTF-8, UTF-16, and UTF-32 are
becoming more frequently used for languages with more complex alphabets, such
as Chinese and Arabic. Many scientists will usually not notice these distinctions

0005078642.3D 33 12/7/2021 1:16:29 PM

 34 Earth Observation Using Python

and may use the terms interchangeably. However, knowing the distinction is
important, as using the wrong format can introduce unwanted character artifacts
that can cause code to erroneously read files.
Of plain text data formats, comma-separated values (.csv files) and tab-
separated values (.tsv files) are highly useful for storing tabular data. Below is
an example of a csv file:

STATION_NUM, STATION_NAME, LATITUDE, LONGITUDE,

ELEVATION USW00014768,ROCHESTER, NY, 43.1167, -77.6767,
164.3
USC00309049,WEBSTER, NY, 43.2419, -77.3882, 83.8

An example of a tsv dataset is the following METeorological Aerodrome

Reports (METAR), which is a text-based format used by pilots for aviation
planning.

KBWI 291654Z VRB04KT 10SM FEW045 27/13 A2999 RMK AO2

SLP155 T02720133

Note that just because text data are “human readable,” does not mean it is
“human understandable” without additional explanation. However, the above
code is very “machine readable” because it follows consistent formatting and uses
a common set of codes to describe the current weather conditions. For instance,
the above METAR can be translated from “machine readable” into a “human-
readable” description using a text-parsing algorithm written in Python:

Location: BWI Airport

Date: August 29, 2019
Time: 16:54Z
Winds: variable direction winds at 4 knots
Visibility: 10 or more sm
Clouds: few clouds at 4500 feet
AGL Ceiling: at least 12,000 feet
AGL Pressure (altimeter): 29.99 inches Hg
Sea level pressure: 1015.5 mb
Temperature: 27.20 C
Dewpoint: 13.3 C

It is somewhat arguable that METARs in the first form are a “user-friendly

aviation weather text product” as described on the official aviation weather web-
page. But to make it human-understandable, the above code can be easily
imported and parsed by Python. In this way, a user can write a function to convert
the data to make it understandable to a layperson.

0005078642.3D 34 12/7/2021 1:16:29 PM

 A Deep Dive into Scientific Data Sets 35

Like binary data, text files can store empirical and abstract data, and some
simple metadata. However, there are some disadvantages of character code.
For one, the data require much more storage space than binary data formats,
which do not require as many bytes to store the data. Second, formatting csv
and tsv files needs to be carefully organized to be machine readable: column head-
ers may not match the number of columns below, the separating or delimiting
character may be inconsistent, or there may be missing values. Third, sometimes
multiple variables are stacked into the same file, rather than organized by rows,
and a special reader must be designed to import the data.
In the next section, I will discuss self-describing datasets, which in more recent
years have become the primary method of storing large satellite datasets. Self-
describing datasets combine the compactness of binary data with the human-
readable descriptions of text files so that users can understand the structure, met-
adata, and formatting of the stored empirical and abstract data.

3.2.3. Self-Describing Data Formats

Self-describing formats are the main vector of communicating satellite data.

In terms of their design, they are a collection of binary data organized in a stan-
dardized way. Because multiple datasets are easily stored in a single file, you can
open the file and learn which empirical variables are available, the time of the
overpass, data quality information, and the geolocation information if the data
are gridded. With these descriptors, the data are ready to use off the shelf. For
instance, if you want to plot aerosol optical depth (AOD), you will be able to
see exactly where in the file the data are located as well as some other useful infor-
mation like the units of the data. If you are comfortable navigating directories on
your laptop, you will find the layout of the binary files similar.
Self-describing formats are powerful, but I admit that they can be incredibly
challenging to learn initially. However, the reward is that your knowledge is trans-
ferrable to many satellite datasets, not to just to one. The increased use of standar-
dized self-describing formats has helped Earth science dramatically where
consistent formats have made it easier to disseminate data through centralized
data portals. In contrast to the Earth sciences, the space sciences community con-
tinues to struggle with a myriad of formats and access points and where previously
each product developer would use any format or structure of their choosing – a
hurdle for early career scientists.
What are the advantages of self-describing binary data over plain text ASCII data
that make this additional complexity worthwhile? The reason is primarily economical
– both in a monetary sense and a computational one. On the NASA Earthdata portal
alone, there are over 850 datasets that extend back to the beginning of the satellite era.
As discussed at the beginning of this chapter, early records were stored on hard drives,
which over time degraded in warehouses (James et al., 2018). Compact formats enable
agencies to retain longer records. Binary-based formats take up less space on a storage

0005078642.3D 35 12/7/2021 1:16:29 PM

 36 Earth Observation Using Python

disk than text data. Each byte of binary data can hold 256 possible values whereas
unextended text (text that is exclusively machine readable) can only store 128 and
human readable text is even less efficient. Computationally, it is faster to read in
binary-based datasets than text, which needs to be parsed before being stored into
a computer’s memory. Because the files are more compact, binary formats are com-
monly used to store large, long-term satellite data.
The most common self-describing format is the Hierarchical Data Format
(HDF5). HDF files are often used outside of the Earth sciences, as they are useful
for storing other big datasets. The next common format is Network common data
format (netCDF4), which is derived from HDF. Standards for netCDF are hosted
by the Unidata program at the University Corporation for Atmospheric Research
(UCAR). As a result, these files are very often found in the Earth sciences. NOAA
remote sensing datasets are almost exclusively netCDF, and more recently, NASA
produced datasets have been more frequently stored in netCDF4. Please note that
older versions of the datasets (HDF4, netCDF3) do exist but are not necessarily
backward compatible with the Python techniques presented in this book.
Before getting started with any data-based coding project, it is a good practice
to inspect the data you are about to work with. Self-describing datasets are a type
of structured binary format. While the variable data itself is binary, it is organized
into groups. I recommend that after opening the dataset, you examine which spe-
cific elements within it are of interest to you. Additionally, self-describing datasets
can have both local and global attributes, such as a text description of the variable,
the valid range of values, or the number that is used to indicate missing or fill,
values (Figure 3.5). In self-describing formats you do have to know where in
the large array your variable of interest is stored. You can extract it by using
the variable name, which points to the address inside of the file. Because the data
can be complex, it is worthwhile to inspect the contents of a new dataset in
advance. You can utilize free data viewers to inspect the dataset contents, such
as Panoply (NASA 2021). Python or command line tools also allow you to exam-
ine what is inside the files. Planning your coding project with knowledge of the
data will allow you to work efficiently and overall will save you time.
The variables can be one-, two-, three-dimensional variables, or more. HDF
files may organize the variables into groups and subgroups. While variables can be
organized into groups in netCDF files, Unidata compliance standards do not
recommended grouping. Most often, data can be separated into other variables.
If the data have the same grid and there is a temporal element, it is more efficient to
increase the dimensionality of the variables and stack the matrices in time.

3.2.4. Table-Driven Formats

Binary data that take on table-driven code form require external tables to
decode the data type. Thus, they are not self-describing. These files follow a meth-
odology of encoding binary data and not a distinct file type. Binary Universal
Form for the Representation of meteorological data (BUFR) and GRIdded

0005078642.3D 36 12/7/2021 1:16:29 PM

 A Deep Dive into Scientific Data Sets 37

NetCDF file = container

2D Variable Dimensions: 3200 × 768

Variable Attributes
Name:
Coordinates
768 Fill Value
Valid Range
...
3200

3D Variable Dimensions: 3200 × 768 × 3

Attributes
Name
Coordinates
Fill Value
768 Valid Range
...

3 3200

Figure 3.5 Example of how netCDF data are organized. Each variable has metadata that
stores units, fill values, and a more detailed description of the contents.

Binary (GRIB) are two common table-driven formats in Earth Sciences, but they
are specific to certain subject areas. I will not cover these formats in significant
detail in this text but will mention them here for your awareness.
• BUFR. BUFR was developed by the World Meteorological Organization
(WMO). Many assimilated datasets are stored in this format. BUFR files
are organized into 16-bit chunks called messages, with the first bits represent-
ing the numeric data type and format codes followed by the bit-stream of the
data. The software that parses these files uses external tables to look up the
meaning of the numeric codes, with the WMO tables as the field standard.
BUFR Table A is known as the Data Category. If the Data Category bit
stream contains the number 21, the Python parser can use the tables to see
that this code corresponds to “Radiances (satellite measured).” An advantage
of BUFR is that the descriptions of the data are harmonized and has superior
compression to text-based formats. When BUFR files follow standards, they
are more easily decoded than plain binary files. However, if the stored data
does not conform to the codes in the WMO tables, then the data must be dis-
tributed with a decoding table (Caron, 2011).

0005078642.3D 37 12/7/2021 1:16:29 PM

 38 Earth Observation Using Python

• GRIB2. American NWS models (e.g., GFS, NAM, and HRRR) and the Euro-
pean (e.g., ECMWF) models are stored in GRIB2. While they share the same
format, there are some differences in how each organization stores its data. Like
BUFR, GRIB2 are stored as binary variables with a header describing the data
stored followed by the variable values. GRIB2 replaced the legacy GRIB1 for-
mat as of 2001 and has a different encoding structure (WMO, 2001).
Models merit discussion in this text because researchers often compare models
and satellite data. For example, models are useful for validating satellite datasets or
supplementing observations with variables that are not easily retrieved (e.g.,
wind speed).
At the time this text was published, many Python readers had been developed
and tested with ECMWF because historically, most Python developers have been
in Europe. For instance, some of the GRIB2 decoders have problems parsing the
American datasets because the American models have multiple pressure dimen-
sions (depending on the variable) while the European models have one. Still, there
are ways the data can be inspected by using the pygrib and cfgrib packages, which
were described later in Section 2.4.

3.2.5. geoTIFF

GeoTIFF is essentially a geolocated image file. Like other image files, the data
are organized in regularly spaced grids (also known as raster data). In addition,
GeoTIFF contains metadata that describes the location, coordinate system, image
projection, and data values. An advantage of this data format is that satellite
imagery is stored as an image, which can be easily previewed using image software.
However, like text data, the data are not compact in terms of storage. While gaining
popularity in several fields, geoTIFF is most used for GIS applications.

3.3. Data Usage

There is a lot of technical jargon associated with satellite data products. How-
ever, having a working knowledge of the terminology will enable you to under-
stand how to appropriately choose what data to use. Below, I discuss how
several data producers (e.g., NASA, NOAA, ESA) define the processing levels,
the level of data maturity, and quality control. The timeliness of the data may also
be of importance, so we describe what is meant by data latency. Finally, the algo-
rithms used to calibrate and retrieve data change over time, so it is important to be
aware of what version you are using for your research.
3.3.1. Processing Levels

Data originating from NASA, NOAA, and ESA are often assigned a proces-
sing level, numbered from 0 to 4 to help differentiate how processed the data are
(Earthdata, 2019; ESA, 2007). Specific definitions can vary between agencies, but

0005078642.3D 38 12/7/2021 1:16:30 PM

 A Deep Dive into Scientific Data Sets 39

in general, Level 0 is raw, unprocessed satellite instrument data that is typically

encoded in binary at full resolution and is not often used for research or visuali-
zation. Instead, Level 0 data are used inside data production environments, which
process the raw data from a binary format to understood units such as radiance,
bending angles, and phase delays (Level 1 data). Level 1 data are still minimally
processed (e.g., converted to an HDF or BUFR) and is further divided into three
categories (A, B, or C) with some subtle distinctions. Level 1A data may contain
the correction and calibration information in the dataset, but it is not applied to
data. Calibration ensures that the raw measurements are consistent and meet spe-
cific requirements for scientific research and numerical weather prediction. Level
1B has calibration applied but is not quality controlled and is not spatially or tem-
porally resamples. Level 1C can be resampled, calibrated, or corrected, and may
also include quality control (Section 3.2.3). Level 2 data are geophysical variables,
such as precipitation rate, AOD, surface type, or insolation. Conversion from
Level 1 to Level 2 is not straightforward, in part because raw signals such as elec-
tromagnetic radiation are often only indirectly linked to the desired variable. Fur-
thermore, there are not always enough channels to obtain unique solutions
without further assumptions or constrains. Retrieval algorithms are developed
to process Level 1 data along with ancillary datasets to make a reasonable estima-
tion of geophysical variables such as temperature, atmospheric composition, or
surface type. Levels 1 and 2 data are often stored in a granule, which is a combi-
nation of all observation within a given time period (e.g., 1 minute of data). In
terms of data management, there will be many relatively small files. Level 3 data
are spatially and temporally resampled geophysical variables from Level 2 data.
For instance, Level 3 data may combine Level 2 data from multiple sensors to cre-
ate a global snapshot or aggregate instantaneous measurements to produce
monthly mean values. Level 4 data synthesizes Level 1–3 satellite observations
with models to create a consistent and often global, long-term reanalysis of the
geophysical state. Depending on aggregation and number of available variables,
Level 3 and 4 files may be significantly larger than a single Level 2 file. Table 3.4
shows a short summary with a few sample products that fall into the categories.

Table 3.4 Levels and Examples of Transformations Performed on the Data

Level Processing Sample Dataset(s)

0 Raw data N/A

1 Calibrated or uncalibrated radiances or brightness VIIRS or ABI radiance
temperatures data
2 Products, converted to estimate a physical quantity Dark Target AOD Product
from Level 1 data
3 Combination of multiple Level 2 data, daily or Hourly CMORPH-2,
monthly average Monthly NDVI
4 Combination of remote sensing, in-situ, or model Merra-2 Global Reanalysis
data

0005078642.3D 39 12/7/2021 1:16:30 PM

 40 Earth Observation Using Python

Generally, Level 3 and Level 4 data may be more appropriate to study long-
term trends or teleconnections. For real-time environmental monitoring or region-
specific analysis, Level 2 datasets may be more useful. Modelers may be interested
in assimilating radiances from Level 1 data, and on occasion Level 2 data, such as
for trace gas assimilation.

3.3.2. Product Maturity

When new NASA or NOAA satellites are launched, there is a test period
where the products are evaluated, which is sometimes referred to as the “alpha”
or “post-launch” phase. After this test period, so long as the data meet minimum
standards, the data are in the beta stage. Beta products can sometimes restricted to
the public, although access may be requested by users from the agencies under spe-
cific circumstances. The next two stages are provisional and validated. At these
maturity stages, you can utilize these datasets with more confidence. Both provi-
sional and validated datasets are appropriate for scientific research.
In terms of appropriate use, data that have beta maturity or below really are
not scientifically rigorous. This data may still contain significant errors in the
values and even geolocation. There might be certain applications where using beta
stage data are appropriate, such as if you want to do a feasibility study, want to
simply become familiar with the data, or if you are evaluating the data accuracy
and precision. If these data are used in published scientific research, you should
clearly state the maturity level in your manuscript or presentation.
In terms of time scales, advancement through maturity stages is slower for
brand-new sensor, anywhere about one to two years from launch. Level 1 products
are always the first to become validated – they have fewer moving parts and pro-
duce the simplest of measurements. Level 2 products can at times rely on other
Level 2 products as inputs. For instance, many of the GOES-16 products rely
on the Level 2 Clear Sky Mask to determine if a scene is clear or cloudy. So,
the Clear Sky Mask had to reach provisional before other products could.
Logistically, this can complicate the review process for downstream products.
For a series of satellites, the review process is expedited. For instance,
GOES-17 was a twin satellite to GOES-16, so the review process for GOES-17
products was faster than for GOES-16 products.

3.3.3. Quality Control

A sensor’s ability to retrieve an atmospheric variable is strongly influenced by

underlying atmospheric and surface conditions. For example, instruments that
measure visible or IR cannot penetrate cloud tops, so the “best” surface observa-
tions occur over cloud-free scenes. Bright background surfaces like snow, desert
sand, or sun glint on the ocean can also impact the accuracy of the retrieval. Sat-
ellite datasets often include quality control flags to inform users if and where the
dataset contains degraded or unphysical observations.

0005078642.3D 40 12/7/2021 1:16:30 PM

 A Deep Dive into Scientific Data Sets 41

There is no standard for communicating quality control across different

retrievals. This is because retrieval algorithms are complex and the needs of the
user community are diverse. Some datasets will use a binary flag ranked as good
(0) or bad (1), while other datasets reverse the numbers. Others will use categorical
labels such as high, medium, or low quality or a continuous scale, 0–100%. Some
do not use a single measure but rather combine many bit flags, which can indicate
data quality according to specific criteria.
For robust scientific research, you will almost always want to use the best-
quality data. However, this, in turn, reduces the number of available observations
for your study. In some cases, it might be reasonable to keep the top-two quality
flags (high and medium) and only discard the lowest-quality data. There are situa-
tions where examining all available data are valuable, such as in data sparse
regions where no other measures are available.

3.3.4. Data Latency

For many scientists, it is important to know when to access data as much as

where and how to access the data. Most research scientists are content with
accessing data days, months, or years after the observations are made. Often
the resulting work is built on long-term means or case studies of a past event.
However, those involved with hazard management, such as flood or severe
weather forecasting, will need to access the data sooner. Data that are provided
immediately after downlinking from the satellite is known as near real-
time data.
As described in Section 1.3, Level 0 data are downloaded from the satellite to
ground downlink stations. From there, antennas distribute the observations to
data processing centers around the world. Level 1 data are often released first,
since they have the least amount of processing, and often no dependency on other
products, and are prioritized for assimilation into forecasting and climate models.
Level 2 and Level 3 datasets will take longer to be released; they require more pro-
ducts as inputs and may call computationally expensive routines such as radiative
transfer models. Recall from Section 1.3, data latency is the time difference bet-
ween when a satellite makes an observation from when the data are made available
to the end user. The latency at which a user receives the data is determined by how
often a satellite is downlinked to the Earth, the speed of the retrieval algorithm,
bandwidth, and the needs of data users. To decrease latency over the United
States, observations from some polar orbiting satellites are downloaded by a net-
work direct broadcast receivers. Retrievals are then processed locally using the
free, open-source Community Satellite Processing Package (CSPP) developed
by the University of Wisconsin. Each individual site can then internally use (or
share via FTP) the data at a faster pace. Figure 3.6 shows the antenna sites across
the United States and Pacific.
A disadvantage of direct broadcast is that the receiver can only download the
local data from polar orbiting satellites during the short time that they are in range.

0005078642.3D 41 12/7/2021 1:16:30 PM

 42 Earth Observation Using Python

WA NH
MT VT ME
ND
OR MN
ID MA
SD WI NY
WY MI RI
PA CT
NE IA
NV OH NJ
IL IN
UT DE
CA CO WV
KS MO VA MD
KY
NC D.C.
TN
AZ OK
NM AR SC
MS AL GA
TK LA
AK
FL

HI Puerto Rico
U.S.
(PR)
Virgin
Islands (VI)
Guam
(GU)

Figure 3.6 Direct broadcast (DB) antenna sites, which can provide data in real-time.
Source: Joint Polar Satellite System/Public Domain.

Data from multiple receivers can be combined into a larger, regional image. How-
ever, global view of the data is only possible at two satellite ground stations located
in Svalbard, Norway, and Troll, Antarctica.
IMERG, a global Level 3 precipitation product that combines many passive
microwave sensors from various platforms, is critical for flood monitoring in
remote regions without access to radar. However, the logistics of combining obser-
vations and running the retrieval make distribution challenging. The solution that
was adopted was to produce three versions of the dataset:
• Early run: Produced with a latency of 4 hours from the time all data are
acquired.
• Late run: Produced with a latency of 12 hours.
• Final run: Research grade product that is calibrated with rain gauge data to
improve the estimate. The latency is 3.5 months.
The required timeliness of datasets varies between different agencies, mis-
sions, and priorities. NASA datasets tend to focus on long-term availability
and consistency of the data to create climate records. In contrast, NOAA datasets
tend to focus on capturing the state of the atmosphere and oceans in the present to
serve operational forecasting. So, you will likely find more real-time datasets gen-
erated by forecast agencies such as NOAA.

0005078642.3D 42 12/7/2021 1:16:30 PM

 A Deep Dive into Scientific Data Sets 43

3.3.5. Reprocessing

Over time, research leads to improvements in the quality of retrieval algo-

rithms. For the data distributors, they may choose to simply make the switch
and not change prior data, or they may choose to reprocess what is often years
of data to make the entire record consistent. NOAA data are seldom reprocessed
because the organization prioritizes forecasts and warnings. NASA records tend
to be available for longer periods of time and are generated using a common
retrieval algorithm. To ensure that you are working with the same version of
the algorithm, this information is typically stored in the global metadata of the
HDF or netCDF files.

3.4. Summary

In this chapter, you learned some of the main scientific data types and com-
mon terminology. In the past, satellite data was distributed as text or binary data-
sets, but now netCDF and HDF files are more common. These self-describing
data are advantageous because they contain descriptive metadata and important
ancillary information such as georeferencing or multiple variables on the same
grid. There is a significant amount of remote sensing data, so it can be overwhelm-
ing for new users to know which data are appropriate to use for research or envi-
ronmental monitoring. Scientific data from major international agencies often
classify them according to processing level to convey information on the degree
of calibration, aggregation, or combination. Furthermore, quality control flags
within the datasets can help discriminate which measurements should and should
not be used.
Because no two satellite datasets are identical, universally understood formats
such as netCDF or HDF make it easier for scientists to compare, combine, and
analyze different datasets. Since there is ongoing research to improve retrieval
algorithms, self-describing formats can also include production information on
the version number and maturity level of the data from new satellite missions.
Overall, unification of datasets, language, and production promotes analysis-
ready data (ARD) so that scientists can more easily use tools like Python for
research and monitoring.

References

Balaraman, Goutham. (2017, March 14). Numpy vs Pandas performance comparison.

Retrieved April 5, 2020, from http://gouthamanbalaraman.com/blog/numpy-vs-pan-
das-comparison.html
Caron, J. (2011). On the suitability of BUFR and GRIB for archiving data. Unidata/
UCAR. Retrieved from https://doi.org/10.5065/vkan-dp10.

0005078642.3D 43 12/7/2021 1:16:30 PM

 44 Earth Observation Using Python

Dwyer, J. L., Roy, D. P., Sauer, B., Jenkerson, C. B., Zhang, H. K., & Lymburner, L.
(2018). Analysis ready data: Enabling analysis of the Landsat archive. Remote Sensing,
10(9), 1363. https://doi.org/10.3390/rs10091363
Earthdata (2019). Data processing levels. Retrieved April 18, 2020, from https://earthdata.
nasa.gov/collaborate/open-data-services-and-software/data-information-policy/data-
levels/
ESA (2007, January 30). GMES Sentinel-2 mission requirements. European Space Agency.
Retrieved from https://www.eumetsat.int/website/home/Data/TechnicalDocuments/
index.html
James, N., Behnke, J., Johnson, J. E., Zamkoff, E. B., Esfandiari, A. E., Al-Jazrawi, A. F.,
et al. (2018). NASA Earth Science Data Rescue Efforts. Presented at the AGU Fall
Meeting 2018, AGU. Retrieved from https://agu.confex.com/agu/fm18/meetingapp.
cgi/Paper/381031
Meier, W. N., Gallaher, D., and G. C. Campbell (2013). New estimates of Arctic and Ant-
arctic sea ice extent during September 1964 from recovered Nimbus I satellite imagery.
The Cryosphere, 7, 699–705, doi:10.5194/tc-7-699-2013.
Murphy, K. (2020). NASA Earth Science Data: Yours to use, fully and without restrictions
| Earthdata. Retrieved April 17, 2020, from https://earthdata.nasa.gov/learn/articles/
tools-and-technology-articles/nasa-data-policy/
NASA (2021). Panoply, netCDF, HDF, and GRIB data viewer. https://www.giss.nasa.gov/
tools/panoply/
Precipitation Processing System (PPS) At NASA GSFC. (2019). GPM IMERG early pre-
cipitation L3 half hourly 0.1 degree x 0.1 degree V06 [Data set]. NASA Goddard Earth
Sciences Data and Information Services Center. https://doi.org/10.5067/GPM/IMERG/
3B-HH-E/06
WMO (2003, June). Introduction to GRIB Edition1 and GRIB Edition 2. Retrieved from
https://www.wmo.int/pages/prog/www/WMOCodes/Guides/GRIB/Introduction_-
GRIB1-GRIB2.pdf
Special thanks to Christopher Barnet for his description of historic plotting
routines.

0005078642.3D 44 12/7/2021 1:16:30 PM

Random documents with unrelated
content Scribd suggests to you:
 “About my Gunnie,” was the prompt and shrill-noted reply. “I
want ’o know just what call you’ve got to come between Gunboat
and me after we’ve been going together for a year and a half! I
want ’o know what right, just b’cause you’re rotten with money,
you’ve got to turn a poor boy’s head and have him say the things
that Gunnie’s just been sayin’ to me! I want——”
“Ruby,” interrupted Teddie, steadying herself, “you are saying
things yourself that are utterly ridiculous. I haven’t either the
intention or the desire to come in any way whatever between you
and the young gentleman you speak of as Gunnie. I——”
“Then just why were you usin’ me, me of all people, to make a
date with him not more than twenty-four hours ago?” demanded the
irate voice over the wire. “And if there’s nothin’ to that, just why is
he runnin’ round in your car to-day?”
“In my car?” echoed Teddie.
“Yes, and bumpin’ into a Fifth Avenue bus with it and havin’ the
ink sleuths from the canary-colored evenin’ papers comin’ and
frightenin’ his poor old mother into a nervous breakdown?”
It took a little time for Teddie to digest this.
“But, my dear girl,” she finally explained, “your Gunnie has no
more claim on that car of mine than he has on me.”
“Well, he thinks he has. And he’s so sure of it he’s even been
advertisin’ that you know he has. And I’ve been goin’ with Gunnie
long enough to realize that that boy never told a lie in his life.”
This declaration of faith in Gunboat Dorgan was followed by a
moment or two of unbroken silence.
“Ruby,” finally called out the bewildered girl at the telephone, “I
want you to come here. I want to see you. I must see you at once.”
“From the way things are breakin’,” clearly and coldly announced
the lady on the other end of the wire, “I don’t think it’s me you want
to see. You’d better do your talkin’ to my lawyer!”
“Ruby!” called the girl at the desk.
 But the wire brought no answer to that repeated call, and Teddie
hung up the receiver. She placed it slowly and carefully on its hook
and sat staring at the cadmium tinted wall, with a look of helpless
protest on her bewildered young face. And for the second time she
found herself face to face with a forlorn and seemingly fruitless
survey of her resources.
Once or twice, in her desperation, she was even tempted to pack
up and scurry off to Hot Springs in the wake of her Uncle Chandler.
But that, she remembered, would be more than cowardly. It would
be foolish, for it would be nothing more than a momentary evasion
of the inevitable. And besides being a sacrifice of dignity, it would
stand as an advertisement of guilt.
Then out of a world that seemed as cold and empty as a glacial
moraine came one faint glow of hope. On the gray sky-line of a
Sahara of uncertainties appeared a tremulous palm-frond or two. For
Teddie, in her misery, had suddenly taken thought of Gerald
Rhindelander West. Gerry, she remembered with a gulp, was not
only one of her own set, but also a corporation lawyer. It wouldn’t
be easy to explain things to Gerry. It would, in fact, involve sacrifices
of pride which made her wince without knowing it. But she had
talked about having an attorney. And it was her duty to find one.
 CHAPTER NINE
When Teddie made ready for her conference with Gerald
Rhindelander West she did so with a particularity which might have
surprised both her recently abandoned maid and the immediate
members of her own family. She went forth to the terra incognita of
Nassau Street cuirassed in tailored and braided trimness and
gauntleted in spotless kid with just the right array of wrinkles about
her glove-tops. She was still further entrenched behind a four-skin
scarf of blue-fox—which wasn’t blue at all—and a canteen muff to
match, to say nothing of seven cyanitic-looking orchids which
completed the color-scheme and fluttered demurely above her
slightly fluttering heart.
For it wasn’t often that Teddie was as excited as she found
herself that morning, just as it wasn’t often that she had turned to
give ponderable thought to the question of armor-plate. But it
loomed up before her as a serious matter, this commandeering of a
clever young attorney to her side of the case, and she felt the need
of not producing an unfavorable impression on Gerry.
Yet even after she had unearthed Gerry’s aerial office-suite in
that seldom explored warren of industry known as Nassau Street,
she found the attorney in question not quite so accessible as she
had anticipated. For she was compelled to send in a card, and cool
her heels in an outer room, and even after being admitted to the
royal presence had to wait for a further minute or two while Gerry
instructed an altogether unnecessarily attractive stenographer as to
the procedure in manifolding a somewhat dignified array of
documents.
He seemed still preoccupied, in fact, as he seated Teddie in a
chair at his desk-end and absently took her muff and put it down
and motioned away a secretarial-looking intruder and crisply asked
just what he could do for her.
 Teddie found it hard to begin. She made two false starts, in fact,
before she was able to begin. And then she refused to be further
intimidated by the paraded professional dignity of a person who’d
once helped her paint zebra-stripes on a Jersey cow.
“Gerry, do you know Raoul Uhlan?” she found herself quite
casually inquiring.
Gerry pondered the question for a moment. He was really
thinking, all the time, how extraordinarily lovely Teddie could look in
blue-fox.
“He’s a man whom I have the privilege of not knowing,” was
Gerry’s retarded but none the less satisfactory reply. “Why do you
ask?”
“Because he’s suing me for twenty-five thousand dollars,” was
Teddie’s altogether unexpected announcement. Gerry, however,
seemed determined to remain immobile.
“Not for breach of promise?” he asked, with an air of diffidence.
“No; it’s for what I suppose you’d call breach of the peace,”
explained his client.
“What did you do?” inquired Gerry, with vivid but secret
memories of the Nero incident.
“I had his nose thumped,” acknowledged Teddie with vigor.
“Why?” asked Gerry, wondering why his mind kept straying back
to one-eyed Russian rats.
Teddie hesitated. It wasn’t an easy thing to talk about. That was
a lesson she had already learned. But Gerry was different. He was
one of her own world and one of her own set, and he’d look at the
thing in the right way, in the only way.
“Why?” he repeated, secretly astounded by this new mood of
humility in which he found Teddie Hayden immersed.
“Because he tried to kiss me,” acknowledged Teddie, meeting
Gerry’s unwavering gaze.
“Fine!” said Gerry, as cool as a cucumber. “But who did the
thumping?”
 “A prize-fighter by the name of Dorgan—Gunboat Dorgan.”
“Better still,” calmly agreed her interlocutor, “for that implies it
was a genuine professional thumping.”
“It was,” conceded Teddie. She was more than serious, she was
even grim about it all. And if Gerry West had laughed at her, at any
moment of that perilous mood, everything would have been over
between them.
But Gerry was solemnity itself. “Go on!” he said, almost bruskly.
“Now Raoul Uhlan claims that he’s lost a valuable commission
through—through what was done to him. And the young lady who’s
interested in Gunboat Dorgan seems to think because I had him
protect me in this way that I’ve interfered with her claim on this hero
of hers.”
“In what way interfered with him?” demanded Gerry.
“That I’ve—that I’ve made love to him,” acknowledged the none
too happy Theodora Lydia.
“Why do you say that?”
“Because she’s seeing her lawyer about it.”
“And this man Uhlan?”
“He sent his attorney, a man named Shotwell, to my studio to
explain that because of his injuries he couldn’t paint his twelve-
thousand-dollar portrait. I was quite willing to pay for that until old
Shotwell put in another claim for twelve thousand dollars for
damages in general and an extra thousand for himself.”
“So they’re all trying after a bite,” commented Gerry, studying his
engagement-pad. “Now, tell me, Miss Hayden——”
“Don’t do that,” was Teddie’s sharp command.
“Don’t do what?”
“Don’t call me Miss Hayden.”
“All right, Teddie,” acquiesced her counsel-at-law, without a break
in his solemnity. “But the first thing you must tell me is just what you
intend doing.”
 “I don’t know what to do. That’s why I came to see you. That’s
what I’m willing to pay you for. But it’s not entirely unnatural, I
think, to nurse a fixed aversion to be chased around the map by an
army of reporters and subpœna-servers.”
“There are several things, of course, that we can do,” explained
Gerry, quite unruffled by this unmasking of the guns of irony. “But
before we go any further there’s a phase or two of the case I must
understand. It was in your studio, you say, that this assault took
place?”
“I hate that word!” interpolated Teddie.
“Well—er—this incident. Now, had you forbidden this man Uhlan
entry, warned him away, and all that sort of thing?”
“No, he was coming there three times a week, to give me
lessons,” explained Teddie.
“For which he was being duly paid?”
“No, nothing was ever said about his being paid,” she
acknowledged. And Gerry’s increase of gravity didn’t altogether add
to her happiness.
“And the day he got his thumping—why did he come to your
studio on that occasion?”
For the second time Teddie hesitated. Life, after all, wasn’t so
simple as she had once imagined it.
“He came to make love to me,” she finally admitted, not meeting
Gerry’s eyes. “And I had Gunboat Dorgan there to give him what he
deserved.”
Gerry wagged his head. He did so with what impressed Teddie as
quite unnecessary solemnity.
“Now, about this man Dorgan: He knew exactly why he was
doing what he did?”
“Of course!”
“And he expected to be duly paid for this—er—service he
rendered you?” asked Gerry, seeming to persist in his determination
that things should not be made too easy for her.
 “No, he declined to have the matter of money come into it at all,”
Teddie rather falteringly acknowledged.
“Then what was the understanding?”
“There was no understanding.”
“Then what did he do, when the thing was over?”
A silence fell between them.
“He kissed me,” slowly acknowledged truthful Teddie, flushing up
to the tiptilted brim of her hat.
Gerry swung sharply about. He swung about and stared out of
the skyscraper window.
“He had no reason, no excuse, for doing anything like that!”
supplemented the tingling Teddie.
“Didn’t he, now!” silently soliloquized Gerry as he swung slowly
back in his swivel-chair and sat staring at her. Then he added, aloud:
“And what happened after that?”
“He presumed on his privileges to the extent of taking my car out
of the garage and going joy-riding in it.”
“Without your knowledge and permission?”
“Entirely! And bumped into a street-bus and broke my lamps.”
“That’s much better,” Gerry surprised her by saying.
“Why?” asked Teddie, vaguely disturbed by her remembered
failure to mention an offhanded proffer of this same car to this same
knight with the cauliflower ear.
“Because we can settle his hash with a larceny action,” retorted
Gerry. “But our biggest nut to crack, I imagine, will be Uhlan!”
“What can we do about him?” asked Teddie, with the faintest
trace of a tremor in her voice.
“There are quite a number of things we can do,” coolly explained
her solemn-eyed counsellor. “I can have him put out of the
Camperdown Club, for one thing, before the week-end. I can
demand an impartial appraisement of his physical injuries. I can see
Shotwell, this attorney of his, and accept service. I can even get
after ’em for blackmail. And there are several other things I can do.
But each and every one of them will result in exactly the end we are
most anxious to obviate. By that I mean publicity, newspaper talk,
the reporters you spoke of as chasing you all over the map. That’s
the one thing, Teddie, we must not and shall not have!”
“No, we mustn’t have that!” echoed Teddie, mysteriously
comforted by the masterfulness of this new-found sage who could
achieve such a cool-headed and clear-eyed view of the entire
tangled-up muddle. It took a load off her mind, to know that she
had some one so adroit and dependable as Gerry to stand beside
her in this fight against the forces of evil. She felt sorry, in fact, that
she hadn’t come to Gerry in the first place. Then she felt rather glad
in remembering that since she had come to him, she hadn’t come
looking like a frump.
“So the best thing you can do, Teddie,” her new-found adviser
was saying to her, “is to leave this entirely in my hands for a day or
two. All I’m going to ask you to do is to keep mum, to sit tight.
Before the week-end, I feel sure, we’ll have the whole thing
straightened out. And, by the way, what’s the name and address of
your prize-fighter’s lady-friend?”
He remained solemnity incarnate as he jotted Ruby Reamer’s
name and address down on his scratch-pad.
“Has it occurred to you,” he said as he wrote, without looking up,
“that this man Dorgan might have been the proper person for Uhlan
to take action against?”
“I imagine he saw about all he wanted to of Dorgan,” announced
Teddie, with the icicle-look once more in her eyes.
“But not all he wanted to of you?” questioned Gerry, pretending
to ignore her eye-flash of indignation. It was not often that he’d
enjoyed the luxury of finding Teddie Hayden on the defensive, and
he intended to make the most of it. “It’s quite apparent he isn’t
afraid of you!”
“I was hoping you could make him that way,” acknowledged
Teddie. She said it quietly, but there was a barb in it which Gerry
couldn’t quite overlook.
“Well, we’ll get him that way,” he announced with vigor, as he
rose to his feet. “If it’s action they’re after, they’ll get all they want!”
A consciousness of clearing skies both elated and depressed the
brooding-eyed Teddie. What Gerry was doing for her was being done
merely in the way of a professional duty for which he would be duly
paid. But they had been friends once, and she had treated him, she
remembered, rather rottenly of late. She wanted to say something
about that, make some effort to explain it away, yet she didn’t quite
know how to get that belated mood of repentance into words.
So, as she rose from her chair, she didn’t even try to put it into
words. She merely smiled softly and gratefully up into Gerry’s eyes
as he stood beside her, with the magnolia-white of her cheeks
tinging into pink as he stared back at her, with his jaw-muscles set
and a quick look of pain on the face that still remained preoccupied.
“It’s—it’s awfully good of you, Gerry,” she said as she held out
her hand to him.
“That’s how I make my living,” was Gerry’s unexpectedly brusk
reply. But, apparently without knowing it, he still held her hand in
his.
“It’s awfully, awfully good of you,” she repeated, as she reached
with her free hand to restore the scarf which had slipped off her
shoulder.
“It’s not a bit good of me,” he countered, almost harshly, as he
put the scarf back where it belonged. And she would have been
afraid of him, with that sudden black look in his eyes, if she hadn’t
remembered that Gerald Rhindelander West was a gentleman, a
man of her own world and her own way of looking at things. And
she rather liked that touch of camaraderie which was expressing
itself in the unconsidered big-brothery weight of his hand on her
unaverted shoulder.
“I feel so—so safe with you,” she reassured him, with that misty
look in her up-raised eyes which can seem so much like a sigh made
visible. And it was beginning to be a luxury, she felt, to find
somebody she could feel that way with.
“Well, you’re not!” he said in a voice that was almost a bark.
“Why do you mean I’m not?” she asked, perplexed, with a still
more searching study of his face.
“I mean because——”
He did not finish. Instead, with his hand still on her shoulder, he
stooped and kissed her.
Teddie recoiled three full steps, and stood with her arms straight
at her sides and a black rage in her startled eyes. Gerry’s own hands
had dropped to his side, and his head fell forward, for all the world
like a chrysanthemum that needed watering.
“O-o-o-o-o-o!” gasped Teddie, with the most unmistakable
accents of loathing and anger in her voice. “Are all men like that?”
“Wait!” called out Gerry, unhappily, pleadingly.
But Teddie had no intention of waiting. She withered him with
one short look of revulsion, of utter repudiation, wheeled about, and
strode out of the office.
She went, leaving behind her a blue-fox canteen muff and a
much bluer young attorney who for quite a number of minutes stood
staring morose and motionless out over the East River. He
contemplated that wharf-fringed waterway very much as though he
should like to take a header down into it. Then, as he slowly and
dejectedly turned about, his eye fell on the forgotten muff.
He crossed to his desk and took the furry pillow up in his hands,
turning it over and over. He meditatively stroked the deep pelt,
sniffed at it, started for the door, and just as suddenly stopped. Then
he quietly removed two tennis racquets and a box of golf-balls
wrapped in a llama-wool sweater-coat from the bottom drawer of his
desk and into this same drawer carefully tucked away the blue-fox
muff—after which he stood, irresolute and unmoving, for another full
five minutes.
 Then Gerry West, as though to make up for lost time, exploded
into a sudden fury of movement. He punched the buzzer-button for
his stenographer, jerked down the messenger-call lever and caught
up the telephone directory with one hand while he possessed
himself of the receiver with the other.
“I’ll show ’em,” he muttered darkly to himself, “I’ll show ’em they
can’t pull that cave-man stuff around my home circle!”
And in half an hour’s time he had an ex-pool-roomer from a
private detective agency busily shadowing Gunboat Dorgan, and
another quiet-moving agent gathering what data he could as to the
physical disabilities of Raoul Uhlan, and an expeditious clerk from
the outer office confirming the address and movements of a certain
Miss Ruby Reamer. Then, having started these wheels into motion,
he hurriedly looked up a point or two of law, consulted his watch,
and called up Louis Lipsett, of The Star, at the Press Club.
“Louis,” he said over the wire, “I’ve got a great news story for
you.”
“Good!” promptly announced the other.
“Yes, it’s so good, in fact, that you’ve got to come and help me
kill it in the bud.”
“Then let me add that what you want isn’t a reporter, but an
undertaker,” retorted the unfeeling young White Hope of his over-
saffroned daily.
“No, I want you, Louis, and I want you as quick as you can
come,” Gerry coolly averred.
“But why me?”
“Because you’re the only ink-coolie on this Island who’d keep
your word if you once promised to. So come over here in a taxi and
let me unload.”
Louis came, and smoked Gerry’s good cigars, and listened, and
remembered his promise with a true inkster’s pang of regret.
“Now, the one thing that Avenue-robin can’t stand, the one thing
be doesn’t want, in all this, is printer’s ink. So it’s up to us to give
him what he’s afraid of. It’s up to us to hold a full-page Sunday story
over his fat head. I want you to go right up to him as a reporter
from The Star, with every detail I’ve given you. I want you to let him
see just what it’ll look like when it’s unrolled, the entire unsavory
story. And if he isn’t sending a hurry-call in for the soft pedal before
you’re out of the elevator I’ll buy The Star and give it to you to play
with when you’ve got writer’s cramp in the coco and can’t dream up
cable-despatches any more.”
“And supposing our Romeo doesn’t weaken?”
“He can’t help it. But if he’s crazy enough not to, I’ll bring
Gunboat Dorgan up there myself. And if that doesn’t turn the trick,
I’ll call the rotter out myself and give ’im what he deserves. And if
that doesn’t work, I’ll put a bullet into him!”
The man from The Star office smiled a bit wearily.
“Say, Gerry, doesn’t this strike you as going pretty far for a mere
client?”
“A mere client!” echoed the other. “A mere client?” he repeated
as he looked his confederate straight in the eye. “She’s a darned
sight more than that. She’s the girl, please God, that I’m going to
marry!”
“So at last I get you,” announced the solemn-eyed Louis as he
reached over the desk-end and solemnly shook hands with the other
man. “And now I’ll know how to put the screws to that palette-
scraper!”
“Then let’s get busy,” suggested Gerry as he reached for his hat
and coat, after a moment’s talk over the wire. “They’ve got that
Reamer girl for me, and the sooner we have our pow-wow the
better!”
 CHAPTER TEN
When Teddie left Gerald Rhindelander West’s office she left
behind her more than a blue-fox canteen muff. She left the last of
her confidence in life, the last of her belief in mankind. She found
herself compelled to face a world that seemed too big and brutal for
even the valorous spirit of youth. And after a vast amount of frantic
and quite fruitless thinking she also found herself compelled to eat
crow. The current was too strong for her. It had tired her out, and
baffled her, and broken down both her will-power and her pride.
Much as she hated to do it, she felt that her only way out was to
compromise with Raoul Uhlan. Right or wrong, she would pay the
man’s claim and get the thing over with.
A quick assessment of her immediate means, however, showed
her that she had little more than half enough money to meet his
demand. So she promptly stopped in at the Waldorf telegraph desk
and sent a message to her Uncle Chandler at Hot Springs.
“Please wire my banker,” she said, “eleven thousand dollars
without delay or foolish questions, as it is urgent. Lovingly, Teddie.”
Her Uncle Chandler, after frowning for a full hour over this
unexpected message, none too willingly wired instructions for eleven
thousand dollars to be placed to the credit of his niece. Then, after
still another hour of troubled thought, he sent a day-letter off to old
Commodore Stillman at the Nasturtium Club explaining that he had
reason to believe that Theodora was in some sort of trouble and
requesting him to drop quietly down to the girl’s studio and have a
look around to see just what was wrong.
And the Commodore in question, instead of being upset by this
calamitous intimation of beauty in distress, adjusted his cravat and
stopped in at Thorley’s for the insertion of a Richmond rose-bud in
the button-hole of his right-hand lapel. Then he toddled blithely
down to the wilds of Greenwich Village, where he arrived at Teddie’s
studio just in time to see an urbane old gentleman pocket, with an
air of quiet but unqualified satisfaction, a narrow slip of paper which
looked remarkably like a bank-check. He stood aside, however, until
this triumphant-eyed old gentleman had bowed himself triumphantly
out, whereupon it came to his attention that his somewhat
abstracted young hostess remained undeniably divorced from the
customary buoyancies of youth.
He was so impressed, in fact, by the shadows of fatigue about
Teddie’s starry eyes and the world-weariness in her forlorn little
smile that he concluded the gravest fears of his old friend the Major
to be quite well founded. But Teddie, accepting him as an emissary
from a world of pomp and order which seemed eternally lost to her,
was glad enough to ensconce him in the brown velvet armchair and
make tea for him in the battered old samovar. It was not particularly
good tea, he soon discovered, but that in no way dampened his
ardor or discouraged him in the object of his visitation. So he
hummed and hawed, and touched lightly on the prerogatives of the
elderly, and ventured the assertion that New York was an extremely
bewildering city, especially for the young, and he became paternal
and platitudinous over the perils of the wide, wide world in general,
and then with rather awkward unconcern announced his hope that
Teddie was making a go of it.
But Teddie wasn’t making a go of it, as she very well knew, and
for one weak moment she was tempted to take this kindly-eyed and
clean-hearted old gentleman into her confidence and exteriorate her
troubles by freely and frankly talking them over with one of her own
kind. But a revival of her old spirit of independence nipped this
impulse in the bud, so she merely gave the Commodore another cup
of tea and somewhat pensively asked if the autumn ball at Tuxedo
had been a success this year. Whereupon the old Commodore
admitted that it had been a success, if you could call such things a
success. But they weren’t like the good old days of the Patriarchs
and the Assemblies and The Howling Swells. The spirit of the times
had changed, had lamentably changed, and the relationship of the
sexes in the younger generation seemed disturbing to the survivors
of the older era when a lady was accepted as a lady and treated as
one. And from this diatribe on the degeneration of the present day
Teddie’s counsellor glided easily and eloquently into the advantages,
for the girl of to-day, of early marriage and adequate guardianship.
Every girl of spirit ought to marry. Even Teddie herself, he finally
ventured, ought to marry.
“No young whippersnapper, mind you,” discreetly qualified the old
Commodore, “but some older and steadier man who knows the
world and its ways, a man to be relied on in times of trouble, a man
who’d be a harbor of refuge when the seas got to kicking up a bit!”
But this didn’t seem to impress Teddie as he had hoped it would.
“I’ve seen all I want to of men,” she announced with unexpected
passion. “I despise ’em, the whole pack of them!”
“And you don’t intend to marry?” demanded the scion of the
statelier years.
“Never!” retorted Teddie, staring fixedly at her unfinished sketch
of the Macauley Mission by Moonlight.
“Then what, may I ask, do you intend doing?” inquired her stiff-
shouldered old visitor.
She had intended to say that she wanted to live for Art. But she
hesitated. For Art, at that particular juncture, seemed a very anemic
and elusive thing to live for. She had no idea, in fact, just what she
did intend living for. She was less impatient of others than she might
once have been. She even recognized kindliness under the intentions
of that over-personal emissary from her older world, however heavy-
handed he may have been in his executions of those intentions. And
that, impinging on her desolated young spirit, intrigued her into a
brief but depressing mood of self-pity. There was no trace of tears in
her eyes, for Teddie was not habitually lachrymose. But before she
found that mood conquered and killed she was unable to resist the
temptation to let her bobbed head sink wearily into the crooked arm
which rested on one end of the none-too-orderly cherrywood table.
“Oh, I say, you know; this sort of thing won’t do!” ejaculated her
obviously disturbed visitor. “It won’t do, my dear,” he repeated as he
patted what was left of the bobbed hair with his fatherly old hand.
 Teddie, however, was without the spirit either to agree or
disagree with that statement. And her unhappiness so melted the
heart of the benignant old Commodore that he took her hand and
stroked it as he talked to her. And so gratified was he to see even
the ghost of a grim little smile about her lips that a paternally
commiserative impulse prompted him to stoop down and kiss the
magnolia-white cheek.
So intent, indeed, had he been on his contemplation of this white
cheek, faintly shot through with its shell-pink, that the door had
opened and a third person had stepped into the studio without his
being conscious of the fact. And it was the voice of this intruder,
more than Teddie’s sudden recoil of startled wonder, that promptly
brought the Commodore to attention.
“So he’s doin’ it too!” called out Gunboat Dorgan, with a quaver
of incredulity in his Celtic young voice. Whereupon he threw down
his hat and advanced slowly toward the table-end. “Say it quick,” he
commanded. “D’ yuh want me to knock his block off?”
“No, no,” cried Teddie, already on her feet. “There’s been too
much of that already!”
“But I saw the old bird tryin’ to kiss yuh!” proclaimed the
indignant youth.
“Who is this young jackanapes?” interrupted the older man, in no
way intimidated by the interloper with the cauliflower ear.
“Didn’t I see this old mutt pullin’ that muggin’-stuff?” persisted
Gunboat, ignoring the stately old gentleman with the rose-bud in his
lapel. But Teddie was herself by this time and she fixed her
champion from the East Side with a cold and steely stare.
“I want to talk to you!” she said, with great deliberation. And she
made that announcement with such an unlooked-for note of
masterfulness that, unimpressed as it left the newcomer, it rather
bewildered the old Commodore.
“And I guess I gotta earful or two to unload to yuh!” countered
Gunboat, betraying that he was laboring under an excitement which
more recent events had only temporarily eclipsed.
 “I should be obliged to know just who this young bounder is,”
repeated the older man, in his most authoritative quarter-deck
manner. But that manner was entirely lost on Gunboat Dorgan.
“Yuh just play dead, yuh old Has-Been, until I say a word or two
to me lady-friend here,” he proclaimed as he confronted Teddie and
gave his back to an all too negligible enemy. “I came here to find out
what right a law-sharp named West has got to take that car of yours
away from me. I wantta know what call he’s got to load Ruby up wit’
a lot o’ talk about me goin’ to State’s Prison. And I may be a prize-
fighter, but I’ve got the right to ask if I ain’t lived decent and done
my work on the square. I’ve got——”
“A prize-fighter?” interrupted the older man in the background.
Then he strode valorously in between the two. “Do you mean to tell
me, Miss Hayden, that a girl of your antecedents has—has come to
have dealings with——”
But he in turn was destined to interruption.
“Say, d’ yuh want me to throw this old cuff-shooter out o’ here?”
was Gunboat Dorgan’s crisp and angry demand of the girl.
“Stop it!” cried Teddie, with a stamp of the foot. “Stop it, right
here and right now! I’m tired of all this. I’m so tired of it I can’t
stand another moment of it!” Then, with a deep breath, she turned
about to the old gentleman with the rose-bud in his button-hole.
“It’s been very kind of you, I’m sure,” she said in a voice of
laboriously achieved patience, “but you can’t possibly help me, and
you can’t possibly do any good by remaining here. So if you’ll permit
Mr. Dorgan and me to talk this quietly over, by ourselves——”
“You are requesting me to leave you?” her would-be benefactor
inquired, as he reached for his hat.
“You must,” announced Teddie.
“Then permit me, Miss Hayden,” said the other with dignity, “to
bid not only you, but also your—your professional boxer, a good
afternoon.”
And the old Commodore buttoned his coat and took his
departure. He sallied forth with considerable trepidation, trepidation
which remained with him even until he stopped in at a telegraph-
office on lower Fifth Avenue and despatched a none too carefully
worded message to the old Major in Hot Springs, announcing that
things looked very dark indeed, as Theodora seemed to be mixed up
with a young prize-fighter by the name of Dorgan, and suggesting
that the sooner Theodora’s uncle could get back to the city the
better it might be for all concerned.
 CHAPTER ELEVEN
Teddie, alone with her irate young prize-fighter, turned and
regarded him with a studiously narrowed eye.
“Now, what do you want to know?” she quietly demanded. She
felt oddly and immeasurably older than she had done but one short
week ago.
“I want ’o know who’s playin’ double in this mix-up,” Gunboat
Dorgan promptly asserted.
“I don’t quite understand,” protested Teddie.
“Well, first thing, I want ’o know just what yuh said about that
car?”
“When?” temporized Teddie. “And where?”
“Just b’fore I kissed yuh, right here in this room,” asserted the
over-honest youth. Whereupon Teddie stiffened and winced and had
to take a grip on herself before she could control her voice.
“I’m sorry there’s been any mistake about it,” explained Teddie,
doing her best to be patient. “I remember now, I said you could
have the car. And, as a matter of fact, you are perfectly welcome to
it, or what’s left of it!”
“Then why’s this man West talkin’ so big about grand larceny and
gettin’ me locked up? What’s he know about what’s been passin’
strictly b’tween yuh and me? Yuh were up ag’inst it, and I could see
it, and I helped yuh out the same as I’d help any girl. And I didn’t
have me hand out when I did it!”
“That was the trouble, Mr. Dorgan,” Teddie tried to tell him. “I
was willing to accept service from you without stopping to consider
whether or not it could be repaid, I mean adequately repaid. And
that’s where I made my mistake. You’ll have to attribute that
mistake, I’m afraid, to the defects in my bringing up. It’s a sort of
penalty for the past. One gets into the habit of accepting things, just
as one accepts cinnamon-toast from the footman, or a trip across
the Hudson from the ferry-boat, without being actively conscious of
any human obligation. That man had made himself unbearably
offensive to me, and I asked you to punch his nose for me, without
remembering the risks it involved, without appreciating the danger I
was bringing——”
“Risks!” cried Gunboat, with a derisive hoot, finally arriving at a
definite idea in what seemed a morass of abstractions. “Where’s the
risks in standin’ up to a big stiff like that?”
“I’m afraid I wasn’t thinking of the risks to you,” Teddie rather
wearily explained. “I was rather selfishly remembering the risks to
myself.”
“Well, yuh ain’t suffered none from it, have yuh?” derided her still
indignant-eyed cross-examiner.
“I’ve just paid Raoul Uhlan twenty-five thousand dollars as
compensation for his injuries,” explained Teddie, as coolly as she was
able.
Gunboat Dorgan fell back, gaped a little, and then swallowed
hard.
“Yuh paid—yuh paid that mutt—that money—for—for what he’d
get tarred and feathered for—down in my Ward!” he gasped, wide-
eyed with incredulity.
Teddie nodded.
And Gunboat, seeing that movement of acquiescence, repeated:
“Twenty-five thousand dollars!” Then he began to stride meditatively
back and forth, pacing the studio-rug with his characteristic panther-
like step. Teddie watched him, without speaking, without moving.
She watched him until he came to an abrupt stop.
“Say, Ruby was right in this, after all,” he suddenly proclaimed. “I
was the guy who got off his trolley. Yuh—yuh looked so good to me
I got my numbers mixed. I got to dreamin’ things. But twenty-five
thousand bucks in cold cash ain’t no dream. And d’ yuh know what
I’m goin’ to do, and do right now? I’m goin’ up to that Uhlan guy
and get that twenty-five thousand back. Just so ’s yuh can see I’m a
little more on the level than yuh’ve been imaginin’. I’m goin’ to make
that studio-lizard come across wit’ that dough—with that dough,” he
amended, remembering, in his excitement, certain old-time
admonitions as to the utterance of his mother-tongue.
“But I don’t want you to do that!” cried Teddie, harboring a
strangely muddled-up and reluctant admiration for the deluded
young fire-eater with the Saint Anthony light in his blazing blue eyes.
“Of course yuh don’t, for the thing’s got yuh buffaloed the same
as yuh got me buffaloed,” proclaimed the knight of the ring. “And
the whole lay-out’s wrong. The only thing that got hurt about that
guy was his dignity. I knew what I was doin’ all the time. I held back
on the sleep-punch, and played wit’ him. I didn’t give him anything
that a pound of beefsteak wouldn’t put right inside o’ twenty-four
hours—and he knows it as well as I do. But now he’s pulled this
blackmail stuff I’m goin’ to put him wise to how I was toyin’ wit’ him.
I’m goin’ to let him see that if he ever so much as opens his trap
about this business he’s goin’ to have it decorated wit’ a double set
o’ plates when I get through wit’ him—when I get through with him.
And the next time he’ll holler so loud for help they’ll be fannin’ him
wit’ a hearse-plume before he’s finished!”
Teddie tried to stop him as he turned away.
“Noth-thing doin’!” he proclaimed with his movie-hero side-
movement of the hand. “I’m Irish, I am, and me Irish is up. Yuh’re
goin’ to see this goob bitin’ on a mouth-gag or yuh’re goin’ to see
crape swingin’ over his door-mat!”
“It’s no use,” Teddie still tried to tell him. “It’s too late. It will only
make things much worse than they already are!”
But Gunboat Dorgan hadn’t been crowned with that soubriquet
of belligerency without fit and proper reason.
“I’m wise to this lay-out now,” he announced from the doorway,
“and I’m goin’ to have a hand in windin’ it up. It’s no use tryin’ to
flag me off. And I ain’t sayin’ yuh’re a quitter, for yuh’re only a girl.
But yuh don’t see me layin’ down in the shafts wit’ a thing like this
under me nose. I’m goin’ through wit’ this, and nobody’s goin’ to
stop me. And maybe this’ll square up a little for—for them lamps o’
yours I put on the blink!”
 CHAPTER TWELVE
Teddie, once she was alone in her studio, experienced a sense of
confinement, a feeling of compression, which had hitherto been
absent from her newer mode of life. She felt the need for
untrammelled movement through fresh air, the craving to get out
into open spaces and leave the suffocation of city walls behind her.
She promptly decided, in fact, to drive her car out to Tuxedo, and
even went to the telephone to order it from the garage. Then she
remembered that she no longer had a car.
But this, in the face of the denudations with which life had been
confronting her, did not impress her as a very vast deprivation. She
merely called for another number and ordered a taxi, contenting
herself with the thought of three gasoline-flavored hours in that rus
in urbe known as Central Park.
But Teddie did not go gloom-riding in Central Park. For when she
opened the door to what she thought to be a taxi-driver she found
Gerry West there with his hat in his hand and a look of triumph in
his eyes.
“Well, I’ve got it back,” he announced, only momentarily abashed
by the iciness of her manner.
“Got what back?” asked Teddie, without so much as asking him
to step inside.
“Your car,” explained Gerry, entering the abode of art on his own
hook. “It’s down at the door. And I had ’em put on a new pair of
lamps on the way over.”
“I’m sure that was very kind of you,” Teddie coldly admitted. But
her attitude was something more than unbending. It was distinctly
hostile. For there were certain things which she wasn’t quite able to
forget.
“Say, Teddie,” demanded her quick-eyed visitor, entirely ignoring
her expression in his comprehensive stare about the studio, “what in
the name of heaven are you doing in a dump like this?”
“It seems to have proved an entirely satisfactory place to me,”
Teddie responded with the utmost dignity.
“But has it?” demanded Gerry, putting down his hat.
“It would, if I were left alone,” said Teddie, biting her lips.
“And what would that mean? What would that bring you?” asked
Gerry, with a suddenly sobered face.
“It would bring me the freedom I want,” retorted Teddie, with a
challenge still in her gaze.
“That is the one thing it could never do, O Helen of the Ruinous
Face!” corrected Gerry. But Teddie, who was in no sense a classical
student, saw nothing remarkably appropriate in this allusion to the
ancients.
“What makes you think that?” she asked, with a tremor in her
voice. She hadn’t intended any retreat from her earlier severity of
tone. She prided herself on not being the sort of girl who would
willingly show the white feather. But Gerry had touched on
something which had been bewildering her, of late, more than she
was ready to acknowledge.
“The things that have been happening around here,” he had the
brutality to retort, “the things I’m now trying to straighten out for
you!”
And remembering those things, the sense of her desolation
returned to her double-fold. She walked to the window, looked out,
and then turned slowly about. She was neither obtuse nor
unsympathetic; she was merely a girl who had been prodigiously
preoccupied with her fight for freedom and the depressing discovery
that it was a losing fight.
“Oh, Gerry, what’s the matter with me, anyway?” she demanded
with an altogether unlooked-for note of wistfulness in her voice.
“Don’t you know?” he said as he followed her to the window.
“Don’t you know, you poor little muddle-headed kid?”
 Teddie shook her head. She was rather foolishly afraid that Gerry
was going to be sympathetic, and she didn’t want that, for
sympathy, of late, seemed the inevitable overture to the unmusical
opera of mushiness.
“I’ll tell you what’s the matter with you, Teddie,” asserted Gerry,
wondering why she was refusing to meet his gaze. “You’re
inflammatory without quite knowing it. You’re provocative, without
being foolish enough to have fathomed the fact. The Lord made you
so lovely, girl, that you put an ache in men’s hearts and a mist in
front of their eyes. You make them forget themselves. And that’s
why I’ve got to take you in hand.”
“Take me in hand?” repeated Teddie, standing up very straight
and white.
“Yes, take you in hand,” repeated Gerry in turn.
“I rather think I’ve something to say about that!”
“Teddie, I’ve loved you all my life,” said Gerry, quite simply,
disregarding even the abysmal scorn in her voice.
“Then this is no time to tell me a thing like that,” she retorted
with spirit.
“And you’re wrong there,” contended Gerry, quite unmoved. “It’s
the only, the essential time.”
“What makes you feel that way about it?” asked Teddie,
disturbed by the darkening light in his eye.
“Because heaven only knows how long we can be alone here,”
was his not altogether satisfactory reply.
“I fail to see any particular advantage arising out of our—our
temporary isolation,” retorted Teddie, with quite unexpected
Johnsonian dignity.
“Teddie!” was Gerry’s sharp cry as he towered over her. “Don’t
you understand?”
“Understand what?” asked the girl with the exasperatingly level
gaze as she surveyed the none too steady hands which he was
holding out toward her.
 “That I can’t help kissing you!” he abandonedly exclaimed as he
just as abandonedly proceeded to do so.
Teddie drew slowly away from him. He had seen children draw
back, that way, from a milk-snake coiled up in a chocolate-box. Her
eyes were blazing.
“Now I know you’re no better than——”
But that was as far as Teddie got. For the door was flung open
and a protesting and much dishevelled Louis Lipsett was piloted into
the room. He was piloted in without ceremony, and by the lapel of
his overcoat. The hand that grasped that collar was Gunboat
Dorgan’s, and the lines of his wide mouth were grim with
determination.
“Call off this wildcat,” gasped Louis as he dropped weakly into a
chair. “Call him or I’ll get a shooting-iron and kill him!”
Gerry tried to remove the steel-corded hand from the uptwisted
coat-collar, but Gunboat Dorgan betrayed no slightest intention of
relaxing his hold.
“Not on your life,” he irately announced. “Not until he eats every
word of it!”
“Of what?” demanded Gerry, with an abstracted and mildly
perplexed inspection of Louis Lipsett’s person.
“Don’t listen to him,” cried the prisoner. “He’s gone crazy. He’s
gummed up the whole game. He came tearing into Uhlan’s studio
when I had the big bounder scared stiff, had him eating out of my
hand and willing to sign any kind of quitclaim I was ready to hand
out. He blew in there ready to eat Uhlan up, until he found out I was
from The Star and heard that tricky brush-tickler swear his lips were
sealed and then step from under by saying it was me and my paper
that were going to open up on a full-page story. Me, mind you, with
all I’d done! Then this East Side rat-terrier let loose, and wouldn’t
even give me a chance to get to a phone and have you put things
straight or call up our sporting-editor to shoot a little reason into his
empty nut. He’s hauled me around like a civet-bag and dragged me
down here across eleven city blocks to say what you very well know
I don’t even need to say. And I call this a fine line of reporting, this
ghost-laying for a bunch of love-sick idiots who’re so afraid of
printer’s ink they’re playing tennis with bank-checks over it. For I’m
not the only thing he’s collared, I want y’ to understand. He collared
old Shotwell as well and shook that twenty-five-thousand-dollar draft
out of him and has got it right here in his jeans while he’s joy-riding
on the back of my neck! But I’m tired of being bullyragged and
manhandled and having my clothes spoilt, and if this rising star of
suburban ring doesn’t get his fingers out of my back hair inside of
ten seconds I’m going to let loose with something more than ink
before the day is over.”
“Let him go!” commanded Gerry, in his most authoritative grand-
jury voice. “This man is acting for Miss Hayden, is very generously
and unselfishly acting for Miss Hayden.”
“Am I now?” gritted Louis Lipsett, breathing hard and writhing his
disordered clothing back into place.
“Well, so am I,” averred Gunboat Dorgan as he tossed Teddie’s
much crumpled check out on the cherrywood table. “And I want ’o
know,” he continued as he confronted Gerry West, “just what call
yuh’ve got for buttin’ in on this?”
“I am acting for Miss Hayden,” Gerry announced with gravity.
“We’re all acting for Miss Hayden!” mocked Louis, with a foolish
upward movement of his hands. But Gunboat ignored that derisive
interruption.
“In what way ’re yuh actin’ for her?” he demanded, with his
shoulders squared and his chin out.
“As her husband,” said Gerry with a grimness which was quite
new to him.
Gunboat swung slowly about and stared at the girl on the other
side of the cherrywood table. He saw a slow flush creep up into the
shell-pink of her cheeks.
“Are yuh married to this mucker?” he demanded, with a thumb-
jerk toward the unobserving Gerald Rhindelander West. And he
swallowed hard as he put the question, just as Teddie used to
swallow hard when she beheld the castor-oil bottle being lifted down
out of the medicine cabinet.
“I am not!” was Teddie’s quiet but distinct-noted reply.
“Are yuh goin’ to be?” queried the narrow-eyed Gunboat.
“I am not going to be,” replied Teddie, with an opaque eye on a
slightly crestfallen young attorney.
Gunboat shook his cauliflower ear in a little nod of
comprehension. Then he turned back to the girl.
“All right, then. I’m takin’ care of yuh. Remember that. We’ll cut
out the hot-air artists and get busy. And that brings us round to this
newspaper boob. He’s got the whole story of what’s been happenin’
here, and he’s talkin’ big about puttin’ it into print. I heard him wit’
my own——”
“He won’t put it into print,” interrupted Gerry.
“What’ll stop him?” demanded the man of battle.
“The knowledge of what we’d both do to him if he tried it,”
announced the expounder of law, doing his best to overlook
Gunboat’s oblique glance of contempt. “And the further knowledge
that he never even intended to put it into print.”
“No, putting things into print doesn’t seem to be my business any
more,” interpolated the morose-eyed Lipsett.
“Then why——” began Gunboat, but he was interrupted by the
trill of the telephone bell. It was Teddie who finally crossed to the
instrument and answered the call.
“It’s for you, Mr. Dorgan,” she said, without emotion, as she
waited with the receiver in her hand.
“Oh, is that yuh, Ruby,” Gunboat was murmuring a moment later,
into the transmitter. He spoke in a strangely altered tone, a tone
with even a touch of meekness in it. “All right, Ruby,” he docilely
agreed after several minutes of an obviously one-sided conversation.
“Sure, Ruby, yuh’re dead right.” Then came the receiver’s turn again,
with an amending “Whatever yuh say, Ruby,” gently intoned into the
transmitter.
 If Teddie garnered any inkling of that capitulating meekness on
Gunboat Dorgan’s part, she gave out no echo of it in her own icy
stare of disapproval as she stood regarding Gerald Rhindelander
West. Even the rueful Louis Lipsett awakened to that oddly sustained
duel of glances between the two silent figures on the far side of the
room. He awakened, in fact, to the all-pervading, three-cornered
preoccupation which surrounded him. And he made hay while the
sun shone. He took advantage of that momentary inattention and
slipped from his chair. He tiptoed discreetly out of the room and
hurried away into the comparative quietness of Fourth Street, where
he caught a Broadway surface-car and headed for the peace of Park
Row.
Gunboat Dorgan, as he meditatively hung up the receiver, did not
even miss the vanished newspaper man. He was too busy watching
the strange couple still confronting each other on the far side of the
studio. The girl, with ice-cold deliberation, pinned a tiptilted turban
on her head, adjusted a four-piece blue-fox throw about her
shoulders, and drew on a pair of wrinkled-topped gloves.
“Where are you going?” demanded the dark-browed expounder
of the law, in a tone savoring unmistakably of the cave-man age.
“I regard that as entirely my own affair,” retorted the girl in blue-
fox, just as unmistakably reverting to the age of ice.
“Where are you going?” repeated the neolithic young giant in
tweeds.
“Will you kindly permit me to open my own door?” said Teddie,
with her chin up.
“Where are you going?” demanded Gerry, for the third and last
time.
For one long moment of silence Teddie inspected him as though
he were something under plate-glass, something behind Zoo bars.
“I’m going home!” she finally told him.
“Why?” exacted her altogether too obtuse tormentor.
 “Because I’m tired of all this!” was the intense but low-noted
reply.
“Of all what?” demanded the bewildered Gerry.
“Of seeing everything that’s most sacred in life mauled over in
public—of being mauled over myself as though I was something
marked down on a bargain-counter—of finding out that all men are
so—so disgusting and degradingly alike!”
It seemed to take time for Gerry to digest this. And even with
time the process appeared to be unattended by any great degree of
satisfaction.
“Haven’t I been doing what I could for you?” he demanded, with
the air of a man who asked only for reason.
“Are you worrying about your fee?” countered the pale-cheeked
Teddie.
“I don’t want a fee,” said Gerry.
“Then what is it you want?”
Gerry tried to square his shoulders.
“I want you!”
She met his eye, but it took an effort. And Gerry, for the life of
him, couldn’t help thinking once more of the milk-snake in the
chocolate-box.
“How about my wishes in the matter?” she asked with a slow and
pointed emphasis which brought a wince to even Gunboat Dorgan’s
Celtic eyes.
“Just a minute, yuh folks,” suggested the perturbed man of the
ring. “This actin’ as though yuh was married for ten years ain’t goin’
to bury any tomahawks and end the war-dance! There’s been too
much pullin’ at cross——”
But it was Gunboat’s turn to be interrupted. That final
interruption came in the form of the unceremonious flinging back of
the studio-door, disclosing the bristling but the immaculate figure of
Teddie’s Uncle Chandler.
 “What’s wrong here, Teddie?” demanded that perplexed-eyed old
gentleman, striding into the room with all the dignity his sciatica
would permit.
“She wants to go home,” said Gerry. And as he said those five
words in a singularly dull tone his hands went down to his sides. The
movement, in some way, was oddly suggestive of flying colors
forlornly lowered.
“Well, that impresses me as an eminently sane and respectable
place to want to go to,” remarked the old Major as he blinked from
one to the other of the odd trio confronting him. But his eye, for
some reason, was on Gerald Rhindelander West when he spoke
next, though his question, obviously, was addressed to Teddie. “And
just when do you want to go, my dear?”
“As soon as you can get me away from here,” was Teddie’s
prompt but low-noted reply.
Ceremoniously the old Major held out his crooked right arm and
dolorously the girl in the blue-fox took it. Neither of them spoke until
they came to a stop beside the wine-colored shopping-car.
“I never intend to speak to Gerry West again as long as I live,”
announced Teddie, with a combined suddenness and fierceness
which made her Uncle Chandler forget his left hip-joint as he
climbed into the car beside her.
He patted her knee, comprehendingly.
“Under the circumstances, then,” he observed as she made the
motor whine with an altogether unnecessary jab on the accelerator,
“it’ll be just as well, Teddikins, if you don’t see him for a week or
two!” . . .
Back in the dismal emptiness of the dismal gray studio Gerry and
Gunboat stood looking at each other. Then Gunboat sighed
fraternally and essayed an owl-like wag of the head.
“They’re all alike, them women!” he remarked with the sagacity
of one who has survived many unfair ordeals at the hands of the fair.
 CHAPTER THIRTEEN
Teddie’s head was much clearer by the time she had motored out
to Tuxedo. Her head was clearer, but the contradictory tides of
feeling that eddied about her troubled young heart seemed as
muddled-up as ever. Even her Uncle Chandler was not entirely
oblivious of the fact that some newer ferment was working in the
depths of that bottled-up young soul. But he asked no questions.
There were two things which he knew too well for that: one was life
in general, and the other was Theodora Lydia Lorillard Hayden in
particular.
As for Teddie herself, she was tyrannical and melting and snappy
and chummy all at the same time. She promptly ordered the
servants back at their posts, and just as promptly proceeded to bully
them in a manner which plainly betokened that she intended to be
master of her fate in at least one quarter of an otherwise
unconquered world. She ordered silver unpacked and moth-bags
banished and the striped ticking off the furniture and the cars
overhauled and the drapes restored and the drive-borders retrimmed
and an absurd amount of cut flowers for every room in the house.
But she prowled moodily about that house, resenting its quietness at
the same time that she gave orders she was at home to nobody. She
tried riding before breakfast, and found her old mount gone soft and
her new groom grown sulky. She tried reading, and discovered how
unbelievably dull all modern books could be. She tried motoring, and
found no interest in maneuvering the old hair-pin curves on two
wheels and no thrill in defying the old speed-traps at sixty miles an
hour. Even the greenhouses, when she invaded them, seemed to
suggest funeral set-pieces and the vanity of all earthly ways. The
very walls about that lordly Hayden demense grew still again
remarkably suggestive of jail walls. And that particular wall which
intervened between her own and the adjacent West estate seemed
to take on a particularly objectionable coloring.
 As for her Uncle Chandler, he punctiliously dressed for dinner,
and punctiliously sat at one end of the big dining-room table while
Teddie just as solemnly sat at the other—though she did once
emerge sufficiently from her self-absorption to remark that they
looked exactly like two palm-trees on the edge of the Sahara. She
also once ventured to ask if Watkins really oughtn’t to have a
passport when he carried the joint all the way from her end of the
table down to the old Major’s end of the table. And her Uncle
Chandler brightened up sufficiently to inquire if he hadn’t better
order a taxi to run them out to the terrace for coffee, so abysmally
vast seemed the distances in that dolorous and empty house.
If the old Major remained suspiciously meek and long-suffering
during these days of trial, it must be acknowledged that he made
divers and undivulged trips in to the City, whence he returned oddly
fortified in spirit and beguilingly abstracted in manner. The only
excursion which brought him obvious displeasure was that when he
brought back to Teddie a motor-truck loaded down with her studio
possessions—which the lady in question solemnly committed to a
bonfire on the rear end of the East Drive. And that afternoon as they
sat taking tea and cinnamon-toast on the Terrace, he finally found
the courage to confront the morose-eyed young lady who sat in the
high-backed willow-chair so moodily tearing an Ophelia rose to
pieces.
“Say, Teddie, isn’t it about time you were loosening up?” the old
Major quietly inquired.
“About what?” demanded Teddie, taking her third slice of
cinnamon-toast.
“About that mix-up down in the Village.”
“It wasn’t a mix-up,” corrected Teddie.
“Then what was it?”
“It was a revelation!”
“A revelation of what?” asked Uncle Chandler as he put his
teacup down.
“Of what men are!” asserted the abstracted-eyed Teddie.
 “Of course,” said the old Major as he took out a chased gold case
and meditatively extracted a cigarette. “So let’s have it, Teddikins,
hook, line and sinker!”
But Teddie shook her head.
“I telegraphed to father,” she inappositely remarked.
“Where is Trummie this summer?” her uncle inquired.
“He’s still at the Arizona Camp Observatory,” explained Teddie.
“Trummie moves so slowly,” complained the old Major. “The poor
man can’t help it, I suppose, trailing that chain of D. S.’s, and F. R.
S.’s and F. R. G. S.’s around after him all the time. But I suppose you
felt he was the proper person to talk such things over with?”
Teddie nodded a slightly abstracted assent.
“Yes, I felt that way. But I had a wire from father this morning.
He says he’ll be through with his spectographic analysis of the Milky
Way nebulæ before the end of October and that as soon as he feels
sure he can synthesize an isotope of hydrogen approximating to
nebulum he’ll come east and have a talk with me!”
The old Major smiled pensively.
“Yes, I remember what he said when the Rubber Trust swallowed
up my little Congolo Company and squeezed me out after I’d
squeezed out the original Amsterdammers: ‘The oysters eat the
diatoms, and we eat the oysters!’ It makes me wish, Teddie, that I
could be a philosopher now and then.”
“I wish women could be,” remarked Teddie.
“Then why not make a stab at it,” ventured the old gentleman
who had been so intently studying her averted face, “by telling me
what the trouble is?”
“There’s really nothing to tell, Uncle Chandler,” solemnly asserted
the young lady with the moody eyes, drawing the striped ticking of
reticence over the brocaded injustices of youth.
The old Major tossed away his cigarette. He sat staring at the
poor little rich girl in the willow lawn-chair. He stared at her so long
and so intently that she finally turned about and looked none too
fraternally into his face.
“What are you thinking about?” she asked.
“It’s queer I never noticed it before,” remarked the old Major,
apparently more to himself than to the girl confronting him.
“Noticed what?” asked Teddie.
“How you’re getting a bit like your mater,” replied the placid-eyed
old gentleman in the armchair, “a bit tamed and trimmed off and
ironed out!”
“I won’t be!” proclaimed Teddie, with quite unlooked-for passion,
as she got up from her chair.
“But how, my dear, are you going to stop it?” asked the still
equable old Major.
“I won’t get like that!” reiterated Teddie, looking for all the world
like a second Artemisia confronting an army of embattled males. She
stood there, as though expecting some retort from him. But he said
nothing. He merely took out another cigarette, lighted it, and
recovered his morning Herald from the grass at his feet. This he
proceeded to peruse with studied unconcern, quite ignoring the
young Artemisia still glowering at him over the edge of it. Then he
looked up, with the ghost of a yawn.
“By the way, I saw the Commodore in town yesterday,” observed
Teddie’s uncle as he leisurely turned a page. “He was telling me a
queer thing about young West.”
“Indeed!” said Teddie, without moving.
“The Commodore was saying that Gerry’s going to marry that
Rivers girl,” offhandedly announced the maculated old scoundrel in
immaculate cricketer’s flannel.
He waited behind his paper, for several seconds. Then he heard a
mirthless little laugh. Then he heard the contemptuous ejaculation of
“That frump!” And then he heard quick steps along the marble walk
that bisected the Terrace.
Welcome to our website – the perfect destination for book lovers and
knowledge seekers. We believe that every book holds a new world,
offering opportunities for learning, discovery, and personal growth.
That’s why we are dedicated to bringing you a diverse collection of
books, ranging from classic literature and specialized publications to
self-development guides and children's books.

More than just a book-buying platform, we strive to be a bridge

connecting you with timeless cultural and intellectual values. With an
elegant, user-friendly interface and a smart search system, you can
quickly find the books that best suit your interests. Additionally,
our special promotions and home delivery services help you save time
and fully enjoy the joy of reading.

Join us on a journey of knowledge exploration, passion nurturing, and

personal growth every day!

ebookbell.com