Instant Access to Earth Observation Using Python A Practical Programming Guide 1st Edition Rebekah B. Esmaili ebook Full Chapters
Instant Access to Earth Observation Using Python A Practical Programming Guide 1st Edition Rebekah B. Esmaili ebook Full Chapters
com
https://ebookmeta.com/product/earth-observation-using-
python-a-practical-programming-guide-1st-edition-rebekah-b-
esmaili/
OR CLICK BUTTON
DOWNLOAD NOW
https://ebookmeta.com/product/algebraic-graph-algorithms-a-practical-
guide-using-python-undergraduate-topics-in-computer-science-k-erciyes/
ebookmeta.com
https://ebookmeta.com/product/real-world-python-a-hacker-s-guide-to-
solving-problems-with-code-1st-edition-lee-vaughan/
ebookmeta.com
The Complete Home Networking Manual 3rd Edition 2022
Papercut
https://ebookmeta.com/product/the-complete-home-networking-manual-3rd-
edition-2022-papercut/
ebookmeta.com
https://ebookmeta.com/product/embodied-differences-the-jews-body-and-
materiality-in-russian-literature-and-culture-henrietta-mondry/
ebookmeta.com
https://ebookmeta.com/product/unleash-core-data-fetching-data-
migrating-and-maintaining-persistent-stores-1st-edition-avi-tsadok/
ebookmeta.com
https://ebookmeta.com/product/the-real-valkyrie-the-hidden-history-of-
viking-warrior-women-nancy-marie-brown/
ebookmeta.com
https://ebookmeta.com/product/love-and-war-1-7-1st-edition-k-webster/
ebookmeta.com
Know How 1st Edition Jason Stanley
https://ebookmeta.com/product/know-how-1st-edition-jason-stanley/
ebookmeta.com
Earth Observation Using Python
EARTH OBSERVATION
USING PYTHON
A Practical Programming Guide
Rebekah B. Esmaili
All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted,
in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as
permitted by law. Advice on how to obtain permission to reuse material from this title is available at
http://www.wiley.com/go/permissions.
The right of Rebekah B. Esmaili to be identified as the author of this work has been asserted in accordance with law.
Registered Office
John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, USA
Editorial Office
111 River Street, Hoboken, NJ 07030, USA
For details of our global editorial offices, customer services, and more information about Wiley products visit us at
www.wiley.com.
Wiley also publishes its books in a variety of electronic formats and by print-on-demand. Some content that appears
in standard print versions of this book may not be available in other formats.
While the publisher and authors have used their best efforts in preparing this work, they make no representations or
warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all
warranties, including without limitation any implied warranties of merchantability or fitness for a particular purpose.
No warranty may be created or extended by sales representatives, written sales materials or promotional statements
for this work. The fact that an organization, website, or product is referred to in this work as a citation and/or
potential source of further information does not mean that the publisher and authors endorse the information or
services the organization, website, or product may provide or recommendations it may make. This work is sold with
the understanding that the publisher is not engaged in rendering professional services. The advice and strategies
contained herein may not be suitable for your situation. You should consult with a specialist where appropriate.
Further, readers should be aware that websites listed in this work may have changed or disappeared between when this
work was written and when it is read. Neither the publisher nor authors shall be liable for any loss of profit or any
other commercial damages, including but not limited to special, incidental, consequential, or other damages.
10 9 8 7 6 5 4 3 2 1
Introduction ...................................................................................................... 1
Part I: Overview of Satellite Datasets ............................................................... 5
1 A Tour of Current Satellite Missions and Products .................................. 7
2 Overview of Python................................................................................ 17
3 A Deep Dive into Scientific Data Sets.................................................... 25
Conclusion..................................................................................................... 253
Index.............................................................................................................. 283
When I first met the author a few years ago, she was eager to become more
involved in the Joint Polar Satellite System’s Proving Ground. The Proving
Ground by definition assesses the impact of a product in the user’s environment;
this intrigued Rebekah because as a product developer, she wanted to understand
the user’s perspective. Rebekah worked with the National Weather Service to
demonstrate how satellite-derived atmospheric temperature and water vapor
soundings can be used to describe the atmosphere’s instability to support severe
weather warnings. Rebekah spent considerable time with users at the Storm Pre-
diction Center in Norman, Oklahoma, to understand their needs, and she found
their thirst for data and the need for data to be easily visualized and understand-
able. This is where Rebekah leveraged her expert skills in Python to provide NWS
with the information they found to be most useful. Little did I know at the time she
was writing a book.
As noted in this book, a myriad of Earth-observing satellites collect critical
information of the Earth’s complex and ever-changing environment and land-
scape. However, today, unfortunately, all that information is not effectively being
used for various reasons: issues with data access, different data formats, and the
need for better tools for data fusion and visualization. If we were able to solve these
problems, then suddenly there would be vast improvements in providing societies
with the information needed to support decisions related to weather and climate
and their impacts, including high-impact weather events, droughts, flooding, wild-
fires, ocean/coastal ecosystems, air quality, and more. Python is becoming the uni-
versal language to bridge these various data sources and translate them into useful
information. Open and free attributes, and the data and code sharing mindset of
the Python communities, make Python very appealing.
Being involved in a number of international collaborations to improve the
integration of Earth observations, I can certainly emphasize the importance
of working together, data sharing, and demonstrating the value of data
fusion. I am very honored to write this Foreword, since this book focuses on these
vii
issues and provides an excellent guide with relevant examples for the reader to
follow and relate to.
This book evolved from a series of Python workshops that I developed with
the help of Eviatar Bach and Kriti Bhargava from the Department of Atmospheric
and Oceanic Science at the University of Maryland. I am very grateful for their
assistance providing feedback for the examples in this book and for leading several
of these workshops with me.
This book would not exist without their support and contributions from
others, including:
The many reviewers who took the time to read versions of this book, several of
whom I have never met in person. Thanks to modern communication systems,
I was able to draw from their expertise. Their constructive feedback and insights
not only helped to improve this quality and breadth of the book but also helped me
hone my technical writing skills.
Rituparna Bose, Jenny Lunn, Layla Harden, and the rest of the team at AGU
and Wiley for keeping me informed, organized, and on track throughout this
process. They were truly a pleasure to work with.
Nadia Smith and Chris Barnet, and my other colleagues at Science and Tech-
nology Corp., who provided both feedback and conversations that helped shape
some of the ideas and content in this book.
Catherine Thomas, Clare Flynn, Erin Lynch, and Amy Ho for their endless
encouragement and support.
Tracie and Farid Esmaili, my parents, who encouraged me to aim high even if
they were initially confused when their atmospheric scientist daughter became
interested in “snakes.”
ix
Earth Observation Using Python: A Practical Programming Guide, Special Publications 75,
First Edition. Rebekah B. Esmaili.
© 2021 American Geophysical Union. Published 2021 by John Wiley & Sons, Inc.
DOI: 10.1002/9781119606925.introduction
Fortran and C/C++. A drawback of printed media is that it tends to be static and
Python is evolving more rapidly than the typical production schedule of a book.
To mitigate this, this book intends to teach fluency in a few, well-established
packages by detailing the steps and thought processes needed for a user needs
to carry out more advanced studies. The text focuses discipline-agnostic packages
that are widely used, such as NumPy, Pandas, and xarray, as well as plotting
packages such as Matplotlib and Cartopy.
I have chosen to highlight Python primarily because it is a general-purpose
language, rather than being discipline or task-specific. Python programmers
can script, process, analyze, and visualize data. Python’s popularity does not
diminish the usefulness and value of other languages and techniques. As with
all interpreted programming languages, Python may run more slowly compared
to compiled languages like Fortran and C++, the traditional tools of the trade.
For instance, some steps in data analysis could be done more succinctly and with
greater computational efficiency in other languages. Also, underlying packages in
Python often rely on compiled languages, so an advanced Python programmer can
develop very computationally efficient programs with popular packages that are
built with speed-optimized algorithms. While not explicitly covered in this book,
emerging packages such as Dask can be helpful to process data in parallel, so more
advanced scientific programmers can learn to optimize the speed performance of
their code. Python interfaces with a variety of languages, so advanced scientific
programmers can compile computationally expensive processing components
and run them using Python. Then, simpler parts of the code can be written in
Python, which is easier to use and debug.
This book encourages readers to share their final code online with the broader
community, a practice more common among software developers than scientists.
However, it is also good practice to write code and software in a thoughtful and
carefully documented manner so that it is usable for others. For instance, well-
written code is general purpose, lacks redundancy, and is intuitively organized
so that it may be revised or updated if necessary. Many scientific programmers
are self-learners with a background in procedural programming, and thus their
Python code will tend to resemble the flow of a Fortran or IDL program. This
text uses Jupyter Notebook, which is designed to promote good programming
habits in establishing a “digestible code” mindset; this approach organizes code
into short chunks. This book focuses on clear documentation in science algorithms
and code. This is handled through version control, using virtual environments,
how to structure a usable README file, and what to include in inline
commenting.
For most environmental science endeavors, data and code sharing are part of
the research-to-operations feedback loop. “Operations” refers to continuous data
collection for scientific research and hazard monitoring. By sharing these tools
with other researchers, datasets are more fully and effectively utilized. Satellite
data providers can upgrade existing datasets if there is a demand. Globally,
satellite data are provided through data portals by NASA, NOAA, EUMETSAT,
ESA, JAXA, and other international agencies. However, the value of these data-
sets is often only visible through scientific journal articles, which only represent a
small subset of potential users. For instance, if the applications of satellite obser-
vations used for routine disaster mitigation and planning in a disadvantaged
nation are not published in a scientific journal, improvements for disaster-
mitigation specific needs may never be met.
Further, there may be unexpected or novel uses of datasets that can drive sci-
entific inquiry, but if the code that brings those uses to life is hastily written and not
easily understood, it is effectively a waste of time for colleagues to attempt to
employ such applications. By sharing clearly written code and corresponding doc-
umentation for satellite data applications, users can alert colleagues in their com-
munity of the existence of scientific breakthrough efforts and expand the potential
value of satellite datasets within and beyond their community. Moreover, public
knowledge of those efforts can help justify the versatility and value of satellite mis-
sions and provide a return on investment for organizations that fund them. In the
end, the dissemination of code and data analysis tools will only benefit the scien-
tific community as a whole.
Overview of Satellite
Datasets
At present, there are over 13,000 satellite-based Earth observations freely and
openly listed on www.data.gov. Not only is the quantity of available data notable,
its quality is equally impressive; for example, infrared sounders can estimate
brightness temperatures within 0.1 K from surface observations (Tobin et al.,
2013), imagers can detect ocean currents with an accuracy of 1.0 km/hr
(NOAA, 2020), and satellite-based lidar can measure the ice-sheet elevation
change with a 10 cm sensitivity (Garner, 2015). Previously remote parts of our
planet are now observable, including the open oceans and sparsely populated
areas. Furthermore, many datasets are available in near real time with image
latencies ranging from less than an hour down to minutes – the latter being crit-
ically important for natural disaster prediction. Having data rapidly available
enables science applications and weather prediction as well as to emergency man-
agement and disaster relief. Research-grade data take longer to process (hours to
months) but has a higher accuracy and precision, making it suitable for long-term
consistency. Thus, we live in the “golden age” of satellite Earth observation. While
the data are accessible, the tools and skills necessary to display and analyze this
information require practice and training.
Earth Observation Using Python: A Practical Programming Guide, Special Publications 75,
First Edition. Rebekah B. Esmaili.
© 2021 American Geophysical Union. Published 2021 by John Wiley & Sons, Inc.
DOI: 10.1002/9781119606925.ch1
Saarwanderung.
Die Fahrt durch das Saartal von St. Johann nach Trier führt uns
zuerst durch das fabrikreiche Bild der zuletzt genannten Orte. Dann
entfaltet das Tal seine liebliche Anmut. Mit fruchtbaren Äckern und
Wiesen ist es freundlich geschmückt. Waldige Höhen begleiten uns
zu beiden Seiten. Wir grüßen die alte Festung Saarlouis (800 Einw.)
und das Ackerbaustädtchen Merzig (7500 Einw.), das in breiterer
Talmulde, umgeben von einem Kranze von Obstpflanzungen, die
das Tal und die ansteigenden Höhen beschatten, umgeben ist.
Enger wird dann das Tal und höher werden die Berge. Denn
zwischen Merzig und Saarburg hat die Saar den Durchbruch durch
den harten Quarzitrücken des Hunsrück erzwingen müssen. Die
Landstraße muß in Serpentinen die Berge hinauf- und
herunterklettern, und für den Eisenbahnreisenden verschwindet
plötzlich das heitere Grün der Landschaft bei der Einfahrt in einen
dunkeln Tunnel. Wir erreichen das verborgen in waldigem Talgrunde
liegende, durch die große Steingutfabrik von Villeroi & Boch aber
weltbekannte Mettlach, und weiter geht’s nach Saarburg, der Perle
des Saartales. Wir steigen empor zur steilen Felswand der Klause
(Abb. 91), wo der blinde König Johann von Böhmen sein stilles Grab
gefunden hat, und bald grüßen uns die Rebenberge, die sich um das
alte Städtchen Saarburg gruppieren. An die Stelle des Quarzits ist
der Schieferfels getreten. Ernst schauen die Reste der alten
Saarburg (Abb. 90), deren erster Bau schon aus dem zehnten
Jahrhundert stammt, hinüber zu den Rebenhöhen, die im warmen
Sonnenlichte glänzen, und hinab in das wonnige Tal, wo in
Schlangenbiegungen die Saar durch grüne Wiesenauen zieht. Nur
der untere Teil des Städtchens Saarburg spiegelt sich in dem Flusse,
der obere jedoch erhebt sich zur luftigen Bergeshöhe. In der
Unterstadt lenkt plötzlich ein tosendes Geräusch unsere Schritte,
und bald stehen wir vor dem wildstürzenden, schäumenden
Wasserfall des Leukbaches. Auf dem Wege von Saarburg nach Trier
treffen wir, dem Flußlauf folgend, die besten Weinbergslagen der
Saar an, so den Scharzhofberg und den Wiltinger Berg. Eine
bekannte Marke ist auch der „Wawerner Herrenberger“.
Einst hatte die Saar, welche bei Conz in die Mosel mündet, auf
ihrer untersten Strecke einen ganz anderen Lauf. Sie machte
mehrere große Biegungen, die sie später abzuschneiden vermochte.
Zuerst zog sie unterhalb Saarburg auf der rechten Seite eine
Schleife. Dann bog sie noch weiter nach Westen aus, und zuletzt
beschrieb sie einen sehr großen Bogen, über Oberemmel und
Niedermennig, wieder nach Osten. Sie mündete aber ungefähr an
der nämlichen Stelle wie heute.
Es öffnen sich nun die Berge, und der helle
Spiegel der Mosel blitzt vor uns auf. Mosella fließt, Trier.
wo sie zuerst uns grüßt, nachdem sie schon weit
gewandert durchs lothringische Land, in ziemlich breitem Tal.
Ostwärts erbreitet sich dieses noch mehr, und nach kurzer Strecke
hat es sich zu einem lieblichen Talkessel gestaltet, der von sanften,
sonnigen Abhängen umschlossen ist. Das ist die Stelle, wo die
Römer in ihrer südlichen Heimat zu weilen glaubten, wo sie an die
sonnigen Gefilde Italiens erinnert wurden und sich im alten Trier (fast
50000 Einw., Abb. 92), dessen Türme bald im Bilde der anmutigen
Landschaft auftauchen, ein zweites Rom — so darf man fast sagen
— schufen.
Eine Inschrift, die am Roten Hause (Abb. 93), dem früheren
Versammlungshause der Ratsherrn, das sich am Hauptmarkt von
Trier erhebt, angebracht ist, sagt: