Download the full version of the textbook now at textbookfull.

com

Python Deep Learning: Understand how deep

neural networks work and apply them to real-
world tasks 3rd Edition Vasilev

https://textbookfull.com/product/python-deep-
learning-understand-how-deep-neural-networks-work-
and-apply-them-to-real-world-tasks-3rd-edition-
vasilev/

Explore and download more textbook at https://textbookfull.com

 Recommended digital products (PDF, EPUB, MOBI) that
you can download immediately if you are interested.

Learn Keras for Deep Neural Networks: A Fast-Track

Approach to Modern Deep Learning with Python 1st Edition
Jojo John Moolayil
https://textbookfull.com/product/learn-keras-for-deep-neural-networks-
a-fast-track-approach-to-modern-deep-learning-with-python-1st-edition-
jojo-john-moolayil/
textbookfull.com

Neural Networks and Deep Learning A Textbook Charu C.

Aggarwal

https://textbookfull.com/product/neural-networks-and-deep-learning-a-
textbook-charu-c-aggarwal/

textbookfull.com

Applied Neural Networks with TensorFlow 2: API Oriented

Deep Learning with Python Orhan Gazi Yalç■n

https://textbookfull.com/product/applied-neural-networks-with-
tensorflow-2-api-oriented-deep-learning-with-python-orhan-gazi-yalcin/

textbookfull.com

Open Access Databases and Datasets for Drug Discovery

Methods Principles in Medicinal Chemistry 1st Edition M.
T. Przewosny
https://textbookfull.com/product/open-access-databases-and-datasets-
for-drug-discovery-methods-principles-in-medicinal-chemistry-1st-
edition-m-t-przewosny/
textbookfull.com
All Rise for the Honorable Perry T Cook First Edition,
Printing. Edition Leslie Connor

https://textbookfull.com/product/all-rise-for-the-honorable-perry-t-
cook-first-edition-printing-edition-leslie-connor/

textbookfull.com

Sensors and Image Processing Proceedings of CSI 2015 1st

Edition Shabana Urooj

https://textbookfull.com/product/sensors-and-image-processing-
proceedings-of-csi-2015-1st-edition-shabana-urooj/

textbookfull.com

Improving Father Daughter Relationships A Guide for Women

and their Dads 1st Edition Nielsen

https://textbookfull.com/product/improving-father-daughter-
relationships-a-guide-for-women-and-their-dads-1st-edition-nielsen/

textbookfull.com

Arduino-Based Embedded Systems: Interfacing, Simulation,

and LabVIEW GUI 1st Edition Rajesh Singh

https://textbookfull.com/product/arduino-based-embedded-systems-
interfacing-simulation-and-labview-gui-1st-edition-rajesh-singh/

textbookfull.com

CONFESSIONS OF AN ILLUMINATI VOL I THE WHOLE TRUTH ABOUT

THE ILLUMINATI AND THE NEW WORLD ORDER 2nd Edition Leo
Lyon Zagami
https://textbookfull.com/product/confessions-of-an-illuminati-vol-i-
the-whole-truth-about-the-illuminati-and-the-new-world-order-2nd-
edition-leo-lyon-zagami/
textbookfull.com
Hedge fund investing a practical approach to understanding
investor motivation manager profits and fund performance
Second Edition Mirabile
https://textbookfull.com/product/hedge-fund-investing-a-practical-
approach-to-understanding-investor-motivation-manager-profits-and-
fund-performance-second-edition-mirabile/
textbookfull.com
Python Deep Learning

Understand how deep neural networks work and apply them

to real-world tasks

Ivan Vasilev

BIRMINGHAM—MUMBAI
Python Deep Learning
Copyright © 2023 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted
in any form or by any means, without the prior written permission of the publisher, except in the case
of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information
presented. However, the information contained in this book is sold without warranty, either express
or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable
for any damages caused or alleged to have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and
products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot
guarantee the accuracy of this information.

Associate Group Product Manager: Niranjan Naikwadi

Associate Publishing Product Manager: Nitin Nainani
Book Project Manager: Hemangi Lotlikar
Senior Editor: Tiksha Lad
Technical Editor: Rahul Limbachiya
Copy Editor: Safis Editing
Proofreader: Safis Editing
Indexer: Subalakshmi Govindhan
Production Designer: Shankar Kalbhor
DevRel Marketing Coordinator: Vinishka Kalra

First published: April 2017

Second edition: January 2019
Third edition: November 2023

Production reference: 1271023

Published by
Packt Publishing Ltd.
Grosvenor House
11 St Paul’s Square
Birmingham
B3 1RB, UK.

ISBN 978-1-83763-850-5

www.packtpub.com
 Contributors

About the author

Ivan Vasilev started working on the first open source Java deep learning library with GPU support in
2013. The library was acquired by a German company, with whom he continued its development. He
has also worked as a machine learning engineer and researcher in medical image classification and
segmentation with deep neural networks. Since 2017, he has focused on financial machine learning.
He co-founded an algorithmic trading company, where he’s the lead engineer.
He holds an MSc in artificial intelligence from Sofia University St. Kliment Ohridski and has written
two previous books on the same topic.
About the reviewer
Krishnan Raghavan is an IT professional with over 20+ years of experience in the areas of software
development and delivery excellence, across multiple domains and technologies, ranging from C++
to Java, Python, Angular, Golang, and data warehousing.
When not working, Krishnan likes to spend time with his wife and daughter, besides reading fiction,
nonfiction, and technical books and participating in hackathons. Krishnan tries to give back to the
community by being part of a GDG, the Pune volunteer group, helping the team to organize events.
You can connect with Krishnan at [email protected] or via LinkedIn
at www.linkedin.com/in/krishnan-Raghavan.

I would like to thank my wife, Anita, and daughter, Ananya, for giving me the time and space to
review this book.
 Table of Contents
Prefacexi

Part 1: Introduction to Neural Networks

1
Machine Learning – an Introduction 3
Technical requirements 3 Reinforcement learning 15
Introduction to ML 4 Components of an ML solution 18
Neural networks 21
Different ML approaches 5
Introducing PyTorch 22
Supervised learning 5
Unsupervised learning 11 Summary26

2
Neural Networks 27
Technical requirements 27 Multi-layer NNs 46
The need for NNs 28 Activation functions 47
The universal approximation theorem 49
The math of NNs 28
Linear algebra 29 Training NNs 52
An introduction to probability 33 GD52
Differential calculus 39 Backpropagation56
A code example of an NN for the
An introduction to NNs 41
XOR function 58
Units – the smallest NN building block 42
Layers as operations 44 Summary64
vi Table of Contents

3
Deep Learning Fundamentals 65
Technical requirements 65 Improved activation functions 72
Introduction to DL 66 DNN regularization 76

Fundamental DL concepts 67 Applications of DL 79

Feature learning 68 Introducing popular DL libraries 82
The reasons for DL’s popularity 69 Classifying digits with Keras 82
Deep neural networks 70 Classifying digits with PyTorch 86

Training deep neural networks 71 Summary89

Part 2: Deep Neural Networks for Computer

Vision
4
Computer Vision with Convolutional Networks 93
Technical requirements 94 Advanced types of convolutions 113
Intuition and justification for CNNs 94 1D, 2D, and 3D convolutions 113
Convolutional layers 95 1×1 convolutions 114
Depthwise separable convolutions 114
A coding example of the convolution operation 98
Dilated convolutions 115
Cross-channel and depthwise convolutions 100
Transposed convolutions 116
Stride and padding in convolutional layers 103

Pooling layers 104 Advanced CNN models 119

Introducing residual networks 120
The structure of a convolutional
Inception networks 123
network106
Introducing Xception 128
Classifying images with PyTorch
Squeeze-and-Excitation Networks 129
and Keras 107
Introducing MobileNet 130
Convolutional layers in deep learning libraries 107
EfficientNet132
Data augmentation 107
Using pre-trained models with
Classifying images with PyTorch 108 PyTorch and Keras 133
Classifying images with Keras 111
Summary134
 Table of Contents vii

5
Advanced Computer Vision Applications 135
Technical requirements 136 Introducing image segmentation 159
Transfer learning (TL) 136 Semantic segmentation with U-Net 160
Transfer learning with PyTorch 138 Instance segmentation with Mask R-CNN 162
Transfer learning with Keras 141 Image generation with diffusion
Object detection 145 models165
Approaches to object detection 146 Introducing generative models 166
Object detection with YOLO 148 Denoising Diffusion Probabilistic Models 167
Object detection with Faster R-CNN 153 Summary170

Part 3: Natural Language Processing and

Transformers
6
Natural Language Processing and Recurrent Neural Networks 173
Technical requirements 174 RNN implementation and training 192
Natural language processing 174 Backpropagation through time 194
Vanishing and exploding gradients 197
Tokenization175
Long-short term memory 199
Introducing word embeddings 180
Gated recurrent units 203
Word2Vec182
Visualizing embedding vectors 186 Implementing text classification 204
Language modeling 187 Summary209
Introducing RNNs 189

7
The Attention Mechanism and Transformers 211
Technical requirements 211 Understanding the attention
Introducing seq2seq models 212 mechanism214
viii Table of Contents

Bahdanau attention 214 Transformer decoder 231

Luong attention 217 Putting it all together 234
General attention 218 Decoder-only and encoder-only models 236
Transformer attention 220 Bidirectional Encoder Representations from
Implementing TA 224 Transformers236
Generative Pre-trained Transformer 241
Building transformers with attention 227
Transformer encoder 228 Summary244

8
Exploring Large Language Models in Depth 245
Technical requirements 246 Training datasets 260
Introducing LLMs 246 Pre-training properties 263
FT with RLHF 268
LLM architecture 247
LLM attention variants 247 Emergent abilities of LLMs 270
Prefix decoder 254 Introducing Hugging Face
Transformer nuts and bolts 255 Transformers272
Models258 Summary276
Training LLMs 259

9
Advanced Applications of Large Language Models 277
Technical requirements 277 Conditioning transformer 290
Classifying images with Vision Diffusion model 292
Transformer278 Using stable diffusion with Hugging Face
Transformers293
Using ViT with Hugging Face Transformers 280
Exploring fine-tuning transformers 296
Understanding the DEtection
TRansformer282 Harnessing the power of LLMs with
Using DetR with Hugging Face Transformers 286
LangChain298
Using LangChain in practice 299
Generating images with stable
diffusion288 Summary302
Autoencoder289
 Table of Contents ix

Part 4: Developing and Deploying Deep Neural

Networks
10
Machine Learning Operations (MLOps) 305
Technical requirements 306 Mixed-precision training with PyTorch 319
Understanding model development 306 Exploring model deployment 320
Choosing an NN framework 306 Deploying NN models with Flask 320
PyTorch versus TensorFlow versus JAX 306 Building ML web apps with Gradio 322
Open Neural Network Exchange 307
Introducing TensorBoard 312
Summary325
Developing NN models for edge devices
with TF Lite 316

Index327

Other Books You May Enjoy 342

Visit https://textbookfull.com
now to explore a rich
collection of eBooks, textbook
and enjoy exciting offers!
 Preface
The book will start from the theoretical foundations of deep neural networks (NN), and it will delve
into the most popular network architectures – transformers, transformer-based large language models
(LLMs), and convolutional networks. It will introduce these models in the context of various computer
vision and natural language processing (NLP) examples, including state-of-the-art applications such
as text-to-image generation and chatbots.
Each chapter is organized with a comprehensive theoretical introduction to the topic as its main body.
This is followed by coding examples that serve to validate the presented theory, providing readers
with practical hands-on experience. The examples are executed using PyTorch, Keras, or Hugging
Face Transformers.

Who this book is for

This book is for individuals already familiar with programming – software developers/engineers,
students, data scientists, data analysts, machine learning engineers, statisticians, and anyone interested
in deep learning who has Python programming experience. It is designed for people with minimal
prior deep learning knowledge, employing clear and straightforward language throughout.

What this book covers

Chapter 1, Machine Learning – an Introduction, discusses the basic machine learning paradigms. It will
explore various machine learning algorithms and introduce the first NN, implemented with PyTorch.
Chapter 2, Neural Networks, starts by introducing the mathematical branches related to NNs – linear
algebra, probability, and differential calculus. It will focus on the building blocks and structure of NNs.
It will also discuss how to train NNs with gradient descent and backpropagation.
Chapter 3, Deep Learning Fundamentals, introduces the basic paradigms of deep learning. It will make
the transition from classic networks to deep NNs. It will outline the challenges of developing and
using deep networks, and it will discuss how to solve them.
Chapter 4, Computer Vision with Convolutional Networks, introduces convolutional networks – the
main network architecture for computer vision applications. It will discuss in detail their properties
and building blocks. It will also introduce the most popular convolutional network models in use today.
xii Preface

Chapter 5, Advanced Computer Vision Applications, discusses applying convolutional networks for
advanced computer vision tasks – object detection and image segmentation. It will also explore using
NNs to generate new images.
Chapter 6, Natural Language Processing and Recurrent Neural Networks, introduces the main paradigms
and data processing pipeline of NLP. It will also explore recurrent NNs and their two most popular
variants – long short-term memory and gated recurrent units.
Chapter 7, The Attention Mechanism and Transformers, introduces one of the most significant recent
deep learning advances – the attention mechanism and the transformer model based around it.
Chapter 8, Exploring Large Language Models in Depth, introduces transformer-based LLMs. It will
discuss their properties and what makes them different than other NN models. It will also introduce
the Hugging Face Transformers library.
Chapter 9, Advanced Applications of Large Language Models, discusses using LLMs for computer vision
tasks. It will focus on classic tasks such as image classification and object detection, but it will also
explore state-of-the-art applications such as text-to-image generation. It will introduce the LangChain
framework for LLM-driven application development.
Chapter 10, Machine Learning Operations (MLOps), will introduce various libraries and techniques
for easier development and production deployment of NN models.

To get the most out of this book

Many code examples in the book require the presence of a GPU. Don’t worry if you don’t have one.
To avoid any hardware limitations, all code examples are available as Jupyter notebooks, executed on
Google Colab. So, even if your hardware is not sufficient to run the examples, you can still run them
under Colab.

Software/hardware covered in the book Operating system requirements

PyTorch 2.0.1 Windows, macOS, or Linux
TensorFlow 2.13 Windows (legacy support), macOS, or Linux
Hugging Face Transformers 4.33 Windows, macOS, or Linux

Some code examples in the book may use additional packages not listed in the table. You can see the
full list (with versions) in the requirements.txt file in the book’s GitHub repo.
If you are using the digital version of this book, we advise you to type the code yourself or access
the code from the book’s GitHub repository (a link is available in the next section). Doing so will
help you avoid any potential errors related to the copying and pasting of code.
 Preface xiii

Download the example code files

You can download the example code files for this book from GitHub at https://github.com/
PacktPublishing/Python-Deep-Learning-Third-Edition/. If there’s an update to
the code, it will be updated in the GitHub repository.
We also have other code bundles from our rich catalog of books and videos available at https://
github.com/PacktPublishing/. Check them out!

Conventions used
There are a number of text conventions used throughout this book.
Code in text: Indicates code words in text, database table names, folder names, filenames, file
extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: “Use
opencv-python to read the RGB image located at image_file_path.”
A block of code is set as follows:

def build_fe_model():
    """"Create feature extraction model from the pre-trained model
ResNet50V2"""

    # create the pre-trained part of the network, excluding FC layers

    base_model = tf.keras.applications.MobileNetV3Small(

Any command-line input or output is written as follows:

import tensorflow.keras

When we wish to draw your attention to a particular part of a code block, the relevant lines or items
are set in bold:

import io
image = Image.open(io.BytesIO(response.content))
image.show()

Tips or important notes

Appear like this.
xiv Preface

Get in touch
Feedback from our readers is always welcome.
General feedback: If you have questions about any aspect of this book, email us at customercare@
packtpub.com and mention the book title in the subject of your message.
Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen.
If you have found a mistake in this book, we would be grateful if you would report this to us. Please
visit www.packtpub.com/support/errata and fill in the form.
Piracy: If you come across any illegal copies of our works in any form on the internet, we would
be grateful if you would provide us with the location address or website name. Please contact us at
[email protected] with a link to the material.
If you are interested in becoming an author: If there is a topic that you have expertise in and you
are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Share Your Thoughts

Once you’ve read Python Deep Learning, Third Edition, we’d love to hear your thoughts! Please click
here to go straight to the Amazon review page for this book and share your feedback.
Your review is important to us and the tech community and will help us make sure we’re delivering
excellent quality content.
 Preface xv

Download a free PDF copy of this book

Thanks for purchasing this book!
Do you like to read on the go but are unable to carry your print books everywhere?
Is your eBook purchase not compatible with the device of your choice?
Don’t worry, now with every Packt book you get a DRM-free PDF version of that book at no cost.
Read anywhere, any place, on any device. Search, copy, and paste code from your favorite technical
books directly into your application.
The perks don’t stop there, you can get exclusive access to discounts, newsletters, and great free content
in your inbox daily
Follow these simple steps to get the benefits:

1. Scan the QR code or visit the link below

https://packt.link/free-ebook/9781837638505

2. Submit your proof of purchase

3. That’s it! We’ll send your free PDF and other benefits to your email directly
 Part 1:
Introduction
to Neural Networks

We’ll start this part by introducing you to basic machine learning theory and concepts. Then, we’ll
follow with a thorough introduction to neural networks – a special type of machine learning algorithm.
We’ll discuss the mathematical principles behind them and learn how to train them. Finally, we’ll
make the transition from shallow to deep networks.
This part has the following chapters:

• Chapter 1, Machine Learning – an Introduction

• Chapter 2, Neural Networks
• Chapter 3, Deep Learning Fundamentals
 1
Machine Learning – an
Introduction
Machine learning (ML) techniques are being applied in a variety of fields, and data scientists are being
sought after in many different industries. With ML, we identify the processes through which we gain
knowledge that is not readily apparent from data to make decisions. Applications of ML techniques
may vary greatly and are found in disciplines as diverse as medicine, finance, and advertising.
In this chapter, we’ll present different ML approaches, techniques, and some of their applications to
real-world problems, and we’ll also introduce one of the major open source packages available in Python
for ML, PyTorch. This will lay the foundation for later chapters in which we’ll focus on a particular type
of ML approach using neural networks (NNs). In particular, we will focus on deep learning (DL).
DL makes use of more advanced NNs than those used previously. This is not only a result of recent
developments in the theory but also advancements in computer hardware. This chapter will summarize
what ML is and what it can do, preparing you to better understand how DL differentiates itself from
popular traditional ML techniques.
In this chapter, we’re going to cover the following main topics:

• Introduction to ML
• Different ML approaches
• Neural networks
• Introduction to PyTorch

Technical requirements
We’ll implement the example in this chapter using Python and PyTorch. If you don’t have an
environment set up with these tools, fret not – the example is available as a Jupyter notebook on
Google Colab. You can find the code examples in the book’s GitHub repository: https://github.
com/PacktPublishing/Python-Deep-Learning-Third-Edition/tree/main/
Chapter01.
Visit https://textbookfull.com
now to explore a rich
collection of eBooks, textbook
and enjoy exciting offers!
4 Machine Learning – an Introduction

Introduction to ML
ML is often associated with terms such as big data and artificial intelligence (AI). However, both are
quite different from ML. To understand what ML is and why it’s useful, it’s important to understand
what big data is and how ML applies to it.
Big data is a term used to describe huge datasets that are created as the result of large increases in data
that is gathered and stored. For example, this may be through cameras, sensors, or internet social sites.

How much data do we create daily?

It’s estimated that Google alone processes over 20 petabytes of information per day, and
this number is only going to increase. A few years ago, Forbes estimated that every day, 2.5
quintillion bytes of data are created and that 90% of all the data in the world has been created
in the last two years.
(https://www.forbes.com/sites/bernardmarr/2018/05/21/how-much-
data-do-we-create-every-day-the-mind-blowing-stats-everyone-
should-read/)

Humans alone are unable to grasp, let alone analyze, such huge amounts of data, and ML techniques
are used to make sense of these very large datasets. ML is the tool that’s used for large-scale data
processing. It is well suited to complex datasets that have huge numbers of variables and features. One
of the strengths of many ML techniques, and DL in particular, is that they perform best when used
on large datasets, thus improving their analytic and predictive power. In other words, ML techniques,
and DL NNs in particular, learn best when they can access large datasets where they can discover
patterns and regularities hidden in the data.
On the other hand, ML’s predictive ability can be successfully adapted to AI systems. ML can be thought
of as the brain of an AI system. AI can be defined (though this definition may not be unique) as a
system that can interact with its environment. Also, AI machines are endowed with sensors that enable
them to know the environment they are in and tools with which they can relate to the environment.
Therefore, ML is the brain that allows the machine to analyze the data ingested through its sensors
to formulate an appropriate answer. A simple example is Siri on an iPhone. Siri hears the command
through its microphone and outputs an answer through its speakers or its display, but to do so, it
needs to understand what it’s being told. Similarly, driverless cars will be equipped with cameras,
GPS systems, sonars, and LiDAR, but all this information needs to be processed to provide a correct
answer. This may include whether to accelerate, brake, or turn. ML is the information-processing
method that leads to the answer.
We’ve explained what ML is, but what about DL? For now, let’s just say that DL is a subfield of ML.
DL methods share some special common features. The most popular representatives of such methods
are deep NNs.
 Different ML approaches 5

Different ML approaches
As we have seen, the term ML is used in a very general way and refers to the general techniques that
are used to extrapolate patterns from large sets, or it is the ability to make predictions on new data
based on what is learned by analyzing available known data. ML techniques can roughly be divided
into two core classes, while one more class is often added. Here are the classes:

• Supervised learning
• Unsupervised learning
• Reinforcement learning

Let’s take a closer look.

Supervised learning
Supervised learning algorithms are a class of ML algorithms that use previously labeled data to learn
its features, so they can classify similar but unlabeled data. Let’s use an example to understand this
concept better.
Let’s assume that a user receives many emails every day, some of which are important business emails
and some of which are unsolicited junk emails, also known as spam. A supervised machine algorithm
will be presented with a large body of emails that have already been labeled by a teacher as spam or
not spam (this is called training data). For each sample, the machine will try to predict whether the
email is spam or not, and it will compare the prediction with the original target label. If the prediction
differs from the target, the machine will adjust its internal parameters in such a way that the next
time it encounters this sample, it will classify it correctly. Conversely, if the prediction is correct, the
parameters will stay the same. The more training data we feed to the algorithm, the better it becomes
(this rule has caveats, as we’ll see next).
In the example we used, the emails had only two classes (spam or not spam), but the same principles
apply to tasks with arbitrary numbers of classes (or categories). For example, Gmail, the free email
service by Google, allows the user to select up to five categories, which are labeled as follows:

• Primary: Includes person-to-person conversations

• Promotions: Includes marketing emails, offers, and discounts
• Social: Includes messages from social networks and media-sharing sites
• Updates: Includes bills, bank statements, and receipts
• Forums: Includes messages from online groups and mailing lists
6 Machine Learning – an Introduction

To summarize, the ML task, which maps a set of input values to a finite number of classes, is
called classification.
In some cases, the outcome may not necessarily be discrete, and we may not have a finite number
of classes to classify our data into. For example, we may try to predict the life expectancy of a group
of people based on their predetermined health parameters. In this case, the outcome is a numerical
value, and we don’t talk about classification but rather regression.
One way to think of supervised learning is to imagine we are building a function, f, defined over a
dataset, which comprises information organized by features. In the case of email classification, the
features can be specific words that may appear more frequently than others in spam emails. The use
of explicit sex-related words will most likely identify a spam email rather than a business/work email.
On the contrary, words such as meeting, business, or presentation are more likely to describe a work
email. If we have access to metadata, we may also use the sender’s information as a feature. Each email
will then have an associated set of features, and each feature will have a value (in this case, how many
times the specific word is present in the email’s body). The ML algorithm will then seek to map those
values to a discrete range that represents the set of classes, or a real value in the case of regression.
The definition of the f function is as follows:
​f : space of features → classes = (discrete values or real values)​
In later chapters, we’ll see several examples of either classification or regression problems. One such
problem we’ll discuss is classifying handwritten digits of the Modified National Institute of Standards
and Technology (MNIST) database (http://yann.lecun.com/exdb/mnist/). When given
a set of images representing 0 to 9, the ML algorithm will try to classify each image in one of the 10
classes, wherein each class corresponds to one of the 10 digits. Each image is 28×28 (= 784) pixels in
size. If we think of each pixel as one feature, then the algorithm will use a 784-dimensional feature
space to classify the digits.
The following figure depicts the handwritten digits from the MNIST dataset:

Figure 1.1 – An example of handwritten digits from the MNIST dataset

In the next sections, we’ll talk about some of the most popular classical supervised algorithms. The
following is by no means an exhaustive list or a thorough description of each ML method. We recommend
referring to the book Python Machine Learning, by Sebastian Raschka (https://www.packtpub.
 Different ML approaches 7

com/product/python-machine-learning-third-edition/9781789955750). It’s
a simple review meant to provide you with a flavor of the different ML techniques in Python.

Linear and logistic regression

A regression algorithm is a type of supervised algorithm that uses features of the input data to predict
a numeric value, such as the cost of a house, given certain features, such as size, age, number of
bathrooms, number of floors, and location. Regression analysis tries to find the value of the parameters
for the function that best fits an input dataset.
In a linear regression algorithm, the goal is to minimize a cost function by finding appropriate
parameters for the function over the input data that best approximates the target values. A cost
function is a function of the error – that is, how far we are from getting a correct result. A popular
cost function is the mean squared error (MSE), where we take the square of the difference between
the expected value and the predicted result. The sum of all the input examples gives us the error of
the algorithm and represents the cost function.
Say we have a 100-square-meter house that was built 25 years ago with three bathrooms and two
floors. Let’s also assume that the city is divided into 10 different neighborhoods, which we’ll denote
with integers from 1 to 10, and say this house is located in the area denoted by 7. We can parameterize
this house with a five-dimensional vector, ​x = ​(​x1​ ​, ​x2​ ​, ​x3​ ​, ​x4​ ​, ​x5​ ​)​ = ​(100,25,3, 2,7)​. Say that we also
know that this house has an estimated value of $100,000 (in today’s world, this would be enough for
just a tiny shack near the North Pole, but let’s pretend). What we want is to create a function, f, such
that f​(​ x)​ = 100000​.

A note of encouragement
Don’t worry If you don’t fully understand some of the terms in this section. We’ll discuss vectors,
cost functions, linear regression, and gradient descent in more detail in Chapter 2. We will also
see that training NNs and linear/logistic regressions have a lot in common. For now, you can
think of a vector as an array. We’ll denote vectors with boldface font – for example, x​ ​. We’ll
denote the vector elements with italic font and subscript – for example, ​xi​​​.

In linear regression, this means finding a vector of weights, w​ = (​ w ​ 1​ ,​ w

​ 2​ ,​ w
​ 3​ ,​ w
​ 4​ ,​ w
​ 5​ )​ ​, such that the dot
product of the vectors, ​x ⋅ w = 100000​, would be ​100 ​w1​ ​+ 25 ​w2​ ​+ 3 ​w3​ ​+ 2 ​w4​ ​+ 7 ​w5​ ​ = 100000​ or​​
∑ ​xi​​​wi​​ = 100000​. If we had 1,000 houses, we could repeat the same process for every house, and
ideally, we would like to find a single vector, w, that can predict the correct value that is close enough
for every house. The most common way to train a linear regression model can be seen in the following
pseudocode block:

Initialize the vector w with some random values

repeat:
  E = 0 # initialize the cost function E with 0
  for every pair (​​
x​(i)​
​
,​
​ t​
​
)​of the training set:
(i)​
​
8 Machine Learning – an Introduction

2
E += ​
      ​ (​x​(i)​
​
​⋅ w − ​t​​ )​
(i)​
​ t​
​​  ​​# here ​ (i)​
​
​is the real house price
MSE = ​
  ​ ______________
  
    E ​ # Mean Square Error
total number of samples

  use gradient descent to update the weights w based on MSE until MSE
falls below threshold

First, we iterate over the training data to compute the cost function, MSE. Once we know the value
of MSE, we’ll use the gradient descent algorithm to update the weights of the vector, w. To do this,
we’ll calculate the derivatives of the cost function concerning each weight, ​wi​.​ In this way, we’ll know
how the cost function changes (increase or decrease) concerning ​wi​​. Then we’ll update that weight’s
value accordingly.
Previously, we demonstrated how to solve a regression problem with linear regression. Now, let’s take
a classification task: trying to determine whether a house is overvalued or undervalued. In this case,
the target data would be categorical [1, 0] – 1 for overvalued and 0 for undervalued. The price of the
house will be an input parameter instead of the target value as before. To solve the task, we’ll use logistic
regression. This is similar to linear regression but with one difference: in linear regression, the output
is x​ ⋅ w​. However, here, the output will be a special logistic function (https://en.wikipedia.
org/wiki/Logistic_function), ​σ​(x ⋅ w)​. This will squash the value of ​x ⋅ w​in the (0:1)
interval. You can think of the logistic function as a probability, and the closer the result is to 1, the
more chance there is that the house is overvalued, and vice versa. Training is the same as with linear
regression, but the output of the function is in the (0:1) interval, and the labels are either 0 or 1.
Logistic regression is not a classification algorithm, but we can turn it into one. We just have to introduce
a rule that determines the class based on the logistic function’s output. For example, we can say that
a house is overvalued if the value of σ​ (​ x ⋅ w)​ < 0.5​and undervalued otherwise.

Multivariate regression
The regression examples in this section have a single numerical output. A regression analysis
can have more than one output. We’ll refer to such analysis as multivariate regression.

Support vector machines

A support vector machine (SVM) is a supervised ML algorithm that’s mainly used for classification.
It is the most popular member of the kernel method class of algorithms. An SVM tries to find a
hyperplane, which separates the samples in the dataset.

Hyperplanes
A hyperplane is a plane in a high-dimensional space. For example, a hyperplane in a one-
dimensional space is a point, and in a two-dimensional space, it would just be a line. In three-
dimensional space, the hyperplane would be a plane, and we can’t visualize the hyperplane in
four-dimensional space, but we know that it exists.
Random documents with unrelated
content Scribd suggests to you:
Paulina's prediction came true, and we were soon busy preparing for her
father's marriage with Miss Cottrell. It took place in our beautiful old church on
the thirty-first of July. The happy pair spent a week at Felixstowe and then
came back to "Gay Bowers" to fetch Pollie. It was with genuine regret that
Aunt Patty and I watched Mr. and Mrs. Dicks and Paulina take their departure.
How different were our feelings now from those with which we had received
the Americans and Miss Cottrell! The paying guests had become our friends.

"Au revoir!" cried Pollie as they drove away. "We are coming back some day.
And, Mr. Faulkner, please don't forget that you are going to bring Nan to
Indianapolis some time."

We watched them pass out of our sight with the sadness most partings
inevitably bring, for who could say whether we should all meet again?

Two days later, Alan's sisters came to spend their holidays at "Gay Bowers."
They were such nice, bright girls that I had no difficulty in making friends of
them, and I am thankful to say they seemed to take to me at once. The
brother, who was their guardian, was so great a hero in their eyes, that I
wonder they thought me good enough for him. It must have been, because
they thought he could not make a wrong choice.

Peggy joined us ere August was far advanced, and we became a very lively
party. By this time Jack had returned to the vicarage. I had the satisfaction of
seeing that Aunt Patty had rightly gauged the depth of his wound. If the news
of my engagement to Alan Faulkner hurt him, the blow was one from which he
quickly recovered. He and Peggy became good comrades; she wanted to
practise sketching during her stay in the country and he helped her to find
suitable "bits," and was her attendant squire on many of her expeditions.

I had heard nothing from Agneta since her return to Manchester, but the news
of my engagement brought me a kind though rather sad letter from her. She
said she thought that I and Professor Faulkner were exactly suited to each
other and she was glad I was going to be happy, for I deserved happiness and
she supposed she never had. She knew now that she had been utterly
deluded when she imagined that Ralph Marshman would make her happy.
She wanted me to know that she was convinced of his worthless character
and of what an escape she had had. She thanked me for the efforts I had
made to save her from her own folly, and she begged me to forgive her for
being so ungrateful at the time. She said she was sick of her life at home. She
wanted her parents to let her adopt a career of her own and live a more useful
life, but her mother refused to entertain the idea for a moment.
"I am trying to be patient," Agneta wrote; "You know you were always
preaching patience to me, Nan; and I mean to do some 'solid' reading every
day. Do send me a list of books you think I ought to read. I know, although you
never said so, that you thought me very ignorant when I was with you. I don't
forget either how you once said that I never should be happy as long as I
made myself the centre of my life. So I try to be unselfish and to think of other
people, but there is really very little I can do for others in the life I lead here. I
almost envy girls who have to work for themselves."

I felt very sorry for Agneta as I read her letter, and yet I should have been
glad, for, if her words were sincere, they augured for her happier days than
she had yet known. For what hope of happiness is there for any one who is
shut up in the prison-house of self? It was good for Agneta, as it had been for
me and for Paulina, to suffer, if her trouble had led her into a larger, fuller, and
more blessed life.

But the story of Aunt Patty's guests, as far as I have known them intimately,
must be brought to a close. After all, I did not stay quite twelve months at "Gay
Bowers." I went home for Christmas and I did not return. There was no longer
any thought of my going up for Matriculation. Even now I regret that I never
did so, but mother was bent upon my entering on a course of domestic
economy, and the value of that study I am daily proving.

Early in the New Year, Olive was married. It was a very pretty wedding and
everything went off charmingly; but her departure for India six weeks later left
us all with very sore hearts. Alan was duly appointed to the professorship at
Edinburgh, and now my home is in that beautiful old city, for in the following
year, at the beginning of the summer vacation, we were married.

I should like to write about that wedding, but Alan thinks I had better not begin.
My three sisters, Alan's two, and Cousin Agneta were my bridesmaids. Mr.
Upsher assisted at the ceremony, and Jack, such a handsome young soldier,
was one of the guests. He still showed himself devoted to Peggy, but I hope
he is not seriously attracted by her, for Peggy declares that she is wedded to
her art and is quite angry if any one suggests that she may marry. She is now
working hard in Paris and promises to develop into a first-rate artist in "black
and white."

Agneta made a very pretty bridesmaid and looked as happy as one could
wish. I say this on mother's authority, for really I cannot remember how any
one looked except Alan. The sun must have been in my eyes all the time, for
my recollection of everything is so vague and hazy. So it was wise of Alan to
advise me not to attempt to describe our wedding. Soon afterwards we heard
of Agneta's engagement, with her parents' approval, to a young medical man,
so I dare say she did look happy.

Alan and I always agree that "Gay Bowers" is the most delightful old country
house we have ever known. Apparently many are of the same opinion, for
aunt seldom has a room to spare in it.

THE END

Richard Clay & Sons, Limited, London and Bungay.

*** END OF THE PROJECT GUTENBERG EBOOK AUNT PATTY'S
PAYING GUESTS ***

Updated editions will replace the previous one—the old editions will
be renamed.

Creating the works from print editions not protected by U.S.

copyright law means that no one owns a United States copyright in
these works, so the Foundation (and you!) can copy and distribute it
in the United States without permission and without paying copyright
royalties. Special rules, set forth in the General Terms of Use part of
this license, apply to copying and distributing Project Gutenberg™
electronic works to protect the PROJECT GUTENBERG™ concept
and trademark. Project Gutenberg is a registered trademark, and
may not be used if you charge for an eBook, except by following the
terms of the trademark license, including paying royalties for use of
the Project Gutenberg trademark. If you do not charge anything for
copies of this eBook, complying with the trademark license is very
easy. You may use this eBook for nearly any purpose such as
creation of derivative works, reports, performances and research.
Project Gutenberg eBooks may be modified and printed and given
away—you may do practically ANYTHING in the United States with
eBooks not protected by U.S. copyright law. Redistribution is subject
to the trademark license, especially commercial redistribution.

START: FULL LICENSE

THE FULL PROJECT GUTENBERG LICENSE
 PLEASE READ THIS BEFORE YOU DISTRIBUTE OR USE THIS WORK

To protect the Project Gutenberg™ mission of promoting the free

distribution of electronic works, by using or distributing this work (or
any other work associated in any way with the phrase “Project
Gutenberg”), you agree to comply with all the terms of the Full
Project Gutenberg™ License available with this file or online at
www.gutenberg.org/license.

Section 1. General Terms of Use and

Redistributing Project Gutenberg™
electronic works
1.A. By reading or using any part of this Project Gutenberg™
electronic work, you indicate that you have read, understand, agree
to and accept all the terms of this license and intellectual property
(trademark/copyright) agreement. If you do not agree to abide by all
the terms of this agreement, you must cease using and return or
destroy all copies of Project Gutenberg™ electronic works in your
possession. If you paid a fee for obtaining a copy of or access to a
Project Gutenberg™ electronic work and you do not agree to be
bound by the terms of this agreement, you may obtain a refund from
the person or entity to whom you paid the fee as set forth in
paragraph 1.E.8.

1.B. “Project Gutenberg” is a registered trademark. It may only be

used on or associated in any way with an electronic work by people
who agree to be bound by the terms of this agreement. There are a
few things that you can do with most Project Gutenberg™ electronic
works even without complying with the full terms of this agreement.
See paragraph 1.C below. There are a lot of things you can do with
Project Gutenberg™ electronic works if you follow the terms of this
agreement and help preserve free future access to Project
Gutenberg™ electronic works. See paragraph 1.E below.
1.C. The Project Gutenberg Literary Archive Foundation (“the
Foundation” or PGLAF), owns a compilation copyright in the
collection of Project Gutenberg™ electronic works. Nearly all the
individual works in the collection are in the public domain in the
United States. If an individual work is unprotected by copyright law in
the United States and you are located in the United States, we do
not claim a right to prevent you from copying, distributing,
performing, displaying or creating derivative works based on the
work as long as all references to Project Gutenberg are removed. Of
course, we hope that you will support the Project Gutenberg™
mission of promoting free access to electronic works by freely
sharing Project Gutenberg™ works in compliance with the terms of
this agreement for keeping the Project Gutenberg™ name
associated with the work. You can easily comply with the terms of
this agreement by keeping this work in the same format with its
attached full Project Gutenberg™ License when you share it without
charge with others.

1.D. The copyright laws of the place where you are located also
govern what you can do with this work. Copyright laws in most
countries are in a constant state of change. If you are outside the
United States, check the laws of your country in addition to the terms
of this agreement before downloading, copying, displaying,
performing, distributing or creating derivative works based on this
work or any other Project Gutenberg™ work. The Foundation makes
no representations concerning the copyright status of any work in
any country other than the United States.

1.E. Unless you have removed all references to Project Gutenberg:

1.E.1. The following sentence, with active links to, or other

immediate access to, the full Project Gutenberg™ License must
appear prominently whenever any copy of a Project Gutenberg™
work (any work on which the phrase “Project Gutenberg” appears, or
with which the phrase “Project Gutenberg” is associated) is
accessed, displayed, performed, viewed, copied or distributed:
 This eBook is for the use of anyone anywhere in the United
States and most other parts of the world at no cost and with
almost no restrictions whatsoever. You may copy it, give it away
or re-use it under the terms of the Project Gutenberg License
included with this eBook or online at www.gutenberg.org. If you
are not located in the United States, you will have to check the
laws of the country where you are located before using this
eBook.

1.E.2. If an individual Project Gutenberg™ electronic work is derived

from texts not protected by U.S. copyright law (does not contain a
notice indicating that it is posted with permission of the copyright
holder), the work can be copied and distributed to anyone in the
United States without paying any fees or charges. If you are
redistributing or providing access to a work with the phrase “Project
Gutenberg” associated with or appearing on the work, you must
comply either with the requirements of paragraphs 1.E.1 through
1.E.7 or obtain permission for the use of the work and the Project
Gutenberg™ trademark as set forth in paragraphs 1.E.8 or 1.E.9.

1.E.3. If an individual Project Gutenberg™ electronic work is posted

with the permission of the copyright holder, your use and distribution
must comply with both paragraphs 1.E.1 through 1.E.7 and any
additional terms imposed by the copyright holder. Additional terms
will be linked to the Project Gutenberg™ License for all works posted
with the permission of the copyright holder found at the beginning of
this work.

1.E.4. Do not unlink or detach or remove the full Project

Gutenberg™ License terms from this work, or any files containing a
part of this work or any other work associated with Project
Gutenberg™.

1.E.5. Do not copy, display, perform, distribute or redistribute this

electronic work, or any part of this electronic work, without
prominently displaying the sentence set forth in paragraph 1.E.1 with
active links or immediate access to the full terms of the Project
Gutenberg™ License.
1.E.6. You may convert to and distribute this work in any binary,
compressed, marked up, nonproprietary or proprietary form,
including any word processing or hypertext form. However, if you
provide access to or distribute copies of a Project Gutenberg™ work
in a format other than “Plain Vanilla ASCII” or other format used in
the official version posted on the official Project Gutenberg™ website
(www.gutenberg.org), you must, at no additional cost, fee or expense
to the user, provide a copy, a means of exporting a copy, or a means
of obtaining a copy upon request, of the work in its original “Plain
Vanilla ASCII” or other form. Any alternate format must include the
full Project Gutenberg™ License as specified in paragraph 1.E.1.

1.E.7. Do not charge a fee for access to, viewing, displaying,

performing, copying or distributing any Project Gutenberg™ works
unless you comply with paragraph 1.E.8 or 1.E.9.

1.E.8. You may charge a reasonable fee for copies of or providing

access to or distributing Project Gutenberg™ electronic works
provided that:

• You pay a royalty fee of 20% of the gross profits you derive from
the use of Project Gutenberg™ works calculated using the
method you already use to calculate your applicable taxes. The
fee is owed to the owner of the Project Gutenberg™ trademark,
but he has agreed to donate royalties under this paragraph to
the Project Gutenberg Literary Archive Foundation. Royalty
payments must be paid within 60 days following each date on
which you prepare (or are legally required to prepare) your
periodic tax returns. Royalty payments should be clearly marked
as such and sent to the Project Gutenberg Literary Archive
Foundation at the address specified in Section 4, “Information
about donations to the Project Gutenberg Literary Archive
Foundation.”

• You provide a full refund of any money paid by a user who

notifies you in writing (or by e-mail) within 30 days of receipt that
s/he does not agree to the terms of the full Project Gutenberg™
License. You must require such a user to return or destroy all
 copies of the works possessed in a physical medium and
discontinue all use of and all access to other copies of Project
Gutenberg™ works.

• You provide, in accordance with paragraph 1.F.3, a full refund of

any money paid for a work or a replacement copy, if a defect in
the electronic work is discovered and reported to you within 90
days of receipt of the work.

• You comply with all other terms of this agreement for free
distribution of Project Gutenberg™ works.

1.E.9. If you wish to charge a fee or distribute a Project Gutenberg™

electronic work or group of works on different terms than are set
forth in this agreement, you must obtain permission in writing from
the Project Gutenberg Literary Archive Foundation, the manager of
the Project Gutenberg™ trademark. Contact the Foundation as set
forth in Section 3 below.

1.F.

1.F.1. Project Gutenberg volunteers and employees expend

considerable effort to identify, do copyright research on, transcribe
and proofread works not protected by U.S. copyright law in creating
the Project Gutenberg™ collection. Despite these efforts, Project
Gutenberg™ electronic works, and the medium on which they may
be stored, may contain “Defects,” such as, but not limited to,
incomplete, inaccurate or corrupt data, transcription errors, a
copyright or other intellectual property infringement, a defective or
damaged disk or other medium, a computer virus, or computer
codes that damage or cannot be read by your equipment.

1.F.2. LIMITED WARRANTY, DISCLAIMER OF DAMAGES - Except

for the “Right of Replacement or Refund” described in paragraph
1.F.3, the Project Gutenberg Literary Archive Foundation, the owner
of the Project Gutenberg™ trademark, and any other party
distributing a Project Gutenberg™ electronic work under this
agreement, disclaim all liability to you for damages, costs and
expenses, including legal fees. YOU AGREE THAT YOU HAVE NO
REMEDIES FOR NEGLIGENCE, STRICT LIABILITY, BREACH OF
WARRANTY OR BREACH OF CONTRACT EXCEPT THOSE
PROVIDED IN PARAGRAPH 1.F.3. YOU AGREE THAT THE
FOUNDATION, THE TRADEMARK OWNER, AND ANY
DISTRIBUTOR UNDER THIS AGREEMENT WILL NOT BE LIABLE
TO YOU FOR ACTUAL, DIRECT, INDIRECT, CONSEQUENTIAL,
PUNITIVE OR INCIDENTAL DAMAGES EVEN IF YOU GIVE
NOTICE OF THE POSSIBILITY OF SUCH DAMAGE.

1.F.3. LIMITED RIGHT OF REPLACEMENT OR REFUND - If you

discover a defect in this electronic work within 90 days of receiving it,
you can receive a refund of the money (if any) you paid for it by
sending a written explanation to the person you received the work
from. If you received the work on a physical medium, you must
return the medium with your written explanation. The person or entity
that provided you with the defective work may elect to provide a
replacement copy in lieu of a refund. If you received the work
electronically, the person or entity providing it to you may choose to
give you a second opportunity to receive the work electronically in
lieu of a refund. If the second copy is also defective, you may
demand a refund in writing without further opportunities to fix the
problem.

1.F.4. Except for the limited right of replacement or refund set forth in
paragraph 1.F.3, this work is provided to you ‘AS-IS’, WITH NO
OTHER WARRANTIES OF ANY KIND, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR ANY PURPOSE.

1.F.5. Some states do not allow disclaimers of certain implied

warranties or the exclusion or limitation of certain types of damages.
If any disclaimer or limitation set forth in this agreement violates the
law of the state applicable to this agreement, the agreement shall be
interpreted to make the maximum disclaimer or limitation permitted
by the applicable state law. The invalidity or unenforceability of any
provision of this agreement shall not void the remaining provisions.
1.F.6. INDEMNITY - You agree to indemnify and hold the
Foundation, the trademark owner, any agent or employee of the
Foundation, anyone providing copies of Project Gutenberg™
electronic works in accordance with this agreement, and any
volunteers associated with the production, promotion and distribution
of Project Gutenberg™ electronic works, harmless from all liability,
costs and expenses, including legal fees, that arise directly or
indirectly from any of the following which you do or cause to occur:
(a) distribution of this or any Project Gutenberg™ work, (b)
alteration, modification, or additions or deletions to any Project
Gutenberg™ work, and (c) any Defect you cause.

Section 2. Information about the Mission of

Project Gutenberg™
Project Gutenberg™ is synonymous with the free distribution of
electronic works in formats readable by the widest variety of
computers including obsolete, old, middle-aged and new computers.
It exists because of the efforts of hundreds of volunteers and
donations from people in all walks of life.

Volunteers and financial support to provide volunteers with the

assistance they need are critical to reaching Project Gutenberg™’s
goals and ensuring that the Project Gutenberg™ collection will
remain freely available for generations to come. In 2001, the Project
Gutenberg Literary Archive Foundation was created to provide a
secure and permanent future for Project Gutenberg™ and future
generations. To learn more about the Project Gutenberg Literary
Archive Foundation and how your efforts and donations can help,
see Sections 3 and 4 and the Foundation information page at
www.gutenberg.org.

Section 3. Information about the Project

Gutenberg Literary Archive Foundation
The Project Gutenberg Literary Archive Foundation is a non-profit
501(c)(3) educational corporation organized under the laws of the
state of Mississippi and granted tax exempt status by the Internal
Revenue Service. The Foundation’s EIN or federal tax identification
number is 64-6221541. Contributions to the Project Gutenberg
Literary Archive Foundation are tax deductible to the full extent
permitted by U.S. federal laws and your state’s laws.

The Foundation’s business office is located at 809 North 1500 West,

Salt Lake City, UT 84116, (801) 596-1887. Email contact links and up
to date contact information can be found at the Foundation’s website
and official page at www.gutenberg.org/contact

Section 4. Information about Donations to

the Project Gutenberg Literary Archive
Foundation
Project Gutenberg™ depends upon and cannot survive without
widespread public support and donations to carry out its mission of
increasing the number of public domain and licensed works that can
be freely distributed in machine-readable form accessible by the
widest array of equipment including outdated equipment. Many small
donations ($1 to $5,000) are particularly important to maintaining tax
exempt status with the IRS.

The Foundation is committed to complying with the laws regulating

charities and charitable donations in all 50 states of the United
States. Compliance requirements are not uniform and it takes a
considerable effort, much paperwork and many fees to meet and
keep up with these requirements. We do not solicit donations in
locations where we have not received written confirmation of
compliance. To SEND DONATIONS or determine the status of
compliance for any particular state visit www.gutenberg.org/donate.

While we cannot and do not solicit contributions from states where

we have not met the solicitation requirements, we know of no
prohibition against accepting unsolicited donations from donors in
such states who approach us with offers to donate.

International donations are gratefully accepted, but we cannot make

any statements concerning tax treatment of donations received from
outside the United States. U.S. laws alone swamp our small staff.

Please check the Project Gutenberg web pages for current donation
methods and addresses. Donations are accepted in a number of
other ways including checks, online payments and credit card
donations. To donate, please visit: www.gutenberg.org/donate.

Section 5. General Information About Project

Gutenberg™ electronic works
Professor Michael S. Hart was the originator of the Project
Gutenberg™ concept of a library of electronic works that could be
freely shared with anyone. For forty years, he produced and
distributed Project Gutenberg™ eBooks with only a loose network of
volunteer support.

Project Gutenberg™ eBooks are often created from several printed

editions, all of which are confirmed as not protected by copyright in
the U.S. unless a copyright notice is included. Thus, we do not
necessarily keep eBooks in compliance with any particular paper
edition.

Most people start at our website which has the main PG search
facility: www.gutenberg.org.

This website includes information about Project Gutenberg™,

including how to make donations to the Project Gutenberg Literary
Archive Foundation, how to help produce our new eBooks, and how
to subscribe to our email newsletter to hear about new eBooks.