Get the full ebook with Bonus Features for a Better Reading Experience on ebookmeta.

com

Machine Learning Algorithms in Depth MEAP V01

Vadim Smolyakov

https://ebookmeta.com/product/machine-learning-algorithms-
in-depth-meap-v01-vadim-smolyakov/

OR CLICK HERE

DOWLOAD NOW

Download more ebook instantly today at https://ebookmeta.com

 Recommended digital products (PDF, EPUB, MOBI) that
you can download immediately if you are interested.

Terraform in Depth (MEAP V01) Robert Hafner

https://ebookmeta.com/product/terraform-in-depth-meap-v01-robert-
hafner/

ebookmeta.com

CSS in Depth, Second Edition (MEAP V01) Keith Grant

https://ebookmeta.com/product/css-in-depth-second-edition-
meap-v01-keith-grant/

ebookmeta.com

Django in Action (MEAP V01) Christopher Trudeau

https://ebookmeta.com/product/django-in-action-meap-v01-christopher-
trudeau/

ebookmeta.com

Fire Safety Law 1st Edition V. Charles Ward

https://ebookmeta.com/product/fire-safety-law-1st-edition-v-charles-
ward/

ebookmeta.com
Berlitz Pocket Guide Sri Lanka Travel Guide eBook 4th
Edition Berlitz

https://ebookmeta.com/product/berlitz-pocket-guide-sri-lanka-travel-
guide-ebook-4th-edition-berlitz/

ebookmeta.com

The Research Companion A practical guide for those in the

social sciences health and development 2nd Edition Petra
M. Boynton
https://ebookmeta.com/product/the-research-companion-a-practical-
guide-for-those-in-the-social-sciences-health-and-development-2nd-
edition-petra-m-boynton/
ebookmeta.com

From Sunlight to Electricity A Practical Handbook on Solar

Photovoltaic Applications 3rd Edition Suneel Deambi

https://ebookmeta.com/product/from-sunlight-to-electricity-a-
practical-handbook-on-solar-photovoltaic-applications-3rd-edition-
suneel-deambi/
ebookmeta.com

Wild at the Library A Curvy Woman Romance 1st Edition Liz

Fox

https://ebookmeta.com/product/wild-at-the-library-a-curvy-woman-
romance-1st-edition-liz-fox/

ebookmeta.com

A Dictionary of Theatre Anthropology The Secret Art of the

Performer 2nd Edition Eugenio Barba

https://ebookmeta.com/product/a-dictionary-of-theatre-anthropology-
the-secret-art-of-the-performer-2nd-edition-eugenio-barba/

ebookmeta.com
Understanding Virtual Reality Interface Application and
Design Second Edition William R. Sherman

https://ebookmeta.com/product/understanding-virtual-reality-interface-
application-and-design-second-edition-william-r-sherman/

ebookmeta.com
Machine Learning Algorithms in Depth MEAP V01
1. Copyright_2022_Manning_Publications
2. welcome
3. 1_Machine_Learning_Algorithms
4. 2_Markov_Chain_Monte_Carlo
5. 3_Variational_Inference
6. 4_Software_Implementation
MEAP Edition

Manning Early Access Program

Machine Learning Algorithms in Depth

Version 1

Copyright 2022 Manning

Publications

©Manning Publications Co. We welcome reader comments about anything in

the manuscript - other than typos and other simple mistakes.

These will be cleaned up during production of the book by copyeditors and

proofreaders.
https://livebook.manning.com/#!/book/machine-learning-algorithms-in-
depth/discussion

For more information on this and other Manning titles go to

manning.com
welcome
Thank you for purchasing the MEAP for Machine Learning Algorithms in
Depth. This book will take you on a journey from mathematical derivation to
software implementation of some of the most intriguing algorithms in ML.

This book dives into the design of ML algorithms from scratch. Throughout
the book, you will develop mathematical intuition for classic and modern ML
algorithms, learn the fundamentals of Bayesian inference and deep learning,
as well as the data structures and algorithmic paradigms in ML.

Understanding ML algorithms from scratch will help you choose the right
algorithm for the task, explain the results, troubleshoot advanced problems,
extend an algorithm to a new application, and improve performance of
existing algorithms.

Some of the prerequisites for reading this book include basic level of
programming in Python and intermediate level of understanding of linear
algebra, applied probability and multivariate calculus.

My goal in writing this book is to distill the science of ML and present it in a

way that will convey intuition and inspire the reader to self-learn, innovate
and advance the field. Your input is important. I’d like to encourage you to
post questions and comments in the liveBook discussion forum to help
improve presentation of the material.

Thank you again for your interest and welcome to the world of ML
algorithms!

— Vadim Smolyakov

In this book

Copyright 2022 Manning Publications welcome brief contents 1 Machine

Learning Algorithms 2 Markov Chain Monte Carlo 3 Variational Inference 4
Software Implementation
1 Machine Learning Algorithms
This chapter covers
Types of ML algorithms
Importance of learning algorithms from scratch
Introduction to Bayesian Inference and Deep Learning
Software implementation of machine learning algorithms from scratch

An algorithm is a sequence of steps required to achieve a particular task. An

algorithm takes an input, performs a sequence of operations and produces a
desired output. The simplest example of an algorithm is sorting: given a list
of integers, we perform a sequence of operations to produce a sorted list. A
sorted list enables us to organize information better and find answers in our
data.

Two popular questions to ask about an algorithm is how fast does it run (run-
time complexity) and how much memory does it take (memory complexity)
for an input of size n. For example, a comparison-based sort, as we’ll see
later, has O(nlogn) run-time complexity and requires O(n) memory storage.

There are many approaches to sorting, and in each case, in the classic
algorithmic paradigm, the algorithm designer creates a set of instructions.
Imagine a world where you can learn the instructions based on a sequence of
input and output examples available to you. This is a setting of ML
algorithmic paradigm. Similar to how a human brain learns, when we are
playing connect-the-dots game or sketching a nature landscape, we are
comparing the desired output with what we have at each step and filling in
the gaps. This in broad strokes is what (supervised) machine learning (ML)
algorithms do. During training, ML algorithms are learning the rules (e.g.
classification boundaries) based on training examples by optimizing an
objective function. During testing, ML algorithms apply previously learned
rules to new input data points to give a prediction as shown in Figure 1.1
1.1 Types of ML Algorithms
Let’s unpack the previous paragraph a little bit and introduce some notation.
This book focuses on machine learning algorithms that can be grouped
together in the following categories: supervised learning, unsupervised
learning and deep learning. In supervised learning, the task is to learn a
mapping f from inputs x to outputs given a training dataset D = {(x1,y1 ),…,
(xn,yn )} of n input-output pairs. In other words, we are given n examples of
what the output should look like given the input. The output y is also often
referred to as the label, and it is the supervisory signal that tells our algorithm
what the correct answer is.

Figure 1.1 Supervised Learning: Training (left) and Testing (right)

Supervised learning can be sub-divided into classification and regression

based on the quantity we are trying to predict. If our output y is a discrete
quantity (e.g. K distinct classes) we have a classification problem. On the
other hand, if our output y is a continuous quantity (e.g. a real number such as
stock price) we have a regression problem.

Thus, the nature of the problem changes based on the quantity y we are trying
to predict. We want to get as close as possible to the ground truth value of y.
A common way to measure performance or closeness to ground truth is the
loss function. The loss function is computing a distance between the
prediction and the true label. Let y = f(x; θ) be our ML algorithm that
maps input examples x to output labels y, parameterized by θ, where θ
captures all the learnable parameters of our ML algorithm. Then, we can
write our classification loss function as follows in Equation 1.1:

Equation 1.1 Loss function for classification

Where 1[] is an indicator function, which is equal to 1 when the argument

inside is true and 0 otherwise. What the expression above says is we are
adding up all the instances in which our prediction f(xi; θ) did not match
the ground truth label yi and we are dividing by the total number of examples
n. In other words, we are computing an average misclassification rate. Our
goal is to minimize the loss function, i.e. find a set of parameters θ, that make
the misclassification rate as close to zero as possible. Note that there are other
alternative loss functions for classification such as cross entropy that we will
look into in later chapters.

For continuous labels or response variables, a common loss function is the

Mean Square Error (MSE), defined as follows in Equation 1.2:

Equation 1.2 Loss function for regression

Essentially, we are subtracting our prediction from the ground truth label,
squaring it and aggregating the result as an average over all data points. By
taking the square we are eliminating the possibility of negative loss values,
which would impact our summation.

One of the central goals of machine learning is to be able to generalize to

unseen examples. We want to achieve high accuracy (low loss) on not just
the training data (which is already labelled) but on new, unseen, test data
examples. This generalization ability is what makes machine learning so
attractive: if we can design ML algorithms that can see outside their training
box, we’ll be one step closer to Artificial General Intelligence (AGI).

In unsupervised learning, we are not given the label y nor are we learning
the mapping between input and output examples, instead we are interested in
making sense of the data itself. Usually, that implies in a lower dimension
and with some properties so it could be easier to understand by human. In
other words, our training dataset consists of D = {x1,…,xn} of n input
examples without any corresponding labels y. The simplest example of
unsupervised learning is finding clusters within data. Intuitively, data points
that belong to the same cluster have similar characteristics. In fact, data
points within a cluster can be represented by the cluster center as an exemplar
and used as part of a data compression algorithm. Alternatively, we can look
at the distances between clusters in a projected lower-dimensional space to
understand the inter-relation between different groups. Additionally, a point
that’s far away from all the existing clusters can be considered an anomaly
leading to an anomaly detection algorithm. As you can see, there’s an infinite
number of interesting uses cases that arise from unsupervised learning, and
we’ll be learning from scratch some of the most intriguing algorithms in that
space.

Figure 1.2 Unsupervised Learning: clusters of data points projected onto 2-dimensional space
Another very important area of modern machine algorithms is deep learning.
The name comes from a stack of computational layers forming together a
computational graph. The depth of this graph refers to sequential computation
and the breadth to parallel computation.

As we’ll see, deep learning models gradually refine their parameters through
back-propagation algorithm until they meet the objective function. Deep
learning models permeated the industry due to their ability to solve complex
problems with high accuracy. For example, Figure 1.3 shows a deep learning
architecture for sentiment analysis. We’ll learn more about what individual
blocks represent in future chapters.

Figure 1.3 Deep Neural Network (DNN) architecture for sentiment analysis
Deep learning is a very active research area and we’ll be focusing on the
modern deep learning algorithms throughout this book. For example, in self-
supervised learning, used in Transformer models, we are using the context
and structure of the natural language as a supervisory signal thereby
extracting the labels from the data itself. In addition to classic applications of
deep learning in Natural Language Processing (NLP) and Computer Vision
(CV), we’ll be taking a look at generative models, learning how to predict
time-series data and journey into relational graph data.

1.2 Why Learn Algorithms from Scratch?

Understanding ML algorithms from scratch has a number of valuable
outcomes for the reader. First, you will be able to choose the right algorithm
for the task. By knowing the innerworkings of the algorithm, you will
understand its shortcomings, assumptions made in the derivation of the
algorithm as well as advantages in different data scenarios. This will enable
you to exercise judgement when selecting the right solution to a problem and
save time by eliminating approaches that don’t work.

Secondly, you will be able to explain the results of the given algorithm to the
stakeholders. Being able to interpret the results and present them to the
audience in the industrial or academic settings is an important trait of ML
algorithm designer.

Thirdly, you will be able to use intuition developed by reading this book to
troubleshoot advanced ML problems. Breaking down a complex problem into
smaller pieces and understanding where things went wrong often requires a
strong sense of fundamentals and algorithmic intuition. This book will allow
the reader to construct minimum working examples and build upon existing
algorithms to develop and be able to debug more complex models.

Fourthly, you will be able to extend an algorithm when a new situation arises
in the real world, in particular, where the textbook algorithm or a library
cannot be used as is. The in-depth understanding of ML algorithms that you
will acquire in this book will help you modify existing algorithms to meet
your needs.

Finally, we are often interested in improving performance of existing models.

The principles discussed in this book will enable the reader to accomplish
that. In conclusion, understanding ML algorithms from scratch will help you
choose the right algorithm for the task, explain the results, troubleshoot
advanced problems, extend an algorithm to a new situation and improve
performance of existing algorithms.

1.3 Bayesian Inference and Deep Learning

Bayesian inference allows us to update our beliefs about the world given
observed data. Our minds hold a variety of mental models explaining
different aspects of the world, and by observing new data points, we can
update our latent representation and improve our understanding of reality.
Any probabilistic model is described by a set of parameters θ modelled as
random variables, which control the behavior of the model, and associated
data x.

The goal of Bayesian inference is to find the posterior distribution p(θ|x) in

order to capture well a particular aspect of reality. The posterior distribution
is proportional to the product of the likelihood p(x|θ) and the prior p(θ),
which follows from the Bayes rule in Equation 1.3:

Equation 1.3 Bayes Rule

Prior p(θ) is our initial belief and can be either non-informative (e.g. uniform
over all possible states) or informative (e.g. based on experience in a
particular domain). Moreover, our inference results depend on the prior we
choose: not only the value of prior parameters but also on the functional form
of the prior. We can imagine a chain of updates in which the prior becomes a
posterior as more data is obtained in a form of Bayes engine shown in Figure
1.4. We can see how our prior is updated to a posterior via Bayes rule as we
observe more data.

Figure 1.4 Bayes engine showing the transformation of a prior to a posterior as more data is
observed
Posteriors that have the same form as the prior are known as conjugate
priors, which are historically preferred since they simplify computation by
having closed form updates. The denominator Z=p(x)=∫p(x|θ)p(θ)dθ is
known as the normalizing constant or the partition function and is often
intractable to compute due to integration in high dimensional parameter
space. We’ll look at a number of techniques in this book that work around the
problem of estimating Z.

We can model relationships between different random variables in our model

as a graph as shown in Figure 1.5 giving rise to probabilistic graphical
models. Each node in the graph represents a random variable (RV) and each
edge represents conditional dependency. The topology of the graph itself
changes according to specific application you are trying to model. However,
the goals of Bayesian inference remain the same.

Figure 1.5 Probabilistic Graphical Model (PGM) for a Gaussian Mixture Model
 Another Random Document on
Scribd Without Any Related Topics
medium, by making their experiments in a vacuum. They selected
with great 150 judgment the conditions of their experiments and
comparisons, making one quantity vary while the others remained
constant. In this manner they found, that the quickness of cooling for
a constant excess of temperature, increases in geometrical
progression, when the temperature of the surrounding space
increases in arithmetical progression; whereas, according to the
Newtonian law, this quickness would not have varied at all. Again,
this variation being left out of the account, it appeared that the
quickness of cooling, so far as it depends on the excess of
temperature of the hot body, increases as the terms of a geometrical
progression diminished by a constant number, when the temperature
of the hot body increases in arithmetical progression. These two
laws, with the coefficients requisite for their application to particular
substances, fully determine the conditions of cooling in a vacuum.

Starting from this determination, MM. Dulong and Petit proceeded

to ascertain the effect of the medium, in which the hot body is
placed, upon its rate of cooling; for this effect became a residual
phenomenon, 20 when the cooling in the vacuum was taken away.
We shall not here follow this train of research; but we may briefly
state, that they were led to such laws as this;—that the rapidity of
cooling due to any gaseous medium in which the body is placed, is
the same, so long as the excess of the body’s temperature is the
same, although the temperature itself vary;—that the cooling power
of a gas varies with the elasticity, according to a determined law; and
other similar rules.
20 See Phil. Ind. Sciences, B. xiii. c. 7, Sect. iv.
 In reference to the process of their induction, it is worthy of notice,
that they founded their reasonings upon Prevost’s law of exchanges;
and that, in this way, the second of their laws above stated,
respecting the quickness of cooling, was a mathematical
consequence of the first. It may be observed also, that their
temperatures are measured by means of the air-thermometer, and
that if they were estimated on another scale, the remarkable
simplicity and symmetry of their results would disappear. This is a
strong argument for believing such a measure of temperature to
have a natural prerogative of simplicity. This belief is confirmed by
other considerations; but these, depending on the laws of expansion
by heat, cannot be here referred to; and we must proceed to finish
our survey of the mathematical theory of heat, as founded on the
phenomena of radiation and conduction, which alone have as yet
been traced up to general principles.

We may observe, before we quit this subject, that this correction of

151 Newton’s law will materially affect the mathematical calculations
on the subject, which were made to depend on that law both by
Fourier, Laplace, and Poisson. Probably, however, the general
features of the results will be the same as on the old supposition. M.
Libri, an Italian mathematician, has undertaken one of the problems
of this kind, that of the armil, with Dulong and Petit’s law for his
basis, in a Memoir read to the Institute of France in 1825, and since
published at Florence. 21
21 Mém. de Math. et de Phys. 1829.

Sect. 6.—Other Laws of Phenomena with respect to Radiation.

The laws of radiation as depending upon the surface of radiating
bodies, and as affecting screens of various kinds interposed
between the hot body and the thermometer, were examined by
several inquirers. I shall not attempt to give an account of the latter
course of research, and of the different laws which luminous and
non-luminous heat have been found to follow in reference to bodies,
whether transparent or opaque, which intercept them. But there are
two or three laws of the phenomena, depending upon the effects of
the surfaces of bodies, which are important.

1. In the first place, the powers of bodies to emit and to absorb

heat, as far as depends upon their surface, appear to be in the same
proportion. If we blacken the surface of a canister of hot water, it
radiates heat more copiously; and in the same measure, it is more
readily heated by radiation.

2. In the next place, as the radiative power increases, the power of

reflection diminishes, and the contrary. A bright metal vessel reflects
much heat; on this very account it does not emit much; and hence a
hot fluid which such a vessel contains, remains hot longer than it
does in an unpolished case.

3. The heat is emitted from every point of the surface of a hot body
in all directions; but by no means in all directions with equal intensity.
The intensity of the heating ray is as the sine of the angle which it
makes with the surface.

The last law is entirely, the two former in a great measure, due to
the researches of Leslie, whose Experimental Inquiry into the Nature
and Propagation of Heat, published in 1804, contains a great
number of curious and striking results and speculations. The laws
now just 152 stated bear, in a very important manner, upon the
formation of the theory; and we must now proceed to consider what
appears to have been done in this respect; taking into account, it
must still be borne in mind, only the phenomena of conduction and
radiation.

Sect. 7.—Fourier’s Theory of Radiant Heat.

The above laws of phenomena being established, it was natural that

philosophers should seek to acquire some conception of the physical
action by which they might account, both for these laws, and for the
general fundamental facts of Thermotics; as, for instance, the fact
that all bodies placed in an inclosed space assume, in time, the
temperature of the inclosure. Fourier’s explanation of this class of
phenomena must be considered as happy and successful; for he has
shown that the supposition to which we are led by the most simple
and general of the facts, will explain, moreover, the less obvious
laws. It is an obvious and general fact, that bodies which are
included in the space tend to acquire the same temperature. And this
identity of temperature of neighboring bodies requires an hypothesis,
which, it is found, also accounts for Leslie’s law of the sine, in
radiation.

This hypothesis is, that the radiation takes place, not from the
surface alone of the hot body, but from all particles situated within a
certain small depth of the surface. It is easy to see 22 that, on this
supposition, a ray emitted obliquely from an internal particle, will be
less intense than one sent forth perpendicular to the surface,
because the former will be intercepted in a greater degree, having a
greater length of path within the body; and Fourier shows, that
whatever be the law of this intercepting power, the result will be, that
the radiative intensity is as the sine of the angle made by the ray with
the surface.
22 Mém. Inst. t. v. 1821, p. 204.

But this law is, as I have said, likewise necessary, in order that
neighboring bodies may tend to assume the same temperature: for
instance, in order that a small particle placed within a spherical shell,
should finally assume the temperature of the shell. If the law of the
sines did not obtain, the final temperature of such a particle would
depend upon its place in the inclosure; 23 and within a shell of ice we
should have, at certain points, the temperature of boiling water and
of melting iron.
23 An. Chim. iv. 1817, p. 129.

This proposition may at first appear strange and unlikely; but it

may 153 be shown to be a necessary consequence of the assumed
principle, by very simple reasoning, which I shall give in a general
form in a Note. 24
24 The following reasoning may show the connexion of the law of
the sines in radiant heat with the general principle of ultimate
identity of neighboring temperatures. The equilibrium and identity
of temperature between an including shell and an included body,
cannot obtain upon the whole, except it obtain between each pair
of parts of the two surfaces of the body and of the shell; that is,
any part of the one surface, in its exchanges with any part of the
other surface, must give and receive the same quantity of heat.
Now the quantity exchanged, so far as it depends on the receiving
surface, will, by geometry, be proportional to the sine of the
obliquity of that surface: and as, in the exchanges, each may be
considered as receiving, the quantity transferred must be
proportional to the sines of the two obliquities; that is, to that of
the giving as well as of the receiving surface.
 Nor is this conclusion disturbed by the consideration, that all the
rays of heat which fall upon a surface are not absorbed, some
being reflected according to the nature of the surface. For, by the
other above-mentioned laws of phenomena, we know that, in the
same measure in which the surface loses the power of admitting,
it loses the power of emitting, heat; and the superficial parts gain,
by absorbing their own radiation, as much as they lose by not
absorbing the incident heat; so that the result of the preceding
reasoning remains unaltered.

This reasoning is capable of being presented in a manner quite

satisfactory, by the use of mathematical symbols, and proves that
Leslie’s law of the sines is rigorously and mathematically true on
Fourier’s hypothesis. And thus Fourier’s theory of molecular extra-
radiation acquires great consistency.

Sect. 8.—Discovery of the Polarization of Heat.

The laws of which the discovery is stated in the preceding Sections

of this Chapter, and the explanations given of them by the theories of
conduction and radiation, all tended to make the conception of a
material heat, or caloric, communicated by an actual flow and
emission, familiar to men’s minds; and, till lately, had led the greater
part of thermotical philosophers to entertain such a view, as the most
probable opinion concerning the nature of heat. But some steps
have recently been made in thermotics, which appear to be likely to
overturn this belief, and to make the doctrine of emission as
untenable with regard to heat, as it had been found to be with regard
to light. I speak of the discovery of the polarization of heat. It being
ascertained that rays of heat are polarized in the same manner as
rays of 154 light, we cannot retain the doctrine that heat radiates by
the emanation of material particles, without supposing those
particles of caloric to have poles; an hypothesis which probably no
one would embrace; for, besides that the ill fortune which attended
that hypothesis in the case of light must deter speculators from it, the
intimate connexion of heat and light would hardly allow us to
suppose polarization in the two cases to be produced by two
different kinds of machinery.

But, without here tracing further the influence which the

polarization of heat must exercise upon the formation of our theories
of heat, we must briefly notice this important discovery, as a law of
phenomena.

The analogies and connexions between light and heat are so

strong, that when the polarization of light had been discovered, men
were naturally led to endeavor to ascertain whether heat possessed
any corresponding property. But partly from the difficulty of obtaining
any considerable effect of heat separated from light, and partly from
the want of a thermometrical apparatus sufficiently delicate, these
attempts led, for some time, to no decisive result. M. Berard took up
the subject in 1813. He used Malus’s apparatus, and conceived that
he found heat to be polarized by reflection at the surface of glass, in
the same manner as light, and with the same circumstances. 25 But
when Professor Powell, of Oxford, a few years later (1830), repeated
these experiments with a similar apparatus, he found 26 that though
the heat which is conveyed along with light is, of course, polarizable,
“simple radiant heat,” as he terms it, did not offer the smallest
difference in the two rectangular azimuths of the second glass, and
thus showed no trace of polarization.
25 Ann. Chim. March, 1813.

26 Edin. Journ. of Science, 1830, vol. ii. p. 303.

 Thus, with the old thermometers, the point remained doubtful. But
soon after this time, MM. Melloni and Nobili invented an apparatus,
depending on certain galvanic laws, of which we shall have to speak
hereafter, which they called a thermomultiplier; and which was much
more sensitive to changes of temperature than any previously-known
instrument. Yet even with this instrument, M. Melloni failed; and did
not, at first, detect any perceptible polarization of heat by the
tourmaline; 27 nor did M. Nobili, 28 in repeating M. Berard’s
experiment. But in this experiment the attempt was made to polarize
heat by reflection from glass, as light is polarized: and the quantity
155 reflected is so small that the inevitable errors might completely
disguise the whole difference in the two opposite positions. When
Prof. Forbes, of Edinburgh, (in 1834,) employed mica in the like
experiments, he found a very decided polarizing effect; first, when
the heat was transmitted through several films of mica at a certain
angle, and afterwards, when it was reflected from them. In this case,
he found that with non-luminous heat, and even with the heat of
water below the boiling point, the difference of the heating power in
the two positions of opposite polarity (parallel and crossed) was
manifest. He also detected by careful experiments, 29 the polarizing
effect of tourmaline. This important discovery was soon confirmed by
M. Melloni. Doubts were suggested whether the different effect in the
opposite positions might not be due to other circumstances; but
Professor Forbes easily showed that these suppositions were
inadmissible; and the property of a difference of sides, which at first
seemed so strange when ascribed to the rays of light, also belongs,
it seems to be proved, to the rays of heat. Professor Forbes also
found, by interposing a plate of mica to intercept the ray of heat in an
intermediate point, an effect was produced in certain positions of the
mica analogous to what was called depolarization in the case of
light; namely, a partial destruction of the differences which
polarization establishes.
27 Ann. de Chimie, vol. lv.

28 Bibliothèque Universelle.

29
Ed. R. S. Transactions, vol. xiv.; and Phil. Mag. 1835, vol. v. p.
209. Ib. vol. vii. p. 349.

Before this discovery, M. Melloni had already proved by

experiment that heat is refracted by transparent substances as light
is. In the case of light, the depolarizing effect was afterwards found
to be really, as we have seen, a dipolarizing effect, the ray being
divided into two rays by double refraction. We are naturally much
tempted to put the same interpretation upon the dipolarizing effect in
the case of heat; but perhaps the assertion of the analogy between
light and heat to this extent is as yet insecure.

It is the more necessary to be cautious in our attempt to identify

the laws of light and heat, inasmuch as along with all the
resemblances of the two agents, there are very important
differences. The power of transmitting light, the diaphaneity of
bodies, is very distinct from their power of transmitting heat, which
has been called diathermancy by M. Melloni. Thus both a plate of
alum and a plate of rock-salt transmit nearly the whole light; but
while the first stops nearly the whole heat, the second stops very
little of it; and a plate of opake 156 quartz, nearly impenetrable by
light, allows a large portion of the heat to pass. By passing the rays
through various media, the heat may be, as it were, sifted from the
light which accompanies it.
 [2nd Ed.] [The diathermancy of bodies is distinct from their
diaphaneity, in so far that the same bodies do not exercise the same
powers of selection and suppression of certain rays on heat and on
light; but it appears to be proved by the investigations of modern
thermotical philosophers (MM. De la Roche, Powell, Melloni, and
Forbes), that there is a close analogy between the absorption of
certain colors by transparent bodies, and the absorption of certain
kinds of heat by diathermanous bodies. Dark sources of heat emit
rays which are analogous to blue and violet rays of light; and highly
luminous sources emit rays which are analogous to red rays. And by
measuring the angle of total reflection for heat of different kinds, it
has been shown that the former kind of calorific rays are really less
refrangible than the latter. 30
30 See Prof. Forbes’s Third Series of Researches on Heat, Edinb.
R.S. Trans. vol. xiv.

M. Melloni has assumed this analogy as so completely

established, that he has proposed for this part of thermotics the
name Thermochroology (Qu. Chromothermotics?); and along with
this term, many others derived from the Greek, and founded on the
same analogy. If it should appear, in the work which he proposes to
publish on this subject, that the doctrines which he has to state
cannot easily be made intelligible without the use of the terms he
suggests, his nomenclature will obtain currency; but so large a mass
of etymological innovations is in general to be avoided in scientific
works.

M. Melloni’s discovery of the extraordinary power of rock-salt to

transmit heat, and Professor Forbes’s discovery of the extraordinary
power of mica to polarize and depolarize heat, have supplied
thermotical inquirers with two new and most valuable instruments. 31 ]
 31 For an account of many thermotical researches, which I have
been obliged to pass unnoticed here, see two Reports by Prof.
Powell on the present state of our knowledge respecting Radiant
Heat, in the Reports of the British Association for 1832 and 1840.

Moreover, besides the laws of conduction and radiation, many

other laws of the phenomena of heat have been discovered by
philosophers; and these must be taken into account in judging any
theory of heat. To these other laws we must now turn our attention.
157
 CHAPTER II.

The Laws of Changes occasioned by Heat.

Sect. 1.—Expansion by Heat.—The Law of Dalton and Gay-Lussac

for Gases.

A LMOST all bodies expand by heat; solids, as metals, in a small

degree; fluids, as water, oil, alcohol, mercury, in a greater
degree. This was one of the facts first examined by those who
studied the nature of heat, because this property was used for the
measure of heat. In the Philosophy of the Inductive Sciences, Book
iv., Chap. iv., I have stated that secondary qualities, such as Heat,
must be measured by their effects: and in Sect. 4 of that Chapter I
have given an account of the successive attempts which have been
made to obtain measures of heat. I have there also spoken of the
results which were obtained by comparing the rate at which the
expansion of different substances went on, under the same degrees
of heat; or as it was called, the different thermometrical march of
each substance. Mercury appears to be the liquid which is most
uniform in its thermometrical march; and it has been taken as the
most common material of our thermometers; but the expansion of
mercury is not proportional to the heat. De Luc was led, by his
experiments, to conclude “that the dilatations of mercury follow an
accelerated march for equal augmentations of heat.” Dalton
conjectured that water and mercury both expand as the square of
the real temperature from the point of greatest contraction: the real
temperature being measured so as to lead to such a result. But none
of the rules thus laid down for the expansion of solids and fluids
appear to have led, as yet, to any certain general laws.

With regard to gases, thermotical inquirers have been more

successful. Gases expand by heat; and their expansion is governed
by a law which applies alike to all degrees of heat, and to all
gaseous fluids. The law is this: that for equal increments of
temperature they expand by the same fraction of their own bulk;
which fraction is three-eights 158 in proceeding from freezing to
boiling water. This law was discovered by Dalton and M. Gay-Lussac
independently of each other; 32 and is usually called by both their
names, the law of Dalton and Gay-Lussac. The latter says, 33 “The
experiments which I have described, and which have been made
with great care, prove incontestably that oxygen, hydrogen, azotic
acid, nitrous acid, ammoniacal acid, muriatic acid, sulphurous acid,
carbonic acid, gases, expand equally by equal increments of heat.”
“Therefore,” he adds with a proper inductive generalization, “the
result does not depend upon physical properties, and I collect that all
gases expand equally by heat.” He then extends this to vapors, as
ether. This must be one of the most important foundation-stones of
any sound theory of heat.
32 Manch. Mem. vol. v. 1802; and Ann. Chim. xliii. p. 137.

33 Ib. p. 272.

[2nd Ed.] Yet MM. Magnus and Regnault conceive that they have
overthrown this law of Dalton and Gay-Lussac, and shown that the
different gases do not expand alike for the same increment of heat.
Magnus found the ratio to be for atmospheric air, 1∙366; for
hydrogen, 1∙365; for carbonic acid, 1∙369; for sulphurous-acid gas,
1∙385. But these differences are not greater than the differences
obtained for the same substances by different observers; and as this
law is referred to in Laplace’s hypothesis, hereafter to be discussed,
I do not treat the law as disproved.

Yet that the rate of expansion of gas in certain circumstances is

different for different substances, must be deemed very probable,
after Dr. Faraday’s recent investigations On the Liquefaction and
Solidification of Bodies generally existing as Gases, 34 by which it
appears that the elasticity of vapors in contact with their fluids
increases at different rates in different substances. “That the force,”
he says, “of vapor increases in a geometrical ratio for equal
increments of heat is true for all bodies, but the ratio is not the same
for all. . . . For an increase of pressure from two to six atmospheres,
the following number of degrees require to be added to the bodies
named:—water 69°, sulphureous acid 63°, cyanogen 64°∙5,
ammonia 60°, arseniuretted hydrogen 54°, sulphuretted hydrogen
56°∙5, muriatic acid 43°, carbonic acid 32°∙5, nitrous oxide 30°.”]
34 Phil. Trans. 1845, Pt. 1.

We have already seen that the opinion that the air-thermometer is

a true measure of heat, is strongly countenanced by the symmetry
which, by using it, we introduce into the laws of radiation. If we 159
accept the law of Dalton and Gay-Lussac, it follows that this result is
independent of any peculiar properties in the air employed; and thus
this measure has an additional character of generality and simplicity
which make it still more probable that it is the true standard. This
opinion is further supported by the attempts to include such facts in a
theory; but before we can treat of such theories, we must speak of
some other doctrines which have been introduced.
 Sect. 2.—Specific Heat.—Change of Consistence.

In the attempts to obtain measures of heat, it was found that bodies

had different capacities for heat; for the same quantity of heat,
however measured, would raise, in different degrees, the
temperature of different substances. The notion of different
capacities for heat was thus introduced, and each body was thus
assumed to have a specific capacity for heat, according to the
quantity of heat which it required to raise it through a given scale of
heat. 35 The term “capacity for heat” was introduced by Dr. Irvine, a
pupil of Dr. Black. For this term, Wilcke, the Swedish physicist,
substituted “specific heat;” in analogy with “specific gravity.”
35 See Crawford, On Heat, for the History of Specific Heat.

It was found, also, that the capacity of the same substance was
different in the same substance at different temperatures. It appears
from experiments of MM. Dulong and Petit, that, in general, the
capacity of liquids and solids increases as we ascend in the scale of
temperature.

But one of the most important thermotic facts is, that by the
sudden contraction of any mass, its temperature is increased. This is
peculiarly observable in gases, as, for example, common air. The
amount of the increase of temperature by sudden condensation, or
of the cold produced by sudden rarefaction, is an important datum,
determining the velocity of sound, as we have already seen, and
affecting many points of meteorology. The coefficient which enters
the calculation in the former case depends on the ratio of two
specific heats of air under different conditions; one belonging to it
when, varying in density, the pressure is constant by which the air is
contained; the other, when, varying in density, it is contained in a
constant space.

A leading fact, also, with regard to the operation of heat on bodies

160 is, that it changes their form, as it is often called, that is, their
condition as solid, liquid, or air. Since the term “form” is employed in
too many and various senses to be immediately understood when it
is intended to convey this peculiar meaning, I shall use, instead of it,
the term consistence, and shall hope to be excused, even when I
apply this word to gases, though I must acknowledge such
phraseology to be unusual. Thus there is a change of consistence
when solids become liquid, or liquids gaseous; and the laws of such
changes must be fundamental facts of our thermotical theories. We
are still in the dark as to many of the laws which belong to this
change; but one of them, of great importance, has been discovered,
and to that we must now proceed.

Sect. 3.—The Doctrine of Latent Heat.

The Doctrine of Latent Heat refers to such changes of consistence

as we have just spoken of. It is to this effect; that during the
conversion of solids into liquids, or of liquids into vapors, there is
communicated to the body heat which is not indicated by the
thermometer. The heat is absorbed, or becomes latent; and, on the
other hand, on the condensation of the vapor to a liquid, or the liquid
to a solid consistency, this heat is again given out and becomes
sensible. Thus a pound of ice requires twenty times as long a time,
in a warm room, to raise its temperature seven degrees, as a pound
of ice-cold water does. A kettle placed on a fire, in four minutes had
its temperature raised to the boiling point, 212°: and this temperature
continued stationary for twenty minutes, when the whole was boiled
away. Dr. Black inferred from these facts that a large quantity of heat
is absorbed by the ice in becoming water, and by the water in
becoming steam. He reckoned from the above experiments, that ice,
in melting, absorbs as much heat as would raise ice-cold water
through 140° of temperature: and that water, in evaporating, absorbs
as much heat as would raise it through 940°.

That snow requires a great quantity of heat to melt it; that water
requires a great quantity of heat to convert it into steam; and that this
heat is not indicated by a rise in the thermometer, are facts which it
is not difficult to observe; but to separate these from all extraneous
conditions, to group the cases together, and to seize upon the
general law by which they are connected, was an effort of inductive
insight, which has been considered, and deservedly, as one of the
most striking 161 events in the modern history of physics. Of this step
the principal merit appears to belong to Black.

[2nd Ed.] [In the first edition I had mentioned the names of De Luc
and of Wilcke, in connexion with the discovery of Latent Heat, along
with the name of Black. De Luc had observed, in 1755, that ice, in
melting, did not rise above the freezing-point of temperature till the
whole was melted. De Luc has been charged with plagiarizing
Black’s discovery, but, I think, without any just ground. In his Idées
sur la Météorologique (1787), he spoke of Dr. Black as “the first who
had attempted the determinations of the quantities of latent heat.”
And when Mr. Watt pointed out to him that from this expression it
might be supposed that Black had not discovered the fact itself, he
acquiesced, and redressed the equivocal expression in an Appendix
to the volume. 36
 36 See his Letter to the Editors of the Edinburgh Review, No. xii.
p. 502, of the Review.

Black never published his own account of the doctrine of Latent

Heat: but he delivered it every year after 1760 in his Lectures. In
1770, a surreptitious publication of his Lectures was made by a
London bookseller, and this gave a view of the leading points of Dr.
Black’s doctrine. In 1772, Wilcke, of Stockholm, read a paper to the
Royal Society of that city, in which the absorption of heat by melting
ice is described; and in the same year, De Luc of Geneva published
his Recherches sur les Modifications de l’Atmosphère, which has
been alleged to contain the doctrine of latent heat, and which the
author asserts to have been written in ignorance of what Black had
done. At a later period, De Luc, adopting, in part. Black’s expression,
gave the name of latent fire to the heat absorbed. 37
37 See Ed. Rev. No. vi. p. 20.

It appears that Cavendish determined the amount of heat

produced by condensing steam, and by thawing snow, as early as
1765. He had perhaps already heard something of Black’s
investigations, but did not accept his term “latent heat”. 38 ]
38 See Mr. V. Harcourt’s Address to the Brit. Assoc. in 1839, and
the Appendix.

The consequences of Black’s principle are very important, for

upon it is founded the whole doctrine of evaporation; besides which,
the principle of latent heat has other applications. But the relations of
aqueous vapor to air are so important, and have been so long a 162
subject of speculation, that we may with advantage dwell a little upon
them. The part of science in which this is done may be called, as we
have said, Atmology; and to that division of Thermotics the following
chapters belong.
ATMOLOGY.