International Series in | Pure and Applied Mathematics - Willam Ted Martin and E H Spanier, Consulling Editors AHLFORS Complex Analysis BELLMAN Stability Theory of Differential Equations BUCK Advanced Calculus BUSACKER AND SAATY Finite Graphs and Networks CHENEY Introduction to Approximation Theory CODDINGTON AND LEVINSON Theory of Ordinary Differential Equations COHN Conformal Mapping on Riemann Surfaces DENNEMEYER Introduction to Partial Differential Equations and Boundary Value Problems DETTMAN Mathematicat Methods in Physics and Engineering EPSTEIN Partial Differential Equations GOLOMB AND SHANKS Elements of Ordinary Differential Equations GRAVES The Theory of Functions of Real Variables GREENSPAN [Introduction to Partial Differential Equations GRIFFIN Elementary Theory of Numbers HAMMING Numerical Methods for Scientrsts and Engineers HILDEBRAND introduction to Numerical Analysts HOUSEHOLDER Principles of Numerical Analysis LASS Elements of Pure and Apphed Mathematics LASS Vector and Tensor Analysts LEPAGE Complex Variables and the Laplace Transform for Engineers McCARTY Topology An Introduction with Applications to Topological Groups MOORE Elements of Linear Algebra and Matrix Theory MOURSUND AND DURIS: Elementary Theory and Application of Numerical Analysis NEF Linear Algebra NEHARI Conformal Mapping NEWELL Vector Analysts RALSTON $A First Course in Numerical Analysis RITGER AND ROSE Differential Equations with Applications ROSSER Logic for Mathematicians RUDIN Principles of Mathematical Analysis SAATY AND BRAM Nonlinear Mathematics SIMMONS Introduction to Topology and Modern Analysis SNEDDON Elements of Partial Differential Equations SNEDDON = Fourter Transforms STOLL Linear Algebra and Matrix Theory STRUBLE Nontinear Differential Equations WEINSTOCK Calculus of Variations WEISS Algebraic Number Theory ZEMANIAN Distribution Theory and Transtorm AnalystsELEMENTS OF LINEAR ALGEBRA AND MATRIX THEORY JOHN T. MOORE The Universily of Florida The University of Western Ontario McGRAW-HILL BOOK COMPANY New York St. Louis San Francisco Toronto London SydneyELEMENTS OF LINEAR ALGEBRA AND MATRIX THEORY Copyright © 1968 by McGraw Hill Inc All Rights Reserved Printed in the United States of America No part of thts publication may be reproduced, stored in a retrieval system, or transmutted, in any form or by any means, electronic mechanical, photocopying, recording or otherwise without the Prior written permisston of the publisher Library of Congress Catalog Card Number 68 11933 42885 234567890 MAMM 754321069PREFACE This book is an outgrowth of a course in linear algebra which | first gave to a class of undergraduates several years ago. In writing it, | have been moti- vated by the desire to produce a text with the following features: It would be of a size such that a major portion of the contents could be covered in a single academic term; it would have an abundance of problems, with degree of difficulty varying from the routine and near-trivial to the challenging; it would use a notation and terminology as near-standard as possible; and it would present a level of mathematics suitable for students whose major interest is in either pure mathematics or any of the pure or applied sciences. In my desire to limit the size of the book, it is inevitable that | am exposing an —Achilles' heel’ by my choice of material for inclusion. However, it is my belief that in a first course one should avoid being encyclopedic but, rather, help to give some sense of direction to the student; and so ! take my stand on what | have done—but with apologies to those whose opinions may differ from mine. Before beginning any writing of my own, | consulted the bulletins of the Committee on the Undergraduate Program in Mathematics (CUPM) and decided to make the course recommendation contained in one of them the nucleus of this book. However, | have not followed these recommendations _Slavishly but have included additional material and been guided by my own’ viiviii = PREFACE preference with regard to arrangement The book has been structured, for the most part, as a continuous study, tn which each section makes a contribu- tron to the next At the same time, there are a few sections that are not essential to the main development of the text; In each such case this fact has been made clear by an introductory remark. In order to emphasize the key purpose of a section, | have limited the number of results included in any one of them to a bare mimmum. As was stated above, I have tried to use standard symbolism and termi- nology, and so | have made free use of the familiar symbols of set theory To be precise, the following symbols occur without explanation: € for set mem bership, L/ for set theoretic unton; % for set-theoretic intersection, C and > for the two kinds of set inclusion, X for the cartesian product of sets In addition, it has been convenient to refer to the cardinality or cardinal number ofa set in an inturtwe sort of way, and I have used [a,b] to denote the interval of real numbers between aand 6, inclusive Although the book is considered to be self contained, itis assumed that the reader \s familiar with the following basic notions’ the ditston and ged processes as they arise in the theory of polynomuals; the partitioning of a set effected by an equrralence relation, each equtvalence class being representable by a single elernent of the set, and the two methods of proof by mathematical induction. If the reader is not familiar with these several concepts, he is urged to acquaint himself with the unfamihar ones by referring to any book on abstract algebra. The problem sets have been arranged into three groupings, physically separated from one another by spaces The problems tn the first groupings are basic but quite simple, and some instructors may find them unnecessary for their classes. However, although this may be true in some instances, [tis my judgment that a students often lost ata point where he falls tocomprehend something simple—and almost trivial—but also very important. It 1s the purpose of the early problems to bring to light and, hopefully, to elirnmnate any such difficulties. The second groupings of problems are complementary to the material in the text, and all of them should be worked out in detail. tn this way, students may feel more personally involved with the proofs of the theorems than if these proofs were all presented in final polished form. In the final groupings, | have included other problems of a more genera! nature, relevant to the particular section understudy. Otherwise, the problems have not been arranged according to difficulty. [masmuch as the first groupings involve problems of such a bastc nature, answers or hints have been provided for most of them at the back of the book The student 1s thus given an opportunity ta work these problems—if he feels the need of doing do—quite Independently of whether this is suggested by his instructor and so develop an early confidence in his ability to grasp the concepts being presented,PREFACE ix There are many people to whom [ am indebted in the writing of this book. | was first given much encouragement by a number of reviewers of an early portion of the manuscript, all of whom were anonymous except for Prof. George Simmons of Colorado College. To them all | extend my most sincere thanks. Ata later stage, much of the manuscript was read by three people who provided many helpful suggestions for its improvement. They are Prof. C. Desoer of the University of California, Berkeley; Prof. D, J. Lewis of the University of Michigan; and Prof. T. L. Wade of Florida State University. 1am sincerely grateful to them for this assistance. The manuscriptin its near-final form was read by Prof. D. J. Sterling of Bowdoin College, and | wish to express my gratitude to him for his very useful suggestions; | am confident that the book is better because of his pertinent commentary. It is understood, of course, that even though these people have seen part or all of the manu- script, their reading of it does not imply any sort of endorsement; any errors or shortcomings in the book are mine alone. My longtime friend, Prof. W. S. Cannon of Presbyterian College, was of very considerable help in collecting Readings from the American Mathematical Monthly. My colleague, Prof. Anne Bode, has been a most gracious consultant on matters which have arisen from time to time, and her generous assistance continued throughout the final stages of the manuscript. Two very good graduate students, Eddy Smet and Jawaid Rizvi, came to my aid at a very busy time and used a portion of a vacation period to supply me with answers for the first groupings of problems. Their further help in reading the galleys wag inestimable, and the book is much Closer to being free.of, errors than it would othérwise have been. 1! offer my most warm and cordial thanks to them all. JOHN T. MOORECONTENTS Preface vii 1 Finite-dimensional Vector Spaces 1 Li 1.2 1.3 14 1.5 1.6 17 1.8 19 Vector Spaces 1 Directed Line Segments as Vectors 7 Geometric Vectors and Coordinate Spaces 16 Subspaces 24 Solutions of Linear Equations: Gauss Reduction 28 Solutions of Linear Equations: Determinants 35 Linear Dependence of Vectors 45 Basis and Dimension 51 Two Important Theorems 57 2 Linear Transformations and Matrices 63 2.1 2.2 2.3 2.4 2.5 2.6 27 2.8 2.9 xi Linear Transformations 63 Properties of Linear Transformations 69 Operations on Linear Transformations 73 Linear Functionals 79 Annihilators 85 Nonsingular Linear Transformations 90 Matrices of Linear Transformations 96 Matrices as Multiplicative Systems 106 Change of Basis 119x1 CONTENTS 210 Rank 127 211 An Important Note of Clarification 134 3 Determinants and Systems of Linear Equations 140 31 Matrices and Linear Systems 140 32 Elementary Matrices and Inverses 146 33 Determinants as Multilinear Functionals 154 34 An Alternative Method for Evaluating a Determinant 160 35 An Introduction to Alternating Multilinear Forms 168 36 Determinants Discovered Anew 173 4 Inner product Spaces 178 41. Inner Productsin Vs 178 42 General Euclidean Space 186 43 The Gram Schmidt Process of Orthogonalization 193 44 Orthogonal Complements 199 45 Orthogonal Transformations 206 46 Linear Functionals and Adjoints 213 47 Inner Products and Positive Operators 221 48 Simple Applications of the Distance Function 229 5 Bilinear and Quadratic Forms 236 51 Brlinear Functions and Forms 236 52 Quadratic Forms 246 53 Diagonal Quadratic Forms under Congruence 252 54 Invarrants of a Symmetric Matrix under Congruence 260 55 Eigenvalues and Elgenvectors 265 56 Orthogonal Reduction of Quadratic Forms 272 6 Similarity and Normal Operators 282 61 The Cayley Hamilton Theorem 282 62 Similarity and Diagonal Matrices 289 63 Complex Vectar Spaces 297 64 The Spectral Theorem 305 65 Invariant Subspaces and Primary Decomposition 313 66 Nilpotent Operators and T cyclic Subspaces 321 67 The Jordan Canonical Form 327 Appendix 336 Selected Readings 340 Glossary of Symbols 341 Answers 342 Index 363ELEMENTS OF LINEAR ALGEBRA AND MATRIX THEORY| FINITE-DIMENSIONAL VECTOR SPACES 1.1 Vector Spaces In this study of linear algebra, it is assumed only that the reader has some familiarity with those ubiquitous mathematical objects known as sels and their mappings. The reader who has escaped these in his mathematical education will find at least some of the related symbols listed at the end of the book. Our basic elements will often be numbers, selected from the sets of natural numbers, integers, rational numbers, real numbers, or complex numbers, designated, respectively, by N, Z,Q, R, and C. In most instances, except in the final chapter, our numbers will be real and so elements of R. We shall assume complete familiarity with all these number systems. Many physical systems occurring in science can be described conveniently in the language of electrical engineering. The central feature of such a system is a ‘black box” to which an inpui is applied and from which an oulpul leaves, the details of the relationship between the input and output depending on the nature of the ‘‘black box” or system. Many of the most interesting and common systems are linear in the following sense: (1) if an input of quantitative measure A yields an output of measure C, the output that results from an input of rA is rC, for any r € R; (2) if, in addition, an input of B units yields an output of D units, an input of A + B will yield an I2 FINITE DIMENSIONAL VECTOR SPACES Input Output —_~—_ eR Fig 2 output of C + D A study of the mathematical abstractions of these linear systems ts the object of a course in {near algebra. = The pure mathematician needs mo such motivation for a study of this sort, but for others it may be encouraging to know that there Is a wealth of applications of this undertaking Inabstracton Moreover, in recent times it has become almost a cliche even in areas of applied science, that the most practical approach to a study of mathematics Is an abstract one Although the exact meaning of linear algebra has varied from time to time, the one just given is probably as comprehensive as ts currently possible = In mathematical terms the inputand output referred to above are sets of vecfors, and the ‘ black box’ 1s a linear fransformation that transforms input vectors into output vectors The tectors are elements of a mathematical system known as a rector space, and so it is a reasonable preview of the content of this book to say that we shall undertake a study of linear syslems through the medium of tector spaces We shall now define this all importantconcept In the context of a real vector space, about to be defined, it is customary to refer to real numbers ie elements of R, as the associated scalars Definition A real vector space V 1s a sel of elements, called vectors, with two operations, called addition (designated by +) and mulliplication by scalars (designated by juxtaposition), such that the following axioms or conditions are salisfied Al With every pair a 8 © V there is associated a unique vector a + 8, called their sum AZ Addition is associative thatis, a + (6 +7) = (a + 8) + 7, for arb trarya fy GV A3_ There extsts a unique (zero) vector 0 with a -+ 0 = @ foranya © V A4 With each a € V there ts associated a unique vector —« (called the addtine mverse of a) such that a + (—a) = 0 As in arithmetic, we shall identify the difference 8 — a, for vectors a and 8, with the sum @ + (—a) AS Addition ts commutatie that is, a +6 =6-+ a, for arbitrary aBEYV M1 With every pair r © Rand a € V theres associated a unique vector ra, called the product of r and a M2 Multiplication by scalars Is associative that ts, r(se) = (rs), for arbitrary r,s & R and any vector a M3 Multiplication by scalars is distributive with respect to vector addi tion thatts, r(a + 8) = re + r8, for arbitrary af & V and anyr GRSEC. 1.1 VECTOR SPACES 3 M4. Multiplication by scalars is distributive with respect to scalar addi- tion: that is, (r + s)a = ra + sa, for arbitrary r,s © R and any vector a. Ms. la = a, for the real number 1 and any a € V. We make no claim that the above axioms are logically independent (for example, see Prob. 23) but present them merely as a convenient characteriza- tion of the mathematical system to be studied. Before giving any examples of vector spaces, it is appropriate that we make a few further comments on their defining properties. (i) The vector space that we have defined is real because the multipliers are real numbers. We refer to these real numbers as the scalars associated with the vector space, but in more general circumstances such scalars or multipliers may be elements of any field F’, and such a vector space is said to be over F. Inasmuch as we are not assuming any knowledge of fields, and since little would be gained from such a generalization, we shall assume thal our vector spaces are real (i.e. over R), except in the final chapter. (ii) Axioms Al to A4 describe V as an additive group, and, if Axiom AS is included, as an additive abelian group. This terminology is convenient and will be familiar to the student of abstract algebra. We shail often refer to this additive structure of V. (iii) The symbol 0 in this text will have many different meanings, but the context should make clear what is meant at any particular occurrence. For example, 0 designates a vector in Axioms A3 and A4, but the valid (Prob. 16a) equality Oa = 0 contains the real number 0 in its left member and the vector 0 in its right member. (iv) We prefer the terminology mulliplication by scalars to the more customary scalar multiplication in order to avoid any possible confusion with another type of product which is variously called a scalar product, dot product, or inner product. This type of product will be introduced in Chap. 4. (v) The student of abstract algebra may be disturbed by our reference to multiplication by scalars as an “‘operation’ in the vector space V, and we have nothing but sympathy for this feeling. indeed, a (binary) operation defined in an algebraic system regularly associates an element of the system with each pair of elements of the same system. n the case of the operation under discussion, however, one member r of the pair (r,a) is from the system of real numbers R, and the other member ais from the space V. This type of “‘operation”’ is more correctly described as an endomorphism of the additive group of V, but it would take us too far astray to make the meaning of this phrase precige. Hence we shall leave the definition of the Operation as given, in the belief that the beginning student will be willing to accept it, in spite of our adversely critical comments, and the more experienced student will under- stand what is meant by this customary terminology. We now turn to some examples of real vector spaces, and of such there is a great abundance. Several of these examples will recur throughout the book,