Optimality Conditions in Convex Optimization A

Finite dimensional View 1st Edition Joydeep

Dutta pdf download

https://ebookgate.com/product/optimality-conditions-in-convex-
optimization-a-finite-dimensional-view-1st-edition-joydeep-dutta/

Get Instant Ebook Downloads – Browse at https://ebookgate.com

Instant digital products (PDF, ePub, MOBI) available
Download now and explore formats that suit you...

Convex Optimization 1st Edition Stephen Boyd

https://ebookgate.com/product/convex-optimization-1st-edition-stephen-
boyd/

ebookgate.com

Convex Optimization in Signal Processing and

Communications 1st Edition Daniel P. Palomar

https://ebookgate.com/product/convex-optimization-in-signal-
processing-and-communications-1st-edition-daniel-p-palomar/

ebookgate.com

Finite Dimensional Algebras and Quantum Groups Bangming

Deng

https://ebookgate.com/product/finite-dimensional-algebras-and-quantum-
groups-bangming-deng/

ebookgate.com

Semidefinite Optimization and Convex Algebraic Geometry

content added 1st Edition Grigoriy Blekherman

https://ebookgate.com/product/semidefinite-optimization-and-convex-
algebraic-geometry-content-added-1st-edition-grigoriy-blekherman/

ebookgate.com
The Finite Element Method for Three Dimensional
Thermomechanical Applications 1st Edition Guido Dhondt

https://ebookgate.com/product/the-finite-element-method-for-three-
dimensional-thermomechanical-applications-1st-edition-guido-dhondt/

ebookgate.com

Cellular Automata A Discrete View of the World Wiley

Series in Discrete Mathematics Optimization 1st Edition
Joel L. Schiff
https://ebookgate.com/product/cellular-automata-a-discrete-view-of-
the-world-wiley-series-in-discrete-mathematics-optimization-1st-
edition-joel-l-schiff/
ebookgate.com

Heterogeneous Networks in LTE Advanced 1st Edition Joydeep

Acharya

https://ebookgate.com/product/heterogeneous-networks-in-lte-
advanced-1st-edition-joydeep-acharya/

ebookgate.com

Adaptationism and Optimality 1st Edition Steven Hecht

Orzack

https://ebookgate.com/product/adaptationism-and-optimality-1st-
edition-steven-hecht-orzack/

ebookgate.com

A First Course in Optimization 1st Edition Charles L.

Byrne

https://ebookgate.com/product/a-first-course-in-optimization-1st-
edition-charles-l-byrne/

ebookgate.com
 OPTIMALITY CONDITIONS 
IN  
CONVEX OPTIMIZATION
A Finite-Dimensional View

© 2012 by Taylor & Francis Group, LLC

K13102_FM.indd 1 9/2/11 11:33 AM

 OPTIMALITY CONDITIONS 
IN  
CONVEX OPTIMIZATION
A Finite-Dimensional View

Anulekha Dhara
Joydeep Dutta

© 2012 by Taylor & Francis Group, LLC

K13102_FM.indd 3 9/2/11 11:33 AM

 CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2012 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business

No claim to original U.S. Government works

Version Date: 20110831

International Standard Book Number-13: 978-1-4398-6823-2 (eBook - PDF)

This book contains information obtained from authentic and highly regarded sources. Reasonable
efforts have been made to publish reliable data and information, but the author and publisher cannot
assume responsibility for the validity of all materials or the consequences of their use. The authors and
publishers have attempted to trace the copyright holders of all material reproduced in this publication
and apologize to copyright holders if permission to publish in this form has not been obtained. If any
copyright material has not been acknowledged please write and let us know so we may rectify in any
future reprint.

Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced,
transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or
hereafter invented, including photocopying, microfilming, and recording, or in any information stor-
age or retrieval system, without written permission from the publishers.

For permission to photocopy or use material electronically from this work, please access www.copy-
right.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222
Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that pro-
vides licenses and registration for a variety of users. For organizations that have been granted a pho-
tocopy license by the CCC, a separate system of payment has been arranged.

Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are
used only for identification and explanation without intent to infringe.
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com

© 2012 by Taylor & Francis Group, LLC

 In memory of
Professor M. C. Puri
and
Professor Alex M. Rubinov

© 2012 by Taylor & Francis Group, LLC

 Contents

List of Figures xi

Symbol Description xiii

Foreword xv

Preface xvii

1 What Is Convex Optimization? 1

1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Basic Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Smooth Convex Optimization . . . . . . . . . . . . . . . . . 15

2 Tools for Convex Optimization 23

2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.2 Convex Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.2.1 Convex Cones . . . . . . . . . . . . . . . . . . . . . . . 39
2.2.2 Hyperplane and Separation Theorems . . . . . . . . . 44
2.2.3 Polar Cones . . . . . . . . . . . . . . . . . . . . . . . . 50
2.2.4 Tangent and Normal Cones . . . . . . . . . . . . . . . 54
2.2.5 Polyhedral Sets . . . . . . . . . . . . . . . . . . . . . . 60
2.3 Convex Functions . . . . . . . . . . . . . . . . . . . . . . . . 62
2.3.1 Sublinear and Support Functions . . . . . . . . . . . . 71
2.3.2 Continuity Property . . . . . . . . . . . . . . . . . . . 75
2.3.3 Differentiability Property . . . . . . . . . . . . . . . . 85
2.4 Subdifferential Calculus . . . . . . . . . . . . . . . . . . . . . 98
2.5 Conjugate Functions . . . . . . . . . . . . . . . . . . . . . . . 111
2.6 ε-Subdifferential . . . . . . . . . . . . . . . . . . . . . . . . . 122
2.7 Epigraphical Properties of Conjugate Functions . . . . . . . 136

3 Basic Optimality Conditions Using the Normal Cone 143

3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
3.2 Slater Constraint Qualification . . . . . . . . . . . . . . . . . 145
3.3 Abadie Constraint Qualification . . . . . . . . . . . . . . . . 154
3.4 Convex Problems with Abstract Constraints . . . . . . . . . 157
3.5 Max-Function Approach . . . . . . . . . . . . . . . . . . . . 159
3.6 Cone-Constrained Convex Programming . . . . . . . . . . . 161

vii

© 2012 by Taylor & Francis Group, LLC

 viii Contents

4 Saddle Points, Optimality, and Duality 169

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
4.2 Basic Saddle Point Theorem . . . . . . . . . . . . . . . . . . 171
4.3 Affine Inequalities and Equalities and Saddle Point Condition 175
4.4 Lagrangian Duality . . . . . . . . . . . . . . . . . . . . . . . 185
4.5 Fenchel Duality . . . . . . . . . . . . . . . . . . . . . . . . . 196
4.6 Equivalence between Lagrangian and Fenchel Duality . . . . 200

5 Enhanced Fritz John Optimality Conditions 207

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
5.2 Enhanced Fritz John Conditions Using the Subdifferential . 208
5.3 Enhanced Fritz John Conditions under Restrictions . . . . . 216
5.4 Enhanced Fritz John Conditions in the Absence of Optimal
Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224
5.5 Enhanced Dual Fritz John Optimality Conditions . . . . . . 235

6 Optimality without Constraint Qualification 243

6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 243
6.2 Geometric Optimality Condition: Smooth Case . . . . . . . . 249
6.3 Geometric Optimality Condition: Nonsmooth Case . . . . . . 255
6.4 Separable Sublinear Case . . . . . . . . . . . . . . . . . . . . 274

7 Sequential Optimality Conditions 281

7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
7.2 Sequential Optimality: Thibault’s Approach . . . . . . . . . 282
7.3 Fenchel Conjugates and Constraint Qualification . . . . . . . 293
7.4 Applications to Bilevel Programming Problems . . . . . . . . 308

8 Representation of the Feasible Set and KKT Conditions 315

8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 315
8.2 Smooth Case . . . . . . . . . . . . . . . . . . . . . . . . . . . 315
8.3 Nonsmooth Case . . . . . . . . . . . . . . . . . . . . . . . . . 320

9 Weak Sharp Minima in Convex Optimization 327

9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 327
9.2 Weak Sharp Minima and Optimality . . . . . . . . . . . . . . 328

10 Approximate Optimality Conditions 337

10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 337
10.2 ε-Subdifferential Approach . . . . . . . . . . . . . . . . . . . 338
10.3 Max-Function Approach . . . . . . . . . . . . . . . . . . . . 342
10.4 ε-Saddle Point Approach . . . . . . . . . . . . . . . . . . . . 345
10.5 Exact Penalization Approach . . . . . . . . . . . . . . . . . . 350
10.6 Ekeland’s Variational Principle Approach . . . . . . . . . . . 355
10.7 Modified ε-KKT Conditions . . . . . . . . . . . . . . . . . . 358
10.8 Duality-Based Approach to ε-Optimality . . . . . . . . . . . 361

© 2012 by Taylor & Francis Group, LLC

 Contents ix

11 Convex Semi-Infinite Optimization 365

11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 365
11.2 Sup-Function Approach . . . . . . . . . . . . . . . . . . . . . 366
11.3 Reduction Approach . . . . . . . . . . . . . . . . . . . . . . . 368
11.4 Lagrangian Regular Point . . . . . . . . . . . . . . . . . . . . 374
11.5 Farkas–Minkowski Linearization . . . . . . . . . . . . . . . . 382
11.6 Noncompact Scenario: An Alternate Approach . . . . . . . . 395

12 Convexity in Nonconvex Optimization 403

12.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 403
12.2 Maximization of a Convex Function . . . . . . . . . . . . . . 403
12.3 Minimization of d.c. Functions . . . . . . . . . . . . . . . . . 408

Bibliography 413

Index 423

© 2012 by Taylor & Francis Group, LLC

 List of Figures

1.1 Lower semicontinuous hull. . . . . . . . . . . . . . . . . . . . 11

1.2 Graph of a real-valued differentiable convex function. . . . . . 16
1.3 Local minimizer is global minimizer. . . . . . . . . . . . . . . 18

2.1 Convex and nonconvex sets. . . . . . . . . . . . . . . . . . . . 24

2.2 F1 , F2 , and F1 ∩ F2 are convex while F1c , F2c , and F1 ∪ F2 are
nonconvex. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.3 Line segment principle. . . . . . . . . . . . . . . . . . . . . . . 33
2.4 Tangent cone. . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
2.5 Normal cone. . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
2.6 Graph and epigraph of convex function. . . . . . . . . . . . . 63
2.7 Epigraphs of improper functions φ1 and φ2 . . . . . . . . . . . 76
2.8 Graph of ∂(|.|). . . . . . . . . . . . . . . . . . . . . . . . . . . 126
2.9 Graph of ∂1 (|.|). . . . . . . . . . . . . . . . . . . . . . . . . . 126

3.1 NC (x̄) is not polyhedral. . . . . . . . . . . . . . . . . . . . . . 153

3.2 C ∩Y. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165

5.1 Pseudonormality. . . . . . . . . . . . . . . . . . . . . . . . . . 222

5.2 Not pseudonormal. . . . . . . . . . . . . . . . . . . . . . . . . 223

9.1 Pictorial representation of Theorem 9.6. . . . . . . . . . . . . 336

xi

© 2012 by Taylor & Francis Group, LLC

 Symbol Description

∅ empty set k.k norm

∞ infinity φ(F ) image space of F under φ
N set of natural numbers gph Φ graph of set-valued map Φ
R real line dom φ effective domain of φ : X
R̄ R ∪ {−∞, +∞} → R̄
Rn n-dimensional Euclidean epi φ epigraph of φ
space lev≤α φ α-lower level set of φ
R+ nonnegative orthant of R δF indicator function to F
Rn+ nonnegative orthant of Rn dF distance function to F
[x, y] closed line segment joining projF (x̄) projection of x̄ to F
x and y σ( . ; F ) support function to F
(x, y) open line segment joining φ∗ conjugate function of φ
x and y Q φ+ (x) max{0, φ(x)}
RI product space I R ∇φ(x̄) derivative or gradient of φ
R[I] {λ ∈ RI : λi 6= 0 for finitely at x̄
many i ∈ I} φ◦ (x̄; d) Clarke directional derivative
[I]
R+ positive cone in R[I] of φ at x̄ in the direction d
supp λ {i ∈ I : λ ∈ R[I] , λi 6= 0} ∂φ
partial derivative of φ with
B open unit ball ∂x
Bδ (x̄) open ball with radius δ > 0 respect to x
and center at x̄ ∂2φ
second-order partial deriva-
cl F closure of F ∂xi ∂xj
co F convex hull of F tive of φ with respect to xi
cl co F closed convex hull of F and xj
af f F affine hull of F ∂φ(x̄) convex subdifferential of φ
int F interior of F at x̄
ri F relative interior of F ∂ǫ φ(x̄) ǫ-subdifferential of φ at x̄
cone F cone generated by F ∂ ◦ φ(x̄) Clarke subdifferential or
F+ positive polar cone of F generalized gradient of φ at
F◦ polar cone of F x̄
x → x̄ x converges to x̄ ∇2 φ(x̄) Hessian of φ at x̄
lim limit Jφ(x̄) Jacobian of φ at x̄
lim inf limit infimum TF (x̄) tangent cone to F at x̄
lim sup limit supremum NF (x̄) normal cone to F at x̄
h., .i inner product Nǫ,F (x̄) ǫ-normal set to F at x̄

xiii

© 2012 by Taylor & Francis Group, LLC

 Foreword

The roots of the mathematical topic of optimization go back to ancient Greece,

when Euclid considered the minimal distance of a point to a line; convex
sets were investigated by Minkowski about a hundred years ago, and fifty
years ago, J.-J. Moreau [87] defined the notion of the subdifferential of a
convex function. In 1970, R.T. Rockafellar wrote his monograph [97] on convex
analysis. Since then, the field of convex optimization and convex analysis has
developed rapidly, a huge number of papers on that topic have been published
in scientific journals and a large number of monographs and textbooks have
been produced. Now, we have a new book at hand and one can ask why read
this book.
A recent topic of research in mathematical optimization is the need to
compute global optima of nonconvex problems. To do that, the problem can
be convexified using the optimal function value of the resulting convex opti-
mization problem as a bound for the problem investigated. Combining this
with an enumeration idea the problem can be solved. The same approach of
convexification plus enumeration can serve as a way to solve mixed-integer
nonlinear optimization problems which is a second challenging problem of re-
cent and future research. Moreover, many practical situations lead directly to
convex programming problems. Hence the need to develop a deep insight into
convex optimization.
The theory of convex differentiable optimization is well established. Every
student will be introduced in basic courses on mathematical optimization to
the Fritz John and Karush–Kuhn–Tucker necessary optimality conditions. For
guaranteeing the Karush–Kuhn–Tucker conditions a constraint qualification
such as the Slater condition is needed. But, in many applications, this condi-
tion is violated. There are a larger number of ways out in such a situation:
Abadie constraint qualification can be supposed, sequential optimality condi-
tions can be used or we can try to filter out full information of (enhanced)
Fritz John necessary optimality conditions. These nonstandard but essential
parts of the theory of convex optimization need to be described in detail and
in close relation to each other.
Nonsmooth analysis (see, for example, Mordukhovich [86]) is a quickly
developing area in mathematical optimization. The initial point of nonsmooth
analysis is convex analysis, but recent developments in nonsmooth analysis are
a good influence on convex analysis.
The aim of this book is to develop deep insight into the theory of convex

xv

© 2012 by Taylor & Francis Group, LLC

 xvi Foreword

optimization, combining very recent ideas of nonsmooth analysis with stan-

dard and nonstandard theoretical results. Lagrange and Fenchel duality use
different tools and can be applied successfully in distinct directions. But in
the end, both are shown to coincide.
If, at an optimal solution, no constraint qualification is satisfied, algorithms
solving the Karush–Kuhn–Tucker conditions cannot be used to compute this
point. And, how to characterize such a point? Roughly speaking one idea is
the existence of a sequence outside of the feasible set with smaller objective
function values converging to that point. These are the enhanced Fritz John
necessary optimality conditions. A second idea is to characterize optimality
via subgradients of the regular Lagrangian function at perturbed points con-
verging to zero. This is the sequential optimality condition. Both optimality
conditions work without constraint qualifications. ε-optimal solutions can be
characterized using ε-subgradients.
One special convex optimization problem is also investigated. This is the
problem to find a best point within the set of optimal solutions of a convex
optimization problem. If the objective function is convex, this is a convex
optimization problem called a simple bilevel programming problem. It is easy
to see that standard regularity conditions are violated at every feasible point.
For this problem, a very general constraint qualification is derived.
A last question is if convexity can successfully be used to investigate non-
convex problems as the maximization of convex functions or the minimization
of a function being the difference of convex functions.
Algorithmic approaches for solving convex optimization problems are not
described in this book. This results in much more space for theoretical proper-
ties. The result is a book illuminating not only the body but also the bounds
and corners of the theory of convex optimization. Many of the results pre-
sented are usually not contained in books on this topic. But, if more and
more (applied) difficult optimization problems need to be solved, we are more
likely be faced with instances where usual approaches fail. Then it is necessary
to search away from standard tools for applicable approaches. I am sure that
this book will be very helpful.
I deeply recommend this book for advanced reading.

Stephan Dempe
Freiberg, Germany

© 2012 by Taylor & Francis Group, LLC

 Preface

This is a book on convex optimization. More precisely it is a book on the re-

cent advances in the theory of optimality conditions for convex optimization.
The question is why should one need an additional book on the subject? How-
ever, possibly the books on convex analysis are much more in number than the
ones on convex optimization. In the books dealing with convex analysis, like
the classic Convex Analysis by Rockafellar [97] or the the more recent Convex
Analysis and Nonlinear Optimization by Borwein and Lewis [17], one would
find convex optimization theory appears as an application to various results
of convex analysis. However, from 1970 until now there has been a growing
body of research in the area of optimality conditions for a convex optimiza-
tion. Many of these results address the question as to what happens when
the Slater condition fails for a convex optimization problem or are there very
general constraint qualification conditions which hold even if the the most pop-
ular ones fail? The books on convex analysis usually do not present results of
this type and thus these results remain largely scattered in the vast literature
on convex optimization. On the other hand, the books dealing with convex
optimization largely focus on algorithms or algorithms and theory associated
with a certain special class of problems like second-order conic programming
or semidedfinite programming. Some recent books like Introductory Lectures
in Convex Optimization by Nesterov [90] or Lectures on Modern Convex Opti-
mization by Ben-Tal and Nemirovskii [8] deal with algorithms and the special
problems, respectively.
This book has a completely different focus. It deals with optimality con-
ditions in convex optimization. It attempts to bring in most of the important
and recent results in this area that are scattered in the literature. However,
we do not ignore the required convex analysis either. We provide a detailed
chapter on the main convex analytic tools and also provide some new results
that have appeared recently in the literature. These results are usually not
found in standard books on convex analysis but they are essential in devel-
oping many of the important results in this book. This book actually began
as a survey paper but then we realized that it has too much material to be
considered as a survey; and then we thought of converting the survey paper
into the form of a monograph.
We would look to thank the many people who encouraged us to write
the book. Professor Stephan Dempe agreed very kindly to write the foreword.
Professor Boris Mordukhovich, Professor Suresh Chandra, Professor Juan En-

xvii

© 2012 by Taylor & Francis Group, LLC

 xviii Preface

rique Martinez-Legaz also encouraged us to go ahead and write the book. We

are indeed grateful to them. We would also like to thank Aastha Sharma of
Taylor & Francis, India, for superb handling of the whole book project and
Shashi Kumar from the helpdesk of Taylor & Francis for helping with the
formatting. We would also like to extend our deepest gratitude to our families
for their support. Joydeep Dutta would like to thank his daughter Naina and
his wife Lalty for their understanding and patience during the time this book
was written. Anulekha Dhara would like to express her deepest and sincer-
est regards and gratitude to her parents Dr. Madhu Sudan Dhara and Dolly
Dhara for their understanding and support. She would also like to thank the
National Board for Higher Mathematics, Mumbai, India, for providing finan-
cial support during her tenure at the Indian Institute of Technology Kanpur,
India.
The book is intended for research mathematicians in convex optimization
and also for graduate students in the area of optimization theory. This could be
of interest also to the practitioner who might be interested in the development
of the theory. We have tried our best to make the book free of errors. But to
err is human, so we take the responsibility for any errors the readers might
find in the book. We would also like to request that readers communicate
with us by email at the address: [email protected]. We sincerely hope that the
young researchers in the field of optimization will find this book helpful.

Anulekha Dhara
Avignon, France

Joydeep Dutta
Kanpur, India

© 2012 by Taylor & Francis Group, LLC

 Chapter 1
What Is Convex Optimization?

1.1 Introduction
Optimization is the heart of applied mathematics. Various problems encoun-
tered in the areas of engineering, sciences, management science, and economics
are based on the fundamental idea of mathematical formulation. Optimiza-
tion is an essential tool for the formulation of many such problems expressed
in the form of minimization of a function under certain constraints like in-
equalities, equalities, and/or abstract constraints. It is thus rightly considered
a science of selecting the best of the many possible decisions in a complex
real-life environment.
Even though optimization problems have existed since very early times,
the optimization theory has settled as a solid and autonomous field only in
recent decades. The origin of analytic optimization lies in the classical calculus
of variations and is interrelated with the development of calculus. The very
concept of derivative introduced by Fermat in the mid-seventeenth century via
the tangent slope to the graph of a function was motivated by solving an op-
timization problem, leading to the Fermat stationary principle. Around 1684,
Leibniz developed a method to distinguish between minima and maxima via
second-order derivatives. The calculus of variations was introduced by Euler
while solving the Brachistochrone problem, which was posed by Bernoulli in
1696. The problem is stated as “Given two points x and y in the vertical plane.
A particle is allowed to move under its own gravity from x to y. What should
be the curve along which the particle should move so as to reach y from x in
the shortest time?” In 1759, Lagrange gave a completely different approach
to solve the problems in calculus of variations, today known as the Lagrange
multiplier rule. The Lagrange multipliers are viewed as the auxiliary variables
that are primarily used to derive the optimality conditions for constrained
optimization problems. These optimality conditions are the building blocks of
optimization theory.
During the second world war, Dantzig developed the simplex method to
solve linear programming problems. The first attempt to develop the La-
grange multiplier rules for nonlinear optimization problem was made by Fritz
John [71] in 1948. In 1951, Kuhn and Tucker [73] gave the Lagrange multiplier
rule for convex and other nonlinear optimization problems involving differen-

© 2012 by Taylor & Francis Group, LLC

 2 What Is Convex Optimization?

tiable functions. It was later found that Karush in 1939 had independently
established the optimality conditions similar to those of Kuhn and Tucker.
These optimality conditions are today famous as the Karush–Kuhn–Tucker
(KKT) optimality conditions. All the initial theories were developed with the
differentiability assumptions of the functions involved.
Meanwhile, efforts were made to shed the differentiability hypothesis,
thereby leading to the development of nonsmooth convex analysis as a subject
in itself. This added a new chapter to optimization theory. The key contrib-
utors in the development of convexity theory are Fenchel [45], Moreau [88],
and Rockafellar [97]. An important milestone in this direction was the publi-
cation of Convex Analysis by Rockafellar [97], where the theory of nonsmooth
convex analysis was presented in detail for the first time. No wonder this text
is by far a must for all optimization researchers. In the early 1970s, his stu-
dent Clarke coined the term nonsmooth optimization to categorize the theory
involving nondifferentiable optimization problems. He extended the calculus
rules and applied them to optimization problems involving locally Lipschitz
functions. This was just the beginning. The subsequent decade witnessed a
large development in the field of nonsmooth nonconvex optimization. For de-
tails on nonsmooth analysis, one may refer to Borwein and Lewis [17]; Bor-
wein and Zhu [18]; Clarke [27]; Clarke, Ledyaev, Stern and Wolenshi [28];
Mordukhovich [86]; and Rockafellar and Wets [101].
However, such developments have not overshadowed the importance of
convex optimization, which still is and will remain a pivotal area of research. It
has paved a path not only for theoretical improvements, but also algorithmic
designing aspects. In this book we focus mainly on convex analysis and its
application to the development of convex optimization theory.

1.2 Basic Concepts

By convex optimization we simply mean the problem of minimizing a convex
function over a convex set. More precisely, we are concerned with the following
problem
min f (x) subject to x ∈ C, (CP )
where f : Rn → R is a convex function and C ⊂ Rn is a convex set. Of course
in most cases the the set C is described by a system of convex inequalities
and affine equalities. Thus we can write

C = {x ∈ Rn : gi (x) ≤ 0, i = 1, 2, . . . , m and
hj (x) = 0, j = 1, 2, . . . , l},

where gi : Rn → R, i = 1, 2, . . . , m are convex functions and hj : Rn → R,

© 2012 by Taylor & Francis Group, LLC

 1.2 Basic Concepts 3

j = 1, 2, . . . , l are affine functions. When C is expressed explicitly as above,

(CP ) is called the convex programming problem.
A set C ⊂ Rn is a convex set if for any x, y ∈ Rn , the line segment joining
them, that is

[x, y] = {z ∈ Rn : z = (1 − λ)x + λy, 0 ≤ λ ≤ 1},

is also in C. A function φ : Rn → R is a convex function if for any x, y ∈ Rn

and λ ∈ [0, 1],

φ((1 − λ)x + λy) ≤ (1 − λ)φ(x) + λφ(y),

while it is an affine function if it is a translate of a linear function; that is, φ

is affine if

φ(x) = ha, xi + b,

where a ∈ Rn and b ∈ R.
It is important to note at the very outset that in optimization theory it is
worthwhile to consider extended-valued functions, that is, functions that take
values in R̄ = R∪{−∞, +∞}. The need to do so arises when we seek to convert
a constrained optimization problem into an unconstrained one. Consider for
example the problem (CP ), which can be restated as

min f0 (x) subject to x ∈ Rn ,

where

f (x), x ∈ C,
f0 (x) =
+∞, otherwise.
All the modern books on convex analysis beginning with the classic Convex
Analysis by Rockafellar [97] follow this framework. However, when we include
infinities, we need to know how to deal with them. Most rules with infinity
are intuitively clear except possibly 0 × (+∞) and ∞ − ∞. Because we will
be dealing mainly with minimization problems, we will follow the convention
0 × (+∞) = (+∞) × 0 = 0 and ∞ − ∞ = ∞. This convention was adopted in
Rockafellar and Wets [101] and we shall follow it. However, we would like to
ascertain that we really need not get worried about ∞ − ∞ as the functions
considered in this book are real-valued or proper functions. An extended-
valued function φ : Rn → R̄ is said to be a proper function if φ(x) > −∞ for
every x ∈ Rn and dom φ is nonempty where dom φ = {x ∈ Rn : φ(x) < +∞}
is the domain of φ.
It is worthwhile to note that the definition of a convex function given
above can be extended to the case when φ is an extended-valued function. An
extended-valued function φ : Rn → R̄ is a convex function if for any x, y ∈ Rn
and λ ∈ [0, 1],

φ((1 − λ)x + λy) ≤ (1 − λ)φ(x) + λφ(y),

© 2012 by Taylor & Francis Group, LLC

 4 What Is Convex Optimization?

with the convention that ∞ − ∞ = +∞. A better way to handle the convexity
of an extended-valued convex function is to use its associated geometry. In
this direction we describe the epigraph of a function φ : Rn → R̄, which is
given as

epi φ = {(x, α) ∈ Rn × R : φ(x) ≤ α}.

A function is said to be convex if the epigraph is convex. We leave it as a simple

exercise for the reader to show that if the epigraph of a function φ : Rn → R̄
is convex in Rn × R, then φ is a convex function over Rn . For more details see
Chapter 2.
In case of extended-valued functions, one can work with the semicontinuity
of the functions rather than the continuity. Before we define those notions, we
present certain notations that will be used throughout.
For any two sets F1 , F2 ⊂ Rn , define

F1 + F2 = {x1 + x2 ∈ Rn : x1 ∈ F1 , x2 ∈ F2 }.

For any set F ⊂ Rn and any scalar λ ∈ R,

λF = {λx ∈ Rn : x ∈ F }.

The closure of a set F is denoted by cl F while the interior is given by int F .

The open unit ball, or simply unit ball, is denoted by B. By Bδ (x̄) we mean
an open ball of radius δ > 0 with center at x̄. Explicitly,

Bδ (x̄) = x̄ + δB.

(y1 , y2 , . . . , yn ) in Rn , the inner

For vectors x = (x1 , x2 , . . . , xn ) and y = P
n
py is denoted by hx, yi = i=1 xi yi while the norm of x is
product of x and
given by kxk = hx, xi. We state a standard result on the norm.

Proposition 1.1 (Cauchy–Schwarz Inequality) For any two vectors x, y ∈ Rn ,

|hx, yi| ≤ kxkkyk.

The above inequality holds as equality if and only if x = αy for some scalar
α ∈ R.

To discuss the concept of continuities of a function, we shall consider the

notions of limit infimum and limit supremum of a function. But first we discuss
the convergence of sequences in Rn .

Definition 1.2 A sequence {xk ∈ R : k = 1, 2, . . .} or simply {xk } ⊂ R is

said to converge to x̄ ∈ R if for every ε > 0, there exists kε such that

|xk − x̄| < ε, ∀ k ≥ kε .

A sequence {xk } ⊂ Rn converges to x̄ ∈ Rn if the i-th component of xk

© 2012 by Taylor & Francis Group, LLC

 1.2 Basic Concepts 5

converges to the i-th component of x̄. The vector x̄ is called the limit of {xk }.
Symbolically it is expressed as

xk → x̄ or lim xk = x̄.
k→∞

The sequence {xk } ⊂ Rn is bounded if each of its components is bounded.

Equivalently, {xk } is bounded if and only if there exists M ∈ R such that
kxk k ≤ M for every k ∈ N. A subsequence of {xk } ⊂ Rn is a sequence {xkj },
j = 1, 2, . . ., where each xkj is a member of the original sequence and the order
of the elements as in the original sequence is maintained. A vector x̄ ∈ Rn is
a limit point of {xk } ⊂ Rn if there exists a subsequence of {xk } converging
to x̄. If the limit point is unique, it is the limit of {xk }. Next we state the
classical result on the bounded sequences.

Proposition 1.3 (Bolzano–Weierstrass Theorem) A bounded sequence in Rn

has a convergent subsequence.

For a sequence {xk } ⊂ R, define

zr = inf{xk : k ≥ r} and yr = sup{xk : k ≥ r}.

It is obvious that the sequences {zr } and {yr } are nondecreasing and non-
increasing, respectively. If {xk } is bounded below or bounded above, the se-
quences {zr } or {yr }, respectively, have a limit. The limit of {zr } is called
the limit infimum or lower limit of {xk } and denoted by lim inf k→∞ xk , while
that of {yr } is called the limit supremum or upper limit of {xk } and denoted
by lim supk→∞ xk . Equivalently,

lim inf xk = lim { inf xr } and lim sup xk = lim {sup xr }.

k→∞ k→∞ r≥k k→∞ k→∞ r≥k

For a sequence {xk }, lim inf k→∞ xk = −∞ if the sequence is unbounded below
while lim supk→∞ xk = +∞ if the sequence is unbounded above. Therefore,
{xk } converges to x̄ if and only if

−∞ < lim inf xk = x̄ = lim sup xk < +∞.

k→∞ k→∞

Now we move on to define the semicontinuities of a function that involve

the limit infimum and limit supremum of the function.

Definition 1.4 A function φ : Rn → R̄ is said to be lower semicontinuous

(lsc) at x̄ ∈ Rn if for every sequence {xk } ⊂ Rn converging to x̄,

φ(x̄) ≤ lim inf φ(xk ).

k→∞

Equivalently,

φ(x̄) ≤ lim inf φ(x),

x→x̄

© 2012 by Taylor & Francis Group, LLC

 6 What Is Convex Optimization?

where the term on the right-hand side of the inequality denotes the limit
infimum or the lower limit of the function φ defined as
lim inf φ(x) = lim inf φ(x).
x→x̄ δ↓0 x∈Bδ (x̄)

The function φ is lsc over a set F ⊂ Rn if φ is lsc at every x̄ ∈ F .

For a function φ : Rn → R̄,
inf φ(x) ≤ φ(x̄).
x∈Bδ (x̄)

Taking the limit as δ ↓ 0 in the above inequality leads to

lim inf φ(x) ≤ φ(x̄).
x→x̄

Thus, the inequality in the above definition of lsc can be replaced by an

equality, that is, φ : Rn → R̄ is lsc at x̄ if
φ(x̄) = lim inf φ(x).
x→x̄

Similar to the concept of lower semicontinuity and limit infimum, we next

define the upper semicontinuity and the limit supremum of a function.
Definition 1.5 A function φ : Rn → R̄ is said to be upper semicontinuous
(usc) at x̄ ∈ Rn if for every sequence {xk } ⊂ Rn converging to x̄,
φ(x̄) ≥ lim sup φ(xk ).
k→∞

Equivalently,
φ(x̄) ≥ lim sup φ(x),
x→x̄

where the term on the right-hand side of the inequality denotes the limit
supremum or the upper limit of the function φ defined as
lim sup φ(x) = lim sup φ(x).
x→x̄ δ↓0 x∈Bδ (x̄)

The function φ is usc over a set F ⊂ Rn if φ is usc at every x̄ ∈ F .

Definition 1.6 A function φ : Rn → R̄ is said to be continuous at x̄ if it is
lsc as well as usc at x̄, that is,
lim φ(x) = φ(x̄).
x→x̄

Alternatively, φ is continuous at x̄ if for any ε > 0 there exists δ(ε, x̄) > 0
such that
|φ(x) − φ(x̄)| ≤ ε whenever kx − x̄k < δ(ε, x̄).
The function φ is continuous over a set F ⊂ Rn if φ is continuous at every
x̄ ∈ F .

© 2012 by Taylor & Francis Group, LLC

 1.2 Basic Concepts 7

Because we will be considering minimization problems, the continuity of

a function will be replaced by lower semicontinuity. Before moving on, we
state a result on the infimum and supremum operations from Rockafellar and
Wets [101].

Proposition 1.7 (i) Consider an extended-valued function φ : Rn → R̄ and

sets Fi ⊂ Rn , i = 1, 2 such that F1 ⊂ F2 . Then

inf φ(x1 ) ≥ inf φ(x2 ) and sup φ(x1 ) ≤ sup φ(x2 ).

x1 ∈F1 x2 ∈F2 x1 ∈F1 x2 ∈F2

(ii) Consider the functions φ1 , φ2 : Rn → R̄ and a set F ⊂ Rn . Then

inf φ1 (x) + inf φ2 (x) ≤ inf (φ1 + φ2 )(x)

x∈F x∈F x∈F
≤ sup (φ1 + φ2 )(x) ≤ sup φ1 (x) + sup φ2 (x).
x∈F x∈F x∈F

Also, for functions φi : Rni → R̄ and sets Fi ⊂ Rni , i = 1, 2,

inf φ1 (x1 ) + inf φ2 (x2 ) = inf (φ1 (x1 ) + φ2 (x2 )),

x1 ∈F1 x2 ∈F2 (x1 ,x2 )∈F1 ×F2

sup φ1 (x1 ) + sup φ2 (x2 ) = sup (φ1 (x1 ) + φ2 (x2 )).

x1 ∈F1 x2 ∈F2 (x1 ,x2 )∈F1 ×F2

(iii) Consider an extended-valued function φ : Rn → R̄ and a set F ⊂ Rn .

Then for λ ≥ 0,

inf (λφ)(x) = λ inf φ(x) and sup (λφ)(x) = λ sup φ(x),

x∈F x∈F x∈F x∈F

provided 0 × (+∞) = 0 = 0 × (−∞).

The next result from Rockafellar and Wets [101] gives a characterization
of limit infimum of an arbitrary extended-valued function.

Lemma 1.8 For an extended-valued function φ : Rn → R̄,

lim inf φ(x) = min{α ∈ R̄ : there exists xk → x̄ satisfying φ(xk ) → α}.

x→x̄

Proof. Suppose that lim inf x→x̄ φ(x) = ᾱ. We claim that for xk → x̄ with
φ(xk ) → α, α ≥ ᾱ. As xk → x̄, for any δ > 0, there exists kδ ∈ N such that
xk ∈ Bδ (x̄) for every k ≥ kδ . Therefore,

φ(xk ) ≥ inf φ(x).

x∈Bδ (x̄)

Taking the limit as k → +∞ in the above inequality,

α≥ inf φ(x), ∀ δ > 0.

x∈Bδ (x̄)

© 2012 by Taylor & Francis Group, LLC

 8 What Is Convex Optimization?

Because δ is arbitrarily chosen, so taking the limit δ ↓ 0 along with the defin-
ition of the limit infimum of φ leads to

α ≥ lim inf φ(x),

x→x̄

that is, α ≥ ᾱ.

To prove the result, we shall show that there exists a sequence xk → x̄
such that φ(xk ) → ᾱ. For a nonnegative sequence {δk }, define

ᾱk = inf φ(x).

x∈Bδk (x̄)

As δk → 0, by Definition 1.4 of limit infimum, ᾱk → ᾱ. Now for every k ∈ N,

by the definition of infimum it is possible to find xk ∈ Bδk (x̄) for which φ(xk )
is very close to ᾱk , that is, in an interval [ᾱk , αk ] where ᾱk < αk and αk → ᾱ.
Therefore, as k → +∞, xk → x̄, and φ(xk ) → ᾱ, thereby establishing the
result. 
After the characterization of limit infimum of a function, the result below
gives an equivalent characterization of lower semicontinuity of the function in
terms of the epigraph and lower level set.

Theorem 1.9 Consider a function φ : Rn → R̄. Then the following condi-

tions are equivalent:

(i) φ is lsc over Rn .

(ii) The epigraph of φ, epi φ, is a closed set in Rn × R.
(iii) The lower-level set lev≤α φ = {x ∈ Rn : φ(x) ≤ α} is closed for every
α ∈ R.

Proof. If φ ≡ ∞, the result holds trivially. So assume that dom φ is nonempty

and thus epi φ and lev≤α φ are nonempty.
We will first show that (i) implies (ii). Consider a sequence {(xk , αk )} ⊂
epi φ such that (xk , αk ) → (x̄, ᾱ). Therefore, φ(xk ) ≤ αk , which implies that

lim inf φ(x) ≤ lim inf φ(xk ) ≤ ᾱ.

x→x̄ k→∞

By the lower semicontinuity of φ,

φ(x̄) = lim inf φ(x),

x→x̄

which reduces the preceding condition to φ(x̄) ≤ ᾱ, thereby proving that epi φ
is a closed set in Rn × R.
Next we show that (ii) implies (iii). For a fixed α ∈ R, suppose
that {xk } ⊂ lev≤α φ such that xk → x̄. Therefore, φ(xk ) ≤ α, that is,
(xk , α) ∈ epi φ. By (ii), epi φ is closed, which implies (x̄, α) ∈ epi φ, that
is, φ(x̄) ≤ α. Thus, x̄ ∈ lev≤α φ, thereby yielding condition (iii).

© 2012 by Taylor & Francis Group, LLC

 1.2 Basic Concepts 9

Finally, to obtain the equivalence, we will establish that (iii) implies (i).
To show that φ is lsc, we need to show that for every x̄ ∈ Rn ,

φ(x̄) ≤ lim inf φ(xk ) whenever xk → x̄.

k→∞

On the contrary, assume that for some x̄ ∈ Rn and some sequence xk → x̄,

φ(x̄) > lim inf φ(xk ),

k→∞

which implies there exists α ∈ R such that

φ(x̄) > α > lim inf φ(xk ). (1.1)

k→∞

Thus, there exists a subsequence, without relabeling, say {xk } such that
φ(xk ) ≤ α for every k ∈ N, which implies xk ∈ lev≤α φ. By (iii), the lower
level set lev≤α φ is closed and hence x̄ ∈ lev≤α φ, that is, φ(x̄) ≤ α, which
contradicts (1.1). Therefore, φ is lsc over Rn . 
The proof of the last implication, that is, (iii) implies (i) of Theorem 1.9
by contradiction was from Bertsekas [12]. We present an alternative proof for
the same from Rockafellar and Wets [101].
It is obvious that for any x̄ ∈ Rn ,

ᾱ = lim inf φ(x) ≤ φ(x̄).

x→x̄

Therefore, to establish the lower semicontinuity of φ at x̄, we need to prove

that φ(x̄) ≤ ᾱ. By Lemma 1.8, there exists a sequence {xk } ⊂ Rn with
xk → x̄ such that φ(xk ) → ᾱ. Thus, for every α > ᾱ, φ(xk ) ≤ α, which
implies xk ∈ lev≤α φ. Now if condition (iii) of the above theorem holds, that
is, lev≤α φ is closed in Rn ,

x̄ ∈ lev≤α φ, ∀ α > ᾱ.

Thus, φ(x̄) ≤ α, which leads to φ(x̄) ≤ ᾱ. Because x̄ ∈ Rn was arbitrarily

chosen, φ is lsc over Rn .
Theorem 1.9 gives equivalent characterization of lower semicontinuity of a
function. But if the function is not lsc, its epigraph is not closed. The result
below gives an equivalent characterization of the closure of the epigraph of
any arbitrary function.

Proposition 1.10 For any arbitrary extended-valued function φ : Rn → R̄,

(x̄, ᾱ) ∈ cl epi φ if and only if

lim inf φ(x) ≤ ᾱ.

x→x̄

© 2012 by Taylor & Francis Group, LLC

 10 What Is Convex Optimization?

Proof. Suppose that (x̄, ᾱ) ∈ cl epi φ, which implies that there exists
{(xk , αk )} ⊂ epi φ such that (xk , αk ) → (x̄, ᾱ). Thus, taking the limit as
k → +∞, the condition

lim inf φ(x) ≤ lim inf φ(xk )

x→x̄ xk →x̄

yields

lim inf φ(x) ≤ ᾱ,

x→x̄

as desired.
Conversely, assume that lim inf x→x̄ φ(x) ≤ ᾱ but (x̄, ᾱ) 6∈ cl epi φ.
We claim that, lim inf x→x̄ φ(x) = ᾱ. On the contrary, suppose that
lim inf x→x̄ φ(x) < ᾱ. As (x̄, ᾱ) ∈
/ cl epi φ, there exists δ̄ > 0 such that for
every δ ∈ (0, δ̄),

Bδ ((x̄, ᾱ)) ∩ cl epi φ = ∅,

which implies for every (x, α) ∈ Bδ ((x̄, ᾱ)), φ(x) > α. In particular for
(x, ᾱ) ∈ Bδ ((x̄, ᾱ)), φ(x) > ᾱ, that is,

φ(x) > ᾱ, ∀ x ∈ Bδ (x̄).

Therefore, taking the limit as δ → 0 along with the definition of limit infimum
of a function yields

lim inf φ(x) ≥ ᾱ,

x→x̄

which is a contradiction. Therefore, lim inf x→x̄ φ(x) = ᾱ. By Lemma 1.8, there
exists a sequence xk → x̄ such that φ(xk ) → ᾱ. Because (xk , φ(xk )) ∈ epi φ,
(x̄, ᾱ) ∈ cl epi φ, thereby reaching a contradiction and hence the result. 
Now the question is whether it is possible to construct a function that is
the closure of the epigraph of another function. This leads to the concept of
closure of a function.

Definition 1.11 For any function φ : Rn → R̄, an lsc function that is con-
structed in such a way that its epigraph is the closure of the epigraph of φ is
called the lower semicontinuous hull or the closure of the function φ and is
denoted by cl φ. Therefore,

epi(cl φ) = cl epi φ.

Equivalently, the closure of φ is defined as

cl φ(x̄) = lim inf φ(x), ∀ x̄ ∈ Rn .

x→x̄

By Proposition 1.10, it is obvious that (x̄, ᾱ) ∈ cl epi φ if and only if

(x̄, ᾱ) ∈ epi cl φ. The function φ is said to be closed if cl φ = φ.

© 2012 by Taylor & Francis Group, LLC

 1.2 Basic Concepts 11

−1 1 −1 1

epi φ epi cl φ

FIGURE 1.1: Lower semicontinuous hull.

If φ is lsc, then it is closed as well. Also cl φ is lsc and the greatest of all
lsc functions ψ such that ψ(x) ≤ φ(x) for every x ∈ Rn . From Theorem 1.9,
one has that closedness is the same as lower semicontinuity over Rn . In this
discussion, the function φ was defined over Rn . But what if φ is defined over
some subset of Rn . Then one cannot talk about the lower semicontinuity of
the function over Rn . In such a case, how is the closedness of a function related
to lower semicontinuity? This issue was addressed by Bertsekas [12]. Consider
a set F ⊂ Rn and a function φ : F → R̄. Observe that here we define φ over
the set F and not Rn . The function φ can be extended over Rn by defining a
function φ̄ : Rn → R̄ as

φ(x), x ∈ F,
φ̄(x) =
+∞, otherwise.
Note that both the extended-valued functions φ and φ̄ have the same epigraph.
Thus from the above discussion, one has φ is closed if and only if φ̄ is lsc over
Rn . Also observe that the lower semicontinuity of φ over dom φ is not sufficient
for φ to be closed. In addition, one has to assume the closedness of dom φ.
To emphasize this fact, let us consider a simple example. Consider φ : R → R̄
defined as

0, x ∈ (−1, 1),
φ(x) =
+∞, otherwise.
Here, dom φ = (−1, 1) over which the function is lsc but epi φ is not closed
and hence, φ is not closed. The closure of φ is given by

0, x ∈ [−1, 1],
cl φ(x) =
+∞, otherwise.
Observe that in Figure 1.1, epi φ is not closed while epi cl φ is closed. There-
fore, we have the following result from Bertsekas [12].

© 2012 by Taylor & Francis Group, LLC

 12 What Is Convex Optimization?

Proposition 1.12 Consider F ⊂ Rn and a function φ : F → R̄. If dom φ is

closed and φ is lsc over dom φ, then φ is closed.

Because we are interested in studying the minimization problem, it is im-

portant to know whether a minimizer exists or not. In this respect, we have
the classical Weierstrass theorem, according to which “A continuous function
attains its minimum over a compact set.” For a more general version of this
theorem from Bertsekas [12], we require the notion of coercivity.

Definition 1.13 A function φ : Rn → R̄ is said to be coercive over a set

F ⊂ Rn if for every sequence {xk } ⊂ F

lim φ(xk ) = +∞ whenever kxk k → +∞.

k→∞

For F = Rn , φ is simply called coercive.

Observe that for a coercive function, every nonempty lower level set is
bounded. Below we prove the Weierstrass Theorem.

Theorem 1.14 (Weierstrass Theorem) Consider a proper lsc function

φ : Rn → R̄ and assume that one of the following holds:

(i) dom φ is bounded.

(ii) there exists α ∈ R such that the lower level set lev≤α φ is nonempty and
bounded.

(iii) φ is coercive.

Then the set of minimizers of φ over Rn is nonempty and compact.

Proof. Suppose that condition (i) holds, that is, dom φ is bounded. Because
φ is proper, φ(x) > −∞ for every x ∈ Rn and dom φ is nonempty. Denote
φinf = inf x∈Rn φ(x), which implies φinf = inf x∈dom φ φ(x). Therefore, there
exists a sequence {xk } ⊂ dom φ such that φ(xk ) → φinf . Because dom φ is
bounded, {xk } is a bounded sequence, which by Bolzano–Weierstrass Theo-
rem, Proposition 1.3, has a convergent subsequence. Without loss of generality,
assume that xk → x̄. By the lower semicontinuity of φ,

φ(x̄) ≤ lim inf φ(xk ) = lim φ(xk ) = φinf ,

k→∞ k→∞

which implies that x̄ is a point of minimizer of φ over Rn . Denote the set

of minimizers by S. Therefore, x̄ ∈ S and hence S is nonempty. Because
S ⊂ dom φ which is bounded, S is a bounded set. Also, S is the intersection
of the lower level sets lev≤α φ, where α > m. For an lsc function φ, lev≤α φ is
closed by Theorem 1.9 and thus S is closed. Hence S is compact.
Assume that condition (ii) holds; that is, for some α ∈ R, the lower level

© 2012 by Taylor & Francis Group, LLC

 1.2 Basic Concepts 13

set lev≤α φ is nonempty and bounded. Consider a proper function φ̄ : Rn → R̄

defined as

φ(x), φ(x) ≤ α,
φ̄(x) =
+∞, otherwise.

Therefore, dom φ̄ = lev≤α φ which is nonempty and bounded by condition (ii).

Since φ is lsc which by Theorem 1.9 implies that dom φ̄ is closed. Also by the
lower semicontinuity of φ along with Proposition 1.12, φ̄ is closed and hence
lsc. Moreover, the set of minimizers of φ̄ is the same as that of φ. The result
follows by applying condition (i) to φ̄.
Suppose that condition (iii) is satisfied, that is, φ is coercive. Because φ
is proper, dom φ is nonempty and thus has a nonempty lower level set. By
the coercivity of φ, it is obvious that the nonempty lower level sets of φ are
bounded, thereby satisfying condition (ii), and therefore leading to the desired
result. 
As we all know, the next concept that comes to mind after limit and
continuity is the derivative of a function. Below we define this very notion.

Definition 1.15 For a scalar-valued function φ : Rn → R, the derivative of

φ at x̄ is denoted by ∇φ(x̄) ∈ Rn and is defined as

φ(x̄ + h) − φ(x̄) − h∇φ(x̄), hi

lim = 0.
khk→0 khk

Equivalently, the derivative can also be expressed as

φ(x) = φ(x̄) + h∇φ(x̄), x − x̄i + o(kx − x̄k),

o(kx − x̄)k
where limx→x̄ = 0. A function φ is differentiable if it is differen-
kx − x̄k
tiable at every x ∈ Rn . The derivative, ∇φ(x̄), of φ at x̄ is also called the
gradient of φ at x̄, which can be expressed as
 
∂φ ∂φ ∂φ
∇φ(x̄) = (x̄), (x̄), . . . , (x̄) ,
∂x1 ∂x2 ∂xn

∂φ
where , i = 1, 2, . . . , n denotes the i-th partial derivative of φ. If φ is
∂xi
continuously differentiable, that is, the map x 7→ ∇φ(x) is continuous over
Rn , then φ is called a smooth function. If φ is not smooth, it is called a
nonsmooth function.
Similar to the first-order differentiability, we have the second-order differ-
entiability notion as follows.

© 2012 by Taylor & Francis Group, LLC

 14 What Is Convex Optimization?

Definition 1.16 For a scalar-valued function φ : Rn → R, the second-order

derivative of φ at x̄ is denoted by ∇2 φ(x̄) ∈ Rn×n and is defined as
1
φ(x̄ + h) − φ(x̄) − h∇φ(x̄), hi − h∇2 φ(x̄)h, hi
lim 2 = 0,
khk→0 khk2

which is equivalent to

φ(x) = φ(x̄) + h∇φ(x̄), x − x̄i + h∇2 φ(x̄)(x − x̄), x − x̄i + o(kx − x̄k2 ).

The matrix ∇2 φ(x̄) is also referred to as the Hessian with the ij-th entry of
∂2φ
the matrix being the second-order partial derivative (x̄). If φ is twice
∂xi ∂xj
2
continuously differentiable, then the matrix ∇ φ(x̄) is a symmetric matrix.
In the above definitions we considered the function φ to be a scalar-valued
function. Next we define the notion of differentiability for a vector-valued
function Φ.

Definition 1.17 For a vector-valued function Φ : Rn → Rm , the derivative

of Φ at x̄ is denoted by JΦ(x̄) ∈ Rm×n and is defined as

kΦ(x̄ + h) − Φ(x̄) − hJΦ(x̄), hik

lim = 0.
khk→0 khk

The matrix JΦ(x̄) is also called the Jacobian of Φ at x̄. If Φ = (φ1 , φ2 , . . . , φm ),

Φ is differentiable if each φi : Rn → R, i = 1, 2, . . . , m is differentiable. The
Jacobian of Φ at x̄ can be expressed as
 
∇φ1 (x̄)
 ∇φ2 (x̄) 
 
JΦ(x̄) =  .. 
 . 
∇φm (x̄)

∂φi
with the ij-th entry of the matrix being the partial derivative (x̄). In
∂xj
the above expression of JΦ(x̄), the vectors ∇φ1 (x̄), ∇φ2 (x̄), . . . , ∇φm (x̄) are
written as row vectors.
Observe that the derivative is a local concept and it is defined at a point
x if x ∈ int dom φ. Below we state the Mean Value Theorem, which plays a
pivotal role in the study of optimality conditions.

Theorem 1.18 (Mean Value Theorem) Consider a continuously differen-

tiable function φ : Rn → R. Then for every x, y ∈ Rn , there exists z ∈ [x, y]
such that

φ(y) − φ(x) = h∇φ(z), y − xi.

© 2012 by Taylor & Francis Group, LLC

 1.3 Smooth Convex Optimization 15

With all these basic concepts we now move on to the study of convexity.
The importance of convexity in optimization stems from the fact that when-
ever we minimize a convex function over a convex set, every local minimum
is a global minimum. Many other issues in optimization depend on convexity.
However, convex functions suffer from the drawback that they need not be
differentiable at every point of their domain of definition and the nondiffer-
entiability may be precisely at the point where the minimum is achieved. For
instance, consider the minimization of the absolute value function, |x|, over R.
At the point of minima, x̄ = 0, the function is nondifferentiable. How this ma-
jor difficulty was overcome by the development of a completely different type
of analysis is possibly one of the most thrilling developments in optimization
theory. This analysis depends on set-valued maps, which we briefly present
below.
Definition 1.19 A set-valued map Φ from Rn to Rm associates every x ∈ Rn
to a set in Rm ; that is, for every x ∈ Rn , Φ(x) ⊂ Rm . Symbolically it is
expressed as Φ : Rn ⇉ Rm . A set-valued map is associated with its graph
defined as
gph Φ = {(x, y) ∈ Rn × Rm : y ∈ Φ(x)}.
Φ is said to be a proper map if there exists x ∈ Rn such that Φ(x) 6= ∅. Φ is said
to be closed-valued or convex-valued or bounded-valued if for every x ∈ Rn , the
sets Φ(x) are closed or convex or bounded, respectively. Φ is locally bounded
at x̄ ∈ Rn if there exists δ > 0 and a bounded set F ⊂ Rn such that
Φ(x) ⊂ V, ∀ x ∈ Bδ (x̄).
The set-valued map Φ is said to be closed if it has a closed graph; that is, for
any sequence {xk } ⊂ Rn with xk → x̄ and yk ∈ Φ(xk ) with yk → ȳ, ȳ ∈ Φ(x̄).
A set-valued map Φ : Rn → Rm is said to be upper semicontinuous (usc) at
x̄ ∈ Rn if for any ε > 0, there exists δ > 0 such that
Φ(x) ⊂ Φ(x̄) + εB, ∀ x ∈ Bδ (x̄),
where the balls are in the respective spaces. If Φ is locally bounded and has a
closed graph, then it is usc. If Φ is single-valued, that is, Φ(x) is singleton for
every x, the upper semicontinuity of Φ coincides with continuity.
For more on set-valued maps, the readers may refer to Berge [10]. A de-
tailed analysis of convex function appears in Chapter 2.

1.3 Smooth Convex Optimization

Recall the convex optimization problem (CP ) stated in Section 1.1, that is,

© 2012 by Taylor & Francis Group, LLC

 16 What Is Convex Optimization?

(y, f (y))

(x, f (x)) (y, f (x) + ∇f (x)(y − x))

graph of f

x y x

FIGURE 1.2: Graph of a real-valued differentiable convex function.

min f (x) subject to x ∈ C, (CP )

where f : Rn → R is a convex function and C is a closed convex set in Rn . Let
us additionally assume that f is differentiable. It is mentioned in Chapter 2
that if f is differentiable, then for any x ∈ Rn ,
f (y) − f (x) ≥ h∇f (x), y − xi, ∀ y ∈ Rn .
Conversely, if the above relation holds for a function, then the function is
convex. This fact appears as Theorem 2.81 in the next chapter. It is mentioned
there as a consequence of more general facts. However, we provide a direct
proof here.
Observe that if f is convex, then for any x, y ∈ Rn and any λ ∈ [0, 1],
(1 − λ)f (x) + λf (y) ≥ f (x + λ(y − x)).
Hence, for any λ ∈ (0, 1),
f (x + λ(y − x)) − f (x)
f (y) − f (x) ≥ .
λ
Taking the limit as λ ↓ 0, the above inequality yields
f (y) − f (x) ≥ h∇f (x), y − xi. (1.2)
For the converse, suppose that (1.2) holds for any x, y ∈ Rn . Setting
z = x + λ(y − x) with λ ∈ (0, 1), then
f (y) − f (z) ≥ h∇f (z), y − zi (1.3)
f (x) − f (z) ≥ h∇f (z), x − zi (1.4)

© 2012 by Taylor & Francis Group, LLC

 1.3 Smooth Convex Optimization 17

The result is obtained by simply multiplying (1.3) with λ and (1.4) with
(1 − λ) and then adding them up. This description geometrically means that
the tangent plane should always lie below the graph of the function. For a
convex function f : R → R, it looks something like Figure 1.2. This important
characterization of a convex function leads to the following result.

Theorem 1.20 Consider the convex optimization problem (CP ) where f is

a differentiable convex function and C is a closed convex set in Rn . Then x̄
is a point of minimizer of (CP ) if and only if

h∇f (x̄), x − x̄i ≥ 0, ∀ x ∈ C. (1.5)

Proof. It is simple to see that as C is a convex set, for x ∈ C,

x̄ + λ(x − x̄) ∈ C, ∀ λ ∈ [0, 1].

Therefore, if x̄ is a point of minimum,

f (x̄ + λ(x − x̄)) ≥ f (x̄),

that is,

f (x̄ + λ(x − x̄)) − f (x̄) ≥ 0.

Dividing both sides by λ > 0 and taking the limit as λ ↓ 0 leads to

h∇f (x̄), x − x̄i ≥ 0.

Because x ∈ C was arbitrarily chosen,

h∇f (x̄), x − x̄i ≥ 0, ∀ x ∈ C.

Also as f is convex, by the condition (1.2), for any x ∈ C,

f (x) − f (x̄) ≥ h∇f (x̄), x − x̄i.

Now if (1.5) is satisfied, then the above inequality reduces to

f (x) ≥ f (x̄), ∀ x ∈ C,

thereby proving the requisite result. 

Remark 1.21 Expressing the optimality condition in the form of (1.5) leads
to what is called a variational inequality. Let F : Rn → Rn be a given function
and C be a closed convex set in Rn . Then the variational inequality V I(F, C)
is the problem of finding x̄ ∈ C such that

hF (x̄), x − x̄i ≥ 0, ∀ x ∈ C.

© 2012 by Taylor & Francis Group, LLC

 18 What Is Convex Optimization?

C λ = λ0

x x̄

FIGURE 1.3: Local minimizer is global minimizer.

When f is a differentiable convex function, for F = ∇f , V I(∇f, C) is nothing

but the condition (1.5). In order to solve V I(F, C) efficiently, one needs an
additional property on F which is monotonicity. A function F : Rn → Rn is
called monotone if for any x, y ∈ Rn ,

hF (y) − F (x), y − xi ≥ 0.

However, when f is a convex function, one has the following pleasant property.

Theorem 1.22 A differentiable function f is convex if and only if ∇f is

monotone.

For proof, see Rockafellar [97]. However, the reader should try to prove it on
his/her own. We have shown that when (CP ) has a smooth f , one can write
down a necessary and sufficient condition for a point x̄ ∈ C to be a global
minimizer of (CP ). In fact, as already mentioned, the importance of studying
convexity in optimization stems from the following fact. For the problem (CP ),
every local minimizer is a global minimizer irrespective of the fact whether f
is smooth or not. This can be proved in a simple way as follows. If x̄ is a local
minimizer of (CP ), then there exists δ > 0 such that

f (x) ≥ f (x̄), ∀ x ∈ C ∩ Bδ .

Now consider any x ∈ C. Then it is easy to observe from Figure 1.3 that there
exists λ0 ∈ (0, 1) such that for every λ ∈ (0, λ0 ),

λx + (1 − λ)x̄ ∈ C ∩ Bδ .

Hence

f (λx + (1 − λ)x̄) ≥ f (x̄).

© 2012 by Taylor & Francis Group, LLC

 1.3 Smooth Convex Optimization 19

The convexity of f shows that

λ(f (x) − f (x̄)) ≥ 0.

Because λ > 0, f (x) ≥ f (x̄). As x ∈ C was arbitrary, our claim is established.

The result can also be obtained using the approach of contradiction as done
in Theorem 2.90.
Now consider the following function

θ(x) = sup h∇f (x), x − yi.

y∈C

The interesting feature of the function is that

θ(x) ≥ 0, ∀ x ∈ C

and if θ(x) = 0 for x ∈ C, then x solves the problem (CP ). Furthermore, if x

solves the problem (CP ), we have θ(x) = 0. The function θ is usually called the
gap function or the merit function associated with (CP ). For the variational
inequality problem, such a function was first introduced by Auslender [5]. The
next question is how useful is the function θ to the problem (CP ). What we
will now show is that for certain classes of the problem (CP ), the function
θ can provide an error bound for the problem (CP ). By an error bound we
mean an upper estimate of the distance of a point in C to the solution set
of (CP ). The class of convex optimization problems where such a thing can
be achieved is the class of strongly convex optimization problems. A function
f : Rn → R is strongly convex with modulus of strong convexity ρ > 0 if for
any x, y ∈ Rn and λ ∈ [0, 1],

(1 − λ)f (x) + λf (y) ≥ f (x + λ(y − x)) + ρλ(1 − λ)ky − xk2 .

If f is differentiable, then f is strongly convex if and only if for any x, y ∈ Rn ,

f (y) − f (x) ≥ h∇f (x), y − xi + ρky − xk2 .

1
Observe that f (x) = hx, Axi, where x ∈ Rn and A is a positive definite n×n
2
matrix, is strongly convex with ρ = λmin (A), the minimum eigenvalue of A
while f (x) = x with x ∈ Rn is not strongly convex.
If f is a twice continuously differentiable strongly convex function, then
∇2 f (x) is always positive definite for each x. Now if f is strongly convex with
modulus of convexity ρ > 0, then for any x, y ∈ Rn ,

f (y) − f (x) ≥ h∇f (x), y − xi + ρky − xk2 ,

f (x) − f (y) ≥ h∇f (y), x − yi + ρky − xk2 .

Adding the above inequalities leads to

0 ≥ h∇f (x), y − xi + h∇f (y), x − yi + 2ρky − xk2 ,

© 2012 by Taylor & Francis Group, LLC

 20 What Is Convex Optimization?

that is,
h∇f (y) − ∇f (x), y − xi ≥ 2ρky − xk2 . (1.6)
The property of ∇f given by (1.6) is called strong monotonicity with 2ρ as the
modulus of monotonicity. It is in fact interesting to observe that if f : Rn → R
is a differentiable function for which there exists ρ > 0 such that for every
x, y ∈ Rn ,

h∇f (y) − ∇f (x), y − xi ≥ 2ρky − xk2 ,

which implies that

h∇f (y) − ∇f (x), y − xi ≥ ρky − xk2 , ∀ x, y ∈ Rn .

Now we request the reader to show that f is strongly convex with modulus
ρ > 0. In fact, if f is strongly convex with ρ > 0 one can also show that ∇f
is strongly monotone with ρ > 0. Thus we conclude that f is strongly convex
with modulus of strong convexity ρ > 0 if and only if ∇f is strongly monotone
with modulus of monotonicity ρ > 0.
It is important to note that one cannot guarantee θ to be finite unless C
has some additional conditions, for example, C is compact. Assume that C
is compact and let x̄ be a solution of the problem (CP ), where f is strongly
convex. (Think why a solution should exist.) Now as f is strongly convex, it
is simple enough to see that x̄ is the unique solution of (CP ). Thus from the
definition of θ, for any x ∈ C and y = x̄,

θ(x) ≥ h∇f (x), x − x̄i.

By strong convexity of f with ρ > 0 as the modulus of strong convexity, ∇f

is strongly monotone with modulus 2ρ. Thus,

h∇f (x), x − x̄i ≥ h∇f (x̄), x − x̄i + 2ρkx − x̄k2 ,

thereby yielding

θ(x) ≥ h∇f (x̄), x − x̄i + 2ρkx − x̄k2 . (1.7)

But by the optimality condition in Theorem 1.20,

h∇f (x̄), x − x̄i ≥ 0, ∀ x ∈ C.

Therefore, the inequality (1.7) reduces to

θ(x) ≥ 2ρkx − x̄k2 ,

which leads to
s
θ(x)
kx − x̄k ≤ .
2ρ

© 2012 by Taylor & Francis Group, LLC

 1.3 Smooth Convex Optimization 21

This provides an error bound for (CP ), where f is strongly convex and C is
compact. In this derivation if ∇f was strongly monotone with modulus ρ > 0,
then the error bound will have the expression
s
θ(x)
kx − x̄k ≤ .
ρ

Observe that as ρ > 0,

s s
θ(x) θ(x)
≤
2ρ ρ

and hence the error bound provided by considering that f is strongly monotone
with modulus 2ρ gives a sharper error bound.
Now the question is can we design a merit function for (CP ) that can be
used to develop an error bound even when C is noncompact. Such a merit func-
tion was first developed by Fukushima [48] for general variational inequalities.
In our context, the function given by
 α 
θ̂α (x) = sup h∇f (x), x − yi − ky − xk2 , α > 0.
y∈C 2

It will be an interesting exercise for the reader to show that

θ̂α (x) ≥ 0, ∀ x ∈ C

and θ̂α (x) = 0 for x ∈ C if and only if x is a solution of (CP ). Observe that
 α 
θ̂α (x) = − inf h∇f (x), y − xi + ky − xk2 , α > 0.
y∈C 2
For a fixed x, observe that the function
α
φα
x (y) = h∇f (x), y − xi + ky − xk2
2
is a strongly convex function and is coercive (Definition 1.13). Hence φαx attains
a lower bound on C. The point of minimum is unique as φα x is strongly convex.
Hence for each x, the function φα x has a finite minimum value. Thus θ̂ α (x) is
always finite, thereby leading to the following error bound.

Theorem 1.23 Consider the convex optimization problem (CP ) where f is

a differentiable strongly convex function with modulus ρ > 0 and C is a closed
convex set in Rn . Let x̄ ∈ C be the unique solution of (CP ). Furthermore, if
α
ρ > , then for any x ∈ C,
2
s
2θ̂α (x)
kx − x̄k ≤ .
2ρ − α

© 2012 by Taylor & Francis Group, LLC

 22 What Is Convex Optimization?

Proof. For any x ∈ C and y = x̄ in particular,

α
θ̂α (x) ≥ h∇f (x), x − x̄i − kx − x̄k2 .
2
By the fact that ∇f is strongly monotone with modulus ρ > 0,
α
θ̂α (x) ≥ h∇f (x̄), x − x̄i + ρkx − x̄k2 − kx − x̄k2 . (1.8)
2
Because x̄ is the unique point of minimizer of (CP ), by Theorem 1.20,

h∇f (x̄), x − x̄i ≥ 0,

thereby reducing the inequality (1.8) to

α
θ̂α (x) ≥ (ρ − )kx − x̄k2 .
2
Therefore,
s
2θ̂α (x)
kx − x̄k ≤ ,
2ρ − α

as desired. 
The reader is urged to show that under the hypothesis of the above theo-
rem, one can prove a more tighter error bound of the form
s
2θ̂α (x)
kx − x̄k ≤ .
4ρ − α

The study of optimality conditions with C explicitly given by functional con-

straints will begin in Chapter 3.

© 2012 by Taylor & Francis Group, LLC

 Chapter 2
Tools for Convex Optimization

2.1 Introduction
With the basic concepts discussed in the previous chapter, we devote this
chapter to the study of concepts related to the convex analysis. Convex analy-
sis is the branch of mathematics that studies convex objects, namely, convex
sets, convex functions, and convex optimization theory. These concepts will be
used in the subsequent chapters to discuss the details of convex optimization
theory and in the development of the book.

2.2 Convex Sets

Recall that for any x, y ∈ Rn , the set
[x, y] = {z ∈ Rn : z = (1 − λ)x + λy, 0 ≤ λ ≤ 1}
denotes the line segment joining the points x and y. The open line segment
joining x and y is given by
(x, y) = {z ∈ Rn : z = (1 − λ)x + λy, 0 < λ < 1}.
Definition 2.1 A set F ⊂ Rn is a convex set if
λx + (1 − λ)y ∈ F, ∀ x, y ∈ F, ∀ λ ∈ [0, 1].
Equivalently, for any x, y ∈ F , the line segment [x, y] is contained in F . Fig-
ure 2.1 present convex and nonconvex sets.
Consider the hyperplane defined as
H(a, b) = {x ∈ Rn : ha, xi = b},
where a ∈ Rn and b ∈ R. Observe that it is a convex set. Similarly, the closed
half spaces given by
H≤ (a, b) = {x ∈ Rn : ha, xi ≤ b} and H≥ (a, b) = {x ∈ Rn : ha, xi ≥ b},

23

© 2012 by Taylor & Francis Group, LLC

 24 Tools for Convex Optimization

convex set nonconvex set

FIGURE 2.1: Convex and nonconvex sets.

and the open half spaces given by

H< (a, b) = {x ∈ Rn : ha, xi < b} and H> (a, b) = {x ∈ Rn : ha, xi > b}
are also convex. Another class of sets that are also convex is the affine sets.
Definition 2.2 A set M ⊂ Rn is said to be an affine set if
(1 − λ)x + λy ∈ M, ∀ x, y ∈ M, ∀ λ ∈ R,
where the set {z ∈ Rn : z = (1 − λ)x + λy, λ ∈ R} denotes the line passing
through x and y. Equivalently, M ∈ Rn is affine if for any x, y ∈ M , the line
passing through them is contained in M .
Note that a hyperplane is an example of an affine set. The empty set ∅ and
the whole space Rn are the trivial examples of affine sets. Even though affine
sets are convex, the converse need not be true, as is obvious from the example
of half spaces.
Next we state some basic properties of convex sets.
Proposition 2.3 (i) The intersection of an arbitrary collection of convex sets
is convex.
(ii) For two convex sets F1 , F2 ⊂ Rn , F1 + F2 is convex.
(iii) For a convex set F ⊂ Rn and scalar λ ∈ R, λF is convex.
(iv) For a convex set F ⊂ Rn and scalars λ1 ≥ 0 and λ2 ≥ 0,
(λ1 + λ2 )F = λ1 F + λ2 F
which is convex.
Proof. The properties (i)-(iii) can be established by simply using Defini-
tion 2.1. The readers are urged to prove (i)-(iii) on their own. Here we will
prove only (iv). Consider z ∈ (λ1 + λ2 )F . Thus, there exists x ∈ F such that
z = (λ1 + λ2 )x = λ1 x + λ2 x ∈ λ1 F + λ2 F.

© 2012 by Taylor & Francis Group, LLC

 2.2 Convex Sets 25

Because z ∈ (λ1 + λ2 )F was arbitrary,

(λ1 + λ2 )F ⊂ λ1 F + λ2 F. (2.1)

Conversely, let z ∈ λ1 F + λ2 F , which implies that there exist x1 , x2 ∈ F such

that  
λ1 λ2
z = λ1 x1 + λ2 x2 = (λ1 + λ2 ) x1 + x2 . (2.2)
λ1 + λ2 λ1 + λ2
λi
Because ∈ [0, 1], i = 1, 2, which along with the convexity of F implies
λ1 + λ2
that
λ1 λ2
x= x1 + x2 ∈ F. (2.3)
λ1 + λ2 λ1 + λ2
Combining the conditions (2.2) and (2.3) lead to

z = (λ1 + λ2 )x ∈ (λ1 + λ2 )F.

As z ∈ λ1 F + λ2 F was arbitrarily chosen,

(λ1 + λ2 )F ⊃ λ1 F + λ2 F,

which along with the inclusion (2.1) yields the desired equality. Observe that
(ii) and (iii) lead to the convexity of (λ1 + λ2 )F = λ1 F + λ2 F . 
From Proposition 2.3, it is obvious that intersection of finitely many closed
half spaces is again a convex set. Such sets that can be expressed in this
form are called polyhedral sets. These sets play an important role in linear
programming problems. We will deal with polyhedral sets later in this chapter.
However, unlike the intersection and sum of convex sets, the union as well
as the complement of convex sets need not be convex. For instance, consider
the sets

F1 = {(x, y) ∈ R2 : x2 + y 2 ≤ 1} and F2 = {(x, y) ∈ R2 : y ≥ x2 }.

Observe from Figure 2.2 that both F1 and F2 along with their intersection are
convex sets but neither their complements nor the union of these two sets is
convex.
To overcome such situations where nonconvex sets come into the picture
in convex analysis, one has to convexify the nonconvex sets. This leads to the
notion of convex combination and convex hull.

Definition 2.4 A point x ∈ Rn is said to be a convex combination of the

points x1 , x2 , . . . , xm ∈ Rn if

x = λ1 x1 + λ2 x2 + . . . + λm xm
Pm
with λi ≥ 0, i = 1, 2, . . . , m and i=1 λi = 1.

© 2012 by Taylor & Francis Group, LLC

 26 Tools for Convex Optimization

0000000
1111111 0
10
10
10
10
10
1 0 1
1
0
1 00
10
1
000
111
0
1
0
1
11
000
10
10
10
1 0
1 0
10
10
1
0
10
1
111
000
1111111
000000011
00 0
1
11
00 0
1
01
1 0
10 1
1 0 10
1
010
10
111
00
11
00
111
000 11
00 111
000
0
100
10
1 0
1 0
100
10
1
F1 111
0000
1
0
1
0
1
0
1
F
0
12
0
1 0
10
1
0
10
1
1111
0000
1111
0000
0
1
11
00 11
00 111
0000
1
11111
000000
10
1 0
1 1111
0000
0
1 0
1
1111
0000
111
000 11
00
111
000 01
10 0
1 0
1
111
000 0 1
1 0 1
1111111111
00000000000
1111111
000000011
00 1111111111
0000000000
11111111111
00000000000
1111111
0000000
1111111
0000000 1111111111
0000000000
F1c F2c

1
0
0
10
1
0
1 0
1
1
0 1
0 1
00
10
1
0
10
1
01
10
0
1 0
1
0
1 0
1 0
1
0
10
10
1
0
10
1
0
1
0
1 0
1
0
1 0
1 0
1
0
1 0
1
0
1
01
1 0
1
0
1
0
1 0
1
1
0
0
1 1
0 1
0
1
0
0
10
10
1
0
0
1
0
1
0
1
1
00
1 0
1
0
1 0
1
0
1 0
1 0
1
0
10
10
1
0
10
1
0
1 0
1 0
1
0
1
1 0
1 0
1
1
00
10
1
0
10
10
1
0
1 0
1 0
0
1 1
0 0
1
0
1
01
100
1
0
1 0
1 0
1
0
1
0
1 0
1
0
1 0
1
0
1
0
1
0
1 11
0
0
1 0 0
1
0
1
0
1
1
0 1
0 1
0
1
0
0
1
0
1 0
1 0
1
0
1
0
1 0
1
0 1
0 1
1 0

F1 ∩ F2 F1 ∪ F2

FIGURE 2.2: F1 , F2 , and F1 ∩ F2 are convex while F1c , F2c , and F1 ∪ F2 are
nonconvex.

The next result expresses the concept of convex set in terms of the convex
combination of its elements.

Theorem 2.5 A set F ⊂ Rn is convex if and only if it contains all the convex
combinations of its elements.

Proof. From Definition 2.1 of convex set, F ⊂ Rn is convex if and only if

(1 − λ)x1 + λx2 ∈ F, ∀ x1 , x2 ∈ F, λ ∈ [0, 1],

that is, the convex combination for m = 2 belongs to F .

To establish the result, we will use the induction approach. Suppose that
the convex combination of m = l − 1 elements of F belong to F . Consider
m = l. The convex combination of l elements is

x = λ1 x1 + λ2 x2 + . . . + λl xl ,

© 2012 by Taylor & Francis Group, LLC

Another Random Scribd Document
with Unrelated Content
 CHAPTER X
A SHIP IN DISTRESS

Had Henry but known it, there was no immediate necessity for his
return to the Iroquois. Indeed, the Coast Guard cutter not only was
unable to sail that day, but she did not cast off her hawsers until
afternoon the day following. Although Henry thus had an
unexpected half day in Boston, he saw no more of the city than he
had seen on the preceding day, for when he awoke rain was pouring
down, a vicious east wind was blowing, and the elements were as
nasty as they well could be.
Even a complete suit of rubber would hardly have kept one dry
very long in the slashing, blustering blasts that came howling
through the Navy Yard. The rain drove in horizontal sheets. It
whipped around corners and under doorways and awnings. It roared
across open decks. It beat against the air-ports. With ever-increasing
force the gusts came tearing in from the sea. Trees bent far over
and groaned before their onslaught. Flags whipped themselves to
ribbons. Halyards beat a very devil’s tattoo against their masts, and
on the Iroquois the cordage fairly shrieked in wild, wailing notes that
made Henry almost shudder.
He was glad enough that the Iroquois had been unable to get
away. Never had the captain’s cabin seemed half so comfortable and
attractive. He told himself that he would be glad enough to spend
the time there, reading some of the interesting books from the
captain’s bookshelf, while waiting for the storm to subside.
Yet the movement of stores went forward without interruption.
Sailors, cased from head to foot in sou’westers, oilskins, and rubber
boots, worked without ceasing in the downpour to finish the
transshipment. Henry could hear them clumping about the deck in
their clumsy footgear. On the pier trucks rattled and banged. Boxes
were wheeled aboard and dumped on the deck. Men swore and
slipped in the wet. Machinery rattled.
While Henry and his host were eating their luncheon, the noises
suddenly ceased. There were a few shouted orders, indistinguishable
in the roar of the storm, then some banging noises as hatches were
closed and battened down, and other openings made fast. Soon all
was quiet. When the luncheon was ended, Henry went up the
companionway and peeped out. Things had been made tight.
Awnings had been removed. Everything was lashed fast. The decks
were bare. The Iroquois was stripped for action.
Henry could hardly believe that Captain Hardwick would leave
port in such a storm. He knew that storm warnings must be showing
all along the New England coast, and perhaps the entire Atlantic
seaboard, but before he could return to the cabin and ask the
captain if he intended to put to sea, he saw the chief electrician
running aft along the slippery deck. Henry threw open the door for
him, and the radio man dodged in out of the blinding rain. He had a
message for the captain.
Henry descended to the cabin with the electrician. Captain
Hardwick took the telegraph blank from Mr. Sharp and studied it a
moment.
“No orders?” he asked.
“Not yet,” said the electrician.
The captain sat down at his desk, drew a red-bordered telegraph
blank from a pigeonhole, and wrote. “Send that,” he directed.
The radio man struggled out into the storm. The captain rose and
touched his call-bell.
“Rollin,” he said, when his attendant appeared, “ask Mr. Farley to
come to me at once.”
The commander picked up the telegram and handed it to Henry.
“You’ll have a chance to see something today, Henry,” he said.
Henry looked at the message in his hands. It was a cry of
distress, an SOS message the radio man had picked out of the air:
 “Steamer Capitol City ashore east
coast Cape Cod forty-two north, seventy
west. Pounding badly. Need immediate
assistance.”
The chief engineer entered the cabin. “Start your engines at once,
Mr. Farley. We just caught a distress call. Steamer ashore on the east
coast of Cape Cod. See that everything is ready for a hard run.”
“All right, sir,” and the chief engineer hurried forward.
Soon the ship began to vibrate. The rumbling noise of the
machinery arose. Yet the Iroquois lay quiet in her dock.
“He’s warming her up,” thought Henry. “It’s a good thing, too, for
this is going to be some struggle. I’ll bet the waves are like
mountains.”
The captain rang his call-bell. “Send Lieutenant Hill to me,” he
said.
The lieutenant came briskly into the cabin. The two officers
conferred a moment. Henry stepped over to the wardroom. Not even
the doctor was there. Cards and books lay on the table in disorder,
as they had been dropped at news of the wireless. Everybody was at
his post, preparing. Henry returned to the cabin. The lieutenant was
gone. The commander was pulling on his oilskins.
“Where can I get a slicker?” Henry inquired.
“Ask Rollin. He will get you one.”
The captain pulled on his sou’wester, tied the strings under his
chin, and mounted to the deck. Henry rang for Rollin and stated his
wish. The attendant brought boots, hat, and slicker. Henry pulled on
a sweater, buttoned his coat up tight, and pulled on the waterproofs.
The gust of wind that struck him as he came out of the
companionway sent him reeling back against that structure. He
could hardly catch his breath. The driving sheets of rain blinded him.
He fought his way forward, and entered the radio shack. The chief
electrician was copying down a message. It was an order for the
Iroquois to go to the rescue of the Capitol City. But Henry knew that
it was not needed. Already the Iroquois was prepared to get under
way.
Henry took the message to the bridge and handed it to the
commander. “Tell Sparks to wire that we are leaving Boston to help
the Capitol City,” shouted the captain.
Henry carried the message to Mr. Sharp and watched him send it.
Then he went back to the bridge. The rain beat on him as irresistibly
as ever, but the weather-cloth offered surprising protection from the
wind. A sailor slipped the hawsers over the posts on the pier. Other
sailors drew in the hawsers and stowed them away. The captain
pressed his signal-bell, and the Iroquois began to move astern. She
backed out into the stream and then turned and headed for the sea,
into the teeth of the driving storm.
The beating rain obscured the view. Fog made the shores almost
indistinguishable, for in from the sea, blown on the breath of the icy
blasts, came racing great clouds of murky white vapor that screened
all they touched. The captain looked grim and inscrutable. His jaw
was set hard. He stood by the wheelhouse, conning the ship. At half
speed the Iroquois slowly nosed her way down the channel. Wiped
from view was the beautiful scene that had so delighted Henry a few
short hours before. Nothing could be seen but occasional glimpses of
shore, the tumultuous, muddy water, and the driving curtains of fog.
One by one the captain made the proper turns in the tortuous
channel. As the Iroquois stood farther and farther out toward the
sea, the waters became ever more tumultuous, the winds roared
more fiercely, and the fog shut in ever denser. Fathom by fathom the
ship crept past one after another of the island defenses along the
way, that served as breakwaters to the sea and broke the sweep of
the winds. When at last the little ship turned eastward at George’s
Island, and faced the storm with the last vestige of protection gone,
she trembled and shook in the grasp of the roaring blasts.
A smother of foam was the sea. Waves rose and broke in
incredible confusion. The waters were churned as by a giant hand.
The racing winds whipped the crests from the combers and flung
them forward in sheets of blinding spray. Fog drove onward in
clouds, now completely hiding the sea, now lifting momentarily, to
expose the wild waste of tossing waters. The fury of the storm was
indescribable.
Mountain high indeed seemed the waves. Before the bow of the
Iroquois they rose up, up, up, as high as the men on the bridge,
then rushed savagely at the little boat, seemingly bent on her
destruction. Down they crashed, and the nose of the cutter was
buried in a smother of foaming water. Sometimes the crests swept
completely over the bow, pouring over the forward deck in great
floods that raced aft and went foaming out of the scuppers. Now
Henry saw why the decks had been cleared of all movable objects.
Indeed, as he watched the smashing combers crash over the bow,
he feared that the big guns themselves would be torn from their
foundations on the iron deck-plates and hurled aft against the
wheelhouse. With blanched face he stood on the bridge, desperately
gripping the rail, and peering with fascinated gaze at the snarling,
hungry seas.
Meantime the captain had sent a reassuring message to the
Capitol City, telling her the Iroquois was on her way to assist her.
And when the ship was fairly in the sea, past all chance of harm by
rock or shoal, the captain left the lieutenant in charge of the bridge
and went himself to the chart-room to plan his coming movements.
With him went Henry. He had seen enough of the sea for a time.
Some of the fear that first gripped his soul had gone. He knew that
the Iroquois was safe so long as she held her course, but he wanted
to shut out for a time the sight of those terrifying billows; so he
staggered to the chart-room, and stepped inside, glad of a relief
from the terrible tension that had held him.
The captain was calmly poring over his charts and guide-books.
“Forty-two north, seventy west,” he muttered, sweeping his glance
over an outstretched map. He placed his pencil on the indicated
spot. “She’s on the shoals almost dead east of Truro,” he said to
Henry.
Then he turned to a Coast Guard directory and leafed it over.
“Thank God!” he cried. “There’s a Coast Guard land station near by.
It may be two or three miles distant, but they’ll come with their
outfit. They must have gotten the wireless flash just as we did. It’s
likely they’re on their way now. What a fight that will be—those
miles across the sand with their heavy boats in this wind. They’ll do
it, though. But maybe we can get to the Capitol City before she
breaks up. I wish we dared go faster.”
He stepped out on the bridge and conferred with the lieutenant.
“Do you think that we dare drive her any faster?” he shouted in the
latter’s ear.
The lieutenant shook his head dubiously. “Well, we’ll try it,” said
the commander.
And he stepped into the wheelhouse and signaled for more speed.
The response was terrifying. Under increased power the Iroquois
drove ahead, bit by bit gaining greater momentum. Savagely she
charged into the seas. Wildly the waves leaped to meet her. The
impact grew ever more terrific. Soon a huge mountain of water
came roaring down on the little cutter. Up it rose and up and up,
while the cutter charged to meet it, and as it broke and crashed
forward, the cutter dived completely under it. The shock was terrific.
Tons of water crashed down on the deck, then went racing aft
against the superstructure. The Iroquois seemed to stop in her
tracks. She trembled from stem to stern. She shook and quivered.
The great wave went roaring aft, twisting and tearing at the ship. A
section of the rail carried away, and went pounding astern. The
tumult was terrifying. Henry, in the chart-room, thought the ship was
breaking to pieces. With blanched face and set teeth he clung to the
side of the room, not knowing what to do. He was reassured when
the captain calmly stepped inside the wheelhouse and signaled for
the engineer to lessen speed.
Hour after hour they wallowed through the storm. Unflinching,
undaunted, in complete mastery of ship and crew, the captain stood
on the bridge, with his right-hand man. Mile by mile, league after
league, he fought his way eastward. The rain ceased, but the winds
blew on, lashing the seas to fiercer and greater fury. The sky was
totally obscured. Gradually the fog lifted, but not for long was the
increased vision. Darkness came apace, and into that roaring
darkness the Iroquois drove with all the power she dared to use.
With anxious eyes the captain watched the logbook. At times he
telephoned to the chief engineer. Again and again he stepped to the
wheelhouse and looked at the compass. Anon he consulted with the
lieutenant. He must know where he was, in order to get safely round
the long arm of Cape Cod. Nor did he dare stand too close to shore
in his run along the eastward coast of the Cape, lest the storm put
the Iroquois also on the shoals. Dead reckoning alone would tell him
when to turn, and carefully the commander considered every feature
that might indicate his position.
Darkness had long since shut in, and the Iroquois was wallowing
through a night as black as pitch before the captain altered his
course and headed south along the outer edge of the Cape.
Gradually the course of the vessel was shifted. To Henry the change
was terrifying. No longer was the Iroquois breasting the storm. The
waves took her abeam. From side to side she rolled until Henry’s
heart stood still with fear. Over and over and over she dipped until
he was certain she would turn upside down. Then slowly she righted
and swung in the opposite direction. And once, when she rolled at
an angle of forty degrees, Henry almost gasped aloud. It seemed
like eternity while the ship lay poised almost on her beam’s ends,
apparently uncertain whether to roll on over or come back on her
keel. Then she slowly righted.
Meantime the chief electrician had been in touch with the
stranded ship. From her signals he knew that she was not far away.
They came crackling out of the air sharp and clear. A distant glow
showed that the guardsmen from the land were already at the
scene. At last the Capitol City wired that she could see the lights of
the Iroquois.
“Make a flare,” flashed back the Iroquois’ wireless man.
And presently, almost dead to leeward of the Iroquois, the
darkness was torn by a flash, and a flaming rocket went streaking up
through the night. Other rockets followed. Then a flaring light arose,
and through their glasses the men on the bridge of the Iroquois
could see the stricken ship, lying in a smother of foam on the outer
edge of the breakers. She was too far from shore for the land crew
to shoot a line to her, and no small boat could live in such a sea. If
the crew of the Capitol City were to be saved, the little cutter alone
could save them.
Cautiously the Iroquois was worked in toward the shoals. Then
slowly she was turned, the captain gave the word, and one of the
anchors was let go. Once more the cutter was heading again into
the seas, and now, little by little, her anchor-chain was paid out,
while seamen with hand-lines took soundings, calling up to the
bridge the depth they found. Fathom after fathom the giant anchor-
chain was paid out. Fathom after fathom the Iroquois rode backward
toward the seething breakers.
Fascinated, Henry watched the attempt to get within reach of the
unfortunate steamer. Gradually the Iroquois drew near to the
smother of white water. The sea was shoaling fast and the tide was
running out, but the captain kept on in the hope that he could get
near enough to shoot a line aboard the Capitol City. His shells, and
the shot-line, loosely wound in the faking-box so that it would run
out freely, were ready for instant use. All that was necessary was to
remove the canvas cover from the little gun on the after rail, insert
the charge, and shoot. But the Iroquois never came within shooting
distance. Too rapidly the water shoaled, and at last, reluctantly, the
commander gave the word and the anchor-chain was held taut. The
Iroquois was rolling, head to the sea, at the very edge of the
breakers, but she was too far away from the Capitol City to put a
shot across her.
“Tell them we’ll float a line down to them, and for them to be on
the lookout for it,” the captain ordered the wireless man.
The latter sat down at his key, but a moment later switched off.
The Capitol City’s wireless was failing. He shifted to the blinkers, and
for the first time Henry had opportunity to see the lights on the
yardarm flash and blink. From the Capitol City came answering winks
from aloft.
 “They’re looking for our line,” the wireless man informed the
captain.

A Coast Guard Cutter Standing by a Stranded Ocean Liner

Meantime empty kegs had been prepared. A light line was made
fast to one of them, and it was thrown into the sea. Rapidly it
floated to leeward, and as fast as the sea carried it shoreward the
line was paid out. The searchlight of the Iroquois was broken out
and its beam kept on the floating keg. Slowly this bore down on the
Capitol City, but it was too far to one side for that ship to get it. The
line was hauled in, and again it was floated toward the helpless ship.
This time a sailor hurled the keg far to one side of the Iroquois, in
the hope that it might now come close enough to the stranded ship
to be caught. But again the attempt failed. The ebb tide, with its
cross current, carried it farther away from the Capitol City than it
had been before.
“Lower the surfboat,” ordered the commander, when he saw the
attempt was useless. Then he called for volunteers. By the dozen the
sailors leaped forward.
“Boatswain Johnson,” said the captain, “I’m going to put you in
charge. Pick your crew.”
The boatswain selected eight sturdy sailors for oarsmen. A life
belt was strapped round each. The boat was lowered to the rail, and
the crew stepped carefully aboard. At a favorable moment the craft
was launched. Quickly she shot away from the side of the Iroquois,
and before another comber broke, she was at a safe distance from
the cutter and heading straight into the breakers.
The ship’s light was trained on her. On she went, now up, now
down, breasting the roaring waves, shooting through the smother of
foam, riding safely where it seemed impossible for a boat to live,
under the skillful guidance of the experienced boatswain. Swiftly she
drew toward the Capitol City, which no longer lay at right angles to
the beach, but had worked a little to one side, making a lee where
the water was calmer. Toward this the boatswain drove the surf
boat. Into it the little craft shot safely, while a sigh of relief went up
from the deck of the Iroquois.
Through powerful glasses, Henry watched breathlessly while the
surfboat drew close to the protected side of the Capitol City. A line
was thrown to the little boat, and a sailor in the bow caught it. Then
the surfboat was drawn close beside the stranded steamer, and a
sailor scrambled down from the rigging and dropped into it.
From seaward a giant comber was rushing toward the tiny craft. If
it caught her, it would crush her against the side of the larger ship as
an eggshell is crushed underfoot. On the Iroquois not a soul
breathed. The onlookers stood tense, waiting the outcome. But the
boatswain had an eye for everything. He, too, saw the great comber
approaching. Back from the steamer’s side drew the little boat, and
the wave passed harmlessly by. Again the surfboat drew up to the
side of the Capitol City, and another man detached himself from the
rigging and dropped into it. But just at that moment a wave, rising
apparently from nowhere, swept over the windward side of the
stranded steamer, poured irresistibly across her deck, and sent its
tons of icy water crashing downward into the little surfboat lying by
the leeward rail. The tiny craft was swept from sight.
“My God!” cried the captain. “They’ve capsized!”
 CHAPTER XI
LOST IN THE SEA

For a moment the crew of the Iroquois stood as though petrified.

In the brilliant beam of the searchlight they could see the dark forms
of their comrades as they bobbed up above the smother of foam.
Down from the bridge and the rigging of the Capitol City, where the
members of her crew had taken refuge from the sweeping waves,
these men now came leaping fearlessly. To rope and life-buoy they
ran, and, seizing them, hurled them far out into the waves toward
the struggling men from the Iroquois. But the strong cross current
that had borne the floating keg so far to one side of the stranded
ship, now carried the struggling men rapidly away from the steamer
and the floating buoys. Desperately the men fought to reach the
Capitol City, but the rushing tide beat them back, sucked them
farther and farther away from the ship, and dragged them out
toward the deeper water.
Not long did the men on the Iroquois remain idly watching their
comrades, helpless in the surf. One instant alone they stood as
though paralyzed. Then, as the captain began to bark his terse
commands, they leaped to action. In an instant the anchor-chain
was released, and as it paid out the Iroquois began to move. Driven
by wind and wave, she bore farther into the breakers. A seaman
with the lead sounded and roared out the depths. Rapidly the water
shoaled, but the captain let the Iroquois drive on. He meant to save
his men if it were humanly possible. Rapidly the tide was sweeping
them outward. The wind was pushing the cutter inward. Every
second brought the struggling men nearer to their comrades, who
waited with ropes and buoys, at the rail of the Iroquois.
Of a sudden the Iroquois was jarred from stem to stern. Again
and again, as she rolled in the waves, came that pounding, jarring
sensation. She had struck bottom. She was on the shoals. The
captain signaled for full speed ahead, and shouted an order to the
men at the anchor-chain. The propeller began to revolve slowly, then
faster, then at full speed. The engine on the forward deck began to
heave in the anchor-chain. The Iroquois seemed to pause and
gather herself for a leap. Almost imperceptibly at first, then faster,
and then with a rush, she moved through the water. An oncoming
comber, towering toward the skies, met her bow on, and again the
little cutter plunged headlong under water. A warning cry went up.
The men on deck grasped rope and rail and clung with all their
might. The great wave went sweeping aft like an avenging fury, but
not a man was swept away. Instantly the captain signaled to lessen
speed. The Iroquois forged ahead more slowly, the leadsman
sounding continually and shouting the depth to the commander on
the bridge. Soon the cutter rode at a safe depth. The engines were
stopped, the anchor-chain was made fast, and once more the
Iroquois rode safe at the edge of the breakers.
All the while the resistless tide was sweeping the struggling men
in the breakers out toward the Iroquois. One by one they were now
borne past the ship, struggling desperately to reach her side. With
trembling hands eager comrades flung ropes and buoys. All fell
short. One man alone came close, bravely fighting his way to the
starboard side of the cutter. A sailor climbed over the rail and down
a ladder, leaned down, and snatched his struggling comrade as the
latter shot upward on the crest of a wave. At that instant the ship
lurched violently to port, the sailor’s grasp was broken, his comrade
was torn from his grip, and the poor fellow was sucked away by a
wave, and, struggling desperately, was borne out to sea. All about
the Iroquois men were fighting with the waves. In desperation their
comrades watched them.
To launch a boat seemed not humanly possible. No little craft
could live in such a sea. But the captain called for volunteers, and as
one man the crew sprang forward.
“Lower away the leeward lifeboat,” roared the captain.
 Eager hands unbent the gripes, the falls were loosened, and the
lifeboat dropped level with the rail. Into her leaped the chosen crew,
with the executive officer in command. Down went the boat. For a
moment it rode the waves in safety, and pulled out toward the
struggling men in the sea. It had almost reached the nearest when a
swirling comber rose beside it, towered a second above it, and then
came crashing down on it, burying men and boat under tons of
yeasty water.
For a moment, boat and men were completely lost to sight. Far
down beneath the swirling seas the ill-fated crew of the lifeboat had
been thrust by the towering comber. The moving searchlight showed
no human form among the savage seas. Then suddenly the sea was
full of struggling men. Some fought their way to the overturned
boats and clung to them. Those whose grips had loosened were
seized by their comrades and dragged back to the pitiful security of
a hold on a floating boat. Some could not gain even this slender
assistance.
At this instant the tide turned. Far to seaward the men from that
first ill-fated boat had now been swept, out into the blackness of the
night, past the possibility of assistance. But the struggling crew of
the other boat were now borne slowly shoreward. Now wave and
wind combined to wash them toward the distant sands. It did not
seem possible that they could safely pass through that seething
caldron. With incredible fury the waves beat down upon them. Like
chips in a mill race they were tossed helplessly this way and that.
But every man of them wore a life belt, and despite the buffeting of
the seas all remained afloat and alive. Bravely they continued to
fight for their lives.
Two boats had been swamped. Two crews were battling for life in
the waves, and one was irrevocably lost. But men still pressed
forward and begged to be allowed to try again. No boat could live in
such an awful sea, yet the men of the Iroquois pleaded for a chance,
a last chance, to save their comrades. The captain ordered one more
boat lowered. Like its predecessors, it lived but a few minutes in the
awful sea.
 Three boats had now been capsized, and three crews were
struggling in the sea. Many were clinging to the overturned boats,
while others had gained some of the buoys thrown to the crew of
the first overturned boat. Numbers were swimming unaided. The sea
was full of boats and floating men. Impotent, heartsick, torn with
anguish, the men remaining on the Iroquois stood watching the
awful sight. They had done their best. They had done all that human
beings could possibly do, but it was not enough. There was nothing
else they could do but pray, and many an agonized seaman, rough
as a barnacle, stood with tears streaming down his rugged face and
prayed for his comrades struggling for their lives in that awful sea.
Perhaps those prayers were heard. With every minute the
incoming tide ran stronger, washing the struggling men toward
shore, where now were burning the welcome beacons of the crews
from landward Coast Guard stations. Again and again they tried to
launch their surfboats, and as often were beaten back. Now they
stood at the edge of the waves, waiting to assist their comrades
from the ship.
Ceaselessly the searchlight of the Iroquois played upon the
breakers, and on her bridge officers stood with glasses and watched
the awful fight. Miraculously the struggling men drove steadily
toward the shore. Soon they were in shallow water. They touched
bottom. And now, fighting their way upward on the sand, they
struggled through the breakers. Again and again inward-rushing
waves beat them down, but always they were flung forward, tossed
landward, driven farther toward the sandy beach and safety. The
flaming fires before them heartened them, encouraged them. New
strength came to them, and singly, in pairs, and even in little groups,
they battled their way onward. The strong helped the weak, and one
or two were seen dragging comrades who were wholly helpless. Out
into the breakers rushed the waiting guards from the shore. Strong
arms were thrown around weak and fainting forms, and limp bodies
were carried bodily through the waves. Again and again the sturdy
guardsmen from the shore rushed back into the waves and aided
more men ashore, while others toiled to resuscitate the few who had
all but succumbed.
Around the fires now pressed the rescued seamen, increasing in
number as man after man gained the sands, until they formed a
great ring about the flames. Their numbers brought joy to the
watchers on the Iroquois. Now the rescued men were seen to be
leaving the fire and trooping off into the darkness.
The commander was talking to the quartermaster, who had been
kept at his post during all the struggle. “We must be a little farther
along the coast than I thought,” said the captain. “They must be
taking our men to a Coast Guard station. Tell Sparks to get in touch
with the station and find how many men were saved.”
The quartermaster climbed down the ladder and made his way to
the radio shack. Young Belford was on duty. “Where is Mr. Sharp?”
demanded the quartermaster.
“Gone,” said the young wireless man, and there were tears in his
eyes.
“Gone! You don’t mean he went in a boat?”
The radio man nodded. For a moment he could not speak, then
he managed to say, “I saw him leap into the last boat.” And a great
sob broke from him.
“He may be safe,” said the quartermaster. “Most of the men got
ashore. We could see them with the glasses.”
“Oh, I hope so!” cried the lad.
The quartermaster ran up on the bridge again. “The chief radio
man is gone,” he said. “He pulled an oar in the last boat.”
“Find young Harper,” said the captain.
“I saw him on deck helping the sailors.”
“Send him to me.”
When the quartermaster had summoned Henry, the commander
said to him, his voice quivering with emotion, “Lad, Mr. Sharp is
gone. He was in the last boat. Are you sure you can operate our
wireless? We need it now as we never needed it before.”
“I can,” said Henry quietly.
 “Then try to get in touch with this land station. I don’t know
which one it is. I thought that we were opposite Truro. We’re
somewhere in that neighborhood. Find which station it is, and ask
how many men got ashore.”
“But what about Belford?” asked Henry. “He’s the regular
assistant. Oughtn’t he to do it?”
The captain glared angrily at Henry. Then he comprehended what
was in the boy’s mind. “Send Belford to me,” he said. “It’s no time
for etiquette now. I want action.”
“You shall have it,” said Henry, and he hastened to the radio
shack.
“Belford,” he said, as he closed the door to shut out the howl of
the wind, “the captain has asked me to send a message. He wants
you to report to the bridge.”
Henry sat down at the desk, hastily searched through the list of
Coast Guard station calls, and began combing the air with the
wireless to carry out the captain’s wishes. Meantime young Belford
mounted to the bridge and stood before his commander.
“Young man,” said the captain, “Mr. Harper is going to take charge
of the wireless. He’s a former government operator and has had
much experience. I am sorry to go over your head, but we must all
do our best. It’s a trying time.”
“I will help all I can,” said the assistant operator loyally.
“Very good,” said the commander. “Go back to the wireless house.
Tell Mr. Harper to come to me the instant he gets word from shore.”
“Yes, sir,” said the radio man, and he hurried back to his post.
Meantime Henry had been flashing abroad call after call. He had
just got his station when young Belford came back. A moment more
and his great question was answered. He turned from his key, his
face aglow.
“The captain wants to see you the minute you’re done,” said
young Belford.
Henry rose and sought the bridge. “Captain Hardwick,” he cried,
“every man except the crew of the first boat is accounted for. They
are all safe at the station.”
 “Thank God!” cried the commander. For a moment he could not
speak. Then, in husky tones, he said, “Henry, I’ve told young Belford
that you are in charge of the radio. Don’t feel any hesitation about
taking charge. There is now a vacancy in our radio staff. You are
regularly appointed to the vacancy and detailed to act as chief.”
 CHAPTER XII
THE RESCUE

Under other circumstances the captain’s words would have caused

Henry to shout with joy. Now there was no sense of jubilation in his
heart. He was stunned by the awful catastrophe that had occurred.
Nine men that he had been living with, and had come to like, had
suddenly been wiped out of existence. The horror of it had laid hold
upon every soul on the Iroquois. Perhaps it was fortunate for those
still left on the ship that there was so much to be done. There was
no time for brooding, or mourning for lost comrades. The roar of the
storm in the darkness was terrifying. The winds still were shrieking
through the cordage. Enormous waves were sweeping down on the
sturdy little cutter, threatening to overwhelm her. Only a bit of iron, a
length of chain, stood between the Iroquois and a fate like that of
the Capitol City; and a chain is no stronger than its weakest link.
There might be a weak link in the chain of the Iroquois. Her work
must be ended and the little ship taken out of danger as quickly as
possible. Every soul on board felt this distinctly. Perhaps no one felt
it more strongly than Henry did. The unaccustomed violence of the
sea appalled him. So the office that had come to him so suddenly
did not for a moment seem to him to be a matter of personal
advancement. It was a call to duty. It was his chance to help forward
the work the Iroquois had set out to do.
Very sober was Henry as he reentered the radio shack. “Belford,”
he said, “the captain thinks my experience as an operator will make
me useful on the Iroquois until Mr. Sharp can get aboard again. He
has asked me to take charge of the wireless room. I did not want to
do it, for you should be in charge. But the captain has asked me to
help, and all I can do is to obey. You’ll help me, won’t you, Belford?
You’ll pull with us, won’t you? We’ve got to work together and do
our best or we may never get out of this situation.”
“Of course, I’ll help you. Don’t give the matter another thought.
I’ll help you just as loyally as I would help Mr. Sharp,” and the lad
held out his hand.
“Thank you, Belford,” said Henry, grasping his hand. “The wireless
will mean a lot to the Iroquois in the next twenty-four hours. We’ll
both stand by the captain to the finish.” Then he added: “Where’s
Black? I want to talk to him, too.”
“I wonder where he is,” said Belford. “I can’t remember seeing
him for several hours past. Mr. Sharp was on watch the first four
hours after we left Boston, and I stood watch the next four. Black
ought to be on duty now. But there’s been so much excitement I
never thought about whose watch it was.”
“You don’t suppose he went in one of the boats, do you?” asked
Henry.
“No, I don’t,” said Belford. “And yet he might. I’ll see.”
He went to the stateroom. Soon he reappeared with a peculiar
expression on his face. “He’s in bed,” he said, “and has been abed all
the while we’ve been trying to save the Capitol City.”
The two young operators looked at each other. The same
sickening suspicion was in the mind of each. But both hesitated to
put it into words.
At that moment the quartermaster entered the room. “The
captain wants you to talk to the Capitol City,” he said, “and find out
how fast she is taking water, and how deep it is where she lies.”
“You do that,” said Henry to his companion. “You’ve had lots of
experience with the blinkers. I haven’t had any.”
Young Belford set the blinkers to winking merrily. The response
was immediate. Colored lights began to flash aloft on the Capitol
City’s yardarm. That vessel was resting easily on the sands, came
the answer, and was taking in water no faster than her pumps could
pump it out again. The tide was rising rapidly. It was already six feet
deep. This news the assistant operator carried to the commander.
 “We’ll save her yet,” said the captain. “This tide is going to be a
very high one, if I am any judge. The wind’s been blowing the water
shoreward now a full twenty-four hours.”
Rapidly the water rose. As the captain had said, the wind had
been blowing it shoreward for a full day. The ebb tide had shown
what the wind could do, for the water was far higher than usual
when the tide turned to flood. Wind and wave both pressed the
flood landward, and now the tide, running in with the wind, mounted
and mounted until it was evident that the captain’s hope was to be
realized. As the tide rose and the water about the cutter deepened,
Captain Hardwick put a leadsman to sounding.
“We must work in with the tide, Lieutenant,” he said to his
assistant on the bridge, “and be ready for action the instant the tide
is at flood. It won’t wait a second for us, though, if this wind holds,
it will delay the ebb. We must not lose a moment.”
Long before the tide was full, Captain Hardwick ordered the
anchor-chain released. At once the cutter began to move toward the
beach, very slowly at first, then faster and faster as wind and wave
gave her momentum. The lead was kept going incessantly, the
leadsman shouting the depths up to the bridge as he made his
soundings. Foot by foot, fathom by fathom, the Iroquois drew nearer
the Capitol City. Steadily the cutter’s searchlight played on the
disabled ship, its brilliant beam boring through the inky dark.
Slowly, almost imperceptibly, yet none the less truly, the wind
abated its violence. Less often the great waves swept over the deck
of the stranded steamship. Not so shrill was the screeching of the
Iroquois’ cordage. The captain, with his wide experience, had
evidently foreseen a change in the weather. He was evidently
expecting the wind to fall, and if it did, it would help in the effort to
float the stranded vessel, for a great pressure against the ship would
be removed. But as the minutes passed, the wind did not become
noticeably less. It still howled angrily, and swept with fitful force over
ship and wave. Now it came in gusts, blowing furiously for a time,
then lulling. But without ceasing the tide drove in, and the waves
crept further and further up the sides of the stranded steamship,
and the combers crashed ever higher up the sandy beach.
Fathom after fathom the Iroquois followed the rising tide
shoreward. When the lead showed questionable depths, the anchor-
chain was made fast, and the little cutter paused for a while in her
progress, marking time, as it were, to the music of the storm. With
unwonted rapidity the tide mounted up, and Captain Hardwick
followed it as fast and as far as he dared.
Plainly there was a good chance to save the Capitol City. As the
two ships came closer, every detail of the stranded ship was visible.
She had suffered astonishingly little, when the violence of the storm
was considered. She lay almost on an even keel. Though not
pointing directly to the waves, the stern was so nearly in line with
them that they were parted as they reached her, sweeping past with
little damage to her hull. A section of her taffrail was gone, and a
part of her rudder was broken off. Otherwise she appeared to have
suffered little, and the success of the pumps in keeping down the
water in the hold showed that even her plates had not been badly
started. Her superstructure had suffered little. One of her small
boats had been washed away, but otherwise she seemed to the
watchers on the Iroquois to be in remarkably good condition.
What was more important, her crew was intact. Huddled high on
the bridge and in the rigging, they had crouched together while the
men from the Iroquois were trying to reach them. But as the tide
ran low, those in the rigging had climbed down and mounted to the
bridge and superstructure, seeking warmth, for the piercing winds
had well-nigh frozen them as they clung to the rattling stays.
Apparently not a man had been swept from the Capitol City. Almost
the full crew was there to work the ship, and Captain Hardwick was
glad, indeed, that there was no such shortage of hands on the
Capitol City as existed on the Iroquois. There would be work for
many hands when the time was ripe for the effort he had in mind.
At last the Iroquois came within reach of her stranded sister. Now
a line could easily be fired across the helpless vessel. Ready was the
faking-box with the shot-line faked neatly on the tall spindles within
it, ready to run out smoothly as it traveled through the air on its
momentous journey. The little brass gun on the after-rail of the
Iroquois was uncovered, a charge was inserted in it, its Camden
projectile, with shot-line bent fast, projected from the muzzle of the
little gun, and all was ready for the effort. The captain himself
sighted the little gun, for the gunner, alas! had been in the Iroquois’
surfboat. A moment the commander waited, until the cutter rode on
an even keel.
“Fire,” he called.
There was a loud explosion, the night was stabbed with a sheet of
flame, and the projectile went hurtling out and up, tearing its way
across the hundreds of yards of raging sea that still separated the
two ships. For an instant those on board the Iroquois were blinded
by the flash of their gun. Then they tried to trace, in the glare of the
searchlight, the flight of the shot-line. Straight and true it winged its
way toward the stranded ship. Then a rush of scurrying forms on the
Capitol City told the watchers on the Iroquois that the shot had
carried true. In a moment more the crew of the Capitol City were
hauling in the little shot-line.
Already a heavier line had been bent to the end of the light shot-
line, but first it had been passed out through one of the quarter
chocks. Steadily the crew of the Capitol City drew this heavier line
aboard their craft. This in turn was followed by a heavy hawser. But
Captain Hardwick had no intention of risking defeat through the use
of so uncertain a towing line as a manila hawser. He meant to make
fast to the Capitol City with a wire cable. To that end young Belford
had been busy with the blinkers, and the flashing lights of the
Capitol City’s yardarm had answered back. Captain Hardwick had
apprised his fellow-commander of his intention, and warned him of
the necessity of heaving the heavy wire cable an inch at a time, as it
was paid out cautiously by the Iroquois. No buoyancy had this line,
like a manila hawser. Like a plummet it would drop to the bottom of
the sea, and once it started to run, out it would go its full length.
Steadily the rope hawser was paid out, and steadily it was pulled
aboard the Capitol City. Then the end of the wire cable, bent to the
hawser, was lowered, and foot by foot, with a caution hardly
credible, the handful of men on the Iroquois responded to the tug
on the line and let the wire cable slide through the quarter chock. At
the same time Captain Hardwick drifted the Iroquois closer and
closer to the vessel on the shoals.
At last the steel cable was aboard the Capitol City, and safe about
her bitts. The other end was now made fast to the bitts of the
Iroquois. The great, unbreakable, steel cable now stretched from
ship to ship. With all the power at her command, the little cutter
would presently strain at this line. At this same time she would
heave in her anchor-chain, and the vast length of this enormous
chain, reaching hundreds of fathoms out into the ocean, and
weighing tons upon tons, would add to the anchor a gripping force
that would hold like the rock of Gibraltar. Like a man pulling himself
up a rope, arm over arm, the little cutter would heave itself along
the length of this anchor-chain with the full power of both its
propeller and its heaving engine, and behind it would come the
Capitol City—perhaps.
Time alone would tell. And now all was ready. Steam was up on
both ships, ready for the supreme effort. It remained only for the
tide to reach flood, but how slowly it now seemed to advance. Up
and up it rose, creeping higher and higher up the sides of the
stranded ship, the lacy edges of the waves foaming ever higher on
the sandy beach. Anxiously the captain kept his watch. Now, with
careful eye, he studied the heavens. Now he bent his gaze upon the
tumultuous sea. Now he went forward, and with his own hands
examined the anchor-chain and looked at all its mechanism for
heaving. Again he went aft and studied the arrangement of the
hawser, appraising with practised eye the lay of the two vessels, the
sweep of the waves, the movement of the tide.
High indeed this was, and as the water mounted ever higher on
the Capitol City’s side, and the leadsman found more and more
depth beneath the Iroquois, the captain’s face showed ever-growing
confidence. From time to time he talked with the master of the
Capitol City, with young Belford as his intermediary. Anon he studied
the skies and noted with satisfaction the steady abatement of the
wind. As the time drew near for the tide to be at flood, the eager
commander paced the deck, impatient for the trial. Yet with eagle
eye he watched the tide. At last the critical moment arrived. The
commander’s judgment told him it was time to be moving. The tide
was not yet quite at flood, but it was high, extremely high. It would
not mount much higher. When it turned, the very volume of it would
cause it to run out fast.
Briskly he mounted to the bridge. “Tell the Capitol City we’re
going to move,” he called to the radio man. “Tell the captain to put
on full speed astern.”
Above, the blinker lights flashed forth their calls, and promptly
from the Capitol City came answering flashes. The ship would turn
on her power. Meantime the indicator in the engine room of the
Iroquois communicated its message to the men at the engines. The
propeller began to move, slowly at first, then faster, then at full
speed. Forward, the heaving engine began to strain at the anchor-
cable. The little cutter trembled and shook with the effort. Loud
rumbled the churning machinery in her hold.
Minute followed minute. The little craft strained and pulled. She
rose and fell in the sea. Her propeller churned the waters into yeasty
foam. Link by link the anchor-chain was heaved in. Foot by foot the
Iroquois crept ahead, but she was only making the wire cable taut.
The Capitol City had not budged. Glasses to eyes, the captain
studied the great steamer as her propeller drove round and round in
the swirling water. Critically he watched the waves sweep past her
sides, for continually the glaring light of the Iroquois was focused on
the helpless steamer. Still the tide rose higher, though now but
slowly. Every inch counted now. A few more inches and the vessel
on the sands ought almost to float of herself.
On deck the sailors watched the tide with anxious eyes. Well they
knew what a little more water would mean to the success of their
efforts. From time to time they dropped bits of wood over the sides,
to see whether the tide was still carrying toward the shore. Anon
they studied the wire cable now stretched tighter than a fiddlestring.
The tide continued to rise, though now almost imperceptibly. Then it
hesitated, halted, and stood still. It was at flood. With every ounce
of power they possessed the two steamships strained and struggled.
The tide paused, as though to give them ample opportunity. Then,
almost imperceptibly, it turned and began to flow out toward the
sea. And at that instant the lights on the Capitol City flashed forth,
and a moment later young Belford came racing up to the bridge with
a message for the captain.
“The Capitol City is moving,” he said. Then he turned and raced
back to his post.
A shout went up as the sailors sensed the import of his message.
Every eye was focused on the stranded steamer. For a moment no
motion was discernible in her. Then plainly she could be seen to
move. The shout was followed by a cheer, for now the big steamer
was plainly ploughing through the waves. Little by little she gained
momentum. Moment by moment the Iroquois drove ahead faster.
But it was no easy task that faced her. No tractable tow was this
behind her. With broken rudder, and advancing stern foremost, the
Capitol City yawed badly. Nevertheless, she came on behind the
Iroquois, as the latter forged ahead, heaving in her anchor-chain
fathom after fathom, and fighting her way out to the depths.
By the time the anchor was heaved aboard, the wind had
lessened markedly. No longer was it blowing from the east. It was
shifting, working around to the north. The tide now was running out
strong. There was no danger that either wind or tide would carry the
rescued vessel back to the shoals again. When the captain of the
Iroquois judged it to be safe, he stopped the cutter. With great care
the crew of the Capitol City shifted the towing cable from stern to
bow, and made it fast to the forward bitts. When this was done, the
Iroquois pointed her nose into the wind, gradually got under way,
and with the disabled steamer behind her, headed once more for the
city of Paul Revere. It was the nearest harbor in which the crippled
vessel could find refuge.
 CHAPTER XIII
HENRY FINDS HE HAS AN ENEMY

Slowly the staunch little cutter steamed ahead. Had it not been
for the broken rudder, the Capitol City could have gone on her way,
unattended, as she was little damaged. But with her rudder injured,
she was in a bad way. Although it was not entirely gone, and it
helped somewhat in guiding the ship, still at times the huge craft
yawed badly. By paying out more cable Captain Hardwick partly
overcame this tendency to yaw. Nevertheless it was difficult enough
to make headway, for the wind had now whipped around almost to
the north, and for a time the two ships had to sail into the teeth of
it. The Capitol City could not use the full power of her engines, but
she kept her propeller turning, and this greatly lessened the burden
on the cutter. Unless some unforeseen difficulty should arise, the two
vessels would have no trouble in reaching their port.
There was much to be done on both ships, however. The captain
of the disabled steamer wanted to get in touch with his agents, and
inform them of the safety of the vessel. His wireless was still out of
commission, and his messages had first to be sent to the Iroquois by
blinkers and then relayed by wireless. So both Henry and his friend
Belford were kept busy for a long time. Then, too, Henry had to get
in touch with the Coast Guard land station where the men from the
Iroquois had found shelter, and send instructions concerning them.
The captain felt that he could operate the Iroquois with his little
force for the short run to Boston and thence to New York. His men
were willing to do double duty. It was necessary for him to get back
to New York at the earliest possible moment. The run to Boston
would require only a few hours. Then he could retrace his steps
immediately. It was simpler to have the seamen go direct to New
York and rejoin the Iroquois there, than to wait for them in Boston.
So Henry got in touch with the land station and communicated the
captain’s orders to that effect.
By the time the two young wireless men had finished their tasks,
they were about worn out. It was far into the night. Belford had
been on duty for many hours. Henry, though not on duty so long,
was almost exhausted by the nervous strain under which he had
been working. Furthermore, he would have to respond, at any
moment, to any call for wireless communication.
“We must get to bed,” he said the moment their tasks were done.
“Won’t you please call the other operator? Then we can arrange our
watches.”
Belford summoned Black, who came into the radio shack,
yawning. “I guess you’ll have to stand watch for a while, Black,” said
Henry. “Things have got a little mixed up in the wireless house, but
we’ll get them straightened out shortly. Suppose you stay until
breakfast. That won’t be long. Then we can rearrange our watches.”
“That isn’t my regular watch,” said the young radio man, sullenly.
“And why should you be telling me what to do, anyway? That’s up to
Mr. Sharp.”
“But Mr. Sharp isn’t here,” replied Henry. “He went in one of the
boats. I am working with you men. We’ve got to do the best we can
under the circumstances. We’ll get things straightened out soon, and
back in their old order.”
“And do you think I am going to take orders from you?” said the
young operator, insolently.
Henry’s face flushed. An angry light leaped to his eyes, and his fist
involuntarily clenched. But he took a grip on himself, and replied
very slowly and deliberately: “It isn’t a question of taking orders,
Black. The captain has asked me to work with you two men, and the
three of us must pull together until we get the ship safe home.
Aren’t you willing to do your share? Belford here has been working
like a Trojan for I don’t know how many hours. He’s all worn out. We
want you to take this trick. You’ll have to watch close, for there’ll
likely be calls for us. Summon me at once if you hear our call.”
 “I reckon I can answer a call as well as anybody on this boat,”
growled the scowling radio man.
“See here, Black,” said Henry. “Why do you want to make trouble?
We’re still at sea, and we’ve got some tall traveling to do before we
are back safe in New York. We’ve all got to pull together. I’m not
asking you to do this for me. It’s for the good of the ship.” And then,
noting the sullen look in the lad’s eyes, Henry continued: “There’s
one thing more. I’m acting under orders from the captain. He has
ordered me to take charge of the wireless. My instructions are to let
no one touch the key but myself. All I can do is to pass those orders
along. If you hear a call for the Iroquois, waken me at once. I’ll
answer it.”
Young Black mumbled an unwilling assent, and swore under his
breath. Henry and the other radio man passed out of the shack to
the stateroom. They noticed that the wind had decreased greatly.
Once in the stateroom, Henry turned to his companion. “What do
you suppose ails that fellow, Black?”
“I don’t know. None of us can make him out. He’s been snappy
and surly ever since he came aboard a few months ago. He’s lazy—
too lazy to study and practice and become a really good operator.
But he is quick and reads wireless calls very well. He spends most of
his time smoking cigarettes and reading dime novels. He’s simply
crazy to use the wireless. He thinks he’s a great operator. The chief
electrician would be glad to let him practice under his own
supervision, the way he does with me. But he doesn’t want either of
us to send when he isn’t present. He’s afraid we might burn out
something. I don’t want to knock Black, but I’d be afraid to have him
monkey with any machine I was responsible for myself. He’s the
most careless, reckless fellow I ever saw.”
“But why should he take such an apparent dislike to me?”
demanded Henry. “I’m a complete stranger to him. He hasn’t any
possible reason for disliking me.”
“He thinks he has,” said Belford.
“He does? What is it?”
 “Why, Mr. Sharp let you send a message on the wireless outfit
yourself.”
“But Mr. Sharp was right beside me, to make sure I did no harm.”
“I know it, but that doesn’t make any difference with a fellow like
Black. He dislikes you very much. He tried to tell me a long story
about it. And to have you put over him as chief is more than he can
stand. He’s a bad egg, I believe. And I’d watch him closely if I were
you.”
“Of course I’ll watch him, if he’s likely to shirk his duty,” said
Henry. “He might get the ship into trouble.”
“I don’t mean to watch him in that way. Watch him on your own
account. I don’t know that he’d really do anything to anybody. But
he’s always talking about fixing this fellow or that fellow. He might
try to do you some harm.”
Henry smiled. “There’s little danger,” he said. “We’ll be in New
York in a couple of days, and I’ll probably never see him again after
that.”
But though Henry smiled at the idea of the young operator’s doing
him harm, he could not dismiss from his mind so easily the feeling
that had come to him of uneasiness for the boat itself. He got his
nightclothes from the captain’s cabin, undressed, and slipped into
the chief operator’s bunk. But weary though he was, he could not
sleep. He was worried, and worry was almost a new sensation to
him. He could not at first understand it. He would be almost asleep,
then he would wake up with a start, his leg or arm jerking nervously.
It was partly the reaction from the long strain he had been under.
That sort of experience was absolutely new to him, and he could not
forget the horrible things he had witnessed. But gradually he came
to realize that there was something more to his restlessness than
the reaction from nervous strain. Something was preying on his
mind. After he had tossed sleeplessly on his mattress for a long
time, he comprehended what was the matter. He was worried about
the wireless.
This was the first time in his life that Henry had ever been placed
in a situation of great responsibility. The responsibility was very real,
too, for the lad understood that in large measure the safety of the
ship now depended upon him. Much more than his own life was at
stake. The lives of all the men left on the cutter, and the safety of
the ship herself, might at any moment become absolutely dependent
upon him. If no danger threatened aboard the Iroquois, there might
yet be other lives and other ships endangered, and upon how well
his instruments were cared for, and how thoroughly the watch was
kept, would depend the possibility of catching those far-flung cries
for help.
Had Henry been put in charge of the wireless thus suddenly and
unexpectedly, under less harrowing circumstances, he doubtless
would have felt the responsibility far less poignantly. But for hours
past he had been watching the fight against death, and even death
itself. He was nervously keyed up to a fearfully high pitch. His nerves
would not let down. Doubtless it was the sense of responsibility that
kept them taut. Again and again he told himself that it was his
business to go to sleep and rest, so that he might be fit for duty
when his turn came. But his arguments had no effect upon his
nerves. He was unable to sleep.
For an hour or two he tossed in his bunk restlessly. Again and
again he fell asleep, only to wake a few minutes later. The situation
preyed on him. He could not keep his thoughts away from the radio
shack. Suppose all was not right there. Suppose young Black was
taking this time to defy orders and fool with the wireless key.
Suppose when Henry woke up he should find the wireless out of
commission. And suppose the Iroquois should break her rudder, or
crack her shaft. She might, with such a strain on it, in such a sea.
Suppose the captain should go wrong in his reckoning and the cutter
should pile up on the sands, as the Capitol City had done. Suppose—
suppose——A hundred terrifying suppositions came into Henry’s
mind. Finally he could stand them no longer. He rose, drew on his
clothes, and made his way toward the radio shack. Light was
beginning to illumine the eastern sky. The sea still heaved violently,
but the wind had lost most of its force. Peace was returning to the
troubled world.
 Then Henry stepped inside of the radio shack, and stood as
though petrified. Young Black lay back in his chair, his eyes tightly
closed, snoring loudly. He was sound asleep. For all that Henry
knew, he might have been asleep for a long time.
“Black,” said Henry sternly, touching the sleeping wireless man on
the shoulder. When the sleeper merely grunted, Henry said, still
more sternly, “Wake up!” And this time he shook the sleeper
vigorously.
Slowly young Black opened his eyes. A savage oath burst from his
lips as he saw who was standing before him and realized his
situation. Then a crafty look came into his eyes. He laughed.
“Fooled you that time, didn’t I?” he chuckled. “You thought I was
asleep.”
“No, you didn’t fool me,” said Henry. “I don’t think you were
asleep. I know it. Put on your headphones and attend to your
business.”
All the ill nature in the other lad’s being leaped to the front. “You
spy!” he snarled, with an oath. “So that’s the sort of chief radio man
we’ve got, is it? A sneaking spy!”
For a moment it was all Henry could do to keep from hitting the
lad. But he took a grip on himself and for a moment made no
answer. Then he said: “I shall report this matter to the captain as
soon as possible. You may go to your bunk. I will finish your watch
myself.”
Slowly the third-class radio man arose and left the shack. At the
door he turned and faced Henry. “You sneaking spy,” he said, with
another oath, “if you tell the captain, I’ll fix you as sure as my name
is Black.”
Welcome to Our Bookstore - The Ultimate Destination for Book Lovers
Are you passionate about books and eager to explore new worlds of
knowledge? At our website, we offer a vast collection of books that
cater to every interest and age group. From classic literature to
specialized publications, self-help books, and children’s stories, we
have it all! Each book is a gateway to new adventures, helping you
expand your knowledge and nourish your soul
Experience Convenient and Enjoyable Book Shopping Our website is more
than just an online bookstore—it’s a bridge connecting readers to the
timeless values of culture and wisdom. With a sleek and user-friendly
interface and a smart search system, you can find your favorite books
quickly and easily. Enjoy special promotions, fast home delivery, and
a seamless shopping experience that saves you time and enhances your
love for reading.
Let us accompany you on the journey of exploring knowledge and
personal growth!

ebookgate.com