Annem Emine Parlar'a ve rahmetli babam Sait Parlar'm amsma

To my mother and to the memory of my father

 Mahmut Parlar

Interactive Operations Research

with Maple
Methods and Models

Springer Science+Business Media, LLC

Mahmut Parlar
DeGroote School of Business
McMaster University
Hamilton. Ontario. L8S 4M4
Canada

Library of Congress Cataloging-in-Publication Data

Parlar.~ut. 1952-
Interactive operations research with Maple: methods and models! Mahmut Parlar.
p. cm.
Includes bibliographical references and index.
ISBN 978-1-4612-7110-9 ISBN 978-1-4612-1356-7 (eBook)
DOI 10.1007/978-1-4612-1356-7
1. Operations research-Data processing. 2. Maple (Computer file) I. Title.

T57.6.P38 2000
658.4'034'0285-dc21 00-031232
CIP

AMS Subject Classifications: 9OC05, 9OC15. 9OC30, 9OC39. 9OC40. 9OA09

Printed on acid-free paper

@ 2000 Springer Science+Business Media New York
Originally published by Birkhauser Boston in 2000
Softcover reprint of the hardcover 1st edition 2000

All rights reserved. This work may not be translated or copied in whole or in part without the written
permission of the publisher Springer Science+Business Media, LLC,
except for brief excerpts in connection with reviews or
scholarly analysis. Use in connection with any form of information storage and retrieval, electronic
adaptation, computer software, or by similar or dissimilar methodology now known or hereafter
developed is forbidden.
The use of general descriptive names, trade names, trademarks, etc., in this publication, even if the
former are not especially identified, is not to be taken as a sign that such names, as understood by the
Trade Marks and Merchandise Marks Act, may accordingly be used freely by anyone.

ISBN 978-1-4612-7110-9

Typeset by the author in ~1l;:X.

Cover design by Joseph Sherman, New Haven, CT.

9 8 7 6 5 4 321
Contents

1 Introduction to Operations Researcb 1

I.l A Brief History . . . . . . . . . . . . . . . . . . . . ..
. . I . . .
1.2 The Method of OR . . . . . . . . . . . . . . . . . . ... . 2. . .
1.3 About This Book . . . . . . . . . . . . . . . . . . ... . .4 . . .

2 A Qukk Tour of Maple 5

2.1 Introduction............................. 5
2.2 Symbolics........................ .. . . . . 6
2.2.1 Expanding, Factoring and Simplifications . . . . . .... 6
2.2.2 Function Definitions Using "- >" and unappl y ( ) ... 8
2.2.3 Lists, Sets and Arrays . . . . . . . . . . . . . . . . . . . 9
2.2.4 Additional Commands to Manipulate Expressions . . . . 12
2.2.5 Solving Equations Analytically .. . . . . . . . . . . . . 14
2.3 Numerics . . . . . . . . . . . . . . . . . . . . . ... . IS
. . . . .
2.3.1 Solving Equations Numerically . . . . . . . . . . . . . . 16
2.3.2 Statistical Computations Using the stats Package. . .. 17
2.3.3 The simplex Package . . . . . . . . . . . . . . . . . . 19
2.4 Graphics...... . . . . . . . . . . . . . . . . . . . . . . . .. 19
2.4.1 Two-dimensional Plots. . . . . . . . . . . . . .. . . .20.
2.4.2 Three--Dimensional Plots . . . . . . . . . . . . . . . . . 24
2.5 Other Useful Commands and Packages. . . . . . . . . . .... 27 .
2.5.1 The piecewise () Command . . . . . . . . . .... . 27
vi Contents

2.5.2 The interface () Command . . . . . . . . . .... 29 .

2.5.3 The infolevel[] Command . . . . . . . . . .... 30.
2.5.4 Exporting to C, FORTRAN and LaTeX . . . . . . . . .31.
2.5.5 Programming in Maple . . . . . . . . . . . . . .. . . 32. .
2.5.6 The finance Package . . . . . . . . . . . . . . . . . . 33
2.6 Summary . . . . . . . . . . . . . . . . . . . . . . . 35 . . . . . . .
2.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

3 Maple and Mathematical Foundations of Operations Research 37

3.1 Introduction... . . . . . . . . .. . . . . . . . . . 37
3.2 Algebra . . . . . . . . . . . . . . . . . . . . . . . . 38
3.2.1 Solution of a Single Equation or Inequality . . . . . ... 38
3.2.2 Solution ofa System of Nonlinear Equations .. . . ... 41
3.3 Calculus.......... .. . . . . . . . . . . . . . . . . . . . 44
3.3.1 Limits..... . . . . . . . . . . . . . . . . . . . . . . . 44
3.3.2 Derivatives......................... 45
3.3.3 Integrals....... .. . . . . . . . . . . . . . . . . . . 49
3.3.4 Finite Sums and Infinite Series . . . . . . . . . . . ... . 54
3.4 Linear Algebra . . . . . . . . . . . . . . . . . . . . .... .55 . .
3.4.1 Matrix Operations . . . . . . . . . . . . . . . . . . . . . 55
3.4.2 Solution of Simultaneous Linear Equations . . . . . ... 58
3.4.3 Eigenvalues, Eigenvectors and Diagonalization . . . .. . 61
3.4.4 Least Squares Fitting to Data. . . . . . . . . . . . .. . 63 .
3.4.5 Special Matrices . . . . . . . . . . . . . . . .... .66 . .
3.4.6 Positive Definiteness . . . . . . . . . . . . . . .. . . 67
. .
3.5 Differential Equations. . . . . . . . . . . . . . . . . . .. . 68 . . .
3.5.1 Ordinary Differential Equations . . . . . . . . . . .. . 69 .
3.5.2 Partial Differential Equations. . . . . . . . . . . . .. . 77 .
3.5.3 Difference (Recurrence) Equations. . . . . . . . . ... . 79
3.6 Transform Methods. . . . . . . . . . . . . . . . . . ... . 81 . . .
3.6.1 Laplace Transforms. . . . . . . . . . . . . . . .. . . .81.
3.6.2 Generating Functions. . . . . . . . . . . . . . .... 85 . .
3.7 Probability.............................. 88
3.7.1 Continuous and Discrete Random Variables. . . . . . .. 88
3.7.2 Expectation......... . . . . . . . . . . . . . . . . 92
3.7.3 Jointly Distributed Random Variables . . . . . . . . .. . 95
3.8 Summary . . . . . . . . . . . . . . . . . . . . . .... . 96 . . . .
3.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

4 Linear Programming 99
4.1 Introduction.................... .. . . . . . . . 99
4.2 Graphical Solution . . . . . . . . . . . . . . . . . . . . . . . . . 100
4.3 The Simplex Method . . . . . . . . . . . . . . . . . . . . . . . .
104
4.3.1 Manual Solution of the Production Problem . . . . . . . 107
4.3.2 Solution Using Maple's simplex Package . . . . . . . . 113
 Contents vii

4.3.3 A Problem with Mixed Constraints . . . . . . . . . . . . 116

4.4 Special Cases and Difficulties . . . . . . . . . . . . . . . . . . . . 118
4.4.1 Infeasibility . . . . . . . . . . . . . . . . . . . . . . . . . 118
4.4.2 Unbounded Solutions. . . . . . . . . . . . . . . . 119 . . . .
4.4.3 Degeneracy . . . . . . . . . . . . . . . . . . . . . . . . . 119
4.5 Other Examples . . . . . . . . . . . . . . . . . . . . . 120 . . . . . .
4.5.1 The Transportation Problem . . . . . . . . . . . . . . . . 121
4.5.2 Two-Person Zero-Swn Games . . . . . . . . . . . . . . . 124
4.5.3 A Linear Program with Randomly Generated Data . . . . 129
4.6 Sensitivity Analysis and Duality . . . . . . . . . . . . . . 130 . . . .
4.6.1 Changes in the RHS Values and Dual Prices . . . . . . . 130
4.6.2 Changes in the Objective Function Coefficients . . . . . . 133
4.6.3 Addition of a New Decision Variable . . . . . . . . . . . 135
4.6.4 D u a l i t y . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
4.7 Integer Linear Programming . . . . . . . . . . . . . . . . . . . . 145
4.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . • . . . . . 147
4.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148

5 Nonlinear Programming 153

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
5.2 Convexity of Sets and FWlctions . . . . . . .. . . . . . . . . . . 154
5.2.1 Positive and Negative Definite Matrices . . . . . . . . 157 . .
5.2.2 Convexity ofa Function and Definiteness of its Hessian . 160
5.2.3 Examples of DefinitenessiConvexity . . . . . . . . . . . . 160
5.2.4 Cholesky Factorization . . . . . . . . . . . . . . . . . . . 164
5.3 Unconstrained Optimization . . . . . . . . . . . . . . . . . . . . 166
5.4 Inequality and Equality Constrained Optimization . . . . . . . . 170
5.4.1 Geometric Interpretation of Kuhn-Tucker Conditions . . .173
5.4.2 Constraint Qualification . . . . . . . . . . . . . . . . . . 174
5.4.3 An Equivalent Formulation Using the Lagrangian . . . . 176
5.4.4 Economic Interpretation of the Lagrange Multipliers . . . 181
5.5 Lagrangian Duality . . . . . . . . . . . . . . . . . . . . . . . . . 183
5.6 Summary . . . • • . • . • . . . . . . . . . . . . . . . . • • . . . 190
5.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190

6 Dynamic Programming 193

6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
6.2 Stagecoach Problem. . . . . . . . . . . . . . . . . . . . 197 . . . . .
6.3 Models with a Linear System and Quadratic Cost . . . . . . . . . 201
6.3.1 The Infinite-Stage Problem . . . . . . . . . . . . . . . . . 207
6.4 Continuous-Time Dynamic Programming . . . . . . . . . . 211. . .
6.4.1 A Problem with Linear System and Quadratic Cost . . . . 214
6.4.2 A Problem with Quadratic and Linear Costs . . . . . . . . 215
6.5 A Constrained Work Force Planning Model . . . . . . . . . 219 . . .
6.6 A Gambling Model with Myopic Optimal Policy . . . . . . . . . 224
viii Contents

6.7 Optimal Stopping Problems . . . . . . . . . . . . . . . . . . . . . 228

6.7.1 The Infinite-Stage Problem . . . . . . . . . . . . . . . . . 230
6.8 Summary . . . . . . . . . . . . . . . . . . . . . . . 233
. . . . . . .
6.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . 233
. . . . . . .

7 Stochastic Processes 137

7.1 Introduction .. . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
7.2 Exponential Distribution and Poisson Process .. .. .. . . . . . 238
7.2.1 Memoryless Property of the Exponential Distribution ... 239
7.2.2 Hazard Rate Function .. . . . . .. . .. . . . . . . . . 240
7.2.3 The Erlang Random Variable . . . . . . . . .. . . . . . 241
7.2.4 Poisson Process. . . . . . . . . . . . . . . . . . . . 243
. . .
7.2.5 Interarrival Times of the Poisson Process . . . . . . . . . 246
7.2.6 Density of the Erlang-An Equivalent Derivation . . .. 246
7.2.7 Nonhomogeneous Poisson Process . . . . . . . . . . . . . 247
7.2.8 Compound Poisson Process . . . . . . . .. .. . . . . . 251
7.3 Renewal Theory . . .. . .. . .. . . . . . . . . . . . . . . . . . 256
7.3.1 Preliminaries .. . . . .. . . . . . . . . .. ... . . . . 257
7.3.2 Distribution of N(t) .. . . . . . . . . . . . . . . . . . . 257
7.3.3 Renewal Function and Renewal Density . . . . . . . . .. 259
7.3.4 Renewal Equation and the Key Renewal Theorem . . . . . 262
7.3.5 Renewal Reward Process . . . . . . . . . . . . . 268
.
7 .4 Discrete-Time Markov Chains . . . . . . . . . . . . . . .271 . . . .
7.4.1 Chapman-Kolmogorov Equations . . . . . . . . . . .. . 276
7.4.2 Limiting Probabilities and the Stationary Distribution .. 278
7.4.3 Classification of States . . . . . . .. . . . . . . . . . . . 287
7.4.4 Imbedded Markov Chain Technique . . . . . . . . . . . . 293
7.4.5 Transient Behavior of Markov Chains . . . . . .... 301
7.5 Continuous-Time Markov Chains. . . . . . . . . . . . . . 305 . . . .
7.5.1 Kolmogorov Differential Equations . .. . . . . . . . .. 308
7.5.2 Limiting Probabilities .. . . . . . . . . . . . . . . . . . 318
7.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327
7.7 Exercises . . . . . . . . ... . . . . . . . . . . . . . . . . . . . 327

8 Inventory Models 331

8.1 Introduction . . . .. .. .. . .. . .. . . . . . ... . . . . . . 331
8.2 Classification ofInventory Models . . . . . . . . . . . . . . 332 . . .
8.3 Costs Associated with Inventory Models . . . . . . . . . . . 334 . . .
8.3.1 Procurement Cost. . . . . . . . . . . . . . . . . 334 . . . . .
8.3.2 Holding (or Carrying) Cost . . . . . . . . . . . . . . . . . 334
8.3.3 Penalty Cost . . . .. . .. . . . . . . . . . . . . . . . . 334
8.4 Deterministic Inventory Models .. . . . . . . . .. . . . . . . . 335
8.4.1 The Basic EOQ Model . . . . . . . . . . . . . . .335 . . . .
8.4.2 The EOQ Model with Planned Backorders . . . . . . . . 338
8.4.3 Analysis of Implied Backorder Costs . . . . . .. . .. . 343
 Contents ix

8.4.4 Quantity DiscoWlts . . . . . . . . . . . . . . . . 346 . . . . .

8.5 Probabilistic Inventory Models . . . . . . . . . . . . . . 349 . . . . .
8.5.1 The Continuous-Review Model: Approximate Fonnulation 349
8.5.2 The Continuous-Review Model: Exact Fonnulation . . .. 354
8.5.3 One-Period (Newsvendor) Model with Random Demand . 358
8.5.4 Dynamic Inventory Models . . . . . . . . . . . . . . . . 361
8.5.5 Diversification Under Yield Randomness .. . . . . . . 366 .
8.6 Summary . . . . . . . . . . . . . . . . . . . . . . . .371
. . . . .
8.7 Exercises .. . . . . . . . . . . . . . . . ... . . . . . . . . . . 371

9 Queueing Systems 375

9.1 Introduction........ . . . . .. .. . . . . . . . . . . . . . 375
9.2 Markovian Queueing Systems . . . . . . . . . . . . . . . . . . . 377
9.2.1 Birth and Death Processes . . . . . . . . . . . . . . . . . 377
9.2.2 MI MIl Queueing System . . . . . . . . . . . . . . . . . 378
9.2.3 Finite Capacity Markovian Queue: MI MIll K . . . . . . 382
9.2.4 MuitiserverQueue MIMIc . . . . . . . . . . . . . . . . 383
9.2.5 Markovian Bulk Arrival System: M X I MIl . . . . . . . . 387
9.3 Transient Solutions . . . . . . . . . . . . . . . . . . . . . . . . . 389
9.3.1 Finite Capacity MI MIll K Queue with K = I . . . . . . 389
9.3.2 Ample Server System MIMloo . . . . . . . . . . . . . . 391
9.4 Queueing Networks . . . . . . . . . . . . . . . . . . . . . . . . . 394
9.4.1 Serial Queue with Blocking .. . . . . . . . . . . .394 . . .
9.4.2 Jackson Networks . . . . . . . . . . . . . . . . . . . . . 398
9.5 Optimization of Queueing Systems . . . . . . . . . . . . . . . . . 401
9.5.1 A Transportation Queueing Process . . . . . . . . . . . . 402
9.5.2 An MI MIl System with Controlled Arrivals . . . . . . . 404
9.5.3 Optimal Dynamic Service Rate Control . . . . . . . . . . 406
9.6 Summary .. ••.• . . . . . • . . . . . . . . ••...••..• 413
9.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413

10 Simulation 417
10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417
10.2 Generating (pseudo-) Random Numbers . . . . . . . . . . .423 . . .
10.2.1 Mixed-Congruential Method . . . . . . . . . . . . . . . . 424
10.2.2 Maple's Unifonn Random Number Generator . . . . . . . 426
10.2.3 Kolmogorov-Smimov Test for Unifonnity . . . . . . . .426 .
10.3 Generating Random Variates from Other Distributions . . . . . . 428
10.3.1 Exponential Random Variates . . . . . . . . . . . . . . . 429
10.3.2 Maple's Random Variate Generators . . . . . . . . . . . . 430
10.4 Monte Carlo Simulation . . . . . . . . . ... ... . . . . . . . 432
10.4.1 Numerical Evaluation of Definite Integrals ... ... .. 432
10.4.2 Simulation of a Static (Single-Period) Problem . . . . . 434 .
10.5 Dynamic Simulation Models . . . . . . . . . . . . . . . . . . . . 435
10.5.1 Simulation of an Inventory System with Random Yield . . 436
x Contents

10.5.2 Simulation ofa Non-Markovian Queue . . . . . . . . .. 438

10.6 Optimization by Random Search . . . . . . . . . . . . . . 441 . . . .
10.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . 445
10.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446

References 449

Index 463
Preface

Interactive Operations Research with Maple: Methods and Models has two ob-
jectives: to provide an accelerated introduction to the computer algebra system
Maple and, more importantly, to demonstrate Maple's usefulness in modeling and
solving a wide range of operations research (OR) problems.
This book is written in a format that makes it suitable for a one-semester course
in operations research, management science, or quantitative methods. A nwnber
of students in the departments of operations research, management science, oper-
ations management, industrial and systems engineering, applied mathematics and
advanced MBA students who are specializing in quantitative methods or opera-
tions management will find this text useful. Experienced researchers and practi-
tioners of operations research who wish to acquire a quick overview of how Maple
can be useful in solving OR problems will find this an excellent reference.
Maple's mathematical knowledge base now includes calculus, linear algebra,
ordinary and partial differential equations, nwnber theory, logic, graph theory,
combinatorics, statistics and transform methods. Although Maple's main strength
lies in its ability to perform symbolic manipulations, it also has a substantial
knowledge of a large nwnber of nwnerical methods and can plot many different
types of attractive-looking two-dimensional and three-dimensional graphs. After
almost two decades of continuous improvement of its mathematical capabilities,
Maple can now boast a user base of more than 300,000 academics, researchers
and students in different areas of mathematics, science and engineering.
My goal in writing this book has been to show that Maple can be immensely
useful in the type of modeling work that is common in operations research, man-
agement science and related disciplines of operations management and industrial
and systems engineering. There are several excellent texts on OR and on Maple,
xii Preface

but none of them describe the use of Maple in solving OR problems. This book
fills the gap and provides the missing link between Maple and its successful use
in OR.
Chapter I starts with an introduction to operations research. Chapter 2 is an
accelerated introduction to Maple that covers the most important commands that
are relevant in OR applications. This is followed in Chapter 3 by a treatment of
some of the mathematical techniques useful in OR modeling: algebra and calcu-
lus, ordinary and partial differential equations, linear algebra, transform methods
and probability theory.
Chapter 4 discusses linear programming, one of the oldest and perhaps the
most frequently used OR technique. In this chapter Maple's symbolic manipula-
tion capabilities are not used since linear programming problems must be solved
numerically. After describing the graphical method to solve linear programming
problems (which is simplified by using Maple's capability to plot regions of in-
equalities) we proceed to a discussion ofthe simplex method, which is algebraic in
nature. Maple's own implementation of the simplex method is also demonstrated
and integer linear programming is mentioned briefly.
Chapter 5 presents an exposition of nonlinear programming and many prob-
lems in unconstrained and constrained optimization are solved explicitly. Chapter
6 covers dynamic programming, in which Maple can be useful in finding opti-
mal policies for dynamic optimization problems in closed form. After describing
a simple numerical example (the "stagecoach problem"), several models with a
linear system and quadratic costs in both discrete and continuous time are exam-
ined. An exact solution of a workforce planning model is followed by the analysis
of some stochastic dynamic programming models, including a gambling problem
that gives rise to a myopic policy. The chapter ends with a discussion of stopping
time problems. Stochastic processes is presented in Chapter 7 where exponen-
tiaVPoisson duality, renewal theory and discrete- and continuous-time Markov
chains are described in detail. These chapters amply demonstrate the success of
Maple in obtaining analytic solutions to many OR models.
Chapters 8 and 9 are concerned with the application of OR methods to inven-
tory management and queueing models, respectively. Here, Maple's symbolic ma-
nipulation capabilities are used extensively to obtain optimal solutions in c1osed-
form. For example, in the chapter on inventory models, implied backorder costs
and optimal base-stock policy for a dynamic stochastic inventory problem are
computed explicitly. In the chapter on queueing models, the operating charac-
teristics of Markovian bulk arrival systems and the transient solutions for some
queues are found in closed form. Where symbolic analysis becomes impossible
to use, we employ Maple's extensive knowledge of numerical analysis. For ex-
ample, solution of the exact model for the continuous-review inventory problem
with backorders requires intensive numerical computations. Similarly, dynamic
optimization of a Markovian queue involves solving differential equations nu-
merically. These computations are performed easily by stringing together a few
lines of Maple code.
 Preface xiii

In Chapter 10, Simulation, Maple's ability to generate random variates from

nearly 20 probability distributions is used to find solutions to different problems.
For example, simulation is used to compute a definite integral that could not be
evaluated in closed form. In this chapter we also describe the famous Monty Hall
"car and goats" problem and present its counterintuitive solution by simulation.
Simulation is frequently used to analyze the properties of dynamic systems that
evolve probabilistically over time. We use Maple's random variate generators to
simulate an inventory system with random yield and find the distribution of in-
ventory level. We also simulate a non-Markovian (G / G/1) queueing system and
compute the average waiting time in the queue.
Each chapter (except Chapter 1) ends with a summary and numerous exercises.
A Solutions Manual for the exercises will be provided to instructors who adopt
the book for their course.

Internet Resources
Every Maple worksheet displayed in the book has been tested with Maple RS.l
and R6. These worksheets can be downloaded from my personal Web page at the
addresswww.business.mcmaster.ca/msis/profs/parlar/.
Waterloo Maple's Web site www.maplesoft.com/is always worth a look
for announcements of new products, demos and Maple support information.
The following yahoo.com page has links to interesting sites concerned with
Maple: dir. yahoo. com/Science/Mathematics/Software/MAPLE.
Readers interested in following the lively Internet discussions on symbolic al-
gebra packages such as Maple and Mathematica may wish to peruse the sci.
math. symbolic newsgroup. A recently established newsgroup that carries
Maple-specific postings is comp. soft-sys .math . maple.

Acknowledgments
During the last three years of writing this book, I received feedback from many
friends and colleagues. First and foremost, I would like to thank Ray Vickson
(University of Waterloo) who encouraged me to undertake this project and pr0-
vided useful comments on early drafts. Martin von Mohrenschildt (McMaster
University) saved me many hours of work by quickly directing me to the correct
Maple commands to solve some of the problems presented in the book. Elkafi
Hassini (University of Waterloo) read many chapters and sent comments. George
Wesolowsky (McMaster University), Paul Stephenson (Acadia University) and
Mustafa Karakul (University of Toronto) are friends and colleagues who took the
time to read parts of the manuscript and comment on its style.
I consider myself fortunate in having acquired my early training in OR under
the guidance of two eminent operations researchers: I took many OR courses
xiv Preface

from Halim DogrusOz and izzet ~ahinl who, in the mid-l 970s, were professors
in the graduate program in Operations Research and Statistics at the Middle East
Technical University, Ankara. I learned from Dr. DogrusOz that there is more
to OR than just high-powered mathematics, and I learned from Dr. ~ahin (who
directed my master's thesis on renewal theory) that there is more to modeling
than just deterministic analysis. I am grateful to both.
My editors Ann Kostant and Tom Grasso at Birkhluser have always been very
encouraging and positive about this project and I thank them for that.
Finally, a big thank you to my wife and children for their understanding during
the long process of writing this book.
I would be delighted to hear from readers who have found the Maple work-
sheets contained in this book useful in their OR-type work.

MAHMUT PARLAR
[email protected]
www.business . mcmaster.ca/msis/profs/parlar/

May 2000
Hamilton, Canada

1Deceased. (Nur ifinde )'Orsin.)

Interactive Operations Research
with Maple
Methods and Models
1
Introduction to Operations Research

1.1 A Brief History

The activity called operations research (OR) was developed during World War
II when leading British scientists were consulted to improve military strategy
against the enemy. A team of physicists, mathematicians, statisticians, engineers
and sometimes even psychologists and chemists conducted research on military
operations under the leadership of P. M. S. Blackett. I These scientists worked
on important military problems such as the deployment of radar, management
of bombing and antisubmarine operations and military logistics. Many of these
problems were solved by applying the scientific method and using mathematical
techniques such as algebra, calculus and probability theory.2
The news of the successful application of scientific method to military opera-
tions spread quickly, and the American military began establishing its own OR
teams, which were instrumental in the successful supply and deployment of U.S.
forces in the Pacific.
After the war ended, many of the scientists who worked on military OR were
discharged from war duty, and they began applying the scientific method and OR

I This team was nicknamed "Blackett's Circus."

2For example, one interesting problem involved the maximization of the probability of hitting
an enemy submarine. It was found that if a depth setting of a charge was made at 3S feet below
the surface of water (rather than the usual I ()() feet), then the probability of hitting the submarine
would be maximized. It was reported that implementing this solution increased the casualty of enemy
submarines by about 100"10. For a more detailed description of these military-related problems and the
history of OR, see Beer [20] and Singh [176].

M. Parlar, Interactive Operations Research with Maple

© Birkhäuser Boston 2000
2 I. Introduction to Operations Research

techniques to the solution of industrial problems. Textbooks began appearing in

the 1950s3, and OR was recognized as a fonnal academic SUbject. At the same
time many universities started offering courses leading to graduate degrees in OR.
Societies such as the Operations Research Society of America, the Institute of
Management Sciences (USA),4 the Canadian Operational Research Society, The
Operational Research Society (England), Societe Fran~aise de Recherche Opera-
tionelle (France) and Centro per la Ricerca Operativa (Italy) were fonned. These
societies also began publishing their own journals describing the theory and ap-
plications of OR to a wide variety of problems.
Since the publication of the early books on OR almost 50 years ago, opera-
tions researchers have written thousands of papers describing the new theoretical
developments and practical applications of OR in different areas. These articles
have appeared in journals such as Operations Research, Management Science,
Interfaces, Mathematics of Operations Research, Naval Research Logistics, /IE
Transactions, Manufacturing and Service Operations, European Journal of 0p-
erational Research, INFOR and Decision Sciences. Interfaces regularly publishes
articles on the real applications of OR. For example, the November-December
1998 issue (Volume 28, Number 6) of Interfaces has articles on optimizing army
base realignment (using integer linear programming), optimal decisions and the
fourth-and-goal conferences at the end of a close football game (using dynamic
programming) and trash-flow allocation (using linear programming under uncer-
tainty). In the May-June 1998 issue of the same journal one can read about the
application of OR techniques to modeling AIDS, and in the March-April 1998
issue one can learn about the impact of the 1990 Clean Air Act Amendments and
an efficient technique for selecting an all-star team.

1.2 The Method of OR

Operations research can be defined as the scientific study of operations in an or-
ganization. Its object is to understand and predict the behavior of an organization
(i.e., a system) by observation, experimentation and analysis in order to improve
the behavior of the organization. This is achieved by first/ormulating the problem
facing the organization. For example, a retailer may realize that the average level
of inventory for an item is consistently high resulting in high inventory carrying

3The first fonnal OR textbook was written by Churchman, AckotT and ArnotT [45J and appeared
in 1958. This was followed by Sasieni, Yaspan and Friedman's book [164J, published in 1959. A
collection of notes on OR topics (such as probability, search theory, Markov processes, queueing,
sequential decision processes and production scheduling) assembled by the OR Center at MIT [139J
based on a summer program of OR lectures also appeared in 1959.
4In the 19905, The Operations Research Society of America and The Institute of Management
Sciences merged to fonn INFORMS (Institute for Operations Research and the Management Sci-
ences), which has a total of approximately 12,000 members including some Nobel prize winners; see
www. informs. org.
 1.2 The Method of OR 3

costs. In this case, an OR analyst would fonnulate the problem while interacting
with the decision maker (i.e., the retailer). This would involve specifying the ob-
jectives (e.g., cost minimization) and identifying those aspects of the inventory
system that can be controlled by the retailer (e.g., magnitude and frequency of or-
ders placed). Problem fonnulation is perhaps similar to a medical diagnosis since
in this stage of an OR study, one usually observes the symptoms (such as high
inventory) and then attempts to make a diagnosis of the problem (such as very
large order quantities).
The next step involves the construction 01a model that represents the behavior
of the system. In OR, mathematical models usually assume the fonn V = I(x, y)
where V is the perfonnance measure (value function) of the organization, sys-
tem or decision maker under study (e.g., average cost, profit per month, etc.). The
set x = {XI, ... , xm} is the set of variables that can be controlled by the deci-
sion maker, y = {YI, . .. , Yn} is the set of parameters that nonnally cannot be
controlled by the decision maker and 1 is the relationship between V and (x, y).
Additionally, there may be equality and/or inequality constraints limiting the val-
ues that can be assumed by the decision variables.
In the inventory example, the OR analyst attempting to solve the retailer's prob-
lem constructs a mathematical model of the inventory system and the ordering
policy used by the retailer. As a very simple example, if the annual demand A for
the item is detenninistic and unifonn, a reasonable model of the average annual
cost C may be expressed as C = K AI Q + !h Q where K is the (fixed) cost of or-
dering, Q is the order quantity (the decision variable) and h is the cost of holding
inventory per unit per year. Once such a mathematical model is constructed, it be-
comes relatively easy to use calculus to find the optimal order quantity that would
minimize the cost function C. In this example, Q is the decision variable, K, A,
and h are the uncontrollable parameters in the problem and C is the perfonnance
measure of the system.
Assuming that the model is properly validated (i.e., that it does represent the
system under study reasonably accurately), the OR analyst must derive a solution
from the model. In the simple case discussed here, in order to optimize (i.e., min-
imize) the average cost, one can use calculus and find the optimal order quantity.
This involves differentiating the cost function, equating the result to zero and solv-
ing for the unknown decision variable Q. As we will show in Chapter 8, Inventory
Models, the optimal solution Q. for this model is

and the minimum average annual cost under this decision is C· = J2AK h.
In the next stage of an OR analysis, an evaluation 01 the solution is perfonned
and the solution is compared to the policy or decision it is expected to replace. At
this stage, the analysts can highlight the differences between the current practice
and the recommended optimal solution. In the inventory example, the analysis
may reveal that under the current policy, the retailer was ordering unnecessarily
4 1. Introduction to Operations Research

large amounts (say, 2Q* units instead of the optimal Q* units) that resulted in
high inventories and carrying costs.
In the final stage of an OR study, the solution is implemented in order to im-
prove the performance of the system. For example, in the inventory problem, the
order decision is revised to Q* (say, down from the nonoptimal 2Q*) to reflect
the optimality of the solution found. This assumes that the decision makers have
accepted the results of the study and are willing to revise their current policy.
Normally, the five stages of an OR study are initiated in the order given. But
very often each stage interacts with others before the termination of the study. For
example, the analyst solving the inventory problem may discover that the demand
is not quite deterministic but exhibits a fair degree of randomness. In that case, the
model construction stage must be revisited and a more accurate model reflecting
the randomness of demand must be constructed.
These nontechnical and philosophical--but important-issues were discussed
and debated in great detail in the 1950s and 1960s by OR pioneers such as Ackoff
and Churchman. For their views on problem formulation, model construction,
systems approach and so on, see [3], [4] and [44].

1.3 About This Book

This book is about the computer algebra system Maple and its use in solving OR
problems using mathematical models. There are no prerequisites for using the
Maple component of the book other than the ability to start the Maple program
installed in a computer. All necessary commands for Maple symbolics, numerics
and graphics are summarized with examples in Chapter 2. However, in order to
understand the OR techniques and applications such as optimization, stochastic
processes, inventory models and queueing processes, the reader will need some
familiarity with calculus, probability and linear algebra. Thus, it is assumed that
the reader has such a background, but in Chapter 3 we do provide a quick review
of these important topics that will be relevant in the subsequent chapters.
2
A Quick Tour of Maple

2.1 Introduction
Maple is a mathematical problem-solving environment that combines symbolic,
numerical and graphical methods used in different areas of mathematics. This
chapter provides an introduction to Maple and presents those aspects of it that can
be useful in operations research applications.
Maple has more than 2700 commands and nearly 40 mathematical packages
(such as linear algebra, integral transforms, Boolean logic, numerical approxima-
tions, financial mathematics and algebraic curves) that can be used to solve prob-
lems from basic arithmetic to general relativity theory. As we will see through-
out this book, the solution of many challenging operations research problems be-
comes extremely easy by loading a package and simply combining a few Maple
commands.
Maple has an extensive on-line help facility. To obtain help about a command,
it is sufficient to type a question mark and then the name of the command. For
example, typing? in t (and pressing the < ENTER> key) provides information on
definite and indefinite integration, including a large number of examples where
the in t command is used. Readers who are new to Maple can take a ''tour'' by
clicking on Help and then New User's Tour on the Maple menu bar.
We start this chapter by discussing some aspects of basic symbolic computa-
tions and expression manipulations such as simplifications. This is followed by
numerical computations such as finding roots, computing definite integrals, solv-
ing differential equations. In many cases important insights about a problem can
be obtained by a graphical representation of, for example, the implicit curves

M. Parlar, Interactive Operations Research with Maple

© Birkhäuser Boston 2000
6 2. A Quick Tour of MapJe

arising from the necessary conditions of an optimization problem or the plot of

a three-dimensional surface of the objective function. We describe Maple's very
impressive array of two- and three-dimensional graphics tools that facilitate such
graphical analyses.
There are several excellent books on Maple dealing with different aspects of
the software. One of the most lucid among these is the official Maple manual
(Maple V Learning Guide) by Heal, Hansen and Rickard [85]. An advanced but
still very readable book is Heck's Introduction to Maple [86]. Israel [97] is mainly
concerned with the use of Maple in calculus and Corless [51] is written for sci-
entific programmers who already have experience in other computer languages.
Redfern and Chandler [154] discuss Maple's use in solving ordinary differen-
tial equations and Klimek and Klimek [III] present a comprehensive treatment
of Maple's graphics command and structures used in plotting curves and sur-
faces. Currently there are more than 300 books on Maple written in different
languages including French, German, Spanish, Russian and Japanese. An up-
to-date list of these books can be obtained from Waterloo Maple's Web site at
www.maplesoft.com/publications/books/maplebooks.html.

2.2 Symbolics
2. 2.1 Expanding, Factoring and Simplifications
Maple's main strength is in its ability to manipulate symbolic expressions. For
example, it can expand an expression such as (x+y)lO. When asked to factor
the result, Maple gives the original expression.
As a simple example, consider the expansion of the expression (x + y) 10. Note
that, in Maple, all characters which follow a sharp character (#) on a line are
considered to be part of a comment. Thus, in the following example, the file name
of the worksheet (i.e., Expand.mws) is understood by Maple to be a comment.
Note that each Maple command must end with a semicolon (;) or a colon (:).
Termination by a semicolon informs Maple to display the results. When the results
are expected to be very long, the colon can be used to terminate the command thus
suppressing the output.
> restart: # Expand.mws
> expand((x+y)A10);

x lO + IOx 9 Y
+ 45x 8 y2 + 120x 7 y3 + 21Ox 6 y4 + 252x 5 y5 + 210x4 y6
+ 120x 3 y7 + 45x2 y8 + lOx y9 + ylO
> factor (%) ;

(x + y)IO
Note that the ditto operator, %, is a shorthand notation for the result of the previ-
ous, i.e., the last, command.
 2.2 Symbolics 7

A Maple expression can be named using the syntax name: =expression.

Several examples using the simplification commands simplify ( ), normal ( )
and combine () are given below.
> restart: # Simplification.mws
The simplify () command is used to apply simplification rules to an expres-
sion:
> x : =a A 3/((a-b)* (a- c))+b A 3/((b-c)* (b-a))
+c A 3/((c-a)* (c-b));
03 b1 c1
x:= + +-----
~-~~-~ ~-~~-~ ~-~~-~
> simplify (x) ;
b+o+c
The normal () command is used to simplify a rational function fig where f
and g i: 0 are polynomials.
> y:=(w 2+3*w+2)/(w 2+5*w+6);
A A

w2 +3w+2
y'-
.- w2 +Sw+6
> normal (y) ;
w+ 1
w+3
Note that x, y and z were the names assigned to the preceding two expres-
sions. If we wish Maple to "forget" these assignments, we can ''unassign'' them
by placing the name inside single forward quotes:
> x :=' x': y :=' y': z: =' z' :
The combine () command attempts to combine terms into a single term. In
some sense, this is-like the factor () command-the opposite of the ex-
pand () command.
> A : =XAy*XAZ;

A :=x'x=
> combine (A) ;
x'x=
In this case combine () did nothing since very often one needs to specify an
option such as power, In, or symbolic to help combine () recognize the
expressions and do the simplification.
> combine(A,power);
x (y+=>
> B:=log(x)/2+log(y)-log(z);
8 2. A Quick Tour of Maple

I
B := 21n(x) + In(y) -In(z)
> combine(B,ln);
I
2ln(x) + In(y) -In(z)
> combine(B,ln,syrnbolic);

> C:=sqrt(x)*sqrt(y);
C:=..(i ,jY
In the following example, the combination of ..;x,jY into ,JXY becomes pos-
sible only after informing Maple---using assurne-that both x and yare posi-
tive quantities since ..(i,jY is not equal to ,JXY for all values of x and y, e.g.,
RR i= J(-I)(-I).
> combine(C);

> assurne(x>O,y>O);
> combine(C);

2.2.2 Function Definitions Using" -> " and unapply ()

The assignment operation (: =) allows the user to name a Maple object as in
f: = 1/ x +x;. Once this is done, one can even evaluate the expression at differ-
ent values using subs () or eval () as in the following examples.
> restart: # Function.rnws
> f:=l/x+x;
I
!:=-+x
x
> subs(x=l,f); eval(f,x=l); subs(x=(a+b)A2,f);
2
2
I 2
(a + b)2 +(a+b)
If the expression is intended to define a function, then it is easier (and prefer-
able) to define the function formally using either the arrow (- > ) notation or the
unapply () command.
> f:=x->l/x+x;
 2.2 Symbolics 9

1
! :=x -+ - +x
x
> f (1) ;
2
> f:=unapply(l/x+x,x);
1
!:=x -+ -+x
x
Now the function !(x) can be evaluated at different values of x.
> f(l); f(a); f((a+b)"2);
2
I
-+a
a

I 2
(a+b)2 +(a+b)
The unapply () command can be useful if, for example, in the middle of a
Maple session, we find an expression that we want to define as a function:
> restart: # Unapply.mws
> Result:=int(int(lambda*exp(-lambda*u),u=O .. t),
lambda) ;
e(-J.t)
Result := - - + A-
t
Here, we create a function out of the Result that depends on two variables
lambda and t.
> F:=unapply(Result,lambda,t);
e(-J.t)
F := (A-, t) -+ - -
t
+ A-
> F(1,5); F(a,b);

~ e(-S) + I
5
e(-ab)
--+a
b

2.2.3 Lists, Sets and Arrays

Lists, sets and arrays are basic types of Maple objects and they play an important
role in the effective use of Maple.
IO 2. A Quick Tour of Maple

Lists
A Maple list can be created by simply enclosing any number of expressions or
other Maple objects in square brackets.
> restart: # Lists.mws
> L:={ a,b,sin(x),int(x"2,x)];

L := [a, b, sin(x), ~ x 3]
The number of elements of the list can be counted using nops () and the
membership of an element can be checked with member ().
> nops (L) ;
4
> member(b,L); member(cos(x),L);
true
false
To access, say, the first and the fourth elements of the list, we enter I.{ 1] and
u: 4] .
> I.{ 1]; I.{ 4] ;

a
1 3
-x
3
The ordered elements of the list L can be converted to an expression sequence
by op (L).
> op (L) ;

a, b, sin(x), ~ x3
Sets
A mathematical set is an unordered collection of elements. In Maple a set is cre-
ated by enclosing the elements in curly brackets.
> restart: # Sets.mws
> A: ={ a, b, 2, Q" 2, 1 / x} ;
1
A := {2, -, a, b, (f)
x
> B:={c,f,10!};
B := {e, J, 3628800}
Operations on sets can be performed with union and intersect.
> A union B;
 2.2 Symbolics II

1
{2, -, a, b, c, f, (fl, 3628800}
x
Since A and B do not have any elements in common, their intersection A nB
is the empty set 0, which Maple denotes by {}.
> A intersect B;
{}
Solving a system of two equations in two unknowns gives the solution using
the set notation. Before the expressions involving x and y can be manipulated, the
solution found here must be assigned.
> Sol:=solve({ a*x+b*y=c,d*x+e*y=f},{ x,y} );
af-cd -ec+ f b}
Sol := {y =
ae-bd
,x = ae-bd
> x; y;
x
y
> assign(Sol); x; y;
-ec+fb
ae-bd
af-cd
ae-bd
It is possible to convert () a list to a set and perform set operations on it.
> L: =[ 9, 6, t, 4, 2] ;
L := [9, 6, I, 4, 2]
> LSet:=convert(L,set);
LSeI := {2, 4, 6, 9, I}
> LSet intersect A;
{2}

Arrays
As a data structure, an array is an extension of a list that is indexed by positive
integers ranging from 1 to nops (list). In an array, one can have many
dimensions and the indices can be negative or zero. Arrays are also related to
objects such as vectors and matrices.
> restart:. Arrays.rows
Here A is a 2 x 2 array with negative and zero indices.
> A:=array(-l •. O,O •. l);
A := array( -1 .. 0, 0.. 1, [])
12 2. AQuick Tour of Maple

Let us make assignments to the first three entries but leave the last one unas-
signed. (The presence of the comment symbol # before A[ 0, 1] : =d indicates
that this is treated as a comment, not as an assignment.)
> A[ -1,0] :=a; A[ -1,1] :=b; A[ 0,0] :=C; #A[ 0,1] :=d;

A_I,o := a
A_I,I :=b
Ao,o:= c
> print (A);

array( - 1..0, 0 .. 1, [
(-1,0) = a
(-I,I)=b
(0,0) =c
(0, I) = AO,I
])
Since A has nonpositive indices, it is not treated as a matrix. But the following
2 x 2 array B acts like a matrix and thus one can manipulate it using matrix
operations with the command evalm ().
> B:=array(1..2,1..2,[[ 3,4] ,[ 2,k]]);

B:= [~ :]
> with (linalg) :
Warning, new definition for norm

Warning, new definition for trace

> evalm(A &* A); evalm(B &* B);
A2

[ 17 12+4kJ
6+2k 8+k 2

2.2.4 Additional Commands to Manipulate Expressions

The map ( ) Command
The map () command applies a procedure to each operand of an expression that
can be a list, set, array. This is a very useful command as it can substantially
simplity the evaluation of a large expression such as a list or a matrix in a single
operation.
> restart: # Map.mws
 2.2 Symbolics 13

> theta:={ O,Pi/4,Pi/2,3*Pi/4,Pi];

o := [0, 41 1r, 21 1r, 43 1r, 1r]

We can use map () to simplify the computation of the sin of each number in
the list O.
> map(sin,theta);
1 1
[0, 2.j2, I, 2.j2, 0]
Differentiating each element of the following array A one by one would be
very cumbersome. But by using map ( ) , we can apply the differentiation operator
di f f to each element of A and automatically obtain the complete result.
> A:=array(1..2,1..2,[[ sin(x),x"2*cos(x)],[ log(x),-
x] ] ) ;

A ._ [ sin (x) x 2 cos(x) ]

.- In(x) -x
> map(diff,A,x);

co~(x) 2 x cos(x) - x 2 sin(x) ]

[
- -I
x

The collect () command

The collect () command is used to collect coefficients of similar powers.
> restart: # Collect.mws
Here, it can simplify expressions by regrouping terms involving exp (x) .
> f:=int(x* (exp(-x)+exp(x)),x);
x 1
f := - eX - eX + x eX - eX
> collect(f,exp(x));
-x - I
(x - l)eX + eX
Here, it collects the coefficients of x" 2.

> collect (g, x) ;

14 2. A Quick Tour of Maple

2.2.5 Solving Equations Analytically

As we will see in later chapters, Maple can find analytic (i.e., symbolic) solutions
for a large class of algebraic and differential equations, hence making it very
useful in operations research modeling and applications. Here are a few examples:

Example I A junction arising in deterministic inventory models. The function

f(x) = a/x + bx arises frequently in operations research and it represents the
average cost of operating an inventory system for particular values of a and b.
The minimum-cost order quantity can be found by differentiating the function,
equating the result to zero and solving for the positive root.
> restart: *
Solve.rnws
> f:=x->a/x+b*x;
a
f :=x -+ -+bx
x
> fx:=diff(f(x),x);
a
fx:=-"2+ b
x
> solve(fx,x);
$a $a
-b-' b
To compute the second derivative of f, we use the shorthand notation diff (
f (x) ,x$2) where the dollar sign $ between x and 2 is an operator that signifies
differentiating the function with respect to x twice.
> fxx:=diff(f(x),x$2);
a
jxx:=2'3
x
Since the function is convex for positive x, (i.e., /xx > 0), the minimum of f
occurs at ,J{jb/ b.
Example 2 A junction arising in stochastic inventory models. The problem of
solving 1- F(Q) = Pwhere F(t) = J~ Ae-).II du arises in single-period stochas-
tic inventory modeling where the demand is assumed to be exponentially dis-
tributed with parameter A. The analytic solution for the optimal order quantity
can again be found by solving the equation.
> F:=unapply(int(larnbda*exp(-larnbda*u),u=O .. t),t);
F := t -+ -e(-).t) +1
> eq:=l-F(Q)=beta; solve(eq,Q);
eq:= e(-).Q} = p
In(p)
A
 2.3 Numerics 15

Thus, the optimal order quantity is obtained as Q = -In(p)/ A. > 0 since

O<P<1.

Example 3 A differential equation arising in stochastic processes. Maple can

also solve differential equations analytically. The next example is encountered
in the description of a Poisson process where A. is the arrival rate and pO(t) is the
probability that there will be zero arrivals between time 0 and time t.
> DE:=diff(pO(t),t)=-lambda*pO(t);
DE := Ii pO(/) = -A. pO(/)
> Sol : =dsolve({ DE,pO(O)=l} ,pO(t»;
Sol := pO(/) = e(-AI)
> assign(Sol); pOSol:=unapply(pO(t),t);
pOSol := 1 ~ e(-A I)

Thus, 1'0(/) = e- A1 •

2.3 Numerics
In many practical problems the user may be interested in obtaining only a nu-
merical solution to a problem. For example, finding the sample mean or sample
variance of a data set or computing the probability of a specific event may re-
quire numerical (not symbolic) computations. In those cases, we can make use of
Maple's abilities to perform the necessary numerics and obtain the results.
Although Maple's symbolic manipulation and solution capabilities can be very
impressive, there are certain instances where it becomes impossible to find solu-
tions analytically. In those cases, we again resort to numerics to find the solution.
Maple can also be used as an interactive calculator that can add and subtract
real numbers.
> restart: * BasicNurnerics.mws
> 2+3; evalf(3452.51-sqrt(Pi»;
5
3450.737546
It knows about complex numbers and their manipulation.
> sqrt(-l); (1+6*I)* (3-5*I)*7*I;
1
-91 +2311
16 2. AQuick Tour of Maple

Maple can deal with very large numbers and manipulate them using (almost)
infinite precision. It can also do integer factorizations with i factor () and per-
form floating-point evaluations to (almost) any precision with evalf ( ) .
> Big:=100!;

Big:=933262154439441526816992388562667004907159682643\
81621468592963895217599993229915608941463976156\
51828625369792082722375825118521091686400000000\
0000000000000000
> ifactor(Big);

(2)97 (3)48 (5)24 (7)16 (11)9 (13)7 (17)5 (19)5 (23)4 (29)3 (31)3 (37)2
(41i (43)2 (47)2 (53) (59) (61) (67) (71) (73) (79) (83) (89)
(97)
> evalf(10g(Big),35);
363.73937555556349014407999336965564
In the following examples we present additional Maple commands that can aid
in numerical analysis.

2.3.1 Solving Equations Numerically

Example 4 A junction arising in stochastic inventory models (continued). Con-
sider again the problem of solving 1- F(Q) = P for the inventory problem where
F(t) = J~ f(u) du is the cumulative distribution function of the random demand.
Let us now assume that demand is normal with mean p and variance q2 and that
the probability of negative demand (for given p and q2) is negligibly small.
> restart: # NumericSolve.mws
> f:=x->1/(sqrt(2*Pi*sigma))*exp(-(x-mu)A2/
(2* sigma A2) ) ;

e<-1/2¥> II
f:=x-+
~
v21(;q
> F:=unapply(int(f(u),u=O .. t),t);

I ,Ji" q (erf( ~ .J2 (-t + p» _ erf(~ p.J2»

F := t -+ __ 2 q 2 q
2 ../iCi
In this problem, Maple attempts to solve 1- F(Q) = P symbolically using
sol ve (), but it can only find a result in terms ofthe error function erf(x) defined
as
erf(x) =
2 r
,Ji"Jo e-I dt.
2
 2.3 Numerics 17

But as soon as we specify the values of the parameters (i.e., Jl = 100, (1 = 10

= =
and p .7), Maple solves the equation and finds Q 86.88 as the optimal order
quantity.
> solve(l-F(Q)=beta,Q)i

RootOf( -erf(! ../i (- - Z + Jl» 1r (1 + erf(! Jl ../i) 1r (1 - 2.Jii7i ,Jii

2 (1 2 (1

+ 2 .Jii7i ,Jii P)
> mu:=100i sigma:=10i beta:=.7i
Jl := 100
(1 := 10
P:=·7
> solve(l-F(Q)=beta,Q)i
86.88641461
Note that the same solution can also be obtained by using the fsolve ()
command that solves equations only numerically. Here, the argument o.. 150
informs Maple that it should search for the numerical solution in the interval
[0,150].
> fsolve(1-F(Q)=beta,Q,0 .. 150)i
86.88641461

2.3.2 Statistical Computations Using the sta ts Package

Maple's stats package contains procedures for performing some of the basic
statistical computations.

Example S Some statistical computations. Suppose the following numbers are

the closing prices of a stock in the last 10 business days.
> restart: # Stats.mws
> with(stats)i
[anova, describe, fit, importdata, random, statevalf, statplots, transform]
> Prices:~ 97.6,99.4,101.3,102.1,96,107,96.5,92,
100 . 4 , 99 . 8] i
Prices:= [97.6,99.4, 101.3, 102.1,96, 107, 96.5, 92, 100.4,99.8]
After loading the describe subpackage, we can numerically compute the
mean, variance and the standard deviation of these prices.
> with(describe)i
18 2. A Quick Tour of Maple

[coefficientofvariation, count, countmissing, covariance, decile,

geometricmean, harmonicmean, kurtosis, linearcorrelation, mean,
meandeviation, median, mode, moment, percentile, quadraticmean,
quantile, quartile, range, skewness, standarddeviation, sumdata,
variance]
> mean(Prices);
99 .21000000
> variance(Prices);
14.74290000
> sqrt(variance(Prices)); standarddeviation(Prices);
3.839648421
3.839648421
It is also possible to group these prices into ranges of half-open intervals of
our choice such as [85,90), [90,95) and so on. After loading the transform
subpackage, we use the tallyinto () command to count the number of obser-
vations in each range.
> Ranges:={ 85 .. 90,90 .. 95,95 .• 100,100 .• 105,105 •• 110];
Ranges := [85 .. 90, 90 .. 95, 95 .. 100, 100.. 105, 105 .. 11 0]
> with(transform);

[apply, classmark, cumulativefrequency, deletemissing, divideby,

frequency, moving, muitiapply, scaleweight, split, standardscore,
statsort, statvalue, subtractfrom, tally, tallyinto]
> PriceList:=tallyinto(Prices,Ranges);

PriceList := [Weight(85 .. 90, 0), 90 .. 95, Weight(95 .. 100, 5),

Weight(l00 .. 105, 3), 105 .. 110]
Thus, there are no observations in the range [85,90), one observation in the
range [90,95), five observations in the range [95,100) and so on.
The result can be plotted as a histogram (not displayed here) after loading the
statsplots subpackage.
> with(statplots);

[boxplot, histogram, scatterplot, xscale, xshijt, xyexchange, xzexchange,

yscale, yshijt, yzexchange, zscale, zshijt]
> *histogram(PriceList);
And finally, we can even generate random prices distributed normally with a
mean of 100 and standard deviation of 5.
> RandomPrices:={ randon{ normald[ 100,5]] (10)] ;
 2.4 Graphics 19

RandomPrices:= [105.8791978,97.18317935, 101.1769700,

92.78724855,94.60401883, 99.88992677, 87.07360818,
97.78364360, 94.98359260, 99.86065130]

Maple's random number generation facilities will be useful in Chapter 10, Sim-
ulation.

2.3.3 The simplex Package

Linear programming (LP) problems are always solved numerically since it is im-
possible to find the solution to such problems analytically in terms of the problem
parameters. Maple has an implementation of the simplex algorithm that finds the
optimal solution to LP problems numerically. We will discuss different aspects of
the simplex method and linear optimization in Chapter 4, Linear Programming.
Example 6 A linear programming problem. As a simple example, consider the
problem of maximizing the objective function z = 2x1 + 3X2 subject to the three
constraints (i) XI + 1.75x2 ~ 650, (ii) 1.5x2 ~ 200, and (iii) XI + X2 ~ 600.
Maple finds the optimal solution as XI = 533.33, X2 = 66.66 and z = 1266.66.
> restart: * Simplex.mws
> with(simplex);
Warning, new definition for maximize

Warning, new definition for minimize

[basis, convexhull, cterm, define_zero, display, dual, jeasible, maximize,

minimize, pivot, pivoteqn, pivotvar, ratio, setup, standardize]
> z: =2* xC 1] +3* xC 2] ;
z :=2xl +3X2
> C:={ xC 1] +1. 75*xC 2] <=650, 1. 5* xC 2] <=200,
xC 1] +xC 2] <=600} ;
C:= {XI + 1.75x2 ~ 650, XI +X2 ~ 600, 1.5x2 ~ 2oo}
> Sol:=rnaximize(z,C,NONNEGATIVE); assign(Sol);
Sol := {XI = 533.3333333, X2 = 66.66666667}
> z;
1266.666667

2.4 Graphics
The third important aspect of Maple is its capability to plot impressive-looking
two- and three-dimensional graphs. The dictum "one picture is worth one thou-
sand words" also applies in operations research modeling and graphing curves on
20 2. A Quick Tour of Maple

the plane or surfaces in space may reveal important information about a problem's
characteristics.
In this section we will consider several two- and three-dimensional Maple
graphs that can be useful in modeling and analysis.

2.4.1 Two-dimensional Plots

Example 7 Plot of the Weibull density. The Weibull random variable arises fre-
quently in reliability and queueing theory where it is used to represent the lifetime
of devices or the time to complete a specific task; see Law and Kelton [118, p.
161]. Figure 2.1 displays the Weibull density f(x) = ap-axa-Ie-(x/PY' , x > 0,
for a specific value of the shape parameter a = 2.5 and shift parameter p = 1.7.
Naturally, these parameters can be easily changed to obtain and plot Weibull den-
sities with different shapes.
> restart: i WeibullPlot.rnws
> alpha:=2.5; beta:=1.7;
a :=2.5
p:= 1.7
> W:=alpha*beta A(-alpha)*x A(alpha-l)
*exp(-(x/beta)Aalpha);
W := .6634645233 x 1.5 e(-·2653858093x 2.s)
> plot(W,x=O .. 4);

0,(,

os
004

0.3

0.2

0.1

O-¥-~~~--r"~·-· . . :2 .. . . :;
x

FIGURE 2.1. Plot of the Weibull density with shape parameter a = 2.5 and scale parameter
p = 1.7.

Example 8 Implicit plot of necessary conditions in an optimization problem.

Suppose we wish to find the stationary points of the function z = -x 3 - 2y 3 +
 2.4 Graphics 21

3xy-x+y, i.e., the set of points that satisfy the necessary conditionszx = Zy = o.
Before analyzing the problem mathematically, it may be useful to plot the neces-
sary conditions as implicitly defined curves in the xy-plane.
> restart: # ImplicitPlot.mws
> z:=-x A 3-2*y A 3+3*x*y-x+y;
z := _x 3 - 2 y 3 + 3x y - x +y
> zx:=normal(diff(z,x»;
zx:= -3x 2 +3y-1
> zy:=normal(diff(z,y»;
zy := -6 + 3 x + Ir
> with(plots):
> implicitplot({ zx,zy} ,x=-l •• l,y=-l .• l,
numpoints=lOOO,color=black);
We see in Figure 2.2 that the two curves for Zx = 0 and Zy = 0 intersect at two
points.

1
0.8

0.2 0.4 xO.6 0.8 1

-0.6
-0.8

FIGURE 2.2. Implicit plots of the necessary conditions Zx = 0 and Zy = 0 for

z = _x 3 - 2y 3 + 3xy - x + y.

Now solving Zx = 0 and Zy = 0 for (x, y) we obtain the two real-valued

solutions.
> Sol:=solve({zx,zy},{x,y});

Sol := {x = RootOf(l8 _z4 + 12 _Z2 - I - 9 _Z),

I
Y = RootOf(l8 _z4 + 12 _Z2 - I - 9 _Z)2 + 3'}
> evalf(allvalues(Sol»;
22 2. AQuick Tour of Maple

{x = .5875626256, y = .6785631723},
{y = .3429560780, x = -.0980955898}, {
y = -.5107596261 - .4653767156/,
+
x = -.2447335179 .9507825483 I}, {
x = -.2447335179 - .9507825483/,
y = - .5107596261 + .4653767156 I}
Example 9 Contour plot and implicit plot displayed jointly. Consider the prob-
lem of minimizing the quadratic function f(x, y) = (x - 2)2 + (y - 3)2 subject
to the linear constraint g(x, y) = x + y ~ 4 for x, y ~ O. In order to obtain in-
sight into the behavior of the objective function and its relation to the constraint,
we can plot the contours of the function f(x,y) using contourplot () along
with the constraint boundary g(x, y) = 0 using implici tplot (). The graph
in Figure 2.3 reveals that since the unconstrained optimal solution is outside the
feasible region, the constrained optimal solution for this problem will be on the
constraint boundary x + y = 4.
> restart : # ContourPlot.mws
> with (plots) :
> f:=(x-2)A2+(y-3)A2i
f := (x - 2)2 + (y - 3)2
> g :=X+y=4i
g :=X +y=4
> fCP:=contourplot(f,x=O .. 5,y=O •• 5,color=black,
scaling=constrained,contours~ 1,2,3,4,~ ,
numpoints=1000) :
> gIP : =implicitplot(g,x=O •. 5,y=O .. 5,color=black,
scaling=constrained) :
> display({ fCP,gIP} ) i

Example 10 Graph of the numerical solution of an ordinary differential equa-

tion. The differential equation encountered in Example 3 had a closed-form s0-
lution as the arrival rate parameter A was a constant. Now suppose that this pa-
rameter is time dependent, i.e., it is given by .1.(t) =
log(t 2 + e-t \ This time
it becomes impossible to solve the resulting differential equation dx(t)/dt =
-.1.(t)x(t) analytically since .1.(t) is now a very complicated function. We can,
however, use Maple's knowledge of numerical solutions of DEs and even plot the
resulting solution trajectory.
> restart: # DEPlot . mws
> lambda : =t->log(t A2+exp(-t A2))i
#plot(lambda(t),t=O .. l)i
A := t ~ log(t 2 + e<-t 2»
> DE : =diff(x(t),t)=-lambda(t)*x(t)i
 2.4 Graphics 23

3
Y
2

FIGURE 2.3. The contours of the objective function f(x,y) displayed with the con-
straint boundary g(x,y) = O.

DE := f, x(t) = -In(t 2 + e(-t 2 » x(t)

Maple makes a good attempt to solve this DE, but it can give the solution only
in tenns of an integral.
> dsolve({ DE,x(O)=l} ,x(t)};

x(t) = e(- J~ In(u2-te(-W2»du)

But using the type=numeric option of dsol ve (), we can ask Maple to
compute numerically the solution trajectory.
> Sol:=dsolve({ DE,x(O}=l} ,x(t},type=numeric);
Sol := proc(rkf45_x) . . . end
Once the solution is found, it can be evaluated at different time points and
eventually plotted using odeplot (), shown in Figure 2.4.
> Sol (O); Sol ( . 5); Sol ( 1 ) ;
[t = 0, x(t) = I.]
[t = .5, x(t) = .9970800089348753]
[t = I, x(t) = .9296351615552543]
> with(plots}:

> odeplot (Sol, [ t, x (t) 1 ,0 .. 1, color=black,

labels=[ t, xl ) ;
24 2. A Quick Tour of Maple

0.99

0.'J8

0.'T1
x
0.%

0.CJ5

0 .')4

0 0.2 OA "t 0.£ 0.8

FIGURE 2.4. Graph of the solution trajectory for dx(t)/dt = -1 (t)x (t), x (0) = 1 where
l(t) = log(t2 + e- t\

2.4.2 Three-Dimensional Plots

Maple's plotting abilities also include plotting three-dimensional surfaces, which
arise frequently in operations research modeling. In this section we consider two
examples.
Example 11 Plot of the objective function surface in an inventory problem. In
[SI, p. 44] Hadley and Whitin consider a detenninistic inventory problem where
backorders are pennitted. In this example we plot the objective function surface
of this problem and calculate the optimal values of the decision variables.
> restart: * BackorderPlot.mws
Here, A. is the annual demand rate, K is the order cost, h is the holding cost per
unit per year and 1C I and 1C 2 are the fixed and variable (in time) costs, respectively,
of backordering demands.
> C:=lambda*K/Q+h* (Q-s)A2/(2*Q)
+ (1/Q)* (pi[ 1) * lambda* s+pi[ 2) * sA2);

C.- A.K ! h(Q-s)2 'fJA.S+1C2S 2

.- Q + 2 Q + Q
> lambda:=200; K:=8; h:=14; pi[ 1) :=.8; pi[ 2) :=25;
A. := 200
K :=S
h:= 14
1C1 :=.S
1C2 := 25
> C;
 2.4 Graphics 25

1600~ 7(Q-S)2 16O.0s+25s 2

Q+ Q + Q
When we assign specific values to the parameters and plot the objective func-
tion C(Q, s), we observe that the function is strictly convex with a unique mini-
mum. The optimal solution is found as Q = 16 and s = 1 with C (16, 1) = 210
by solving the necessary conditions.
> plot3d(C,Q=10 .. 30,s=0 .. 5,axes=boxed,shading=none,
orientation={ 107,67] ) ;
> CQ:=normal(diff(C,Q)); Cs:=normal(diff(C,s));

CQ .- -1600 + 7 g2 - 32s - 160.0s

2
.- Q2
14 Q - 64s - 160.0
Cs:=
Q
> Sol:=fsolve({numer(CQ),numer(Cs)} ,{Q,s});
Sol:= {Q = 16.00000000, s = 1.0OOOOOOOO}
> assign(Sol);
> C;
210.0000000

FIGURE 2.5. Objective function of the inventory problem where backorders are permitted.

Example 12 Bivariate normal density. Now we consider the surface plot of the
density f(x,y) of the bivariate normal random vector (X, Y). We will plot the
density using the plot3d () command.
> restart: * BivariateNormalPlot.mws
26 2. A Quick Tour of Maple

> q:= (x-mu[ 1] ) "2/sigma[ 1] "2

-2* rho* (x-mu[ 1] ) * (y-mu[ 2] ) / (sigma[ 1] * sigma[ 2] )
+ (y-mu[ 2] ) "2/sigma[ 2] "2;
(x - "'1)2 P (x - "'I) (y - "'2) (y - "'2i
q := - 2 + ..::.....---:.,,-=--
0'1 2 0'10'2 0' 22

> c:=1/(2*Pi*sigma[ 1] * sigma[ 2]*sqrt(1-rho"2));

1 1
JI - p2
C'--
.- 2 1f 0' 10'2

> f:=c*exp(-q/(2*' (1-rho"2)' ));

(r-'1)2 -2 p(r-S'~Y'-1'2) +~)

( -1/2 "I I 2 "2
I-pi
f'- ! _e_ _ _ _ _-===--_ _
.- 2 1f 0' 10'2 JI - p2
Note that in the previous expression we enclosed 1 - p2 in single quotes (' ')
so that Maple would not immediately evaluate 2(1 - p2) to 2 - 2p2. The single
quotes can be useful in checking the accuracy of symbolic data entry.
> mu[l]:=lO; mu[2]:=5; sigma[l]:=7; sigma[2]:=12;
rho:=.7;
"'I := 10
"'2 := 5
0'1 := 7
0'2 := 12
p :=.7
We check by double integration that the joint density integrates to 1, i.e., that
f~f~f(x,y)dydx = 1.
> int(int(f,x=-infinity .. infinity),
y=-infinity .. infinity);
.9999999998
We may assume, for example, that the return X from stock A and the return
Y from stock B have a bivariate normal distribution with parameters
"'2 = 5%,0'1 = 7«'10,0'2 = 12% and p = 0.7. With these mean values, stan-
= 1()o1o, "'I
dard deviations and correlation coefficient, we plot the surface of the probability
density in Figure 2.6. The orientation argument is used in order to view the
surface from a position that is nearest to the lowest values assumed by x and y,
i.e., -5 and -15, respectively.
> plot3d(f,x=-5 .. 25,y=-15 .. 25,shading=none,
axes=boxed, orientation={ -112,54] ) ;
 2.5 Other Useful Commands and Packages 27

FIGURE 2.6. The surface plot of the bivariate normal density f(x,y) with PI = 10,
P2 = 5,111 = 7,112 = 12 andp = .7.

2.5 Other Usefid Commands and Packages

2.5.1 The piecewise () Command
In many important applications some functions are given in a "piecewise" fonn.
For example, in inventory problems with quantity discounts the unit price C(Q)
is usually a step (i.e., piecewise) function of the order quantity Q. In probabilistic
modeling, some random variables-such as the triangular-bave densities that
are specified as piecewise functions. Maple's piecewise () is a very powerful
command that can manipulate expressions involving piecewise functions.
We discuss two examples.
Example 13 Inventory problem with quantity discounts. In many inventory prob-
lems, the unit price C(Q) is a decreasing function of the order quantity Q given
as a step function. For specified quantities (break levels) qO = 0, ql, . . . ,qN with
q; < q;+1 and qN+1 = 00, if Q units are ordered such that qj ~ Q < q;+I, then
the unit price is C(Q) = C; dollars per unit and the total purchase cost is C; Q
dollars.

Consider the following case.

> restart: *
piecewise.mws
The break-points are given as 1000, 2000 and 3000.
> q[ 0] :=0; q[ 1] :=1000; q 2] :=2000; q[ 3] :=3000;

qO :=0
ql := 1000
q2:= 2000
28 2. A Quick Tour of Maple

q3 := 3000
The piecewise unit cost function C(Q) is defined in tenns of the break-points.
> C:=piecewise (Q<q[ 1] ,5, Q<q[ 2] ,4,
Q< q[ 3] , 3, 2);

5 Q < 1000
C'= { 4 Q < 2000
. 3 Q < 3000
2 otherwise
> eval(C,Q=1234);
4
The total cost QC(Q) is also a piecewise function as plotted in Figure 2.7.

8000

6000

4000

2000

o 500 1000 1500 02000 2500 3000 3500

FIGURE 2.7. Plot of the total purchase cost function QC(Q) in an inventory model with
quantity discounts.

> plot(Q*C,Q=O .. 3500,discont=true,color=black);

Example 14 The triangular random variable. An even more impressive use of
piecewise () is demonstrated in the following example. We consider the tri-
angular random variable X whose density f(x) is given as a piecewise function
[12, p. 218].
f( ) - { (b:~)(:2a)' ifa ::: x ::: b
x - 2~-.x) ifb
(e- He-a)' , < x ::: c.
> assume(a<b,b<c);
> f:=x->piecewise(a<=x and x<=b,
2* (x-a) / ((b-a)* (c-a)),
b<=x and x<=c,
2* (c-x) / ((c-b)* (c-a))) :
> f (x);
 2.5 Other Useful Commands and Packages 29

2 x-a a-x<Oudx-b<O
{ (b - c-x
a)(c - a) - -
2 b - x < 0 and x - c < 0
(c-b)(c-a) - -
with assumptions ona, b andc
We check and see that the density indeed integrates to I, i.e., .r: f (x ) dx = 1.
> int(f(x),x=a .. c);

f:
Evaluating the integral x f(x) dx to compute the mean of X gives E(X) =
!(a+b+c).
> EX:=int(x*f(x),x=a .. c);

1 1 1
EX := 3" b + 3" a + 3" c
with assumptions on b, a and c
We can easily compute the cumulative distribution function F (x) = f: f (u) du,
a ~ x ~ c and display the result as another piecewise function.
> F:=normal(convert(int(f(u),u=a .. x),
piecewise,x»;

o x~a
x 2 -2ax +a2
x~b
F .-
·- (-b + a)(a - c)
-a c + a b - x 2 + 2 ex - c b
x ~ c
(-c + b)(a - c)
c<x
with assumptions on a, b andc

2.5.2 The interface () Command

Sometimes it may be useful to see the body of a procedure in order to exam-
ine the written code. This is made possible by first entering interface (ver
boseproc=3) followed by readlib ( 'procedurename ' ) .1
For example, to see the code for the fu t ureval ue () command that is a part
of the finance package, we do the following.
> restart: # VerboseProcedure.mws
> interface(verboseproc=3);
> readlib('finance/futurevalue');

1The readlib () command became obsolete in Maple Release 6.

30 2. A Quick Tour of Maple

proc(Amount, Rate, Nperiods)

option
'Copyright (c) 1994 by Je'roAme M. Lang. All rights reserved.';
description 'future value at Time of Amount invested at Rate';
Amount x (l + Ratefperiods
end
Thus, we see that futurevalue () takes three inputs (Amount, Rate and
Nperiods) and produces one output computed using the formula Amount*(I+
Rate)ANperiods.
To see the code for commands that are not part of a specific package, one should
use op (command) as in op ( 'sqrt').
> op( 'sqrt '):

2.5.3 The infolevel [J Command

In Maple it is possible to obtain very general information on how a problem is
being solved with the infolevel[] : =Level command. For example, to see
how an integral is being evaluated, we enter infolevel[ int] : =5; before
performing the integration. At each stage of solution process, Maple displays the
technique it uses to attack the problem.

Example 15 Definite integration. Consider the problem of evaluating the definite

integral of the exponential density, i.e., J~ Ae-.l.u duo
> restart:. infolevel.mws
> infolevel[ int] :=5;
infolevelinl := 5
> int(lambda*exp(-lambda*u),u=O .. t);
int/elliptic: trying elliptic integration

int/ellalg/elltype: Checking for an elliptic integral

lambda*exp(-lambda*u) freeof(u) u

int/indefl: first-stage indefinite integration

int/indefl : first-stage indefinite integration

int/indef2: second-stage indefinite integration

int/indef2: applying derivative-divides

int/indefl: first-stage indefinite integration

 2.5 Other Useful Commands and Packages 31

_e(-At) +1
Example 16 Solution ofa differential equation To follow Maple's steps in solv-
ing a differential equation, we enter infolevel[ dsol vel : =5; . Consider the
problem of solving dx/dt = ax (t), x(O) = xo.
> infolevel[ dsol vel : =5;
infoleveldsotve := 5
> dsol ve ({ di ff (x (t) , t) =a* x (t) , x (0) =x[ 0] } ,x (t» ;
Methods for first order ODEs:

trying to isolate the derivative dy/dx

successful isolation of dy/dx

-> trying classification methods

trying a quadrature

trying 1st order linear

1st order linear successful

x(t) = xoe(at)

2.5.4 Exporting to C, FORTRAN and laTeX

Maple has facilities that allow exporting its output to some programming lan-
guages and typesetting processors. As an example, let us assume that we wish to
export the expression f := 1C sin(Ox) 1n(..(;3) to C, FORTRAN and LaTeX. This
is done as follows:
> restart: # Export.mws
> readlib(C): readlib(fortran): readlib(latex):
> f:= Pi*sin(theta*x)*ln(sqrt(x A 3»;

f := 1C sin(O x) 1n(..(;3)
> C(f);
to = O.3141592653589793El*sin(theta*x)*log(sqrt
(x*x*x» ;

> fortran(f);
to = O.3141593El*sin(theta*x)*alog(sqrt(x**3»
32 2. A Quick Tour of Maple

Output from C and fortran can also be written to an external file for further
processing; see? C and? fortran .
If a complete Maple worksheet is to be exported to LaTeX, it is preferable to
do it by choosing Iile/&xport As/LaTeX from the Maple menu bar. To format
and print documents exported and saved in this manner, the LaTeX style package
maple2e. sty is needed. In Wmdows installations, this file is located in the
Etc folder where Maple is installed.2
> latex(f);

\pi \,\sin(\theta\,x)\ln (\sqrt {{x}"{3}})

The C and FORTRAN output can be optimized resulting in a sequence of as-
signment statements in which temporary values are stored in local variables be-
ginning with the letter t.
> C(f,optimized);

t2 sin(theta*x);

t4 x* x;

t6 sqrt (t4* x) ;

t7 log(t6);

t8 O.3141592653589793El*t2*t7;
> fortran(f,optimized);
t2 = sin(theta*x)

t4 = x** 2

t6 = sqrt(t4*x)
t7 = alog (t6)

t8 = O.3141593El*t2*t7

2.5.5 Programming in Maple

In this book we will be using Maple interactively to solve operations research
problems. But Maple also has a very sophisticated programming language that
can be used to develop programs (procedures) automating the computations per-
formed.

2Maple worksheets used in this book were produced with maple2e . sty, mapleenv. def,
mapleenv. sty, mapleplots. sty, maplestyle. sty, mapletab. sty and maple-
uti! . sty. These style files are written by Stan Dewitt ofWaterJoo Maple, Inc.
 2.5 Other Useful Commands and Packages 33

For example, suppose we wish to write a program to compute the factorial n!.
This is done by creating a procedure, naming it (say, f) and then entering the
statements that define the factorial function.
> restart: # Program.mws
> f:=proc(n) if n=O then 1 else n*f(n-l) fi end;
f:= proe(n) ifn = othen 1 elsen x f(n - I)fiend
Once the program is written, we can run it by simply evaluating the ftmction
fen) for different values of n.
> seq(f(n),n=O .. 5);
I, 1,2, 6,24, 120
As another example, the next procedure called Di f fIn t computes the deriva-
tive and the indefinite integral of a given ftmction y(x).
> Difflnt:=proc(y,x)
> local yDiff, ylnt;
> yDiff:=diff(y,x);
> ylnt:=int(y,x);
> [ y,yDiff,ylnt]
> end;

Di.fI1nt := proe(y, x)
IoealyD(tf, ylnt;
yD(tf:= diff(y, x); ylnt := int(y, x); [y, yD(tf, ylnt]
end
x
The input ftmction 2 , its derivative 2x and integral -lx 3 follow.
> Difflnt(x A 2,x);

, 2x '3"!'x ]
[X2 3

For an excellent coverage of Maple's programming facilities, we refer the reader

to Monagan, Geddes, Labahn and Vorkoetter [132].

2.5.6 The finance Package

In addition to its sophisticated mathematical packages such as integral transforms
and linear algebra and others, Maple also has a somewhat simple finance pack-
age that performs basic financial calculations. Using this package, one can calcu-
late present and future values, compute annuities and option prices and so on.

Basic Finance
> restart: # Finance.mws
> with(finance);
34 2. A Quick Tour of Maple

[amortization, annuity, blackscholes, cashjlows, effectiverate, futurevalue,

growingannuity, growingperpetuity, levelcoupon, perpetuity,
presentvalue, yieldtomaturity]
Suppose that in three years we have promised to pay someone $100. If the interest
rate is 6% per year, then $83.96 deposited now (i.e., the present value) will grow
to $100 at the end of year 3.
> presentvalue(100,0.06,3);
83.96192830
Suppose we deposit $100 in our bank account, which grows at a rate of 6% per
year. The initial deposit will grow to $119.10 in 3 years.
> futurevalue(100,0.06,3);
119.101600
If the interest rate is 6% per year, then the present value of an annuity paying
$100 at the end of each year for the next 15 years is $971.22.
> annuity(100, .06,15);
971.2248989

Option Pricing
Maple's blackscholes () command computes the arbitrage-free value of a
European-style call option. If the spot price of a stock is $20, the strike price of
the option is $ 18, risk-free interest rate is 7%, time to maturity of the option is 90
days and the volatility of the stock is .09, then the correct value of the option now
using the Black/Scholes formula [34] is $2.63. Maple also finds that the option
seller should hold 0.81 units of the stock now in order to hedge and eliminate the
risk in his portfolio (consisting of the option and the stock).
> BS:=blackscholes(20,18,0.07,90/365,sqrt(0.09),
hedge) ;

BS:= lOerf(.09259259260 (In( ~) + .02835616438) ~ v'73-/2)

+ 1.154009520 - 8.845990480erf(~(
.1851851852 (In( I:) + .02835616438) ~v'73

- .004109589041 ~v'73)-/2)
> evalf(BS); evalf(hedge);
2.630815160
.8153042532
 2.6 Summary 35

For a discussion of Black/Scholes fonnula, financial derivatives and financial

engineering, see Baxter and Rennie [17], Luenberger [128], Neftci [136] and
Wilmott [194].

2.6 Summary
Maple's ability to perfonn symbolic and numeric computations and its two- and
three-dimensional plotting features can be helpful in modeling and solving OR
problems. In this chapter we provided a brief introduction to those Maple com-
mands that will be used in subsequent chapters. In particular, we used exam-
ples to describe the following features of Maple: function definitions, lists, sets
and arrays, algebraic simplifications and expression manipulations, solution of
equations symbolically or numerically, perfonning statistical and financial com-
putations, plotting graphs, obtaining infonnation about Maple's internal opera-
tions and exporting the Maple output to other programming languages such as C
and FORTRAN. Although Maple's individual commands are powerful enough to
solve a wide range of OR problems by just pasting together a few commands, we
also described briefly Maple's powerful programming language. With this basic
Maple background, the reader should now be able to follow the solution of OR
problems that will be covered throughout the book.

2.7 Exercises
1. Simplify
X4-I
x -I
using normal ( ) . Is it always a good idea to use normal () to simplify
expressions? Would you use normal () to simplify (x 50 - I)/(x - I)?
Why, or why not?

2. Let /(t) = Ae- At be the density of the exponential random variable T

defined using the «->" notation. First compute the cumulative distribution
function F(t) = J~ /(u)du by using the U_>" notation. Now compute
F(t) by using the unapply () command. What is the difference between
the two results? Which method evaluates its input?

3. Let L be a list defined as [tfl, I, eel'] where a and b are real numbers. Find
the logarithm of the elements of this list using the map command.

4. Consider the nonlinear equations x2 + r = 9 and x 2 - r = 4. First,

use the implici tplot () command to plot and graphically locate the
roots of this system. Next, use the sol ve () command to compute all the
roots exactly. If the solution is obtained in tenns of the RootOf function,
36 2. A Quick Tour of Maple

use a 11 val u e s () to generate all the roots. Find the floating point ap-
proximation for the root in the first quadrant using evalf (). For addi-
tional information on these functions, use Maple's help facility by entering
a question mark followed by the command name, e.g., ? sol ve.
5. Generate 50 random variates distributed uniformly between 0 and 100 and
find their mean. Is the computed sample mean close to the true mean? Now
plot these numbers in a histogram with five intervals [0,20), [20,40), ...,[80,
100).
6. Maximize the linear objective function z = XI +X2 +X3 subject to the linear
constraints 2x1 + 3X2 ~ 8 and 2x2 + 5X3 ~ 10 where the decision variables
Xi, i = I, 2, 3 are required to take nonnegative values. Find the optimal Xi,
i=I,2,3andz.
7. Consider the differential equation N(t) = [hIN(t) -m]N(t), with N(O) =
No a given constant. Let hI = 0.4, m = 50 and N = 100. Solve this dif-
ferential equation and plot its solution. Now increase hI = 10 and resolve
the problem. What happens to the trajectory of N(t) as hI increases? (This
is a simplified model of a population growth--or decline-where the birth
rate is proportional to the population size and m is the death rate; see e.g.,
Bellman and Cooke [25, p. 13].)
8. Consider the differential equation x(t) = u(t)x(t) with x(O) = 1 where
u(t) is a piecewise defined function given by

.2, if 0 ~ t < 1
u(t) = { .2, if2 ~ t < 3
.1, otherwise.

Solve this DE and plot the solution trajectory for t E [0, 5]. Can you assign
an economic interpretation to this problem?
3
Maple and Mathematical Foundations
of Operations Research

3.1 Introduction
Although there have been some isolated cases I of successful applications of oper-
ations research that did not involve any mathematics, operations research model-
ing generally requires a good understanding of basic university-level mathematics
such as algebra, calculus, linear algebra, differential equations, transfonn methods
and probability theory.
In this chapter we review these important mathematical topics and describe the
Maple statements that can be used to perfonn the required operations in order to
solve equations and inequalities, evaluate integrals and derivatives, find the solu-
tion oflinear systems, solve ordinary and partial differential equations and differ-
ence (recurrence) equations, invert Laplace transfonns and generating functions
and find the joint probability distribution of functions of random variables.

JWhen the human element is involved in a decision environment, it may be impractical or impossi-
ble to construct a mathematical model of the decision problem. A case in point is the widely circulated
"elevator problem" (Taba, [182, pp. 1-2]). This problem involved elevators in an office building with
slow service and waiting times that were perceived to be long by the users who complained loudly.
A mathematical solution based on queueing theory Wlfl found unsatisfactory. But after a careful ex-
amination of the problem, it Wlfl discovered that the users' complaints were due to boredom since
the actual waiting times were not that long. A nonmathematical solution for this problem Wlfl pro-
posed that involved installing full-length mirrors at the entrances to the elevators. Once the mirrors
were installed, the users' complaints disappeared IfI they kept occupied by watching the others (and
themselves) while waiting for the elevators.

M. Parlar, Interactive Operations Research with Maple

© Birkhäuser Boston 2000
38 3. Maple and Mathematical Foundations of Operations Research

We will see that Maple's ability to perfonn symbolic manipulations will pennit
us to easily solve difficult mathematical problems that are relevant in OR. For ex-
ample, evaluating manually an integral such as f(k log(x) - 2x 3 + 3x 2 + b)4 dx
is a nontrivial task since expanding the integrand results in 35 tenns such as
-l44k log(x)x s and 12k3[log(x)]3x 2. While integrating each of these tenns may
not be very difficult, the sheer task of mathematical bookkeeping that involves
collecting the tenns, simplitying them makes this an error-prone operation ifper-
fonned manually. We will see that Maple can evaluate this integral (and many
others) symbolically and produce the correct result in a fraction of a second. As
always, it is important to check the results obtained, but that too can be achieved
relatively easily with Maple's help.

3.2 Algebra
In this section we describe the use of the sol ve () function which can success-
fully solve a large class of equations and inequalities.

3.2.1 Solution ofa Single Equation or Inequality

First consider the quadratic equation ax 2 + bx + c = O. Maple solves this easily
in tenns of the coefficients and gives the well-known result.
> restart: # QuadraticCubic.mws
> Quadratic:=a*x~2+b*x+c;

Quadratic := a x 2 + b x +c
> SolQ:=solve(Quadratic,x);
1 -b + ./b2 - 4 a c I -b - .Jb2 - 4ac
SolQ := 2' a ' 2' a
> Sol<1 1]; Sol<1 2] ;
I -b + .Jb2 - 4ac
2 a
I -b -.Jb2 - 4ac
2 a
Next, we attemptto solve a cubic equation given by x 3-2x 2+x+21 = O. The
exact solution of this cubic involves complex numbers (denoted by I in Maple).
The solution is simplified using the evalf () command. The correctness of the
solution is also checked by substituting it back into the original equation.
> Cubic:=x~3-2*x~2+x+21;

Cubic:= x 3 - 2x2 +x + 21
> SolC:=solve(Cubic,x);
 3.2 Algebra 39

sole:= -! %1(1/3) - ~ + ~ ~ %1(1/3) +!----,~

6 3 %1(1/3) 3' 12 3 %1(1/3)
+ ~ +! 1../3 (-! %1(1/3) + ~ ) ~ %1(1/3)
3 2 6 3 %1 (1/3) , 12
I I 2 I r-; I 0 (1/3) 2
+ 3 %1(1/3) +3 - 2 / ,,3 (-6 Vol + 3 -%-'1("""'1/=3»
%1 := 2276 + 36 ,./3997
> evalf (SolC) ;

-2.135720892, 2.067860447 - 2.357265088/,

2.067860447 + 2.3572650881
> seq (simplify (subs (x=Solq i) , Cubic) ), i=1. .3) ;
0,0,0
Maple also knows how to solve inequalities such as Ix + 31 < 2 and (x - 3 +
a)(x-I +a)(x -2+a) < 0 for the unknownx. The solution of these inequalities
is found as follows.
> restart: # Inequality.mws
> solve(abs(x+3)<2,x);
ReaIRange(0pen(-5), Open(-I»
> solve ((x-3+a)* (x-1+a)* (x-2+a) < 0, {x});
{x < I - a}, {2 - a < x, x < 3 - a}

Example 17 Aircraft engines. Suppose that an airplane makes a successful flight

if at least 50010 of its engines are operating. If I - P is the probability that an engine
will fail independently of the other engines, we want to find a range of values for
p such that a 8-engine plane is preferable to a 4-engine plane.
Using simple probabilistic reasoning it is easy to see that the number of en-
gines operating is binomial. Thus, it follows that an 8-engine plane will make a
successful flight with probability

and a 4-engine plane will make a successful flight with probability

Solving the inequality Ps > P4 manually would clearly be a challenging task. But
using Maple's sol ve ( ) command and then evaluating the result with eval f ( ) ,
we find that p > .586.
40 3. Maple and Mathematical Foundations of Operations Research

> restart: # Airplanes.mws

> Digits: =6;
Digits:= 6
> p[ 8) : =sum(binomial (8, n) *p"n* (l-p)" (8-n) ,n=4 .. 8) ;

Ps := 70 p4 (I - p)4 + 56 p5 (I - p)3 + 28 p6 (I - p)2 + 8 p7 (I _ p) + ps

> p[ 4) :=sum(binomial(4,n)*p"n* (1-p)"(4-n),n=2 .• 4);

P4 := 6 p2 (I - p)2 + 4 p3 (1- p) + p4
> Sol: =sol ve (p[ 8) > p[ 4) ,{ p} ) :
> evalf(Sol);
{p < -.238776}, {p < I., .586886 < pI, {1.22332 < p}
The graphs of both Ps and P4 as a function of p E (0, I) can be plotted after
removing the # sign before the plot command.
> #plot ([ p[ 8) ,p[ 4) ) ,p=O .. 1) ;
Of course, sometimes it may be impossible for Maple to find the solution
of an equation using the solve () command. But in some cases where there
are no symbolic parameters, using the floating point version of the solver, i.e.,
fsol ve (), may result in a solution.

Example 18 Solution ofa single-period stochastic inventory problem. As an ex-

ample, consider the equation I - foQ f(x) dx = p that arises in stochastic inven-
tory theory. Here, Q is the order quantity decision variable, f(x) is the demand
density and p is a constant between 0 and I that is computed using parameters
such as the order cost, unit revenue, shortage cost and salvage value. Supposing
that p =0.65 and that demand is normal with mean f.l =
1000 and standard
deviation (1 = 100, we first attempt to solve this equation with sol ve ( ), which
fails. Next, trying fsol ve () but with a specific interval over which the search
should be conducted, i.e., from 500 to 1500, we find the correct solution.
> restart: # NewsboyEquation.mws
> with(stats);
[anova, describe, fit, impartdata, random, stateva/f, stalplots, transform]
> Eq:=l-statevalf[ cdf,normald[ 1000,100)) (Q)=0.65;
Eq := I - staleva/fcdj, IIOrmaldJOoo, 100 (Q) = .65
> solve(Eq,Q);

RootOf(20 staleva/fcdj, normalduXJlJ, 100 l.Z) - 7)

> fsolve(Eq,Q=500 .. 1500);
961.4679534
Thus, the optimal order quantity should be Q* = 961.46 units.
 3.2 Algebra 41

3.2.2 Solution 0/a System o/Nonlinear Equations

Maple's powerful solve () function can also solve systems of equations. First
consider a simple example with two equations: a circle defined as x 2 + y2 = r2
and a line passing through the origin defined as y = ax. We know from the
geometry of these equations that the system has two solutions. Maple finds them
as follows.
> restart: # CircleLine.rnws
> circle:=xA2+yA2=rA2;

> line:=y=a*x;
line:= y = ax
> Sol: =sol ve ({ circle, line} ,{ x, y} );

Sol:= {x = RootOf((l + a2LZ2 -1)r,

y = a RootOf«(l +a2)_Z2 - l)r}
Since the solution is not in a form that is very convenient to interpret, we use
the all values () command to obtain the set of two solutions.
> allvalues(Sol);
r ar r ar }
{x= ~,y= ~},{x= ~,y=
-vl+a- -vl+a- ~
Example 19 A constrained multi-item inventory problem. As another example,
consider the problem of finding the solution to the system
I AI
-hi - '1-
2 Qt = 0

I A2
-h2 - '1-
2 ~
= 0

AI A2
-
QI
+ -
Q2
= B

with unknowns QI, Q2 and 'I. This problem arises in the optimization of a con-
strained multi-item inventory system where there is a constraint on the total num-
ber of orders that can be placed. The parameters hi correspond to the holding
costs, Ai are the annual demands and B is the maximum number of orders that can
be placed per year; see Hadley and Whitin [81, pp. 56-57]. We assume hi = 10,
h2 = 5, AI = 100, A2 = 150 and B = 50 and solve the system with Maple.
> restart: # ThreeEqns.rnws
> h[ 1] :=10; h[ 2] :=5; lambda( 1] :=100;
larnbda[ 2] : =150; B: =50;
hi := 10
42 3. Maple and Mathematical Foundations of Operations Research

h2:= 5
A.I := 100
A.2 := 150
B :=50
> ecJ 1] :=(1/2)*' h[ 1]' -eta*' lambda{ 1]' 1\1.. 1] "2=0;
1 'IA.I
eql:= -hl- - 2 =0
2 QI
> eq 2] :=(1/2)*' h[ 2]' -eta*' lambda{ 2]' 1\1.. 2] "2=0;
1 'I A.2
eq2 := - h2 - - 2 = 0
2 Q2
> ecJ 3] : =' lambda{ 1] , 1\1.. 1] +' lambda{ 2] , 1\1.. 2] =' B' ;
A.I A.2
eq3 :=-+-=B
QI Q2
> solve({ ecJ 1] ,eq[ 2] ,eet 3]} ,{ \1.. 1] ,\1.. 2] ,eta} );

1 1
(Q2 = RootOf(-3 - 6_Z + _Z2), QI = 2RootOf(-3 -6_Z + _Z2) + 2'
1
'I = - (RootOf(-3 - 6 Z + Z2) + 1)2}
80 - -
> Sol:=evalf(allvalues(%));

Sol := {'I = .6964101616, Q2 = 6.464101616, QI = 3.732050808},

{Q2 = -.464101616, 'I = .003589838475, QI = .267949192}
Thus the correct solution with positive order quantities is found as QI = 3.73,
Q2 = 6.46, and 'I = 0.69.

Example 20 A marketing game. As a final example consider a problem adapted

from Ackoff and Sasieni [4, p. 345). In a duopolistic environment, there are two
companies that spend X; dollars (i = 1, 2) on advertising. Each company's adver-
tising effectiveness factor is a;. Total market size is S and the market is split in
proportion to the money spent on advertising. Thus, if company 1 spends XI and
company 2 spends X2 and if the factors are a I and a2, then their sales are

Sa;x;
i = 1,2.

Suppose that the gross profit of each company is g;, i = 1, 2, so the profit P; of
company i is obtained as

PI. -_ g;Sa;x;
-X;, i = 1,2
alXI + a2X2
 3.2 Algebra 43

The solution to this competitive problem can be obtained by computing each com-
pany's ''reaction curve"

R. _ ap;
,- ax;
and solving for XI and X2 from RI = R2 = o.
> restart: # Marketing.mws
> p[ 1] : =g[ 1] * S* alpha[ 1] * x[ 1] / (alpha[ 1] * x[ 1]
+alpha[ 2] * x[ 2] ) -x[ 1] ;

PI:= gl Salxl -XI

al XI +a2x2

> p[ 2] : =g[ 2] * S* alpha[ 2] * x[ 2] / (alpha[ 2] * x[ 2]

+alpha[ 1] * x[ 1] ) -x[ 2] ;

P2:= g2 Sa2x2 -X2

al XI +a2x2

> R[ 1] :=diff(p[ 1] ,x[ 1] )=0; R[ 2] :=diff(p[ 2] ,x[ 2] )=0;

RI := gl Sal glSal 2 xI -1=0

alXI +a2x2 (alxl+a2 x2)2

R2 := K2 S a2 K2 S a2 2 X2 _ 1= 0
al XI + a2X2 (al XI + a2 x2)2
The optimal advertising amounts XI, X2 and the profits PI, 1'2 are found as
follows.

> sol: =sol ve ({ R[ 1] , R[ 2] } ,{ x[ 1] , x[ 2] } ) ;

> assign(sol);

> simplify (P[ 1] ); simplify (P[ 2] );

al 2 Sgl3
(gl al + g2 a2)2
a2 2 Sg2 3
44 3. Maple and Mathematical Foundations of Operations Research

3.3 Calculus
Calculus plays an important role in mathematical modeling of operations re-
search problems. In this section we describe some examples of calculus tech-
niques whose implementation can be substantially simplified using Maple.

3.3.1 Limits
Computing the limits using 1 imi t () is easy with Maple. Consider the following
examples.
> restart: # Limits.mws
Maple knows that one dollar invested now that compounds continuously at a
rate of r% per annum will grow to e"t dollars in I years.
> f{1]:=(1+r/n)A(n*t);
r
II := (l + _)(nt)
n
> limit (f{ 1] , n=infinity) ;
e(rt)

The next limit operation needs I'Hopital's rule, which Maple supplies.
> f{ 2] :=(a"'x-1)/x;
tr - I
12:=--
x
> limit (f{ 2] , x=O);
In(a)
Maple recognizes that the next limit does not exist. However, if the direction is
specified as right or left, then Maple finds the limits.
> f{ 3] : =abs (x) / x;

/J:=~
x
> limit (f{ 3] , x=O) ;
undefined
> limit (f{ 3] , x=O, left) ;
-I
> limit (f{ 3] , x=O, right) ;

Maple also knows how to take multidimensional limits.

> limit ( (x+y) / (x* y) , { x=Pi/4, y=Pi/ 4} ) ;
 3.3 Calculus 45

8.!.
1r
In the next example Maple appears to be unable to find the result since its initial
attempt fails to produce a limit. However, we realize that Maple has done the right
thing since it does not know the sign of the parameter b. As soon as we inform
Maple that b > 0, the limit is correctly found as O.
> f[ 4] :=sin(a*x)*exp(-b*x);

14 := sin(ax)e(-bx)
> limit (f[ 4] , x=infinity);
lim sin(ax)e(-bx)
X-HXl

> assurne(b>O);
> limit (f[ 4] , x=infinity) ;
o

3.3.2 Derivatives
Partial Derivatives
Maple has two means of performing ordinary and partial differentiation: either it
uses the diff () function or the D () operator. The main (and subtle) difference
between the two is that the D operator computes derivatives of mappings (Le.,
functions), while di f f () computes derivatives of expressions.
Here are some examples.
We first define an expression in two variables f: =a* x"2* y+b* x* y"2; and
differentiate it with respect to x and y using di f f ( ). We also find the mixed
partial derivative di ff ( f, x, y) ; of f.
> restart: # Derivatives.mws
> f:=a*x"2*y+b*x*y"2;

f
:= a x 2 y + bxI
> diff(f,x); diff(f,y);

2axy+b y 2
ax 2 +2bxy
> di ff ( f, x, y) ;
2ax+2by
Next, we define a mapping 9 (i.e., a function) involving the same expression as
in f and perform differentiation using the D operator. Note that the output is also
obtained as a function.
> g:=(x,y)->a*x"2*y+b*x*y"2;
46 3. Maple and Mathematical Foundations of Operations Research

g:= (x, y) --+ ax 2 y + bx y2

> q 1] (g); q 2] (g);

(x, y) --+ 2 a x y + br
(x, y) --+ ax 2 + 2bx y
> q 1,2] (g);
(x, y) --+ 2 a x + 2 by
Finally, we see that using the 0 operator, one can differentiate composite func-
tions.
> f : =' f': g :=' g' :
> f:=g->COS(gA2);
f := g --+ coseg2)
> u:=v->cos(vA2);

> 0 (u@v) (x);

Example 21 Inventory model with lost sales. As another example, consider the
objective function of a deterministic inventory problem where lost sales are al-
lowed. Defining Q as the order quantity and T as the length of time during which
sales are lost, Hadley and Whitin [81, p. 48] show that the average annual cost C
is obtained as a function of Q and T. This cost function and its partial derivatives
are easily found as follows.
> restart: # LostSales.mws
> C:=lambda*K/(Q+lambda*T)+(1/2)*h*Q A2/(Q+lambda*T)
+pi*lambda A2*T/(Q+lambda*T);
C ._ A K h g2 ! 1C A. 2 T
.- Q + A T + 2 Q + A. T + Q + A. T
> CQ:=normal(diff(C,Q»;

CQ '= _! 2 A K - h Q2 - 2 h Q A. T + 21C A.2 T

. 2 (Q + A. T)2
> CT:=normal(diff(C,T»;

CT '= _! A. (2 A. K + h g2 - 21C A. Q)
. 2 (Q + A. T)2
 3.3 Calculus 47

We can even solve for the (Q, T) pair satisfying the necessary conditions. It is
clear that the sign of (1r Ai - 2A.Kh plays an important role in the detennination
of the optimal solution. For details, see Hadley and Whitin [81, p. 49].
> solve({ numer(CQ) ,numer(CT)} ,{ Q,T} );

-2 K +%I1r}
{Q=%I, T=
%1 h -1r A.
%1 := RootOf(2 A. K + h _Z2 - 21r A._Z)
> allvalues(%);

Leibniz's Rule
In some operations research problems (especially those arising in inventory the-
ory) the objective function involves an integral whose upper and/or lower limits
may be a function of the decision variable. Such integrals must be differentiated
using Leibniz's rule of differentiation under the integral sign. Fortunately, Maple
is aware of this rule as the following statements demonstrate.
> restart: *
Leibniz.mws
> C:=Int(g(Q,t),t=a(Q) .. b(Q));

C := l
a(Q)
b(Q)
g(Q, t) dt
> diff (C, Q) ;

l b(Q)

a(Q)
fa g(Q, t) dt + (fa b(Q» g(Q, b(Q»
- (fa a(Q» g(Q, a(Q»
> C:='C':
As an example, consider the following cost function that appears in the analy-
sis of a continuous-review inventory model where lead-time randomness can be
reduced by investment; see Gerchak and Parlar [70]. The decision variables are Q
(order quantity), r (reorder point) and a (investment amount). The partial deriva-
48 3. Maple and Mathematical Foundations of Operations Research

tives of the cost function w.r.t. the three variables are easily computed using the
dif f () function.
> C:=K*lambda/Q+h* (Q/2+r-mu)+b*lambda*alpha
*Int(FBar(x),x=u(r,alpha) .. infinity)/Q
+G(Q,alpha);

C.= K)' +h(!Q+r-Jl)+

b ). a 1 00

u(r.a)
FBar(x) dx
. Q 2 Q
+G(Q, a)
> CQ:=diff(C,Q);

CQ := - ~; + ~ h _
b ). a 1~ 00

u(r.
FBar(x) dx

+ (~G(Q, a»
> Cr:=diff(C,r);
b).a (t,. u(r, a»FBar(u(r, a»
Cr:= h - --~--~---~
Q
> Calpha:=diff(C,alpha);

Calpha :=
b ).1 00
FBar(x) dx
_..:.....;;;;u(:;.:r•..:;;a'-)_ _ __
Q
b).a (%0. u(r, a»FBar(u(r, a» iJ
Q + (iJa G(Q, a»
As discussed in Gerchak and Parlar [70], these equations are solved numeri-
cally to find the optimal decisions.

Implicit Differentiation
In some problems it may be impossible to express uniquely the dependent vari-
able in terms of the independent variable. For example, the expression (i.e., the
r
relation) x 2+ = r2 represents a circle of radius r, but solving for the dependent
variable y one obtains y = ±Jr2 - x 2, indicating that y is not a function of x. In
those cases, to compute the derivative of y w.r.t. x, we use implicit differentiation
that has been implemented by the Maple function implicitdi ff (f, y, x) ,
which computes dy / dx, the (implicit partial) derivative of y with respect to x.
In the first example, we implicitly differentiate x 2 + = r2 and obtain r
dy/dx = -x/yo
> restart: # ImplicitDiff.mws
> implicitdiff(x A 2+y A 2=r A 2,y,x);
 3.3 Calculus 49

x
y
Example 22 Continuous-review inventory problem. A more difficult case involves
(yet another) inventory example where the partial derivatives of a particular cost
function C(Q, r) are found as
I
CQ = + 7r: B(r)] + "2hgl = 0
-)'[A
Cr = hQ -7r:).K(r) = 0
where B(r) and K(r) are functions ofr. Implicitly differentiating CQ and Cr to
compute dr/dQ, we obtain the following.
> CQ:=-lambda* (A+pi*B(r))+h*Q A 2/2;
1
CQ:= -). (A + 7r: B(r» +"2 h gl
> Cr:=h*Q+pi*lambda* (-K(r));
Cr := h Q - 7r: ). K(r)
> implicitdiff(CQ,r,Q);. r (Q) in CQ
hQ
). 7r: O(B)(r)
> implicitdiff(Cr,r,Q);. r (Q) in Cr
h
). 7r: O(K)(r)
These results can be used to investigate the shape of the curves represented by
CQ = 0 and Cr = 0 in the Qr-plane to see if there is a unique solution to the
problem; see Hadley and Whitin [81, p. 170].
Maple can also compute the second-order implicit derivatives d 2r /dgl for
each relation and obtain the following results.
> implicitdiff(CQ,r,Q$2);. r'(Q) in CQ
h «O(2»(B)(r) h g2 -). 7r: O(B)(r)2)
).2 7r:2 O(B)(r)3
> implicitdiff(Cr,r,Q$2);. r'(Q) in Cr
(O(2»(K)(r) h2
).2 7r:2 O(K)(r)3

3.3.3 Integrals

J:
Maple possesses a truly impressive knowledge of integration. A symbolic integral
such as x 2 dx can easily be evaluated to obtain (b 3 - a 3)/3:
> restart:. Integrals.mws
50 3. Maple and Mathematical Foundations of Operations Research

> Int(x A 2,x=a .. b);

> value(%);

!bl _ !al
3 3
This is, of course, a very simple problem that could have been quickly solved
by anyone who has some passing knowledge of integrals. But consider the fol-
lowing problem that arose in aerospace engineering in the study of turbulence
and boundary layers. 2

Example 23 An indefinite integral arising in aerospace engineering. The func-

tion to be integrated is given by f(x) = (k In(x) - 2xl + 3x 2 + b)4.
> f:=(k*log(x)-2*x A 3+3*x A 2+b)A4;
f:= (kln(x) - 2x 3 + 3x 2 + b)4
When expanded, the expression whose integral is sought involves 35 terms.
Evaluating this integral manually may be possible if one is willing to spend sev-
eral hours to check and recheck every term in the integration process. Even then,
there still is a possibility that human errors may creep into the computations and
one may be unsure of manually obtained results.3 However, by using Maple, we
quickly find the result.
> expand(f); nops(%);

16x l2 - 96x" + 216x lO + b4 + I08kln(x)x 6 +4k l ln(x)l b

+ 24 ~ In(x )2 x 6 - 72 ~ In(x )2 x 5 + 54 ~ In(x)2 x4
+ 6 k2 In(x)2 b2 + 12 klln(x)l x 2 - 8 k l ln(x)3 xl
+ 108x6 b + 144x 8 b + k41n(x)4 + 24x 6 b2
- 216x 7 b - 72x 5 b2 - 8x l bl + 54x 4 ~ + 12x2 bl
- 216x 9 + 8lx 8 + 48kln(x)x 6 b - 144 kIn(x)x 5 b
- 24k In(x) xl b2 + 108k In(x)x 4 b
+ 36k In(x)x 2 b2 - 24k21n(x)2 xl b
+ 36 k2 1n(x)2 x 2 b - 32 k In(x) x 9 + 144 k In(x)x 8
- 216k In(x) x 7 + 4k In(x)b l - 32x 9 b
35

2Tbis problem appeared in a Macsyma advertisement in the September 1984 issue of ScienJijic
American.
3Legend has it that the engineer who was working on this problem tried to evaluate the integral
for more than three weeks with pencil and paper, always arriving at a different result. With Maple the
correct solution is found in a fraction of a second.
 3.3 Calculus 51

> Result:=int(f,x);

Result := k4 x In(x)4 - 4 k4 x In(x)3 + 12 k4 x In(x)2

- 24k4 x In(x) + 3 Jc2 bx 4 1n(x) + 4k bx 6
_ 108 kbx 5 _ 108 k 2 x 5 1n(x)-2k3 x 4 1n(x)3
25 25
+ - k 3 x41n(x)2 - -3 k 3 x 4 1n(x) + -24 Jc2 x71n(x)2
3
2 4 7
48
- 49 Jc2 x 7 1n(x) + 4k2 x 6 1n(x) - 4k~ x 3 - 24k3 bx

+ 12k2 b2 x - 4k 3 x 3 1n(x)2 +!3 k 3 x 3 1n(x) - 4kb 3 x

- 6 k ~ x 4 1n(x) + 4 k b 3 x In(x) + 4 k 3 b x In(x)3

- 12k3 bx In(x)2 + 24 k 3 bx In(x) - 6Jc2 bx 4 1n(x)2
+ 6k2 ~ x In(x)2 - 12k2 b 2 x In(x) - 8k2 bx 3 1n(x)
8 ._2 3 b 16 13
+-.rx +-x +- x - - x - - x
8 k 10 16 k 9 16 b 10
3 13 25 9 5
b4 24 b2 7 48 k2 7 108 k 7 108 k2 5
+ x + "1 x + 343 x - 49 x + 125 x

- !9 k 3 x 3 _ ~ Jc2 x 6 -
3
2 b3 x4 + 27 k x 8 - ~ Jc2 b x4
8 4
_ ~ kxl0ln(x) - 48 kbx7 + ~ k b2 x4
5 49 2
+ - k bx7In(x) + 24k4 x - 8x l2 + -216 xII
48
7 II
108 lO 54
- - x - 12Jc2In(x)2 x 6 + - Jc2ln(x)2 x 5
5 5
+ 4k 3 1n(x)3 x 3 _ 27 x 8 b _ 12x 6 ~ + 108 x 7 b
7
54
+ _x 5 b2 +4x 3 b 3 + 9x 9 - 24kln(x)x6 b
5
108
+ -5- kln(x)x 5 b + 12kln(x)x 3 b2
+ 12Jc2ln(x)2 x 3 b + 16k In(x)x 9 - 27 k In(x) x 8
108 3
+ -;;- kln(x)x7 + 16x9 b + 16 k 3 x4
> simplify(diff(Result,x)-f);
o

Differentiating the (horrendous looking) result and subtracting it from the orig-
inal expression gives 0, indicating that Maple was able to find the correct solution.
52 3. Maple and Mathematical Foundations of Operations Research

Example 24 Definite integral of the unit normal density. Maple's knowledge of

integration also includes numerical evaluation of integrals for which no closed-
form solution is available. For example, to evaluate the probability that the unit
normal random variable Z '" N(O, I) takes a value between, say, -0.5 and 0.35,
we must compute

Pr(-0.5 ~ Z ~ 0.35) = 1 0.35

-0.5
t/J dz

where
I
t/J = --e-~-.
1.2

-.Iii
Since functions of the form e-~ cannot be evaluated analytically, this integral has
to be computed numerically.
> restart: # NormalDensity.mws
> phi:=(1/ (' sqrt(2*Pi)' ) )*exp(-z"2/2);
e(-1/2=2)
t/J._-=-
.- .ffi
> Int('phP ,z=-0.5 . . . 35); evalf(%);

1 ·35

- .5
t/Jdz

.3282931124
Thus, Pr( -0.5 ~ Z ~ 0.35) = .3282931124. The fact that this is the correct
result can be seen by finding the probability by an alternative method that uses
Maple's knowledge of the distribution of the normal density. Defining <I>(z) =
J:oo t/J(u) du, we find the same result from <1>(0.35) - <1>(-0.5) = .3282931124.
> with(stats):
> Phi:=z->statevalf{ cdf,normalc{ 0,1]] (z);
<I> := stateva/fcdj, norma/do, 1
> Phi(.35)-Phi(-0.50);
.3282931124

Example 25 Integration of the exponential density. As another example, let us

consider the integration of the exponential density f(/) = Ae- lt , 1 ~ O. Since
f(t) is a density, we should have Jooo Ae- lt dt = I, provided that A > O.
> restart: # ExponentialDensity.mws
> f:=lambda*exp(-lambda*t);
f:= Ae(-lt)
 3.3 Calculus 53

> int(f,t=O •. infinity);

Definite integration: Carr t determine if the integral is
convergent.

Need to know the sign of --> lambda

Will now try indefinite integration and then take limits.

lim - e(-At) +1
t~oo

Naturally, Maple is unable to evaluate the integral since we have not specified
the sign of the parameter A.. As soon as we indicate, using assume ( ) , that A. > 0,
Jo
Maple correctly integrates the density and finds oo A.e- At dl = I.
> assume(lambda>O);
> int(f,t=O .. infinity);

Example 26 A multiple integral. Maple can also evaluate multiple integrals. As

an example, consider a problem in probability where X and Y are independent
uniform random variables distributed over the interval (0, a) and we wish to eval-
uate the distribution Fw(w) of the product W = XY. It can be shown (see, Siva-
zlian and Stanfel [177, p. 54]) that this problem can be solved by evaluating the

1a1a
double integral

Fw(w) = 1- 1
1.dxdy,
wla wlya
We find the result using Maple's help:
> restart: *
Doublelntegral.mws
> F:=1-Int(Int(1/a A 2,x=w/y .. a),y=w/a .. a);
F:=unapply(value(F),w);

F := 1 - lla
- -a
a y
a
w w 1.
1 dx dy

w
_a2 + w + wIn(a) - wIn(-)
F~w~l+ a
a2
> simplify(F(w));
w
w (I + In(a) -In(-))
a
a2
> limit(F(w),w=O); F(a A 2);

°
54 3. Maple and Mathematical Foundations of Operations Research

3.3.4 Finite Sums and Infinite Series

Maple also knows how to compute finite sums and infinite series as we demon-
strate in the next few examples.
> restart: *
SumsSeries.mws
> Sum(a*xAk,k=O .. n-l); value(%);
n-\
Lax le
1e=O

ax n a
- -
x-I
- - -
x-I
The next sum is obtained correctly, but one should be careful in using it, as the
result applies only when -I < x < I.
> Sum(a*xAk,k=O .. infinity); value(%);
00

Lax le
1e=O
a
x -I
> Sum(k A3*x Ak,k=1 .. infinity); value(%);

Lk x
00
3 le
Ie=\

I x x2
x «x _ 1)2 - 6 (x _ 1)3 + 6 (x - 1)4)
> Sum(xAk/k,k=l .• infinity); value(%);
x le
t;T
00

-1n(I - x)
> factor(simplify(sum(k,k=l .• n)));
I
2 n (n+l)
> factor(simplify(sum(k A2,k=1 .. n)));
I
6n(n+ 1)(2n+ I)
> factor(simplify(sum(k A3,k=1 .• n)));

4I n2 (n+I)2
 3.4 Linear Algebra 55

Example 27 Geometric random variable. The next example concerns the sum
of the probabilities of the geometric random variable N such that Pr(N = k) =
p(1- p)k for k = 0, 1,2, ... Provided that 0 < p < I, the sum L~P(1- p)k
converges to I. Since Maple does not check for the correct interval of conver-
gence of these infinite sums, it is up to the user to find out about the convergence
properties.
> Surn(rho* (l-rho)Ak,k=O .. infinity); value(%);

Example 28 Moment generatingfunction ofthe binomial. The moment generat-

ing function of the binomial random variable is defined as tP(t) = Lk=O e'k C)
pk(1 - p)n-k. Maple easily finds the result that involves the use of the binomial
theorem, i.e., (x + y)n = Lk=O (Z)xkyn-k.
> rngf:=Surn(exp(t*k)*binornial(n,k)*pAk* (l-p)A(n-k),
k=O .. n) ;

mgf:= L e(1 k) binomial(n, k) pk (I _ p)(n-k)

n

k=O
> assurne(n,integer); sirnplify(value(rngf));

3.4 Linear Algebra

Linear algebra is probably as important as calculus in the modeling and anal-
ysis of operations research problems. In this section we demonstrate Maple's
deep knowledge of linear algebra by providing examples relevant to operations
research.

3.4.1 Matrix Operations

A matrix in Maple is represented as a two-dimensional array with rows and columns
indexed from I. Matrices can be input either directly, as a two-dimensional array,
or using the matrix command in the linear algebra package.
Consider the following examples.
After loading the linear algebra package linalg, we define two matrices A
and B each having a symbolic entry. Matrix algebra (such as multiplying the ma-
trix A by a constant y, adding A and D, multiplying A and D) is performed using
the command evalrn ().
> restart: # Matrices.rnws
56 3. Maple and Mathematical Foundations of Operations Research

> with (linalg) :

Warning, new definition for norm

Warning, new definition for trace

> A:=matrix(2,2,[ 1,3,2,a]);

A := [ ; !]
> B: =matrix (2, 2,[ b, 4,3,7] ) ;

B:= [~ ~]
> evalm(gamma*A);

3Y ]
ya
> transpose (A) ;

> evalm(A+B);

[ I +h 7 ]
5 a+7
The detenninant of a matrix is computed with the det () command. We see in
this example that as long as the parameter a "# 6, the A matrix should possess an
inverse, A-I, which we compute with the inverse () command. The result is
checked by multiplying the inverse by the original matrix.
> det (A);
a-6
> AI: =inverse (A) ;

Ab[ ]
a I
-3--
a-6 a-6
I I
-2--
a-6 a-6
> simplify(evalm(A &* AI));

[~ ~]
An easy way to input an n x n identity matrix is to use the array (identi ty
,1 •. n, 1 .. n) command.
> ID:= array(identity,1 .. 2,1 .. 2);
ID := array(itientity, 1..2, 1..2, [])
 3.4 Linear Algebra 57

> print (ID) ;

> evalm(ID &* A);

Vectors can be input in a manner similar to matrices. However, the shape of a

vector (i.e., whether it is a row or column vector) is decided by Maple when the
vector is involved in a vector/matrix mUltiplication.
> C:=vector(2,( 3,c] );
C:= [3, c)
> CA:=evalm(C &* A);
CA:= [3 +2c, 9+ca]
> AC:=evalm(A &* C);
AC:=[3+3c,6+ca]
If it is important to have control over the actual dimensions of a vector, it should
be input as an n x I matrix (i.e., a column vector) or as a I x n matrix (Le., a row
vector).
> evalm(C &* C);
Error, (in evalm/evaluate) improper op or subscript
selector
> C:=matrix(2, 1,[ 3,c] );

C:= [ !]
> evalm(A &* C);

[ 3+3C]
6+ca
> C:=matrix(1,2,[ 3,c] );

C:= [3 c]
> evalm(C &* A);

[3+2c 9+ca]

The next example evaluates the product Clx2A2x2Cfxl' which results in a

scalar.
> evalm(C &* A &* transpose(C));

[9+6c+(9+ca)c]
58 3. Maple and Mathematical Foundations of Operations Research

3.4.2 Solution o/Simultaneous Linear Equations

Solving linear equations plays an important role in many branches of operations
research. Consider the linear system Ax = b where A is an n x n matrix, b is an
n x I column vector and x is also an n x I column vector of unknowns. If A is
=
nonsingular (i.e., if its inverse exists), then we may solve for x as x A-I b. The
computations involvingA-1bcan be performed in Maple with the linsolve ()
command.
Consider a simple example.
> restart : *
SLE.mws
> with (linalg) :
Warning, new definition for norm

Warning, new definition for trace

> A: =matrix ([ [ 3,1,1) ,[ 1,2,1) ,[ 1,1,7)) );

A := [i ~ :]
1 1 7
We find (before solving the problem) that the inverse of A exists; hence we
should be able to compute the solution vector x.
> inverse (A) ;
13 -3 -I
32 16 32
-3 5 -I
-
16 8 16
-I -I 5
32 16 32
> b : =vector([10,20,30);
b := [10, 20, 30]
Using the geneqns () function, we can view the linear equations arising from
the system Ax = b.
> geneqns (A,[ x, y, z) ,b);
{3x + y + z =
10, x + 2y + z = 20, x + y + 7 z = 30}
> linsolve(A,b) ;

[ -5 35 25J
8 ' 4' 8
Thus, we find x = (-5/8,35/4,25/8) as the solution of the system.
It is, of course, possible that some linear systems may have infinitely many
solutions or no solutions at all. This is determined by computing the rank of a
matrix. The rank ofa matrix A [denoted by r(A)] is defined as follows: If there
 3.4 Linear Algebra 59

exists at least one nonsingular square submatrix in A of order k and if all square
submatrices of order k + i, i ~ 1 (ifany) are singular, then r(A) = k.
Consider an n x n matrix A and an n x 1 vector b. The A matrix augmented
with b is denoted by (A I b). The following rules are useful in determining the
nature of the solution:

• If, r(A) = r(A I b) = n, then the system has a unique solution.

• If r(A) = r(A I b) = m < n, then the system Ax = b has an infinite

number of solutions.

• Finally, ifr(A) < r(A I b), then the system has no solution.

These results are illustrated in the following examples.

The example with n = 3 had a unique solution and this is again noted by the
fact that r(A) = r(A I b) = 3.
> Ab:=augment(A,b);

Ab:= [l ~ ! ~~]
1 7 30
> rank(A); rank(Ab);
3
3
For the next example with n = 2, we see that r(A) = r(A I b) = 1 < 2; thus
the system has infinitely many solutions.
> A:=matrix([[ 3,1],[ 6,2]]);

A := [~ ~]
> b :=vector ([ 4, 8] ) ;
b := [4,8]
> geneqns(A,[ x,y,z] ,b); linsolve(A,b);
{3x+y=4,6x+2y=8}
LIt. -3 _1\ + 4J
> Ab:=augment(A,b);

Ab:= [~ ~ :]
> rank(A); rank(Ab);
60 3. Maple and Mathematical Foundations of Operations Research

The final example with n = 2 involves a system that has no solution. This is
observed by the fact that r(A) = I < r(A I b) = 2.
> A:=matrix([[ 1,1),[ 1,1)));

A := [! !]
> b: =vector ([ 2, 4) ); linsol ve (A, b) ;
b:= [2,4]
> geneqns (A,[ x, y) , b); linsol ve (A, b) ;
{x + y = 2, x + Y = 4}
> Ab:=augment(A,b);

Ab:= [ 12]
I 4
> rank(A); rank(Ab);

Homogeneous System of Linear Equations

As we pointed out, every system of linear equations has either (i) one solution,
(ii) infinitely many solutions or (iii) no solutions at all. We now consider the case
of a homogeneous system of n linear equations in n unknowns, i.e., Ax = 0, and
discuss the conditions under which the system has solutions.
First note that if det(A) =f:. 0, that is, if A -I exists, then the zero vector x =
A-IO = 0 is a trivial solution. However, if det(A) = 0, then infinitely many
nontrivial solutions exist, as we see in the following example.
> restart: *
HomoSLE.mws
> with (linalg) :
Warning, new definition for norm

Warning, new definition for trace

> A:=matrix([[ 1,5,3),[ 5,1,-1),[ 1,2,1)));

> det (A);

15
A:= [ 5 I
I 2
-:]
o
> basis ([ col (A, 1) , col (A, 2) , col (A, 3») ) ;
[[1,5, I], [5, 1,2]]
 3.4 Linear Algebra 61

> b: =vector ([ 0, 0, 0] ) ;
b := [0,0,0]
> geneqns (A,[ x, y, z] ,b);
{x + 5 Y + 3 z = 0, 5x + y - z = 0, x + 2 y + z = O}
> linsolve(A,b);

[-~2 tl- ' -tl' -~2-tl]

Since we find that x = -t /2, y = -t and z = - 3t /2, the system has infinitely
many nontrivial solutions.

3.4.3 Eigenvalues, Eigenvectors and Diagonalization

Let A be an arbitrary square matrix of order n, and let x be an n-dimensional
nonzero vector. The vector x is called an eigenvector of A if Ax is a scalar multiple
of x, i.e., Ax = AX for some scalar A. The scalar A is called the eigenvalue of A
and x is said to be an eigenvector corresponding to A. To find the eigenvalues of
A we rewrite Ax = h as Ax = AIx (where I is the n x n identity matrix) or
equivalently as
(A - AI)x = O. (3.1)
For A to be an eigenvalue, the system (3.1) must have a nonzero solution x#; O.
But this implies that the columns of (A - AI) are linearly dependent and (A - AI) is
singular. Hence, the scalar A is an eigenvalue of A if and only if det(A - AI) = O.
Expanding this gives a polynomial in terms of A, which must be solved to find
the eigenvalues. (Some eigenvalues may be repeated.) To find an eigenvector X;
corresponding to the eigenvalue Ai, we form the system (A - Ai I)X; = 0 and solve
for the vector Xi.
These results are useful in determining whether the original matrix A can be
diagonalized. That is, does there exist a Q matrix such that Q-I AQ = D is a
diagonal matrix? It can be shown that the matrix A is diagonalizable if and only if
A has linearly independent eigenvectors and that Q is formed using eigenvectors
of A. An additional result states that if the eigenvalues of A are distinct, then A
can be diagonalized; see Anton [7, p. 277]. If A can be diagonalized, then the
nth power of A can be computed as An = QonQ- I. Since D is diagonal, its nth
power can be computed much more quickly than the nth power of A.
Consider the following example to illustrate these ideas.

Example 29 Transition probabilities ofa three-state Markov chain. Suppose we

have a three-state discrete-time Markov chain with the transition probability ma-
trix A given as

!
o
I tI)
! .
! 0
62 3. Maple and Mathematical Foundations of Operations Research

We wish to compute, say, the 10th power of this matrix to find the probability
that the chain will be in some state j = 1, 2, 3 after 10 transitions given that it
is in some state i = 1, 2, 3 now. In order to do this, we find the eigenvalues and
the corresponding eigenvectors of A, diagonalize it as D and form the Q matrix.
Computing Q IY' Q-I then gives the nth power of A.
> restart: # EigensMC3.rnws
> with(linalg):
Warning, new definition for norm

Warning, new definition for trace

> A:=rnatrix (3,3,[ 0,1/2,1/2, 1/2,0,1/2, 1/2,1/2,0]);

1
o 2 2
.-
A'- - 0 -
2 2
1
- - 0
2 2
We could use Maple's charpoly () function to find the characteristic poly-
nomial, i.e., det(A - AI) = 0, and then compute the polynomial's roots for the
eigenvalues, which are obtained as AI = 1, A2 = A3 = -!.
(Note that the eigen-
value -!
is repeated; thus its multiplicity is 2.)
> CharacteristicPolynornial:=charpoly(A,larnbda);

CharacteristicPoiynomiai := A3 - 43 A - 41
> solve(CharacteristicPolynornial);
-1 -1
I'T'T
However, it is easier to use the eigenvectors () function to find all eigen-
values and the corresponding eigenvectors v[ 1] and v[ 2] .
> v:~eigenvectors(A)];

-1
v := [[1, 1, HI, 1, 1m, [T' 2, H-l, 0, 1], [-1, 1, Om]
> v[ 1]; v[ 2] ;
[1, 1, ([1, 1, 1m
-1
[T' 2, ([-1,0, 1], [-1, 1, om
Note here that the list v[ 1] contains the information related to the eigenvalue
1 and its eigenvector: This eigenvalue's multiplicity is 1, and its eigenvector is
 3.4 Linear Algebra 63

[ 1, 1, 1] . The list v[ 2] indicates that the mUltiplicity of eigenvalue -1/2 is 2

and the corresponding eigenvectors are [ -1, 0, 1] and [ -1, 1, 0] .
> Eigenvectors: ={ v[ 1][ 3, 1], v[ 2][ 3, 1], v[ 2][ 3,2] ] ;
Eigenvectors:= [[1, 1, 1], [-1,0, 1], [-1, 1,0))
Using the basis () function, we note that the eigenvectors are linearly inde-
pendent; thus the A matrix is diagonalizable.
> basis(Eigenvectors);
[[1, 1, 1], [-1,0, 1], [-1, 1,0))
> Q:=transpose(rnatrix(3,3,Eigenvectors»;

1
Q:=
1
[! -~ -!]
0
> Diagona1izedA:=evalrn(inverse(Q) &* A &* Q);

10

~1 1
-1
DiagonalizedA := [ 0 T
o 0
Next, we evaluate QDIOQ-I to find the 10th power of A. As a check, we also
compute A 10 directly and see that the results agree.
> evalrn(Q &* (DiagonalizedA)A10 &* inverse (Q) ) ;
171 341 341
-
512 1024 1024
341 171 341
1024 512 1024
341 341 171
1024 1024 512
> evalrn(AA10);
171 341 341
512 1024 1024
341 171 341
-
1024 512 1024
341 341 171
1024
--
1024
-512

3.4.4 Least Squares Fitting to Data

An interesting application oflinear algebra is to "fitting" a curve to a set of experi-
mentally obtained n data points (XI, YI), (X2, n), ... , (xn, Yn) in order to develop
64 3. Maple and Mathematical Foundations of Operations Research

a mathematical relationship of the form Y = f(x). Based on theoretical consid-

erations or the graphical display of data that may reveal a pattern, a general form
such as Y = a + bx or y = a + bx + cx 2 + dx 3 may be used. The curve-fitting
problem then involves determining the parameters (coefficients) a, b, etc., of the
model using the given n data points.
Consider the simplest case of fitting a straight line y = a + bx to a given set
of n data points. If the data were all on the same line (i.e., collinear), then the
coefficients would satisfy YI = a + bXI, ... ,Yn = a + bXn, which can be written
in matrix form as y = Xa where

y =( ~I )
Yn nxl
X = (I
I
X:I)
Xn nx2
, a=(~) 2xl .
Thus, for collinear data points, we would have y - Xa = O. However, when the
data are not collinear, it is impossible to find a and b that satisfy y = Xa. In this
case one attempts to find a vector a that minimizes the Euclidean length of the
difference (deviation) lIy-Xall 2 = :L7=1[Y; - (a + bx;)j2. It can be shown [7,
Section 8.3] that the vector minimizing the deviations is given by

a = (X'X)-IX'y.

Thus, the coefficients a and b can be found by matrix algebra that involves ma-
trix inversion and matrix multiplications. This has been implemented by Maple,
as we will demonstrate.
The same technique can be generalized to fitting a polynomial of degree m, i.e.,
Y = ao + alx + ... + amxm to the n data points (XI, YI), (X2, n), ... , (xn, Yn).
In this case the X becomes an n x (m + I) matrix and a an (m + I) x I vector.
The optimal solution is again found as a = (X'X)-I X'y.
Maple's implementation of the least squares solution is provided in the fi t ()
subpackage of the stats package. Thus both of these packages must be loaded
before doing least squares computations.
> restart: # LeastSquares.mws
> with(stats); with(fit);

[anova, describe, fit, importdata, random, statevalf,

statplots, transform]
[/eastmediansquare, leastsquare]
The first example fits a straight line Y = a + bx to four data points with coor-
dinates defined as lists xData and yData. Next, using the leastsquare{ ]
function, we find that Y = 18.12 - 1.03x.
> xData:={ 0,1,2,3];
xData := [0, I, 2, 3]
 3.4 Linear Algebra 6S

> yData:={ 21, .3,41,4];

yData:= [21, .3,41,4]

> leastsquare[ [ x, y], y=a+b* x] ([ xData, yData] ) ;

y = 18.12000000 - 1.030000000x
In the next example, we fit a quadratic y = a + bx + cx 2 to five data points and
find that y = 1.05 - I2.03x + 35.5x 2 •

> xData:={ .1, .2, .3, .4, .5]; nops(xData);

xData:= [.1, .2, .3, .4, .5]
5

> yData:={ .18,.31,.03,2.48,3.7~; nops(yData);

yData:= [.18, .31, .03,2.48, 3.73]
5

> leastsquare[ [ x, y], y=a+b* x+c* x"2] ([ xData, yData] ) ;

y = 1.050000000 - I 2.03000000 x + 35.50000000x 2

The next example finds the analytic solution for the coefficients a and b in
linear regression.

Example 30 Coefficients of linear regression equation. We now directly mini-

mize the sum L7=1
[y; - (a + bx;)j2 by using calculus techniques.

> restart: * LSEClosedForm.mws

> f:=Sum«y[ i) -a-b*x[ i) )"2,i=1..n);
n
f := L (y; - a - b x;)2
;=1

> fa:=diff(f,a); fb:=diff(f,b);

n
fa:= L (-2y; + 2a + 2bx;)
;=1
n
fb:= L(-2(y; -a-bx;)x;)
;=1

> solve({fa,fb},{a,b}):
66 3. Maple and Mathematical Foundations of Operations Research

n n n n
b= (~x; y;) (~I) - (~x;) (~Y;)

_(t (t (t
{

x ;)2 + X;2) I)

a= _ -<t,J<) (t.Xi') + <t,Xi)(t,Xi Yi) }

-(t (t ;2) (t
x;)2 + x I)

This is a rather impressive result since Maple was not given the specific value
of n for the number of data points. This approach can be used even for fitting data
to a higher-order polynomial such as the quadratic, cubic, etc.

3.4.5 Special Matrices

In operations research applications certain special matrices and vectors are fre-
quently encountered. For example, the Hessian matrix is used to check the con-
vexity/concavity of a multivariable function. The Jacobian matrix is used to find
the joint probability distribution of functions of random variables. The gradient
of a function is a vector of first partial derivatives of the function. Maple has im-
plemented these (and other matrices) in the linalg package. The next example
illustrates the use of these matrices.
> restart: *
SpecialMatrices.mws
> with (linalg) :
Warning, new definition for norm

Warning, new definition for trace

The Hessian matrix H of a function of three variables I = I(x, y, z) is com-
puted as

H= (Z: fzx
7~
f,y
Z:)
/zz
.
In our example, I(x, y, z) = x 2yz + 3xrz and the Hessian is
> f:=x"2*y*z+3*x*y"2*z;

I := x 2 y z + 3 x rz
> hessian(f,[x,y,z));

2yz
[ 2xz+6yz
2xz+6yz 2X Y +3
6xz x 2 +6xy
r]
2xy+3r x 2 +6xy o
 3.4 Linear Algebra 67

If each component of a vector A is a function of two variables, say, A =

[/(r, 8), g(r, 8)], then the Jacobian matrix is given by

(i I),
In our example, A = [r cos(8), r sin(8)] and the Jacobian is
> A:= vector([ r*cos(theta) , r*sin(theta)));
A := [r cos(8), r sin(8)]
> jacobian (A, [ r, theta) ) ;

[ cos(8) -r sin(8) ]
sin(8) r cos(8)
> simplify(det(%));
r
Using the function I, we find its gradient VI = th, Iy , 1=] as
> grad(f, [x,y, z) );
[2xyz+3,rz, x 2 z+6xyz, x 2 y+3x,r]

3.4.6 Positive Definiteness

Maple has implemented a procedure called defini te () for checking the con-
vexity or concavity of a multivariable function-or, equivalently the positive (or
negative) definiteness (or semidefiniteness) of its Hessian. This can be very useful
in optimizing nonlinear functions as we see in the following example.
> restart: * PositiveDefinite.mws
> with(linalg): with(student):
Warning, new definition for norm

Warning, new definition for trace

The interface () command below asks Maple to indicate the variables with
assumptions (such as positive, real, integer, etc.) as a list displayed at the end of
an expression.
> interface(showassumed=2);
> assume (x,real, y,real, z,real):
> f:=(x,y,z)->exp(x-y)+exp(y-x)+exp(x A 2)+z A 2;

1:= (x, y, z) ~ e(x-y) + e(Y-x) + e(x 2 ) + z2

> H:=hessian(f(x,y,z),[ x,y,~);
68 3. Maple and Mathematical Foundations of Operations Research

e(X-Y) + e(Y-X) + 2 e(x2) + 4 x 2 e(x2) _e(X-Y) - e(Y-x) 200 ]

H := [ -e(x-y) _ e(Y-x) e(x-y) +
e(Y-x)

o o
with assumptions on x and y
> definite(H,'positive_def');

_e(X-Y) - e(Y-x) - 2 e(x2)4 x 2 e(x2) < 0 and

-

_(e(x-y) + e(Y-x) + 2 e(x2) + 4 x 2 e(x2» (e(x-y) + e(Y-x»

+ (-e(x-y) _ e(y-x»2 < 0 and - 4 e(x2) e(x-y) - 4 e(x2) e(Y-x)
- 8x 2 e(x2) e(x-y) _ 8x 2 e(x2) e(Y-x) < 0
with assumptions on x and y
By using the is () function, we see that these conditions are satisfied for all
real x, y and z, thus making the function a convex one.
> is(%);
true
We can now use the extrema () function to optimize (i.e., minimize) f. We
find that f's optimum value is 3 and the optimal solution is at (x, y, z) = (0, 0, 0).
> extrema (f (x, y, z),{} ,{ x, y, z} " sol' );
{3}
> sol;

{{z = 0, x = 0, y = On
with assumptions onz, x andy

3.5 Differential Equations

Ordinary differential equations (ODE) play an important role in modeling pr0b-
lems where the system's behavior is time-dependent. For example, in order to find
the probability that a queueing system will be in a particular state at a finite time t,
one normally needs to solve a system of ordinary differential equations. Similarly,
applications of optimal control theory require a good understanding of differential
equations and their exact or numerical solution.
Partial differential equations (POE) have also found applicability in operations
research in the analysis of some types of queues where they are used to repre-
sent the generating function of transient probabilities. To optimize some types of
continuous-time systems, one usually applies Bellman's principle of optimality,
which results in the so-called Hamilton-Jacobi-Bellman partial differential equa-
tion.
 3.5 Differential Equations 69

In this section we describe Maple's powerful differential equation solvers and

discuss some operations research applications where the solution can be obtained
analytically. We also consider some problems that must be solved nwnerically
since either an exact solution is not possible or Maple cannot find it.

3.5.1 Ordinary Differential Equations

First, we examine some problems in ordinary differential equations.

Analytic Solutions
Example 31 Savings account. One of the simplest ordinary differential equations
arises in the modeling of growth of cash deposited in a bank account. If the ini-
tial deposit in a savings account is So dollars and the savings grows at r percent
per annwn compounded continuously, then the first-order homogeneous linear
ordinary differential equation with a constant coefficient representing the level of
savings 8(t) at time t is given by

d~~t) = r8(t). 8(0) = So given.

Maple solves this ODE as follows.
> restart: *
Savings.rnws
> DE1:=diff(S(t),t)=r*S(t);
DEI := f, S(t) = r S(t)
> Ie: =S (0) =S[ 0] ;
Ie := S(O) = So
> Sol1:=dsolve({DE1,IC) ,S(t»;
Soil := S(t) = So e(r t)
Thus we find that at time t the level of savings account is given as 8(t) = Soer t .
The fact that the solution is correct is verified using the odetest () function,
which results in a 0 signaling that Maple has solved the ODE correctly.
> odetest(Soll,DE1);
o
When the interest rate r(t) is a function of time t, then the ODE becomes
d8(t) .
-;It = r(t)8(t), 8(0) = So gIVen.
This is a slightly more challenging problem to solve since now the ODE has a
variable coefficient, r(t). But this creates no difficulty for Maple, which finds that
8(t) = Soefo r(u)du .
> DE2:=diff(S(t),t)=r(t)*S(t);
70 3. Maple and Mathematical Foundations of Operations Research

DE2 := -/; S(t) = r(t) S(t)

> 8012: =dsolve ({ DE2, IC} ,8 (t) ) ;

Sol2 := S(t) = So e(- J~ -r(u)du)

> odetest(8012,DE2);
o
Numerical Solutions
In many problems, it may be impossible to solve the ODE analytically. When this
is the case, Maple can still be useful by generating a numerical solution of the
differential equation.

Example 32 Van der Pol's nonlinear differential equation. One interesting ex-
ample is the well-known van der Pol equation that represents the behavior of an
electronic circuit used in radars, or the behavior of the heart; see Bellman and
Cooke [25, p. 128]. This nonlinear ODE is given by

y" - (1 -l)y' +y = 0
where the initial conditions y(O) and y'(0) must be specified. We attempt to solve
this ODE (with y(O) = 1 and y'(O) = 0) hoping for an analytic solution, but
Maple fails to produce any results. But using the type=numeric option of
dsol ve ( ) we are able to find the numerical solution for the problem.
> restart: # VanDerPol.mws
> DE:=diff(y(t),t$2)-(1-y(t)A2)*diff(y(t),t)
+y(t)=O;

DE := (~y(t» - (1 - y(t)2) (-/; y(t» + y(t) = 0

> IC:=y(O)=l,D(y) (0)=0;
Ie:= y(O) = 1, D(y){O) = 0
> dsolve({DE,IC},y(t));
The numeric solver using t ype=n ume ric returns a Maple list that consists of
time t, the value of the variable y(t) at time t and the value of the first derivative
y' (I) at time I.
> F:=dsolve({ DE,IC} ,y(t),type=numeric);
F := proc(rkf45_x) ... end
> F(O);
[I = 0, Y(/) = 1., -/; Y(/) = 0]
> F(l);

[I = 1, y(t) = .4976154397782068,
-/; y(t) = -1.044238324424431]
 3.5 Differential Equations 71

To plot the solution we convert the numerical solution into a function. Plotting
the solution for t e [0, 15] in Figure 3.1 indicates the periodic nature of the
solution.
> Y:=t->rhs(op(2,F(t)));
Y := t ~ rhs(op(2, F(t»)
> pIot(Y,0 •. 15);

FIGURE 3.1. Solution of van der Pol's nonlinear ODE for 0 ~ t ~ 15.

If it is known that the ODE does not have an analytic solution (or that Maple
cannot find it), then one can use an easier method to plot the numerical solution of
the ODE. This method requires loading the differential equations tools package
(DEtools) and using the DEpIot () function. Removing the "i" sign before
DEpIot () and executing the worksheet produces the same graph as obtained in
Figure 3.1.
> restart: i vanderPolDETools.rnws
> with(DEtools):
> iDEpIot(diff(y(t),t$2)-(1-y(t)A2)*diff(y(t),t)
+y(t)=0,y(t),t=0 .. 15,[[ y(O)=l,D(y) (0)=0]],
y=-3 .• 3,stepsize=.05);

Systems of Differential Equations

There are many problems in operations research where systems of ordinary dif-
ferential equations arise naturally. For example, transient solutions of queueing
problems are found by solving systems of ODEs. Application of the maximum
principle to solve optimal control problems requires the solution of a two-point
boundary value problem with several differential equations.
In some cases, Maple is successful in solving a system of ODEs analytically.
When this is not possible, we can resort to a rich collection of numerical methods
that Maple knows about.
72 3. Maple and Mathematical Foundations of Operations Research

Example 33 Lancaster s equations. As an example of a case that can be solved

analytically, consider the system of ODEs
dx(t)
= -kpy(t), x(O) = xo
dt
dy(t)
= -kax(t), y(O) = yo.
dt
This system of ODEs is known as Lancaster's differential equations. F. W. lan-
caster analyzed these ODEs during World War I in the context of ''mathematics of
warfare." For a brief description of these equations and comments on Lancaster's
"square law," see Saaty [161, p. 71].
This system can be solved very easily with Maple's dsolve() function.
> restart: *
Lancaster.mws
> DEx:=diff(x(t),t)=-k*beta*y(t); ICx:=x(O)=x{ 0];

DEx := f, x(t) = -k Pyet)

lCx := x(O) = xo
> DEy:=diff(y(t),t)=-k*alpha*x(t); ICy:=y(O)=~ 0];

DEy := f, yet) = -k a X(/)

ICy := yeO) = ~
> System:=DEx,DEy; ICSystem:=ICx,ICy;

System := f, X(/) = -k PY(/), f, yet) = -k a X(/)

lCSystem := x(O) = xo, yeO) = ~

> Sol: =dsol ve ({ System, ICSystem) , ( x (t) , y (t)} ) ;

sinb(%I)xoa
Sol:= { Y(/) = cosh(%I)~ - ...;ap ,

...;ap(Sinh(%I)~a- coSh~XOa)]
X(/) =

%1 :=k...;apt
We use odetest () and see that the solution found does satisfY the original
system of ODEs.
> odetest (Sol,{ System) ) ;
{OJ
Another important class of systems of ODEs that can be solved using dsol ve ( )
is the two-point boundary value problem (TPBVP) arising in optimal control ap-
plications; see Kamien and Schwartz [99] and Sethi and Thompson [169].
 3.5 Differential Equations 73

Example 34 An optimal control problem. Consider the following nontrivial ex-

ample adapted from Sethi and Thompson [169, Section 5.1] corresponding to the
optimal control of a production/inventory system. The objective is to find the op-
timal production rate u(t) in order to minimize the total cost of deviations from
the constant target levels of inventory (x) and production rate (ii), i.e.,

foT {~h[X(t) - x]2 + ~C[u(t) - ii]2} dt

subject to the state equationx(t) = u(t)-S(t), with the initial condition x (0) as a
given constant.4 The parameters h and c represent the costs of deviations from the
target levels and S(t) is the estimated sales (demand) rate. Applying the maximum
principle we find the (unconstrained) optimal production rate as u(t) = ii + A,(t)
and

x(t) = x + ~A,(t)
c
- S(t), x(O) = xo
l(t) = h[x(t) - x], A,(T) = O.

where A,(t) is the adjoint variable. This TPBVP where x(t) is specified at t = 0
and A,(t) is specified at t = T is solved by Maple as follows.
> restart: # TPBVP.rnws
We first assign values to the parameters and specify the exact form of the de-
mand rate S(t). (The parameters uH and xH correspond to uand x, respectively.)
> uH:=50; xH:=25; T:=8; h:=5; c:=l; xO:=20;
uH:= 50
xH:=25
T:=8
h :=5
c:= 1
xO:= 20
> S:=t->t* (t-4)* (t-8)+40; #plot(S(t),t=O .• T);
S := t ~ t (t - 4) (t - 8) + 40
> inventory:=diff(x(t),t)=uH+larnbda(t)/c-S(t);
inventory := f, x(t) = 10 + A,(t) - t (t - 4) (t - 8)
> adjoint:=diff(larnbda(t),t)=h* (x(t)-xH);
adjoint := %t A,(t) = 5 x(t) - 125
> sys:=inventory,adjoint;

4The dot (.) above x(t) denotes the time derivative of x(t).
74 3. Maple and Mathematical Foundations of Operations Research

sys := f, X(/) = 10 + ).(/) - 1 (I - 4) (I - 8), f, ).(/) = 5 X(/) - 125

> funes : ={ x ( t) , lambda ( t )} ;

Junes := {x(t), ).(t)}
The dsol ve () ftmction succeeds in solving the TPBVP exactly.

> dsolve({ sys,x(O)=xO,lambda(T)=O} ,funes);

{X(/) = _ 291 (../SI) _ 291 (-../SI) __1_ %2,J5 e(-"/sI)

50 e 50 e 250 %1 +1
_1_ %2,J5e(../SI) ~ 2 791 _ 24 ).() _ 291 '5e(-"/sI)
+ 250 %1 +1 + 51 + 25 5 I, 1 - 50 v~
291 rr 1 %2e(../SI) 1 %2e(-"/sI) 74
--,J5e(vSI)+- +- -12/2_-
50 50 %1 +1 50 %1 +1 5
166
+ - 1 + /3}
5
%1 := (e(S../S»2
0/02 := -291 ,J5 + 291 ,J5 %1 + 260 e(s../S)
> assign(%);
> sirnplify(subs(t=O,x(t»);
sirnplify(subs(t=T,lambda(t»);
20
o
Once we find the optimal state trajectory, the optimal control trajectory can also
u
be detennined using U(/) = + ).(/). The plots of the trajectories for S(/), X(/)
and U(/) can then be easily plotted with the plot () ftmction. We leave this to
the reader as an exercise.

> #u : =unapply(uH+lambda(t)/e,t);

> #plot({x(t)} ,t=O •• T,title='Optirnal x(t) ');

> #plot({u(t)} ,t=O •. T,title='Optirnal u(t) ');

Example 3S Numerical solulion of sySlem of ODEs. In some cases, Maple may

not be able to find the exact solution to a system of ODEs. When this happens, it
may be necessary to compute the solution numerically. For example, consider the
problem of solving the ODE system

P' (I) = P(/)Q(/), P(O) = I

 3.5 Differential Equations 75

where P(t) = [Po(t), ... , P4(t)], P'(t) = [PQ<t), ... , p"(t)] and

-l(t) l(t)
/J -[l(t) + /J] l(t)
Q(t) =( /J -[l(t) + /J] l(t)
/J -[l(t) + /J] l(t) ).
/J -/J

which arises in queueing analysis of computer systems; see Kao [10 I, Section
2.3]. Assuming, for example, that l(t) = lOOt (1-t) is the time-dependent arrival
rate and that /J = 7, the problem is to solve this system offive ODEs in order to
compute the transient (time-dependent) probabilities of being in different states.

> restart: * ComputerSystem.mws

> lambda:=t->100*t* (1-t); mu:=7;
iplot(lambda(t),t=0 .. 1);
l := t ~ lOOt (1- t)
/J := 7
> DE[ 0] :=diff(pO(t),t)=-pO(t)*' lambda (t)'
+p1 (t)*' mu' ;

DEo := f, pO(t) = -pO(t) l(t) + pI (t) /J

> DE[ 1] :=dHf(p1(t),t)=pO(t)*' lambda (t)'
-p1 (t)*' (lambda (t) +mu)' +p2 (t)*'
mu' ;

DEI := f, pl(t) = pO(t) l(t) - pl(t) (l(t) + /J) + p2(t) /J

> DE[ 2] :=diff(p2(t),t)=p1(t)*' lambda (t)'
-p2(t)*' (lambda (t)+mu)' +p3(t)*'
mu' ;

DE2 := f, p2(t) = pI (t) l(t) - p2(t) (l(t) + /J) + p3(t) /J

> DE[ 3] :=diff (p3 (t), t)=p2 (t)*' lambda (t)'
-p3 (t)*' (lambda (t) +mu)' +p4 (t)*'
mu' ;

DE3 := f, p3(t) = p2(t) l(t) - p3(t) (l(t) + /J) + p4(t) /J

> DE[ 4] :=dHf(p4(t),t)=p3(t)*' lambda (t)'
-p4 (t) *, mu' ;

DE4 := f, p4(t) = p3(t) l(t) - p4(t) /J

> DESystem: ={ seq (DE[ i] , i=O .• 4)} ;

76 3. Maple and Mathematical Foundations of Operations Research

DESyslem := {I,- p2(/) =

100 pi (I) 1 (1 - I) - p2(/) (100 1 (1 - I) + 7) + 7p3(/)
, I,- pO(/) = -100pO(/) 1 (1 - I) + 7pl(/), I,- pl(/) =
lOOpO(/) 1 (1- I) - pi (/)(100 1 (I - I) + 7) + 7p2(/)
, I,- p4(/) = 100 p3(/) 1 (1 - I) - 7 p4(/), I,- p3(/) =
100p2(/) 1 (I - I) - p3(/) (100 1 (1 - t) + 7) + 7 p4(/)
}
> ICSystern:={ pO(O)=1,seq(p.i(O)=O,i=1..4)};

ICSystem := {
pO(O) = I, pl(O) = 0, p2(0) = 0, p3(0) = 0, p4(0) = 0
}
After trying very hard, Maple fails to find the exact solution for this problem.
> #dsolve(DESystern union ICSystern,
{seq(p.i(t) ,i=O .. 4)}); #This fails
But using the type=nurneric option of dsol ve (), we can compute the s0-
lution numerically and plot the transient probability trajectory for the loss proba-
bility P4(t) as given Figure 3.2.
> F:=dsolve(DESystern union
ICSystern, [ seq (p. i (t) , i=O .. 4)] , type=nurneric) ;
F := proc(rkf45y) ... end
> F(O);
[I = 0, pO(/) = 1., pi (I) = 0, p2(/) = 0, p3(/) = 0, p4(t) = 0]
> F ( .25) ;

[I = .25, pO(t) = .1977703300783685,

= .2605045316892459,
P I (I)
p2(t) = .2344519154105270,
p3(/) = .1667040464906527,
p4(/) = .1405691763312056]
> F(.5);

[t = .5, pO(/) = .01776293908215261,

p I (I) = .04255366365736456,
p2(/) = .09076540588829907,
=
.2133530755905687,
p3(t)
p4(/) = .6355649157816149]
> P4:=t->rhs(op(6,F(t)));
P4 := t ~ rhs(op(6, F(/)))
 3.5 Differential Equations 77

> plot (P4, 0 .. 1) ;

0.7

0.£

05

OA

0.3

0.2

0.1

0 0.2 OA 0.£ 0.8

FIGURE 3.2. Numerical solution of the loss probability, P4(/), 0 ~ 1 ~ I, with

1(1) = 100/(1 - I).

It is interesting to note that the loss probability trajectory P4(/) has a shape that
resembles the time-dependent arrival rate function ..1.(1) = 100/(1 - I), which is
concave and reaching its maximum at 1 = 0.5. This resemblance is not a coin-
cidence since the increasing arrival rates result in a higher probability of a busy
system.

3.5.2 Partial Differential Equations

Partial differential equations (PDE) do not appear very frequently in operations
research, but they have been used in the modeling and solution of some prob-
lems including the transient solution of ample server queue (Gross and Harris
[76, p. 139]), manpower planning (Gerchak, Parlar and Sengupta [71]) and the
optimal control of distributed parameter systems (Derzlm, Sethi and Thompson
[61]). In this section we demonstrate the solution of some PDEs using Maple's
pdsol ve () function.
Consider the PDE
au + au + u(x, y) = ~+2Y
ax ay
with the initial condition u(x, 0) = O. To solve this PDE for the unknown function
u(x, y) we proceed as follows.
> restart: *
PDE.rows
>
= exp(x+2*y); u(x,O)=*
PDE:=diff(u(x,y),x) + diff(u(x,y),y) + u(x,y)
is the Ie °
PDE:= (! u(x, y» + (%y u(x, y» + u(x, y) = e(x+2y)
78 3. Maple and Mathematical Foundations of Operations Research

> pdsolve(PDE);
I e(·J+2y) eX + 4 FI(y - x)
u(x, y) = 4 ----ex--=:.-~~
Maple finds the general solution that involves an unknown function _FI(·) that
must be detennined using the initial condition.
> Sol:=combine(expand(%»;

So/:= u(x, y) = 4I e(.r+2y) + e(-X) _FI(y - x)

> solve(subs(y=O,u(x,O)=O,Sol),_Fl(-x»;
leX
-4 e(-X)
> _Fl:=unapply(simplify(subs(x=-x,%»,x);

-
Fl := x -+
4
-! e(-2x)
Thus,_FI(x) = -e- /4 and so _Fl(y - x) = _e-2y+2x /4.
2x

> _Fl (y-x);

_! e(-2y+2x)
4
> combine(Sol);
I 1
u(x, y) = 4e(X+2y) - 4e(X-2Y)

It then follows that the solution is

u(x,y) = ~eX+2Y - ~eX-2Y.

> assign(%); u(x,y);
! e(x+2y) _ ! e(x-2y)
4 4
To check the result we substitute the solution in the original POE and find that
it is satisfied.
> is(diff(u(x,y),x) + diff(u(x,y),y) + u(x,y)
= exp (x+2* y) ) ;
true

Example 36 Ample server queue. The following POE arises in the transient anal-
ysis of the Markovian ample server queueing system M / M /00 with exponen-
tial interarrival times (rate l) and exponential service times (rate J.l). Defining
G(z, t) = L~ Pn(t)zn as the generating function of the transient probability
Pn(t) of having n customers in the system at time I, it can be shown that G(z, I)
satisfies
aG aG
at = J.l(l - z) az - l(l - z)G(z, I)
 3.5 Differential Equations 79

with the initial condition G(z, 0) = I assuming that at time I = 0 the system is
empty.
Maple solves this POE as follows.
> restart: *
AmpleServer.mws
> PDE:=diff(G(z,t),t)=mu* (l-z)*diff(G(z,t),z)
-lambda* (l-z)*G(z,t);
PDE := f, G(z, I) = Il (I - z) (;= G(z, I» -). (I - z) G(z, I)
> Sol:=pdsolve(PDE,G(z,t»;
-11l+in(-I+z) (ll)
Sol:= G(z, I) = ]I( )e P
Il
> pdetest(Sol,PDE);
o

Using the initial condition, it can be shown that the exact solution of this POE
is
- -.t(=-I)(e-'P-I)/p .
G(z, I) -e

We will provide a detailed solution of this POE in Section 9.3.2 of Chapter 9,

Queueing Systems.

3.5.3 Difference (Recurrence) Equations

Maple can solve a large class of difference (i.e., recurrence) equations with the
rsol ve () command. This command can solve linear recurrences with constant
coefficients, systems of linear recurrences with constant coefficients, many first-
order linear recurrences, and some nonlinear first-order recurrences.

Example 37 Fibonacci sequence. The Fibonacci sequence is obtained as the s0-

lution to the difference equation In = In-I + In-2, with the initial conditions
10 = /J = I. Maple solves this difference equation as follows.
> restart: * Fibonacci.mws
> rsolve({ f(n)=f(n - l)+f(n-2), f(O)=l, f(l)=U, f);

2 ../5 (2 1 ~)n ../5 (-2 I )n

_ -1+ v 5 +~ 1+../5
5 -1+../5 5 1+../5
Ifwe use the' makeproc' option, then rsol ve () returns a procedure for
evaluating the function defined by the difference equations.
> f:=rsolve({ f(n)=f(n-l)+f(n-2), f(O)=l, f(l)=U,
f,' makeproc' ) ;
80 3. Maple and Mathematical Foundations of Operations Research

f:= proc(n)
locali, s, I, bipow;
bipow:= proc(n) ... end;
ifl < nargs or not type(n, integer) tben 'procname'(args)
else
s := bipow(n - I);
1:=0;
fori to2dol := I +SI,; x (array(l .. 2,[(2)=1,(1)=1])); od;
I
fi
end
> f(1); f(2); f(5); f(10);
I
2
8
89
Example 38 Coin loss. In this more complicated example, consider a biased coin
for which p = Pr(Heads) and q = Pr(Tails) . The coin is tossed repeatedly and
stopped when two heads (HH) occur in succession for the first time. Ifwe let X
denote the number of such trials needed, and an = Pr(X = n), then it follows that
for n = 1,2 and 3, we have al = 0, 02 = p2, a3 = p2q as the initial conditions.
Using probabilistic arguments as in Kao [10 I, Chapter I], one can show that

an = qan-I + pqan-2. n = 4, 5, ...

We now solve these difference equations with rsol ve ().

> restart: * BiasedCoin.mws
Maple can find the exact solution of these difference equations but the result
looks unattractive. Instead, we choose to solve the problem using the ' make-
proc' option and obtain a procedure for the solution an.
> a:=rsolve ({ a (n) =q* a (n-1) +p* q* a (n-2), a (1) =0,
a (2) =p"2, a (3) =p"2*q} , a,' makeproc' ) :
Following are the first 10 terms of the solution.
> seq ([ n, simplify (a (n) )] , n=1. .10) ;

[1,0], [2, r), [3, p2q), [4, q p2 (q + p»), [5, q2 (q +2p»), r

r r
[6, q2 (q2 + 3 pq + r»), [7, q3 (q + p) (q + 3 p»),
r r
[8, q3 (q3 + 5 q2 P + 6 q + p3»),
[9, q4 p2 (q + 2 p)(q2 + 4 pq + 2 r»),
r
[10, q4 p2 (q4 + 7 q3 P + 15 q2 + 10 p3 q + p4»)
 3.6 Transfonn Methods 81

Next, we solve the same problem with the' genfunc' (z) option that pro-
duces the generating fimction Px(z) of the distribution. Since the mean E(X) can
be found by differentiating the generating fimction and substituting z = 1, i.e.,
E(X) = Px(l), we find that the mean of X is E(X) = (1 + p)/ p2.
> restart:
> PXz:=rsolve({a(n)=q*a(n-l)+p*q*a(n-2),a(1)=O,
a(2)=p"2,a(3)=p"2*q} ,a,' genfunc' (z));

PXz:= _
-1+qz+ pqz2
rzl
> simplify (subs (q=l-p,normal (subs(z=l,
diff(PXz,z)))));
I+p
IT

3.6 Transfonn Methods

Many problems in probability and stochastic processes can be successfully solved
using transform techniques such as Laplace transforms (L1) and generating fimc-
tions. When the direct solution of a problem-such as finding the distribution of
a random variable-is not available, it may sometimes be possible to express the
problem in terms of the Laplace transform (or the generating fimctions if the ran-
dom variable is discrete). If the transform can be inverted using invlaplace ( ) ,
then the solution to the original problem is obtained.

3.6.1 Laplace Transforms

The Laplace transform l(s) of a real-valued fimction f(t) is defined as

l(s) = 10 00
e-st f(t)dt

provided that the integral exists. When f(t) is the p.d.f. of a nonnegative random
variable, l(s) = E(e-Sx ), i.e., the LT of the density is the expected value of
e-sx . Thus, it follows that f(n)(s) = (_l)n E(xne-Sx ), so the nth moment of X
is E(xn) = (_l)n f(n) (0).

Example 39 Erlang density with four stages. Consider the Erlang density with
k = 4 stages, i.e.,
le-.b (lti- I
f(t) = (k _ I)! ' t > O.
> restart: # ErlangLT.mws
> with(inttrans);
82 3. Maple and Mathematical Foundations of Operations Research

[addtable, fourier, fouriercos, fouriersin, hankel, hilbert, invfourier,

invhilbert, invlaplace, invmellin, laplace, mellin, savetable]
> k:=4;
k :=4
> f: =lambda* exp (-lambda* t) * (lambda* t) " (' k' -1) I (' k' -
1) ! ;

lee-AI) (It)(k-I)
f:= (k _ I)!
The LT of this density is found using laplace () as

l(s) = (_l )4
l+s
> fLT:=laplace(f,t,s);
l4
jLT:= (s + l)4
Inverting the transfonn gives the original density, as expected.
> invlaplace(fLT,s,t);

~ l4 e(-A I) t 3
6
Finally, the expected value of the Erlang is obtained as E(X) = 4/ l using the
fact that E(X) = (-I)f'(O).
> EX:=-subs(s=O,diff(fLT,s));

EX:=4±

Example 40 Partial fraction expansion. As another example, suppose that we

wish to find the function f(t) whose LT is

If we were solving this problem manually, we would first find the partial fraction
expansion of the LT and then invert each tenn by referring to tables of Laplace
transfonns. We see below that by simply using the invlaplace () function on
l(s) we immediately obtain the function f(t).
> restart: # LT.mws
> with(inttrans):
> fLT:=s"3/((s+1)"2* (s-1)"3);
s3
.- ~--:-:;--:----:-:-:;"
JLT .- (s + 1)2 (s _ 1)3
 3.6 Transfonn Methods 83

> PF:=convert(fLT,parfrac,s);
11 3111 I I 31
PF:=- ----+- +- +---
8 (s + 1)2 16 s + I 4 (s - 1)3 2 (s - 1)2 16 s - I
> invlaplace(PF,s,t);

!Ie(-t) - ~e(-t) + !/2 e' + !Ie' + ~e'

8 16 8 2 16
> invlaplace(fLT,s,t);
I 3
- 1 e(-t) - - e(-t) + -1 12 e' + -1 1 e' + -3 e'
8 ]6 8 2 ]6
Numerical Inversion of Laplace Transfonns
Quite often, it may be impossible (or very difficult) to invert a Laplace transfonn
algebraically. In such a case, numerical inversion of the Laplace transfonn may
be a feasible alternative.
If j(s) is the LTof /(/), then the numerical inversion of j(s) can be perfonned
by using

/Numeri~(/) = ~;2 R(j (~)) + ~/2 %;(-I)k R(j (A +2~bj))

(3.2)
where A is a parameter related to the approximation error, j is the complex number
and R(z) is the real part of the complex number z. This numerical method was
suggested by Abate and Whitt [2J who advise using A = v 10(10) for an accuracy
of 10-11 ; see also Kao [101, p. 25J. When 1/(/)1 ~ ] for all 1 ~ 0, it can be shown
that the error is bounded by
e- A
1/(/) - /Numerical(/)1 ~ I
-A'
-e
so when e- A is small, the error is approximately e- A •
10 the next example, we attempt to invert the transfonn
- 1
/(s) = (1 +.JS)(1 + s)'
> restart: *
NurnericalLT.rnws
> with(inttrans):
> fLT:=s->l/«l+sqrt(s))* (l+s));
1
jLT:= s -') (J +.JS) (J +s)
> f:=unapply(invlaplace(fLT(s),s,t),t);

,Ji (,Ji ,JJJi e-VI erfc(£[Ji) - ,Ji) e( -t+_VI)

/ := 1 -') loo
t
-
1
-
2
-
1m
",_UJ
-..fii
d U1
-
84 3. Maple and Mathematical Foundations of Operations Research

Maple makes a valiant attempt to invert this LT but finds only an integral in-
volving the complementary error function erfc () and the exponential function.
Plotting the resulting function (which we do not display) reveals that f(l) ~ 0
as 1 -+ 00.

> v:=5.; A:=v*ln(10.}; exp(-A};

v:=5.
A := 11.51292547
.9999999950 10-5
> iplot(f(t},t=0 .• 10};
Next, we compute the triple [I, f(I), J~ f(u) dU] for increasing values of 1
and observe that f(l) appears to be a density of a (yet to be discovered?) random
variable since lim Hoo J~ f(u) du =
1.
> for t from 5 to 40 by 5 do [t,evalf(f(t}},
evalf(Int(f(u},u=O .. t}}] od;
[5, .03317546275, .7344982429]
[10, .009359468114, .8200628136]
[15, .004943879701, .8538194324]
[20, .003184776647, .8736012833]
[25, .002270662056, .8870247002]
[30, .001724058011, .8969068485]
[35, .001366589281, .9045757418]
[40, .001117717361, .9107517465]
Finally, using the numerical inversion formula in (3.2) we compare the values
for f(l) and fNwnerical(l) and find that the errors resulting from the numerical
approximation are quite negligible.
> fNumer:=t->exp(A/2}/(2*t}*Re(fLT(A/(2*t}}}
+exp(A/2}/t
* sum ( (-1) "k*Re (fLT ((A+2* k* Pi* I) / (2* t)}} I
k=1. .1000} ;

1
> for t from 0.01 to 1 by .1 do [t,evalf(f(t}},
fNumer(t}] od;
 3.6 Transfonn Methods 85

[.01, .1028407591, .1027793021]

[.11, .2652275434, .265022770]
[.21, .3116393936, .311356022]
[.31, .3294644327, .329119930]
[.41, .3336666867, .333270567]
[.51, .3302478027, .329806145]
[.61, .3222873512, .321804697)
[.71, .3115759949, .311055639]
[.81, .2992342356, .298678991]
[.91, .2859958883, .285407838]

3.6.2 Generating Functions

If X is a discrete-valued random variable with probability density an = Pr(X =
n), n = 0, I, ..., then the generating function of X is defined as o(z) = E(zx) =
L:.oanz". Note that 0(1) = I and the series converges for at least Izi ~ I .
Additionally, E(X) =0'(1), and E(X2) =
0"(1) + 0'(1) so that Var(X) =
a' (I) + a(I) - [0'(1)]2. For an excellent review of generating functions see
Kleinrock [110, Appendix 1).5
Manipulation of (rational) generating functions in Maple is achieved by first
loading the genfunc () package. Two important functions are rgf _encode ( )
and rgf expand ( ) .The first function finds the generating function o(z) given
the sequcmce an, and the second inverts a given generating function o(z) to obtain
the original sequence an.

Example 41 The use of rgf encode () and rgf expand (). We start with
the density of the geometric r~. an = Pr(X = n) = .0(1 - p)n, n = 0, I, ...

> restart : * Genfunc.rnws

> with(genfunc);

[rgf_charseq, rgf_encode, rgf_expand, rgfJintirecur, rgf_hybrid,

rgf_norm, rgf....P/rac, rgfJelate, rgf_sequence, rgf_simp, rgf_term,
termscale]

SSome authors (including Kleinrock [110, p.327» use the tenn ":-transfonn" for generating ftmc-
tions. Maple also ha a procedure called ztrans (), but it should not be confused with generating
functions. Maple's ztrans () is defined a ~oQn:-n , which is not used very frequendy in oper-
ations research applications.
86 3. Maple and Mathematical Foundations of Operations Research

Maple finds the generating function o(z) easily using rgf _encode ( ). To
compute the mean and variance we find 0'(1) and 0"(1) and see that E(X) =
(1- p)jp and Var(X) = (1- p)jp2.
> aGF:=rgf_encode(p* (I-p) An, n, z); *Geometric
aGF:= p
1- (1- p)z
> aGFz:=eval(diff(aGF,z),z=l);

aGFz:= -I+p
p
> aGFzz:=eval(diff(aGF,z$2),z=1);

aG'':;' ._ 2 (-I + p)2

rZZ.- p2
> Var:=normal(aGFzz+aGFz-(aGFz)A2);
-I +p
Var:=
p2
In the next example, suppose that the generating function of a r. v. is given bY;

A( 8
a z) = (2 _ z)(3 _ z)3'

We now use rgf _expand () to invert the generating function.

> aGF:=8/ «2-z)* (3-z) A3);
I
aGF := 8 -----=-
(2 - z)(3 - z)3
> a:=rgf_expand(aGF,z,n);

a := 4 (.!.)n - .!. .!. .!.

(n + I) ( n + I) ( .!.)n + (-8 n _ 8) ( .!.)n _ )n ! (.!.
2 27 2 3 9 3 3 3
We first evaluate the result for some small values of n and see that the numbers
are all between zero and one. Summing the result from zero to infinity gives one,
implying that we have found the correct density.
> evalf(seq(eval(a,n=k),k=O •. lO));

.1481481481, .2222222222, .2098765432, .1598079561, .1073388203,

.06647233653, .03892635777, .02190183153, .01196702103,
.006397479330, .003364327190
> sum(a,n=O .. infinity);

6It is easy to see that since g(l) = I, this must be the generating function of a proper discrete
random variable.
 3.6 Transfonn Methods 87

Finally, we also compute the mean in two different ways and find that it equals
512.
> eval(diff(aGF,z),z=I);
5
2
> sum(a*n,n=O .. infinity);
5
-2

Numerical Inversion of Generating Functions

As in Laplace transfonns, it may sometimes be difficult (or impossible) to invert
a generating function a(z) = L~ anz" of a probability density an. Fortunately,
there is an easy-to-use numerical inversion method due to Abate and Whitt [I]
that works for n ~ I. If we let a~umerical denote the probability an, then

01.'........ = ~" {o(r) + <-I)"o<-r) + 2%<-l y n (a (rexp (Kn)) l

where r e (0, I) and R«() is the real part of the complex number ( . To have an
accuracy of 10-11 , we let r = 10-1112n. It can be shown that the error bound is

I
an - an
Numerical I
~
r2n
I _ r2n .
Example 42 Numerical inversion ofa generating function. We invert the gener-
ating function discussed in Example 41
~( 8
a z) = (2 -z)(3 -z)3
and compare the results to what we already have obtained as the exact expressions.
> restart: * NumericalGF.mws
> v : =5;
v:=5
> aGF:=z->8/«2-z)* (3-Z)A3);
I
aGF := z ~ 8 (2 _ z) (3 _ z)3
> aNumer:=n->(I/(2*n*r An))* (aGF(r)+(-I)A n*aGF(-r)
+2* Sum ( (-1) Aj* Re (aGF(r*exp (Pi* j* I/n))), j=1. .n-l));

aNumer :=

1 aGF(r) + <-I)" aGF<-r) + 2 ~ <-IY ~<aGF<r e<~»»)

n~------------=----------
2 nrn
88 3. Maple and Mathematical Foundations of Operations Research

> r:=lOA(-v/(2*n»;
r := 1O(-5/2~)
We see that for this problem the numerical inversion gives results that are very
close to what we had found in Example 41.
> seq([ n,evalf(value(aNurner(n»)] ,n=1. .25);

[I, .2222238606], [2, .2098769451], [3, .1598080311], [4, .1073388311],

[5, .06647233500], [6, .03892634146], [7, .02190184246],
[8, .01l96702514], [9, .006397466901], [10, .003364318742],
[II, .001747392703], [12, .0008990553030],
[13, .0004592782611], [14, .0002333670562],
[IS, .0001180794478], [16, .00005955754688),
[17, .00002997095156], [18, .00001506825305],
[19, .756699756710-5 ], [20, .377575952510-5],
[21, .188531982410-5 ), [22, .968806883310-6 ],
[23, .4798412623 10-6], [24, .248370557910-6 ],
[25, .1l643506341O-6 ]
> n: =' n' :
We also compute the remaining probability ~umcrical = I - :L:=I a~umeric:al
where n is such that a~umerical ~ 0 for n ~ n. This gives a~umerical = 0.148146.
> surn{evalf(value(aNurner(n»),n=1 .. 25);
.8518538100
> evalf(l-%); # a(O)
.1481461900

3.7 Probability
Maple recognizes a large number of continuous and discrete random variables
whose density function, cumulative distribution function and inverse cumulative
distribution function can be evaluated numerically. These functions are loaded
by entering the command wi th (stats) ;. By using Maple's symbolic manip-
ulation capabilities, one can also perform certain types of probabilistic analysis
such as computing expectations and finding the joint probability distribution of
functions of random variables.

3.7.1 Continuous and Discrete Random Variables

Maple knows about the following 13 continuous random variables: beta, Cauchy,
X2, exponential, F, gamma, Laplace, logistic, lognormal, normal, t, uniform and
 3.7 Probability 89

Weibull. To evaluate (or plot) the density function of a continuous random vari-
able, we use s ta teval f[ pdf, name] (). To evaluate the cumulative distribu-
tion function, the first argument is replaced by edf, and to evaluate the inverse
cumulative distribution function, the first argument is replaced by iedf.
Maple also knows about the binomial, discrete uniform, hypergeometric, nega-
tive binomial and Poisson discrete random variables. To evaluate the density func-
tion of ad iscrete random variable, we use statevalf[ pf, name] (). The first
argument is replaced with dedf and idedf to evaluate the cumulative distribu-
tion and inverse cumulative distribution functions, respectively.
Detailed information about the evaluation of these functions can be obtained
with? stats.
Example 43 Normal random variable. In this example we evaluate and plot the
density, distribution function and inverse distribution function of the unit normal
random variable with mean 0 and variance 1. The density
1
5
I 2
f(x} = e -'1. X • -oo<x<oo

is defined as f : =x- > s ta teval f[ pdf, normald[ 0, 1]] (x).

> restart: # ContinuousDistributions.mws
> with(stats);
lanava, describe, fit, importdata, random, statevalJ, stalplots, transform]
The distribution function F (x) = f~oo f (u) du and inverse distribution func-
tion F- 1(x) are defined similarly.
Evaluating these functions, we see that f(O) = .398, F(3) = .998 and F- 1(.3)
= -.524. These three functions are plotted on the same graph with the dis-
play () command and presented in Figure 3.3
> f:=x->statevalf[ pdf,normald[ 0,1]] (x):
> F:=x->statevalf[ edf,normald[ 0,1]] (x) :
> iF:=x->statevalf[ iedf,normald[ 0,1]] (x) :
> f(O); F(3); iF(.3);
.3989422803
.9986501020
-.5244005127
> fPlot:=plot(f(x),x=-3 .. 3,linestyle=1,
eolor=blaek,thiekness=2) :
> FPlot:=plot(F(x),x=-3 .. 3,linestyle=2,
eolor=blaek,thiekness=2) :
> iFPlot:=plot(iF(x),x=-1 .. 1,linestyle=3,
eolor=blaek,thiekness=2) :
> with(plots):
> display ([ fPlot, FPlot, iFPlot] ) ;
90 3. Maple and Mathematical Foundations of Operations Research

1.51
1
I
j .. _-_..-..__.._-_.
.•........ ...../ -
0.. . .!
~~ ; ..

-3 -2 -1 • i 1 x 2 3
i
-0.5 !
I
:
!
-1 !
:

-1.5/
FIGURE 3.3. The density f(x), cumulative distribution function F(x) and inverse cumu-
lative distribution function F-I (x) of the unit nonnal random variable drawn as a solid
line, dotted line and dashed line, respectively.

Example 44 Poisson random variable. Next, we consider the Poisson random

variable with parameter l = 3 and density
lne-A
p(n)=-,-, n=O,I, ....
n.
This is defined in Maple as p: =s ta teval f[ pf, poisson{ 3)) (n);.
> restart:. DisereteDistributions.rnws
> with(stats);
[anova, describe, fit, importdata, random, statevalf, stOlplots, transform]
The distribution function P(n) = Lk=O p(k) and the inverse distribution func-
tion p- I (n) are defined in a similar manner. We find, for example, that p(S) = .1,
P(S) = .91 and P-I(.8) = 3; i.e., there is a 0.10 chance that the Poisson random
variable with rate l = 3 will take the value 5 and a 0.91 chance that it will take a
value less than or equal to S. The first two functions are plotted in Figure 3.4. The
inverse distribution is not included in this graph but it too can be plotted using the
command plot (iP (n), n=O .. 1,0 .. 8) ;.
> p:=n->statevalf[ pf,poisson{ 3)) (n);
p := statevalJpf,poWon)
> P:=n->statevalf[ dedf, poisson{ 3)) (n);
P := statevalfdcd/,poisMJn)
> iP:=n->statevalf[ idedf,poisson{ 3)) (n);
iP := statevalf1dcd/,poWon)
> p(5); P(5); iP(.8);
 3.7 Probability 91

0.8

0,,(,

OA

0.2

0~~~~~2~~~3~~4~~T5~~'~~7~~8
n

FIGURE 3.4. The density p(n) and the cumulative distribution function P(n) of Poisson
with parameter l = 3 drawn as a solid line and a dotted line, respectively.

. 1008188134
.9160820583
3.
> pPlot:=plot(p(floor(n)),n=O •. 8,linestyle=1,
color=black,thickness=2) :
> PPlot:=plot(P(floor(n)),n=O .. 8,linestyle=2,
color=black,thickness=2):
> with(plots):
> display ([ pPlot, PPlot] ) ;
> #plot(iP(n),n=O .. 1,O •. 8):
In addition to the nearly 20 frequently used distributions, Maple also allows the
definition of arbitrary discrete empirical distributions defined over the integers
1, ... , N. This is made possible by using the empirical command.

Example 45 A discrete empirical distribution. Consider a discrete random vari-

able X with density px(I) =
0.2, px(2) =
0.3, px(3) =
0.5. The probabilities
are entered as an expression sequence named Probs and the density is defined
simply as p: =empirical[ Probs] .
> restart: # Empirical.mws
> with(stats):
> with (random) ;

[p, binomiald, cauchy, chisquare, discreteunifonn, empirical, exponential,

[ratio, J', laplaced, logistic, lognormal, negativebinomial, normald,
poisson, studentst, uniform, weibull]
> Probs:=.2,.3,.5;
92 3. Maple and Mathematical Foundations of Operations Research

Probs := .2, .3, .5

> p:=empirieal[ Probs] ;
p := empiricaJ.2, .3, .s
The density and the cumulative distribution function of X can be evaluated
using the pf and dedf functions. It is also possible to evaluate the inverse cu-
mulative distribution function using idedf and by supplying as input a number
between 0 and I.
> statevalf[ pf, p] (1);
.2
> statevalf[dedf,p] (2);
.5
> statevalf[ idedf,p] (.215);
I.
> seq (statevalf[ dedf, p] (n), n=1. .3) ;
.2, .5, 1.0
Finally, it is also possible to generate random variates from this distribution
using the random command.
> randorr( p] (10);
2.0, 3.0, 3.0, 3.0, 2.0, 1.0, 1.0, 3.0, 1.0, 3.0

3.7.2 Expectation
Since the computation of moments of a random variable requires evaluating inte-
grals or sums, Maple is well suited for performing such operations.
Example 46 Mean and the variance ofthe normal distribution. For example, let
us consider the normally distributed random variable X with parameters Jl and (12
and compute its mean E(X) and variance Var(X).
> restart:. NormalMoments.mws
> interfaee(showassumed=2); assume(sigma>O);
> f:=(l/(sigma* (2*Pi)A(1/2)))*exp(-(x-mu)A21
(2* sigma A2));

I v'2e(-1/2~)
f := 1. (1 ,.fii
with assumptions on (1
> Int(f,x=-infinity •• infinity);
 3.7 Probability 93

1 00

-002
1 J2e(-)!2~)
-
(1-JiC
dx

with assumptions on (1
Maple easily evaluates the integral of density and finds the result as I. It also
finds that E(X) = f.l and Var(X) = (12.
> value(%);

> EX:=int(x*f,x=-infinity .. infinity);

EX:=f.l
> VarX:=int((x-mu)A2*f,x=-infinity .• infinity);

VarX:= (12
with assumptions on (1
Example 47 A discrete random variable takingjinitely many values. Suppose a
random variable X takes the value x with probability ax, x = I, ... , N. We wish
to find the mean E(X) and variance Var(X) after detennining the proper value of
a.
> restart: * Expectation.mws
> f:=a*x;
!:=ax
> a:=solve(sum(a*x,x=l •• N)=l,a);
1
a :=2 N(N + I)
Thus, a is found to be2/[N(N + I)].
> f;
x
2 N(N + I)
> EX:=normal(sum(x*f,x=l .• N));
2 1
EX:=3"N+3"
> EX2:=normal(sum(x A2*f,x=1 .. N));
I
EX2 := "2 N (N + I)
> VarX:=normal(EX2-EX A2);
1 2 1 1
VarX := 18 N + 18 N - 9
94 3. Maple and Mathematical Foundations of Operations Research

Example 48 Cauchy distribution. We wish to find a constant c(a, b) for which

the function f(x) = c(a, b)(l + x 2 )-1 for a < x < b is a density.
> restart: *
Cauchy . rnws
> f:=c/(1+x"2);
c
f:= I +x2
> p:=Int(f,x=a .. b);

p:= l a
b
-1--2 dx
+x
c

To do this, we solve J:
c(a, b)(l + x 2 )-1 dx = I for c(a, b) and find that
c(a, b) = [arctan(b)-arctan(a)r l . When x e (-I, I), we havec(-I, I) = 211C
and when x e (-00, 00), we obtain c( -00, 00) = II 1C which gives the Cauchy
density
I
f(x) = 1C(l +x2)' -00 < x < 00.
> c:=solve(value(p)=l,c);
I
c·-------------~
.- arctan(b) - arctan(a)
> lirnit(limit(c,b=l),a=-l);

2.!.
1C
> lirnit(limit(c,b=infinity),a=-infinity);

1C
The fact that the Cauchy density has no expected value can be shown as follows.
We write E(X) = i-oo x f(x) dx + Iaoo xf(x) dx . The second integral evaluates
to ~ In(l +02), which diverges to infinity as a ~ 00. Thus, E(X) does not exist.
> f:=l/(Pi* (1+x"2));
I
f:= 1C 0 +x2)
> Integral[ 2] : =Int (x* f, x=O .. a) ;
value (Integral[ 2] );

[nJegral2 := loa 1C 0 : x 2) dx

I InO +a2 )
2 1C
> lirnit(%,a=infinity);
00
 3.7 Probability 95

3.7.3 Jointly Distributed Random Variables

Let X and Y be jointly distributed continuous random variables with density
fx,Y(x,y). Suppose we define two new random variables U = u(X, Y) and
V = v(X, Y) and assume that the system of nonlinear equations u = u(x,y)
and v = v(x, y) can be solved uniquely for x = x(u, v) and y = y(u, v) in terms
of u and v. Then the joint density of the random vector (U, V) is given by

fu,v(u, v) = fx,Y(x (u, v), y(u, v» IJ(u, v)1 (3.3)

where
J(u, v) = det (iiou ~)
iJo
is the determinant of the Jacobian of the transformation; see Harris [82, p. 169]
and Stirzaker [181, p. 270].

Example 49 Joint distribution ofa random vector. We assume that the joint den-
sity of the random vector (X, Y) is fx,Y(x,y) =
12 e -Ax e -Ay, i.e., X and Yare
individually distributed as exponential random variables with parameter I .
> restart: # JointDensity.rnws
> with (linalg) :
Warning, new definition for norm

Warning, new definition for trace

> assurne(larnbda>O);
> fXY:=(x,y)->larnbda*exp(-larnbda*x)*larnbda
* exp (-larnbda* y) ;
jXY := (x, y) -+ 12 e(-Ax) e(-Ay)
Using the transformation U = X + Yand V = X/(X + Y), we obtain the
inverse transformation as x = uv and y = u - uv and the determinant of the
Jacobian as -u.
> uxy:=u=x+y; vxy:=v=x/(x+y);
uxy:= u =X +y
x
.- -
vxy·-v --
- x+y
> solve({ uxy,vxy} ,{ x,y} );
{y =u- v u, x = v u}
> assign(%);
> A:=evalrn(vector([ x,~ ));
tf :=[vu, u-vu]
> jacobian (A, [ u, t1c );
% 3. Maple and Mathematical Foundations of Operations Research

> J:=det(%);
J:=-u
> fXY(x,y)*abs(J);
,l2 e(-Ao II) e(-·HII-olI» lui
Applying (3.3) gives fu,v(u, v) = ,l2e-Alllul, u > 0, which integrates to I, as
expected.
> fUV:=sirnplify(%);
jUV := ,l2 e (-AII) lui
> int(fUV,u=O • • infinity);

3.8 Summary
Modeling and solution of many operations research problems requires a knowl-
edge of a wide spectrwn of mathematical topics such as algebra and calculus,
linear algebra, differential equations, transfonn methods and probability theory.
This chapter covered these topics and their treatment with Maple. We surveyed
these mathematical topics by providing a large number of examples. With the
detailed explanation of the useful mathematical techniques as presented in this
chapter, we are now ready to begin our study of OR techniques and models.

3.9 Exercises
I. Find the values of x that satisfy the inequality x 2 + x + 1 ~ 3. Check your
result by comparing the plots of x 2 + x + 1 and 3 on the same graph.

2. The nonlinear equation 1 - foQ f(x) dx = p (where 0 < P < 1 and

f(x) is the demand density) arises in stochastic inventory theory. Solve
this equation for the order quantity Q under the following assumptions:

(a) f(x) = I/(b - a), a < b (unifonn demand)

(b) f(x) = ,le-Ax(,lx)n-I/(n - 1)!, for n = 3, P = 0.7 and,l = 0.01
(gamma demand).

3. Find the points of intersection of the two circles x 2 +

•.2 _ I
r = 1 and (x _1)2 +
Y -4·
 3.9 Exercises 97

4. The function C(u) = L~ e-anu(k + cA.u) arises in the analysis of an

inventory problem with discounted cost objective function; see Zipkin [199,
p. 63] and Hadley and Whitin [81, p. 80]. Differentiate this function with
respect to u and find the u* that satisfies C'(u*) = 0 assuming c = 1,
A. = 10, k = 10, a = 0.1.

5. Consider the function G(u) = I: f6(u -x)dx + I~o f6(x - u)dx. Find
the derivative of this function in two different ways and locate the point that
minimizes G(u).

6. Evaluate the integral 10

xQe- bx dx assuming that a and b are strictly pos-
itive. HINT: Use the assume () statement.
7. Determine whether the linear system of equations Ax = b with

A [~ ~ ~6]' b ~
=
3 6 -9
= [
0
]

has a unique solution by comparing the ranks ofr(A) and r(A I b). If the
system has a unique solution, find it by (i) evaluating A-\ b, and (ii) using
linsol ve ( ) .

8. Find the 10th power of the transition probability matrix

p=[t tJ
after diagonalizing it. Does your result agree with ptO obtained directly?

9. The second order differential equation x"(/) = rx'(/) + arises in the!

minimization of the functional It
e-rt ([x' (/)]2 +2x(/)} dl with conditions
x(O) = xo and x(T) = XT. Solve the differential equation and plot the
trajectory assuming r = 0.10, T = I, xo = 10 and XT = 8.
10. Consider the system of differential equations:

m) (I) = A. - Jlm) (I), m) (0) = ;

m2(t) = A. + (21 + Jl)m)(t) - 2Jl m2(t), m2(0) = ;2.
This system arises in the analysis of an infinite server queueing system
where A. and Jl are positive arrival and departure rates, respectively. The
functions m) (t) and m2(t) are the expected number of busy servers and
mean-squared number of busy servers. Solve this system by dsol ve ( )
and use the rnethod=laplace option. Show that limt-+oo m) (t) = )./ Jl.
Next, define the variance of the number ofbusy servers as M) (I) = m2 (t)-
[m)(I)j2 and show that limt-+oo M) (I) = A./ Jl. (Ghosal et al. [72, p. 124].)
98 3. Maple and Mathematical Foundations of Operations Research

11. Solve the difference equation mk = 2mk-1 + 1, with initial conditions

mo = 0, m I = 1 using generating functions.
12. Some differential equations can be solved easily using Laplace transfonns
(Ln. Consider the second order differential equation y"(t) + 3y'(t) +
2y(t) = 0 with y(O) = 1 and y' (0) = 3. Let y(s) be the LT of y(t). Find the
LT of the individual tenns of this differential equation, isolate y(s), and in-
vert the resulting transfonn to find y(t). Compare your result to what Maple
would find using dsol ve ( ) .
13. Invert the generating function

A( )
az =----~
18
(3 - z)(4 - z)2

using rgf _expand () to find the sequence a". Is the resulting sequence
a probability density of a discrete random variable? If it is, find its mean
using two different methods.
14. Consider two random variables X and Y with joint density given by f(x, y)
= e-(x+y) for 0 < x < 00 and 0 < y < 00 and zero otherwise. Find the
distribution function and the density of the ratio XI Y.
15. The Euclidean distance R of a random point (X, Y) from the origin is given
by R = JX2 + y2. If the joint density of (X, Y) is f(x, y) = 1/(1rz?) for
x 2 + y2 ~ u 2 and zero otherwise, find the mean distance E(R).

16. An experiment consists of n independent and identical trials where each

trial can result in one of three classifications for the items tested: (i) grade A
classification, (ii) grade B classification or (iii) defective with probabilities
p, q and 1 - p - q, respectively. Let X denote the number of grade A items,
and Y denote the number of grade B items. Detennine the joint density of
X and Y, their expectations, variances and covariance.

17. Consider two random variables X and Y distributed jointly with density

f(x,y) = 21rIU 2 exp ( x22:t) , - 00 <x < 00, - 00 <y < 00.

Let two new random variables(R, 9) be related to (X, Y) asX = Rcos(9)

and Y = Rsin(9). Detenninethejointdensity g(r, 0) of(R, 9). Show that
fooo fi IC g(r, 0) dO dr = 1.
4
Linear Programming

4.1 Introduction
Linear programming (LP) is a flexible and powerful optimization technique that is
used to detennine the nonnegative values of n decision variables x j, which satisfy
m linear constraints
n
LaijXj{~,=,~}bi' ;= 1,2, . .. ,m
j=)

and maximize (or minimize) a linear objective function

n
Z = ~:CjXj
j=)

where the parameters aij, bi and Cj are given constants.

The origins of LP date back to the late 1940s when o. B. Dantzig [58] de-
veloped a very efficient and general algebraic method known as the "simplex
method" that could solve LP problems.) About a decade later, Dantzig [59] pub-
lished a major survey covering the theoretical developments and applications of
linear programming up to the early 1960s. Since the publication of Dantzig's orig-
inal paper, practically hundreds of books and thousands of research articles on LP
have appeared in dozens of different languages.

IIn late 1930s the Russian mathematician Kantorovich [100] had, however, studied linear program-
ming models for production planning.

M. Parlar, Interactive Operations Research with Maple

© Birkhäuser Boston 2000
100 4. Linear Programming

Any textbook on operations research published since the 1950s has at least one
chapter describing LP. Linear programming is now a very important and estab-
lished branch of operations research that has found wide applicability in business
and governmental decisions.
Despite the mathematical nature of the simplex method, which requires a good
understanding of linear algebra, LP has become one of the most useful operations
research techniques and is frequently used by practitioners-perhaps due to the
proliferation of easy-to-use PC software such as LINDO and the inclusion of the
simplex algorithm in almost every spreadsheet software such as Excel and Quat-
troPro. A recent survey by Lane, Mansour and Harpell [116] indicates that along
with simulation and statistical methods, LP has found its way as an important
technique into the practitioner's toolkit of analytical methods.
In this chapter, we will briefly describe the mathematics of the simplex method
and present some examples of LP modeling and solution using Maple. Since LP
problems cannot be solved symbolically (i.e., the parameters ajj, bj and Cj must
be real nwnbers), Maple's symbolic manipulation capabilities will not be used in
this chapter. However, since Maple can plot regions of linear inequalities with the
command inequal ( ) , this plotting feature will be very useful in identifying the
feasible region and the optimal solution of an LP with n = 2 decision variables.

4.2 Graphical Solution

To motivate the discussion, we first formulate a simple LP with two decision
variables and three constraints and present its solution using the graphical method.

Example SO A simplified production problem. Suppose we are in the business of

producing toy cars and toy trains. The accounting department has analyzed the
costs and revenues and found that each car made (and immediately sold) results
in a profit of $30 and the profit from each train is $40. We have two departments
where these toys are made. The car department has a daily production capacity of
90 units and the train department 60 units. A complicating factor in the production
of these toys is a special part that has to be purchased from an outside source that
can provide only 600 parts per day. The engineering department has done some
computations and found that each car needs 5 parts and each train needs 6 parts.
The problem is to find the optimal values of the decision variables that maximize
the daily profit.
In this problem it is clear that there are n = 2 decision variables. These are
defined as XI : nwnber of toy cars to produce per day and X2 : nwnber of toy trains
to produce per day. The objective is to maximize the daily profit z = 30xI +40X2.
The car department's production capacity indicates that the solution must satisfy
XI ~ 90. Similarly for the train department the constraint is X2 ~ 60. Finally, the
special parts constraint is given as 5xI + 6X2 ~ 600. Since negative production is
impossible, we also have the "natural" constraints XI ~ 0 and X2 ~ O. Thus, the
 4.2 Graphical Solution 101

linear programming problem for the toy manufacturer can be stated as

max z = 30xI + 40X2

s.t. XI ~ 90
X2 ~ 60
5xI + 6X2 ~ 600
XI, X2 ::: 0

In this simple case, it is possible to plot the feasible region F (i.e., the set of all
points (XI, X2) satisfYing all constraints) and find the point that maximizes the
objective function z = 30xI + 40X2. We do this using Maple's inequal ( )
function that can plot linear inequalities and hence the feasible region F.

At this point we introduce the concepts of a convex set and an extreme point
that play an important role in the theory of linear programming.

Definition 1 A set:F is said to be convex if, for any two points x I, X2 E :F, tire line
segmentjoining the two points lies entirely within the set, i.e., .hI +(I-l)X2 E :F
forO~l~1.

Intuitively, a convex set cannot have any "holes" in it and its boundaries are
always "flat" or "bent away" from the set, as we will see in the current example.

Definition 2 A point x is said to be an extreme point ofa convex set if and only
if there do not exist other points XI, X2 (XI i= X2) in the set such that X = Axl +
(I - l)x2for 0 < l < I.

Intuitively, an extreme point cannot be "between" any other two points of the
set.
We start by introducing the constraints, which are placed in a list named CS
(Constraint Set). The feasible region is plotted (but not yet displayed) using the
inequal () function. We also plot-but do not yet display-three iso-profit
lines corresponding to profit levels of z = 1200, 2400 and 3600. These lines will
be useful when we attempt to identifY the optimal solution. The feasible region :F
(and the iso-profit lines) are displayed with the display () command. We find
that the feasible region is a convex set bounded by the lines X2 = 0, XI = 90,
5xI + 6X2 = 600, X2 = 60 and XI = 0 with the comer points at 0, A, B, C and D.
> restart: # ToyLP.mws
> with (plots) :
> CS: ={ xC 1] <=90, xC 2] <=60,5* xC 1] +6* xC 2] <=600,
xC 1] >=0, xC 2] >=0] i
CS:= [XI ~ 90, X2 ~ 60, 5xI +6X2 ~ 600, 0 ~ XI, 0 ~ X2]
> FeasibleRegion:=inequal(CS, xC 1]=0 •• 120,
xC 2)=0 .• 120,optionsfeasible=(color=white),
optionsexcluded=(color=yellow)):
102 4. Linear Programming

120

FIGURE 4.1. The feasible region with extreme points 0, A, B, C and D and the iso-profit
=
lines for z 1200,2400 and 3600.

> ProfitLines:=contourplot(30*x[ 1] +40*x[ 2] ,

x[ 1] =-10 •• 120,x[ 2] =-10 • • 120,
contours~ 1200,2400,3600] ,linestyle=2,
numpoints=2000,color=black) :
> display([ FeasibleRegion,ProfitLines])i
It can be shown that in linear programming problems, such feasible regions are
always convex; see Hadley [79, p. 60]. Additionally, if the convex feasible region
corresponding to the linear programming problem is nonempty, it must have at
least one extreme (comer) point. In our example, the feasible set has five extreme
points (denoted by 0, A, B, C and D), which are located at the intersection of the
constraint boundary lines; see Figure 4.1. It is also important to add that if there
is a finite optimal solution to the linear program, then this optimal solution must
be an extreme point of the feasible region. For a discussion of these important
properties of the solution of linear programming problems, see Luenberger [127,
Chapter 2].

°
Returning to our problem, we observe that the extreme points of the feasible
region----except the uninteresting point at the origin where there is no produc-
tion activity-are easily computed by Maple as follows.
> with(sirnplex):
Warning, new definition for display

Warning, new definition for maximize

Warning, new definition for minimize

 4.2 Graphical Solution 103

We start by solving for the intersection point of the boundaries XI = 90 and

X2 = 0 of the constraints CS[ 1] and CS[ 5] , respectively. This is achieved by
first converting the inequality constraints XI ~ 90 and X2 ~ 0 to equalities using
the convert () function. The obvious solution corresponds to the comer point
A on the feasible set. Substituting the solution XI = 90 and X2 = 0 in z gives
z = 2700.
> sol ve (convert ({ CS[ 1] ,CS[ 5] } ,equality) ) ;
assign(%); z:=30*x[ 1] +40* x[ 2] ;
x[ 1] : =' x[ 1]' : x[ 2] : =' x[ 2]' :
{X2 = 0, XI = 90}
z:= 2700
The coordinates of the next three points S, C and D are similarly computed and
the objective function is evaluated at each point.
> sol ve (convert ({ CS[ 1] ,CS[ 3] } ,equality) ) ;
assign(%); z:=30*x[ 1] +40* x[ 2]; x[ 1] :=' x[ 1]' :
x[ 2] : =' x[ 2] , :
{X2 = 25, XI = 90}
z:= 3700
> sol ve (convert ({ CS[ 3] ,CS[ 2] } ,equality) ) ;
assign(%); z:=30*x[ 1] +40*x[ 2]; x[ 1] :=' x[ 1)' :
x[ 2) : =' x[ 2] , :
{X2 = 60, XI = 48}
z:= 3840
> solve (convert ({ CS[ 4] ,CS[ 2]} ,equality));
assign(%); z:=30*x[ 1] +40*x[ 2]; x[ 1] :=' x[ 1]' :
x[ 2) : =' x[ 2]' :
{X2 = 60, XI = O}
z:= 2400
Thus, we find that the optimal extreme point is C with coordinates (XI, X2) =
(48,60) that gives rise to the highest profit level of z = 30 x 48 +40 x 60 = 3840
dollars per day.

Although in this example it was not too difficult to examine each extreme point
individually and find the optimal solution, this method is not very efficient if com-
putations are being performed manually. In problems with several constraints, the
number of extreme points may be quite large, which makes the computation of
the coordinates of the extreme points and the corresponding profit values unnec-
essarily cumbersome. In those cases, it would be preferable to use a visual method
whereby an iso-profit line is drawn on the feasible region for an arbitrary value of
z and moved in a direction to maximize the profit.
We start by arbitrarily assigning a fixed value to the daily profit z, say, z =
1200, and ask whether it is possible to find at least one feasible production com-
104 4. Linear Programming

bination (XI, X2) that gives rise to the value of z = 1200. We see in Figure 4.1
that for z = 1200, the lowest iso-profit line passes through the points (40,0) and
(0, 30), and hence eve!)' point between and including these points gives a profit
ofz = 1200.
Increasing the value of z (say, doubling it to 2400) and redrawing the line, we
see that it is still possible to find some feasible production combinations. Hence,
by visual inspection (perhaps by using a ruler and sliding it upward parallel to the
first two iso-profit lines) we find that the optimal point must be at the intersection
of the boundary lines 5xI + 6X2 = 600 and X2 = 60. Since the solution of this
system can be found easily, we conclude that the optimal result is at point C with
coordinates XI = 48 and X2 = 60 with a maximum profit of z = 3840. It is
important to note that with the iso-profit line method it is not necessary to solve
for the coordinates of eve!)' extreme point. Once we visually identify the optimal
extreme point, its coordinates can then be found by solving a system of two linear
equations in X I and X2. '
It should be clear that when there are only n = 2 decision variables, this
graphical method (while cumbersome) can always be used to locate the opti-
mal solution-when it exists. If there are n = 3 decision variables, the graphical
method can stilI be used but it becomes vel)' difficult to identify the feasible re-
gion as it becomes a solid geometrical figure (a polytope) bounded by planes in
three dimensions. When there are four or more variables, the graphical method
can no longer be used to find the optimal solution since it becomes impossible
to plot objects in dimensions higher than three. Real-life models that may have
thousands of variables and constraints are commonly solved using the simplex
method.

4.3 The Simplex Method

Essentially, implementing the simplex method involves nothing more difficult
than solving a sequence of system of linear equations. This requires the con-
version of linear programming problems with a mixture of~, = or ~ constraints
to the fonn Ax = b where A is an m x n matrix of constraint coefficients (with
columns denoted by aI, a2, ... , an), b is an m x I vector of right-hand-side (RHS)
constants and x is an n x I vector of unknown decision variables. The vector x
contains the original set of decision variables and a new collection of variables
introduced to convert the inequalities to equalities.
For example, the toy manufacturing problem with three constraints can be con-
verted to a linear system of the fonn Ax = b as follows. Since the first constraint
is XI ~ 90, by introducing a slack variable X3 ~ 0, we can write XI + X3 = 90.
Similarly, for the second and third constraints X2 ~ 60 and 5xI + 6X2 ~ 600,
we introduce the slack variables X4 2: 0 and xs 2: 0, respectively, and write
X2 + X4 = 60 and 5xI + 6X2 + xs = 600. These conversions result in the follow-
 4.3 The Simplex Method 105

ing problem with m = 3 equality constraints and n = 5 variables:

max z = 30xI + 40X2 + OX3 + OX4 + Oxs

s.t. XI + X3 = 90
X2 + X4 = 60
SXI + 6X2 + xs = 600
allxj ~ 0

It is usually assumed that the slack variables make no contribution to the ob-
jective function nor do they result in any cost. Thus, in this example X3, X4 and
xs appear in the objective function with zero coefficients. Of course, if there is
a profit or cost associated with the slack variables it should be reflected in the
objective function. See Ignizio [96, p. 115] for a more detailed discussion of this
issue.
Note that the constraint equations XI + X3 = 90, X2 + X4 = 60 and SXI +
6X2 + XS = 600 form a system of m = 3 equations in n = 5 unknowns. Thus,
the optimal solution of the optimization problem must be a solution to this set
of linear equations. But since m < n, this system will have an infinite number of
solutions.2 However, in the simplex method only a finite number of these solutions
(known as the basic solution set) will be of interest.
Given a system Ax = b, we may select any m x m nonsingular submatrix of A
(termed the basis matrix), set the remaining n - m nonbasic variables to zero and
solve the resulting system for the m basic variables. This gives a basic solution
denoted by XB.
Denoting the basis matrix by B, the resulting system after setting n - m vari-
ables to zero may be written as BXB = b or XB = B- 1b. Thus, finding the basic
solution XB involves a matrix inversion that can be easily performed by Maple.
Since the decision variables must be nonnegative, the simplex method examines
only the basic feasible points for which XB ~ O. We see that the number of basic
solutions must be limited by n!J[m!(n - m)!] since this is the number of combi-
nations of n variables taken m at a time. For example, as we will show, the toy
manufacturing problem with n = 5 and m = 3 has a total of eight basic solutions,
which is less than the maximum possible of5!J[3!(S - 3)!] = 10.
Suppose we set X4 = XS = o. In this case the basic variables would be XI, X2
and X3 for which the basis matrix B and the basis XB are obtained as follows.
> restart: # BasicSols.rnws
> with (linalg) :
Warning, new definition for norm

Warning, new definition for trace

=
2More accurately, if the system AI b is consistent (i.e., the rank of the coefficient matrix, r(A),
is equal to the rank of the augmented matrix, r(A I b» and ifr(A) < n, then the system has an infinite
number of solutions; see Ignizio [96, Chapter 3].
106 4. Linear Programming

> System: =[ x[ 1] +x[ 3] =90, x[ 2] +x[ 4] =60,

5* x[ 1] +6* x[ 2] +x[ 5] =600] ;
System := [XI + X3 = 90, X2 + X4 = 60, 5 XI + 6X2 + xs = 600]
> A:=genmatrix(System,[ x[ 1] ,x[ 2] ,x[ 3] ,x[ 4] ,x[ 5]],
RHS) ;

1 0 I 00]
A := [ 0 I 0 I 0
5 600 I
> b:=evalm(RHS);
b := [90, 60, 600]
> for j from 1 to 5 do a[ j] :=col (A, j) od:
> B:=augment(a[ 1] ,a[ 2] ,a[ 3]);

B:= [ 50I 60I 00I]

> xB:=evalm(inverse(B) &* b);
xB := [48, 60, 42]
Thus, XB = (XI, X2, X3) = (48,60,42)', which is a feasible point. This solution
corresponds to producing 48 cars and 60 trains, leaving an idle capacity of 42
mits in the car department.
But when X2 = X4 = 0, the resulting basis matrix for the columns (ai, a3, as)
is

B = ( 05I 00I 0)
0
I
whose inverse does not exist since X2 + X4 = 0 + 0 =F 60.
> B:=augment (a[ 1] ,a[ 3] ,a[ 5] ) ;

B:= [ 05I 00I10]

0

> xB:=evalm(inverse(B) &* b);

Error, (in inverse) singular matrix
If we choose (ai, a2, as) as the columns of the basis matrix (i.e., if X3 = X4 =
0), then we find that XB = (XI, X2, xs) = (90,60, -210)', which is an infeasible
basic solution. This corresponds to producing 90 cars and 60 trains that would
require a total of 90 x 5 + 60 x 6 = 810 parts, which is 210 more than the
available 600.
> B: =augment (a[ 1] ,a[ 2] ,a[ 5] ) ;
 4.3 The Simplex Method 107

10
B:= [ 0 1
5 6
> xB:=evalm(inverse(B) &* b);
xB:= [90, 60, -210]
The following table swnmarizes results for all possible solutions, some of which
do not exist and some of which are infeasible. The five comer points 0, A, B, C
and D as shown in Figure 4.1 are indicated in the first colwnn labelled "POINTS."

POINTS,!. Xl X2 X3 X4 xs PROPERTY
1 (C) 48 60 42 0 0 Feasible
2 (B) 90 25 0 35 0 Feasible
3 90 60 0 0 -210 Infeasible
4 120 0 -30 60 0 Infeasible
5 ? 0 ? 0 ? Singular B
6 (A) 90 0 0 60 150 Feasible
7 0 100 90 -40 0 Infeasible
8 (D) 0 60 90 0 240 Feasible
9 0 ? 0 ? ? Singular B
10(0) 0 0 90 60 600 Feasible

Essentially, the simplex method locates the optimal solution by examining only
a small fraction of the basic feasible solution generated using the procedure de-
scribed. It is well known that most linear programming problems are solved within
the range of 105m to 3m iterations (i.e., comer point evaluations) where m is the
nwnber of constraints; see Wagner [190, p. 116].

4.3.1 Manual Solution ofthe Production Problem

We now demonstrate the steps involved in the simplex method by solving the toy
manufacturing problem with Maple's help.
We first define the system of three equations in three unknowns arising from the
introduction of the slack variables X3, X4 and xs. Next, the coefficient matrix A is
generated from these equations using genmatrix () . We also extract the right-
hand-side vector b and define the objective function coefficient vector e. The jth
column of the A matrix is denoted by a j, which is indicated by a[ j] in Maple
notation.
> restart: # ToyLPManualSimplex.mws
> with (linalg) :
Warning, new definition for norm

Warning, new definition for trace

108 4. Linear Programming

> System: 9: x[ 1] +x[ 3] =90, x[ 2] +x[ 4] =60,

5* x[ 1] +6* x[ 2] +x[ 5] =600] :
System:= [XI +X3 = 90, X2 +X4 = 60, 5xI +6X2 +xs = 600]
> A:=genmatrix(System,[ x[ 1] ,x[ 2] ,x[ 3] ,x[ 4] ,x[ 5]],
RHS) ;

10 I 0 0 ]
A := [ 0 I 0 I 0
5 600 I
> b:=evalm(RHS);
b := [90, 60, 600]
> c:=vector (5,[ 30,40,0,0,0] );
C := [30, 40, 0, 0, 0]
Next, we fonn a basis matrix B from A by using the last three columns in the
order B = [a3, 84, as]. Thus, bl = a3, h2 = 84 and b3 = as.
> ft a[ 3], a[ 4] and a[ 5] are in basis
> for j from 1 to 5 do a[ j] :=col(A,j) od:
> B:=augment(a[ 3] ,a[ 4] ,a[ 5]):

B := [~0 0~ 1~]
The basic solution XB = (XBI,XB2,XB3)' = (X3, X4, xs)' given by this basis
matrix B is found as XB = B-1 b = (90,60,600)'. This implies that if XI = X2 =
0, then X3 = 90, X4 = 60 and xs = 600; i.e., if nothing is produced, then there
will be an idle capacity of 90 units and 60 units in the car department and train
department, respectively. With no production, the 600 parts will also not be used.
This corresponds to Point 0 in Figure 4.1.
The basic solution XB is also associated with a I xm vector(B = (CBI, CB2, ••• ,
CBm) of the objective function coefficients of the basic variables X B I , X B2, ..• ,
XBm. In this example, (B = (0,0,0) since the coefficients of the slack variables
are all zero. Hence, given a basic feasible solution XB ~ 0, the value of the ob-
jective function, z, is found as z = (BXB . In our case, we find z = 0 since
(B = (0,0,0) and "B = (90,60,600)'.
> xB:=evalm(inverse(B) &* b);
xB := [90, 60, 600]
> cB: =vector ( 3, [ c[ 3] ,C[ 4] ,C[ 5]] ) ;
cB := [0, 0, 0]
> z:=evalm(cB &* xB);
z :=0
 4.3 The Simplex Method 109

Since the basis matrix B consists of linearly independent vectors bl, h7, ... , bm ,
any column 8j of the coefficient matrix can be written as a linear combination of
the columns ofB, i.e.,
m
8j = LY;jb;
;=1
= BYj

where Yj = (Ylj,Y2j, . . . ,Ymj )'. Since the inverse ofB exists, we can also write
Yj = B- 18 j. In our example we find YI = (I, 0, 5)' and Y2 = (0, I, 6)'. Note that
the entries of the Yj vectors correspond to an amount by which a basic variable's
value will change if one unit of a nonbasic variable x j is introduced into the
solution. For example, if we decide to produce one car (i.e., increase XI from to °
I while keeping X2 at 0), then YI3 = 5 implies that the value of xs will be reduced
by 5 units since

xs = 600 - 5xI - 6X2

= 600 - 5xI - 6 x 0
= 6OO-5xl.

These values are computed in the next line.

> y[ 1] :=eva1m(inverse (B) &* a[ 1] );
y[ 2] : =eva1m (inverse (B) &* a[ 2] );
YI := [I, 0, 5]
Y2 := [0, I, 6]
Using the definition of the Yj 's provided, we now define a new scalar quantity
Zj = L~I Y;jCB; = CBYj. This important quantity is interpreted as the decrease
in the value of the objective function that will result if one unit of a nonbasic
variable X j is brought into the solution. At this particular stage of the solution, we
find that ZI = (0,0,0) x (1,0,5)' = 0 and Z2 = (0, 0,0) x (0, 1,6)' = O.
> z[ 1] :=eva1m(cB &* y[ 1] );
z[ 2] :=eva1m(cB &* y[ 2] );

ZI :=0
Z2:=0
Now we are faced with the important question of improving the currently avail-
able solution. To do this we define Cj - Zj as the net contribution resulting from
introducing one unit of a nonbasic variable x j into the solution. The x j that gives
the highest Cj - Z j value is chosen as a new basic variable. (Ties can be broken
arbitrarily.) In our example, we see that since CI - ZI = 30 and C2 - Z2 = 40, we
decide to make X2 a new basic variable-i.e., include the column vector 82 in the
basis matrix ~and assign X2 the highest possible value in order to increase the
profit Z as much as possible.
110 4. Linear Programming

> c[ 1] - z[ 1]; c[ 2] - z[ 2] ;
30
40
> .. So, a{ 2] enters
However, this cannot be done arbitrarily. For example, the parts constraint is
5XI + 6X2 + XS = 600, which can be written as xs = 600 - 6X2 since XI = o.
Thus, the highest possible value that X2 can take without making xs infeasible is
600/6 = 100. But recalling that the train production constraint was X2 + X4 = 60,
or X4 = 60 - X2, we see that the value of X2 = 100 cannot be allowed. Thus, the
highest value that X2 can take would be 60 units, which is the minimum of the two
ratios 60/1 = 60 for X4 and 600/6 = 100 for xs.
These results can be summarized using a table that indicates that the nonbasic
variable X2 should enter the basis and X4 should leave the basis (i.e., the vector a2
should replace the vector 84 in the basis matrix B).
XI X2 X3 X4 xs
Basis Cj 30 40 0 0 0 b Ratio
X3 0 1 0 1 0 0 90 -
X4 0 0 I 0 1 0 60 60/1 = 60 (out)
xs 0 5 6 0 0 1 600 600/6 = 100
Zj 0 0 0 0 0
Cj -Zj 30 40 0 0 0 0
(in)
> i f y( 2] [ 1] >0 then print (xB[ 1] Iy( 2] [ 1]) else
print("No Ratio") fi;
''No Ratio"
> if y( 2] [ 2] >0 then print (xB[ 2] Iy( 2] [ 2]) else
print("No Ratio") fi;
60
> i f y( 2] [ 3] >0 then print (xB[ 3] Iy( 2] [ 3]) else
print("No Ratio") fi;
100
> .. So, a{ 4] leaves
This example suggests that the column (say, b,.) of the basis matrix B that
should be replaced by a j is the one that satisfies

XBr
Yrj
= min (XBi,
I Yij
Yij > 0). (4.1)

The rest of the Maple worksheet implements the simplex procedure until the
optimal solution is reached. Now with (a3. a2, as) in the basis, we find that IB =
(X3, X2. xs) = (90,60,240)' with an objective fimction value as Z = 30 x 0 +
40 x 60 = 2400 since Xl = 0 and X2 = 60. This corresponds to Point D in Figure
4.1.
 4.3 The Simplex Method III

> B: =augment (a[ 3] , a[ 2] , a[ 5] ) ;

B:= [~o 6~ ~]
1
> xB:=evalm(inverse(B) &* b);
xB := [90, 60, 240]
> cB:=vector (3,[ c[ 3] , c[ 2] , c[ 5]] ) ;
cB := [0, 40, 0]
> z:=evalm(cB &* xB);
z:= 2400
> y[ 1] : =evalm (inverse (B) &* a[ 1] ) ;
y[ 3] :=evalm(inverse (B) &* a[ 3] );
YI := [I, 0, 5]
Y3 := [1,0, 0]
> z[ 1] :=evalm(cB &* y[ 1] );
z[ 3] :=evalm(cB &* y[ 3] );
ZI := 0
Z3 :=0
> c[ 1] - z[ 1]; c[ 3] - z[ 3] ;
30
o
In this iteration we find that al should enter and as should leave the basis.
> ..So, a[ 1] enters
> if y[ 1] [ 1] >0 then print (xB[ 1] /y[ 1] [ 1]) else
print("No Ratio") fi.
90
> if y[ 1] [ 2] >0 then print (xB[ 2] /y[ 1] [ 2]) else
print("No Ratio") fi;
"No Ratio"
> if y[ 1] [ 3] >0 then print (xB[ 3] /y[ 1] [ 3]) else
print("No Ratio") fi;
48
> .. So, a[ 5] leaves
Now, with (a3, a2, al) in the basis, the basis vector is found as llB = (X3, X2, XI)
= (42,60,48), which gives rise to an objective function value of Z = 30 x 48 +
40 x 60 = 3840 since XI = 48 and X2 = 60. This corresponds to Point C in
Figure 4.1.
> B:=augment(a[ 3],a[ 2],a[ 1]);
112 4. Linear Programming

oI 1]
0
6 5
> xB:=evalm(inverse(B) &* b);
xB := [42, 60, 48]
> cB: =vector ( 3, [ c[ 3] ,C[ 2] ,C[ 1] ] ) ;
cB := [0, 40, 30]
> z:=evalm(cB &* xB);
z:= 3840
> y[ 4] :=evalm(inverse (B) &* a[ 4] );
y[ 5] : =evalm (inverse (B) &* a[ 5] ) ;

Y4 := [~, I, ~1
YS := [~I ,0, ~J
> z[ 4] :=evalm(cB &* y[ 4] );
z[ 5] :=evalm(cB &* y[ 5]);
Z4 :=4
ZS :=6
We also find that C4 - Z4 = -4 and cs - zs = -6. This implies that we have
found the optimal solution since introducing any of the nonbasic variables (X4 or
xs) back into the basis would lower the objective function.
> c[ 4] - z[ 4]; c[ 5] - z[ 5] ;
-4
-6
> * Optimal solution reached!
In summary, the optimal solution is to produce XI = 48 cars and X2 = 60 trains
that would result in a maximum profit of Z = $3840. The final simplex table
follows.

XI X2 X3 X4 xs
Basis c· 30 40 0 0 0 b
X3 0 0 0 I 6/5 -1/5 42
X2 40 0 I 0 I 0 60
XI 30 I 0 0 -6/5 1/5 48
30 40 U 4
~~ 3840
Zj
Cj -Zj 0 0 0 -4 -6
 4.3 The Simplex Method 113

4.3.2 Solution Using Maple s simplex Package

Fortunately, it is not necessary to solve LP problems ''manually.'' Maple has an
implementation of the simplex algorithm that can be used either to maximize or
minimize a linear objective function subject to linear constraints. (Maple's imple-
mentation of the simplex algorithm is based on the initial chapters of Chvatal's
Linear Programming [46].)
We now return to the toy manufacturing problem and solve it using Maple's
simplex package. In order to see the steps used by Maple, we insert the com-
mand infolevel[ simplex] : =5 before loading simplex. Note that it is
not necessary to introduce the slack variables as Maple automatically adds them
to each constraint.
> restart: * ToyLPSimplex.mws
> infolevel[ simplex] : =5;
info/eve/simpla := 5
> with (simplex) :
Warning, new definition for maximize

Warning, new definition for minimize

> z : =30* xC 1] +40* xC 2] ;
z := 30xI +40X2
> Const[ 1] : =x[ 1] <=90; Const[ 2] : =xC 2) <=60;
Const[ 3) : =5* x[ 1] +6* xC 2] <=600;
Constl := XI ~ 90
Const2 := X2 ~ 60
Const3 := 5xI + 6X2 ~ 600
The optimal solution is found as XI = 48, X2 = 60 and z = $3840, as before.
> Sol:=maximize(z,{ seq (Const[ i) ,i=1. .3)} ,
NONNEGATIVE);
simple~ feasible) verify that constraints are linear

simple~ feasible) the variables are of type NONNEGATIVE

simple~ feasible) initial equations:

[ _ SLI = 90-~ 1], _ SL2 = 60-~ 2], SL3 600-5* ~ 1] -6* ~ 2] ]

simple~ feasible) initial equations already feasible

maximize: eqns =

maximize : trans =
114 4. Linear Programming

{}
simplex/max: eqns = [_SL1 = 90-x[ 1), _SL2 = 60-x[ 2) ,
_SL3 =600-5*x[ 1) -6* x[ 2))

simplex/max: t [_AR, _SL1, _SL2, _SL3, x[ 1), x[ 2)) •

eqns =, LSLl = 90 - XI, _SL2 = 60 - X2, _SLJ = 600 - 5xI - 6X2]

obj =, 30xI + 40X2

pivot variable =, XI
pivot equation =, _SLl = 90 - XI

simplex[ pivot) working on

XI

_SLl =90-XI

simplex/max: * [_AR, _SL1, _SL2, _SL3, x[ 1), x[ 2)) •

eqns =, [XI = - _SLl + 90, _SL2 = 60 - X2, _SLJ = 150 + 5 _SLl - 6X2]

obj =, -30 _SLl + 2700 + 40X2

pivot variable =, X2

pivot equation =, _SL3 = 150 + 5 _SLl - 6 X2

simplex[ pivot) working on

simplex/max: * [_AR, _SL1, _SL2, _SL3, x[ 1), x[ 2)) I

I 5
eqns =, [XI = - _SLl + 90, _SL2 = 35 + 6_SL3 - 6_SL1,
I 5
X2 = -- SL3 + 25 + - SLl]
6- 6-
. to 20
obJ =, 3" _SLl + 3700 - 3" _SL3
pivot variable =, _SLl
Pivot equation =, - SL2 = 35 +!6-SL3 - 5 SLl
-6 -
simplex[ pivot) working on
 4.3 The Simplex Method 115

I 5
[XI = -_SLJ + 90, _SL2 = 35 + -6-SL3 - - SLJ
6-'
I 5
X2 = -6 _SL3 + 25 + 6_SLJ]
SLJ
I 5
SL2 = 35 + - SL3 - - SLJ
- 6- 6-
simplex/max: tttttt final description tttilt

6 I 6 I
eqns =, [XI = -5 - SL2 + 48 - -5-SL3, - SLJ = --
5-SL2 + 42 + -5 - SL3,
X2 = 60 - _SL2]

obj =, -4 _SL2 + 3840 - 6 _SL3

Sol := {XI = 48, X2 = 60}
> assign(Sol); Z;
3840

It is interesting to note that Maple follows a slightly different path in finding

the optimal solution for this problem. As an exercise, we leave it to the reader
to follow Maple's steps and detennine the points examined by Maple. It is worth
noting that Maple also infonns the user that in the optimal solution the final basis
consists of XI, X2 and X) (which is denoted as _SL I by Maple), and hence the
nonbasic variables are X4 (denoted by _SL2) and xs (denoted by _SL2). Using
the infonnation under the section of the output titled "final description,"
we can easily reconstruct the final simplex table. The last set of equations found
in this section are given as

6 I
XI = - SL2+48-- SL3
5- 5-
6 I
_SLI = -- SL2 +42 + - SL3
5- 5-
X2 = 60- SL2.

Collecting the variables on the left-hand side and writing _S L I, _S L2 and _S L3

as X), X4, and xs, we get

6 I
XI - -X4 + -xs = 48
5 5
X2+X4 = 60
6 I
X) + -X4 - -xs = 42.
5 5
116 4. Linear Programming

This is exactly the same set of equations that is implied by the infonnation pre-
sented in the final table at the end of Section 4.3.1. Moreover, infonnation about
the objective function that is given as obj =, -4_SL2 + 3840 - 6_SL3 corre-
sponds to the Cj - Zj row in the final simplex table: If _S L2(= X4) is made a basic
variable, the objective function decreases by $4 and if_SL3(= xs) is made a basic
variable, the objective function decreases by $6. Since _SL2 = 0 and _SL3 = 0
in the optimal solution, the maximum value of the objective is Z = $3840.

4.3.3 A Problem with Mixed Constraints

LP problems with a mixture ofless-than-or-equal-to, greater-than-or-equal-to and!
or equality constraints require special care before they can be solved with the sim-
plex method. While each less-than-or-equal-to constraint can easily be converted
to an equality constraint by the inclusion of a slack variable, the other constraint
types necessitate additional steps.
Consider, for example, the following problem with two variables and three con-
straints:
min Z = al + 4X2
s.t. XI + 5X2 ::s 80
4xI + a2 ~ 20
XI + X2 = 10
XI, X2 ~ 0

Using the graphical method, it can be shown that this problem has the optimal
solution where XI = 10, X2 = 0 with the minimum value of the objective function
Z =20.
If we want to solve this problem using the "manual" version of the simplex
method as demonstrated in Section 4.3.1, we need to convert the inequalities to
equalities. The first constraint is converted by introducing a slack variable, say,
X3 ~ 0, which results in X I + 5X2 + X3 = 80. By introducing a "surplus" variable,
say, X4 ~ 0, and subtracting it from the left-hand side of the second constraint,
we obtain 4xI + 2x2 - X4 = 20. Since the last constraint is already an equality,
we obtain the linear system and the objective function as

min Z = 2x1 + 4X2 + OX3 + OX4

s.t. XI + 5X2 + X3 = 80
4xI + 2x2 X4 = 20
XI + X2 = 10
allxj ~O

Although we now have a system of equations, the A matrix of coefficients does

not contain an immediately obvious starting feasible basis. To obtain a convenient
starting basis that is also feasible, we introduce two new ("artificial") variables
and add them to the left-hand side of the second and third constraints. This results
 4.3 The Simplex Method 117

in the following coefficient matrix A with columns (a I, a2, a), 114, as, 116).

= 80
- X4 + xs = 20
+ X6 = lO

=
It is now easy to see that the initial basis matrix can be chosen as B (a), as, 116),
which results in the initial solution of XB = = =
B- 1b (X), xs, X6) (80,20, 10).
But we note that the artificial variables must eventually be eliminated from the
basis, i.e., they must be forced to take a value of 0 in the optimal solution. In order
to assure this outcome, we assign a very high cost of keeping these variables in
the solution. That is, we rewrite the objective function as

where M is a "big" number.)

Since minimizing a function is equivalent to maximizing its negative, we may
also convert the objective function to

max -z = -2x1 - 4X2 - OX) - OX4 - Mxs - MX6

and solve this problem as a maximization. We leave the solution of this problem
to the reader as an exercise.
Fortunately, if Maple is used to solve this problem with the simplex pack-
age, there is obviously no need to introduce the slack, surplus or the artificial
variables as Maple takes care of this step automatically. Following is the solu-
tion using simplex where we have chosen to minimize the original objective
function min z = 2x1 + 4X2.
> restart: *
MixedConstraints.mws
> with(simplex}:
Warning, new definition for maximize

Warning, new definition for minimize

> z:=2*x[ 1] +4* x[ 2] ;
z:= 2xI +4X2
> Const[ 1] :=x[ 1] +5* x[ 2] <=80;
Const[ 2] : =4* x[ 1] +2* x[ 2] >=20;
Const[ 3] : =x[ 1] +x[ 2] =10;
Cons/l := XI + 5 X2 ::: 80
Consl2 := 20::: 4xI + 2X2
Consl) := XI + X2 = lO

3The "big" M could be approximately ten times larger than the largest parameter in the problem.
118 4. Linear Programming

> Sol:=rninirnize(z,{ seq (Const[ i) ,i=1..3)},

NONNEGATIVE);
Sol := {X2 = 0, XI = 10}
> assign(Sol); z;
20

4.4 Special Cases and Difficulties

Some linear programming problems may have unusual characteristics resulting
in certain complications in formulation and/or solution. We now examine these
special cases and attempt their solution using Maple.

4.4.1 Infoasibility
It is sometimes possible to formulate LP problems that do not have any feasible
points! As an example, consider the following formulation and an attempt to solve
the problem using Maple.
> restart: # Infeasible.rnws
> with (sirnplex) :
Warning, new definition for maximize

Warning, new definition for minimize

> z: =2* x[ 1] +3* x[ 2] ;

z :=2xl +3X2

> Const[ 1] :=x[ 1] +x[ 2] <=10; Const[ 2] :=x[ 1] +x[ 2] >=20;

Constl :=XI +X2 ~ 10
Const2 := 20 ~ XI + X2
> Sol:=rnaxirnize(z,{ seq (Const[ i) ,i=1..2)},
NONNEGATIVE);
So/:= {}
> assign(Sol); z;
2xI + 3X2
Clearly, since it is impossible to satisfy both the constraints XI + X2 ~ 10
and XI + X2 ~ 20, Maple cannot find a solution to this problem and gives the
"solution" as Sol: ={} ; -an empty set!
 4.4 Special Cases and Difficulties 119

4.4.2 Unbounded Solutions

Another type of unusual case is when the objective function can be maximized at
infinity. An LP model that gives rise to such a case cannot be the representation
of a real system. In this situation the analyst who modeled the system must return
to the drawing board and correct the faulty formulation.
Here is a simple example that produces an unbounded solution.
> restart: * Unbounded.mws
> Hnfolevel[ simplex] : =5;
> with (simplex) :
Warning, new definition for maximize

Warning, new definition for minimize

> z:=-2*x[ 1] +x[ 2] ;
z:= -2xl +X2
> Const[ 1] :=5*x[ 1] -x[ 2] <=20; Const[ 2] :=x[ 1] <=5;
Constl := 5 XI - X2 :5 20
Const2 := XI :5 5
> Sol: =maximi ze (z, { seq (Const[ i] ,i=1. .2)} ,
NONNEGATIVE);
801:=
> assign(Sol); z;
-2xl +X2

*
In this case Maple does not even attempt to make a statement about the exis-
tence of the solution and gives Sol: =. If the symbol before infolevel[ sim-
plex] statement is removed and the problem is solved again, one would see that
Maple correctly identifies the unboundedness of the solution and flags it.

4.4.3 Degeneracy
Normally, in an LP problem with m constraints, the optimal basis consists of m
basic feasible variables all assuming strictly positive values. In some "degenerate"
cases, the optimal basis may contain fewer than m positive basic variables as the
following example demonstrates.
> restart: * Degeneracy.mws
> with (simplex) :
Warning, new definition for maximize

Warning, new definition for minimize

120 4. Linear Programming

> infolevel[ simplex) : =2:

> z:=2*x[ 1) +6* x[ 2) ;
z := 2xI +6X2
> Const[ 1) : =x[ 1) +2* x[ 2) <=20;
Const[ 2) : =2* x[ 1) +3* x[ 2) <=30;
Const[ 3) : =x[ 2) <=10;
Constl := XI + 2X2 ~ 20
Const2 := 2xI + 3X2 ~ 30
Const) := X2 ~ 10
> Sol:=maximize(z,{ seq (Const[ i) ,i=1. .3)} ,
NONNEGATIVE);

simplex/max: tttttt final description tttttt

2 1 2 1
eqns =, LSLl = 3" _SL2 + 3"XI' X2 = -3"XI - 3" _SL2 + 10,
2 1
- SLJ = -XI
3 + -3-SL2]
obj =, -2_SL2+6O-2xl
Sol := {XI = 0, X2 = IO}
> assign(Sol); z;
60

Thus, in this case with m = 3 constraints, we would expect three basic variables
taking strictly positive values. But as we see in the final simplex table, the optimal
basis consists of the variables LSL1,X2,_SL3) = (0,10,0). Thus, two basic
variables are equal to zero, implying degeneracy.
Unlike the cases of infeasibility and unboundedness, problems having a degen-
erate solution do not present any serious difficulties. In the simplex method of
solution, degeneracy arises when the ratio test (4.1) produces a result whereby
the minimum ratio is not unique and the variable that is to leave the basis is not
immediately obvious. In such a case an arbitrary choice is made and the ties are
broken according to a specific rule such as choosing the variable with largest Yij
or selecting the variable with the smallest i index. For a good discussion of this
issue, see the classical text by Hadley [79, p. 113].

4.5 Other Examples

In this section we present a few more LP formulations and their solution with
Maple's simplex package. Unlike the examples in the previous sections where
 4.5 Other Examples 121

we considered problems with only a few variables and constraints, the examples
in this section involve problems that can potentially have a very large number of
variables and constraints.

4.5.1 The Transportation Problem

The transportation problem arises frequently in the optimal distribution of goods
from several supply points (e.g., factories, plants) to demand points (e.g., ware-
houses, customers, retail outlets). The objective is usually to find the best dis-
tribution plan to minimize the total cost of shipments from the supply points to
demand points.

Example SI Ski Shipments. We illustrate this model by considering a problem

faced by SunSno, a multi-national company that operates three factories in (I)
Jasper, Canada, (2) Seoul, Korea, and (3) Toronto, Canada. SunSno ships skis to
four company owned warehouses in (I) Frankfurt, Germany, (2) New York, USA,
(3) Paris, France, and (4) Yokohama, Japan. The weekly production capacities Si,
; = I, 2, 3 of the m = 3 factories and the weekly demands dj, j = 1, ... ,4 of
the n = 4 warehouses are given in the following table where we also indicate the
unit transportation costs Cij, ; = I, ... , 3 and j = I, ... , 4 in bold face.

From,l.. "To-+ Frankf't (I) NY (2) Paris (3) Yok'ma(4) SUPPLY

19 7 13 8
Jasper (1) 100
IS 21 18 6
Seoul (2) 300
II 3 12 20
Toronto (3) 200
DEMAND 150 100 200 150 600

To solve this problem as a linear program, we define Xij as the munber of units
shipped from factory i to warehouse j for factories; = I, ... , m and warehouses
j = I, ... , n. Thus, the LP model of the transportation problem takes the follow-
ing fonn:
m n
min LL CijXij
i=1 j=1

subject to
n
LXij = Si, ; = I, ... ,m
j=1
m
LXij = db j = I, ... ,n
j=1
122 4. Linear Programming

with Xjj ~ 0, ; =I, ... , m and j =

I, ... , n.
Specializing this to the SunSno company's decision problem, we obtain the
following fonnulation.

min z = 19xI\ + 7x12 + 13x13 + 8x14 + +15x21 + 21xll + 18x23 + 6X24

+llx31 + 3X32 + 12x33 + 20X34

subject to

XII +X12 +XI3 +X14 = 100 (Jasper capacity)

X21 + Xll + X23 + X24 = 300 (Seoul capacity)
X31 + X32 + X33 + X34 = 200 (Toronto capacity)
XII + X21 + X31 = ISO (Frankfurt demand)
XI2 +Xll +X32 = 100 (N.Y. demand)
XI3 + X23 + X33 = 200 (paris demand)
XI4 + X24 + X34 = 150 (Yokohoma demand)
Allxjj ~ O.
Note that, in general, fonnulating the transportation problem as a linear pro-
gram results in m x n decision variables and m + n constraints. However, although
there is a total of m + n constraints in transportation problems, the total number
of basic variables is never more than (m + n - I) due to the redundancy in the
constraints-if a solution is found that satisfies the first m + n - I constraints,
it must automatically satisfY the last one since the total supply equals the total
demand.
= =
In this problem we have 3 x 4 12 variables and 3 + 4 7 constraints. Also,
since the total supply ofL.~=1 Sj = 600 units equals the total demand L.:=I dj =
600, the factories will be shipping all that they can produce and the demands of the
warehouses will be satisfied exactly. Transportation problems where L.~=I Sj =
L.j=1 dj are said to be balanced.
Fonnulation and solution of this problem using Maple is now a straightforward
exercise in entering the objective function and the constraints. Once that is done,
we again use simplex to find the optimal solution.
> restart: * Transportation.mws
> m:=3: n:=4:
> c:=matrix(m,n,[[ 19,7,13,8],[ 15,21,18,6],
[ 11, 3, 12 , 20] ] ) ;

19
13 7
8]
c := [ 15 21
18 6
II 3
12 20
> z:=sum(sum(c[ i,j] *x[ i,j] ,j=1. .n) ,i=1. .m):
> s:=array([ 100,300,200] ); sum(s[ i] ,i=1. .m);
S := [100, 300, 200]
600
 4.5 Other Examples 123

> d:=array ([ 150,100,200,150] ); sum(c( j] , j=1. .n);

d := [ISO, 100, 200, ISO]
600
> for i from 1 to m do
Supply[ i) :=sum(x[ i,j] ,j=1..n)=s[ i) od:
> i: =' i' :
> for j from 1 to n do
Demand[ j] :=sum(x[ i, j] , i=1. .m»=d[ j] od:
> j : =' j' :
> constraints:={ seq (Supply[ i) ,i=1. .m)} union
{ seq (Demand[ j] , j=1. .n)} :
> with(simplex):
Warning, new definition for maximize

Warning, new definition for minimize

> Sol:=minimize(z,constraints,NONNEGATIVE);

Sol:= {XI, I = 0, XI,2 = 0, X3,4 = 0, XI,4 = 0, X2,2 = 0, XI,3 = 100, X3,3 = 100,
X3,2 = 100, X2,3 = 0, X3,I = 0, X2,I = ISO, X2,4 = ISO}
> assign (Sol) :
> z;

5950

Thus, the optimal solution is obtained with a minimum cost of z = $5, 950. We
summarize the optimal solution in the following table.

From..j, "'T~ Frankf't (I) NY (2) Paris (3) Yok'ma(4) SUPPLY

19 7 13 8
Jasper (I) 0 0 100 0 100
15 21 18 6
Seoul (2) 150 0 0 150 300
11 3 12 20
Toronto (3) 0 100 100 0 200
DEMAND ISO 100 200 ISO 600
In some cases, total supply may not equal total demand. If total supply exceeds
total demand, i.e., when :L7'::1 S; > :Lj=1 dj. then the only modification neces-
sary is to rewrite the supply constraints as :Lj=1 X;j ::; S;, j = I, ... , m. In this
case excess supply would appear as a slack, which should be interpreted as the
WlUSed supply that is not shipped to any destination.
When total supply is less than total demand, i.e., when :L~=I S; < :Lj=1 dj.
then the LP formulation would be infeasible. In this case we would introduce a
124 4. Linear Programming

dummy origin with an artificial supply equal ~j=1 dj - ~~I Si, the difference
between the total demand and total supply. The optimal "shipment" amounts from
the dummy origin to a particular destination would then correspond to the amount
by which the destination's demand is unsatisfied.
When the number of origins and destinations is not excessively large, moderate-
size transportation problems can be solved using the simplex package. But for
a realistic problem with, say, m = 200 origins and n = 500 destinations, we
would have m + n = 700 constraints (less one due to redundancy) and m x n =
100000 variables! Thus, to solve these problems one would need more efficient
procedures. Such special-purpose procedures for solving transportation problems
indeed exist, but they have not been implemented by Maple. For a discussion of
these procedures see, for example, HiHier and Lieberman [92, Chapter 7].

4.5.2 Two-Person Zero-Sum Games

Competitive situations are characterized by the presence of two or more players
(decision makers) whose decisions influence the objective functions of alJ players.
A zero-sum (or a constant sum) game is one in which the gains of one player (or
players) are equivalent to the losses of another player (or players).
As an example, consider the simple coin-matching game with two players PI
and P2 who select a head (H) or a tail (T). If the outcomes match, i.e., both players
select H or T, player PI wins $1 from player P2. Otherwise, PI loses $1 to P2.
In this game, the 2 x 2 payoff matrix, expressed in terms of PI's payoff, is as
folJows.
PlayerP2
H T
Player PI H I -I
T -I I

A solution to a zero-sum game is obtained when each player determines his

"best" strategy without knowing, a priori, what the other player wiH do. In gen-
eral, the best strategy for a player is found by assigning a probability distribution
over the player's set of strategies. MathematicalJy, we let

Xi probability that PI will use strategy i = I, ... , m

Yj probability that P2 wiH use strategy j = I, ... , n
where m and n are the number of strategies available to PI and P2, respectively,
and ~~I Xi = I and ~j=1 Yj = 1. The vectors x = (XI, ••• , xm) and y =
(YI, ... ,Yn) are referred to as mixed strategies. But if some Xi = I (and alJ other
probabilities are zero), the player PI is said to use a pure strategy.
A generalJy agreed-upon objective is the expected payoff resulting from ap-
plying the mixed strategies. Denoting Pij as the payoff to PI if PI uses strategy
; = I, ... , m and P2 uses strategy j = I, ... , n, we find PI's expected payoff E I
 4.5 Other Examples 125

as
m
EI = LX;Yjp;j.
;=1

We now assume that PI will choose XI, ... , Xm in such a way that regardless of
how P2 chooses YI, ... , Yn, the gain to PI is at least equal to some value v where
v is to be made as large as possible. Thus, PI should choose XI, ... , Xm such that

m
LX;P;j ~ v, j = I, ... ,n
;=1

and that v is maximized. Note that the variable v is, in general, unrestricted in
sign. Additionally, since the XI, ... , Xm constitute a probability distribution, we
must have L7=1 X; = I with X; ~ 0,; = 1, ... , m.
For example, in the case of the coin-matching game, PI's problem would be to

max v
s.t. XI - X2 ~ v
-XI +X2 ~ v (4.2)
XI +X2 I =
XI,X2 ~ 0

where v is unrestricted in sign. Solving this problem (as we show below using
Maple) gives the optimal solution as XI = X2 = and v = o. !
Now considering P2's problem, we see that by using the same arguments em-
ployed for PI's strategy, P2 would choose YI, ... , Yn so that the gain to PI is at
most equal to some value u where u is to be made as small as possible where
variable u is, in general, unrestricted in sign. (Note that minimizing the gain to
PI maximizes P2's gain in zero-sum games.) Thus, P2 should choose YI, ... , Yn
such that
n
LYjP;j~U, ;=I, ... ,m
j=1

and that u is minimized. As before, since the YI, ... , Yn constitute a probability
distribution, we must have Lj=1 Yj = I with Yj ~ 0, j = 1, ... , n.
For the case of the coin-matching game, P2's problem would be to

min u
s.t. YI - Y2 ~ u
-YI + Y2 ~ u (4.3)
YI+Y2=1
YI,Y2 ~ 0

where u is unrestricted in sign. Solving this problem (as follows) gives the optimal
!
solution as YI = Y2 = and u = O. Hence, for this game the maximum PI can
expect to gain is equal to the minimum P2 can expect to lose, that is, u = v.
126 4. Linear Programming

The following Maple worksheet solves this simple coin-matching game for
each player. Note that the variables v and u in the first problem and second
problem, respectively, are allowed to take any positive or negative value since
they are not restricted. Normally, such unrestricted variables must be represented
as the difference of two nonnegative variables in a problem formulation, e.g.,
u = u' - u" where u' ::: 0 and u" ::: O. If the option NONNEGATIVE is not
included in simplex, all variables are asswned to be unrestricted in sign by
default-unless otherwise indicated as in { seq (x[ iJ >=0, i=1. .m)} .
> restart: * CoinTossGame.mws
> with (simplex) :
Warning, new definition for maximize

Warning, new definition for minimize

> P:=matrix(2,2,[ 1,-1,-1,1]);

P:= [ 1-I]
-I 1
> m:=2; n:=2;
m:=2
n :=2
> * Player l' s problem
> for j from 1 to n do
Player1[ j] : =add (x[ iJ * p[ i, j] , i=1. .m) >=v od;
Playerl l := v ~ XI - X2
Player12 := v ~ -XI + X2
> Unity:=add(x[ iJ ,i=1..m)=l;
Unity:= XI +X2 = 1
> maximize (v,{ seq (Player1[ j] ,j=1. .n)} union
{Unity} union { seq(x[ iJ >=0, i=1. .m)} );
1 1
{X2 = 2' XI = 2' v = O}
> * Player 2' s problem
> for i from 1 to m do
Player2[ i] : =add(y[ j]*P[ i,jJ ,j=1..n)<=u od;
Player2 1 := YI - Y2 ~ u
Player22 := -YI + Y2 ~ u
> Unity:=add(y[ j] ,j=1. .n)=l;
Unity := YI + Y2 = 1
> minimize (u, { seq (Player2[ i) , i=1. .m)} union
{Unity} union {seq(y[ j] >=O,j=1. .n)} );
 4.5 Other Examples 127

I I
{u = 0, Y2 = -, YI = -}
2 2

We now consider another example of a competitive situation with a larger num-

ber of strategies available to each player.

Example 52 Advertising policies. Consider a marketing problem where two firms

are about to launch competitive products [60, p. 34]. Firm PI has enough funds
in its advertising budget to buy three one-hour blocks of TV time and firm P2 has
enough money for two such blocks. We assume that the day is divided into three
periods: morning (M), afternoon (A) and evening (E). Each firm must purchase
its TV time in advance but naturally keeps the information confidential. It has
been determined that 20% of the TV audience watch in the morning, 30010 in the
afternoon and the remaining 50% in the evening. To keep the analysis simple, we
assume that no person watches in more than one period.
Past experience indicates that if a firm buys more time in any period than its
competitor, it captures the entire audience in that period. If both firms buy the
same number of blocks in any period (including zero blocks), then each gets half
the audience. For example, suppose firm PI chooses two morning hours and one
evening hour (MME) and firm P2 chooses one morning hour and one evening
hour (ME). In this case, PI captures alJ morning audience of2001o as it bought two
Ms. Since each firm has no afternoon hours they each capture half the afternoon
audience of 30010. FinalJy, since firm each has one E block, they again capture half
the evening audience of 50010. This means that PI's percentage of the market is
1.0 x 20010 + 0.5 x 30010 + 0.5 x 500/0 = 60%. The market share (i.e., the payoff)
matrix for P I where it has a total of 10 strategies and P2 has a total of 6 strategies
is given in the Table 4.1.

AM AA EM EA EE

MMM XI
I~
60
Y2
45
13
45
Y4
35
Y5
20
Y6
35
AMM X2 65 60 45 50 35 50
AAM Xl 55 65 60 40 50 50
AAA X4 55 55 65 30 40 40
EMM X5 75 70 70 60 45 35
EAM X6 80 75 70 65 60 50
EAA X7 80 80 75 55 65 40
EEM X8 65 60 70 75 70 60
EEA X9 80 65 60 80 75 65
EEE XIO I 65 50 60 65 60 75
TABLE 4.1. The market share matrix for player PI.
128 4. Linear Programming

Note that this is a constant-sum (not a zero-sum) game since the sum of both
players' market shares equals 1000/0. However, as both players still have diamet-
rically opposed interests the game can be reduced to a zero-sum game and solved
using the procedure described.
A careful examination of the payoff table reveals that EEM (xs) dominates the
three strategies AMM (X2), MMM (XI) and AAA (X4), and EAM (X6) dominates
EMM (xs) and AAM (X3). Thus, we could discard PI's first five strategies. If this
is done, then the P2's strategy AM en) dominates MM (YI). The game could
then be reduced to one with only five strategies for each player. We leave this
as an exercise and solve the complete problem with ten strategies for PI and six
strategies for P2.
> restart: * MarketingGame.mws
> with (simplex) :
Warning, new definition for maximize

Warning, new definition for minimize

> m:=10: n:=6:

> P: =matrix (m, n,[
[ 60,45,45,35,20,35] ,
[ 65,60,45,50,35,50] ,
[ 55,65,60,40,50,50] ,
[ 55,55,65,30,40,40] ,
[ 75,70,70,60,45,35] ,
[ 80,75,70,65,60,50] ,
[ 80,80,75,55,65,40] ,
[ 65,60,70,75,70,60] ,
[ 80,65,60,80,75,65] ,
[65,50,60,65,60,75]]) :
> * Player l' s problem
> for j from 1 to n do
Player1[ j] :=add(x[ i) *P[ i,j] ,i=1. .m»=v od:
> Unity:=add(x[ i) ,i=1..m)=l:
> maximize (v,{ seq (Playerl[ j] ,j=1. .n)} union
{Unity} union { seq(x[ i) >=0, i=l .. m)} );

{XI= 0, X2 = 0, X3 = 0, X4 = 0, xs = 0, X7 = 0, XIO = 0, X6 = 0,
2 I 190
X9 = -, xs = -, v = -}
3 3 3
> * Player 2' s problem
> for i from 1 to m do
Player2[ i) :=add(y[ j] *P[ i,j] ,j=1. .n)<=u od:
> Unity:=add(y[ j] ,j=1..n)=l:
> minimize (u,{ seq (Player2[ i) ,i=1..m)} union
{Unity} union { seq(y[ j] >=0, j=1. .n)} );
 4.5 Other Examples 129

2 4 I 190
{YS = 0, Y6 = 5" .Y2 = Is' Y3 = 3" YI = 0, Y4 = 0, U = 3"}
Clearly, the dominated states; = I, ... , 5 for PI and j = I for P2 do not
appear in the solution indicating that they would never be used. As in the coin-
tossing game, the maximum PI can expect to gain is equal to the minimum P2
= =
can expect to lose, i.e., U v 190/3.

4.5.3 A Linear Program with Randomly Generated Data

In this section we present a Maple worksheet that randomly generates the vector c
of objective function coefficients, the matrix A of constraints coefficients and the
vector b of RHS constants of a linear program.
We set m = 4 and n = 6 and generate the integer entries of the c vector
distributed uniformly between 0 and 10 using the command c: =randvector
(n, entries=rand (0 •• 10) ) ;. Using similar statements, we also generate
the A matrix and the b vector. Once the objective function and constraints are
defined using the mul tiply () command, the randomly generated problem is
solved in the usual way. In the optimal solution we find that XI = X2 = X3 =
Xs = 0, butx4 = ~ and x6 = W.
> restart: # RandomLP.mws
> m:=4;
m :=4
> n:=6;
n :=6
> with(linalg): with (simplex) :
Warning, new definition for norm

Warning, new definition for trace

Warning, new definition for basis

Warning, new definition for maximize

Warning, new definition for minimize

Warning, new definition for pivot

> readlib(randomize); randomize();

proc(n)
description 'Reset the seed for the random number generator'
end
943045446
130 4. Linear Programming

> c:=randvector(n,entries=rand(O .. 10));

c := [3, 2, 3, 7, 7, 8)
> A:=randmatrix(m,n,entries=rand(O .. 150));

102 116 57 125 51 4]

[ 118 57 127 6 64 24
A := 14 97 47 134 15 77
85 23 83 49 97 79
> b:=randvector(m,entries=rand(O •. 1000));
b := [87, 864, 565, 165)
> x:=vector(n);
x := array(1..6, [))
> z:=multiply(c,x):
> LHS:=multiply(A,x):
> constraints: ={ seq (LHS[ k] <=b[ k] ,k=1. .m)} :
> Sol:=maximize(z,constraints,NONNEGATIVE);
16362 6213
Sol := (XI = 0, X2 = 0, X3 = 0, Xs = 0, X6 = 9679' X4 = 9679)
> assign(Sol); evalf(z);
18.01704722
The reader can change the parameters m and n and solve different random
problems with the same worksheet. However, since every element of A is positive,
for large problems (e.g., with m = 200 and n = 400) one should expect Maple to
take several minutes to find the optimal solution.

4.6 Sensitivity Analysis and Duality

Very often, after solving an LP problem completely, it is advantageous to investi-
gate if changing any of the parameters affects the optimal solution. This could be
important since the values of parameters such as demand, revenue, or cost used
as inputs to the problem fonnulation are only estimates that are likely to deviate
from their true values when the solution is actually implemented. Such sensi-
tivity (postoptimality) analysis is an important part of any application of linear
programming. We now discuss several issues related to sensitivity analysis.

4.6.1 Changes in the RHS Values and Dual Prices

Let us return to the toy manufacturing problem introduced in Section 4.2 and
investigate if it is worthwhile to acquire additional resources. We know from the
solution to the original fonnulation that it is optimal to produce XI = 48 toy cars
 4.6 Sensitivity Analysis and Duality 131

and X2 = 60 toy trains that would result in a daily profit of z = $3840. With
this solution the capacity in the train department is utilized completely since the
constraint X2 ~ 60 is satisfied as an equality. Thus, we now wish to detennine if
it would be desirable to acquire additional resources and increase the production
capacity in the train department.
Let us suppose that the train production capacity is increased by 10 units to 70.
Re-solving the problem using Maple gives
> restart : # ToyLPRHS70.rnws

> with(sirnplex):
Warning, new definition for maximize

Warning, new definition for minimize

> z:=30*x[ 1] +40*x[ 2] ;

z := 30xI +40X2
> Const[ 1] : =x[ 1] <=90; Const[ 2] : =x[ 2] <=70;
Const[ 3] : =5* x[ 1] +6* x[ 2] <=600;
Constl := XI ~ 90

Const2 := X2 ~ 70
Const3 := 5xI +6X2 ~ 600
> Sol:=rnaxirnize(z,{ seq(Const[ i] ,i=1..3)},
NONNEGATIVE);
Sol := {XI = 36, X2 = 70)
> assign(Sol); z;
3880

With an extra capacity of 10 units in the train department, it is now optimal to

produce XI = 36 cars and X2 = 70 trains with a maximum daily profit of z =
$3880. Hence, the maximum profit increases by an amount of $40. This implies
that the marginal increase in profit for a unit increase in the train production
capacity is $40/10 = $4. This $4 is known as the dual price of the train production
department capacity. Naturally, if the marginal cost of increasing the capacity in
this department is less than the marginal benefit of $4, then management should
increase the capacity since the net benefit is positive.
For small increases in the available capacity in the train department, we ob-
serve that the dual price is $4. Does this result apply regardless of the amount of
capacity added to the train department? Increasing the capacity in increments of
ten units and resolving the problem gives the following.
132 4. Linear Programming

Train Department
Production Capacity XI X2 Z($)
60 48 60 3840
70 36 70 3880
80 24 80 3920
90 12 90 3960
100 0 100 4000
110 0 100 4000
120 0 100 4000

We see that if, for example, the new capacity is In. = 120, the optimal solution
is found as XI = 0, X2 = 100 and Z = $4000. Thus, for an increase of 60 units in
the capacity, the maximum profit increases by $4000 - $3840 = $160, implying
that the marginal benefit per unit is now only $160/60 = $2.67 instead of $4.
Why is the dual price no longer equal to $4? Using the graph of the feasible
region as in Figure 4.1, the reader can easily show that for such a substantial
increase in the train department capacity, this department's constraint becomes
redundant and thus increases in this capacity no longer add to the profit beyond a
certain threshold level.
If this is the case, there must be some range of values for the train department
capacity for which the dual price would still be $4. This is indeed the case as we
now demonstrate using Maple's help.
First, recall from the analysis of the problem in Section 4.3.1 that in the optimal
solution, the basic variables are XI = 48, X2 = 60 and X3 = 42 with the basis
matrix B = (aI, a2, a3). Now, let us introduce a parameter ~2 corresponding to
the change in the train department production capacity so that the RHS vector b
assumes the form b = (90,60+ ~2, 600)'.
With this change in the train production capacity, Maple tells us that the new
values of the optimal basic variables are now XB = B-Ib = (48 - ~~2' 60 +
~2, 42 + ~ ~2)' provided that XB ~ o. It is an easy matter to solve the resulting
set of inequalities for ~2, which gives -35 ~ ~2 ~ 40 as we shall see. This
result implies that as long as the train department capacity is not increased by
more than 40 units or is not decreased by more than 35 units, the optimal basis
(XI, X2, X3) remains optimal. Hence, for this range of ~2 values, the dual price of
$4 is still valid. Equivalently, this implies that as long as the new capacity in the
train department is between 25(= 60-35) and 100(= 60+40) units, the optimal
basis does not change and the dual price is still $4. These results are presented in
the following Maple worksheet.
> restart: * ToyLPRHSDelta2.mws
> with (linalg) :
Warning, new definition for norm

Warning, new definition for trace

 4.6 Sensitivity Analysis and Duality 133

> System: =( x[ 1] +x[ 3] =90, x[ 2] +x[ 4] =60+Delta[ 2] ,

5* x[ 1] +6* x[ 2] +x[ 5] =600] ;
System:= [Xl +X3 = 90, X2 +X4 = 60 + a2, 5XI + 6X2 +XS = 600]
> A:=genrnatrix(System,[ x[ 1] ,x[ 2] ,x[ 3] ,x[ 4] ,x[ 5]] ,
RHS) :
> b:=evalm(RHS);
b:= f)0, 60+ a2, ~
> for j from 1 to 5 do a[ j] :=col (A, j) od:
> B:=augment(a[ l],a[ 2],a[ 3] ):
> xB:=evalm(inverse(B) &* b);

xB := [48 - %a2, 60 + a2, 42 + %a2]

> solve({ seq(xB[ i) >=O,i=1. .3)} );
{-35 ~ a2, a2 ~ 40}

Using the approach described, it can also be shown that the dual price for the toy
car department production capacity is $0 and for the special parts constraint is $6.4
Moreover, it is easy to show that for the car production capacity, the pennissible
range of values for the change a I is -42 ~ a I, and for the special parts the range
is - 240 ~ a3 ~ 210. These findings are summarized in the following table.

Dual Allowable Current Allowable

Constraints Price Decrease in RHS RHS Increase in RHS
Car 0 42 90 00
Train 4 35 60 40
Parts 6 240 600 210

4.6.2 Changes in the Objective Function Coefficients

The second problem in sensitivity analysis is concerned with the effects of changes
in the solution that result from changes in the objective function coefficients of the
decision variables. For example, suppose that due to cost savings in the toy car
production department, the unit profit from each car increases to CI = $35 (up
from $30). Does this change the original optimal solution? It is easy to show that
the optimal solution now moves to a different comer point (to B in Figure 4.1)
where Xl = 90, X2 = 25 and z = $4150. Thus, for this increase of $5 in the
unit profit CI, the optimal solution changes. But if the increase in CI were small,
say, only $2, would the solution still have changed? Or more generally, for what

4Note that if there is any unused capacity in the car department (in this case, X3 = 42 units), adding
more capacity would not improve the profit since the new units would also not be used.
134 4. Linear Programming

range of CI values (the range ofoptimality) does the original optimal solution not
change?
Suppose that the profit from each car is 30 + 0 I dollars where 0 I is the amount
by which the unit profit changes. Thus, the objective coefficient vector now be-
comes C = (30 + 01,40,0,0,0) and since (XI, X2, X3) constitute the optimal ba-
sis, we have CB = (30 + 01, 40,0). This change in CB affects the Zj = CBB-Iaj
values and hence the Cj - Zj row in the final simplex table. Indeed, as the fol-
lowing Maple worksheet demonstrates, we find that C4 - Z4 = -4 + ~Ol and

°
C5 - Z5 = -6 - !Ol. In order for the original solution to remain optimal, we must
of course have C4 - Z4 ::: and C5 - Z5 ::: 0 for these nonbasic variables X4 and
X5 . Maple easily solves the inequalities and finds that for -30 ::: 01 ::: lj, the
original basis remains optimal.
> restart: # ToyLPCosttheta1.mws
> with (linalg) :
Warning, new definition for norm

Warning, new definition for trace

> System:={ x{ 1] +x{ 3] =90, x{ 2] +x{ 4] =60,
5* x{ 1] +6* x{ 2] +x{ 5] =600] :
> A:=genmatrix(System,[ x{ 1] ,x{ 2] ,x{ 3] ,x{ 4] ,x{ 5]] ,
RHS) :
> b:=evalm(RHS):
> c:=vector(5,[ 30+theta[ 1] ,40,0,0,0]);
C := pO + 01, 40, 0, 0, q
> for j from 1 to 5 do a[ j] : =col(A,j) od:
> B:=augment(a[ 1] ,a[ 2] ,a[ 3]):
> xB:=evalm(inverse(B) &* b);
xB := [48, 60, 42]
> cB:=vector(3,[ 30+theta[ 1] ,40,0]);
cB := pO + 0 I, 40, q
> for j from 1 to 5 do
y[ j] : =evalm(inverse(B) &* a[ j]);
z[ j] : =evalm(cB &* y[ j]) od:
> seq (c[ j] -z[ j] <=0, j=l. .5) ;

°: : °: : °
0, 0,
6 1
< 0, -4 + -5 01 ::: 0, -6 - -5 01 ::: °
> solve({ seq(c[ j] -z[ j] <=0,j=1..5)});
10
{-30::: 01 , 01 ::: T}
 4.6 Sensitivity Analysis and Duality 135

Next, writing 40 + fh as the new value of the unit profit from each toy train
(where fh is the change in this unit profit), it can be shown that as long as -4 :::
fh, the original optimal solution (XI = 48, X2 = 60 and z = $3840) remains
optimal. These results are summarized in the following table.
Allowable Current Allowable
Variable Decrease Coefficient Increase
XI (Car) 30 30 3.33
X2 (Train) 4 40 00

4.6.3 Addition ofa New Decision Variable

We now examine a third important issue in sensitivity analysis. Suppose that the
company is considering adding a new product (say, toy bulldozers denoted by
xp) to their product line where the profit for each bulldozer is cp = $50. Each
bulldozer needs some of the resources available in the car and train department.
In particular, each bulldozer needs labor and machine time resources from the
truck depamnent equivalent to 112 car, labor and machine time resources from the
tractor depamnent equivalent to I train, and JO special parts. Thus, the column
vector ap that would correspond to the new variable xp is a p = (!, I, 10)'.
Given this information, should the company should produce the bulldozers?
Although this problem could be formulated and solved with the addition of the
new variable x p, it is often easier to consider the effect of the new variable on the
optimal solution.
To do this, we first find the updated vector y p corresponding to xp by the trans-
formation YP = B-Iap, which gives Yp = (~, I, -in)'. Recall that the entries
of the yp vector correspond to the amounts by which a basic variable's value
will change if one unit of the nonbasic variable X p, is introduced into the s0-
lution. This is the vector that would have appeared in the final solution table if
we had initially introduced the xp into the formulation. Hence, zp = CBYp =
(30,40,0) x (~ , I, -in)' = 64, which is the amount by which the objective func-
tion value would decrease if one unit of xp is introduced into the solution. This
gives cp - z p = 50 - 64 = -14, implying that it is not beneficial to introduce
the new product into the solution. We see that profit from each bulldozer would
have to increase by $14 before it would be worthwhile to produce a bulldozer. The
Maple commands that perform the above computations follow.
> restart: * ToyLPNewVar . mws
> with (linalg) :
Warning, new definition for nonn

Warning, new definition for trace

> System: =[ x[ 1] +x[ 3] =90, x[ 2] +x[ 4] =60,
5* x[ 1] +6* x[ 2] +x[ 5] =600] :
> A:=genmatrix(System,[ x[ 1] ,x[ 2] ,x[ 3] ,x[ 4],x[ 5]) ,
RHS) :
136 4. Linear Programming

> b:=evalm(RHS):
> for j from 1 to 5 do a[ j] :=col(A,j) od:
> B:=augment (a[ 1] , a[ 2] , a[ 3] ) :
> c[p] :=50;

cp:= 50
> a[ p] :=vector (3,[ 1/2,1,10] );

Qp := [~, I, 10]
> y[ p] : =evalm (inverse (B) &* a[ p] );

YP := [~, I, ~~]
> cB: =vector (3,[ 30,40,0] ) ;
cD := [30, 40, 0]
> z[ p] :=evalm(cB &* y[ p] );
zp:= 64
> c[ p] - z[ p] ;

-14

These arguments also imply that while it is not optimal to produce bulldozers,
iffor some reason the management has to produce one unit (i.e., if the constraint
x p ~ I is introduced, which would result in x p = I), then the optimal profit
would decrease by S14. This amount is also known as the reduced cost associated
with the nonbasic variable xp.

4.6.4 Duality
In Section 4.6.1 we learned that the dual prices of the car and train production de-
partment resources and the special parts resource were SO, $4 and S6, respectively.
As we discussed, these dual prices can be found by increasing the right-hand side
of a particular constraint by one unit (thereby increasing the associated resource
by one unit) and re-solving the problem. The change in the objective function with
the higher value of the resource was the dual price for that resource constraint.
We now follow a different route and attempt to compute the dual prices as
the optimal solution to a linear programming problem ("dual program") that is
intimately related to the original problem (''primal program.")
Example 53 Furniture production. In this example, we consider a simplified ver-
sion of a furniture manufacturing problem where only tables (XI) and chairs (X2)
are made by using wood and labor as resources. The problem data are summarized
in the following table.
 4.6 Sensitivity Analysis and Duality 137

Unit Requirements Amount

Resouree Table Chair Available
Wood (board feet) 5 4 240
Labor (hours) 2 5 130
Profit/wit 10 5

Following is the primal program for this problem and its solution using Maple.

max zp = lOx I5X2 +

s.t. 5XI 4X2 + :s 240
2x1 5X2 + :s 130
Xl, X2 2: 0
>

>
restart:
* FurnitureDual.mws
infolevel[ simplex] :=2;
info/eve/Simpla := 2
> with (simplex) :
Warning, new definition for maximize

Warning, new definition for minimize

> zPrimal: =10* x[ 1] +5* x[ 2] ;

zPrima/ := 10 XI + 5 X2
> Constraintsp:=[ 5*x[ 1] +4*x[ 2] <=240,
2* x[ 1] +5* x[ 2] <=130] ;
ConstraintsP := [5 Xl + 4 X2 :s 240, 2 Xl + 5 X2 :s 130]
> PrirnalSol:=rnaximize(zPrimal,ConstraintsP,
NONNEGATIVE) ;

simplex/max: •••••• final description ••••••

1 4 2 17
eqns = , [XI = --
5-
SLl +48 - -X2 SL2 = 34 + - SLl- -X2]
5 '- 5- 5
obj =, -2_SL1+480-3x2
Prima/So/ := {X2 = 0, Xl = 48}
Note that by inserting the command infolevel[ simplex] : =2; we have
requested Maple to provide detailed information on the solution. In order to con-
serve space, we deleted the output wtil the final description, which gives
the final simplex table. The final stage in the solution indicates that the two basic
variables are Xl and _SL2, which is the slack variable for the second constraint.
(Using our standard notation, _SL2 would be denoted by X4.) Writing out the
equations after collecting the variables of the constraints on the left, we obtain the
following final simplex table for the primal problem.
138 4. Linear Programming

XI X2 _SL1(= X3) _SL2(= X4)

Basis Cj 10 5 0 0 b
XI 10 0 4/5 1/5 0 48
SL2(= X4) 0 1 17/5 -2/5 1 34
Cj - Z j 0 -3 -2 0 480
Thus, the optimal solution to the primal program is to produce XI = 48 tables
and X2 = 0 chairs resulting in a maximum profit of Z p = $480. It is easy to show
that if the wood availability increases by 1 board foot, then the optimal profit
increases by S2, which is the dual price for the wood resource. On the other hand,
increasing the labor availability does not increase the profit, implying that the dual
price for labor is SO.
10 the development of the dual program we assume that the dual prices of the
resources are not yet known, but they are the decision variables in a different
problem. Thus, for each of the two constraints in the primal program we introduce
unknown variables defined as UI : dual price for the wood capacity resource and
U2 : dual price for the labor capacity resource. We now generate a new linear
program whose solution gives us the values of the dual prices.
Suppose that management wishes to know the total value of the two resources
available. This value is 240uI + 130U2 since there are 240 board feet of wood
and 130 hours of labor available. Suppose now that management decides to re-
duce table production (XI) by one unit, which would release the wood and labor
resources for alternative use. Specifically, 5 board feet of wood and 2 hours of
labor would be released, which could then be sold in the marketplace. How much
would company management be willing to sell these resources for in order to be
at least as well off as before?
Since the value of released resources is SUI + 2U2 and the since the profit
foregone (by reducing the table production by one unit) is S10, the constraint
(4.4)
must be satisfied in order for the furniture company to be at least as well off as
before. Similarly, if the chair production is reduced by one unit, then the constraint
(4.5)
must be satisfied in order for the furniture company to be at least as well off as
before.
Finally, let us suppose that a prospective customer has been found who is will-
ing to buy the resources from the company. This customer would pay UI and U2
dollars for the wood and labor resources, respectively, while minimizing

ZD = 240uI + 130U2
and satisfying the two constraints (4.4) and (4.5).5

SIfthc constraints are not satisfied, company management would not sell the resources.
 4.6 Sensitivity Analysis and Duality 139

These arguments lead to the fonnulation of the dual program given by

min ZD = 240uI + 130U2
s.t. SUI + 2U2 > 10
4uI + 5X2 > 5
UI, U2 ~ 0
Maple has implemented a procedure (dual) that simplifies the generation (and
hence the solution) of the dual program. This is illustrated below. Note that we
enclosed the original constraints in a list (in square brackets [] rather than the
curly brackets {} of a set) since the order in which the constraints appear is
important.
> assign(PrimalSol); zPrimal;
480
> xC 1] : =' xC 1] , : xC 2] : =' xC 2]' :
> z[ P] :=10*xC 1] +5*xC 2] ;
zp := 10xi + 5X2
> DualProgram:=dual(zPrimal,ConstraintsP,u);
DuoIProgram:= 240ul + 130u2, [10 ~ SuI +2u2, 5 ~ 4ul + 5u2]
> DualSol:=minimize(DualProgra~ 1] ,DualProgra~ 2] ,
NONNEGATIVE) ;
simplex/max: tiit.. final description ••••••
I 2 4 17
eqns =, [ul=2+- SLl--u2, SL2=3+- SLl+-
5 u2]
5- 5 - 5-
obj =, -480 - 48 _SLl - 34 u2
DuoISoI := {u2 = 0, ul = 2}
> assign(DualSol); DualProgra~ 1];
480
Maple finds that the dual prices are UI = 2 and U2 = 0 with the minimum value
of the dual objective function ZD = $480. 6 It is interesting to note that without
solving the primal problem we could have found its optimal solution by using the
results for the optimal solution for the dual. The final table for the dual can be
constructed as follows.
UI U2 SLI SL2
Basis Cj 240 130 0 0 b
- SL2 0 0 -17/5 -4/5 I 3
UI 240 1 8/20 -1/5 0 2
Cj - Z j 0 -34 -48 0 480

6It is not clear why Maple gives obj =, -480 - 48_SL I - 34u2, which would in<:orrectly imply
that the minimum value of the dual objective is =D = -480 instead of +480.
140 4. Linear Programming

This example demonstrates several important properties of the primal and dual
problems that are true in general:

• The optimal objective function value for both problems is the same, i.e.,
zp=zD=480.
• The optimal values of the primal decision variables (x j) are the negatives
of Cj - Zj entries for the _SL.i variables in the final dual table. For the
primal, XI = 48 and X2 = O. These values could also be deduced from the
information in the final table in the dual: Cj - Zj = -48 for _SL 1 so that
XI = -(-48) = 48, and Cj - Zj = 0 for _SL2 so that X2 = -0 = o.

• The optimal values of the dual variables (u;) are the negatives of Cj - Zj
entries for the _SL.i variables in the final primal table. These values could
also be deduced from the information in the final table in the primal: Cj -
Zj = -2 for _SLI so that UI = -(-2) = 2, and Cj -Zj = 0 for _SL2 so
that U2 = -0 = o.
We can now generalize these ideas by presenting the concept of duality in a for-
mal manner. We first assume that the primal problem has been written in the form
of maximization with :s constraints. This is always possible since a ~ constraint
can be written as a :s constraint by multiplying both sides of the former by -1,
for example, XI +X2 ~ 3 can be written as -XI -X2 :s -3. Similarly, an equality
constraint can be converted to two inequality constraints; XI + 2x2 = 1 can be
written equivalently as XI + 2x2 :s 1 and XI + 2x2 ~ 1 (or, -XI - 2x2 :s -1).
The primal program is stated as
max Zp = ex
s.t. Ax :s b
x ~ O.

With this form of the primal, the dual will always be

min ZD = b/u
s.t. A/u ~ c'
U ~ O.

We now present some of the main theoretical results associated with the pair of
primal/dual programs.

• Suppose x and u are feasible solutions to the primal and dual programs,
respectively. Given x and u, the value of the primal objective function is
always less than or equal to the value of the dual objective function. This
result easily follows by noting that the inequality Ax :s b can be written as
x' A' :s b/. MUltiplying both sides on the right by any feasible u ~ 0 gives
x' A/u :s b/u = ZD. Now, writing A/u ~ c' as e' :s A/u and multiplying
both sides on the left by x ~ 0, we get zp = xc' x
:s A/u. But since
x' A/u :s b'u = ZD, we obtain zp :s ZD.
 4.6 Sensitivity Analysis and Duality 141

As an example, consider the furniture production example. The vector I =

(10, 10)' is a feasible solution for the primal problem and at that point the primal
objective function is Zp = ex = (10,5) x (10, 10)' = 150. Now consider the
vector u = (5, 5)', which is also feasible for the dual for which the dual objective
function iszD = b'u = (240, 130) x (5,5)' = 1850> zp = ISO.

• Another important property in duality theory that is a corollary to the previ-

ous result is that if i is feasible for the primal and uis feasible for the dual
such that ci = b'u, then i is an optimal solution for the primal and ii is an
optimal solution for the dual.

As an example, consider again the furniture production problem. For i =

(48,0)' and ii = (2,0), we have ci = (10,5) x (48,0)' = 480 and b'ii =
(240, 130) x (2, 0) = 480. But, the i and ii vectors are optimal for the primal and
the dual, respectively.

• A third important property concerns the existence of the optimal solutions

for the primal and dual: If either the primal or the dual has an optimal
solution, then the other has an optimal solution, too, and the correspond-
ing values of the objective functions are equal. Moreover, if B is the basis
matrix for the primal for an optimal solution I and if CB is the objective
function coefficient vector for the basic variables, then u' = cBB- I . Note
that Zj = CBYj = cBB-Iaj. Since Cj - Zj ~ 0 (when the primal is opti-
mal), then c-cBB- 1A ~ 0, which becomes c- u'A ~ 0 (after substituting
u' = cBB- I ), or A'u ~ c', which is, of course, the dual problem.

Returning again to the furniture example, we see that the optimal primal basis
is IB = (XI, _SL2) = (XI, X4). The solution for the dual variables u' = cBB- 1 =
(UI, U2) = (2,0) is given by the following Maple worksheet.
> restart: # DualForrnula.rnws
> cB: =vector (2,[ 10,0] ) ;
cB:= [10,0]
> B: =rna t r ix (2, 2, [ 5, 0, 2, 1] ) ;

> with (linalg) :

Warning, new definition for norm

Warning, new definition for trace

> u:=evalrn(cB &* inverse(B));

U := [2,0]
142 4. Linear Programming

Example 54 Dual of the problem with mixed constraints. As another example,

consider the problem with mixed constraints discussed in Section 4.3.3:

min z = 2x1 + 4X2

S.t. XI + 5X2 ::: 80
4xI + 2x2 ~ 20
XI + X2 = 10
XI, X2 ~ 0
This problem must first be converted to the standard form max z p = ex sub-
ject to Ax ::: b and x ~ O. After converting the objective to maximization, we
let Maple take over and do the necessary conversions of the constraints using the
convert (, stdle) command. This gives rise to a problem in standard form
with four ::: constraints where the last two are obtained from the equality con-
straintxI +X2 = 10.
Maple finds that UI = U2 = U3 = 0 and U4 = 2 where the last dual price
corresponds to -XI - X2 ::: -10. The minimum value of the dual program's
objective function is found as ZD = -20. The fact that UI = U2 = 0 implies that
increasing the RHS of the first two constraints in the primal program would not
affect the optimal value of the objective Z p. But U3 = 0 and U4 = 2 imply that
increasing the RHS of -XI - X2 ::: -10 by I to -10 + I = -9 would increase
the objective function z p = -20 by 2 units to -20 + 2 = -18. Translating these
to the original problem statement, we find that if XI + X2 = 10 were written as
XI + X2 = 9, then the optimal value of2x1 + 4X2 that is to be minimized would
be lower and reach a level of 18. The Maple worksheet that produces these results
follows.
> restart: # MixedConstraintsDual.mws
> with (simplex) :
Warning, new definition for maximize

Warning, new definition for minimize

> zPrimal: =-2* xC 1] -4* xC 2] ;
zPrimal := -2xI - 4X2
> ConstraintsP: =[ xC 1] +5* xC 2] <=80,4* xC 1] +2* xC 2] >=20,
xC 1] +xC 2] =10] ;
ConstraintsP:= [XI +5X2::: 80, 20::: 4xI +2X2, XI +X2 = 10]
> ConstraintsPStandard:=
convert(ConstraintsP,stdle);

ConstraintsPStandard :=
[XI + 5 X2 ::: 80, -4 XI - 2 X2 ::: -20, XI
+ X2 ::: 10, -XI - X2 ::: -10]
> PrimalSol:=maximize(zPrimal,ConstraintsPStandard,
NONNEGATIVE) ;
PrimalSoI := {X2 = 0, XI = 1O}
 4.6 Sensitivity Analysis and Duality 143

> assign(PrimalSol); zPrimal;

-20
> XC 1) : d XC 1)' : XC 2) : d XC 2) ,
> zPrimal;
-2xl - 4X2
> DualProgram:=dual(zPrimal,ConstraintsPStandard,u);

DuoIProgram:= 80ul- 20u2 + IOu3 - IOu4,

[-2::: ul - 4 u2 + u3 - u4, -4::: 5 ul - 2 u2 + u3 - u4]
> DualSol:=minimize(DualProgram 1) , DualProgram 2) ,
NONNEGAT IVE) ;
DuoISoI := {ul = 0, u2 = 0, u3 = 0, u4 = 2}
> assign(DualSol); DualProgram 1);
-20

Example 55 Duolity in game theory. In Section 4.5.2 we presented two examples

of the LP fonnulations ofzero-sum games and solved them using Maple. In both
examples, the minimum value of the second player's objective and the maximum
value of the first player's objective were found to be equal: In the first example
we found min u = max v = 0, and in the second example min u = max v =
190/3. The fact that the minimum value of the objective of one problem equals
the maximum value of the objective of another problem would suggest that there
may be a duality-type relationship between the two problems. This is indeed the
case and we now demonstrate it for the case of the coin-tossing example.
Without loss of generality, we will suppose that the second player's LP problem
is the primal program since its constraints are in ::: fonn. Recalling that in general,
the variable u is unrestricted in sign, we let u = u' - u" where u', u" 2: 0 and
rewrite the problem as

max -(u' - u")

s.t. Yt - Yl ::: u' - u"
-YI + Yl ::: u' - u"
YI+Yl=l
YI,Yl 2: 0

where the original objective function min u = min (u' - u") is converted to a
maximization objective as max -(u' - u").
At this stage, we let Maple take over and find the dual of Player 2's problem.
> restart: # CoinTossDual.mws
> with (simplex) :
Warning, new definition for maximize
144 4. Linear Programming

Warning, new definition for minimize

> P:=rnatrix(2,2,[ 1,-1,-1,1]);

> m:=2; n:=2;

m :=2
n:=2
> * Player 2' s problem
> for i from 1 to m do
Player2[ i) : =add (y[ j] * p[ i, j] , j =1. . n) <=up-upp od;
Player2 1 := YI - Y2 ~ up - upp
Player22 := -YI + Y2 ~ up - upp
> Unity: =add (y[ j] ,j=1. . n)=l;
Unity := YI + Y2 = I
> Constraints: =[ seq (Player2[ i) , i=l •• m) , Uni ty] ;
Constraints := [YI - Y2 ~ up - upp, -YI + Y2 ~ up - upp, YI + Y2 = I]
> ConstraintsSTDLE:=convert(Constraints,stdle);

ConstraintsSTDLE := [YI - Y2 - up + upp ~ 0, -YI + Y2 - up + upp ~ 0,

YI +Y2 ~ I, -YI-Y2 ~ -I]
> dual(-up+upp,ConstraintsSTDLE,x);

xJ -x4, [0 ~ xl -x2 +xJ -x4, 0 ~ -xl +x2 +xJ -x4, -I ~ -xl -x2,
I ~ xl +x2]
Maple finds the dual as follows:

min (Xl -X4)

s.t. XI - X2 + (Xl - X4) ~ 0
-XI + +
X2 (Xl - X4) ~ 0
XI +X2 ~ I
XI +X2 ~ l.

Letting v = Xl -X4 and noting that the two constraints XI +X2 ~ I and XI +X2 ~ 1
imply XI +X2 = I, we obtain the problem for Player 1 as given in (4.2).
Thus, since the two players' problems are the duals of each other, it is sufficient
to solve only one player's problem and deduce the solution for the other player
using the Cj - Z j values from the final optimal table.
 4.7 Integer Linear Programming 145

4.7 Integer Linear Programming

In many realistic applications of linear programming, the optimal values of de-
cision variables must be integer. For example, when a decision problem requires
finding the optimal number of employees to be assigned to jobs or number of
aircraft to purchase, the decision variables must be obviously limited to integer
values. Since the simplex method does not necessarily produce integer optimal
solutions, alternative means must be found to compute the optimal integer solu-
tions.
As a simple example of linear programming requiring integer values for the
decision variables, consider the following.
Example 56 Aircraft purchase. Air NoFrills is a small airline company that has
decided to acquire new aircraft for its operations. After some preliminary analysis,
management has decided that they will limit their choice to Boeings and Airbuses.
The relevant information on profitability, price and maintenance requirements are
as follows.
Boeing Airbus
Profit/year ($000) 200 100
Price/unit ($000,000) S 4
Maintenance timelyear (hours) 200 SOO

The company has a budget of$24,OOO,000 available for the purchases of these
aircraft, and the annual maintenance time available per year is 1,300 hours. The
problem is to find the optimal number of Boeings (XI) and Airbuses (X2) to pur-
chase in order to maximize the annual profit.
We first solve this problem as an ordinary linear program using Maple's sim-
plex package and find that the solution is not integer since XI = 4.8 and X2 = 0
withz = 960.
> restart: * AircraftLP.mws
> with (simplex) :
Warning, new definition for maximize

Warning, new definition for minimize

> z: =200* x[ 1] +100* x[ 2] ;
z:= 200xl + IOOX2
> Constraints: = { 5* x[ 1] +4* x[ 2] <=24. ,
200* x[ 1] +500* x[ 2] <=1300} ;
Constraints:= (SXI +4X2 ~ 24., 200xl +SOOX2 ~ 1300)
> maximize(z,Constraints,NONNEGATIVE);
(XI = 4.800000000, X2 = O)
> assign(%); z;
146 4. Linear Programming

960.0000000
At this stage one may be tempted to round the non integer solution up and down
to the nearest integer with the hope of obtaining an optimal integer result. This
usually does not work, as we see in the following lines. Rounding XI = 4.8 up to
5, we find an infeasible solution, and rounding down to 4 gives an integer result
that is not optimal!
> x[ 1] : =5; x[ 2] :=0; Z; Constraints;
XI :=5
X2 :=0
1000
{25 :'S 24., 1000 :'S l300}
> x[ 1] :=4; x[ 2] :=0; Z; Constraints;

XI :=4
X2 :=0
800
{20 :'S 24., 800 :'S l300}
Although Maple has not implemented any algorithms to solve integer linear
programs, it includes a share package written by Anu Pathria in its share folder. 7
This package is named i 1P () and it can be loaded by using the commands
with (share) ; and with (ilp) ;.
The ilp () function's syntax is identical to that used in simplex as can be
seen in the following Maple worksheet. Note that i 1 P () maximizes the objective
function in an integer programming problem. For minimization problems, the user
must first multiply the original objective function by -I and then maximize the
resulting function subject to the usual constraints. Once the optimal solution is
found, taking the negative value of the objective function would give the minimum
of the original objective.
> restart: * AircraftIP.rnws
> with (share) : with(ilp);
See ?share and ?share,contents for information about the
share library

Share Library: ilp

Author: Pathria, Anu.

7Anu Pathria's ilp () is available with Release 5.1 .ld it implements the branch-and-bound algo-
rithm for solving integer programming problems. A good reference on this algorithm is Zionts [198,
Chapter IS).
 4.8 Summary 147

Description: an integer linear programming algorithm for

maximizing a linear function given a set of linear
constraints
["i1p"]

> z: =200* x[ 1] +100* x[ 2] ;

z := 200xI + IOOX2
> Constraints: = { 5* x[ 1] +4* xl 2] <=24. ,
200* xl 1] +500* x[ 2] <=1300} ;
Constraints:= {5xl +4X2 ~ 24., 200xI +500X2 ~ 1300}

> ilp(z,Constraints,NONNEGATIVE);
{XI = 4, X2 = I}
> assign(%); z; Constraints;
900
{24 ~ 24., 1300 ~ 1300}

We thus see that it is optimal to purchase XI = 4 Boeings and X2 = I Airbuses

resulting in a maximum profit of z = $900,000.

4.8 Summary
Linear programming (LP) is one of the oldest and most thoroughly studied OR
technique. We started this chapter by introducing a simple example ofLP model-
ing with two decision variables and solved the resulting problem using the graphi-
cal method. For larger problems with four or more variables the graphical method
fails since plotting the feasible set becomes geometrically impossible. In that case,
the (algebraic) simplex method can be used to locate the optimal solution. We
provided a step-by-step description of the simplex method that involves inverting
matrices. This was made relatively painless by using Maple's inverse () com-
mand. We also explained the implementation of Maple's own simplex package
and discussed certain types of problems that may have unusual characteristics
such as infeasibility and unbounded solutions. Sensitivity analysis is always an
important component of any application of LP. This topic was also discussed in
some detail. The chapter ended with a very brief description of a share package
known as ilp () that can be used to solve integer linear programming problems.
148 4. Linear Programming

4.9 Exercises
1. Consider the following LP problem with two decision variables.

max 20xI +30X2

s.t. ~XI +X2 ~ 8
XI +X2 ~ 10
X2 ~ 4
XI ~ 0

(a) Solve this problem graphically using Maple's inequality plotting fa-
cilities.
(b) Solve the same problem manually (but with Maple's help) using the
simplex method.
(c) Now solve the problem using Maple's simplex.
(d) Formulate and solve the dual problem and interpret its solution.

2. The Kingdom of Albricol's national oil company that produces oil special-
ity additives purchases three grades of petroleum distillates A, B and C.
The company then combines the three according to King Peter the First's
specifications of the maximum and minimum percentages of grades B and
C, respectively, in each blend.
Max % Min % Selling Priee of
Blend Allowed for B Allowed for C Mixture (SIL)
Royal Deluxe 60% 10% 3.75
Princely Standard 20010 30% 3.10
Pauper's Special 1% 65% 1.99
The supply of the three additives and their costs are:
Maximum Quantity Cost of Distillate
Distillate Available per day (L) (SIL)
A 16000 .37
B 19000 .29
C 12000 .24
Formulate a linear program to determine the daily production policy that
will maximize profits for King Peter and his Kingdom. Solve using sim-
plex.
3. The Claus Cookie Manufacturing Society packages three types of cookies
into two different one pound assortment packages. Each week, 1000 pounds
of chocolate chip cookies, 2000 pounds of peanut butter cookies and 1500
pounds of coconut crispies are available. Assortment A contains 50 percent
chocolate chip cookies, 25 percent peanut butter cookies and 25 percent
 4.9 Exercises 149

coconut crispies. Assorbnent B contains 50 percent peanut butter cookies,

30 percent coconut crispies and 20 percent chocolate chip cookies. The net
profit contribution for each pound of Assorbnent A cookies is fifty cents,
while for Assorbnent B cookies it is forty cents.

(a) Fonnulate an appropriate linear program for Claus Cookie Manufac-

turing Society to find its optimal mix and solve using simplex.
(b) Fonnulate and solve the dual problem and interpret its solution.

4. Captain's Boats Inc. makes three different kinds of boats. All of these
boats can be made profitably, but the company's monthly production is con-
strained by the limited amount of wood, screws, and labor available each
month. The director will choose the combination of boats that maximizes
his revenue in view of the following infonnation:
Requirements per boat
Input Row BOAT CANOE KAYAK Available/month
Labour (hr.) 12 7 9 1,260 hrs.
Wood (board ft.) 22 18 16 19,008 bd. ft.
Screws (lb.) 2 4 3 3961bs.
Selling Price $400 $200 $500
Fonnulate this problem and solve using Maple's simplex.

(a) How many boats of each type will be produced and what will be the
resulting revenue?
(b) How much wood will be used to make all of these boats? If extra
wood were to become available, how much would Captain's Boats be
willing to pay for some?
(c) You will find that very few canoes are made because their selling price
is so small. Suppose someone offered to buy canoes for $x each. For
.what value of x, would Captain's Boats produce any canoes?
(d) Suppose Kayaks were sold for $y per unit rather than $500. For what
value of y, fewer Kayaks would be produced?
(e) What is the highest hourly wage Captain's Boats would be willing to
pay for extra workers?

5. Smart Bucks, Inc., an investment agency, has been asked to advise one of
its clients from the province of Kebek on how to invest all of his $100,000
among the 3 assets described below:
Units of Risk Rate of
Assets per Dollar Invested Retum
1. Northern Mines Shares 4 0.13
2. Royal Bank of Commerce Shares 3 0.12
3. Kebek Savings Bonds I 0.14
150 4. Linear Programming

The client would like as high an annual return as is possible to receive while
incurring an average of no more than 2.9 risk units per dollar invested. Ke-
bek Savings Bonds unfortunately are available only up to a value ofSl5,OOO
per Kebek resident.
Fonnulate and solve this problem using Maple's simplex and answer the
following questions.

(a) How much of each asset will be bought and what will be the average
rate of return on these?
(b) How much risk is being incurred on average? What is its shadow
(dual) price?
(c) Obviously Kebek Savings Bonds are paying an excessive rate of re-
turn in view of their relatively low level of risk. How low a rate of
return could the province of Kebek have offered for its bonds without
this investor having invested differently than he did?
(d) If the Royal Bank of Commerce raised the rate ofreturn on its shares
to 12.9%, would it be able to increase its sale of shares to the particular
investor we are discussing? What would be the new average rate of
return on this investor's portfolio?
(e) Fonnulate and solve the dual problem and interpret its solution.

6. Solve the following integer programming problem using Maple's ilp ( )

share package if this package is available in your system.

min 12x1 + 9X2 + 8x3 + 6X4

s.t. 4xI +X2 -X4 ~ 16
XI +X2 +X3 ~ 20
3xI +3X3 +X4 ~ 36
XI, X2, X3, X4 ~ oand integers
7. An airplane with a weight capacity of 40,000 pounds and a volume capacity
of 7,000 cubic feet will be sent to a disaster area. The weight and volume
of the container for each of the five items is given below. Each item has
an associated utility index that depends on the number of people who will
need these items.

Weight per Volume per Utility

Item Container (Ib.) Container (W) Index
Blood 80 25 7
Drugs 150 30 80
Supplies 130 40 1
Food 40 7 2
Water 70 5 5
 4.9 Exercises 151

Fonnulate the problem to find the optimal number of containers to ship in

order to maximize the total utility index. Assume that at least 50 containers
of blood, 15 containers of medical drugs, and 100 containers of water must
be shipped. Use the i 1 P () share package to solve this problem if this
package is available in your system.
5
Nonlinear Programming

5.1 Introduction
As we discussed in Chapter 4, in linear programming (LP) applications, our pur-
pose is to optimize (i.e., maximize or minimize) a linear objective function subject
to linear constraints. Although a large number of practical decision problems can
be accurately modeled as linear programs, it should be emphasized that in some
cases using LP to formulate an inherently nonlinear decision problem (i.e., forcing
the world to fit the model) may not provide an accurate representation of reality.
In many OR problems due to economies/diseconomies of scale (e.g., quantity
discounts), interaction among decision variables (e.g., variance ofa portfolio) or
transformations that result in nonlinearities (e.g., a linear stochastic programming
formulation transformed into an equivalent nonlinear deterministic one), an LP
formulation becomes impossible to use. In these cases, we would need to utilize
the techniques of nonlinear programming (NLP) that can deal with both nonlinear
objective functions and nonlinear constraints.
A general NLP problem with both inequality and equality constraints is ex-
pressed as follows:

min I(x)
s.t. g; (x)~ 0, i = I, . .. , m
hj(x) = 0, i = I, ... , P
XE s.
Here, I(x) is the objective function with x = (XI, ... , Xn)' as the column
vector of n decision variables; g;(x) ~ 0, i = 1, . . . , m, are the m inequality

M. Parlar, Interactive Operations Research with Maple

© Birkhäuser Boston 2000
154 5. Nonlinear Programming

constraints, hi(x) = 0, i = I, ... , P are the p equality constraints and S is a

subset of the n-dimensional Euclidean space Rn. (Of course, since max f(x) =
- min[- f(x)] , an NLP problem can be stated as a maximization problem. Also,
inequality constraints of the type gi(X) ~ 0, i = I, ... , m can be transformed to
this form ifboth sides of these constraints are multiplied by -I.)

Example 57 Portfolio selection. As a simple example of the formulation of a

nonlinear programming problem, consider a portfolio selection problem where
we wish to invest all of $10,000 in two stocks. Assume Wi is the random annual
return on $1 invested in stock i = 1,2 with the expected returns E(WI) = 0.10,
E(W2) = 0.05, variances Var(WJ) = 0.15, Var(W2) = 0.06, and the covariance
COV(WI, W2) = 0.07. If the objective is to minimize the variance of the portfolio
with an expected annual return of at least 91'10, we can define decision variables
Xi = amount invested in stock i = 1,2 and find the expected annual rate ofretum
as

which must be at least equal to 0.09, that is, O.lOxl +0.05X2 ~ 900. The variance
of the portfolio is computed as

Var(Wlxl + W2X2) = Var(WI)· xi + Var(W2)· x~ + 2COV(WI, W2)· XIX2

= O.15xf +o.06xi +O.14xIX2
Since the amounts invested must be nonnegative and we wish to invest all of
$10,000, this nonlinear programming problem can be formulated as follows.

min f(XI, X2) = 0.15xi + O.06x~ + 0.14x1X2

s.t. g(XI, X2) = 900 - 0.1 OXI - 0.05X2 ::: 0
h(XI,X2) = 10000 -XI -X2 = 0
XI ~ 0, X2 ~ o.
The optimal solution for this problem is obtained as XI = $8, 000, X2 =
$2, 000 and J = 1.208 x Iif with the standard deviation of the portfolio as
J1.208 x I ~ 3475; see Example 68.
The next section introduces the concept of convexity (of sets and functions),
which plays a very important role in the study of nonlinear programming.

5.2 Convexity of Sets and Functions

Let Rn be the n-dimensional Euclidean space. A nonempty set S c ~ is said to
be a convex set if the line segment joining any two points of S also belongs to S.
Stated mathematically, if XI, X2 E S, then the convex combination of XI and X2
also belongs to S, i.e., AXI + (1 - A)X2 E S where A E [0, I].
 5.2 Convexity of Sets and Functions 155

Let / : S ~ RI with S as a convex set. The function /(x) is said to be a

convex function on S if / (A. x I + (1 - A.)X2) ::: A./(XI) + (1- A.)/(X2) for each
XI, X2 e S where A. e [0, I]. Geometrically, this has the interpretation that when
the function / is convex, its value at points on the line A.XI + (1-A.)X2 is less than
or equal to the height of the cord joining the points (XI, /(XI» and (X2, /(X2».
Note that if A. e (0, I) and if the inequality is satisfied strictly, then / is a strictly
convex function. Also, if / is (strictly) convex, then - / is (strictly) concave.

Example 58 Convexity 0/x 2 -4x -8. Using the foregoing definition of convexity,
we now use Maple's assume () facility and is () function to check whether the
function /(x) = x 2 - 4x - 8 is convex. We let XI = a and X2 = b.
>
>
restart: *
convex.mws
assume (lambda>=O, lambda<=l, lambda, real,
a,real,b,real);
> f:=x->x A 2-4*x-8;
/ := X ~ x 2 - 4 X - 8
> LHS:=f(lambda*a+(l-lambda)*b);
LHS := (A. a + (I - A.) b)2 - 4 A. a - 4 (I - A.) b - 8
> RHS:=lambda*f(a)+(l-lambda)*f(b);
RHS := A. (02 - 4 a - 8) + (I - A.) (b 2 - 4 b - 8)
> is (LHS<=RHS) ;

Maple indicates that the condition LHS ::: RHS is always true indicating that
/ (A.a + (I - A.)b) ::: A./(a) + (I - A.)/(b), for all A. e (0, I) and all real a and
b. This proves the convexity of /(x) = x 2 - 4x - 8.
The following intuitive result is concerned with the sum of convex functions.

Theorem 1 Let the functions I; (x), i = I, ... , k be convex over some convex set
S eRn. Then the function /(x) = L~=I I; (x) is also convex over S.•

A useful theorem that relates the convexity of functions to the convexity of sets
is the following.

Theorem 2 [80, p. 86J If /(x) is a convex function over the nonnegative orthant
0/ R n, then if the set o/points S satisfying /(x)
::: b and x ::: 0 is not empty, Sis
a convex set. •

For example, since /(x) = xf + xi

is a convex function, the set S = {x :
/(x) ::: I, x ::: O}-which corresponds to the portion of the unit circle in the first
quadrant-is also convex.
156 5. Nonlinear Programming

Let the nonempty set S be a subset of Rn and let f : S ~ R'. If the function
f is differentiable at some point i e S, then its gradient vector at i is

Vf(i) = (Of (i) , ... , of(i»)'

ox, oXn

where of(i)/ox; is the partial derivative of f with respect to X; at the point i.

Maple excels in computing the partial derivatives-and hence the gradient of
a function-symbolically. In order to compute the gradient of an expression (or
function) using the grad () function, the linear algebra package must be loaded
with the command wi th (linalg) .
Example 59 Te"ain depth. Consider the following fimction f of two variables
adapted from Schmidt and Davis [167, p. 161] that is purported to describe the
depth of the terrain beneath a harbor's waters. With X and y as measured in miles
due east and due north, respectively, from the center of the harbor, the function
f(x, y) is given as

f(x,y) = -(5/IO)x 2 + (l/IO)y3 - (2/10)1 + (4/100)xy + (l/IOO)x +

(3/100)y - (5/10).

We find the gradient vector of the function as follows.

> restart: *HARBOUR1.MWS
> with (linalg) :
Warning, new definition for norm

Warning, new definition for trace

> f:=-(5/10)*x A 2+(1/10)*y A 3-(2/10)*y A 2
+(4/100)*x*y+(1/100)*x+(3/100)*y-(S/IO);

f := -!2 x 2 + ..!.. y3 - ! 1 + ..!.. X y + _1_ X + 2- y - !

10 5 25 100 100 2
> grad(f,[x,y));
1
[-x + 25 1
Y + 100' 10 1
3 2 1
- 5 y + 25 X + 100
3]
For a twice differentiable function f, we define the Hessian matrix H(x) of f
at x as a square matrix with the ith row and jth column entry as the second partial
derivative 02f(x)/oxjOXj:

0 2 f(x) 0 2 f(x)
,
aT ox,oxn
H(x) =
a2 f(x) a2 f(x)
oxnox, a;r
n
 5.2 Convexity of Sets and Functions 157

Provided that of(x)/ox;, i = I, .. . ,n and 02 f(x)/ox;oxj. i,j = I, . .. ,n are

continuous, the Hessian is a symmetric matrix; see [8, p. 121].
Maple has a hessian () function that simplifies the computation of the Hes-
sian. We demonstrate its use on the function f:

]
> H:=hessian(f,[x,y));

-I
[ -25
H:= 215 ~y-~
5 5

5.2.1 Positive and Negative Definite Matrices

Now, let Q be an n x n symmetric matrix. The matrix Q is called positive definite
if the quadratic form x'Qx > 0 for all nonzero x e Rn and positive semidefinite
if x'Qx ~O for all x e Rn. When the inequalities are reversed, we obtain the
definitions for negative definiteness and negative semidefiniteness, respectively.
The matrix Q is called indefinite if x'Qx is positive for some values of x and
negative for others. These definitions are summarized in the following table.

I >0
Sign oftbe Quadratic Form x'Qx
~O <0 ~O Mixed
-'Jo~X=FO for all x for x =F 0 for all x
Property Positive -PosltlVe-- Negative--- Negative Indefinite
of Q Definite Semidefinite Definite Semidefinite

Example 60 Quadratic forms. The quadratic fonn f(XI, X2) = x~ + is pos- xi

itive definite since f is positive except when x = 0, and the quadratic fonn
f(XI, X2) = (XI + X2)2 is positive semidefinite since f is positive except for
XI = -X2. But the quadratic fonn f(XI, X2) = x~ - xi
is indefinite since f is
positive for IXII > IX21 and negative for IXII < IX21.

It is usually difficult to check the positive and negative definiteness properties of

a symmetric matrix using the stated definitions. We will now present an easy-to-
use operational result that makes use of the signs of certain types of submatrices
of Q for examining these properties.

Tbeorem 3 (Sylvester s Theorem) {148, p. 16] For the n x n symmetric Q matrix,

we define Il" to be the determinant ofthe upper left-hand corner k x k submatrix
ofQfor k = I, ... , n, and call it the kth principal minor ofQ. Then, Q is positive
definite if and only if Il" > 0 for k = I, ... , n; and negative definite if and only
if (-I)" Il" > 0 for k = I, ... , n. Otherwise, Q is indefinite. These results are
summarized in the following table.
158 5. Nonlinear Programming

I °
Sign oftbe ~k'S, k = 1, ... , n
°
Q
~k >
Property of Positive
Definite
(_I)k ~k >
Negative
Definite
Mixed
Indefinite •
These results must be interpreted and used with care since merely replacing the
> sign by ~ (or < by:::) does not mean that the Q matrix is positive (or negative)
semidefinite. For example, if
> restart: * notsemi.mws
> Q:=rnatrix (3,3,[ 2,2,2,
> 2,2,2,
> 2,2,1]);

Q:= [ ; ; ;]
221

such as x = (!,!, -I)', i.e.,

°
then it is easy to show that ~ I > 0, ~2 = and ~3 = 0. But for a nonzero vector
> x:=array ([ 1/2,1/2, -1] );

x := [~, ~, -1].
we obtain x'Qx as
> eva 1m (x &* Q &* x);
-I
implying that Q is not positive semidefinite.
There is, however, a special case where it can be shown that if ~k > 0, k =
1, ... , n - I, and ~n = 0, then Q is positive semidefinite [148, p. 19]. Similarly,
if ( _I)k ~k > 0, k = 1, ... , n - I but ~n = 0, then Q is negative semidefinite.
In general, in order to determine the positive semidefiniteness of a symmetric
matrix, the following procedure is used: Let Qr be the matrix obtained by remov-
ing the first r rows and r columns of the original matrix Q, r = 0, ... , n - I (with

°
Qo == Q). Define ~rk to be the kth principal minor of Qr. Then, Q is positive
semidefinite if and only if ~rk ~ for r = 0, ... , n - 1, k = I, ... , n - r;
°
and negative semidefinite if and only if (-1 i ~rk ~ for r = 0, ... , n - 1,
k = 1, ... , n - r. For example, if n = 3 and

Q= [:~: :~ :~!],
q31 q32 q33

then for positive semidefiniteness we would need to show that

~02 = det [qll

q21
ql2]
q22
>°
-,
~03 = det Q ~ 0,
 5.2 Convexity of Sets and Functions 159

~Il = q22 ~ 0, ~12 = det [q22

q32
q23]
q33
~ 0, and ~21 = q33 ~ o.
Note that this method-which is actually implemented by Maple--of checking
the positive semidefiniteness of symmetric matrices requires the computation of
n + (n - 1) + ... + 2 + I = n(n + 1)/2 determinants. Although it would be
cwnbersome to implement this test manually, for Maple it is straightforward, as
we demonstrate in the next example.
Example 61 Maple's test for semi-definiteness with definite (). We let n =
3 and ask Maple to find the conditions under which a general (i.e., symbolic)
symmetric 3 x 3 symmetric matrix is positive semidefinite.
> restart: *
semid33.mws
> with (linalg) :
Warning, new definition for noon

Warning, new definition for trace

> H:=array(1 •. 3,1 .. 3,' symmetric);
H := array(symmelric, 1..3, 1..3, [])
> definite(H,'positive_semidef');

-HI,I ~ 0 and - + HI,22 ~ 0 and - HI,I H2,2 H3,3 + HI,I H2,3 2

HI,I H2,2

+ HI, 22 H3,3 - 2 HI,2 HI,3 H2,3 + HI,32lh,2 ~ 0 and -lh,2 ~ 0

and - H2,2 H3,3 + H2,3 2 ~ 0 and - H3,3 ~ 0
There is another method that makes use of the eigenvalues and provides infor-
mation on all aspects of positive and negative definiteness and semidefiniteness
of a symmetric matrix Q. We summarize this result in the following theorem.
Theorem 4 [148, p. 30J The symmetric matrix Q is positive (negative) definite if
and only if all the eigenvalues ofQ are positive (negative) and is positive (nega-
tive) semidefinite ifand only if all the eigenvalues ofQ are nonnegative (nonpos-
itive). However, matrix Q is indefinite if and only ifQ has at least one positive
eigenvalue and at least one negative eigenvalue.
Sign of the Eigenvalues of Q
All> 0 All > 0 All < 0 All < 0 Mixed
Property Positive Positive Negative Negative Indefinite
ofQ Definite Semidefinite Definite Semidefinite

Provided that the eigenvalues' signs can be easily determined, this test is sim-
•
pler to implement than the principal minor test. We should also note that although,
in general, the eigenvalues may asswne complex values, due to the symmetric na-
ture of the Q matrix that we are assuming, it can be shown that its eigenvalues are
always real [91, p. 32].
160 5. Nonlinear Programming

5.2.2 Convexity ofa Function and Definiteness of its Hessian

There is a very important relationship between the convexity of a function 1 and
the positive semidefiniteness of its Hessian matrix. For a proof of the following
theorem on this relationship, see Bazaraa and Shetty [18, Section 3.3].

Theorem 5 Let 1 : s ~ RI be a twice differentiable function on a nonempty

open convex set S c Rn. Then 1 is convex if and only if the Hessian is positive
semidefinite at each point 01 S. Also, if the Hessian is posidve definite at each
point 01 S, then the function 1 is strictly convex. But, if 1 is strictly convex, then
the Hessian is positive semidefinite at each point in S . •
The fact that strict convexity of 1 does not imply positive definiteness can
be seen if we take the strictly convex function I(x) = -b.x4. The Hessian is
H(x) = f" (x) = x 2 , which is positive semidefinite at x = o.
As demonstrated, Maple can be very helpful in identifying the positive or neg-
ative definiteness or semidefiniteness properties of a symmetric matrix. Since the
Hessian H is symmetric, Maple's definite () function automates the task of
determining the definiteness properties ofH and hence the convexity or concavity
of the function I .

5.2.3 Examples ofDefiniteness/Convexity

Example 62 Terrain depth (continued). Consider again the terrain depth func-
tion 1 discussed in Example 59. We first check to see if the Hessian is positive
(semi)definite. This being the irregular terrain beneath the water, we would not
expect such a property to hold true.
> restart: *
HARBOUR2.MWS
> with (linalg) :
Warning, new definition for norm

Warning, new definition for trace

> f:=-(S/10)*x A 2+(1/10)*y A 3-(2/10)*y A 2
+ (4/100) * x* y+ (11100) * x+ (3/100) * y- (SilO) ;
1 1 1 1 1 31
1 := -2 x 2 + lO y3 - 5 y2 + 25 x Y + 100 x + 100 Y - 2
> H: =hessian(f,[ x,y] ):
Now let US examine the negative (semi)definiteness of I. In this case, we would
expect the terrain to have this property, but due to the cubic term in I, the function
may be concave only over some regions:
> definite (H,' positive def' );
definite (H,' positive=:semidef' );
lalse
 5.2 Convexity of Sets and Functions 161

Ialse
> definite(H,'negative de~);
definite(H,'negative=semide~ );
125y - 83 < 0
125 y - 83 $ 0 and 3 y - 2 $ 0
Thus, we see that only in some regions, the terrain is negative definite (or nega-
tive semidefinite), and hence 1 is strictly concave (or concave). We can ask Maple
to solve these inequalities to obtain the region over which the terrain is, say, neg-
ative semidefinite:
> solve({definite(H,'negative_semide~)} ,y);

83
{y $ 125}
Since the defini te () function would not be directly useful to determine
where the Hessian is indefinite, (i.e., where 1 is neither convex nor concave) we
make use of the eigenvalue test and obtain the eigenvalues ofB as
> s:=eigenvalues(H);

7 3 1
s := - 10 + 10 Y + 50 J229 + 450 Y + 225 y2,
7 3 1
--
10
+ -10 y - - J229 +450y +225r
50
The two eigenvalues are given in terms of the variable y; thus, we ask Maple
to solve for the region of values of this variable for which the eigenvalues assume
opposite signs:
> solve ({ s[ 1] >0, s[ 2] <O} , y) ;
83
{125 < y}
This result implies that for {(x,y) : y > 83/125}, the terrain of the surtace
beneath the water resembles a part of a saddle without any convex or concave
regions. See Figure 5.1 for the plot of the surtace of the terrain of function 1
produced by the following Maple three-dimensional plotting command.
> plot3d(f,x=-15 .• 15,y=-15 •• 15,axes=boxed,
shading=none, orientation=[ -33,76] );

Example 63 Nonconvexity 01the backorder fimction in continuous-review (Q, r)

inventory model. We now use the above results and discuss an incorrect statement
made by Hadley and Whitin in their classic text on inventory theory [81, p. 221,
problem 4-6]. Under suitable assumptions, Hadley and Whitin (H- W) find the
average number of backorders per year for the (Q, r) model as

B(Q,r) ..1.1r
=Q 00
(x-r)k(x)dx
162 5. Nonlinear Programming

FIGURE 5.1. Surface plot of the terrain depth function f(x,y).

where Q is the order quantity, r is the reorder point, A. is the average demand per
year, and k(x) is the density of demand during the leadtime. It is easy to show that
the B(Q, r) function is separately convex in Q and convex inr, but despite H-W's
claim it is not jointly convex in both Q and r. Let us use Maple to demonstrate
this fact.
> restart: *backord.mws
> B:=lambda*int((x-r)*k(x),x=r •. infinity)/Q;

B .-
.-
A. 1 00
(x - r) k(x) dx
.....:!.!..r_ _ _ _ __
Q
> with (linalg) :
Warning, new definition for norm

Warning, new definition for trace

> H:=hessian(B,[Q,r]);

2
A. 1
r
00
(x -r)k(x)dx
Q3
A. 1
00
- k(x)dx

Q2
H .-
·-
1
A.
00
- k(x) dx A. k(r)
Q2 Q
In order for B to be positive semidefinite, the diagonal elements and the de-
terminant must be nonnegative. The diagonal elements are nonnegative, but the
determinant is computed as
> det(H);
 5.2 Convexity of Sets and Functions 163

A.2 (21°O -(-x+r)k(X)dXk(r)-(l°O -k(x)dxi)

Q4
Since A.2 / (t > 0, we see that if at r = 0 the density fimction has the prop-
erty that k(O) = 0, the detenninant assumes a negative value as it reduces to
-(A.2 / (t) [- CO k(x )dx]2 < O. Thus, contrary to H-W's claim, it is not, in gen-
eral, true that the backorder cost fimction is always convex.
However, as has been shown by Brooks and Lu [38], if ~ is a number such that
k(x) is nonincreasing for x ~ {, then B(Q, r) is convex in the region {(Q, r) :
o < Q < 00 and ~ =:; r < oo}.
Example 64 Convex;tyo/ /(a, b) = Ja 2 + 1J2. The Euclidean distance fimction
/(a, b) = Ja 2 + IJ2 (also known as the t2 distance fimction) plays a very im-
portant role in location theory. In this example we will show using Maple that
this fimction is convex over the real plane R2 except at the origin where it has no
derivative.
> restart: * sqrt.mws
> f:=sqrt(a A 2+b A 2);

> diff(f,a);
a
Ja 2 + IJ2
> limit (%,{ a=O,b=O) );
undefined
> with (linalg) :
Warning, new definition for norm

Warning, new definition for trace

> H:=hessian(f,[ a,b] );

H := [ (a 2 +~)(3/2) + J a21+ IJ2 (a2 } ;)(3/2) ]

ba b2 1
(a 2 + b2)(3/2) - (a 2 + b2)(3/2) + J a2 + IJ2
> H11 : =normal (H[ 1, 1] , expanded) ;
H22:=normal(H[ 2,2] ,expanded);
b2
HII := (a2 + b2)(3/2)

a2
H22 := (a2 + b2)(3/2)
164 5. Nonlinear Programming

> assume(a<O,a>O,b<O,b>O);
> is(Hll>O); is(H22>O);
true
true
> det(H);
o

Since the first principal minor of the Hessian is 111 = HII > 0 for 0 =
{(a, b) : aE RI " (OJ, be RI " {OJ} and the determinant of the Hessian is zero,
it fo]]ows that the Hessian is positive semidefinite over O. Thus, the function
f(a, b) = Ja 2 + j)2 is convex in o.

5.2.4 Cholesky Factorization

In closing this section we wi]] mention a simple but effective algorithm for testing
if a symmetric matrix Q is positive definite, positive semidefinite or indefinite.
This method is based on Cholesky factorization and it is briefly described in Gill,
Murray and Wright [73] but more thoroughly covered in Vickson [I89].
In a nutshe]], the Cholesky factorization of a positive definite or a positive
semidefinite matrix Q is Q = LL' where L is a real lower-triangular matrix. When
this factorization exists, Q is positive definite or positive semidefinite; but if L
fails to exist or if it contains some complex numbers then Q is indefinite. A sim-
ple recursive algorithm has been developed based on these arguments and Maple
has implemented a portion of this algorithm that works only on positive definite
matrices: If Q is positive definite, Maple indicates that this is the case by produc-
ing the L matrix; when Q is positive semidefinite or indefinite, Maple produces
an error message indicating that the matrix is not positive definite.
Example 6S Maple:SO implementation ofthe Cholesky factorization. A simple ex-
ample with the Hessians of positive definite, positive semidefinite and indefinite
quadratic forms f = x 2 + l , g = (x + y)2 and h = x 2 - l , respectively, is
given below.
> restart: # cholesky.mws
> with(linalg):
Warning, new definition for norm

Warning, new definition for trace

f:=x2+l
g:= (x + y)2
 5.2 Convexity of Sets and Functions 165

h :=x 2 - y2
> Hf:=hessian(f,[ x,y]); Hg:=hessian(g,[ x,y]);
Hh: =hessian (h, [ x, y] );

HI:= [~ ~]
Hg:= [ ; ; ]

Hh:= [~ _~]
> cholesky(Hf);

[ ../i 0 ]
o ../i
> cholesky(Hg);
Error, (in cholesky) matrix not positive definite

> cholesky(Hh);
Error, (in cholesky) matrix not positive definite
In the next example, Maple is able to compute the lower triangular matrix L of
a 3 x 3 symmetric matrix Q, which, by implication, is positive definite.

[=! -~ -n
> Q:=matrix([[ 2,-1,-3], [-1,2,4], [-3,4,9]]);

Q:=

> L:=cholesky(Q);

o
!J6
2
~J6
6
As a check of the results, when we ask Maple whether LL' is equal to the
original matrix Q, the answer is obtained in the affinnative.
> equal(L &* transpose(L), Q);
true
166 5. Nonlinear Programming

5.3 Unconstrained Optimization

In unconstrained optimization, the objective is to optimize the function f(x) with-
out any constraints on the decision vector x. In operations research applications,
such problems arise infrequently since in many business and industrial models the
decision variables are required to assume nonnegative values (e.g., the number of
units of production, order quantity, number of people to hire, amount of money
to invest). Additionally, these decision variables are usually constrained to take
values within some feasible region due to resource or budget limitations.
However, there are a few cases (such as the single-facility location problem to
be discussed) where the decision variables do not have to be constrained. More-
over, the optimality conditions for the constrained problems are natural extensions
of the conditions for unconstrained problems, which provides additionaljustifica-
tion for the study of unconstrained optimization.
We first give some definitions that will be useful in the analysis of general
optimization problems.

Definition 3 Let f(x) be afunetion defined on S eRn. A point XES is

(a) a global minimizer of f(x) on S if f(x) ~ f(x) for all XES,

(b) a local minimizer of f(x) if there exists an e > 0 such that f(x) ~ f(x)
for all XES for which x E Nc(x) = {y : lIy - xII ~ e}.

Note that replacing the inequality sign (~) by the strict inequality sign «)
gives the definitions for strict global minimizer and strict local minimizer.
The next two theorems, which give the necessary and sufficient conditions for
the optimality, are proved in Bazaraa and Shetty [18, Section 4.1].

Theorem 6 Suppose f(x) is differentiable at i. Jfx is a local minimum, then the

first order condition on the gradient) is

Vf(i) = (Of (x) , ... , of(X»)' = o.

OX) OXn

Also, if f(x) is twice differentiable and x is a local minimum, a second-order

condition is that the Hessian H(x) is positive semidefinite. •

This theorem provides the necessary conditions for the local optimality of i;
these conditions are true for every local optimum, but a point satisfYing these
conditions does not have to be a local minimum, the classical example being
f(x) = x3. The following theorem provides a sufficient condition for the lo-
cal minimum.

IThe points of f(x) obtained as a solution of Vf(i) = 0 are called the critical (or statioaary)
poiats.
 5.3 Unconstrained Optimization 167

Theorem 7 Suppose f(x) is twice differentiable at i. lfVf(i) = 0 and Hessian

H(i) is positive definite, then i is a local minimum. •

Let us now illustrate the necessary and sufficient conditions using a specific
function.

Example 66 Analysis of f(x) = (x - 4)4 (x + 3l The following steps in Maple

compute the derivative of this seventh-degree polynomial, compute its stationary
points, which are obtained from the necessary condition f' (x) = 0, and evaluate
the Hessian f" (x) at each stationary point.
> restart: * necsuf.mws
> f:=x-> (X-4)A4* (X+3)A3;
f := x -+ (x - 4)4 (x + 3)3
> solve(diff(f(x),x),x);
0, -3, -3, 4, 4, 4
> H:=diff(f(x),x$2);
H := 12 (x - 4)2 (x 4)3 + 6 (x - 4)4 (x
+ 3)3 + 24 (x + 3)2 (x - + 3)
> subs(x=O,H); subs(x=-3,H); subs(x=4,H);
-4032
o
o
The function is negative definite at x = 0, but it is negative semidefinite at x =
-3 and x = 4. Using Maple's is () function, we can check the characteristics
of these two points. We hypothesize that both of these stationary points are local
minima, which should imply that the value of the function at these points is lower
than the neighboring points, i.e., f(-3 - E) > f(-3) and f(-3) < f(-3 + E),
and f(4 - E) > f(4) and f(4) < f(4 + E).
> epsilon:=.Ol;
E:= .01
> is( f(-3-epsilon»f(-3) );
is( f(-3)<f(-3+epsilon) );
false
true
> is( f(4-epsilon»f(4) );
is( f(4)<f(4+epsilon) );
true
true
This demonstrates that x = -3 is neither a local minimum nor a local max-
imum and that x = 4 is a local minimum. See Figure 5.2 depicting the graph
168 5. Nonlinear Programming

of the function f(x) = (x - 4)4(x + 3)3 obtained using the Maple command
plot(f(x),x=-4 .. 5).

10000

8000

x
-2000

-4000

FIGURE 5.2. Plot of the function f(x) = (x - 4)4 (x + 3)3.

Remark 1 The results obtained for the stationary points x = -3 and x = 4

using the is () Junction of Maple could have been shown more easily using the
following result: Ifthe function ofa single variable f(x) and its first n derivatives
are continuous, then f(x) has a (local) maximum or minimum at i if and only
if n is even, where n is the order of the first nonvanishing derivative at i. The
function has a maximum or minimum at x according to whether f(n)(x) < 0
or > 0, respectively [77, p. 29}. Maple can conveniently compute the higher-
order derivatives and evaluate them at the stationary points -3 and 4. Thus, the
following commands and their output indicate that the former point is neither a
local maximum nor a local minimum since f(3) (- 3) > 0 and that the latter point
is a local minimum since f(4)(4) > O.
> subs(x=-3,diff(f(x),x$2));
subs(x=-3,diff(f(x),x$3));
subs(x=-3,diff(f(x),x$4));
o
14406
-32928
> subs(x=4,diff(f(x),x$2)); subs(x=4,diff(f(x),x$3));
subs(x=4,diff(f(x),x$4));
o
o
8232
> plot(f(x),x=-4 •• 6);
 5.3 Unconstrained Optimization 169

Example 67 Solution of the single facility location problem. Location theory

deals with finding the optimal geographic location of one or more sources (e.g.,
plants or factories) to meet the requirements of destinations (e.g., warehouses or
markets). Suppose that there are n destinations located at the coordinates (x;, y;),
i = I, ... , n, and we wish to locate optimally a source at a point with coordi-
nates (a, b). The Euclidean distance from the source to destination i is given as
d; = J (a - x;)2 + (b - y;)2. We assume that C; is the cost of shipping one unit
per kilometer (S/unitldistance) from the source to destination i and m; is the re-
quirement (in units) at destination i. Thus, the total cost of shipments as a fimction
of c;, m; and the distance is S(a, b) = L7=1 c;m;J(a - x;)2 + (b - y;)2, which
we wish to minimize. Unfortunately, due to the presence of the square root fimc-
tion, it is impossible to find a c1osed-fonn solution for this problem; see Love,
Morris and Wesolowsky [126, Chapter 2]. We demonstrate this using Maple.
>
>
restart: * locgen.rnws
S:=sum(c[ i) *n( i) *, sqrt' ((a-x{ i) ) "2+ (b-y[ i) )"2),
i=1..n);

S:= LC;
" m; J(a -x;)2 + (b - y;)2
;=1
> Sa:=diff(S,a);

Sa'- i: (! c; m; (2a - 2x;) \

.- ;=1 2 Ja 2 -2ax;+x;2+b2-2by;+y;2)
> Sb:=diff(S,b);

Sb'- i: (! C; m; (2b -2y;) \

.- ;=1 2 Ja 2 -2ax;+x;2+1J2-2by;+y;2)
> solve({ Sa,Sb},{ a,b});

Understandably, Maple finds no solution!

The decision variables a and b appear in every term both in the numerator and
inside the square root in the denominator of the necessary conditions as/aa ==
Sa = 0 and as/ab == Sb = 0, which makes it impossible to solve these equa-
tions analytically. But we can use Maple to easily obtain a numerical solution that
minimizes the total cost for this unconstrained optimization problem.
We know that the fimction S(a, b) must be convex, since we have shown that
the distance fimction J a2 + b2 is convex. As the sum of convex fimctions is also
convex, any numerical solution we find for the necessary conditions as/aa =
as/ab = 0 will be the global minimum for this problem.
Consider a problem with n = lO destinations and shipment costs c;, demands
m;, and the coordinates (x;, y;) of the destinations given as follows.
> restart: * location.rnws
> c: =( 21, 1, 3, 2, 2, 3, 4, 3, 2, 15] ;
170 5. Nonlinear Programming

c:= [21, 1,3,2,2,3,4,3,2, 15]

> m: =[ 40, 70, 56, 38, 59, 83, 38 , 57 , 86, 19] ;
m := [40, 70, 56, 38, 59, 83, 38, 57, 86, 19]
> x : =[ 1, 6, 9, 13, 8, 1 7 , 11, 6, 8, 9] ;
x:= [1,6,9,13,8,17,11,6,8,9]
> y : =[ 8, 9, 4 , 13, 8, 6, 1, 13, 9, 10] ;
y := [8, 9, 4, 13, 8, 6, 1, 13, 9, 10]
The objective function S is written conveniently with Maple's sum () function:
> S:=sum(c[ i]*rr( i]*sqrt((a-x[ i) ) "2+ (b-y[ i) )"2),
i=1..10);

S:= 840Ja2 - 2a + 65 + b2 - 16b + 70Ja2 - 12a + 117 +b2 - 18b

+ 168Ja2 -18a+97+b2 -8b
+ 76 J a2 - 26 a + 338 + b2 - 26 b
+ 118 J a2 - 16 a + 128 + b2 - 16 b
+ 249 Ja 2 -34a+325+b2 -12b
+ 152 J a2 - 22 a + 122 + b2 - 2 b
+ 171 J a2 - 12 a + 205 + b2 - 26 b
+ 172Ja2 - 16a + 145 + b2 - 18b
+ 285 J a2 - 18 a + 181 + b2 - 20 b
> Sa:=diff(S,a): Sb:=diff(S,b):
Next, we use Maple's floating point solver function fsol ve () to compute the
optimal solution.

> fsolve ({ Sa, Sb} ,{ a,b} );

{a = 6.966982752, b = 8.539483288}
> sol:=assign(%):
> subs(sol,S);
12216.13251

The optimal location of the source is thus found to be at the coordinates (a, b) =
(6.96,8.53) with a total minimum cost of $min = $12,216.13.

5.4 Inequality and Equality Constrained Optimization

We now extend the above results to the following general nonlinear programming
problem with inequality and equality constraints:
 5.4 Inequality and Equality Constrained Optimization 171

Problem P: min f(x)

s.t. g;(x) SO, i = I, . . . , m
h; (x) = 0, i = I, . .. , P
xe S.
With the inclusion of these constraints, the optimization problem becomes sub-
stantially more difficult to solve. Nevertheless, by using the theory developed by
H.W. Kuhn and A.W. Tucker [114], we can develop necessary (and sufficient)
conditions to solve these problems.
Theorem 8 KMhn- Tucker (K-T) Necessary Conditions For the Problem P let
f(x), g; (x), i= I, .. . ,m and h; (x), i =
I, ... , p be continuously differentiable.
Suppose that i is a feasible point that solves P locally and that the gradient
vectors Vg;(i)for i e B = (i : g;(i) = OJ and Vh;(i) are linearly independent.
Then there exist scalars A;, i = I, ... , m and p;, i = I, ... , p such that
m p
V f(i) + LA; Vg;(i) + LP; Vh;(i) = 0 (5.1)
;=\ ;=\
A;g;(i) = 0, i = I, ... , m (5.2)
Ai ~ 0, i = I, ... , m, (5 .3)

with the original structural constraints

g;(i) S 0, i=I, ... ,m, (5.4)

h;(i) = 0, i = I, . .. ,p. (5.5)

The scalars A;, i = I, .. . ,m and p;, i = I, ... , P are known as Lagrange

•
multipliers, and they have an important economic interpretation that will be dis-
cussed in Section 5.4.4 below. The conditions A;g;(i) = 0, for i = I, ... , mare
known as the complementary slackness conditions, stating that if the constraint
g; (i) is not binding, i.e., if g; (i) < 0, then A; = O. Similarly, if A; > 0, then
the constraint must be binding, i.e., g; (i) = O. Finally, note that there is no sign
restriction on the multipliers p ; that correspond to the equality constraints.
Theorem 8 provides necessary conditions for the optimal solution. For a dis-
cussion of the sufficient conditions that involve the forms of the ftmctions, i.e.,
pseudoconvexity of f and quasiconvexity of g; and h;, see Bazaraa and Shetty
[18, Section 4.3].
In general, the solution of the conditions is not a trivial matter since it involves
solving a system ofnonlinear equalities and inequalities. But Maple's sol ve ( )
ftmction can deal with such a system and compute the values of all the variables
exactly. We demonstrate this with the portfolio selection problem.
Example 68 Solution of the portfolio problem using Maple. Recall that in this
problem the objective was to minimize f(x\, X2) = 0.15xf + O.06x~ +0.14x\X2
172 5. Nonlinear Programming

subject to the constraints g(XI,X2) = 900 - O.lOXI - 0.05X2 ~ 0, h(XI,X2) =

lOOOO-xl -x2 =Oandxl ~ 0,X2 ~ O.
After inputting the data for the objective function and the constraints, we check
and see that f is strictly convex since its Hessian is positive definite. This assures
that the solution that will be found by Maple is the global optimum.
> restart: # portfoli.mws
> with (linalg) :
Warning, new definition for norm

Warning, new definition for trace

> f:=(15/100)*x[ 1] "2+(6/100)*x[ 2] "2
+(14/100)*x[ 1] *x[ 2] ;

. 3 2 3 2 7
f .= 20 XI + 50 X2 + 50 XI X2
> g:=900-(1/10)*x[l]-(5/100)*x[2]; # 9 <= 0
I I
g := 900 -10 XI - 20 X2
> h:=10000-x[ 1] -x[ 2]; # h = 0
h:= 10OOO-xl -X2
> vars: =[ x[ 1] ,x[ 2]] :
> H:=hessian(f,vars); definite(H,' positive_def' );

H:= [I: ~]
50 25
true
> grad_f:=grad(f,vars):
> grad_g:=grad(g,vars):
> grad_h:=grad(h,vars):
> eq[ 1] :=grad_f[ 1] +lambda*grad_g( 1] +mu*grad_h[ 1] ;
371
eql := 10 XI + 50 X2 -10 1 - p
> eq[ 2] :=grad_f[ 2] +lambda*grad_g( 2] +mu*grad_h[ 2] ;
371
eq2 := 25 X2 + 50 XI - 20 1 - p
> comp_slack:=lambda*g=O;
I I
comp_slack := 1 (900 - 10 XI - 20 X2) = 0
> structural[ 1] :=g<=O;
 5.4 Inequality and Equality Constrained Optimization 173

1 1
Structurall := 900 - -XI - -X2<0
10 20-
> structural[ 2] :=h=O;
structurah := 10000 - XI - X2 = 0
> solve({ eq 1] ,eq 2] ,cornp slack,larnbda>=O,
structural[ 1] , structuraI[ 2] ,
xi 1] >=0, xi 2] >=O} ) ;
{A. = 26400, P = 40, XI = 8000, X2 = 2000}
> sol:=assign(%):
> subs(sol,f);
12080000
The results indicate that it is optimal to invest XI = $8000 in the first stock and
X2 = $2000 in the second stock, giving rise to a portfolio variance of 12080000.

5.4.i Geometric interpretation ofKuhn-Tucker Conditions

There is an interesting geometric interpretation of the K-T conditions V/(i) +
L7'=1 A.; Vg;(i) + Lf=1 p; Vh;(i) = O. Defining B = (i : g;(i) = O} as the set
of binding inequality constraints and rewriting this condition, we have
p
LA.; Vg;(i) + LP; Vh;(i) = -V/(i).
;eB ;=1

In other words, the (negative) of the gradient vector V/(i) of the objective func-
tion / is a linear combination of the gradient vectors Vg; (i), i E B of the binding
inequality constraints and the gradient vectors Vh; (i) of the equality constraints.
(If any of the inequality constraints were not binding, then the corresponding mul-
tiplier would be zero due to the complementary slackness conditions.)
For the portfolio problem, at the optimal solution i = (8000, 2000) we have

Vg(i) = [ ~~, ;~ J, Vh(i) = [-1, -I]' and V/(i) = [2680, 13601'.

Thus, the values we have found for A. = 26400 and P = 40 will satisty

A. [ ~-I] + [ P
-1 ]
-1 = -
[ 2680 ]
1360 '

implying that i = (8000, 2000) is a Kuhn-Tucker point. Geometrically, then, the

vector -V /(i) is in the n-dimensional cone spanned by the vectors Vg(i) and
Vh(i).
174 5. Nonlinear Programming

5.4.2 Constraint Qualification

Are there nonlinear programming problems whose optimal solutions do not sat-
isfy the Kuhn-Tucker conditions? Consider the following problem from Kuhn and
Tucker [114].
min !(x,y) =-x
s.t. g\(x,y) = -(1 _x)3 + y ~ 0
K2(x,y) = -y ~ O.

As shown on the graph of the feasible region and the objective function in
Figure 5.3, the optimal solution for this problem is clearly at x = 1, y = O. But
let us attempt to solve this problem using Maple.
> restart: # CONSQUAL.MWS
> f:=-x; #Minimize
!:=-x
> gl:=-{1-x)A3+y; #gl <= 0
gJ := _(l-x)3 +y
> g2:=-y; #g2 <= 0
g2:= -y
> with{linalg):
Warning, new definition for norm

Warning, new definition for trace

> vars: ~ x, y] :
> grad_f:=grad{f,vars): #Gradients
> grad_g1:=grad{gl,vars):
> grad_g2:=grad{g2,vars):
> eq 1] : =grad f[ 1] + lambda[ 1] * grad gl[ 1]
+ lambda[ 2] * grad g2[ 1] : -
# Linear independence of gradient vectors
> eq[ 2] : =grad f[ 2] +lambda[ 1] * grad g2[ 1]
+lambda[ 2] *grad_g2[ 2] : -
> comp slack[ 1] : =lambda[ 1] * gl :
# Complementary slackness •..
> comp_slack[ 2] : =lambda[ 2] * g2:
> structural[ 1] : =gl<=O:
# Structural constraints .••
> structural[ 2] : =g2<=0:
> solve{{ eq 1] ,eet 2] ,comp slack[ 1] ,comp slack[ 2] ,
lambda[ 1] >=0, lambda[ 2] >~O -
} );
 5.4 Inequality and Equality Constrained Optimization 175

I I
p 2 = 0, y = 1 - 3 x + 3 x 2 -
o x 3 , AI = 3' (x _ 1)2' x = X)

-1

-2

FIGURE 5.3. Optimal solution is al i = (i,y) = (1,0) where constraint qualification is

not satisfied since al i the vectors Vgl (i) = [0, I)' (pointing up) and VK2(i) = [0, -It
(pointing down) are linearly dependent

As we see, Maple tries its best to find a solution and produces one. But due
to the unusual nature of this problem, the solution found by Maple (using Kuhn-
Tucker conditions) does not seem right, especially when we note that at x = I,
the multiplier AI = 00. What went wrong?
What happened in this problem was the lack of linear independence of the
gradient vectors Vg; (i), i = I, 2 at the point i = (I, 0). Since

Vgl(i) = [0, 1]', V82(i) = [0, -I)' and Vf(i) = [-1,0)'.

the system

AI [ ~ ] + A2 [ ~I ] = _ [ ~I ]

* This
has no solution, as can be seen from the following Maple output:
> sol:~x=l,y=O}; is known to be the optimal
solution
sol := {y = 0, x = I}
> grad_gl:=subs(sol,evalrn(grad_gl»;
grad...x1 := [0, I]
> grad_g2:=subs(sol,evalrn(grad_g2»;
grad...x2 := [0, -I]
> grad_f:=subs(sol,evalrn(grad_f»;
gradJ:= [-1,0]
> vector_space:=stackrnatrix(grad_gl,grad_g2)i
176 5. Nonlinear Programming

vector_space:= [~ _!]
> linsolve(transpose(vector_space),-grad_f)i
Maple finds no solution!
The fact that the vectors [0, I]' and [0, -I]' are linearly dependent is shown
in the next set of commands where Maple indicates that the matrix fonned by
these vectors has rank 1 and the solution of the linear system for AI, and A2 gives
AI = A2 = _tl:
> rank(vector_space)i

> linsolve (transpose (vector_space) ,[ 0,0] ) i

Ltl, _t~
Thus, at the optimal solution i = (I, 0), the linear independence eonsmint
qualifieation is not satisfied, which is the reason for the failure of the Kuhn-
Tucker conditions at this optimal point.
The graph of the feasible set in Figure 5.3 can be obtained with the following
Maple commands.
> with (plots) : with(plottools):
> Delg1:=arrow([ 1,0], [1,1], .01, .05, .2,
color=black) :
> Delg2:=arrow([ 1,0] ,[ 1,-1] ,.01, .05, .2,
color=black) :
> Delf:= arrow([ 1,0], [0,0], .03, .15, .2,
color=black) :
> fr:=textplot ([ -1,1/2, 'Feasible Region '] ,
align={ ABOVE, RIGHT} ) :
> glplot:=irnplicitplot(gl,x=-2 •• 3,y=-2 •• 3,
color=black,nurnpoints=5000,th
ickness=2) :
> g2plot:=irnplicitplot(g2,x=-1 •• 2,y=-1 .• 2,
color=black,nurnpoints=5000):
> display({ fr,glplot,g2plot, Delg1,
Delg2, Delt) ) i

5.4.3 An Equivalent Formulation Using the Lagrangian

It would be convenient to introduce the Lagrangian function £ associated with
the original constrained problem as
m p
£(x, ~,I') = /(x) + LA;g;(x) + LJl;h;(x)
;=1 ;=1
 5.4 Inequality and Equality Constrained Optimization 177

where..\ = (AI, ... , Am) and I' = (JIb . .. , Jlp). The K-T conditions can now be
expressed in tenns of the Lagrangian as

aL(x, ..\, 1')

aXj
l aL(x,..\, 1') =
I all
llgl(x) = 0, i = I, ... , m
II ~ O,i=l, ... ,m
aL(x, ..\, 1')
= g; (x) ~ 0, i = I, . .. , m
all
aL(x,..\,I')
= h;(x) = 0, i = 1, ... , p.
aJlI

Using this equivalent fonnulation, we now present another constrained mini-

mization example that can be solved more conveniently with Maple.
Example 69 A problem with a quadratic objective function and mixed constraints.
Suppose we wish to minimize f(xl, X2) = (XI - 3)2 + (X2 - 2)2, subject to two
mixed (i.e., nonlinear and linear) inequality COnstraintsgl(XI, X2) = xf+xi-5 :s
0, and g2(XI, X2) = XI + 2x2 - 4 :s O. We enter the problem data as
restart: *
KTMIN.MWS
* Minimize
>
> f:=(x[l]-3)"2+(x[2]-2)"2;
/ := (XI - 3)2 + (X2 - 2)2
> g( 1] :=x[ 1] "2+x[ 2] "2-5; * g( 1] <= 0

gl := XI 2 +X2 2 - 5
> g( 2] : =x[ 1] +2* x[ 2] -4; .. g( 2] <= 0
g2 := XI + 2 X2 - 4
> vars:=[x[l],x[2]];
vars := [XI, X2]
and generate the K-T conditions with the following commands:
> L:=f+sum(lambda[ k] *g( k] , k=1. .2);

L := (XI - 3)2 + (X2 - 2)2 + II (X1 2 +X22 - 5) + l2 (XI + 2X2 - 4)

> L_x:=seq(diff(L,x[ k] ),k=1..2);
L_x:= 2xI - 6 +2ll XI + l2, 2X2 - 4 +2ll X2 + 212
> eq: =seq (lambda[ k] * diff (L, lambda[ k] ), k=1. .2) ;

eq := II (X1 2 +X2 2 - 5), l2 (XI +2X2 -4)

> ineq_lambda:=seq(lambda[ ~>=O,k=1 •• 2);
ineq_lambda := 0 :s AI, 0 :s l2
178 5. Nonlinear Programming

> ineq_g:=seq(diff(L,lambd~ ~ )<=O,k=1 .. 2);

ineq...E := XI 2 + X2 2 - 5 ~ 0, XI + 2 X2 - 4~0
> L_x, ineq_g;

2xI - 6 + 2 AI XI + A2, 2X2 - 4 + 2 AI X2 + 2 A2, XI 2 + X2 2 - 5 ~ 0,

XI +2X2 -4 ~ 0
Next, the solve () command is used to find the solution to the K-T condi-
tions:
> sol:=solve({L x,eq,ineq g,
r
ineq_lambda} , x[ 1] , x[ 2] ~lambda[ 1] , lambd~ 2] } ) ;
I 2
sol:= {X2 = 1, XI = 2, AI = 3' A2 = 3}
At the optimal solution, the objective function takes the minimum value f = 2:
> evalf(subs(sol,f));
2.
The following commands are used to show that the constraint qualification is
satisfied at the optimal solution:
> with(linalg):
Warning, new definition for norm

Warning, new definition for trace

> grad_f:=grad(f,vars);
gradJ:= (2xl - 6, 2X2 - 4J
> grad_g[ 1] : =grad (g[ 1] , vars) ;
grad...E1 := (2xl, 2xiJ
> grad_g[ 2] : =grad (g[ 2] , vars) ;
grad...E2 := [1, 2]
> Opt:={ x[ 1] =2,x[ 2] =1} ;
Opt := {X2 = I, XI = 2}
> grad_f:=subs(Opt,evalm(grad_f));
gradJ := [-2, -2]
> grad_g[ 1] :=subs(Opt,evalm(grad_g[ 1] ));
grad...E1 := [4, 2]
> grad_g[ 2] :=subs(Opt,evalm(grad_g[ 2] ));
grad...E2 := [1, 2]
> A:=transpose(matrix(2,2,[ grad_g[ 1] ,grad_g[ 2]] ));
 5.4 Inequality and Equality Constrained Optimization 179

A:= [i ~]
> linsolve(A,-grad_f);

[i, ~]
The final result indicates that the gradient of the objective ftmction can be ex-
pressed as a linear combination of the gradients of the constraints evaluated at
i = (2, I) when ~ = (.l, j). The graph of the feasible set, the objective function
and the optimal solution presented in Figure 5.4 are plotted using the next set of
Maple commands.
> with(plots):
> f: =subs ({ x[ 1] =x, x[ 2] =y} , f) ;
f := (x - 3)2 + (y - 2)2
> g1: =subs ({ xC 1] =x, x[ 2] =y} , g[ 1] ) ;
gJ:=x 2 +r-5
> g2: =subs ({ x[ 1] =x, x[ 2] =y} , g[ 2] ) ;
g2 :=X +2y-4
> solve({g1,g2});
II -2
(y = I, x = 2), (y = 5' x = 5)
> feas_g1:=implicitplot(g1,x=2 •• 3,y=-1 •• 1):
> feas_g2:=implicitplot(g2,x=-1 •• 2,y=1 •. 3):
> obj:=contourplot(f,x=-2 .• 6,y=-2 •• 3,contours=
[0.5,1,1.5,2,2.5] ,coloring=[ white,blue]):
> display({ feas g1,feas g2,obj},
scaling=constrained);-

Example 70 An application in microeconomic theory: The Cobb-Douglas pro-

duction junction. Consider a simple production ftmction that is defined as q =
f(x), X2) where X) > 0 and X2 > 0 are the variable inputs (e.g., labor and capital)
and q is the quantity of the output. A special class of these functions known as the
Cobb-Douglas (CD) production ftmction [48] is defined as q = Ax~\ x~ where
a) > 0 and 02 > O. The CD function has an interesting homogeneity property: If
the inputs are each increased by a factor of (J, we have q8 = A «(JX) )Q\ «(JX2)t12 =
A(J(Q\+Q2)X~\ X~2. Hence, if a) + 02 = I, the function is linear homogeneous and
we have constant returns to scale; if a) + 02 > I « I) we have increasing (de-
creasing) returns to scale [87], [94].
In this example, we discuss an optimization problem that involves the CD pro-
duction function. Suppose that the desired output is specified as q and that each
unit of input X) costs $c) and each unit of input X2 costs Sc2. Thus, we wish
180 5. Nonlinear Programming

FIGURE 5.4. The feasible set, objective function and the optimal solution of the problem
with a quadratic objective function and mixed constraints.

to minimize the total cost C = CIXI + C2X2 of producing q units subjcct to the
constraint that Axfl x~ = q.
We first fonn the Lagrangian as L = CIXI + C2X2 + Jl(q - Axfl x~). Using
Maple's implicit differentiation function, we observe that the curve q = Axfl x~
in the (XI, X2) plane is decreasing and strictly convex. Since the cost function
C = CIXI +C2X2 is linear, this implies that the solution found from the application
of the Kuhn-Tucker theorem will give the unique minimizing point.
>
>
restart: * cobbdouglas.rnws
L: =c[ 1] * x[ 1] +c[ 2] * x[ 2]
+rnu* (q-A*x[ 1] "a( 1] *x[ 2] "a( 2] );
L := CI XI + C2 X2 + Jl (q - A XI Q1 X2D2)
The first-order implicit derivative is obtained as a negative quantity implying
that X2 is decreasing in XI,
> irnplicitdiff (q-A* x[ 1] "a( 1] * x[ 2] "a( 2] ,
x[ 1] ,x[ 2] );
02 XI
°IX2
whereas the second order implicit derivative is always positive implying that X2 is
a strictly convex function of XI.
> irnplici tdiff (q-A* x[ 1] "a( 1] * x[ 2] "a( 2] ,
x[ 1] ,x[ 2] $2);
02 XI (02 + 01)
012 x2 2
> e1:=diff(L,x[ 1] )=0;
Jl A XI Q1 01 X2 D2
el :=CI - =0
XI
 5.4 Inequality and Equality Constrained Optimization 181

> e2:=diff(L,x[ 2] )=0;

J.l AXIal X2 t12 tl2
e2 :=C2 - =0
X2
> e3:=diff(L,mu)=0;
e3:= q - =0
AXIal X2 t12

The necessary conditions give rise to a system of three nonlinear equations in

three unknowns, which is solved easily by Maple's sol ve () command.
> Sol:=solve({ e1,e2,e3},{ x[ 1],x[ 2] ,mu});

(_-ID(1)':;~(iffi))1
X2 =e

Expanding to clear the logarithms and simplifying gives the analytic solution
for the optimal values of the inputs XI and X2 and the Lagrange multiplier J.l .
> expand(simplify(Sol,power,symbolic));

5.4.4 Economic Interpretation ofthe Lagrange Multipliers

So far we have treated the Lagrange multipliers as a purely algebraic device to
convert a constrained problem into an unconstrained one. The solution of the
problem assigned a nonnegative value to these multipliers, but we did not pro-
vide a discussion of the economic interpretation of Lagrange multipliers.
In the portfolio selection problem of Example 68 we had found that 1 = 26400
and J.l = 40, and in Example 69 with a quadratic objective function with mixed
constraints we had 11 = 1/3 and 12 = 2/3. One may justifiably ask whether these
values have an economic meaning related to the constraints they are attached to.
The answer, as in linear programming, is in the affirmative and the Lagrange
multipliers provide information on the approximate improvement in the objective
function if a particular constraint's RHS constant is increased by a small amount.
182 5. Nonlinear Programming

If, for example, in the portfolio problem we were to increase the RHS of the
first constraint by one unit, i.e., 900 - 0.10xl - 0.05X2 ::: lor, equivalently,
gl (XI, X2) = 899 - 0.1 OXI - 0.05X2 ::: 0, and solve the problem again, we would
obtain the optimal solution as XI = 7980, X2 = 2020, and f = 12053628,
a reduction (improvement) in the objective function (i.e., portfolio variance) of
12,080,000 - 12,053, 628 = 26,372 units. This difference is very close to the
value of the Lagrange mUltiplier A. = 26400 found as the solution of the original
problem.
We formalize this important result in the following theorem.
Theorem 9 [127, p. 317] Consider the problem
Problem P: min f(x)
s.t. g;(x):::a;,i=I, ... ,m
h; (x) = b;, i = 1, ... , p.
Suppose that for a = (al, ... , am) = 0, b = (b), ... , bp) = 0 there is a so-
lution i that satisfies the constraint qualification with the Lagrange multipliers
A.; ~ 0, i = 1, ... , m, and p;, i =1, ... , p. Thenfor every (a, b) E Rm+p in
the neighborhood of (0, 0) there is a solution x(a, b) depending continuously on
(a, b) such that x(O, 0) = i and such that x(a, b) is a relative minimum of the
problem P. Moreover,

af (x(a,
aa;
b»] 0,0
= -A.;, i = 1, ... ,m
: ( (x(a, b» 1,0 = -p;, i = 1, ... , p.

Thus, in Example 69 (ktmin • mws), if the first constraint is written as xf+x~-

•
5 ::: 0.1 or equivalently as gl (XI, X2) = xf + x~ - 5.1 ::: 0, the value l1al x A.I =
(0.1) x 1/3 = 1/30 should be the approximate improvement (i.e., reduction) in
the optimal value of the objective function f(XI, X2). This increase in the RHS of
the first constraint by 0.1 units enlarges the feasible set and should improve the
objective function by approximately 0.033 units as we see in the following Maple
output. (The actual improvement is equal to 2 - 1.968468797 = 0.031531203
units.)
*
*
> restart: KTMin2.mws
> f:=(x[ 1] -3)"2+(x[ 2] -2)"2: Minimize
> g[ 1] : =x[ 1] "2+x[ 2] "2-5.1; * g[ 1] <= 0

gl := XI 2 +X2 2 - 5.1
> g[2] :=x[l]+2*x[2]-4: * g[2] <= 0
> vars:={ x[ 1] ,x[ 2]] :
> L:=f+surn(larnbda{ k] *g[ k] ,k=1. .2):
> L_x:=seq(diff(L,x[ k] ),k=1..2):
 5.5 Lagrangian Duality 183

> eq: =seq (lambda( k] * diff (L, lambda( k] ), k=1. .2) :

> ineq_Iambda:=seq(lambda( ~>=0,k=1 •. 2):
> ineq_g:=seq(diff (L, lambda( ~ )<=0,k=1 •• 2):
> L x, ineq g:
> sol:=solve({L x,eq,ineq g,
ineq_Iambda} ,{x[ 1] , x[ 2] ;-lambda( 1] , lambda( 2] } ) ;

sol:= {AI = .2977713690, X2 = .9835585997, A2 = .7235658095,

XI = 2.032882801}
> evalf(subs(sol,f»;
1.968468797

5.5 Lagrangian Duality

In Section 5.4.3 we presented the Lagrangian as
m p
L(x,.\, p.) = f(x) + LAIgI(X) + LIl1h/(x)
1=1 1=1

without providing any geometric interpretation for this function. In this section we
show that the Lagrangian has the important geometric property that it has a saddle
point at the optimal solution (i, ..x, j.&). To motivate the discussion, we first present
a very simple example and plot the surface of L to observe its three-dimensional
shape.
Consider the problem of minimizing f(x) = (x - 2)2 + I subject to g(x) =
x - I !S 0 for which the obvious optimal solution is i = 1. The Lagrangian
is formed as L(x, A) = (x - 2)2 + 1 + A(X - 1), which is a function of two
variables (x, A). Using Maple's plot3d () function, we plot the surface of the
Lagrangian for the range x=- 2 .. 3 and lambda =0 •. 4 and obtain the graph of
the Lagrangian depicted in Figure 5.5.2
> restart: *
saddle.rnws
> L:=(x-2)A2+1+lambda* (x-1);
L := (x - 2i + 1 + A (x - 1)
> plot3d(L,x=-2 .. 3,lambda=0 .. 4,axes=boxed,
shading=none,orientation~ -152,70],
labels=[ "", "" , ""] ) ;
This graph suggests that the Lagrangian appears to have a saddle point at
around (x, A) = (1,2), i.e., that L(1,A) !S L(1,2) !S L(x,2). Indeed, when

2Maple cannot use different fonts (such as the Roman x and Greek 1) to display the labels in the
same graph. For this reason, the axis labels in Figure S.S are suppressed. Note, however, that x takes
values between -2 and 3 and .4 between 0 and 4.
184 5. Nonlinear Programming

16r---r::::.____ _
14
12
10
8
6
4
2

FIGURE 5.5. The Lagrangian function L = (x - 2)2 + I + l(x - I) has a saddle point at
(x, 1) = (1,2).

we apply the eigenvalue test to the Hessian of L, we obtain one positive and one
negative eigenvalue, implying that the Lagrangian does have a saddle point:
> with (linalg) :
Warning, new definition for norm

Warning, new definition for trace

> H:=hessian(L,[ x, lambda] );

H:= [i ~]
> eigenvalues(H);
I +.J2, I - .J2

We should now be able to compute the saddle point of the Lagrangian, i.e., the
x value that minimizes L and the 1 value that maximizes L. We do this using the
following Maple commands:
> restart: # SADDUAL.MWS
> f:=(x-2)A2+1; # Minimize
f := (x - 2)2 +1
> g:=x-l; # 9 <= 0
g :=x-l
> L:=f+lambda*g;
 5.5 Lagrangian Duality 185

> diff (L, x) i

2x -4+A.
> xsol:=solve(%,x)i
1
xsol := 2 - - A.
2
> subs(x=xsol,L);

!4 A.2 + 1 + A. (l - !2 A.)
> diff(%,larnbda);

1 -!2 A.
> lambdasol:=solve(%,lambda);
lambdosol := 2
> xsol:=subs(lambda=larnbdasol,xsol);
xsol:= 1
These computations show that L(l, A.) ~ L(l, 2) ~ L(x, 2).
To compare the results obtained by our graph based on heuristic arguments, we
re-solve the same problem using Kuhn-Tucker conditions (5.1 )-{5.5):
restart: * SADDLEKT.MWS
*
>
> f:=(x-2)A2+1i Minimize
f := (x - 2)2 + 1
> g:=x-l; * g <= 0
g :=x-l
> L:=f+lambda*gi
L := (x - 2)2 + I + A. (x - I)
> L_x:=diff(L,x)i
L_x := 2x - 4 + A.
> eq:=lambda*diff(L,larnbda)i
eq := A. (x - l)
> ineq_lambda:=lambda>=Oi
ineq_lambda:= 0 ~ A.
> ineq_g:=diff(L,lambda)<=Oi
ineq..s := x - I ~ 0
> sol:=solve({L x,eq,ineq g, ineq lambda},
{ x, lambda} ) i - - -

sol := (A. = 2, x = I)
186 5. Nonlinear Programming

Interestingly, for this problem the solution obtained using the saddle-point anal-
ysis of the Lagrangian and the Kuhn-Tucker conditions agree. This is not a coinci-
dence and when the objective function and the constraint satisfy certain regularity
conditions (e.g., convexity of the functions involved), there is a valid intimate re-
lationship between the saddle-point criteria and the K-T conditions. This is stated
in the following theorem:
Theorem 10 [18, Section 6.2) The feasible point ifor Problem P and the La-
grange multipliers .oX ::: 0 and p satisfy the Kuhn-Tucker conditions (5.1H5.5), if
and only if(i,.oX, p) satisfy the saddle-point conditions L(i,..\, 1') ~ L(i,.oX, p)
~ L(x, .oX, p). • .

In the simple problem of minimizing f(x) = (x - 2f + 1 subject to g(x) =

x-I ~ 0, we used Maple to differentiate L(x, A.) with respect to x for any
given value of A.. This gave x(A.) = 2 - A./2 as the optimal value of x in tenns
of A.. When we substituted x (A.) into the Lagrangian we obtained a new function
l(A.) = A.2/4 + 1 + A.(1 - A./2). Maximizing l with respect to A. ::: 0 gave A. = 2
which, when substituted back into x(l), produced the optimal value of x = 1.
Thus, at the optimal values found by this approach, we obtained f(x = 1) = 2 =
l(l = 2), i.e., minf = maxl.
Although the approach we used to find the optimal solution via a derived func-
tion such as l(l) may seem arbitrary, it has wide-ranging implications in theo-
retical and computational aspects of nonlinear programming: The function lO
is known as the Lagrangian dual, and its maximum and the minimum of the
original (primal) problem's objective function ffs.) are intimately related.
We define the Lagrangian dual D problem as
max l(..\,I') = min [I(x) + L~=I A.igi(l.) + Lf=1 Pihi(l.) : I. E S]
s.t. ..\ ::: o.
The relationship between the values of the functions f(x) and l("\, 1') ofProb-
lems P (the primal problem) and D (the dual problem) is given in the following
theorem.
Theorem 11 Duality 1'IIeorem [18, Section 6.2) Under certain regularity condi-
tions (i.e., convexity of the functions f and gi, linearity ofhi in terms of ~ and a
constraint qualification), the follOWing result holds:

min [1(1.) : XES, gi(X) ~ 0, i = 1, ... , m, hi(x), i = 1, ... , p]

= max [l("\, 1') : ..\ ::: 0]

Moreover, the function l("\, 1') is concave over any convex subset of its domain.

This theorem can be very useful in solving constrained nonlinear programming

•
problems where the decision vector x can be expressed in terms of (..\, 1') after
minimizing the Lagrangian L(x, ..\, 1'). If this is possible, then x = x(..\, 1') can be
 5.5 Lagrangian Duality 187

substituted back into L to obtain the Lagrangian dual function l()', 1'), which is
shown to be concave; hence its global maximum can be found using the standard
methods. Finally, once the optimal (X, ji.) are computed, they can be substituted
back into x to generate the optimal value of the decision vector.
As an application of the Lagrangian duality, we now discuss the solution of
a discrete-time optimal control problem with quadratic objective and linear state
dynamics.
Example 71 A discrete linear quadratic control problem. The problem is to min-
imize the objective
N-I
J = L(q;x; +r;u~) +qNx~
;=0
subject to the state dynamics

X;+) = a;x; + b;u;, i = 0, ... , N - 1

where xo, {q; }~, {r; }~I, {a; }~I, {b; }~I are given constants. To solve this
control problem as a nonlinear programming problem, we treat all of the state
variables X; and control variables U; as decision variables, i.e., we define the de-
cision vector as X = (XI, ... , XN, uo, •. . ,UN-I).
The Maple commands to solve this problem with N = 2 are as follows.
> restart: * LQ.MWS
The next set of commands defines the objective and the state dynamics.
> for i from 0 to 1 do c{ i) :=q[ i) *x[ i) "2
+r[ i) * u[i) "2 od;

Co := qO x02 + ro u02
CI := ql XI 2 + rl Ul 2
> f:=sum(c{ n] ,n=O •. l)+q[ 2] *x[ 2] "2;

f:= qox02 +rou02 +ql XI 2 +rl Ul 2 + q2 x 22

> for j from 1 to 2 do h[ j] :=x[ j] -x[ j-l] -u[ j-l] od;
hI := XI - xo - uo

h2 := X2 - XI - UI
The Lagrangian L is introduced as
> L:=f+sum(mu[ n] *h[ n] ,n=1. .2);

L := qox02 + ro u0 2 + ql XI 2 + rl Ul 2 + q2 X2 2 + #1 (xl - xo - uo)

+ #2 (X2 - XI - UI)
and differentiated with respect to all decision variables in the decision vector x.
> Lu[ 0] : =diff (L, u[ 0] ) ;
Luo := 2ro uo - #1
188 5. Nonlinear Programming

> Lx[ 1] : =diff (L, x[ 1] ) ;

lxI := 2 ql XI + PI - P2
> Lu[ 1] : =diff (L, u[ 1] ) ;
LUI := 2rl UI - P2
> Lx[ 2] : =di ff (L, x[ 2] ) ;
Lx2 := 2q2 x2 + P2
The decision variables are computed in tenns of the two Lagrange multipliers
Pi'; = 1,2.
> s01xu: =s01 ve ({ Lx[ 1] ,Lx[ 2] ,Lu[ 0] ,Lu[ 1] } ,
{ u[ 0] ,x[ 1] ,u[ 1] ,x[ 2] } ) ;

IP2 IPI-P2 Ipl IP2

solxu := {UI = - - , XI = -- , Uo = - - , X2 = -- - }
2 n 2 ql 2 TO 2 q2
> assign(s01xu);
The Lagrangian duall(PI, P2) is computed and maximized after substituting
x{J.ll, P2) in L.
> e1:=subs(s01xu,L);

1,,2 I ( )2 1 2 1,,2
el:=qox02+-_rl_+- PI-P2 +_P2 + __r2_
4 ro 4 ql 4 TI 4 q2
1 P2
+pd--1 PI - P2 1 PI
-xo---)+P2(---+-
1 P2 1 PI - P2
---)
2 ql 2 TO 2 q2 2 ql 2 TI
> e1mu[ 1] :=diff(e1,mu[ 1] );

1 1 1 1 1 P2
eimul :=-XO+PI(------)+--
2 ql 2 TO 2 ql
> e1mu[ 2] : =di ff (e1 ,mu[ 2] ) ;

1 PIli 1 1 1 1
elmU2:= - - +P2(--- - - - - - - )
2 ql 2 q2 2 ql 2 TI
The Lagrange multipliers are found in tenns of the model parameters as fol-
lows.
> s01mu: =s01 ve ({ e1mu[ 1] ,e1mu[ 2] } ,{ mu[ 1] ,mu[ 2] } ) ;

.-{ - 2 XOTO(qITI+q2 TI+q2ql)

sol mu .- PI - - ,
ql TI + q2 TI + TI TO + q2 TO + q2 ql
XOToq2TI }
P2 = - 2
ql n + q2 n + TI TO + q2 ro + q2 ql
> assign(s01mu};
Finally, the optimal solution for the complete problem, i.e., the optimal values
ofuo, XI, UI andx2, are computed in tenns of model parameters.
> uopt[ 0] :=simplify(u[ 0] };
 5.5 Lagrangian Duality 189

uOpto :=
ql rl +q2 rl +rl rO +q2 rO+q2ql
> xOpt[ 1] :=simplify(x[ 1] );
xoro (rJ + q2)
XQ'Ptl := ----;.....;......:.....;-...:.;;.;----
ql rl +q2rl +rl ro +q2 rO+q2ql
> uOpt[ 1] :=simplify(u[ 1] );
xOrOq2
uQ'Ptl :=
ql rl +q2 +rl ro +q2 rO+q2ql
rl
> xOpt[ 2] : =simplify (x[ 2] );
XOrOrl
X Opt2:= -----..;,..,..;;.....;.-----
ql rl +q2 rl +rl ro +q2 rO+q2ql
Although the solution obtained is of an "open-loop" variety, a careful look at
the structure of the optimal decision vector reveals that the optimal controls U; are
linear functions of the optimal states X; at each time; i.e., in this problem we also
have a closed-loop feedback control.
> uOpt[ 0] Ix[ 0] ;

ql rl + q2 rl + rl ro + q2 ro + q2 ql
> uOpt[ 1] IxOpt[ 1] ;

-~
rl +q2
For another application of this duality theorem, consider a discrete-time vector
optimal control problem with piecewise-quadratic objective function containing
dead-zones, i.e.,
N-I
minJ = QN(XN, iN) + L[Q;(X;, i;) + (u; - u;)' R;(u; - u;)]
;=0

subject to
X;+I = A; x; +B;u; +e;,i =O, . .. ,N-I
where
Qij[x;j - (x;j - aij)f, ifxij::; X;j - a;j
Q;j (xij, X;j) = { 0, ifx;j E [xij - aij, xij + Pij]
Qt[x;j - (x;j + p;j)]2, ifx;j ~ X;j + P;j .

In this multidimensional optimal control problem X; are n x 1 state vectors, U; are

e;
m x 1 control vectors, are n x 1 "disturbance" vectors, A; are n x n matrices,
B; are n x m matrices, R; are positive-definite m x m matrices, Qij, Qt are
strictly positive scalars, and a;j and P;j are nonnegative scalars. This nonstandard
control problem with the piecewise-quadratic cost that adds increased generality
to the well-known linear quadratic control model has been solved as a nonlinear
190 5. Nonlinear Programming

programming problem using the Lagrangian duality. For details of the solution
method and numerical examples with N = 25 periods, we refer the reader to the
paper by Parlar and Vickson [146].

5.6 Summary
Many realistic decision problems are nonlinear in the sense that the objective
function and/or some of the constraints may not be linear. In those cases one
would need to use nonlinear programming (NLP) techniques to model and solve
the problem. This chapter started with an introduction of the fundamental con-
cepts ofNLP such as the convexity of sets and functions. This was followed by a
discussion of unconstrained nonlinear optimization. Next, the important problem
of optimizing a nonlinear objective function subject to nonlinear constraint(s) was
presented. Kuhn-Tucker conditions, constraint qualification and the economic in-
terpretation of the Lagrange multipliers were described with several examples.
Once more, Maple's ability to manipulate symbolic quantities was found useful
in presenting and solving relevant problems.

5.7 Exercises
1. Let the functions f; (x), ; = 1, ... , k be convex over some convex set S c
Rn. Prove that the function f(x) = L:=t f;(x) is also convex over S.

2. Consider the function f(x, y) = xy. Show that f is not convex over R2
but it is convex over a line passing through the origin.
3. The average number of backorders function in the continuous-review (Q, r)
inventory model is given by

B(Q,r) = ~l°O(X -r)k(x)dx.

Q r

Show that if ~ is a number such that k(x) is nonincreasing for x 2: ~, then

B(Q, r) is convex in the region {(Q, r) : 0 < Q < 00 and ~ ~ r < oo}.

4. Optimize the function f(x) = 1 - e- x2 •

5. Consider the function f(x, y) = x 3 + y3 + 2x2 + 4r. Find the stationary

points of this function and identify them (as local minimum, local maxi-
mum or saddle point).
6. Maximize the function f(x, y, z) = xyz subject to the constraint (x/a)2 +
(y / b)2 + (z / c)2 = 1 where a, b and c are positive constants. Can you find
a geometric interpretation for this problem?
 5.7 Exercises 191

7. Consider the objective function f (x, y) = x 2 +4y3. Find all the points that
are local maxima and local minima for f subject to the equality constraint
x 2 +2r = 1.
8. Is the constraint qualification satisfied for the problem of minimizing the
r
distance from the origin to the implicitly defined curve = (x - 1)31
9. Consider the following problem.

min
s.t. -x +y = I
!
x+y ~ 1
x,y ~ 0
(a) Solve this problem graphically.
(b) Check your answer using the Kuhn-Tucker theory.
(c) Find the dual function t(>",,.,,) and show that it is concave.

1O. Minimize
N
J = L(qx; + ruf) + qxj
;=0
subject to the state dynamics

Xi+1 = ax; + bu;, ; = 0, ... , N - 1

where N = 2, q = 2, r = 4, a = 1 and b = 0.5.

6
Dynamic Programming

"Remember, today is the first day

ofthe rest ofyour life·"
Popular mantra-circa 1970

6.1 Introduction
Dynamic programming (DP) is a simple yet powerful approach for solving certain
types of sequential optimization problems. Most real-life decision problems are
sequential (dynamic) in nature since a decision made now usually affects future
outcomes and payoffs. An important aspect of optimal sequential decisions is the
desire to balance present costs with the future costs. A decision made now that
minimizes the current cost only without taking into account the future costs may
not necessarily be the optimal decision for the complete multiperiod problem.
For example, in an inventory control problem it may be optimal to order more
than the current period's demand and incur high inventory carrying costs now in
order to lower the costs of potential shortages that may arise in the future. Thus,
in sequential problems it may be optimal to have some "short-term pain" for the
prospect of "long-term gain."
Dynamic programming is based on relatively few concepts. The state variables
of a dynamic process completely specifY the process and provide information
on all that needs to be known in order to make a decision. For example, in an
inventory control problem the state variable X, may be the inventory level of the
product at the start of period t. Additionally, if the supplier of the product is not

M. Parlar, Interactive Operations Research with Maple

© Birkhäuser Boston 2000
194 6. Dynamic Programming

always available, then the supplier's availability status may also be another state
variable describing the inventory system.
In the inventory control example, ordering decisions are made at certain times.
Such times are called stages and they may be discrete or continuous. If the inven-
tory is managed using a periodic-review system where decisions are made, say,
once every week, then the stage variable would be discrete and it may be sequen-
tially numbered by the integers t = 0, I, 2, .... If, however, a continuous-review
(transactions reporting) system is in place, then the order decisions may be made
at any time and the stage variable would be continuous.
Having defined the state and stage variables, we now introduce the concept of
a decision, which is an opportunity to change the state variable. For example, if
the inventory level at time t is x, units, then ordering u, units is a decision that
increases the current inventory to x, + u, units. The decision to order u, units
in period t results in a cost in the current period and depending on the demand
w" period t may end with negative or positive inventory resulting in shortages
or surpluses that may give rise to additional costs for the current period t. If the
process continues for, say, N periods into the future, the optimization problem
is to find the best order policy that will minimize the total cost of the inven-
to~ ~stem. In general, the total cost incurred over N periods can be written as
1:,::0 L,(x" u" w,) + LN(XN) where L,(·) is the cost incurred in period t.
The processes that are studied by dynamic programming pass from stage to
stage and thus the states undergo a transformation represented by the equation
Xt+1 = f(x" u" w,). In the inventory example, after a demand realization of w,
units in period t, the next period's state is computed from Xt+1 = x, + u, - w,
where the transformation function takes the form f(x" u" w,) = x, + u, - w,.
Depending on the nature of the demand w" the transformation may be determinis-
tic or stochastic. In the former case, the next period's state variable Xt+ I is known
with certainty, but in the latter case Xt+1 is a random variable whose distribution
can be obtained as a function ofx" u" and w,.
Finally, we define an optimal policy as a decision rule that computes an op-
timal decision u, for each conceivable value of the state variable x, at stage
t = 0, I, ... , N - I, i.e., 1r = [po(xo), PI(XI), ... , PN-I(XN-I)]. Using the
optimal policy 1r, the optimal decision is computed from u, = p,(x,). For ex-
ample, in the inventory problem, an optimal policy may be shown to be of the
form

p,(x,) = °
{ s, -x, ifx, < S,.
otherwise

where S, is the "base-stock" level computed using the problem parameters [166].
At any stage, the optimal decision (a number) is computed using the above policy.
If at the start of period 4 we have X4 = 3 and, say, S4 = 10, then the optimal order
quantity U4 in period 4 is obtained as U4 = P4(3) = to - 3 = 7 units.
 6.1 Introduction 195

Using the "principle of optimality" I that was first enunciated by Richard BeIl-
man [21], [23, p. 83], dynamic programming is used to "divide and conquer" a
sequential optimization problem by decomposing it into a series of smaller prob-
lems. Once the smaller problems are solved, they are combined to obtain the s0-
lution of the complete problem.
For example, consider a three-period inventory problem with t = 0 as the be-
ginning of December, t = I as the beginning January and t = 2 as the beginning
February. Using the "backward-recursion" approach ofDP, we would first find the
optimal solution for February-the last stage-and compute the optimal decision
U2 = ,u2(X2) as a fimction of any possible value of the inventory X2 at the start
of February. With the information on the optimal decision available, we would
then compute the minimum cost V2(X2) for February. Stepping back one month,
the optimal policy ,u I (XI) for January would be found by adding the costs for that
month to the minimum cost V2(X2) that was found for February. This solution
would then be used to compute the minimum cost VI (XI) ("cost to go") for peri-
ods I and 2 (January and February). Finally, the optimal solution for December
would be computed taking into account the costs for that month plus the cost to
go VI (XI) for the remaining months of January and February. Once the optimal
policy is found for each month as a fimction of the entering inventory for that
month, as time progresses and state variables X, are observed, actual decisions
would be computed using the optimal policy fimctions ,u,(x,), t = 0, 1,2.
As we indicated, the three-month inventory control example uses the backward-
recursion approach of DP. For an interesting discussion of a puzzle and a mathe-
matical proof using the backward approach, see Nemhauser [138, pp. 19-22 ].
When the process studied is deterministic, it may also be possible to use the
"forward-recursion" approach, which can sometimes be more intuitive. For de-
tails of the forward approach, we refer the reader to Bertsekas [31, p. 52], Cooper
and Cooper [49, p. 114-122], Dane [57, pp. ISO-ISS] and Larson and Casti [117,
p.233].
Sequential optimization problems with discrete stages can be conveniently for-
mulated using the state-equation approach as in Bertsekas [31]. The state trans-
formation of the basic problem is represented by a discrete-time dynamic system

Xt+1 = f,(x" u" w,), t = 0, I, ... , N - I (6.1)

where X, is the state (with xo a given constant), u, is the control and w, is the
uncontrol1able exogenous disturbance parameter that may be deterministic or ran-
dom. There may be constraints on the control variables such as u, ~ 0 so that u,
may have to be an element of a constraint space C,. When the problem is deter-
ministic, there may also be constraints on the state variables so that X, may have
to be an element of the constraint space S,. If L,(x" u" w,) is the cost incurred

JUAn optimal policy has the property that whatever the initial state and initial decision are, the
remaining decisions must constitute an optimal policy with regard to the state resulting from the first
decision."
196 6. Dynamic Programming

during period t, then the objective would be to minimize

N-I
L Lt(xt, Ut, Wt) + LN(XN),
t=O
(6.2)

which is the total cost function for the N -stage sequential problem. Note that if
the disturbance Wt is a random variable, then the proper statement of the objective
would involve expectations with respect to all the random variables, i.e.,

N-I
EWO.WI ••••• WN_1 L Lt(x" U" w,) + LN(XN).
t=O

To solve the sequential optimization problems given by (6.l}-(6.2), we will

first define the cost to go (or value) function J't(Xt) as the minimum (expected)
cost that can be obtained by using an optimal sequence of decisions for the re-
mainder of the process starting from an arbitrary state Xt in an arbitrary stage
t = 0, I, ... , N - I, N . This function can be written as
N-I
J't(Xt) = u,.~~~_1 ~ L;(x;, U;, Wi) + LN(XN), t = 0, I, .. . , N - I
I=t

VN(X) = LN(XN)

where the decision U; must be chosen from the feasible set C;, ; = t, ... , N - I.
Using the separability properties of this additive cost function and the princi-
ple of optimality, it can be shown that (see, for example, Bertsekas [31, p. 19],
Nemhauser [138, pp. 28-31], Larson and Casti [117, pp. 45-47]) the value func-
tion J't(Xt) is the solution ofa functional equation given as

J't(Xt) = min {Lt(xt, Ut, Wt)

u,eC,
+ J't+I[ft(X" Ut, Wt)]}, t = 0, I, ... , N - I

= LN(XN).

We note here that when Wt is random, we minimize the expected value of the
term inside the brackets, i.e., E w, {Lt(x" u" w,) + J't+I[ft(X" u" w,)]}, where,
of course, the expectation is taken with respect to the probability distribution of
W,. In this case J't(Xt) is defined as the minimum expected cost to go for stages
t,t+I, ... ,N.
If the objective function is in the multiplicative form, i.e., if we are trying to
optimize n~ Lt(xt. Ut. Wt), then the functional equations for the value function
are obtained as

J't(Xt) = min {Lt(x" Ut, Wt)· J't+I[ft(X" u" Wt)]}, t

U,Ee,
= 0, I, .. . ,N - 1

VN(XN) = min LN(XN, UN, WN).

uNeCN
 6.2 Stagecoach Problem 197

For details of this problem, see Larson and Casti [117, pp. 48-49] and Nemhauser
[138, p. 37]. The multiplicative objective was originally used by Bellman and
Dreyfus [26] to model some types of reliability problems.
We will now discuss some examples of sequential decision problems and present
their analytic solution using Maple. We will see that Maple's ability to manipulate
symbolic expressions makes it an ideal tool for extracting the optimal policies for
these problems.

6.2 Stagecoach Problem

Our first example is a simple but illustrative deterministic dynamic program-
ming problem that is known in the operations research literature as the "stage-
coach problem." It deals with a hypothetical 19th-century stagecoach company
that transports passengers from California to New York. Although the starting
point (California) and the destination (New York) are fixed, the company can
choose the intermediate states to visit in each stage of the trip.
We assume that the trip is completed in four stages (legs) where stage 1 starts
in California, stage 2 starts in one of three states in the Mountain Time Zone (say,
Arizona, Utah or Montana), stage 3 starts in one of three states in the Central
Time Zone (say, Oklahoma, Missouri or Iowa) and stage 4 starts in one of two
states in the Eastern Time Zone (North Carolina or Ohio). When stage 4 ends, the
stagecoach reaches New York, which is the final destination.
Since in those days travel by stagecoach was rather dangerous because of at-
tacks by roaming criminals, life insurance was offered to the traveling passengers.
Naturally, the cost of the insurance polky was higher on those portions of the trip
where there was more danger. The stagecoach company thus faced the problem
of choosing a route that would be cheapest and thus safest for its passengers.

1(2,3) I --+ 7 I(3,3) I \.1

\.4 \.4
\.8 I(4,2) I
)"2 )"3 )"6 \.5
@JTI --+ 4 1(2,2) I --+ 2 I(3,2) I I(5,1) I
\.3 \.4 \.3 )"4
)"6 I(4,1) I
)" 1 )"3
I(2,1) I --+ 4 I(3,1) I )"3
TABLE 6.1. Data for the stagecoach problem.
198 6. Dynamic Programming

The nodes in the diagram in Table 6.1 are denoted by (i, j) where i corresponds
to a stage (i = I, 2, ... , 5) and j corresponds to a state in state i. Given the state
j in stage i, the decision to travel to state k in the next stage i + I results in a cost
(i.e., insurance premium) of cij(k). These costs are indicated next to the arrows
corresponding to each decision. For example, clI(2) = 4 and C21(3) = 6. The
problem is to find the route that results in the minimum cost.
For this problem, the value function V;j is the minimum cost from state (i, j)
to the final state (5, I) using the optimal policy. Thus, the dynamic programming
recursive equations are written as

VSI =0

V;j = mkin {cij(k) + V;+I,k}

where the expression inside the parenthesis is minimized by a suitable choice of
the decision variable k.
In the Maple worksheet that follows we enter the cost data as lists, e.g., c( 1, 1]
: ~ 3, 4 , 2] . This means that if we are now in state (I, I), the cost of going to
states (2,1), (2,2) and (2,3) are 3, 4 and 2, respectively.
> restart: * StageCoach.rows
> c( 1, 1] : ~ 3, 4 ,2] ;
CI,I := [3, 4, 2]
> c( 2, 1] : ~ 4, 1, 6]; c( 2, 2] : ~ 4, 2, 3] ;
c( 2, 3] : ~ 8, 4, 7] ;
C2,1 := [4, I, 6]
Cl,2 := [4, 2, 3]
Cl,3 := [8, 4, 7]
> c( 3, 1] : ~ 3, 3]; c( 3, 2] : ~ 3, 6]; c( 3, 3] : ~ 4, 1] ;
C3, I := [3, 3]
C3,2 := [3, 6]
C3,3 := [4, I]
> c( 4, 1] : ~ 4]; c( 4, 2] : ~ 5] ;
C4,I := [4]
C4, 2 := [5]
For stage 5, the boundary condition for the value function is trivially obtained
asO.
> i :=5;
i:= 5
> V[ i, 1] : =0;
 6.2 Stagecoach Problem 199

VS,I :=0
Stage 4 computations are also trivial since, once in stage 4, the stagecoach must
travel to the last state.
> i:=4;
; :=4
> for j from 1 to 2 do
> V[ i,j] : =min (seq(c[ i,j][ k] +V[ i+l,k] ,k=1. .1));
> Tq i,j] :={ seq(c[ i,j][ k] +V[ i+1,k] ,k=1..1)];
> for k from 1 to 1 do
u[ i, j, k] :=is (Tq i, j] [ k] =V[ i, j]) od od;
V4,I :=4
TC4,I := [4]

V4,2 := 5

TC4,2 := [5]
> for j from 1 to 2 do for k from 1 to 1 do
print([ i,j,k,u[ i,j,k] ,Tq i,j][ k]]) od od;
[4, 1, 1, true, 4]
[4, 2, 1, true, 5]
For stage 3, we find the value function for each state and the corresponding
optimal decision. For example, [3,1,1, true, 7] means that in state [3, 1] it is
optimal to choose k = 1, which results in a minimum overall cost of 7. Thus,
the word true next to a decision (the third component in the list) implies the
optimality of that decision. On the other hand, the word false implies that the
corresponding decision is not optimal.
> i: =3;
;:= 3
> for j from 1 to 3 do
> V[ i, j] : =min (seq (c[ i, j] [ k] +V[ i+1, k] ,k=1. .2) ) ;
> Tq i,j] :={ seq(c[ i,j][ k] +V[ i+1,k] ,k=1. .2)];
> for k from 1 to 2 do
u[ i, j, k] : =is (Tq i, j][ k] =V[ i, j]) od od;
V3,I :=7

TC3,I := [7, 8]

V3,2 := 7

TC3,2 := [7, 11]

V3,3 := 6
200 6. Dynamic Programming

1'C3.3 := [8, 6]
> for j from 1 to 3 do for k from 1 to 2 do
print([ i,j,k,u[ i,j,k] ,Tq i,j][ k))) od od;
[3, I, I, true, 7]
[3, I, 2,false, 8]
[3, 2, I, true, 7]
[3,2, 2,false, 1I]
[3,3, I,false, 8]
[3, 3, 2, true, 6]
Stage 2 and stage I calculations are perfonoed in a similar manner:
> i:=2;
i :=2
> for j from 1 to 3 do
> V[ i,j] :=rnin(seq(c[ i,j][ k] +V[ i+l,k] ,k=1. .3»;
> Tq i,j] :={ seq(c[ i,j][ k] +V[ i+1,k] ,k=1..3)];
> for k from 1 to 3 do
u[ i, j, k] :=is (Tq i, j] [ k] =V[ i, j]) od od;
V2.) :=8

1'C2.) := [II, 8, 12]

V2.2 := 9

1'C2.2 := [lI, 9, 9]
V2. 3 := II

1'C2.3 := [15, lI, 13]

> for j from 1 to 3 do for k from 1 to 2 do
print([ i,j,k,u[ i,j,k] ,Tq i,j][ k]]) od od;
[2, I, I, false, lI]
[2, I, 2, true, 8]
[2, 2, I, false, lI]
[2, 2, 2, true, 9]
[2, 3, I,false, 15]
[2,3,2, true, II]
> i:=l;
i:= I
> for j from 1 to 1 do
 6.3 Models with a Linear System and Quadratic Cost 201

> V[ i,j] :=min(seq(c( i,j][ k] +V[ i+l,k] ,k=1. .3));

> Tq i,j]:={ seq(c( i,j][ k] +V[ i+l,k] ,k=1..3)];
> for k from 1 to 3 do
u[ i,j,k] :=is(Tq i,j] [k] =V[ i,j]) ad ad;
VI,I:= 11
TCI, I := [II, 13, 13]
> for j from 1 to 1 do for k from 1 to 3 do
print([ i,j,k,u[ i,j,k] ,Tq i,j][ k]]) ad ad;
[I, I, I, true, II]
[I, I, 2,/a/se, 13]
[I, I, 3,/a/se, 13]

We can now extract the optimal solution from the results. From the initial
state (1,1), it is optimal to travel to (2,1) at a one-stage cost of 3 since we find
[1,1,1,lrue,II].2 Next, from (2,1) it is optimal to go to (3,2) at a cost of I since
[2,1,2,lrue,8].3 From (3,2), the stagecoach should travel to (4,1) at a cost of 3,
and finally from (4,1) it will go to (5,1) at a cost of 4. Adding the one-stage costs
we find 3 + I + 3 + 4 = II as we found from [1,1,I,true,II].

6.3 Models with a Linear System and Quadratic Cost

Now, consider the sequential problem with the cost function 'L.;=o(x; + + x~ u;)
and the system equations XH I = X, + u" 1 = 0, 1,2 with xo given as a constant.
In this deterministic problem with quadratic costs and linear system dynamics,
it is desirable to bring the state as close as possible to the origin in the cheapest

are penalized using quadratic terms. We will assume that xo =F (otherwise the °
possible way. Deviations from the origin for both the state X, and the decision u,

solution would be trivial) and that there are no constraints on the state and the
control.
Although this model with linear system and quadratic cost appears simple, its
generalizations have found applications in economics and operations research;
see Bensoussan, Hurst and Naslund [29, Chapter 3], Holt, Modigliani, Muth and
Simon [93], Parlar [140], Parlar and Gerchak [144] and Parlar and Rempala (145).
Forming the DP functional equation, we have

V, (x,) = min[x;+u;+v,+I(x,+u/)],
u,
1=0,1,2

2This also indicates that the overall minimwn cost is II.

31fthe trip had started in state (2,1), the overall minimum cost to New Yoric would be 8.
202 6. Dynamic Programming

We will now use Maple to solve this functional equation and generate the opti-
mal policy for the problem.
> restart: * LQ2.rnws
We begin by infonning Maple that the boundary condition is V3(X) = x 2•
> V[ 3] :=x->x"2;

V3 :=X ~ x 2
Now, for period 2 the cost expression that should be minimized is C2 = x~ +,q +
V3(X2 + U2), i.e., the sum of the current cost and the future minimum cost with
X3 = X2 + U2 as the state for stage 3.
> e[ 2] : =x"2+u"2+V[ 3] (x+u);

C2 := x 2 + u2 + (x + u)2
Differentiating this expression, equating the result to zero and solving gives the
optimal policy in period 2 as P2 = -!x.
> deriv[ 2] :=diff(e[ 2] ,u);
deriV2, := 4u + 2x
> rnu[ 2] : =so1ve(deriv[ 2] ,u);
1
P2:= --x
2
Using these results, the value function V2 (x) for period 2 is found as a quadratic
function of the state variable.
> V[ 2] :=unapp1y(subs(u=rnu[ 2] ,e[ 2] ) ,x);
3
V2 :=x ~ _x 2
2
Repeating the same procedure for the other stages, the optimal decision and the
value function is computed explicitly as a function of the state variable in each
stage:
> e[ 1] : =x"2+u"2+V[ 2] (x+u);

CI := x 2 + u2 + ~2 (x + u)2
> deriv[ 1] :=diff (e[ 1] ,u) ;
derivl := 5 u + 3x
> rnu[ 1] :=so1ve(deriv[ 1] ,u);

3
PI :=-5 x
> V[ 1] : =unapp1y (subs (u=rnu[ 1] ,e[ 1] ) ,x) ;

8
VI :=X ~ _x 2
5
> e[ 0] : =x"2+u"2+V[ 1] (x+u);
 6.3 Models with a Linear System and Quadratic Cost 203

co := X2 + U2 + ! (x + u)2
5
> derive 0] :=diff(c[ 0] ,u);
26 16
derivo:= Su+Sx
> rnu[ 0] :=solve(deriv[ 0] ,u);
8
Ilo:= -Ox
> V[ 0] :=unapply(subs(u=rnu[ 0] ,c[ 0] ),x);

21
Vo :=X ~ _x2
13
The results (to three significant digits) are summarized in the table below.

t 1l,(X) v, (x)
0 -0.615x 1.615x 2
1 -0.6x 1.6x 2
2 -0.5x 1.5x 2
3 x2
Thus, if the initial value of the state is, say, xo = 1, then using the optimal
decisions prescribed by the policy

1r = (IlO(X), III (x), 1l2(X» = (-0.615x, -0.6x, -0.5x),

one would obtain a total minimum cost of Vo(l) = 1.615 for the periods 0, 1,2
and 3. For this case, the numerical values of the optimal decisions are obtained as
follows:
t x, u,
o 1 -0.615
1 0.385 -0.231
2 0.154 -0.077.
3 0.077

Now suppose that the initial decision is not made optimally and instead of uo =
-0.615 we use, say, uo = -0.5, which brings the system to XI = 1 - 0.5 = 0.5.
What is the best decision now at the start of period 1 given this value of the
state? As the principle of optimality states, "An optimal policy has the property
that whatever the initial state and initial decision are, the remaining decisions
must constitute an optimal policy with regard to the state resulting from the first
decision." Thus, in period 1 we make the optimal decision for that period using
UI = III (0.5) = -0.6 x 0.5 = -0.3. This is the power and elegance of the
dynamic programming approach: Regardless of what may have transpired in the
past, at the start of any period we know exactly what to do since DP makes the
optimal policy available for all periods and for any value of the state.
204 6. Dynamic Programming

In this example we can see another aspect of the power and flexibility of dy-
namic programming. Now consider the following case: After applying the correct
decision uo = -0.615, what happens if for some reason there is an unexpected
random disturbance on the system and we find XI = 0.4 (instead of 0.385)1
The answer is fortunately very simple. Since we have the optimal policy avail-
able to us as P.I (XI) = -0.6XI, we just compute the new (optimal) decision as
UI = -0.6 x 0.4 = -0.24. Even though the correct initial decision uo = -0.615
resulted in an unexpected value of the state XI = 0.4, we can still make the next
decision in stage I optimally using the optimal policy P.I (x I) for that stage.

Solution as a Nonlinear Programming Problem

We note that the sequential optimization model described above can also be solved
as a standard nonlinear programming (NLP) problem with six decision variables
(x, u) = (XI, X2, X3, Uo, UI, U2), the objective function I(x, u) = L~=o(X; +
u~)+xj and three equality constraints hI = XI -XO-UO = 0, h2 = X2 -XI-UI =
oand h3 = X3 - X2 - U2 = 0 with xo as a given constant. The methods discussed
in Chapter 5, Nonlinear Programming, can be applied and using the method of
Lagrange multipliers, the optimal solution can be found.
Writing the Lagrangian as
3
LeX, u, 9) = I(x, u) + LO,h,(x, u)
,=1
where 9 = (01,02, (3) are the Lagrange multipliers, we obtain the solution for
this problem using Maple as follows.
> restart: * LQ2NLP.mws
> x[ 0] :=1:
> f:=sum(x[ t] "2+u[ t] "2,t=0 •• 2)+x[ 3] "2:
> h[ 1] : =x[ 1] -x( 0] -u[ 0] :
> h[ 2] :=x( 2] -x( 1] -u[ 1] :
> h[ 3] :=x[ 3] -x[ 2] -u[ 2] :
> X:=seq(x[ t] ,t=1..3):
> U:=seq(u[ t] ,t=O .. 2):
> L: =f+sum (theta[ k] * h[ k] ,k=1. .3) :
> L_x:=seq(diff(L,x[ k] ),k=1. .3):
> L_u:=seq(diff(L,u[ k] ),k=0 •. 2):
> L _theta: =seq (diff (L, theta[ k] ) , k=1. .3) :
> Digits:=3:
> evalf (solve ({ L x, L u, L theta} ,{ X, U, theta[ 1] ,
theta[ 2] ,theta[-3] } f); -

{U2 = -.0769,01 = -1.23, XI = .385, UI = -.231,02 = -.462, X2 = .154,

X3 = .0769,03 = -.154, uo = -.615}
 6.3 Models with a Linear System and Quadratic Cost 205

Naturally, the solution here is the same as the one we fotmd before. However,
note that the NLP solution is in some sense "frozen" because tmlike the DP s0-
lution it cannot directly deal with a situation where after applying uo = -0.615
there is an tmexpected random disturbance on the system and we find XI = 0.4
(instead of 0.385). To answer this question, the NLP problem must be re-solved
for stages I, 2 and 3 with XI = 0.4 as the initial value of the state in stage
I. This shows that the DP approach is, in general, much more powerful than a
competing optimization technique since the DP solution is obtained in terms of
dynamic policies rather than the static-frozen---decisions produced by the com-
peting techniques. 4
The simple model described and solved using DP can easily be generalized us-
ing Maple. For example, one may assume that there are ideal (i.e., target) levels
{x" t = 1,2, ... , N} and {il" t = 0, I, ... , N - I} for the states and deci-
sions, respectively, and that deviations from these trajectories are to be penalized
quadratically at a cost of q, for the states and r, for the decisions. If the dis-
turbances also follow a particular trajectory {WIt t = 0, ... , N - I}, then the
sequential optimization problem can be written
N-I
min L{q,(x,-x,)2 +r,(u, - u,)2} +qN(XN -XN)2
,=0

subject to the linear constraints

Xt+1 = a,x, + b,u, + WIt t = 0, I, ... , N - I

with given a, and bIt t = 0, I, ... , N - 1. Following is a Maple program that

automates the solution of this more general problem via DP where N = 3 with
the arrays xHat, q, uHat, r, a, b and w having the obvious meanings.
> restart: t LQ2Auto.mws
> N:=3: tAssume N=3
> xHat:=array(O .• N,[ 4,3,2,1]):
q:=array(O •• N,[ 1,2,1,2]):
> uHat:=array(O •• N-1,[ 1,2,3]):
r:=array(O •. N-1,[ 2,2,3] ):
> a:=array(O •. N-1,[ 1,7,2] ):
b:=array(O •. N-1,[ 4,2,1]):
> w:=array(O .. N-1,[ 2,-3,1]):
> Digits : =3:
> V[N] :=unapply(q:N]* (x-xHat[N] )"2,X)i

41n engineering terminology, nonlinear programming would produce "open-loop" decisions

whereas dynamic programming gives rise to "closed-loop" policies; see Bertsek~ [31, p. 4J and Kirk
[108, pp. 14-16].
206 6. Dynamic Programming

V3 := x ~ 2 (x _ 1)2
> for t from N-1 by -1 to 0 do
> e[ t] :=q( t] * (x-xHat[ t] ) "2+r[ t] * (u-uHat[ t] ) "2
+V[ t+1] (a[ t] * x+b[ t] * u+~ t]
) ;
> deriv[ t] :=diff(e[ t] ,u);
> mu[ t] :=evalf(solve(deriv[ t] ,u»;
> V[ t] : =
unapply(evalf (simplify (subs (u=mu[ t] ,e[ t] »),x);
> od;

C2 := (x - 2)2 + 3 (u - 3)2 + 2 (2x + u)2

deriv]. := 10 u - IS + SX
P2 := I.S0 - .soo X
V2 := x ~ 5.S0x 2 + lO.4x + 14.S

CI := 2(x - 3)2 + 2(u - 2)2 + 5 S

. 0(7x + 2u - 3)2 + 72.Sx
+ 20.S u - 16.4
derivl := 50.4u - 56.S + 162. x
PI := 1.13 - 3.21x

VI := x ~ 24.6x 2 + .I92x + 29.S

co := (x _4)2 + 2(u _1)2 + 24.6 (x +4u + 2)2 + .192x

+ 7. 6Su +30.2
derivo := 791. u + 391. + 197.x
Po := -.494 - .249x

Vo := x ~ 1.12x2 - 6.5 Ix + 50.3

The results for the optimal policy and the value function for each period t =
0, ... ,3 are summarized in the following table.
t p,(x) V,(x)
o -0.249x - 0.494 l.12xl - 6.5 Ix + 50.3
I -3.2Ix + 1.13 24.6x 2 +0.192x + 29.S
2 -O.Sx + I.S 5.S0x 2 + lO.4x + 14.S
3 2(x _ 1)2

It is worth noting that in this more general problem the optimal policy is linear
and the value function is quadratic in the state variable. This result would be valid
even when the states and the controls are multidimensional and the disturbances
 6.3 Models with a Linear System and Quadratic Cost 207

are random provided that the cost is quadratic and the system equations are linear
with no constraints on the states and controls. For a discussion of this issue, see
Bertsekas [31, Section 4.1].

6.3.1 The Infinite-Stage Problem

In many realistic sequential optimization problems arising in business and ec0-
nomics, there may either be a very large number or infinitely many stages. For
example, a long-term planning problem involving the next 10 years where a deci-
sion has to be made each month would have 120 stages. Although this number is
finite, it is large enough so that approximating the problem with an infinite number
of stages may be useful in understanding the structure of the optimal policy.
As an example, consider the simple sequential problem discussed above. Ifwe
now assume that N ~ 00, the problem can be written as
00

min ~)x; + u~)

t=Q

subject to
Xt+) = Xt + u" t = 0, I, . .. .
For such a problem, the functional equations V,(Xt) for, say, t = 100 and t = 99
would be given by

VIOO(XIOO) = min
11100
[xfoo + u~oo + VIOI(XIOI)]

and
V99(X99) = min [x~ +';99 + VIOO(XIOO)].
1199

Since there is some regularity in the system equations Xt+ I = f, (Xt, Ut) = Xt + Ut
and the cost function L t = xi-
+ u~ (i.e., that t does not appear explicitly in f, or
L t ), under certain conditions the functional equation for the value function can be
written as
V(X) =min {L(x, u) + V[f(x, u)]},
II

which can be solved (usually iteratively) for the unknown function V (x ). 5

Successive Approximations
One of the methods that can be used to solve this problem is known as "successive
approximations" or "approximation in function space." The method starts with a

SIn order for Vex) to remain finite, three conditions an:: required: (i) L(x, u) must be bounded for
finite x and II, (ii) a state i and control ii must exist such that L(i, ii) = 0 and (iii) the specified state
i must be reachable from any admissible state by applying a finite number of admissible controls; see,
Larson and Casti [117, p. 222].
208 6. Dynamic Programming

guess V(O)(x) for the solution V(x). Using this guess, a new value of the function
V(I) (x) is computed from

V(I)(x) =min (L(x, u) + V(O)[f(x, u)]} (6.3)

"
where the corresponding policy p(l)(x) is found as the value of u that mini-
mizes the right-hand-side of (6.3) for each x. The successive approximations
method continues in this manner while computing the value function's nth ap-
proximation v(n) from v(n-I) using v(n)(x) = min (L(x, u)+ v(n-I)(f(x, u)]}.
Here, p(n-I) is determined as the value of u that" minimizes the right-hand side
L(x, u) + v(n-')[f(x, u)] for each x. This is a relatively easy method to apply
using Maple.
> restart: * LQlnf.mws
> Digits: =4;
Digits := 4
In the finite horizon version of this problem we had observed that the value
function was quadratic in the state variable x. Armed with this information, we
make the initial guess for the value function as V(O) = 2x 2• The following lines
automate the successive approximation of the value function and the policy p.
> L:=(x,u)->x A 2+u A 2;
L := (x, u) -+ x 2 + u2
> V[ 0] :=x->2.*x 2;A

Vo := x -+ 2.x 2
> for n from 0 to 5 do
> c[ n] :=(x,u)->L(x,u)+V[ n] (x+u):
> diff (c[ n] (x, u), u) :
> uApp:=solve(%,u);
> subs (u=uApp, c[ n] (x, u) ) :
> V[ n+1] :=unapply(simplify(%),x);
> od;
co := (x, u) -+ L(x, u) + Vn(x + u)
6.u +4.x
uApp:= -.6667x

1.667x 2
VI :=x -+ 1.667x 2
CI := (x, u) -+ L(x, u) + Vn(x + u)
5.334u + 3.334x
 6.3 Models with a Linear System and Quadratic Cost 209

uApp:= -.6250x

1.625x 2
V2 := X ~ 1.625x 2
C2 := (x, u) ~ L(x, u) + Vn(x + u)
5.250u + 3.250x
uApp := -.6190x

1.619x 2
V3 := x ~ 1.619x 2
C3 := (x, U) ~ L(x, u) + Vn(x + u)
5.238u + 3.238x
uApp := - .6182x

1.618x 2
V4 := x ~ 1.618x 2
C4 := (x, U) ~ L(x, u) + Vn(x + u)
5.236u + 3.236x
uApp := -.6180x

1.618x 2
Vs := x ~ 1.618x 2
cs := (x, u) ~ L(x, u) + Vn(x + u)
5.236u +3.236x
uApp := -.6180x

1.618x 2
V6 := x ~ 1.618x 2
We thus see that, after about five iterations, the solution quickly converges to
V(x) = 1.618x 2 and p(x) = -0.6ISx .

A Closed-Fonn Solution for a More General Cost Function

Now consider a more general problem where we wish to minimize L~(QX; +
Ru~) subject to the usual state dynamics Xt+1 = Xt + Ut . Here, deviations from
the origin for the state and control variables are penalized differently.
In this case the functional equation for the value function asswnes the fonn
V(x) =min [Qx 2 + Ru2 + V(x + u)]. Using the infonnation obtained for the
u
210 6. Dynamic Programming

structure of the optimal policy and the fonn of the value function, we assume that
V (x) = Ax2 and attempt to find a closed-fonn fonnula for the coefficient A. The
symbolic manipulation of the expressions to solve this problem is relegated to
Maple, which finds the solution easily.
> restart: *LQClosedForm.mws
> V:=x->A*x"2;
V :=x ~ Ax2
> c:=Q*x"2+R*u"2+V(x+u);
c := Q x 2 + R'; + A (x + ui
> cu:=diff(c,u);
cu:= 2Ru +2A (x + u)
> usol:=solve(cu,u);
Ax
usol:=---
R+A
> RHS:=normal(subs(u=usol,c));

RHS := x 2 (A Q + R A + Q R)
R+A
> Asol:=solve(V(x)=RHS,A);
1 1 1 1
Asol:= 2 Q + 2 JQ2 +4 QR, 2 Q- 2JQ2 +4Q R
> subs (A=Asol[ 1] , usol) ;
1 1
(2 Q+ 2 JQ2 +4Q R)x

R+~Q+~JQ2+4QR
2 2
Note that since Q - JQ2 +4QR < 0 and since the minimum cost V(x)
must be nonnegative, we choose the first solution Asol[ 1] for A in the above
computations.
> V:=subs (A=Asol[ 1] , V (x) ) ;
1 1
V := (2 Q + 2 JQ2 +4 Q R)x 2
To summarize, the optimal policy and the value function for this general case
are obtained as p(x) = -Ax/(R + A) and V(x) = Ax2 where

A= ~ (Q + J Q2 + 4Q R) .
Naturally, for the special case when Q = R = I, we obtain A = 1.618 so that
V(x) = 1.618x 2 and p(x) = -O.6ISx as we had found previously.
 6.4 Continuous-Time Dynamic Programming 211

The Model with a Discount Factor

In most practical problems with long planning horizons, it would be important to
discount the future cash flows to account for the concept of time value of money.
Thus, if P = 1/(1 + r) is the discount factor with r as the interest rate and
P E (0, I), then the objective would be to minimize the net present value of
the future costs, which would now be written as min L:O P' (xl + u~). With this
objective, the functional equation for the value function assumes the fonn V (x) =
min (L(x, u) + PV[!(x, u)]}, which can be solved for V(x) in the usual way.
u
Naturally, in this case the value function V (x) would be defined as the minimum
discounted cost that can be obtained by using an optimal sequence of decisions
for the remainder of the process starting from an arbitrary state x in an arbitrary
stage.
Although we do not present it here, it would be a simple matter to apply the suc-
cessive approximation method to solve a more general problem with discounted
cost and stationary target values, i.e.,

00

min LP'{Q(x, -ii + R(u, - 14)2}

,=0

subject to
Xt+1 = x, + u" 1 = 0, I, .. ..
For example, assuming Q = 2, i = I, R = 3, 14 = 1.5, and P = 0.9 and starting
with the initial guess of V(O) = 4x 2 + 2x + 60, after about seven iterations the
method converges to the value function V(x) = 3.SSx2 + l.lSx + 61.6 and to
policy ",(x) = 0.642 - 0.516x.

6.4 Continuous-Time Dynamic Programming

A different type of infinite-stage sequential optimization problem arises when
the system evolves continuously over time and a decision must be made at each
instant. In this case, even if the decision horizon is finite, the continuous nature of
the process results in an infinite number of decisions.
Let us now suppose that the system evolves according to the differential equa-
tion ;i(/) = ![X(/), U(/), I), 0 ~ 1 ~ T with the initial value of the state x (0) =
Xo given as a constant. This differential equation is the continuous-time analogue
of the discrete-time state equations that were written as X,+I = !(x" u" I),
1 = 0, I, ... , N - 1 in Section 6.3. We wish to minimize the cost functional
Jt L[X(/), U(/), I) dl + S[x(T), T), which is the continuous-time analogue of
the sum L~I L,(x" u" w,) + LN(XN).
Similar to the discrete-time fonnulation, let us define the value function V(I, x)
as the minimum cost that can be obtained starting in state X(/) at time 1 E [0, T).
212 6. Dynamic Programming

The value function can then be written as

V(I, x) =min [T L(x, u, r)dr + S[x(T), T] (6.4)

u I

subject to
x = I(x, u, r), X(I) = x
with the condition that at the final time I = T, we have V[x(T), T] = S[x(T), T].
Ifwe write the integral in (6.4) as

V(I,X) =min ( [ t+t.t Ldr + iT Ldr +s)

u I t+41
where At is an infinitesimally small interval, then by the principle of optimality
we obtain

V(I,X)= mjn [[t+4tLdr+( muin iT Ldr+s)] (6.5)

19St+41 I t+4t9ST t+41

subject to
x = I(x, u, r), X(I + al) = x + ax.
This follows from the principle of optimality because the control u( r) for 1+ al ~
r ~ T is optimal for the problem starting at time 1+ al in state x(1 + At) =
x+ax.
Now using the definition of V(I , x), we can rewrite (6.5) as

t+41 ]
V(I, x) = muin [[ Ldr + V(I + al,X + ax) .
ISfSt+4t I

Assuming V (x, I) to be twice differentiable and expanding V (I + At, x + ax)

using Taylor's theorem gives

V(I,x) = min[L(X,u,l)al+ V(1,x)

u
+ V,(I,x)al
+V.J(I, x)ax + higher order terms].
Subtracting V (I, x) from both sides, dividing by al and letting al --+ 0 we obtain

o = min[L(x,u,l) + V,(I,x) + V.J(I,X)x]

u
= min[L(x, u, I)
u
+ V, (I ,x) + V.J(I, x)/(x, u, I)].
Since the term V, (I ,x) does not involve the decision u, we move it to the left and
obtain the partial differential equation (POE) for the value function V (I, x) as

-V,(I,x) = min[L(x, u, l) + V.J(I,x)/(x, u,I)]

u
 6.4 Continuous-Time Dynamic Programming 213

with the boundary condition V[x(T), T] = S[x(T), T]. The resulting PDE is
known as the Hamillon-Jacobi-Bellman (HJB) equation since it was originally
developed by Bellman [22] to solve problems in the calculus of variations. For
discussions of the development of this PDE and examples of its solution, see
Bertsekas [31, p. 91-92], Bryson and Ho [39, p. 135], Kamien and Schwartz [99,
pp. 238-240], Kirk [108, pp. 86-90] and Sage and White [162, pp. 76-77].

A Remark on Discounting
Note that if we have an infinite horizon problem and if continuous discounting is
applied with a discount factor e-rt , then the optimization problem would be to
minimize.roo e-rt L(x, u, I)dl subject to the state equation x = f(x, u) with the
initial state as x (0) = xo. In this case, the HJB equation would be written as

- V, (I, x) = min[e-
u
rt L(x, u, I) + Vx(/, x)f(x, u, I)]

without any boundary condition on the value function. Here the value function is
defined as the minimum cost discounled 10 lime zero that can be obtained starting
at time I in state x and using the optimal policy. However, it can be shown that for
"autonomous" systems where time does not appear in the cost function (except
for the discount term) and in the system equations, a similar ordinary differential
equation (ODE) can be found for the HJB problem where the value function V (x)
would be defined as the minimum cost discounted to the current lime.
In autonomous systems, the problem is to minimize .roo e-rt L (x, u) dl subject
to x = f(x, u). For this problem we write the HJB equation where discounting is
to the current time (rather than to time zero) as

V(/,x) = min[L(x,
u
u) + e-rM V(t + M,x + ~x)].

Since e-r Ilt ~ 1 - r ~I and the value function at (I + ~/, x + ~x) is expanded
as
V(t + M,x + ~x) ~ V(/,x) + V,~I + Vx~x,

after simplifications and usual limiting operations the HJB equation is obtained
as an ODE in terms of V (x) as follows: 6

r V(x) = min[L(x,
u
u) + V'(x)f(x, u)].

For a more detailed discussion of this issue, see Beckmann [19, Part IV] and
Kamien and Schwartz [99, pp. 241-242].
We now consider two examples where the HJB partial differential equation can
be solved explicitly to find the value function and the optimal policy.

6Notc that in the limiting operations we have limAI-+O Ilx/Ilt =i = f(x, u).
214 6. Dynamic Programming

6.4.1 A Problem with Linear System and Quadratic Cost

Consider the following continuous-time version of the sequential optimization
problem with quadratic cost and linear state equation adapted from Kamien and
Schwartz [99, pp. 240-241]. We wish to minimize

10 00
e-"(Qx 2 + Ru2)dt

subject to i = u with the initial condition x (0) = xo > o. We now solve the HJB
equation - Vi = minu[e-rl (Qx 2 + Ru2) + Vxu] for this problem using Maple.
> restart: *
LQPDE.rnws
> L:=(x,u)->exp(-r*t)* (Q*x"2+R*u"2);
L := (x, u) ~ e(-rl) (Qx 2 + Ru2)
> f:=u;
! :=u
> RHS:=L(x,u)+diff(V(t,x),x)*f;
RHS:= e(-rl) (Qx 2 + Ru2) + (/x V(t, x»u
The optimal policy is obtained in terms of Vx as follows.
> uSol:=cornbine(solve(diff(RHS,u),u),exp);
V(t, x»e(rl)
•. cool .=
UI.J<.
__2I ...(.Q.
0:::::;(:.-_ _ __
R
Using this policy we develop the HJB partial differential equation and attempt
to solve it using Maple's pde () function.
> HJB:=-diff(V(t,x),t)=L(x,uSol)
+diff(V(t,x),x)*uSol;

HJB:= -(f, V(t, x» =

e(-rl) ( Qx2 + _ ox V(t ' x»2 (e(r 1»2) __I ....
I (.Q. (.Q. V(t
o::::x_ _ x»2_
' ....;.;.. e(r_
I)

4 R 2 R
> pdesolve(HJB, V(t,x»;
Error, (in pdesolve/exact/charac) dsolved returned
multiple answers,
[{ t ( Fl) = - Fl+ C4}, {p[ 2] ( Fl) =
DESoI({ (diff(diff( Y( FI), FI), Fl)*R
+r*diff(t( FI), FI)*dIff( Y( Fl),
Fl)*R+Q* Y( Fl»/R},{ Y(-Fl)})
* C3, x( FI)-=-1/2*exp(r*t( FI»/Q
*diff (DESol ({ (diff (diff ( Y (-Fl), FI), FI)*R
+r*diff(t( FI), FI)*diff( Y( FI); FI)*R
+Q* Y ( FI» IR} , r
Y ( FI)}); Fl> * C3},
{P[I] (-Fl) =Int(I/4*r*(4*exp(-2*r*t( FI»
*Q*x (_FI) "2*R+P[ 2] (_FI) "2)*exp(r*t (_FI» I
 6.4 Continuous-Time Dynamic Programming 215

R, Fl) + C2}, {U ( Fl}

Int«-PClJ ( Fl)*R+1/2*p( 2J ( Fl)"2
*exp(r*t(_FI»)/R,_Fl)+_Cl}]
Unfortunately, Maple fails to find a solution to this POE and returns an error
message. Thus, using our intuition obtained from the discrete-time version of this
type of problem with quadratic cost and linear system dynamics, we try a solution
in the form V(/,x) = e-rt Ax2 where the constant A is to be determined.
> v:=(t,x)->exp(-r*t)*A*x"2; Vt:=diff(V(t,x),t);
Vx:=diff(V(t,x),x);
V := (I, x) --+ e(-rt) Ax2

VI := -r e(-r t) A x 2

Vx:= 2e(-rt) A x
> HJB;

re(-rt) Ax2 =
e(-rt) x 2 (_Q R + A2)
R
Substituting and simplifying, we finally reduce the problem to the solution of a
quadratic equation in terms of the unknown A:
> ASol:=solve(HJB,A);
1 1 1 1 ~~~---
ASoI := -2: r R + 2: Jr2 R2 + 4 Q R, -2: r R - 2: Jr2 R2 + 4 Q R
Since the value function must be positive, we choose the first solution ofASol
so that we have A = -!r R + !Jr2 R2 + 4QR. The optimal policy is then 0b-
tained as u = -(A / R)x.
> uSol:=combine(uSol);
Ax
uSo/:= - - -
R

6.4.2 A Problem with Quadratic and Linear Costs

Many of the examples discussed are somewhat standard because they involve a
quadratic cost function with linear state equations. We now describe the solu-
tion of a more difficult problem where the cost function consists of a mixture of
quadratic and linear terms. We will see that Maple is again very helpful in manip-
ulating expressions that give rise to the value function and the optimal policy for
the model.
216 6. Dynamic Programming

Consider the following problem that appears as an exercise in Kamien and

lei
Schwartz [99, p. 242]. We wish to minimize the cost functional (C) u2 +c:zx) dl
subject to x = u with x(O) = 0 and x(T) = B. Kamien and Schwartz suggest
using V(/, x) = a + bXI + hx 2 /1 + kl 3 as the trial form for the value function.
Keeping this hint in mind, we proceed with the development of the HJB partial
differential equation.
> restart: # LPDE.mws
> L:=(x,u)->c:[ 1) *u"2+c:[ 2) *x;
L := (x, u) -+ C) u2 +c:zx
> f:=u;
/:=u
The right-hand side of the HJB equation that will be minimized is L(x, u) +
Vx/(x, u). Differentiating this expression, equating to zero and solving for u
gives the optimal policy function in terms of the unknown Vx •
> RHS:=L(x,u)+diff(V(t,x),x)*f;
RHS:= C) u2 +C2X + (! V(/, x»u
> uSol:=solve(diff(RHS,u),u);
I !
V(/, x)
uSaf := - - -=---
2 C)

> HJB:=-diff(V(t,x),t)=L(x,uSol)
+diff(V(t,x),x)*uSol;
I (..£. V(/, x»2
HJB:= -(I; V(/, x» = --4 OX
C)
+C2X
An attempt to solve the PDE explicitly using Maple fails.
> pdesolve(HJB, V(t,x));
Error, (in pdesolve/exact/charac) dsolved returned
multiple answers,
[ { p[ 1] ( F1) = C4}, {P[ 2] ( F1) = -c( 2] * Fl + C3} ,
{U( Fl)-=Int«=p[ 1] ( F1)*c(-1] - -
+1/2*P[2] (F1)"2)/c(1], F1)+ C2}, {x(J1)
Int(1/2/c[ l]*p[ 2] ( F1),-F1)+-C1},
{t(J1) = -J1+_csfl - -

At this stage we make use of the trial form of the value function and compute
its derivatives with respect to I and x.
> V: = (t, x) ->a+b* x* t+h* x"2/t+ k* t "3;
Vt:=diff(V(t,x),t); Vx:=diff(V(t,x),x);V(T,B);
hx 2
V := (I, x) -+ a + b x I + - + k 13
I
 6.4 Continuous-Time Dynamic Programming 217

hx 2
VI := b x - -2- + 3 k 12
I
hx
Vx:=bl+2-
I

a+bBT+ rhB2 + kT 3

> HJB:=lhs(HJB)-rhs(HJB);
hx 2
hx2 I (bl +2-)
HJB:= -bx + - -3k1 2 + - I -C2X
12 4 Ct
The HJB equation is now reduced to a nonlinear algebraic equation in tenns of
x, 12 and x 2/ 12 with unknown constants A, B, H and K, which we now compute
analytically.
The coefficients of x and 12 are easily extracted using the Maple eoe f f ( )
function.
> ex:=eoeff(HJB,x);
bh
ex := -b + - - C2
Ct
> et2:=eoeff(HJB,t A 2);
I b2
cI2 := -3 k + - -
4 Ct
2
To extract the coefficient of x / 12 , we use a somewhat obscure Maple function
algsubs () that perfonns more general algebraic substitutions than the substi-
tutions subs () function can make. This substitution will have a temporary effect
of defining x 2/1 2 as z2, thus making the expression a polynomial so that the eo-
e f f () function can be applied.
> x2t2:=algsubs(x A 2/t A 2=z A 2,expand(HJB));
(bCt-bh+C2Ct)X I (l2kCt-b2)/2 hz2(c1 +h)
x212 := - - + -......;...~--'-
Ct 4 CI CI
> ex2t2:=eoeff(x2t2,z A 2);
ex212 := h (Ct + h)
Ct
To make sure that we have not missed any tenns while extracting the coeffi-
cients, we perfonn a simple check and observe that everything is in order.
> simplify (ex* (x)+et2*(t A 2)+ex2t2* (x A 2/t A 2)-HJB);
o
Now that we have the algebraic expressions for the coefficients of x, 12 and
ex, et2 and ex2t2) and the final time condition that V[T, x(T)] =
X 2 /1 2 , (i.e.,
218 6. Dynamic Programming

V(T, B) = 0, we can solve the resulting system of four nonlinear equations in

the four unknowns a, b, h and k.
> solve({cx,ct2,cx2t2,V(T,B)} ,{a,b,h,k});

{h = 0 k = 1. C2 2 a = _1. C2 T (-12 B c) + C2 T2) b = _""'} {h = -c)

, 12 C) , 12 c) ,~.I. , ,

k = 1. C2 2 , b = _.!. C2 a = _1. -24c2 B T2 c) - 48c)2 B2 + C2 2 r}

48 C) 2' 48 T C)
Maple finds two solutions but we use the one where the solution is nonzero.
> assign (%[ 2] );
> a; b; h; k;
1 -24C2 B T2 C) - 48C)2 B2 + C2 2 T4
48 Tc)
1
-2 C2

1 C2 2
48 e)
The value ftmction V (I, x) and the optimal production rate x= U are now
easily computed as ftmctions of the stage I and state x .
> V(t,x); * Value function
1 -24C2 B T2 C) - 48c)2 B2 + C2 2 T4 1 C) x 2 1 C2 2 13
-- --C2XI---+---
48 T e) 2 I 48 C)
> Vx;
1 C) x
--C21 - 2 -

> expand(sirnplify(uSol));
2
* Optimal production rate
I

1 IC2 x
--+-
4 C) I
To obtain an analytic expression for the optimal production rate u(l) and for the
optimal inventory level X(I), we solve x(l) = U(I) using the form of the optimal
policy just determined.
> dsolve({diff(x(t),t)=(t*c[ 2] /(4*c[ 1] ))+x(t)/t,
x(O)=O, x(T)=B} ,x(t));

X(I) =.!. 12 C2 _.!. I(C2 T2 -4Bc)

4 C) 4 Te)
> assign(%);
 6.5 A Constrained Work Force Planning Model 219

> xOpt:=unapply(normal(x(t)),t);
uOpt:=unapply(normal(diff(x(t),t)),t);

XOpI:= 1 -"7 _~ 1 (-021 T +02 T2 -4 BCI)

4 TCI

UOpI:=I-"7 _~ -202IT+02T2_4Bcl
4 TCI

6.5 A Constrained Work Force Planning Model

Many of the models discussed so far were relatively simple since the costs were
quadratic and there were no constraints on the state or the decision variables.
We now present a more complicated sequential decision problem of work force
planning with constraints on both state and decision variables. For a discussion
of similar models, see Bensoussan, Hurst and Nllslund [29, pp. 58-72], Hillier
and Lieberman [92, pp. 345-350], Holt, Modigliani, Muth and Simon [93] and
Sasieni, Yaspan and Friedman [164, p. 280].
The work force requirements in a particular factory fluctuate due to seasonality
of the demand. The estimated work force requirements during each of the four
seasons for the indefinite future are as follows:
Season Summer Fall Winter Spring
Requirements (r,) rl = 210 r2 = 250 r3= 190 r4= 260
Due to the undesirability-and costlin~fhiring and firing of the workers,
management wishes to minimize fluctuations in work force levels. It is estimated
that the total cost of changing the work force level from one season to the next
is s = $100 times the square of the difference in the levels between the two
seasons. For example, if the Summer employment level were 215 people and if an
additional 10 workers were hired for the Fall, the cost of this change in the work
force would be $100 x (225 - 215)2 = $10000. Any employment level that is
above the levels given in the table is considered a "waste" that costs the company
approximately w = $1000/person/season. For example, a Summer level of215
would result in a waste ofSlO00 x (215 - 210) = $5000 for that season.
Although this is a problem with an infinite number of stages, each year starts
an identical cycle. Since the cost data are assumed to be stationary, we can con-
sider only one cycle of four seasons that end with Spring. We will assume that
the Spring production is the most important and that the management has decided
to keep the Spring employment level at U4 = 260. Also, note that since the esti-
mate of 260 for the Spring is the highest, it would not be optimal to increase the
employment level at any season above 260.
We denote the initial Summer season by 1 = 1 and the other periods as 1 =
2, ... ,4. If we define the decision variables u" t = 1, ... ,4 as the employment
levels used in period t (with U4 = 260), it is easy to see that this problem can be
220 6. Dynamic Programming

fonnulated as a nonlinear programming problem. For this problem the objective

function would be
4
min ~)I00(u, - U,_1)2 + l000(u, -r,)]
'=1

subject to the constraints 210 :5 UI :5 260,250 :5 U2 ~ 260, 190 :5 U3 ~ 260

(and U4 = 260). If this problem were solved using the techniques of nonlinear pro-
gramming, we would obtain the open-loop decisions as UI = 252.5, U2 = 250,
U3 = 252.5 and U4 = 260 with a minimum total cost of $117,500. Fractional
levels such 252.5 are assumed to be acceptable since they correspond to the em-
ployment of part-time workers.
We will now solve this problem with Maple's help using dynamic program-
ming. Since in the current stage the state (i.e., the available infonnation about the
system) is the previous stage's decision (i.e., the employment level chosen), we
write Xt+1 = U" t = 1,2,3 as the state equations with XI = 260. This gives
rise to the constraints on the states X" i.e., 210 ~ X2 ~ 260, 250 ~ X3 ~ 260,
and 190 :5 X4 ~ 260. Defining V,(x,) as the value function for stage t-the
minimum cost obtainable for periods t, t + 1, . .. , 4-when the optimal policy is
implemented, the DP functional equation becomes

V,(x,) = min
r/~II/~r4
[100(u, _x,)2 + l000(u, -r,)+ Vt+I(Xt+I)],t = 1, ... ,4
with the boundary condition

V4(X4) = l00(U4 - X4)2 + l000(U4 - r4)

= 100(260 - X4)2

since in stage 4 we must have U4 = r4 = 260.

We start by assigning values to the seasonal work force requirements and com-
pute the maximum requirement as R = max, {r,} = 260.
> restart: *
Manpower.rnws
> with (linalg) :
Warning, new definition for norm

Warning, new definition for trace

> 5:=100; w:=1000;
s := 100
w:= 1000
> r:=array(1 •. 4,[ 210,250,190,2600);
R:=rnax(seq(r[ i] ,i=1. .4));
r := [210, 250, 190,260]
R :=260
 6.5 A Constrained Work Force Planning Model 221

For stage 4 the value function V4(X4) is defined as the boundary condition.
> V[ 4] : =unapply (s* (r[ 4] -x) "2, x) ;

V4 := x ~ 100(260 - x)2
For stage 3 the recursive relationship is

whereC3(x3, U3) = lOO(U3 -X3)2 + lOOO(U3 -190) + V4(X4). We show thatc3

is convex in U3 given X3.
Differentiating C3 with respect to U3 = u, equating the result to 0 and solving
=
gives an expression for "3 (or the policy #3) in tenns of the state variable X3 x.
Recall that U3 is feasible in the interval from'3 = 190 to'4 = R = 260.
Evaluating the policy #3 at the end points of X3, we find #3(X3 = '2) = 252.5
and #3(X3 = = R) 257.5. This shows that the policy #3 is in the feasible interval
for any X3; i.e., we have #3 = !X3 + ¥ for 250::; X3 ::; 260.
With this infonnation about the optimal U3 we easily compute the value func-
tion V3(X) for stage 3.
> c[ 3] :=s* (u-x) "2+w* (u-r[ 3] ) +V[ 4] (u);

C3 := 100 (u - x)2 + 1000 u - 190000 + 100 (260 - u)2

> cp[ 3] :=diff(c[ 3] ,u); cpp[ 3] :=diff(c[ 3] ,u$2);
cP3 := 400 u - 200 x-51 000
cPP3 :=400
> mu[ 3] : =sol ve (cp[ 3] ,u) ;
I 255
#3 :=2 X +T
> * u[ 3] is feasible from r[ 3] =190 to R=260
> evalf (subs (x=r[ 2] ,mu[ 3] ) ); evalf (subs (x=R, mu[ 3] ) ) ;
252.5000000
257.5000000
> V[ 3] :=unapply(subs (u=rnu[ 3] ,e{ 3] ) ,x);
I ~5 2M I
V3 :=X ~ lOO(--x + _)2 + 500 x -62500+ 100(- - _x)2
2 2 2 2
For stage 2, the recursive equation becomes

where C2 (X2 , U2) = 100(U2 -X2)2 + lOOO(U2 -250) + V3(X3). Here the analysis
becomes more complicated.
222 6. Dynamic Programming

Differentiating C2 with respect to U2 we get OC2/0U2 = 3OOU2 - 200X2 +24500.

Next, equating this result to zero and solving for U2 we find the candidate policy
as #2 = jX2 + ~. Now, recall that U2 is feasible in the interval from r2 = 250
to r4 = R = 260, but evaluating the policy #2 at the extreme points of X2 we
find #2(X2 = r) = 210) = 221.67 (infeasible) and #2(X2 = R = 260) = 255
(feasible).
To find the range of X2 for which U2 is feasible, we solve #2 = 250 for X2
and obtain X2 = 252.5. This shows that the policy #2 applies only in the interval
for X2 from 252.5 to 260. For a given X2 $ 252.5, we find that C2 is convex and
increasing in U2. Hence, we set #2 = 250 for X2 $ 252.5. This gives a piecewise
function for the policy #2; see Figure 6.1.

/
",
/
255 /

250

221.67 -

252.5 260

FIGURE 6.1. The fonn of the policy Jl2(X2).

The value function V2(X2) and C) are easily computed in terms of the piece-
wise () function. But we suppress the output of these functions in order to con-
serve space.
> c[ 2] :=s* (u-x) "2+w* (u-r[ 2] ) +V[ 3] (u);

1 255
C2 := l00(u -x)2 + 1500u - 312500 + 100(-2 u + 2)2
+ 100 (265 _ ~ u)2
2 2
> cp[ 2] :=diff(c{ 2] ,u); cpP[ 2] :=diff(c{ 2] ,u$2);
cP2 := 300 u - 200 x - 24500
 6.5 A Constrained Work Force Planning Model 223

CPP2:= 300
> mu( 2J : =501 ve (CP[ 2J ,U) ;
2 245
1l2:=3"x+T
> i u( 2J is feasible from r[ 2J =250 to R=260
> evalf (subs (x=r[ 1J ,mu( 2] ) ); evalf (subs (x=R, mu( 2J ) ) ;
221.6666667
255.
> xLow:=evalf(solve(mu( 2]=2S0,x));
xLow := 252.5000000
> xHigh:=evalf(min(R,solve(mu( 2J=260,x)));
xHigh := 260.
> uLow:=subs(x=xLow,mu( 2J);
uLow:= 250.0000000
> uHigh:=subs (x=xHigh,mu( 2] );
uHigh := 255.0000000
> mu( 2] :=piecewise(x<=xLow,r[ 2] ,x<R,mu( 2] );
250 x ~ 252.5000000
1l2:= { 2 245 260
3"x+T x <
> cp[ 2J ;
300 U - 200 x - 24500
> assume(210<x,x<252.S,250<u,u<260); is(cP[ 2J>0);
true
> x : =' x'; u : =' u' ;
x :=x
U:=U
> VI 2] : =unapply (subs (u=mu( 2] ,c[ 2J ) ,x) :
> c[ 1J :=5* (u-r[ 4] ) "2+w* (u-r[ 1] ) +V[ 2] (u):
Although we do not present the graph of the CI function here, we include a
Maple command plot (c[ 1] ,u=190 .. 260) that can be used to note that CI
is convex in UI. Thus differentiating this function using the Maple diff () com-
mand and solving gives the optimal numerical value of the first period's decision
as UI = 252.5. The minimum total cost for all four periods is now easily found as
VI = 117500 by substituting the optimal UI into CI.
> iplot(c[ 1J ,u=190 .. 260,discont=true);
> cp[ 1J : =dif f (c[ 1J ,u) :
224 6. Dynamic Programming

> rnu[ 1) : =eval f ( sol ve (cP[ 1) ,u) ); u[ 1) : =rnu[ 1) ;

PI := 252.5000000
"I := 252.5000000
> V[ 1) : =evalf (subs (u=rnu[ 1) ,C( 1) »;
VI := 117500.0000
The optimal employment levels for the other periods are easily computed as
"2 = 250, "3 = 252.5 and "4 = 260.
> x: =rnu[ 1) ; u[ 2) : =rnu[ 2) ;
x := 252.5000000
"2 := 250
> x: =rnu[ 2) ; u( 3) : =evalf (rnu[ 3) );
x := 250
"3 := 252.5000000
> x: =evalf (rnu[ 3) ) ; u[ 4) : =r{ 4) ;
x := 252.5000000
260
"4 :=
The following table summarizes the solution.

t I 2 3 4
~ 210 250 190 260
", 252.5 250 252.5 260

6.6 A Gambling Model with Myopic Optimal Policy

In his seminal paper entitled "A New Interpretation of Infonnation Rate," Kelly
[103] considered an unscrupulous gambler who was receiving prior infonnation
concerning the outcomes of sporting events over a noisy communication channel
(e.g., a phone line) before the results became common knowledge. After receiving
the word ''win'' or "Iose"-which he could hear incorrectly due to communica-
tion difficulties-the gambler would place his bet (a nonnegative amount up to his
present fortune) on the original odds. With P as the probability of correct trans-
mission and q = I - p the probability of an incorrect transmission, Kelly showed
using calculus techniques that if p > q, the gambler's optimal wager" (i.e., the
fraction of his capital bet each time) was simply" = p - q. If p ~ q, then " = O .
He also showed that the maximum growth of capital occurred at a rate equal to
the capacity of the channel given as Gmax = 10822 + p log2 P + q log2 q. (Inter-
estingly, this was the same result obtained by Shannon [171] from considerations
of coding of infonnation.)
 6.6 A Gambling Model with Myopic Optimal Policy 225

Kelly's work attracted the attention of a large number of researchers including

economists, e.g., Arrow [10], psychologists, e.g., Edwards [63] and applied math-
ematicians such as Bellman and Kalaba [27], [28], [24] who extended and reinter-
preted Kelly'S results. In particular, Bellman and Kalaba re-solved the same prob-
lem after formulating it as a dynamic program with a logarithmic utility ftmction
for the terminal wealth of the gambler. They assumed that the gambler is allowed
N plays (bets) and that at each play of the gamble, he could bet any nonnegative
amount up to his present fortune.
In the formulation of the gambling problem using OP we define Vn(xn) as the
maximum expected return if the gambler has a present fortune of Xn and has
n gambles left. (This is a slight change in notation compared to what we had
used before. In this model n = N - I is the number of gambles left where n =
0, 1, ... , N.) At each play, the gambler could bet any nonnegative amount up to
his present fortune and win the gamble with a probability of p. The objective is to
maximize the terminal utility of his wealth which, is assumed to be logarithmic;
i.e., when there are no gambles left, the gambler's utility of his final wealth is
Vo(xo) = log(xo). When n gambles are left, we define Un as the fraction of the
current wealth to gamble with 0 ::: Un ::: 1. Then the state equation assumes
the form Xn-I = !n(Xn, Un, Wn) = Xn + UnXnWn, n = 1, ... , N where the
random variable Wn takes the values 1 or -1 with probability p and q = 1 - p,
respectively. Thus, the OP ftmctional equation is written as

max Ew" Vn-I (xn

0:91,,:::;1
+ UnXnWn)
= max (PVn-I(Xn + UnXn) + q Vn-I (Xn - UnXn)]
O:::;u,,:::;1

with the boundary condition Vo(xo) = log(xo).

We now describe the OP solution of Kelly's problem using Maple. Our notation
and exposition closely follow those of Ross [159, pp. 2-4].
First, we show that when p ::: !,
it is optimal not to bet (u = 0) and that
Vn(xn) = log(xn). Starting with n = 1, we see that the CI ftmction is monotone
!,
decreasing in u, so the optimal decision is to bet O. (Actually when p = setting
U = 2p - 1 makes CI constant. But since U = 2p - 1 = 0, the statement about
the optimal decision is still valid.) With PI (XI) = 0, we obtain VI (x» = 10g(xI).
For other values of n > 1, the same solution can be shown to be valid proving
!,
that when p ::: Pn(xn) = 0 and Vn(xn) = log(xn).
> restart: * Gamble.mws
> assume (O<=p,p<=1/2) : additionally(O<=u,u<=l):
> q:=1-p;
q:= 1- P
> c[ 1] :=p*log(x+u*x)+q*log(x-u*x);
CI := p In(x + ux) + (1 - p) In(x - ux)
> c[ 1,1] :=normal (diff (c[ 1] ,u));
226 6. Dynamic Programming

2p - 1 - u
CI,I :=
(l+u)(-l+u)
> is(c{1,1]<=0);
true
> V[ 1] :=simplify(subs(u=O,c{ 1]»;

VI := In(x)
We now tum to the case where p > !.
> restart: * Gamble.mws (Part 2)
> assume(x>O,p>O);
> q:=1-p;
q:= 1- P
The boundary condition is the utility of the final fortune, which is defined as
the logarithmic function in x.
> V[ 0] :=x->log(x);

Vo:= log
When there is one play left, we find that it is optimal to bet 1'1 = 2p - 1 =
P - (l - p) = p - q fraction of the current wealth:
> c[ 1] : =p* log (x+u* x) +q* log (x-u* x) ;

CI := p In(x + ux) + (l - p) In(x - ux)

> deriv[ 1] :=diff(c{ 1] ,u);

derivl:=
px (1 - p)x
x+ux x-ux
> mu[ 1] :=solve(deriv[ 1] ,u);
1'1 := 2 p - 1
The value function VI (x) is obtained in tenns of logarithms that can be written as
VI (x) = C +Iog(x) where C = log(2)+ p 10g(p)+log(1- p)- p 10g(1- p) =
log(2) + p log(p) + q log(q).
> V[ 1] :=unapply(expand(simplify(
subs (u=mu[ 1] , c{ 1] » ) , x) ;
VI := x ~ p In(p) + In(2) + In(x-) + In(1 - p) - p In(1 - p)
When there are two plays left, the optimal fraction to bet is again 1'2 =P - q
with the value function assuming the fonn V2(X) = 2C + log(x).
> c{ 2] : =p* V[ 1] (x+u* x) +q* V[ 1] (x-u* x) ;

C2 := p (p In(p) + In(2) + In(x + ux) + In(1 - p) - p In(1 - p»

+ (l - p) (p In(p) + In(2) + In(x - u x) + In(1 - p) - p In{l - p»
> deriv[ 2] :=diff(c[ 2] ,u);
 6.6 A Gambling Model with Myopic Optimal Policy 227

deriv2:=
px (1 - p)x
x+ux x-ux
> mu[ 2] :=solve(deriv[ 2] ,u);
#2:=2p-1
> ~ 2] :=unapply(expand(simplify(
subs (u=mu[ 2] ,e{ 2] ))) ,x);
V2 := x -+ 2 P In(p) - 2 p In(1 - p) + 2 In(2) + In(x-) + 2 In(1 - p)
This procedure can be automated to produce similar results.
> for n from 3 to 4 do:
> e{ n] :=p*~ n-l] (x+u*x)+q*~ n-l] (x-u*x):
> deriv[ n] :=diff(e{ n] ,u):
> mu[ n] :=solve (deriv[ n] ,u) :
> ~ ~ : =unapply (expand (simplify (
subs (u=mu[ n] ,e{ n] ))) ,x):
> od;

C3 := p (2 P In(p) - 2 p In(1 - p) + 2In(2) + In(x + ux) + 2 In(1 - p»+

(1- p)
(2 P In(p) - 2 p In(1 - p) + 2 In(2) + In(x - ux) + 2 In(1 - p»

deriv3:= px (1- p)x

x+ux x-ux
#3 := 2p-1
V3 := x -+ 3 P In(p) - 3 p In(1 - p) + 3 In(2) + In(x-) + 3 In(1 - p)

C4 := p (3 P In(p) - 3 p In(l - p) + 3In(2) + In(x + ux) + 3 In(l - p»+

(1- p)
(3 p In(p) - 3 p In(1 - p) + 3 In(2) + In(x - u x) + 3 In(l - p»

deriv4 := px (I - p)x
x+ux x-ux
#4:= 2p-1
V4 := x -+ 4 P In(p) - 4 p In(l - p) + 4 In(2) + In(x-) + 4 In(1 - p)
!
Using induction, it can then be shown that when p > the optimal policy is
to bet #n = P - q fraction of the current wealth at any stage of the game. This
gives rise to a value function that is in the form Vn(x) = nC(p) + log(x) where
C(p) = q In(q) + p In(p) + In(2).
One of the most interesting features of this problem is the nature of its solution:
The optimal strategy is myopic (invariant) in the sense that regardless of the num-
ber of bets left to place (n) and the current wealth (x), the optimal (nonnegative)
!
fraction to bet in each period is the same, i.e., u = p - q when p > and u = 0
228 6. Dynamic Programming

!.
when p ~ Optimal myopic policies of this type are usually difficult to obtain,
but they exist for some models. For example, in some periodic-review inventory
problems with random demand it has been shown that provided that some terminal
cost is appended to the objective function, the optimal order quantity is the same
for all periods; i.e., the policy is myopic (Veinott (188], Heyman and Sobel [89,
Chapter 3]). In some dynamic portfolio problems (Mossin [133], Bertsekas [31,
pp. 152-157]) myopic policy is also optimal provided that the amount invested is
not constrained and the investor's utility function satisfies certain conditions.
In a recent paper, C;etinkaya and Parlar [41] argue that the assumption of the
simple logarithmic function in the Kelly-Bellman-Kalaba model is somewhat un-
realistic and that the problem should ideally be solved with a more generalloga-
rithmic utility function. They thus assume that, in general, the gambler's terminal
utility is given as Vo(x) = log(b + x) where b is a positive constant. This gener-
alization results in the disappearance of the myopic nature of the solution and the
optimal strategy assumes a form that depends on the stage (n) and the state (x) of
the gambling process.

6.7 Optimal Stopping Problems

Let us suppose that a professor has N days to leave a country where he has spent
the past year on sabbatical. Before leaving the country he wants to sell his car and
hence places an advertisement in the local newspaper.
After the publication of the advertisement each day he receives random offers
W(), WI, ..• , WN-I, which are independent and identically distributed. If the pro-
fessor accepts an offer, then the process ends; however if he rejects the offer, he
must wait until the next day to evaluate the next offer. We assume that offers that
are rejected in the past are not retained-the potential buyer goes elsewhere. (For
the case where past offers are retained, see Bertsekas [31, p. 173].) The problem
is to find a policy that will maximize the professor's revenue when he leaves the
country.
Problems of this type are known as Slopping-rule problems and their origin can
be traced back to Wald's research on sequential analysis in the 1940s [192]. For
later surveys of stopping-rule problems see Breiman [37] and Leonardz [121].
The second part of Howard's delightful review of dynamic programming [95]
discusses an elementary stopping-rule problem (called an "action-timing" prob-
lem by Howard). For a rigorous exposition of the theory of stopping rules, Chow,
Robbins and Siegmund [43] can be consulted. Applications in economics of job
search are discussed in Lippman and McCall (123], [124].
We let Xt = Wt-I be the state variable at the beginning of period I (or, equiv-
alently, at the end of period I - I). Defining Vr (Xt) as the value function, the DP
functional equation is obtained as

V. ( ) { Xt if the decision is to accept the offer of Wt-I

t Xt = max E[ v,+ I (Wt)] if the decision is to reject the offer of Wt-I.
 6.7 Optimal Stopping Problems 229

Note that if the last offer W,_I is accepted, then at the start of period t, the pro-
fessor will have a revenue of x, and the process will "stop." If the offer is re-
jected, then V,(x,) is equal to the expected value of continuing optimally in peri-
ods t + I, t + 2, ... , N, i.e., E[V,+I (w,)] = .io
V,+I (w) dF(w) where F(w) is
the distribution function of the offer w.
Thus, the optimal policy is to accept the offer x, = W,_I in period t if x,
exceeds a critical threshold level of(J, = E[V,+I(W,)] and reject the offer if it
falls below (J" i.e.,

{ Accept if x, > 8,
JJ,(x,) = Reject if x, < (J" (6.6)

with either acceptance or rejection being optimal if x, = (J,. Hence the value
function is V,(x,) = max (x" (J,). Note that since xo == 0, we have Vo(O) =
max(O, (Jo) = (Jo.
As the threshold value (J, at time t is obtained in terms of E[V,+I(W,)], we
can develop a difference equation in terms of this value and compute (J" t =
0, I, ... , N recursively using Maple.
To obtain the difference equation, we write
(J, = E[ V,+I (w,)] = E[max(xt+l, (Jt+I)] = E[max(w" (Jt+I)]
= (Jt+1 t
(01+1 dF(w) + JO wdF(w)
10 101+1
= (Jt+IF«(J'+I) + roo wdF(w)
10/+1

with the boundary condition being (IN = 0 (since the professor would be willing
to accept any offer on his last day). Thus, the threshold levels are found as the
solution of the nonlinear difference equation

(J, = (Jt+1 F«(Jt+I) + A«(Jt+I), t = 0, I, ... , N - I

with A«(Jt+I) = 1000 wdF(w) and (IN = o. We now discuss some special cases
where the distributf~~s of the random variables assume specific forms.
As a simple example, assunie first that the professor has N = 5 days to leave
the country and his departure is scheduled for next Friday morning. Thus the
process of receiving offers starts on day t = 0 (Sunday) and the first decision is
made on Monday morning. Ifwe assume that the offers are distributed uniformly
between 0 and I, then their density is f (w) = I for 0 ~ w ~ 1. With these
data we find F«(Jt+I) = (Jt+1 and A«(J,+I) = ! - !(J~+I which gives rise to the
following difference equation:
I 2
(J, = 2(1 + (J'+I)' t =0, 1, ... ,4, (IN =0.

Unfortunately, Maple is unable to solve this nonlinear difference equation in

closed form with the rsol ve () function:
230 6. Dynamic Programming

> restart:. UniformDP.mws

> rsolve({ theta(t) = (1/2)* (1+(theta(t+1)A2)),
theta (5) =O} ,
theta);

rsolve({O(/) = 21 + 21 0 (1 + 1)2, 0(5) = OJ, 0)

However, the 0, values for I = 0, 1, ... , N - 1 can easily be computed numer-
ically as follows.
> restart:. UniformDP.mws (Part 2)
> Digits:=4:
> f : =w->l;. Uniform w
!:= 1
> N:=5; theta{ N] :=0;
N :=5
Os :=0
> for t from N-1 by -1 to 0 do:
> Ft: t+1] :=int(f,w=O •• theta{ t+1]):
> A[ t+1] :=int(w*f,w=theta{ t+1] .. 1):
> theta{ t] : =evalf (theta{ t+1] * Ft: t+ 1] +A[ t+ 1] ) :
> od:
> seq (theta{ t] ,t=O .. N-1) ;
.7751, .7417, .6953, .6250, .5000
Thus, we see that at the start of the process, the professor expects to gain 0.7751
monetary units ifhe uses the optimal policy p,(x,) that is prescribed in (6.6).

6. 7.1 The Infinite-Stage Problem

What happens if the horizon N is large? Does the sequence 0, approach a limit
for large N? We now examine these issues.
First note that for large N, we must discount the future rewards, or the thresh-
old levels would approach the maximum possible value attainable by the random
variable representing the offer. For example, for the uniform example we would
have 0, ~ 1 as N gets large. This would be a strange result since in that case the
optimal policy would nol allow accepting any offers.
Now using discounting and assuming that 1/( I + r) is the discount factor per
period, we can write V,(x,) = max{x" (1 +r)-I E[Vt+1 (w,)]), 1= 0,1, ... , N-
1, with VN(XN) = XN, which gives 0, = (1 + r)-I E[V,+I (w,)]. It can then be
shown using induction that [31, p. 161]

V,(x) ~ V,+I (x) for all x ~ 0 and I.

 6.7 Optimal Stopping Problems 231

Since 0, = (1 + r)-I E[Vt+1 (w»), we obtain 0, ~ 0t+I . This means that as the
end of horizon gets one period closer, the professor would be willing to accept
a lower offer since he has one less chance of getting an improved offer. For the
discounted case, the nonlinear difference equation assumes the form

0, = (1 + r)-I F(Ot+I) + (1 + r)-I A(O,+I), t = 0, 1, ... , N - 1.

Now, 0 ~ F(O) ~ 1 for all 0 ~ OandO ~ ~ wdF(w) ~ E(offer) < 00 for

all t. Thus, using the property that 0, ~ 0t+ I, ::~ obtain an algebraic equation in
terms of a constant 0 whose solution would give us the limiting value of 0:

(1 +r}1J = OF(O) + 1000 wdF(w).

For the uniformly distributed offers with a discount rate of r = 0.10, the limit-
ing result is obtained simply as the solution of a quadratic equation:
> restart: *UniformDP.mws (Part 2)
> r:=O.10;
r := .10
> F:=theta;
F:=O
> A:=int(w,w=theta .. l);
1 1
A := 2 - 202
> Asy:=(l+r)*theta=theta*F+A;
1 1
Asy:= 1.100 = _02 +-
2 2
> solve(Asy,theta);
.6417424305, 1.558257570
Thus, we obtain 0 = 0.641 as the number of opportunities (periods) to receive
offers approaches infinity.
Let us now consider a more general case where the offers have a beta distribu-
tion so that the density f(w) assumes the form

f( w) = r(a+b) wa- I (I- w)b-I, forO < w < 1

fWfW - -

where f(y) is the gamma function defined by f(y) = 10 uy-Ie-u du.lt is well
known that when y is a positive integer, we obtain f(y) = (y - I)', and that
when a = b = 1, the beta density reduces to the uniform so that f(w) = 1 for
O~w~1.
The Maple output for a = 5 and b = 2 follows where for N = 5, we compute
the threshold levels 0" t = 0, I, ... , 4 .
232 6. Dynamic Programming

> restart: # BetaDP.mws

> Digits:=3;
Digits:= 3
> a:=5; b:=2;
a:= 5
b :=2
> Mean:=a/(a+b);
5
Mean:= ;;
> f:=GAMMA(a+b)/(GAMMA(a)*GAMMA(b))*
w" (a-1)* (l-w)" (b-1);
f := 30 w 4 (1 - w)
> N:=5; thet~m :=0;
N:=5
85 :=0
> for t from N-1 by -1 to 0 do;
> El t+1] : =int (f, w=O •. theta[ t+1] ) ;
> A[ t+1] :=int(w*f,w=theta[ t+1] .. 1);
> theta[ t] : =evalf (theta[ t +1] * El t +1] +A[ t +1] );
> od:
> seq(thet~ ~ ,t=O .. N);
.838, .830, .806, .765, .714, 0
We find the limiting value 1J = 0.704 (with r = 0.10) as the unique real valued
root of a polynomial of degree 5:
> restart: # BetaDP.mws (Part 2)
> Digits:=3;
Digits:= 3
> r:=0.10;
r:= .10
> a:=5;b:=2;
a:= 5
b :=2
> f:=GAMMA(a+b)/(GAMMA(a)*GAMMA(b))
*w"(a-1)* (1-w)"(b-1);
f := 30 w 4 (1 - w)
> F:=int(f,w=O .. theta);
 6.8 Summary 233

F:= -506 + 60 5
> A: =int (w* f, w=theta •. 1) ;

A := ~ + 370 07 - 506
> Asy:=(l+r)*theta=theta*F+A;
5 30
Asy:= 1.100 = 0(-506 + 6( 5 ) + 7 + 1"87 - 506
> fsolve(Asy,theta);
.704

Compared to the limiting case 1J = 0.641 obtained for uniformly distributed

offers (with mean 112), the beta-distributed offers with a mean of a/(a+b) = 5/7
have higher threshold levels. This should be expected since with the higher mean
of the latter case, the decision maker can afford to be picky and reject offers that
are not high enough.

6.8 Summary
Dynamic programming (OP) is useful in solving different types of sequential de-
cision problems with Bellman's principle of optimality. We started this chapter
by introducing a simple discrete-time, discrete-state sequential decision problem
known as the stagecoach problem. Sequential decision problems with a quadratic
performance criterion and a linear system normally give rise to closed-form s0-
lutions that may require an iterative process of solving a sequence of recursive
equations. Using Maple's symbolic manipulation capabilities, we were able to
develop closed-form solutions for a large class of such problems. A finite horizon
workforce planning problem with constraints was solved with the help of Maple's
piecewise () function. A gambling problem modeled using stochastic DP was
solved that gave rise to a myopic policy. Finally, optimal stopping problems were
considered where the stopping boundary was computed by solving a nonlinear
difference equation numerically.

6.9 Exercises
1. Consider the following road network where a driver can travel only on the
one-way streets as indicated by arrows. The driver starts at the intersection
labelled ''A;' and must reach the final destination at the intersection labelled
"H". The travel distances between any two intersections are indicated next
to the arrows. Define the states and decisions and find the optimal route for
234 6. Dynamic Programming

the driver.

A ~S D ~3 E ~7 H
,1.3 t6 ,1. 1 t2
B ~8 C ~2 F ~3 G

2. Suppose we are given a positive quantity x. Use dynamic programming

to divide x into n parts in such a way that the product of the n parts is
maximized.

3. Consider the following optimization problem:

N
min J = L(qx; +ru;) +qxj
;=0

subject to the state dynamics

XHI = ax; + bu;,; = 0, ... , N - 1

where N = 2, q = 2, r = 4, a = 1 and b = 0.5.

(a) Use dynamic programming to solve this problem.
(b) The same problem appeared as a nonlinear programming problem in
Exercise 10 of Chapter S; see page 191. Comment on the nature of the
solution produced by the two approaches.

4. Consider the problem of minimizing J = L~[u7 + (Xt - Ut)2] subject

to the state dynamics XH I = a(xt - Ut) with 0 < a < 1 and 0 :s Un :s Xn,
n = 0, I, 2, .... Find the optimal policy and the value fimction for this
infinite-stage problem.

S. Use the Hamilton-Jacobi-Bellman equation to minimize the performance

criterion J = !x 2(T) + I[
!u 2 (t) dt subject to the state dynamics x'(t) =
x(t) + u(t) where T < 00.

6. Suppose we need to purchase a particular commodity within the next N

days. Successive prices Wt for this commodity are random and i.i.d. with
density f(Wt). Determine an optimal policy to purchase this commodity
in order to minimize the expected cost assuming that Wt are distributed
uniformly between 0 and I.

7. Consider a machine whose probability of failure depends on the number of

items it has produced. Assume that if the machine fails while producing an
item, there is a repair cost of Cf. The machine can be overhauled before it
fails at a cost of Co < Cf. If the machine is repaired or overhauled, then it is
considered good as new. After the production of each unit we may decide
 6.9 Exercises 235

to overhaul the machine or attempt to continue with the production of the

next unit.
Define Vn(x) to be the minimum expected cost of producing n more units
when the machine has already produced x parts. Let p(x) be the proba-
bility of the machine failing while producing the (x + l)st unit given that
it already has produced x units. Develop the DP functional equations for
Vn(x) and use the following data to compute VI (x), V2(X), ... , V6(X) for
x = 0, 1,2, ... ,5.

(0) p(I) p(2) p(3)

o 0.2 0.3 0.5

8. A set of items is called "group-testable" if for any of its subsets it is possi-

ble to perform a simultaneous (group) test on the subset with an outcome
of "success" or "failure"; see Bar-Lev, Parlar and Perry [14]. The "success"
outcome indicates that all the tested units are good, and the "failure" out-
come indicates that at least one item in the tested subset is defective without
knowing which (or how many) are defective.
In industrial problems that involve, e.g., the identification of good electronic
chips in a large batch that contain good and bad ones, chips of 1000/0 quality
cost much more than the chips of l00q% quality where q is a positive
constant that is usually greater than 0.9 but strictly less than 1.
Let K be the set-up cost for each test, u be the group size decision variable,
D be the requirement (demand) for 1000/0 quality items, N be the number of
l00q% units purchased. Define the state variables x as the number of units
of demand that is yet to be satisfied, y be the number of untested units. Let
q be the probability that a given unit in the group is "good" and 1r be the
unit cost of having a shortage of 100% quality items after the testing ends.
Define V(x, y) as the minimum expected cost (until the process terminates)
when we are currently at state (x, y) and an optimal policy is followed until
termination. Write down the DP functional equation for the value function
V (x, y) and solve the problem using the following data:

D N K 1r q
3 5 20 100 0.95
7
Stochastic Processes

7.1 Introduction
Operations research models can be categorized broadly as deterministic and prob-
abilistic. When the variable of interest-e.g., the number of customers in a service
system or the number of units in inventory-randomly evolves in time, a proba-
bilistic model provides a more accurate representation of the system under study.
A collection of random variables indexed by a parameter T (such as time) is
called a slochaslic process (or "random" process). When the process is observed
at discrete time points 0, 1,2, .. . , the process is usually denoted by {Xn, n =
0,1, ... }. The number of students enrolled in an MBA-level operations research
course at the start of each academic year is an example of a discrete-time stochas-
tic process. When the index parameter is continuous the process is usually denoted
by (X(/), 1 ~ OJ. An example of a continuous-time stochastic process is the price
of a company's stock observed at time I.
Stochastic processes are also characterized by their state space S, i.e., the val-
ues taken by Xn (or by X(/». If the set S is countable, then the process has a
discrete state space. For example, if X(/) represents the number of customers in
the system at time I, then S is countable and X(t) has a discrete state space. If Sis
a subinterval of the real line (an uncountable set), then the process has continuous
state space. For example, if X(/) measures the water content ofa dam, then X(t)
has a continuous state-space. Thus, a stochastic process can be placed in one of
four possible categories depending on the nature of its parameter set T (discrete
vs. continuous) and state space S (discrete vs. continuous).

M. Parlar, Interactive Operations Research with Maple

© Birkhäuser Boston 2000
238 7. Stochastic Processes

The theory of stochastic processes plays a very important role in operations

research. Many realistic problems arising in queueing, inventory, reliability and
maintainability and other areas are modeled using stochastic processes as the fu-
ture values of the variables representing the evolution of these systems are not
known in advance. For an excellent coverage of applications of stochastic pro-
cesses in queueing theory, see Cooper [50], Gross and Harris [76], Medhi [130]
and Wolff [196]. One of the classical texts in stochastic inventory theory is by
Hadley and Whitin [81]. Arecent monograph by ~ahin [163] provides a rigorous
analysis of regenerative stochastic inventory systems. Porteus [151] gives a very
readable account of the recent results in stochastic inventory theory. For a review
of the applications of stochastic processes in reliability and maintainability, see
Shaked and Shantikumar [170].
Several excellent textbooks provide a general overview of the theory and appli-
cations of stochastic processes, e.g., Bhat [33], C;mlar [47], Cox and Miller [53],
Heyman and Sobel [88], Kao [101], Karlin and Taylor [102], Medhi [131], Ross
[157] and Tijms [186].
We will start our discussion of stochastic processes by considering a very sim-
ple continuous-time, discrete state space process known as the Poisson process.
Then, we will discuss renewal theory that generalizes the Poisson process. This
will be followed by discrete- and continuous-time Markov chains.

7.2 Exponential Distribution and Poisson Process

In this section we first describe the properties of exponential distribution as it is
intimately related to the Poisson process.
Suppose T is an exponential random variable with parameter A. > 0 and density

[(I) = A.e-J.t, 1 ~ O.

As the following Maple results demonstrate, for the exponential r.v., the cumula-
tive distribution function is Pr(T ~ I) == F(/) = f~ A.e- Ax dx = 1 - e-A.t and
the survivor probability is Pr(T > I) == F(/) = 1,00 A.e- Ax dx = e-A.t. If, for
example, T is the lifetime of a component, then F(/) gives the probability that
the component will fail at or before 1 time units and the survival probability F(/)
gives the probability that the component will fail after 1 time units.
> restart: * exponential.mws
> assume(lambda>O);
> f:=unapply(larnbda*exp(-lambda*t),t);
[:= 1 -+ A.e(-A.t)
> F:=unapply(int(f(x),x=O .. t),t);
F:= 1 -+ _e(-A.t) + 1
> FBar:=unapply(l-F(t),t);
 7.2 Exponential Distribution and Poisson Process 239

FBar := t ~ e(-A I)

7.2.1 Memoryless Property of the Exponential Distribution

We are interested in computing the conditional probability that the component
will fail after s + t time units given that it has already worked for more than t time
units. This can be written as

Pr(T > s +1 IT > Pr(T > s + t and T > t) Pr(T > s + t)

I) = =.
Pr(T > t) Pr(T > I)

Evaluating this expression, Maple gives

> Conditional:=FBar(s+t)/FBar(t);
e(-A (HI»
Conditional := e(-A ,)
> simplify(Conditional);
e(-As)

Thus, the conditional probability is obtained as

Pr(T> s +1 IT> s) =e- As •

Comparing this to the probability Pr(T > s) that the (new) component will fail
after s time units, we find
> FBar(s);
e(-As)

This implies that for the exponential random variable we have the interesting
property that

Pr(T > s + tiT> t) = Pr(T> S +t)

Pr(T> t)
= Pr(T > s)
or
Pr(T > s + t) = Pr(T > t)Pr(T > s). (7.1)
This is a very significant result-known as the "memoryless" property of the
exponential-as it states that distribution of the remaining life of the component
does not depend on how long it has been operating. In other words, given that
the component has operated for t time units, the conditional probability that it
will operate for at least s time units is independent of how long it has been in
operation. It is this memory less property of the exponential distribution that fa-
cilitates the analysis of a large number models in queueing, inventory theory, etc.
For example, the Markovian models presented in Chapter 9, Queueing Systems,
make the assumption that either the interarrival or the service time is memory less,
thereby considerably simplifying the analysis.
240 7. Stochastic Processes

Writing (7.1) as F(s + I) = F(/)F(s) gives a functional equation with the

unknown function FO. It can be shown that the only solution that satisfies this
equation is of the fonn F(/) = e- At ; thus the only continuous memoryless distri-
bution is the exponential. For a proof of this result, see Ross [157, pp. 24-25].

7.2.2 Hazard Rate Function

Suppose we now wish to compute the conditional probability that the component
will fail in the short time interval (/,1 + d/] given that it has already survived t
time units. Denoting the lifetime r.v. of the component by T, this probability is
written as
Pr(t < T ::: t + dl and T > I)
Pr(t < T ::: 1 + dl IT> I) =
Pr(T> t)
Pr(t < T ::: 1 + d/)
=
Pr(T> I)
~ /~) dl = r(/) dl
F(t)

where r(/) = /(/)/ F(/) is the hazard (failure) rate function of the component.
Naturally, if /(/) = Ae- At , then we find that r(/) = A, a constant. For most
mechanical equipment, the hazard rate function would be an increasing func-
tion of time (Shaked and Shantikumar [170)). In manpower planning applica-
tions, Bartholomew [16] has shown that the hazard rate function for the completed
length of service (CLS) distributions estimated for the British Civil Service is a
decreasing function of time. This implies that the longer a civil servant works for
the government, the smaller is the probability that he will resign his job and move
elsewhere.
Clearly, for a given density /(/), the hazard rate function r(/) can be uniquely
detennined from the equation r(/) = /(/)/ F(/). For example, suppose T is a
Weibull r. v. with distribution F and density /:
> restart: *
weibull.mws
> F: =unapply(l-exp(-lambda*xAa),x);
F :=x ~ I-e(-h")
> f:=unapply(diff(F(x),x),x);
AXQ ae(-Ax")
/:=x~-----
x
Using r(/) = /(/)/ F(t) the hazard rate function r is found as
> r:=unapply(f(x)/(l-F(x)),x);
AxQ a
r : =x~-­
x
 7.2 Exponential Distribution and Poisson Process 241

We now deal with the inverse problem: Does the hazard rate function r(t)
uniquely determine the density /(t)? The answer to this is in the affirmative and
it is obtained by solving a simple ordinary differential equation.
Recall that F(t)= 1,00 /(u) du, which implies F' (t) =-
/(t). Rewriting
r(t) = /(t)/ F(t) we obtain an ordinary differential equation F'(t) = -r(t)F(t)
with the initial condition F(O) = 1. Solving this ODE using Maple's dsol ve ( )
function, we find the expression F(t) in terms of the hazard rate function r(t).
> restart: i rODE.mws
> ODE:=diff(FBar(t),t)=-r(t)*FBar(t);
ODE := f, FBar(t) = -r(t) FBar(t)
> Solution:= dsolve({ODE,FBar(O)=l} ,FBar(t»;
Solution := FBar(t) = e(- f~ r(II)dll)

> assign(Solution);
Next, differentiating -F'(t) gives the unique solution for the density /(1) in
terms of the hazard rate function r(t).
> f:=unapply(-diff(FBar(t),t),t);
1 := 1 ~ r(l) e(- J~ r(II)dll)
Thus, given the hazard rate function r(t), the density 1(1) is obtained uniquely
by evaluating an expression involving an integral.

1(1) = r(t)e- J~r(II)dll. (7.2)

Returning to the Weibull example, we see that with the hazard rate function
r(l) = A.Xa-1a, the correct density /(t) of the Weibull is obtained from (7.2), as
expected.
> restart: i weibul12.mws
> F:=unapply(l-exp(-lambda*xAa),x):
> f:=unapply(diff(F(x),x),x):
> r:=unapply(f(x)/(l-F(x»,x):
> assume(a>O,lambda>O);
> f:=r(t)*exp(-int(r(x),x=O .. t»;
A. t a a e(-t" A)
/:=----
t

7.2.3 The Erlang Random Variable

Suppose 1i and T2 are two independent and identically distributed (U.d.) expo-
nential random variables (r.v.) with parameter A.. In a queueing example with two
242 7. Stochastic Processes

stages, each of these r.v.'s could correspond to the time spent in one stage of the
system. The total time spent in the system is then S2 = T, + T2.
To find the distribution of the new random variable S2, we load the int-
trans () package and compute the Laplace transfonn (LT) of the exponential
r.v., which we denote by fLT[ 1) .
> restart:. Erlang.mws
> with(inttrans);

[addtable, fourier, fouriercos, fouriersin, hankel, hilbert, inv/ourier,

invhilbert, invlaplace, invmellin, laplace, mellin, savetable]
> assume(lambda>O);
> f:=lambda*exp(-lambda*t);
f:= le(-At)
> fLT[ 1) :=laplace(f,t,s);
l
fLT, :=--1
S+A
Since S2 is the sum of two r.v.'s, the LT of S2 (denoted by fLT[ 2) ) is obtained
as the product of the LT of the individual exponential r.v.'s:
> fLT[ 2) : =fLT[ 1) * fLT[ 1) ;
l2
fLT2 := (s + l)2
Inverting the resulting transfonn, we obtain
> invlaplace(fLT[ 2) ,s,t);
l2 t e(-At)
which is the density of the Erlang r.v. with parameters (2, A). Thus, the r.v. S2 has
the density
fSz(t) = l2 te-At, t ~ o.

Continuing with this approach, we now fonn the new sum S3 = 2.7=, T; whose
LT-denoted by fLT[ 3) -is found as
> fLT[ 3) :=(fLT[ 1) )"3;
l3
fLT3 := (s + l)3
Inverting this LT, we find
> invlaplace(fLT[ 3) ,s,t);

~ l3 t2 e(-At)
2
 7.2 Exponential Distribution and Poisson Process 243

as the density of Erlang with parameters (3, A.). Thus, we see that the r.v. S2 has
density
/~(I) = ~A.312e-At, I ~ O.
2
Generalizing these results to a positive integer n by using induction it can be
shown that the finite sum Sn = "L7=1 T; has the Erlang density
A.e- At (A.I)n-1
/s,,(I) = (n-I)! ' I ~0

with parameters (n, A.).

7.2.4 Poisson Process

Suppose we "count" the total number of events N(I)-e.g., customer arrivals--
that have occurred up to time I . Such a counting process has the properties that (i)
it is nonnegative and integer valued, (ii) if s < I, then N(s) :5 N(I), and (iii) for
s < I, the quantity N (I) - N (s) equals the number of events that occurred in the
interval (s, I].
A counting process N(I) is said to have independent incremenls if the (ran-
dom) number of arrivals in two disjoint intervals are independent. The slalionary-
incremenl property of a counting process says that the distribution of the number
of events depends only on the length of the interval, not on the position of the
interval.
The Poisson process {N (I), I ~ O} with rate A. > 0 is a special type of counting
process with the properties that (i) N(O) = 0, (ii) N(I) has independent and
stationary increments, (iii) for a short time interval oflength h, Pr(N(h) = I] =
A.h + o(h) and (iv) Pr[N(h) ~ 2] = o(h), where o(h) is the "little-oh" function
that approaches zero faster than h, i.e., limh-+O o(h) / h = O.
Based on these four properties, we now show using Maple's help that the num-
ber of events in an interval of length I has the distribution
(A.t)ne-At
Pr(N(I) = n] = Pn(l) = , n = 0, 1,2, ...
n!
with mean A.I.
We first start with n = O. Using the properties of the Poisson process, we can
write an expression for the probability of having no events between (0, 1+ h]:
pO(1 + h) = Pr[N(1 + h) = 0]
= Pr[N(I) = 0 and N(I + h) - N(I) = 0] (independence)
= Pr[N(I)= 0] Pr(N(1 + h) - N(I) = 0] (stationarity)
= pO(I)[1 - A.h + o(h)].
Subtracting PO(I) from both sides of the last equation, dividing by h and letting
h -+ 0 gives
PO(I) = -A.pO(l)
244 7. Stochastic Processes

with the initial condition po(O) = I since we assume that initially the system is
empty, i.e., N (0) = O. Solving this ODE with Maple gives
> restart: *
Poisson.mws
> de.O:=diff(p.O(t),t)=-lambda*p.O(t);
deO := f, pO(/) = -). pO(/)
> IC.O:=p.O(O)=I;
ICO := pOcO) = I
> assign(dsolve({de.O,IC.O} ,{p.O(t)} ));
> p.O(t);
e(-At)

That is, PO(/) = e- At •

For n = I we have

PI(I + h) = PI (/)[1 - )'h + o(h») + PO(/)[).h + o(h»).

Subtracting PI (I) from both sides of the last equation, dividing by h and letting
h ~ 0 gives a new ODE

with the condition PI (0) = I. Maple solves this ODE as

> de.I:=diff(p.I(t),t)=-lambda*p.I(t)+lambda*p.O(t);
del := :, pl(/) = -). pl(/) + ).e(-At)
> IC.I:=p.I(O)=O;
ICI := pi (0) = 0
> assign (dsolve ({ de. I, IC.I} ,{ p.l (t)} ));
> p.l(t);
e(-A t) ).1

That is, PI (I) = e- At At.

In general, for n = 1,2,3, ... , we obtain a set of differential-difference equa-
tions as

Pn(1 + h) = Pn(/)[1 - )'h + o(h») + Pn-I (/)[).h + o(h»).

With the usual operations of subtracting pn (I) from both sides, dividing both sides
by h and letting h ~ 0, we find an infinite system of ODEs (i.e., differential-
difference equations)

p~(/) = -).Pn(/) + ).Pn-I (I), n = 1,2,3, ...

 7.2 Exponential Distribution and Poisson Process 245

We now again use Maple to solve these ODEs and obtain the solution for a few
small values of n:
> for n from 2 to 10 do
> de.n:=diff(p.n(t),t)=-lambda*p.n(t)
Tlambda*p. (n-1) (t);
> IC.n:=p.n(O)=O;
> assign (dsolve ({ de.n, IC.n} ,{ p.n (t)} »;
> p.n(t);
> od:
> p.2(t); p.3(t); p.4(t); p.5(t);

! e(-J.t) .(2 t2
2
! e(-J.t) ,l.lt3
6
J.. e(-J. t) ,l4 t 4
24
_1_ e(-J. t) ,l S tS
120
To show the general result, we use mathematical induction and assume that the
result is true for n = k, i.e., that

is true.
> p.k(t) :=exp (-lambda* t)* (lambda*t)Ak/k!;
e(-J.t) (,It)k
pk(t) := k!
With this assumption we attempt to solve the ODE for n = k + 1, i.e.,

and hope to find

e-J.t(..l.t)k+l
Pk+l(t)= (k+l)! .

(We denote k + 1 by kPlus1 in the Maple worksheet.)

> de. (kPlus1) :=diff(p. (kPlus1) (t),t)=
-lambda*p. (kPlus1) (t)
+lambda*p.k(t); IC. (kPlus1):=p. (kPlus1) (0)=0;
,l e(-J.t) (,l t)k
dekPlusl := %t pkPlusl(t) = -,l pkPlusl(t) + k!
246 7. Stochastic Processes

ICkPlusl := pkPlusl(O) = 0
> assign(dsolve({de. (kPlusl),IC. (kPlusl)},
{p. (kPlusl) (t)} ));
> p. (kPlusl) (t);
e(-At) (A I)(HI)

f(k + I)(k + I)

Maple solves the ODE with the initial condition PH 1(0) = I and finds the correct
result
e- At (Ali+) e- At (Ali+ 1
PHI (I) = f(k + I)(k + I) = (k + I)! '
thus proving the proposition.

7.2.5 lnterarrival Times ofthe Poisson Process

There is a very important property that relates the Poisson process to its inter-
arrival times. Suppose we have a Poisson process and T" is the time between
the arrival of the nth and (n - I )st event, n = I, 2, .... We now show that T",
n = 1, 2, .. . is exponentially distributed.
First we compute the survivor probability Fh (I) = Pr(TI > I) of the arrival
time of the first event TI. Since the events {TI > I} and {N(/) = O} are equivalent
(the first event takes place after time I if and only if there are no arrivals until I),
we can write
- (A/)O
FT. (I) = Pr(TI > I) = Pr(N(/) = 0] = e -At Or = e-At .
This implies that Jr. (I) = -FT. (I) = Ae- At , i.e., TJ is exponential with parame-
ter A.
To find the distribution of n, we condition on the arrival time of the first event
TI :

Pr(T2 > I I TI = s) = Pr{no events in (s,s + I] I TI = s)

= Pr{no events in (s, s + t]) (independence)
= e- At (stationarity)
Repeating the arguments we see that the nth interarrival time T", n = I, 2, ... is
distributed exponentially with parameter A.

7.2.6 Density ofthe Erlang-An Equivalent Derivation

In Section 7.2.3 we showed using Laplace transforms that the density of the Er-
lang random variable SrI = L7=1 T; with parameters (n, A) was
Ae- At (A/),,-I
!s,,(/) = (n _ I)! ' I ~ O.
 7.2 Exponential Distribution and Poisson Process 247

We now show the same results using a much simpler argument.

It is easy to see that the two events {S" ~ t} and (N(t) 2: n} are equivalent
since the nth event takes place at or before time t if and only if the number of
events until t is at least equal to n. Thus,

00 (..tty
Fs,,(t) = Pr(S" ~ t) = Pr[N(t) 2: n] = ~::e-.l.t-.,-.
j=" J.

Differentiating Fs,,(t) w.r.t. t and simplifying we find the density /s,,(t) = Fs.. (t)
as
> restart: # Erlang2.mws
> FSn:=Sum(exp(-lambda*t)* (lambda*t)Aj/j!,
j=n .. infinity) ;
00 e(-.l.t) (..t tY
FSn:= ~ .,
j=n J.
> fSn:=simplify(diff(value(FSn),t));
e(-.l.t) (A t)"
jSn:= t r(n)
Rewriting the result in a more familiar format, we obtain the density /s,,(t) of the
Erlang r.v. with parameters (n, A).

7.2.7 Nonhomogeneous Poisson Process

Very often, the arrival rate of a Poisson process may be time dependent. For ex-
ample, the number of vehicles arriving at a bridge or the number of customers
arriving at a restaurant may be different at different times of the day. Thus, it may
be more accurate to model such phenomena by assuming that the arrival rate A(t)
is a function of time t.
A nonhomogeneous Poisson (NHP) process N(t) with arrival rate A(t) is a
process with the following properties: (i) N(O) = 0, (ii) N(t) had independent
increments,! (iii) Pr[N(t + h) - N(t) = 1] = A(t)h + o(h) and (iv) Pr[N(t +
h) - N(t) 2: 2] = o(h).
The integrated arrival rate function for the NHP is defined as

M(t) = lot A(U) duo

This quantity is the mean value function of the process that gives the expected
number of arrivals until t. For an ordinary Poisson process it reduces to M(t) =
At.

'Note that an ordinary (homogeneous) Poisson process with constant). has stationary increments.
248 7. Stochastic Processes

For the NHP, the probability distribution of the number of arrivals in the interval
(t, t+ s] is still Poisson but with parameter M(t + s) - M(t), i.e.,
Pr{N(t+s)-N(t) = n] = e-[M(Hs)-M(t)] [M(t +s) - M(t)1 n , n = 0, 1,2, ...
n!
As in ordinary Poisson processes, we show this result using induction.
= =
For afixed t, define Pn(S) Pr{N(t + s) - N(t) n]. For n 0 we have =
pO(s + h) = Pr{N(t + s) - N(t) = 0]
= pO(s)[I - A.(t + s)h + o(h)]

with pO(O) = 1. The usual operations lead to an ordinary differential equation

with a variable coefficient given by

Po(s) = -A.(t +s)pO(s).

We now solve this ODE using Maple and show that pO(s) = e-[M(Hs)-M(t)].
> restart: *
NHP.mws
> de.O:=diff(p.O(s),s)=-lambda(t+s)*p.O(s);
deO:= :s pO(s) = -A.(t +s)pO(s)
> IC.O:=p.O(O)=l;
leo := pO(O) = I
> assign (dsolve ({ de. 0, IC. O} ,{ p. 0 (s)} ));
> p.O(s);
e(- J~ l(Hu)du)

Thus, we find that pO(s) = e- J~ l(Hu) du. It is not immediately obvious how the
function M(t) appears in this result, so we attempt to convert it to a different fonn
using a change of variable that requires the student package.
> with(student);

[D, Diff, Doubleint, Int, Limit, Lineint, Product, Sum, Tripleint, changevar,
combine, completesquare, distance, equate, extrema, integrand,
intercept, intparts, isolate, leftbox, leftsum, makeproc, maximize,
middJebox, middJesum, midpoint, minimize, powsubs, rightbox,
rightsum, showtangent, simpson, slope, summand, trapezoid, value
]
> Integral:=int(lambda(t+u),u=O .• s);

Integra/:= loS A.(t + u) du

> changevar(t+u=z,Integral,z);
 7.2 Exponential Distribution and Poisson Process 249

11
1+$
A(Z) dz

Since 1,1+$ A(Z) dz = M(t + s) - M(t), the result follows.

For n = I, 2, ... the differential-difference equations are obtained as

p~(s) = -A(t + s)Pn(s) + A(t + S)Pn-1 (s)

which can be solved in a similar manner. Using induction we obtain the required
result.

Example 72 lime-dependent arrival rate in a restaurant. Consider a restaurant

where customers arrive in different rates at different times of the day. The time-
dependent arrival rateA.(t) is conveniently specified using Maple's piecewise ()
function and displayed with the plot () function in Figure 7.1.
> restart: iNHPExample.mws
> lambda:=t->piecewise(t<=3,4+2*t,t<=5,lO,
t<=9,lO-(t-3»;
A := t ~ piecewise(t ~ 3, 4 +2t, t ~ 5, 10, t ~ 9, 13 - t)
> plot(lambda(t),t=O .. 9,O .. lO,discont=true);

10

o 2 4 t 6 8

FIGURE 7.1. Time-dependent arrival rate 1(1).

In this example, we assume that the restaurant opens at 8 a.m. and stays open
for 9 hours. Thus, t = 0 corresponds to 8 a.m. and t = 9 corresponds to 5 p.m.
Maple easily computes the integrated arrival rate function M(t) (denoted by
MIn t) as follows:
> MInt:=int(lambda(u),u=O .. t);
 250 7. Stochastic Processes

MIni := -~ t 2 Heaviside(1 - 5) - ~ Heaviside(t - 5) - 12 Heaviside(1 - 3)

- 9 Heaviside(t - 3) + 4 t + 12 + 3 I Heaviside(t - 5)
+ 1~3 Heaviside(1 _ 9) - 13 I Heaviside(1 - 9)

+ ~ t 2 Heaviside(t - 9) + 6 t Heaviside(t - 3)

I
Since the result is in tenns of Heaviside ftmctions defined as
0 ift < 0
Heaviside(t) = undefined if I = 0
1 ift > 0,

we convert it to the piecewise fonn and use the unapply () ftmction so that
M(/) can be evaluated for different values of I.
> M:=unapply(convert(Mlnt,piecewise,t),t);

M:= 1-+ piecewise(1 ~ 3, 4t + 12, I ~ 5, -9 + 10/, I ~ 9, -~ t 2 _ 2; + 13/,

9 < t, 65)

I-t/2
Thus, we find that
12 +4/, 0~/~3
M(t) = 10/ - 9, 3~t~5
+ 131- ¥, 5~/~9

which is plotted in Figure 7.2.

> plot(M(t),t=O •. 9,color=black);
Next we define the probability Pr[N(t+s)-N(/) = n] as PrNHP: = (t, s, n)
- > ... which is a ftmction of three variables (I, s, n). We use the arrow notation
so that this probability can be computed by entering the three inputs (I, s, n). For
a specific set of values (2, 2, IS) we find that the probability that IS customers
will enter the restaurant between the times I = 2 and I + s = 4 (i.e., between 8
a.m. and 12 noon) is found to be approximately 0.065:
> PrNHP:=(t,s,n)->exp(-(M(t+s)-M(t)))
* (M(t+s)-M(t))"n/n!;
e(-M(Hs)+M(t)) (M(I +s) - M(t))n
PrNHP:= (I, s, n) -+ ,
n.
> evalf(PrNHP(2,2,15));
.06504430041
Suppose we are not aware of the above results and attempt to find the probabil-
ity that 15 customers will arrive between the times I = 2 and I + s = 4 by using
the fonnula for the ordinary Poisson process. In this case Poisson's arrival rate A
 7.2 Exponential Distribution and Poisson Process 251

60

50

40

30

20

10

FIGURE 7.2. The integrated arrival rate function M(t) = fJ 1(u)du.

t
could be computed using the average arrival rate of AAvg = f~ A( r) dr = 7.22
customers. The Poisson probability formula. which would provide on1y an ap-
proximation to the true probability, gives 0.10 I.
> lambda[Avg] :=evalf(int(lambda(t),t=O •. 9)/9);
AAvg := 7.222222222
> Poisson:=(t,n)->exp(-lambd~Av~*t)
* (lambda[ Avg] * t) "n/n! ;
e( -).A>w t) (A t)n
Poisson:= (t, n) ~ , Avg
n.
> Poisson(2,15);
.10 13609134
> Error:=evalf((Poisson(2,15)-PrNHP(2,2,15))
/PrNHP(2,2,15));
Ernor:=.5583365914
This results in an error of about 55.8%, indicating that the Poisson model pro-
vides a poor approximation to the nonhomogeneous Poisson process.

7.2.8 Compound Poisson Process

Consider the following generalization of the ordinary Poisson process N(t) with
rate A. Let {Y;} be discrete-valued U.d. random variables with mean E(y), vari-
ance Var(Y) andthep.g.f. Gy(z) = E(zY) = L~OnZn whereOn = Pr(Y = n).
We assume that the Poisson process N (t) and the sequence {Y; } are independent.
252 7. Stochastic Processes

By combining the Poisson process N (I) and the sequence {Y;} in a particular
way, we obtain a new process
N(t)
X(t) = LY;. (7.3)
;=0

This new process X(t) is called the compound Poisson process.

Such a generalization may be useful in applications where each customer arriv-
ing in a store spends a random amount of Y dollars. In this case, X(t) would cor-
respond to the total amount spent by all customers until time t. In a maintenance-
type application Y may correspond to the number of hours of work done on a
particular vehicle in which case X(t) would be the total time spent on all vehicles
until t.
We will now compute the p.g.f. GX(t)(z) = E[zX(t)] of the compound Pois-
son process X(t) and find its mean and variance assuming that Y is geometric.
Conditioning on the number of Poisson events that take place until time t, we can
write
GX(t)(z) = E[zX(t)]
00

= LE [zX(t) I N(t) = i] Pr[N(t) = i]

;=0
00
= LE [Z Yt+Y2+ ..•+Y1 ] Pr[N(t) = i)
;=0
00

= L[E(zY)t Pr[N(/) = i)
;=0

= f[Gy(z)t e - At ()...~); .
;=0 I.

This infinite sum is easily evaluated by Maple to give

> restart: *CompoundP.rnws
> G[ Xl :=unapply(' sum ( (G[ Y] (z)) "i
*exp (-larnbda* t)* (larnbda*t) "i/i!,
i=O •• infinity)' , z);
00 Gy(z); e(-At) (11);
Gx:=z-+L .,
;=0 I.
> G[X] (z);
e(-At) e(Gy(:)At)

Hence, we see that the p.g.f. of the compound Poisson process X(/) is obtained
as GX(t)(z) = e-AteGy(=)At.
Now consider the case where Y follows the geometric distribution an = Pr(Y =
n) = (I - p)~-I, n = 1,2, ... :
 7.2 Exponential Distribution and Poisson Process 253

> restart: # Stutter.rows

> a[ n] :=(1-p)*p"(n-1);
on := (1 - p) p(n-I)
The p.g.f. of Y is easily found from Gr(z) = L~I onzn as
> ~ ~ :=unapply(suro(~ ~*z"n,n=l •• infinity),z);
(1 - p)z
Gr :=z ~
pz-I
> ~~ (z);

(I - p)z
pz-I
Next, evaluating the expression for the p.g.f. GX(t)(z) = e- At eGy(=)At of X(t).
we find
> G[ X] :=unapply(suro( (Q: ~ (z» "n
*exp (-larnbda* t)* (larnbda*t) "n/n!,
n=O .. infinity),z);
( -zAt+zAlp)
Gx:=z~e(-At)e pz I

Expected Value of X(t)

The first moment E[X(t)] of the X(t) process is equal to G:r(t)(l) (denoted
by G[ X, 1] ). To compute the first moment, we first ask Maple to differentiate
G X(t)(z) w.r.t. z and find
> ~X,l] :=unapply(diff(~X] (z),z),z);
-A) -It+1tp (-zlt+z1tp)p CzAI+zAIP)
Gx I := z ~ e( t ( - )e p:-l
, pz-I (pz_I)2
Substituting z = I and simplifYing. we obtain a simple expression for E[X(t)] as
> EX:=siroplify(Q:X,l] (1»;
1t
EX .-
·-
-I +p

Variance of X(t)
To compute the variance of X(t) we need the second factorial moment of X(t).
which is found by evaluating G X(t)(1) . The second derivative GX(t)(z) (denoted
by G[ X, 2] ) is a very complicated expression and it would be rather difficult to
compute it manually. But using Maple, this is achieved relatively easily. Comput-
ing GX(t)(z). substituting z = I and simplifYing, we find
> ~ X, 2] :=unapply (diff (Q: X] (z), z$2), z);
254 7. Stochastic Processes

(-A) (-).I+).lp)p (-Z).I+Z).lp)r C: A1t: A,,)

GX. 2 := Z ~ e I (-2 (pz _ 1)2 +2 (pz _ 1)3 )e p:-l
(-AI) -).I+).lp (-Z).I+Z).lp)p 2 C:A1-;:Alp)
+e ( - ) e p. I
pz-I (pZ_I)2
> simplify (C{ X, 2] (1));
).1 (2p + ).1)
(-I + p)2
Since the second moment of X(/) is E{[X(/)]2} = GX(I)(l) + GX(I) (I),
> EX2:=simplify(C{X,2] (1)+C{X,1] (1));
EXl:= ).1 (p +).1 + I)
(-I + p)2
we obtain the variance Var[X(/)] = E{[X(/)]2} - (E[X(/)]}2 of the compound
Poisson process X(/) simply as
> VarX : =simplify(EX2-EX A 2);
VarX := ).I(P+ I)
(-I + p)2

Distribution of X(/)
The results found in the preceding paragraphs provide the exact expressions for
the first few moments of the compound Poisson process for geometrically dis-
tributed Y . When Y is a nonnegative integer random variable, it is also possible to
compute symbolically the exact distribution of the X(/) process using a recursive
scheme due to Adelson [5]. Also see TIjms [187, pp. 29--30] who provides a faster
method based on the fast Fourier transform.
As before, let an = Pr(Y = n), n = 1,2, .. . be the density of Y . Defining
rn(/) = Pr[X(/) = n], n = 0, I, .. . as the probability density of X(/), Adelson's
method computes these probabilities as
ro(/) = e-AI(I-ao),
). n-l
rn(/) = ..!.. ~(n - j)an-jrj. n = 1, 2, ...
nj=O

Note that if, for example, Y corresponds to the number of units of a product pur-
chased by customers who arrive according to a Poisson process with rate )., then
X(/) is the total number of units purchased until time I andrn(/) is the probability
that exactly n units will be purchased by time I.
The recursive equations for, say, n = 0, I, 2, 3 are easily formed and computed
with Maple.
> restart : *
Adelson.mws
> Total:= 3;
 7.2 Exponential Distribution and Poisson Process 255

Tola/:= 3
> r[ 0] :=exp (-lambda* t* (l-a[ 0] ));

ro := e(-At(\-ao»
> for n from 1 to Total do
r[ n] := (lambda* t/n)* sum( (n-j)* a[ n"';'j] * r[ j] ,
j=O .. n-l) od:
The expressions for r) (I), r2(/) and r3(/) are found in tenns of the as yet un-
specified values of an, n = 0, 1,2,3.
> r[ 1]; r[ 2]; r[ 3] ;

A. 1 al e( -A t (\-ao»
1
- A. 1 (2Q2 e(-At (\-ao» + al 2 A. 1 e(-At (\-ao»)
2
1 1
3" A. 1 (3 a3 %1 + 2 Q2 A. 1 al %1 + 2" al A. 1 (2 Q2 %1 + al 2 A. t %1»
%1 := e(-At(\-ao»
If we asswne that Y is geometric, i.e., that an = Pr( Y = n) = (I _ p)p"-I ,
n = 1,2, ... , we can obtain the exact fonns of the probabilities rn(/), n = 1,2,3.
> a[ 0] :=0;
ao :=0
> for n from 1 to Total do a[ n] :=(l-p)*p"(n-l) od;
al := 1- p
Q2 := (I - p) p
Q):= (1- p)p2
> r[ 1]; r[ 2]; r[ 3] ;

A. 1 (l - p) e(-At)

1
-1t(3 (1 - p) p2 e(-At) + 2 (1 - pi pA.l e(-At)
3
+ ~ (1 -p)A. 1 (2 (1 - p) pe(-At) + (1 - p)2 A. 1 e(-At»)
2
We now let A. = 1 and p = 0.6 and plot the probability functions for rn(/),
n = 0,1,2,3. These functions are displayed in Figure 7.3.
> lambda:=l; p:=0.6;
A. := 1
256 7. Stochastic Processes

p:=.6
> plot ([ r[ 0] , seq (r[ n] , n=l .. 3)] , t=O .. 4, color=black) ;

0.8

0.6

0.4

0.2

o 3 4

FIGURE 7.3. Probability functions rn(t) = Pr[X(t) = n] for n = 0, 1,2,3 and 0 :5; t :5; 4
where rO(O) = I.

7.3 Renewal Theory

Renewal theory is concerned with extending the results found for the exponen-
tiallPoisson duality. As we showed in Section 7.2.5, in a Poisson process the dis-
tribution of the interarrival times are i.i.d. exponential random variables. When
the interarrival time distribution is non exponential, the resulting counting pro-
cess becomes a renewal process.
Because of its generality, renewal theory has found wide applicability in dif-
ferent areas of operations research. Many problems in inventory and queueing
theory have been solved using tools from renewal theory. For excellent cover-
age of the inventory applications of renewal theory, see TIjms [187, Chapter 1).
~ahin's monograph [163] provides a rigorous renewal-theoretic analysis of a class
of (s, S) inventory systems. A book by Nelson [137] describes the use of re-
newal theory in queueing and performance modeling. Bartholomew [16] uses re-
newal theory to model manpower systems and other social processes. Crane and
Lemoine [56] use the regenerative method of renewal theory to perform simu-
lation analyses more efficiently. As these examples indicate, renewal theory and
renewal theoretic arguments have been successfully used to solve a large number
of important problems in operations research.
 7.3 Renewal Theory 257

7.3.1 Preliminaries
We consider a sequence ofi.i.d. random variables {Xn, n = 1,2, ... } with com-
mon distribution F, density I, mean Il = E(Xn) and variance q2. The r.v. Xn de-
notes the interarrival time between the nth and (n-I)st events (arrivals/renewals).
We assume F(O) = Pr(Xn = 0) < I, so the mean time between events Il > o.
The quantity 1/ Il is the arrival rate of the process.
We define Sn = :L7=1 Xi, n = 1,2, ... as the arrival time of the nth event with
So = 0 indicating that there is an event at time t = O. As before, we define N (t) as
the number of renewals in the interval (0, t] and write N(t) = max{n ; Sn ::; t}.
In Section 7.2.6 we argued that the two events {Sn ::; t} and {N(t) ~ n} were
equivalent since the nth event takes place at or before time t if and only if the
number of events until t is at least equal to n. These arguments are still valid even
if the interarrival times are non exponential. Thus to compute the distribution of
the process N (I), we can write
Pn(/) = Pr[N(/) n]=
= Pr[N(/) ~ n] - Pr[N(/) ~ n + I]
= Pr(Sn::; t) - Pr(Sn+1 ::; t)
= Fn(/) - Fn+1 (t) (7.4)
where Fn(t) = FI (I) ® Fn- I(t) = Jooo FI (I - U)/n-1 (u) du is the n-fold convo-
lution of F with itself.

7.3.2 Distribution ofN(t)

We now show using Laplace transforms (Ln that it is relatively easy to develop
explicit expressions for the probability Pn(/) provided that the LT of the interar-
rival density is not too complicated.
Let p(s) = .c{Pn(/)}, Fn(s) = .c{Fn(/)} and jn(S) = .c{fn(t)} be the LTs of
Pn(/), Fn(t) and In (t), respectively. Since
- I -
Fn(s) = - In(s)
S

and jn(S) = [j(s)]n, taking the LT of both sides of (7.4), we obtain

I - -
Pn(s) = -[/(s)]n[l - I(s))'
s
We now describe an example where we can invert Pn(s) to obtain closed-form
expressions for the probability Pn(t).
Example 73 Distribution 01N (t) wilh Erlang(2. l) intera"ivals. In this example,
we assume that the interarrivals are distributed as Erlang with parameters (2, l),
i.e.,
I(t) = l2 te -l,.
This Erlang r.v. has mean Il = 2/l and variance q2 = 2/l2.
258 7. Stochastic Processes

> restart: * DistN.mws

> k: =2;
k :=2
> f:=lambda* (lambda*t)A(k-1)*e~p(-lambda*t)/(k-1)!;

1:= l2 t e(-lt)
We load the integral transfonns package inttrans () in order to use the
inverse transfonnation facilities of Maple.
> with(inttrans);

[addtable, lourier, louriercos, louriersin, hankel, hilbert, invfourier,

invhilbert, invlaplace, invmellin, laplace, mellin, savetable)
The LT ofErlang(2, l) is easily found as
> fLT:=laplace(f,t,s);
l2
jLT:= (s + l)2
For n = I, we attempt to find the probability Pn(t). We fonn the LT of PI (I)
using PI (s) = Hj(s») I [I - j(s)].
> pLT[ 1] :=(l/s)*fLT* (l-fLT);
l2
l 2 (1 - (s + l )2 )
pLTI := s (s + l)2

Inverting PI (s) gives the probability PI (I) = l2 (ilt 3e-1t + !t2e-1t ) :

> p[ 1] :=invlaplace(pLT[ 1] ,s,t);
I
PI := l2 (6 l t 3 e(-lt) + 2I t 2 e(-lt»
For n = 2 and n = 3, similar but more complicated expressions for the Laplace
transfonns Pn(t) are obtained. These are easily inverted to give
> pLT[ 2] := (l/s)* fLTA2* (l-fLT);
l2
l4 (1 - (s + l)2)
pLT2 := s (s + l)4
> p[ 2] : =invlaplace (pLT( 2] , s, t) ;
1 1
P2 .=
.
l4 ( - l t S e(-lt) + - t 4 e(-lt»
120 24
> pLT( 3] :=(1/s)*fLT A3* (l-fLT);
 7.3 Renewal Theory 259

A2
._ A6 (1- (S + A)2)
p LT3 .- S (s + A)6
> p[ 3] : =invlaplace (pL'I1 3] , s, t) ;
I I
P3:= A6 (--At 7 e(-lt) + _ t6 e(-lt»
5040 720
It is worth noting that if the interarrival density were /(t) = Ae-lt , i.e., ex-
ponential, then the distribution of the counting process N(t) would be Poisson,
as expected. The reader is invited to check this result by setting k = I on the
second line of the Maple worksheet and executing the worksheet with \Edi t
\Execute \Worksheet.

7.3.3 Renewal Function and Renewal Density

As we observe, if the interarrival distribution is not too complicated, then it may
be possible to find explicit expressions for the distribution of the counting process
N(t). However, in some cases this may not be possible or we may need only to
compute the mean of the renewal process, i.e., M(t) = E[N(t»). This determinis-
tic function is called the renewal function and it can often be computed relatively
easily using Laplace transforms.
It is easy to see that

00

M(t) = E[N(t)] = LnPr[N(t) = n]

n=1
00

= Ln[Fn(t) - Fn+!(t)]
n=1
00

= LFn(t)
n=1
where the last equality is obtained using telescopic cancellations.
It is usually easier to work with renewal density m(t), which is the first deriva-
tive of the renewal function, i.e., m(t) = M'(t) = L:'1 /n(t). The renewal den-
sity has an interesting and useful interpretation: Since /n(t)dt is approximately
the probability that the nth renewal occurs in (t, t + dt), the sum m(t) dt =
L:'1 /n(t) dt is the probability that one renewal (the 1st or the 2nd or 3rd and
so on) occurs in (t, t + dt).
Defining ,;;(s) = £{m(t)} as the LT of m(t), and performing the usual opera-
tions on m(t) = L:'1 /n(t), we find
00 00

,;;(s) = Lin(s) = L[i(s)]n

n=1 n=1
260 7. Stochastic Processes

l(s)
= I -1(s)
(7.5)

Note that the infinite swn L,,~ I [1(s ) In converges since

0< l(s) = 10 00
e-st /(/)dl < 10 00
/(/)dl = 1.

Example 74 Renewalfunclion M(/)/or Er/ang(3,J) interarriva/s. In this exam-

ple, the interanival r.v. is Erlang with parameters k = 3 and 1 = I having mean
Jl = k/1 = 3 and variance q2 = k/12 = 3. Since k = 3, the density is given by
!/
/(/) = 2e-t •
> restart: *RenewalFunc.mws
> with (inttrans) :
> k:=3i
k :=3
> lambda:=li
1:= I
> f:=lambda* (lambda*t) A (k-l)*exp(-lambda*t)/(k-l) !i
I
/ := - 12 e(-t)
2
The LT l(s) of the interanival density is computed easily as
> fLT:=laplace(f,t,s)i
I
fLT:= (1 +s)3
Using the result for;;' (s) in (7.5), we find the LT of the renewal density in tenns
of l(s):
> mLT:=fLT/(l-fLT)i

mLT:= I
(1 +s)3 (1- (l +s)3)

The resulting transfonn ;;,(s) appears complicated, but Maple easily inverts it
and finds the renewal density m(/). Since the renewal function M(/) = J~ m(u)du
is computed by integrating the renewal density m(/), we also find M(/) as follows.
> m:=unapply(invlaplace(mLT,s,t),t)i

m := 1 ~ ~ - ~ e(-3/2t) cos( ~,J3 I) - ~ e(-3/2t),J3 sin( ~,J3 I)

3 3 2 3 2
> M:=int(m(u),u=O •• t)i
I I I
M := - 1 + - e(-3/2t) cos( - .J3 I) + -I e(-3/2t).J3 sm(
.1
-.J3 I) - -I
3 3 2 9 2 3
 7.3 Renewal Theory 261

We note that the M(t) function has a linear asymptotic tenn (jt), two transient
tenns that approach zero as t ~ 0 and a bias tenn (the constant -i).
We find that
> lirnit(M/t,t=infinity);

i.e.,
lim M(t)
,-too-t- = -; = 3'
This limiting result holds in general and it is known as the elementary renewal
theorem.
Now, plotting the renewal function M(t) and its asymptote it -i
on the same
graph, we see that the latter approximates the fonner very accurately for t ~ 2.
These results are displayed in Figure 7.4.
> *plot ({ M, t/3-1/3} ,t=O •• 4) ;

0.8

0.6

0.4
0.2

o
-0.2

FIGURE 7.4. The renewal function M(t) and its asymptote it - i for Erlang(3,1) inter-
arrival times.

Finally, a simple check reveals that

lim [M(t)
,-too
-!.] = _!.
3 3

This result is generalized to obtain

lim
,-too
[M(t) - -J.lt] = q22-J.l 2J.l2 '
which can be seen to be true from the following Maple statements.
> rnu:=k/larnbda; sigrna:=sqrt(k/larnbda A 2);
262 7. Stochastic Processes

J.L :=3
(I :=,J3
> limit(M-t/mu,t=infinity);
-I
3
> (sigma A 2-mu A 2)/(2*mu A 2);
-I
3

7.3.4 Renewal Equation and the Key Renewal Theorem

In this section we use the powerful ''renewal argument" and write an integral equa-
tion for the renewal function M(t). We then use Laplace transfonns to solve the
resulting integral equation. We also provide a brief discussion of the key renewal
theorem that is useful in obtaining limiting results as time tends to infinity.

Renewal Equation
Suppose we fix t and condition on the occurrence of the first renewal time XI.
Thus
M(t) = E[N(t)] = 10 00
E[N(t) I XI = x] f(x)dx.

Depending on the relative values of x and t, the conditional expectation assumes

different values. If the first renewal occurs after t, then there must have been no
renewals untilt, and hence E[N(t) I XI = x] = o. On the other hand, if the first
renewal occurs at or before t, then the expected number of renewals until t is I
(the first renewal) plus M(t - x). These arguments give

E[N(t)IXI =x]= { o1'+M(t-x), forx>t

fon $.t.

Thus, we can write

M(t) = jooO'f(t)dt+ 1o'[I+M(t-X)]f(X)dX, (7.6)

or
M(t) = F(t) + 10' M(t-x)f(x)dx.
This is known as the renewal equation (or the integral equation ofrenewal theory)
and it is a computationally more useful result than M(t) = L~I Fn(t), which is
given in tenns of the n-fold convolutions of the distribution F.
 7.3 Renewal Theory 263

Now consider the more general integral equation

g(t) = h(t) + lo' g(t - x)/(x) dx, t 2: 0 (7.7)

where h(t) and /(t) are known functions and g(t) is the unknown function. If we
let h(t) = F(t) and g(t) = M(t), then the integral equation for g(t) reduces to
the renewal equation in (7.6). We now solve (7.7) using Laplace transforms.
Let g(s) and h(s) be the Laplace transforms of g(t) and h(t), respectively.
Taking the LT of equation (7.7), we obtain

g(s) = h(s) + g(s)j(s).

Solving for g(s) gives

g(s) = h(s) -
_ = h(s) [I + j(S)] - + h(s),;,(s)
_ = h(s) - (7.8)
I - /(s) I - /(s)

where the last equality is obtained from (7.5). Inversion of(7.8) gives

g(t) = h(t) + lo' h(t - x)m(x)dx.

Specializing this result to the renewal equation, we find

M(t) = F(t} + lo' F(t - x }m(x) dx

as the solution for M(t}. Provided that the renewal density is not too complicated,
the integral may be evaluated explicitly. We now provide an example of this result
using Maple.
Example 75 Solution o/the renewal equation with Erlang(2, 2) interarrival times.
In this example the interarrivaI times are Erlang(k,A.} with k = 2 and A. = 2. This
implies that the mean interarrival time is Jl = k/ A. = I. After defining the density
/(t), we easily find the distribution F(t) = J~ /(u) du using Maple's int ()
function.
> restart: * RenewalEqn.mws
> with{inttrans):
> k: =2;
k:=2
> lambda:=2;
A.:= 2
> f:=unapply{lambda* (lambda*t) A (k-l)
*exp{-lambda*t)/{k-l) !,t);
/:= t -+ 4te(-2,)
264 7. Stochastic Processes

> F:=unapply(int(f(u),u=O •. t),t);

F := I ~ -21 e(-2t) - e(-2t) + I
We compute the LTs of the probability density i(s) and the renewal density
ir(s) and invert the latter to find m(/) = I - e-4t .
> fLT:=laplace(f(t),t,s);
I
jLT := 4 (s + 2)2
> rnLT:=fLT/(l-fLT);

mLT:=4 I
(s + 2)2 (I - 4 (s + 2)2)
> m:=unapply(invlaplace(rnLT,s,t),t);
m := I ~ I - e(-4t)
Now using the solution M(/) = F(/) + J~ F(I - x )m(x) dx, we compute the
renewal function M(/).
> M: =unapply(F(t)+int(F(t-x)*m(x),x=O •• t),t);

M := I ~ I + -I e(-4t) + e(-4t) I e(4t) _ e(-4t) e(4t) __I e(-2t) e(2t)

4 4
> simplify(M(t));
I I (-4t)
--+-e +1
4 4
Thus, M(/) = -! !e-
+ 4t + I.
As a final check, we find the renewal function using the obvious result M(/) =
J~ m(u)du, which agrees with the solution found.
> Mlntm:=int(m(u),u=O .. t);

Mlntm := -4I + 4I e(-4t) + I

Key Renewal Theorem

It is worth noting that in this case-and in other cases-it may be much easier to
simply integrate the renewal density m(/) and find M(/) instead of using the more
complicated solution that involves the integral J~ F(I - x)m(x)dx. We discuss
this solution of the renewal equation since it has other important uses in renewal
theory, such as finding the limiting probabilities. These are found using the key
renewallheorem (KRT), which says that for large I, provided that h(/) is direclly
 7.3 Renewal Theory 265

Riemann integrable,2 (DRI), then

lim g(t) = -1
t~oo Il 0
10 00
h(t)dt. (7.9)

For example, Ross [157, p. 67] discusses an application of this result to find
the limiting probability that an alternating renewal process (ARP) will be found
in one of its two possible states. An alternating renewal process can be in either
ON state (for example, working) or in OFF state (for example, idle). Initially the
system is ON and it stays there for a random length of time XI; after that it stays
idle for YI. Then it goes to the ON state and stays there for X2 time units and
then switches to OFF state and stays OFF for a length Y2. This process continues
forever. Assuming that the vectors (Xn , Yn ), n = 1,2, ... are i.i.d., Ross shows
that if P(t) is defined as the probability that the system is ON at time t, then

P(t) = Fx(t) + 10' Fx(t - z)mz(z)dz. (7.10)

Here, with Fx(t) as the distribution of the ON random variable X, Fx(t) = I -

Fx(t) is the survivor function and mz(t) is the renewal density for the cycle time
Z = X + Y. We now present an example that computes the probability P(t) for
finite and infinite t.
Example 76 The ARP and the Key Renewal Theorem. We assume that the ON
times are Erlang(I,3}-that is, exponential with parameter A.X = 3-and the OFF
times are Erlang(I,2)--exponential with ~eter A.y = 2. This implies that
i, !
Il x = Il y = and Il z = Il x + Il Y = ~ are the mean times for the ON period
X, the OFF period Y and the cycles time Z, respectively.
> restart: *
KRT.mws
> with(inttrans):
> k[X] :=1;

kx:= I
> lambda[ X] : =3;
A.X := 3
> mu{ X] : =k[ Xl /lambda[ X] ;
I
Ilx:= -
3
> k[ Y] :=1;

ky := I
> lambda[ Y] : =2;

2Sufficient conditions for h(t) to be DRI are as follows: (i) h(t) ~ 0, (ii) h(t) is nonincreasing,
roo
and (iii) h(t)dt < 00; see Ross [157, p. 64] and Feller [65, pp. 362-363].
266 7. Stochastic Processes

ly:=2
> mu( Y] : =k[ Yl /lambda[ Y] ;

1
f.Jy:=-
2
> mu( zl : =mu( Xl +mu( Y] ;

5
f.Jz :=-
6
For the ON period, we have the density lx, distribution Fx, survivor function
Fx and the Laplace transfonn ix as follows:
> f[ xl : =unapply (lambda[ Xl * (lambda[ Xl * t) " (k[ Xl -1)
* exp (-lambda[ Xl * t ) / (k[ X] -1) ! , t) ;

Ix := t -+ 3 e(-3t)
> F{X] :=unapply(int(f[Xl (u),u=O •• t),t);

Fx := t -+ _e(-3t) +1
> FBar[ xl :=t->l-F{ Xl (t); FBar[ Xl (t);

FBarx:= t -+ 1 - Fx(t)
e(-3t)

> fLT[ Xl :=laplace (f[ Xl (t), t, s);

1
ftT x := 3 s +3
Similarly, for the OFF period we define the density Iy and distribution Fy, and
find the survivor function Fy and the Laplace transfonn iy:
> f[ Y] : =unapply (lambda[ Y] * (lambda[ Y] * t) " (k[ Y] -1)
* exp (-lambda[ Y] * t) / (k[ Y] -1) ! , t) ;

Iy := t -+ 2e(-2t)

> F{ Y] : =unapply (int (f[ Y] (u), u=O •• t) , t) ;

Fy := t -+ _e(-2t) +1
> FBar[ Yl :=t->l-F{ Y] (t); FBar[ Y] (t);

FBary := t -+ 1 - Fy(t)
e(-2t)

> fLT[Y] :=laplace(f[ Y] (t),t,s);

1
ftT y :=2 s +2
Since Z = X + Y, with independent X and Y, the LTof Z is the product of the
LTs ix and iy.
> fLT[ zl : =fLT[ Xl * fLT[ Y] ;
 7.3 Renewal Theory 267

fLT z := 6 (s + 3)(s + 2)
Using the expression for the LT of the renewal density ,;,(s) in (7.5) we find the
LT of the renewal density for Z:
> mLT( ZJ :=fLT( Z] I (l-fLT( ZJ);
I
mLTz :=6 1
(s + 3)(s + 2)(1 - 6 (s + 3)(s + 2»
Inversion of the LT gives the renewal density mz(t) as
> ~ zJ :=unapply(invlaplace(mLT( Z] ,s,t),t);

6 6
mz := t -+ 5 - 5e(-SI)
The probability P(t) is computed using equation (7.10). We see that P(t) =
~ + ie-sl and limHoo P(t) = ~; i.e., the limiting probability of being in the ON
state is 215.
> P:=unapply(FBar( X] (t)+int(FBar( Xl (t-x)*~ Z] (x),
x=O • • t) , t) ;

P := t -+ ~ e(-SI) e(SI) + ~ e(-SI)

> limit(P(t),t=infinity);
2
5
We also plot the P(t) function and observe that it converges to 215 as indicated.
This is presented in Figure 7.5.
> *plot(P(t),t=O •• lO,O • • l);
From the KRTwe know that since P(/) = Fx(t) + J~ Fx(t-z)mz(z)dz and
since Fx(/) is DR!,

.
11m P(t) = -
1 10 00
-
Fx(t)dt =
Px
= l =
-'-I 2
-5'
HOO Pz 0 Px +Py 1 +!

We obtain the expected results.

> int (FBar( X] (t), t=O •• infinity) Imu{ Z] ;
2
5
> mu{ Xl I (mu{ Xl +mu{ y) ) ;
2
-5
268 7. Stochastic Processes

FIGURE 7.5. The probability P(t) converges to ~ as t tends to infinity.

7.3.5 Renewal Reward Process

This process generalizes the compound Poisson process that was discussed in
Section 7.2.8. Consider a renewal process N(t) with interarrival times XI, X2, ...
having the common distribution F and suppose that a "reward" Rn is earned at
the time of the nth renewal. We assume that the rewards {Rn, n = 1,2,} and the
vectors {(Xn , Rn ), n = 1,2, ... } are i.i.d.
Similar to the construction of the compound Poisson process, we let
N(t)
R(t) = ~Rn
n=1

where R(t) denotes the total reward earned until t. Computing the limiting values
of the average reward is an extremely important problem in stochastic optimiza-
tion applications found in, for example, inventory theory and queueing processes
and it is given by the following theorem; see, Ross (157, pp. 78-79].

Theorem 12 Renewal Reward Theorem (RRT): If E(R) and Jl = E(X) are both
finite, then the long-run average expected reward per unit time is

. R(t) E(R)
hm - = (X) with probability 1.
t~oo t E

Consider a stochastic system that replicates itself probabilistically at renewal

points XI, XI +X2, XI +X2 +X3, . . . Ifwe suppose that a renewal signifies the
start of a eye/e, then this theorem states that the average reward per unit time is
simply the expected reward per cycle divided by the expected cycle length. This
is a very powerful result in optimization of stochastic systems as it reduces an
infinite horizon optimization problem to a static (calculus) problem.
 7.3 Renewal Theory 269

We now discuss an example where an application of the RRT greatly simplifies

the analysis of the problem.
Example 77 Age replacement policy for a car. This example is adapted from
an exercise in Ross [158, Problem 7.14]. Consider a car whose lifetime X has
distribution F and density f. The replacement policy adopted by the owner is as
follows: "Buy a new car as soon as the old car either breaks down or reaches the
age of T years." The cost of a new car is CJ dollars. If the car has to be replaced
after it breaks down, then there is an additional cost ofC2 dollars. If the owner sells
aT-year old car in working order, then he receives R(T) dollars. The problem is
to find the long-run average cost and determine the optimal replacement time T*
that minimizes the owner's average cost.
It is easy to see that every time a new car is purchased, either due to a break-
down or planned replacement after using the car for T years, a new cycle starts.
To compute the long-run average cost, we need to determine the expected cycle
cost E(C) and expected cycle length E(L).
Conditioning on the occurrence time of the first breakdown, we can write the
expected cycle costas E(C) = 1000 E(C I XI =x)f(x)dx where

E(C I XI = x) = { CJ - R(T), x > T

CJ +C2, x:::; T.
Thus,

E(C) = 1r00[CI-R(T)]f(X)dX+ foT(CJ +C2)f(x)dx

= CI + C2F(T) - R(T)[1 - F(T»).

Similarly, we can write the expected cycle length as E(L) = 10 E(L I XI =

x)f(x)dx where
E(L I XI = x) = { T, x> TT
x, x:::;
so that

E(L) = Iroo
Tf(x)dx + foT xf(x)dx

= [I - F(T)]T + foT xf(x)dx.

Now using the RRT, we obtain the long-run average expected cost per unit time
as
C(T) = E(C) = CI + C2 F (T) - R(T)[I - F(T)] .
E(L) [I - F(T)]T + It xf(x)dx
Optimization of this cost function for CI = 16 and C2 = 1 with R(T) = 10 - T
is performed as follows.
270 7. Stochastic Processes

> restart: i CarReplace.mws

> c[ 1] : =16; c[ 2] : =1 ;
C) := 16
C2:= 1
> R:=T->10-T;
R:= T~ 10-T
We assume that the lifetime random variable is unifonn with density f(x) =
I/(b -a) where a = 2 and b = 8.
> a:=2; b:=8;
a:=2
b:=8
> f:=x->l/(b-a);
1
f:=x~ -b-
-a
The distribution F(x) = f;
f(u) du is easily computed to give
> F:=unapply(int(f(u),u=2 •. x),x);
1 1
F:=x~ 6x-3"
> F (x) ;
1 1
6 x -3"
The cost function C(T) is obtained as a somewhat complicated nonlinear ex-
pression.
> Top:=c[ 1] +c[ 2] *F(T)-R(T)* (l-F(T));
47 1 4 1
Top:= 3" + 6 T - (10 - T)(3" - 6 T)
> Bottom:=int(x*f(x),x=O •• T)+T* (l-F(T));

Bonom·=..!. T2
. 12
+ T(~3 -!6 T)
> C:=unapply(Top/Bottom,T);

-47 + -1 T - (10 - T)(- - - T)

4 1
.- T
C .- 3 6 3 6
---, 1 4 1
-'0.

12 T2 + T(3" - 6 T)
When the cost function is plotted-which we leave to the reader-one observes
that it is a strictly convex function with a unique minimum in the interior of the
interval [a, b) = [2,8].
> iplot(C(T),T=a •• b);
 7.4 Discrete-Time Markov Chains 271

Differentiating the cost function, equating the result to zero and solving-using
fsol ve () -gives the optimal replacement time as T* = 5.15 years.
> CT:=normal(diff(C(T),T));

CT := 2 3 T2 + 28 T - 224
T2 (T - 16)2
> TStar:=fsolve(CT(T),T,a .. b);
TStar:= 5.153946575
As a final check we find the second derivative C"(T) of the cost function and
evaluate it at T* = 5.15, which results in a positive value implying that the solu-
tion we just found is indeed the global minimum. A comparison of the value of
the cost function at the end points a and b with T* also indicates that we have
located the true optimal solution.
> CTT:=unapply(normal(diff(C(T),T$2)),T);
CIT ._ T _ 3 T3 + 42 T2 - 672 T + 3584
.- ~ 4 T3 (T _ 16)3
> CTT(TStar);
.03771347490
> evalf(C(a)); C(TStar); evalf(C(b));
3.428571429
3.054091250
3.187500000

The renewal reward theorem was popularized by Ross [156] and it has been ap-
plied successfully in different areas of operations research. For example, Gilrler
and Parlar [78] and Parlar [143] use the RRT to model a stochastic inventory sys-
tem with two unreliable suppliers who may be unavailable for random durations.
Taylor (184] uses it to model a repair problem for a deteriorating item and Tijms
[186, Ch. 1] describes different applications ofRRT in queueing theory.

7.4 Discrete-Time Markov Chains

Consider a discrete-time stochastic process {Xn, n = 0,1,2, ... } with finite or
countably infinite state space S. Suppose that this process has the property that the
conditional distribution of X n+ I given the past infonnation X n, X n-I, ... , X I , X 0
depends only on the value of X n but not on X n-I, ... , X I , X 0, i.e., for any set of
values j, i, in-I, ... , io belonging to the state space S,
Pr(Xn+1 =j Xn = i, Xn-I = in-I, ... , Xo = io)
= Pr(Xn+1 = ). I X n = I.) = Pij(n,n+l)
272 7. Stochastic Processes

=
transition . Pij'
I Itles (n n+l) are constant over tune;
. .
I.e., lor °
for n 0, I, 2, .... A time-homogeneous Markov chain has the property that the
. . probab·l· ~ a II n = , I , 2 , ...

Pr(Xn+1 =} I Xn = i) = Pij. i,} E S.

We assume throughout this chapter that the Markov chains we consider are time

°
homogeneous. The transition probabilities Pij form the one-step transition matrix
P = [Pij] and they satisfy (i) Pij 2: for i, } E S and (ii) LjeS Pij = I for
i E S, thus making P a "stochastic matrix." The distribution of the initial state Xo
and the transition matrix P completely determine the distribution of a future state
X n, n = 1,2, ... of the Markov chain.

Example 78 Random walle. As a simple example, consider a coin-tossing game

where the possible outcomes are (i) heads (H) with probability P and (ii) tails (1')
with probability q = I - p. Suppose that H and T correspond to a gain oUI and
a loss oUI, respectively. Ifwe denote the outcome of the nth toss by Zn, then

Z _ { I ifheads with probability P

n - -I if tails with probability q.

Now consider the partial sumXn = ZI +Z2+·· ·+Zn. AssumingXo = 0, the

sum Xn may represent the accumulated wealth of a gambler in the first n plays
of the coin-tossing game who earns a random amount Zn in each play. Since we
can write Xn+1 = Xn + Zn+l, it is easy to see that the Xn process (known as the
''random walk") is a Markov chain as the distribution of Xn+1 depends only on
the value of Xn (and on the r.v. Zn+I), not on Xn-I, ... , XI,XO.
To compute the transition probabilities Pij = Pr(Xn+1 =} I Xn = i), i,} E
S = {... , -2, -1,0, 1,2, ... } of this simple Markov chain we proceed formally
as follows:

Pij = Pr(Xn+1 =} I Xn = i)
= Pr(X" + Z,,+I = j I XII = i)
= Pr(i + Zn+1 =})
= Pr(Zn+1 =} - i).

Since Zn+1 = ±I, we see that} = i + I ~ Zn+1 = I and} = i-I ~ Zn+1 =

-I. Thus, if the process is in state i, the only possible movements are (i) to ; + I
(''up'') with probability P and (ii) to i-I ("down") with probability q = I - p,
i.e., Pi,i+) = P and Pi,i-I = q, i E S. The infinite dimensional transition matrix
 7.4 Discrete-Time Markov Chains 273

P of this random walk process is then obtained as

7' -2 -1 0 2

-2 0 P
-1 q 0 P
0 q 0 P
1 q 0 P
2 q 0

Transition matrices of Markov chains can be displayed graphically using a "di-

graph" (or a ''transition diagram"). The digraph for the random walk example is
displayed in Figure 7.6.

q q q q

FIGURE 7.6. Digraph for the random walk example.

The one-period dependency exhibited in this example (that Xn+1 is a function

of Xn only) is known as the Markovian propenyl of a Markov chain. Despite the
simplicity of this property a surprisingly large number of practical problems in
inventory theory, queues, telecommunications and so on have been successfully
analyzed using Markov chains; see, for example, Tijms [186, Chapter 2] for an
excellent discussion of these applications.

Example 79 A periodic-review (s, S) inventory system. Consider a single product

for which demands Yn during successive weeks n = 1, 2, ... are i.i.d. with the
common density gy(k) = Pr(Y = k), k = 0,1,2, . . .. We let Xn be the inventory
position at the end of week n and assume that excess demand is lost. Management
uses an (s, S) ordering policy, that is, after observing the level of X n, they order

31t is interesting to note that the Russian mathematician Andrei Andreivich Markov (1856-1922)
arrived at the notion of Markov chains when he investigated the alteration of vowels and consonants
in Pushkin's poem Onegin and discovered that there was indeed some kind of dependency between
the occurrence of these vowels and consonants (TaUcs [183, p. 6]).
274 7. Stochastic Processes

Qn units at the end of week n where

Q _ {S-Xn ifO:sXn <s

n- 0 if s :s Xn :s S.
In inventory theory S and s are known as the "order-up-to level" and the "re..
order point;' respectively. We may assume that orders are placed on a Saturday
evening before the weekend closing and they arrive on Monday morning just be-
fore the store reopens. Thus, for modeling purposes, deliveries can be assumed
to be instantaneous. Since the excess demand is lost and the maximum possible
value of Xn is S, the state space of this stochastic process is S = to, 1,2, ... , S}.
We now show that X n is a Markov chain.
If, at the end of week n the inventory position Xn is below the reorder point s,
i.e., if Xn < s, then the management orders S - Xn units, which increases the
inventory position at the start of the following week (on Monday morning) to S.
Since demand during n + 1 is a random quantity represented by Yn+1 and since
excess demand is lost, we obtain
Xn+1 = max(S - Yn+l, 0), if 0 :s Xn < s
as the inventory position at the end of the following week. However, if s :s Xn :s
S, then no action is taken and we have
Xn+1 = max(Xn - Yn+l, 0), if s :s Xn :s S.
We see that Xn+1 depends only on Xn but not on X n-I,Xn-2, ... ,XI, Xo; thus
Xn is a Markov chain. We now find the transition matrix P = [Pij] of this Markov
chain.
Assume that the demand r. v. Y is Poisson with rate ,t, i.e.,
,tke-l
gy(k) =~, k = 0,1,2, ...
and define a complementary cumulative-type probability as
k-I
G(k) = Pr(Y ~ k) = I - Lgy(m)
m=O
For 0 :s ; < s, we have
Pij = Pr(Xn+1 = j I Xn = i)
= Pr[max(S - Yn+l, 0) = j]
= {Pr(Yn+1 ~ S) = G(S), ifj =0
Pr(Yn+1 = S - j) = g(S - j), ifj = 1, ... ,S.
Similarly, for s :s i :s S, we obtain
Pij = Pr(Xn+1 = j I Xn = i)
= Pr[max(i - Yn+l, 0) = j]
= {Pr(Yn+1 ~ ;) = G(i), ifj = 0
Pr(Yn+1 =; - j) = g(i - j), if j = I, ... , S
 7.4 Discrete-Time Markov Chains 275

with the condition that if i - j < 0 then Pr(Yn+1 = i - j) = g(i - j) = O. With

,t = I and (s, S) = (1, 4), the transition matrix P is found as follows.
> restart: # PeriodicsS.mws
> lambda:=li
,t := I
> g:=k->'if'(k>=O,lambdaAk*exp(-lambda)/k!,O);
,tk e(-l)
g := k ~ 'if'(O ::; k, k! ' 0)
> GHat:=k->1-sum(g(m),m=O .. k-1);
k-I
GHat:= k ~ 1- (:Lg(m»
m=O
> s:=l; S:=4;
s := I
S:=4
The next two Maple commands generate the probabilities Pij for 0 ::; ; < s
and s ::; ; ::; S, j E S using the results obtained.
> for i from 0 to s-l do
for j from 0 to S do if j=O then
p[ i, j] :=GHat (S) else
p[ i, j] := g (S-j) fi od od;
> for i from s to S do
for j from 0 to S do
i f j=O then p[ i,j] :=GHat(i)
else p[i,j]:= g(i-j) fi od od;
A simple check reveals that the probabilities in each row indeed sum to I, i.e.,
LjeSPij = 1 for; E S:
> for i from 0 to S do
evalf(sum(p[ O,column] ,column=O .. S))
od;
1.
1.
1.
1.
1.
Each row of the P matrix is generated and placed into an array named PArray
whose row and column indices are o.. S and o.. S.
> for i from 0 to S do
Row[ i] :=evalf([ seq(p[ i,j] ,j=O .. S)] ,4)
od;
276 7. Stochastic Processes

Rowo := [.0188, .06133, .1840, .3679, .3679]

Row I := [.6321, .3679, 0, 0, 0]
R0W2 := [.2642, .3679, .3679,0, 0]
Row) := [.0802, .1840, .3679, .3679, 0]
:= [.0188, .06133, .1840, .3679, .3679]
ROW4
> PArray:=array(O •• S,O .• S,[ seq (Row[ i] ,i=O •• S)] ):
Converting the array to a matrix we obtain the transition matrix P:
> P:=convert(PArray,matrix,4);

.0188 .06133 .1840 .3679 .3679]

.6321 .3679 0 0 0
P := [ .2642 .3679 .3679 0 0
.0802 .1840 .3679 .3679 0
.0188 .06133 .1840 .3679 .3679
Note that if the process is in state 1, i.e., Xn = 1, it is impossible to make a
transition to states 2, 3 or 4 since no order will be placed at the end of week n.
This means that week n + 1 will start with 1 unit and end with either 0 units (if
there is a demand) or 1 unit (if there is no demand). The other entries in rows 2
and 3 with zero transition probabilities have a similar interpretation.

7.4.1 Chapman-Kolmogorov Equations

In the foregoing discussion, we defined the one-step transition as Pij = Pr(X"+1 =
j I Xn = i). We now generalize and define the n-step transition as p~) =
Pr(Xm+n = j I Xm = i) for n = 0, 1,2, ... and i, j E S. Since we asswnea that
the Markov chain is time homogeneous, i.e., that the probability is invariant in m,
we can write the n-step transition as
Pr(Xm+n = j I Xm = i) = Pr(X" = j I Xo = i).
To compute p~) we proceed by conditioning on the state visited in the first tran-
sition:
(n) __
Pij Pr(Xn =j I Xo = i)
= LPr(Xn =j I XI = k,Xo = i)Pr(XI =k I Xo = i)
iceS
= ""p°lcp(n-I)
~, Icj • (7.11)
iceS

Ifwe now define p<n) = [p~)] as the matrix ofn-step transition probabilities, we
see that Chapman-Kolmogorov equations in (7.11) can be written conveniently in
matrix notation as
p(n) = p x p<n-I).
 7.4 Discrete-Time Markov Chains 277

Since p(2) = P X p(l) = p2, p(3) = P X p(2) = p3 and SO on. we find that the
n-step transition matrix p(n) is simply the nth power P" of the one-step transition
matrix P, i.e.,
p(n) = pn.
Eumple 80 A periodic-review (s, S) inventory system (continued). Using Ma-
ple's eva 1m ( ) function we compute the nth power of the transition matrix for
different values of n. First, we set Digi ts: =4 in order to fit the output on a
printed page. (Note that Digits is an environment variable that specifies the
number of digits carried in floating point calculations with a default of 10.)
> Digits:=4i * PeriodicsS.mws (continued)
Digits:= 4
For n = 2, 8 and 64 we compute P" and note that for n = 64 the probabilities
seem to have (almost) converged to the same values in each row. This is not a
coincidence and as we will discuss later, for a large class of Markov chains this
convergence is achieved regardless of the initial state.
> eva1m(P"2) i
.1817 .2743 .2777
[ .1241
.2444 .1742 .1163 .2325 .14n ]
.2325
.3347 .2870 .1840 .09720 .09720
.2445 .2757 .2856 .1649 .02951
.1241 .1817 .2743 .2777 .1423
> eva1m (P"8) i
.2225 .2220 .2051
.2220 .2223 .2209 .2051 .1293]
[ .n17 .1303
.2231 .2231 .2212 .2041 .1293
.2226 .2233 .2220 .2043 .1287
.2217 .2225 .2220 .2051 .1293
> eva1m(P"64)i

.2235 .2240 .2228 .2058 .1301]

.2235 .2240 .2228 .2058 .1301
[ .2235 .2240 .2228 .2058 .1301
.2235 .2241 .2229 .2059 .1301
.2235 .2240 .2228 .2058 .1301
%1 := [.2235, .2240, .2228, .2058, .1301]
We should point out that since p~) = Pr(Xn = j I Xo = i), the probabilities
we have computed so far are all c01iditional on the knowledge of the initial state.
To find the unconditional probabilities 1r }n) = Pr(Xn= j) of finding the process
278 7. Stochastic Processes

in state j after n, we condition on the initial state Xo and write

1C;n) = LPr(Xn=jIXo=i)Pr(Xo=i)
ieS
= ~P~'.')1C~O). (7.12)
~ '1 I
ieS

Now defining the vector of n-step unconditional probabilities as n(n) = [",~n),

f)
1C ~n), •.• ] with n(O) as the initial distribution vector with LieS 1C = I, we can
write (7.12) using matrix/vector notation as
n(n) = n (0) p(n) .

Equivalently, if we had conditioned on the position of the process at the (n - 1)st

step as 1C ;n) = LieS Pr(Xn = j I Xn-I = i) Pr(Xn-1 = i), then n(n) would be
obtained as
n(n) = n(n-I)p. (7.13)

7.4.2 Limiting Probabilities and the Stationary Distribution

It is useful to note that (7.13) can be written as
n(n) _ n(n-I) = n(n-I)p _ n(n-I)
= n(n-I)Q

where I is the identity matrix and Q = p - I is a matrix with rows adding up to

O. If the vectors n (n) converge to a constant vector n = [1C 0, 1C 1, •.• ] for large
n, then 0 = nQ, or n = np. Thus, the limiting probabilities can be found by
solving the linear system n = np with the normalizing condition LieS 1C i = 1.
To formalize this argument, we suppose that limn-+oo pt)
= 1C j for all i E
S = {O, 1,2, ... }; i.e., after a long time the conditional probability
Pr(process in state j I process started in i)

)
becomes independent of the initial state i. This implies that there exists the limit

lim p(n) = 2
n-+oo
~ (:~1Co ::
1C1
:~ :• ::.•
1C2
··· ...
where {1C j} are the limiting (steady-state) probabilities.
Now consider the n-step unconditional probabilities 1C ;n) = LieS'" :0)pt).
Taking the limit of both sides, we have
lim 1C (~)
n-+oo 1
= lim ~ 1C ~O) p~'.')
n-+oo ~ I '1
ieS
 7.4 Discrete-Time Markov Chains 279

Thus, 1tjn) converges to the same limit as p~) and it is independent of the initial
probabilities.
When they exist, the unconditional probabilities 1t j can be found solving n =
np with the nonnalizing condition ne = I where e = [I, I, . . . , 1]' is a column
vector of Is. The solution to n = np and ne = I (when it exists) gives the
stationary distribution of the Markov chain.

Example 81 The stationary distribution of the periodic-review (s, S) inventory

system. Returning to the periodic-review (s, S) inventory example, we now find
the stationary distribution of the Markov chain representing the inventory position
Xn•
The next eleven lines are the same as before; therefore they are tenninated with
a colon (:) in order not to repeat any of the already known results.
> restart: # PeriodicsSLimit.mws
> lambda : =l:
> g:=k->'if'(k>=O,lambdaAk*exp(-lambda)/k!,O) :
> GHat:=k->1-sum(g(m),m=0 .. k-1):
s:=l : S:=4:
°°
>
> for i from to s-l do
for j from to S do if j=O then
~ i,j] :=GHat(S) else
~ i,j] := g(S-j) fi
od od;
>
for j from °
for i from s to S do
to S do
i f j=O then ~ i, j) :=GHat (i)
else ~ i, j) := 9 (i-j) fi

°
od od;
> for i from to S do
evalf(sum(~ 0, column] ,column=O .. S»
od:
> for i from °
to S do
Row[ i) :=evalf([ seq(~ i,j) ,j=O •. S)] ,4)
od:
> PArray:=array(O •. S, 0 .. S,[ seq (Row[ iJ ,i=O •• S)] ):
> P:=convert(PArray,matrix,4);
280 7. Stochastic Processes

.0188 .06133 .1840 .3679 .3679

.6321 .3679 0 0 0
.- .2642 .3679 .3679 0
p.- 0
.0802 .1840 .3679 .3679 0
.0188 .06133 .1840 .3679 .3679
We define the vector of unknown stationary probabilities lC j

> pi : =vector ( 5, [] );
lC := array(l ..5. [])
and evaluate II = IIP.
> EqnList:=evalm(pi - pi &* P);

EqnList:= [.9812 lC I - .6321lC2 - .2642lC) - .0802lC4 - .0188lCS •

.6321lC2 - .06133 lC I - .3679lC) - .1840 lC 4 - .06133lC S•
.632IlC) - .1840lCI - .3679lC4 - .1840lCS •
.6321lC4 - .3679 lC I - .3679lCS •.6321lCs - .3679lC 1]

> EqnSetR:=convert(EqnList,set);

EqnSetR:= (.632IlC4 - .3679lCI - .3679lCS •

.9812lCI - .6321lC2 - .2642lC) - .0802lC4 - .01881CS •
.6321lC2 - .06133lC I - .3679lC) - .1840 lC4 - .06133lCs •
.632IlC) - .1840lCI - .3679lC4 - .1840lCs •.6321lCs - .3679lCIJ
Since one of the equations in the system II = IIP is always redundant (see
Heyman and Sobel [88. p. 243] for a discussion). we delete an arbitrary equation-
say. the first one-using the set theoretic minus () function. We also introduce
the nonnalizing condition IIe = I as a new equation.

> EqnSet:=EqnSetR minus {EqnList[ 1]};

EqnSet:= (.632IlC4 - .3679lCI - .3679lCS •

.6321lC2 - .06133lCI - .3679lC) - . 1840 lC4 - .06133lCs •
.632IlC) - . 1840 lC I - .3679lC4 - .1840lCs • .6321lCS - .3679lCIJ
> SumProb: ={ sum (pi[ k] ,k=l .• 5) =1} ;
SumProb := (lC 1+ lC2 + lC) + lC4 + lCS = I}
After defining the linear system as the union () of two sets of equations, we
solve it and obtain the stationary distribution. As a check. we note that the sum of
these probabilities is unity.

> Sys:=EqnSet union SumProb;

 7.4 Discrete-Time Markov Chains 281

Sys := {.6321 'f4 - .3679 HI - .3679 H5,

.6321H2 - .06133 HI - .3679H3 - .1840H4 - .06133Hs,
.6321H3 - .1840HI - .3679H4 - .1840Hs, .6321Hs - .3679HI,
HI +H2+H3+H4+HS = I}
> Solution:=solve(Sys);

Solution:= {HS = .1293182802, H2 = .2225162422, HI = .2221856072,

H3 = .2213947096, H4 = .2045851608}
> assign (Solution); surn(pi[ k] ,k=1.. 5);
1.000000000

We should note that since we had to define n as a vector with five entries
indexed as 1,2,3,4, 5, the indices of the solution must be adjusted by shifting
them by -I, i.e., Pr(stationary inventory level is j - I) = H j, j = I, 2, ... , 5.
The fact that one of the equations in the linear system n = np is redundant
can be illustrated as follows.
First consider a Markov chain with two states and the transition matrix given
by
P _ I (0.7 0.3)
- 2 0.4 0.6 .
Expanding the system n = np we find
HI = 0.7H I + 0.4H2
H2 = 0.3H I + O.6H2.

Ifwe express H2 in the first equation in terms of HI, we find H2 = iH I. Similarly,

writing H2 in the second equation in terms of HI again gives H2 = i HI. Thus,
these two equations are linearly dependent.
To illustrate the redundancy of one of the equations in a three state Markov
chain we use Maple's three-dimensional plotting facilities. Consider the Markov
chain with the transition matrix

P=
.3 .2 .5) .
( .4.3.3
.1.2 .7
> restart: # RedundantEqn.mws
> with (linalg) :
Warning, new definition for norm

Warning, new definition for trace

282 7. Stochastic Processes

> with(plots):
> Row[ 1) : ={ • 3, .2, . 5) ; Row[ 2) : ={ • 4, . 3, . 3) ;
Row[ 3] : ={ • 1, • 2, . 7] ;
Row I := [.3, .2, .5]
ROW2 := [A, .3, .3]
RowJ := [.I, .2, .7]
> P:=array(1. .3,1. .3,[ seq (Row[ i] ,i=1. .3)] );

P:= [ ..13 ..2.32 .5]

.4 .3
.7
We first define the vector of unknown probabilities as n = [x, y, z].
> pi :=vector (3,[ x, y, z] );
1C := [x, y, z]

> EqnList:=evalm(pi - pi &* P);

EqnList:= [.7x -.4y -.I z, .7y - .2x - .2z, .3z - .5x - .3y]
> solve({ seq (EqnList[ k] ,k=1. .3)} );
{y = 1.066666667 x, z = 2.733333333 x, x = x}
When Maple solves the three equations generated by n = np, it finds in-
finitely many solutions as y = 1.06x and z = 2.73x for 0 ~ x ~ I. We also use
Maple's plot3d () function to plot the three planes corresponding to the linear
system n = np. The planes intersecting at infinitely many points in the (x, y, z)
space are shown in Figure 7.7.

FIGURE 7.7. Three planes generated by n = np intersect at infinitely many points.

 7.4 Discrete-Time Markov Chains 283

> EqnSetR:=convert(EqnList,set);
EqnSetR:={.7x-.4y-.lz, .7y-.2x-.2z, .3z-.5x-.3y}
> for k from 1 to 3 do
Planer k] :=solve(EqnList[ k] ,z) od;
Plane 1 := 7.x -4.y
Plaf1e2 := 3.500000000 y - 1. x
Plane3 := 1.666666667 x +y
> p[ 1] :=plot3d (Plane[ 1] , x=O •. 1, y=O •. 1) :
> p[ 2] : =plot3d (Plane[ 2] , x=O •. 1, y=O •. 1) :
> p[ 3] :=plot3d(Plane[ 3] ,x=O •. l,y=O .. 1):
> idisplay ({ p[ 1] , p[ 2] , p[ 3]
} ,axes=boxed,shading=none,orientation~ 52,47]);
After introducing the nonnalizing condition ne = I, the new system is solved
to give the unique stationary probabilities as x = .2083, y = .222 and z = .5694.
> EqnSet:=EqnSetR minus {EqnList[ 1]};
EqnSet:= {.7 y - .2x - .2z, .3z -.5x -.3 y}
> SumProb:={ sum (pi[ j] ,j=1. .3)=1} ;
SumProb:= {x + y +z = I}
> Sys:=EqnSet union SumProb;
Sys:= {.7y - .2x - .2z, .3z -.5x - .3y, x + y+z = I}
> Solution:=solve(Sys);
Solution:= {z = .5694444444, Y = .2222222222, x = .2083333333}
> assign (Solution) ; sum (pi[ , k'] ,'k' =1. .3);
.9999999999

Example 82 Machine maintenance (Heyman and Sobel f88]). Consider a ma-

chine that is examined once a day and classified as either working perfectly (state
0), having minor defects (state I), or having failed (state 2). Suppose the number
of consecutive days a machine is in state 0 (up) follows the geometric distribu-
tion u"-I (I - u), k = I, 2, ... , and the number of consecutive days a machine is
in state 1 (down) follows the geometric distribution d"-I (1 - d), k = 1,2, ....
When the machine leaves state 0, it makes a transition to state 2 (i.e., it Jails)
with probability J and when it leaves state 1 (with probability d), it always enters
state 2. Assume that repair activity lasts a random number of days and is also
geometric with distribution r"-I (1 - r), r = I, 2, .. .. The repair process is not
totally reliable and it results in a machine that is in perfectly working state 0 with
probability w and in minor defects state 1 with probability 1 - w.
284 7. Stochastic Processes

We define X" to be the state of the machine on day n and show that X" is a
Markov chain by using the memoryless property of the geometric distribution.
First, suppose the machine is in state O. The length of time the machine is in
this perfectly working state is independent of the states before it entered this state.
When X" = 0, it does not matter how long the machine has been in state 0; the
probability that it will stay in the same state is u, i.e.,

pOO = Pr(X,,+1 = 01 X" = 0, X,,_I = i,,-I, ... , Xo = io) = u.

Similarly, we obtain

pOI = Pr(X,,+1 = 1 1X" = 0, X,,_I = i,,_), ... , Xo = iO) = (l - u)(l - f)

since the probability of making a transition to 1 from 0 requires that (i) the ma-
chine makes a transition out of state 0, and (ii) it moves into state 1 with prob-
ability 1 - I. Using the same argument, the transition from state 0 to state 2 is
obtained as

p02 = Pr(X,,+1 = 21 X" = 0, X,,_I = i,,-I, ... , Xo = iO) = (1- u)f·

The other probabilities can be determined using similar arguments and we find
the transition matrix as

pOO pOI p02) ( U (1 - u)(1 - f) (1 - u)f )

P= ( PIO PII PI2 = 0 d 1- d .
P20 P21 P22 (l - r)w (1 - r)(l - w) r

It is easy to check that the transition probabilities satisfy Pij ~ 0 for i, j E S

and LjES Pij = 1 for i E S = to, 1, 2}.
Although this Markov chain's transition probabilities are given symbolically,
Maple easily solves the equations II = llP and lle = 1 and finds the stationary
distribution in terms of the symbolic parameters u, I, d, r and w:
> restart: * Machine.mws
> Row[ 0] :={ u , (l-u)* (I-f) I (l-u)*f] ;
Rowo := [u, (l - u) (l - f), (l - u) f1
> Row[ I]:={ O/d / l-d];
Rowl := [0, d, 1 - d]
> Row[ 2] :={ (l-r)*w, (l-r)* (l-w) I r] ;
Ro'W2 := [(1- r) w, (l - r) (l - w), r]
> for i from 0 to 2 do
simplify (sum (Row[ i] [j] Ij=1. .3)) od;
 7.4 Discrete-Time Markov Chains 285

> PArray:=array(O .. 2,0 .. 2,[ seq (Row[ i) ,i=O .• 2)]):

> P:=convert(PArray,matrix,3);

(1-~r)W
(l - u)(1 - 1) (l - u) / ]
d I-d
p ,= [ (I-r)(l- w) r
> pi : =vector ( 3, [ ] ) ;
1C := array(1..3, [])
> EqnList:=evalm(pi - pi &* P);

EqnList := [1C 1 - 1fI U - 1C 3 (l - r) w,

1C2 - 1C 1 (1 - u)(1 - 1) - 1C2 d - 1C3 (1 - r)(1 - w),
1C3 - 1C 1 (1 - u) f - 1C2 (1 - d) - 1C3 r]
> EqnSetR:=convert(EqnList,set);

EqnSetR := {1C 1 - 1C 1 U - 1C3 (l - r) w,

1C2 -1CI (1- u)(1- 1) -1C2d -1C3 (1-r)(1 - w),
1C3 -1CI (1- u)f -1C2(1-d) -1C3 r }
> EqnSet:=EqnSetR minus {EqnList[ 1]};

EqnSet:= {1C2 -1CI (1- u)(1- 1) -1C2d ~ 1C3 (1-r)(1- w),

1C3 -1CI (1-U)f-1C2(1-d)-1C3r}
> SumProb: ={ sum (pi[ k] , k=l .. 3) =1} ;
SumProb := {1C 1 + 1C2 + 1C3 = I}
> Sys:=EqnSet union SumProb;

Sys:= {1C2 -1CI (1- u)(1- 1) -1C2d -1C3 (1-r)(1- w),

1CI +1C2 +1C3 = I, 1C3 -1CI (1-U)f-1C2(1-d) - 1C 3r}
> Solution:=solve(Sys,{ seq (pi[ k] ,k=1..3)});

w(-r -d+ 1 +rd)

So/ution := {1C 1 =
%1
u-I-rwf-wuf+wf-ur+r+rwu/
1C2 =
%1
- "u-d+ud+ 1
1C3 = %1 }
%1 := d-r wf -r wd - wuf + wf -ud- ur - w +r +r wuf
+rw-2+2u+wd
> assign (Solution) ; simplify (sum (pi[ k] ,k=1. .3»;
1
286 7. Stochastic Processes

As in the inventory example, since we had to define TI as a vector with three

entries indexed as 1,2,3, the indices of the solution must be adjusted by shifting
them by -1, i.e., Pr(machine is in state j - 1) = 1r j. j = 1,2,3.

In some "pathological" cases, although the stationary distribution exists, the

limiting (steady-state) probabilities do not exist; i.e., limn-+oo p(n) does not con-
verge. As a simple example of this unusual case, consider the following problem.
Example 83 A "pathological" case. Suppose the transition matrix is given by

p=(~ ~).
When we compute the higher powers of P, we observe an interesting result:
> restart: # Pathological.mws
> Row{ 1] : =( 0, 1] ; Row{ 2] : =( 1, 0] ;
Row) := [0, 1]

:= [1, 0]
ROW2
> P:=matrix(2,2,[ seq (Row{ k] ,k=1..2)]);

P:= [~ ~]
> evalm(P"2);

> evalm(P"3);

> evalm(P"4);

I~
These imply that

p(n) = ~ ~~ if n is even
o1 01 ' fn 'IS odd;
1

thus the limit limn-+oo p(n) does not exist. Hence, this Markov chain has no limit-
ing probabilities.
However, if we solve the linear system TI = TIP with TIe = 1, we find 1r) =
1r2 = !
as the stationary distribution of this pathological Markov chain:
> pi : =vector ( 2 , [] ) ;
 7.4 Discrete-Time Markov Chains 287

1r := array(1 ..2, [])

> EqnList:=evalm(pi - pi &* P);

EqnList := F\ - 1r2, 1r2 - 1r 11
> EqnSetR:=convert(EqnList,set);
EqnSetR := {1r \ - 1r2, 1r2 - 1r d
> EqnSet:=EqnSetR minus {EqnList[ 1]};
EqnSet := {1r2 - 1r d
> SumProb: ={ sum (pi[ k] , k=l .. 2) =1} ;
SumProb := {1r \ + 1r2 = I}
> Sys:=EqnSet union SumProb;
Sys:= {1r2 -1rJ, 1r\ +R'2 = I}
> Solution:=solve(Sys);
I I
Solution:= {1r\ = -, 1r2 =-}
2 2

What is interesting about this case is that although Iimn--+oo pen) does not exist,
the process periodically visits states I and 2 in a deterministic fashion. Thus, the
process spends 50010 of the time in state I and 50% of the time in state 2, implying
that the long-run proportion of time the system stays in state j = I, 2 is !. It
can be shown that, in general, the probabilities 1r j found by solving II = llP
and lle = I can be interpreted as the long-run proportion of time the (periodic)
Markov chain is in state j; see Ross [157, p. Ill].

7.4.3 Classification ofStates

The states of a Markov chain can be classified according to some properties of
the system. By classifYing the states of a Markov chain, it becomes possible to
identifY its limiting behavior.
We say that state j is reachable (or, accessible) from state i if prj> > 0 for
some n ~ O. This is denoted by i ~ j. For example, if

o (\ \)
P2 = I fi i '
then state I is reachable from 0 (0 ~ I) but 0 is not reachable from I (I ..... 0).
States i and j are said to communicate (denoted by i ~ j) if they are
reachable from each other. For example, if

x
x
288 7. Stochastic Processes

where x denotes a positive probability, then 2 -+ I (since 2 -+ 0 and 0 -+ I),

o ~ 2, but I ...... 2. Thus, in this Markov chain not all states communicate.
The set of all states that communicate with each other is called an equivalence
class. For example, if
x
x

then states to, I} and {2, 3, 4} form two distinct equivalence classes. When a
Markov chain has a large number of states it may be difficult to identify its equiv-
alence classes by visual inspection. For an efficient algorithm for identifying the
equivalence classes, see Kao [101, pp. 172-173].
If all the states of a Markov chain belong to one equivalence class, the chain
is said to be irreducible. In such a Markov chain all states communicate. For
example, the Markov chain representing the (s, S) inventory system is irreducible
since all its states communicate. The three Markov chains with transition matrices
P2, P3 and P4 are not irreducible.
The equivalence classification of states takes into account the external (i.e.,
between "neighbors") relationship between states. The following concepts of pe-
riodicity and recurrence identify states with respect to their internal nature.
The period d(i) of a state i is the greatest common divisor (g.c.d.) of all in-
tegers n = I, 2, ... for which pf;)
> O. When the period of a state is I, the
' calied apen·od·IC. For exampIe, suppose Pii(I) > 0,Pii(2) > 0,Pii(3) = 0,
state IS
pf:) > 0, .... Since the g.c.d. of I, 2, 4, ... is I, the state i is aperiodic. In the
random walk model in Example 78, for all i we have pf~) > 0 and n+l ) = 0 p::
for n = 0, I, 2, .... Thus, d(i) = 2 for all i and the chain is periodic.
Now define the first passage time probability as

J/tn) = Pr(Xn = i, X" #- i, k = 1,2, ... ,n -II XO = i).

Thus, J/tn) is the probability that the process visits state i for the first time in n
steps after starting in state i. The probability of ever visiting state i after starting
in the same state is then J;i = L~I !;~n).
The state i is said to be recurrent (persistent) if J;i = I, transient if J;i < 1.
This means that for a recurrent state i, the process eventually returns to i with
probability I. Thus for a recurrent state i we have L~ I !;~n) = I, implying that
(J/tn), n = 1,2, ... } is a p.d.f. It is usually difficult to test J;i for the value it takes,
but it can be shown that state i is recurrent if and only if ~I pf;) = 00; see
Ross [157, p. 105].
Example 84 Random walk revisited As an example of identifying the recurrent
and transient states of a Markov chain, consider again the random walk model
 7.4 Discrete-Time Markov Chains 289

introduced in Example 78. Here the transition probabilities are given as

Pi,HI =P = I - Pi,i-I, i = 0, ±1, ±2, ...

for 0 < P < 1. In this chain all states communicate, thus all states are either
recurrent or transient. We therefore consider state 0 and examine its properties.
Since it is impossible to return to 0 in an odd number of transitions, we have
(2n+I) _ 0 n = 0,1,2, ....
Poo -,

But, there is a positive probability (binomial) of returning to 0 after an even num-

ber of transitions:

(2n)
Poo = (2n)
n
n n (2n)! n n
P q = - - P q.
n!n!
n = 1,2, ....
This expression is simplified using Stirling's approximation of a factorial as n! '"
nn+! e-n£ where '",' is a shorthand notation for "approximated in the ratio."
> restart: *
RecurrenceRW.rnws
> Stirling:=n->n A(n+l/2)*exp(-n)*sqrt(2*Pi);
Stirling := n -+ n(n+I/2) e(-n) .../2i
Substituting and simplifying, Maple finds that the probability pg:) of returning
to 0 in an even number of transitions is
(2n) (4pq)n
,Jnii.
Poo =
> Even:=(Stirling(2*n)/(Stirling(n)*Stirling(n)))
*pAn*qAn;
1 (2 n)(2n+If2) e(-2n) .,fi ~ qn
Even·- -
"""""'--'-::=-~~~:-:--:--~";""'-
.- 2 ,jii (n(n+I/2»2 (e(-n»2
> c:=sirnplify(Even);
4n ~qn
c:= .;n,jii
We now analyze the properties of the infinite sum L:'I pfr).
Ifit converges,
then we conclude that the state 0 is transient; otherwise it is recurrent.
To simplify the analysis, we write r = 4 pq and first assume that 0 < r < I, i.e.,
!.
p '" Using the ratio test of calculus we compute Iimn -+ oo lan+ 1/an I. Intuitively,
!
when p '" there is a positive probability that the process, initially at the origin,
will drift to plus (or minus) infinity when p > q (or, p < q) without returning to
the origin.
> assurne(O<r,r<l);
> a:=n->rAn/sqrt(Pi*n);
290 7. Stochastic Processes

r"
a:= n -+ C7:
v 7rn
> limit(abs(a(n+l)/a(n)),n=infinity);
r
Since the limit converges to r e (0,1), we conclude that when p =f:. the sum !,
~I p:r)
converges and hence the chain is transient.
When r = 1 (or p = !),
the integral test of calculus reveals that the integral
(X>_I_ dn
11 .,f1in
diverges to infinity. Thus, in this case, the chain is recurrent.
> b:=n->l/sqrt(Pi*n);
1
b:= n -+ r=-=
v 7rn
> int(b(n),n=l .. infinity);
00
> evalf(int(b(n),n=1 .. 100000));
355.6964440

!,
Intuitively, when p = the process fluctuates around the origin and even if strays
away, it eventually returns to state O.
For a recurrent state i, mean recurrence time P;; is defined as the expected
number of transitions needed to return to i, i.e., Pii = L~I nJ/;"). If P;; < 00,
the state i is said to be positive recurrent, and if Pii = 00 the state i is null
recurrent.
Finally, we define a state to be ergodic if it is (i) aperiodic and (ii) positive
recurrent.
With these definitions, we are now ready to present a very important theorem
that identifies the long-run behavior of a Markov chain. For a proof, see Ross
[157, p. 109].
Theorem 13 An irreducible Markov chain belongs to one of the following three
classes:
Periodicity

Aperiodic Periodic
p;j
Transient
Null-Recurrent
Positive-Recurrent
-+ 0 and 7r j
do not exist
p;j -+ 7r j are uniquely p;j -+?
•
found from 7r j are long-run
II = llP and lle = I fraction of time in j
 7.4 Discrete-Time Markov Chains 291

We now discuss some examples where this theorem can be used to identifY the
limiting properties of the Markov chains.
Example 85 A recurrent and periodic chain. Consider the Markov chain with the
transition matrix

p=~(Hn).
! j4 0 0
First, we note that all states of the chain commWlicate; hence the chain is irre-
ducible and Theorem 13 can be applied.

After finding P" and printing the first few values of p~~) for n = 2, 3, ... , 50,
we see that p~~) = 0 for n = 1,2,3 and p~~) > 0 for n = 4,6, 8, .... This implies
that the chain is periodic with dO) = 2.
> restart: # PeriodicChain.mws
> Row[ 1] : ={ 0, 1, 0, 0]; Row[ 2] : ={ 0, 0, 1, 0] ;
Row[ 3] : ={ 0, 0, 0, 1]; Row[ 4] : ={ .2, 0, . 8, 0] ;
Row) := [0, 1, 0, 0]
Ro'W2 := [0, 0, 1, 0]
Row) := [0, 0, 0, 1]
ROW4 := [.2,0, .8, 0]
> P:=matrix(4,4,[ seq (Row[ i] ,i=1..4)]);

p:=[H! ~].2 0 .8 0
> for n from 2 to 50 do
Rn] :=evalm(PAn) od: n:~n':
> for n from 2 to 15 do
print (n,R n] [ 1,1]) od; n:~ n' :
2,0
3,0
4, .2
5,0
6, .16
7,0
8, .168
9,0
292 7. Stochastic Processes

10, .1664
11, 0
12, .16672
13, 0
14, .166656
IS, 0
Next, we define the partial sum SA: = L!=1 p~~) and find that the difference
SA:+ 1 - SA: approaches the constant 0.166. This has the implication that the infinite
sum L~1 p~~) diverges; hence all states of the chain are recurrent.

> for k from 2 to 50 do

S[ k] :=evalf(sum(p[ n] [ 1,1] ,n=2 •• k)) od:

> for k from 35 to 49 do

Difference[ k] : =S[ k+ 1] -S[ k] od;
Difference35 := .166666667
DifferenceJ6 := 0
Difference37 := .166666667
Difference38 := 0
Difference39 := .166666667
Difference40 := 0
Difference41 := .166666667
Difference42 := 0
Difference43 := .166666667
Difference« := 0
Difference45 := .166666667
Difference4fJ := 0
Difference47 := .166666667
Difference48 := 0
Difference49 := .166666667
At n = 50, the process has stabilized but the rows of pSo are not identical.
Thus, the chain does not possess limiting probabilities.

> evalf (evalm (p[ 50] ),4);

 7.4 Discrete-Time Markov Chains 293

.1~67 o .S333 0 ]
[ .1667 o .S333
.1667 o .S333 0
o o.1667 .S333
However, the linear system n = np with ne = I does have a solution imply-
ing that the stationary distribution exists. We find this distribution with Maple's
help as follows.
> pi: =vector ( 4 ,[] );
1r := array(1..4, [])
> EqnList:=evalm(pi - pi &* P);
EqnList := J.1r I - .2 1r 4, 1r 2 - 1r I, 1r J - 1r 2 - .S 1r 4, 1r 4 - 1r J]
> EqnSetR:=convert(EqnList,set);
EqnSetR:= {1rJ .S1r4, 1r1 - .21r4, 1r2 -1rI, 1f4 -1rJ}
-1r2 -
> EqnSet:=EqnSetR minus {EqnList[ 1]}; k:~ ~:
EqnSet := {1rJ - 1r2 - .S1r 4, 1r2 - 1r I, 1r4 - 1fJ}
> SumProb: ={ sum (pi[ k] ,k=1 •. 4) =1} ;
SumProb:= {1r1 + 1r2 + 1rJ + 1r4 = I}
> Sys:=EqnSet union SumProb;
Sys:= {1r1 + 1r2 + 1rJ + 1r4 = I, 1rJ -1r2 - .S1r4, 1f2 -1fI, 1f4 -1rJ}
In this case 1r I = 1r2 = 0.OS3, 1rJ = 1r 4 = 0.4166, indicating that the process
spends about S.3% of its time in states I and 2 and about 42% of its time in states
3 or 4.
> Solution:=solve(Sys);

Solution:= {1r1 = .08333333333, 1r4 = .4166666667, 1rJ = .4166666667,

1r2 = .OS333333333}
> assign(Solution); sum(pi[ k] ,k=1..4);
1.000000000

7.4.4 Imbedded Markov Chain Technique

Consider the single-server queueing system with Poisson arrivals and general ser-
vice time S with distribution B(t), density b(t) and mean E(S) = 1/ p.4 We are
interested in the properties of the stochastic process X(t) defined as the number
of customers in the system at time t.

4As we will discuss in more detail in Chapter 9, Queueing Systems, this queue is denoted by the
shorthand notation M / G/ I .
294 7. Stochastic Processes

Naturally, this process is not a (discrete-time) Markov chain since its time index
t is continuous. The process X(/) is not even Markovian since the knowledge of
the system at time 1 is not sufficient to determine the distribution of the state at
time 1 + s. For example, suppose we wish to determine the probability that there
is one customer present in the system at time 1 + M given that there was one
customer present at time I. This probability can be written as

Pr[X(1 + M) = 1 I X(/) = 1] = [1 - 1M + o(M)]

x Pr{no departures in (I, 1 + M])
= [I - 1M + o(M)][1 - rB(u)M + o(~/)]
where rB(u)M is the probability that a customer who has already spent u time
units in service by time 1 will leave the system within the next M time units. Here,
rB(u) is the hazard rate function for the service time random variable S. For an
exponentially distributed service time S with parameter p, the hazard rate would
be a constantrB(u) = p. But when S is general, rB(u) requires the knowledge of
the past (prior to I), thus making the X(/) process non-Markovian.
Using the method of supplementary variables due to Cox [52], it is possible to
expand the state space of this process and transform it to a Markovian system. In
this section, we will describe a different technique known as the imbedded Markov
chain approach that uses the concept of regeneration point. This approach is due
to Kendall [105].
In order to make the process Markovian we proceed as follows. Instead of ob-
serving the system at all points of time I, we focus our attention on those epochs
when a customer leaves the system. We define Xn to be the number of customers
left behind by the nth departure, and Yn the number of customers arriving during
the service period of the nth customer (n 2: 1). This gives

X { Yn+l ifXn =0
n+l = Xn - 1 + Yn+l if Xn > 0,
indicating that the process Xn is now a Markov chain since Xn+l depends only
onXn .
To develop the transition probabilities Pij for this Markov chain, we need to
find the probability distribution of the number of customers that arrive during an
arbitrary customer's service interval. Conditioning on the length of the service
time we find

Pr(Yn =j) = 10 00
Pr(Yn =j I S=s)dB(s)

= roo
Jo
e-).s (1~;
J.
dB(s), j = 0, 1,2, ....
For i = 0, we have
poo = Pr(Xn+l = 0 I Xn = 0) = Pr(Yn+l = 0) = 10 00
e- As dB(s) = ao
 7.4 Discrete-Time Markov Chains 295

pOI = Pr{Xn+1 = 1 I Xn = 0) = Pr{Yn+1 = 1) = 10 00

e- As AS dB{s) = al

= J. I Xn = 0) = . 10 e- As (AS Y
00
pOj = Pr{Xn+1 Pr{Yn+1 = J) = . dB{s) = aj .
o J!
Proceeding in a similar manner for i = 1, 2, ... , we obtain the transition matrix
P of the imbedded Markov chain for the M / G /1 queue as

o ao al 02 a3
1 ao al 02 a3
P= 2 o ao al 02
3 o 0 ao al

The digraph for this Markov chain is displayed in Figure 7.8.

FIGURE 7.S. The digraph for the transition matrix of the embedded Markov chain of the
MIGII queue.

Inspection of the transition matrix and the digraph reveals when aj > 0 U =
0, 1,2, ...) the chain is irreducible and aperiodic. Since the chain has infinitely
many states, we need to examine its recurrence and transience properties to de-
tennine whether it is ergodic. We define p = E(Yn ) to be the mean number of
arrivals during service time of an arbitrary customer.5 Prabhu [152, p. 62] has
shown that the chain is positive recurrent, null recurrent or transient according to
p < 1, p = 1 or p > 1. We assume that p < 1 and proceed to find the stationary
distribution of the imbedded Markov chain using TI = TIP and TIe = 1.

SNotcthatp = E(Yn) = fo E(Yn I S=s)dB(s) = fooo AsdB(s) =.lE(S) = l/p.

296 7. Stochastic Processes

Writing the equations in the system TI = TIP explicitly, we obtain

HO = HoaO+Hlao
HI = Hoal + H lal + H200
H2 = Hoa2 + HI02 + H2al + H3aO

We let G(z) = L~ H"z" and A(z) = L~ a"z" be the probability-generating

function (p.g.f.) of {H,,} and {a,,}, respectively. Multiplying both sides of the jth
equation in the system TI = TIP by zj, summing and collecting the terms gives

G(z) = HoA(z)(z - I) (7.14)

z - A(z)
where H 0 is the stationary probability of the empty system.
Given the p.g.f. A(z), the only undetermined quantity in (7.14) is HO. We now
find an expression for HO with Maple's help.
We define A(z) and form the function G(z). Since we know that G(l) =
L~ a" = I, we attempt to solve for Housing this property. But this attempt fails
since we find that a direct substitution of z = 1 in G (z) results in G (I) = 0/0.
> restart: * ImbeddedMC.mws
> A:=z->Sum(~ n]*zAn,n=O .. infinity);

00

A := z -+ La"z"
,,=0
> G:=z-> (pi[ 0] *A(z)* (z-l» / (z-A(z»;
G := z -+ HO A(z)(z - I)
z -A(z)
> subs(z=l,numer(G(z»);
o
> subs(z=l,denom(G(z»);
00

1 - (La,,)
,,=0
Thus, with the hope offinding the limit of G(z) as z -+ I, we use I'Hopital's
rule and compute the following:

1 = lim G(z) = lim (Ho[A(z) + (z - I)A'(z)]}

=-+1 =-+1 [I - A'(l)]
> diff(numer(G(z»,z);
Top:=subs(z=l,diff(numer(G(z»,z»;
 7.4 Discrete-Time Markov Chains 297

00

Top:= 1£'0 (Lan)

n=O
> diff(denom(G(z)),z);
Bottom:=subs(z=l,diff(denom(G(z)),z));

I_(tan;nn)

00

Bottom:= 1 - (Lan n)
n=O
Thus,1 = 1£'0/(1 - p) since L~an = 1 and L~nan = E(Y) = p
implying that 1£'0 = I - p.
> solve (l=Top/Bottom, pie 0] ) ;
00

-I + (Lann)
n=O

In conclusion, the p.g.f. G(z) of the unknown probabilities {1£' j} is obtained as

G(z) = (I - p)(z - I)A(z).

z - A(z)
We can now compute the probabilities 1£' n using

l£'n = ~ (d"G(Z») . (7.15)

n! dzn ==0

It is also possible to compute the expected number of customers in the system by

evaluating
00

E(X) = G'(I) = Lnl£' n. (7.16)

n=O
Before we compute these quantities, we will present a simplification of the
p.g.f. A(z) and write it in terms of the Laplace transform (LT) of the service time
S. Since
A(z) = fanz n = f~ [10 e-At(~:n dB(t)].
00

n=O n=O
Maple evaluates this infinite sum as
> restart: # pgfA.mws
> A:=z->Int(exp(-lambda*t)*b(t),t=O .. infinity)
* Sum ( (lambda* t* z) "n/n! , n=O .. infinity) ;
298 7. Stochastic Processes

(A. t Z)n)
A := z --+ 10o 00
e(-At) b(t)dt L --,
COO
=0
-
n.
> value(A(z));

Rewriting this result as A(z) = Jo

oo e-().-).;)t dB(t) =
b(A. - A.z), we see that
the p.g.f. A(z) can be expressed as the LT b(-) of the service time S evaluated at
A. - A.z. Hence, the p.g.f. of the distribution {1rn} is given by

G(z) = (I - p)(z _- l)b(A. - A.z),

z - b(A. - A.z)
which is known as the Pollaczek-Khinchine (P-K) formula. We now present some
examples where the P-K formula is used to find the stationary distribution {1r n}.

Example 86 The M / M /1 model. In this model the service time S is assumed

exponential with rate J-l. With this assumption, Maple finds the p.g.f. G(z) as
follows where bLT is the Laplace transform of S.
> restart: # PKMM1.mws
> rho:=lambda/rnu;
A.
p:=-
J-l
> G:=z->((l-rho)*bLT(lambda-lambda*z)* (z-l))
/(z-bLT(lambda-lambda*z));
G := z --+ (I - p) bLT(A. - A. z)(z - 1)
z - bLT(A. - A.z)
> with(inttrans):
> b:=mu*exp(-mu*t);
b := J-l e(-p t)
> bLT:=unapply(laplace(b,t,z),z);

bLT:= z --+ _J-l_

z+J-l
> normal(G(z));
-J-l + A.
A.Z-J-l
After loading the genfunc () package, we invert the resulting p.g.f. explicitly
and obtain the well-known result for the stationary probabilities of the M / M /1
queue as 1rn = (I - p)pn, n = 1,2, ....
> with (genfunc) :
 7.4 Discrete-Time Markov Chains 299

> rgf_expand(G(z),z,n);

#
The exact values of 1C n for n = 1, 2, 3 and the expected number of customers in
the system E(X) = ).1(# -).) are also easily computed using (7.15) and (7.16),
respectively.
> for k from 1 to 3 do
pi[ k] :=subs(z=O,normal(diff(G(z),z$k)))/k!
od;

1C1 :=

).2 (_# + ).)

1C2 := - 3
#
).3 (_# +).)
1C3 :=
#4
> EX:=normal(limit(diff(G(z),z),z=l));

EX .-
·-

The next example deals with a more complicated case where the service time
S is Erlang with k stages (with mean ) 1#).
Example 87 The MI Ek/} model. Here we have essentially the same set of com-
mands as in the previous example except that the density bet) of k-stage Erlang
r. v. S is given by6
(#kltk-le-kl'l
bet) = (k _ I)! , t > O.

We set k = 2 and again find that Maple successfully inverts the resulting p.g.f.
G (z) and finds the stationary probabilities exactly.
> restart: # PKErlang.mws
> rho : =lambda/mu;
).
p:=-
#
> G: =z-> ((l-rho)*bLT(lambda-lambda*z)* (z-l))
/(z-bLT(lambda-lambda*z));

6This is a slighdy different but an equivalent way of writing the density of the Erlang r.v. with k
stages. Here, each stage is exponential with rate k I' and the mean duration of the Erlang is k[ I / (kl') ] =
1/Il; see Kleinrock [110, pp. 119-126].
300 7. Stochastic Processes

G := z ~ (I - p) bLT(A. - A.z)(z - 1)
z - bLT(A. - A. z)
> with(inttrans):
> k:=2;
k :=2
> b:= (rou* k) Ak* tA (k-l)*exp (-k*rou*t) / (k-l) !;
b := 41-'21 e(-2p t)
> bLT:=unapply(laplace(b,t,z),Z);
1-'2
bLT := z ~ 4 (z + 2 1-')2
> norroal(G(z));
4 I-' (I-' - A.)
A. 2 Z 2 -4A.zl-' -A.2z+41-'2
> with (genfunc) :
> pi:=unapply(rgf_expand(G(z),z,n),n);

J8A.I-'+A. 2 1-'(I-'-A.)(2 ~ 2)n

1C := n ~ -8 4 I-' + A. + 8 A. I-' + A.
A. (8 I-' + A.) (4 I-' + A. + J8 A. I-' + A. 2)
J8 A. I-' + A. 2 I-' (I-' - A.)(-2 A. )n
_ 8 -4 I-' - A. + J8 A. I-' + A. 2
A. (8 I-' + A.)( -4 I-' - A. + J8 A. I-' + A. 2)
To check the results we let A. = 5 and I-' = 7 and compute the first 1000 terms
of 1C n, n = 0, I, 2, ... , 1000 which gives 'L!c::/
1C n = 1.0, as expected.

> lambda: =5; rou:=7;

A. := 5
I-' := 7
> Sum(pi(n),n=O .• lOOO); evalf(%);

1000 (
~
112 J305(10 I
33 + v305
MM)n 112 J305(-1O I
-33 + v305
MM)n)
~ - 305 33 + J305 - 305 -33 + J305

1.000000000
> 1 ambda : =' 1 ambda' ; rou: =' rou' ;
A. := A.
1-':=1-'
 7.4 Discrete-Time Markov Chains 301

The first few tenns in {tr n} and the expected mnnber of customers E(X) are
also found exactly as follows.
> for k from 1 to 3 do
pi[ k] :=subs (z=O,normal (diff(G(z), z$k))) /k!
od;
1 {Jl - l)A. (4 Jl + ;')
'1'1 := 4' Jl3

1 {Jl - ;')A.2 (12 Jl2 + 8;' Jl + ;.2)

tr2 := 16 Jls

1 (Jl - ;')A. 3 (l 3 + 40 Jl2 l + 32 Jl3 + 12;.2 Jl)

'1'3 := 64 Jl7
> EX:=normal(limit(diff(G(z),z),z=l));

EX:= ~ (-;. + 4 Jl)A.

4 Jl(Jl-;')

7.4.5 Transient Behavior ofMarkov Chains

As we discussed in Section 7.4.1, the n-step transition matrix P(") can be com-
puted by finding P". However, as n increases, the computation ofP" may become
tedious and may even give rise to numerical inaccuracies for some ill-conditioned
transition matrices. We now present an exact approach for computing the transient
probabilities p(n) that is based on generating functions. This approach is a partic-
ularly attractive one since Maple has powerful facilities for inverting generating
functions.
First recall that
p(n) = p(II-I)p (7.17)
= p(O)p" , n = 1,2, ...
where p(O) is defined as the identity matrix I. We define
00
G(z) = Lp<n)zn
n=O
as the matrix-generating function of p(n). Multiplying both sides of (7.17) and
summing over n = 1, 2, ... gives
00 00
LP(n)zn = LP(n-l)pz".
n=1 n=1
Adding and subtracting P(°)zO, we get
00 00
LP(n)z" + p(O)zO _ p(O)zO = z Lp<n-I)pz"-I
n=1 11=1
302 7. Stochastic Processes

G(z) - p(O)ZO = zG(z)P.

Solving for G(z), we find

G(z) = P(O)(I - zp)-I.

Since pen) = p(O)pn and G(z) is the generating fimction ofp(n), the matrix ex-
pression (I -zpr l must be the generating fimction of the nth power pn as the
following diagram indicates:
G(z) = p(O) (I - zp)-I
~ ~ ~
pen) = P(O) P"

Thus, to compute the n-step transient probability matrix pn, one only needs to
invert the matrix-generating fimction (I -zP)-I. We now present an example of
this method.
Example 88 Transient solution ofa three-state Markov chain. Consider a Markov
chain with the transition matrix given as

p = (1~2 ~ 1~2).
3/10 1/2 1/5
After loading Maple's linalg () and genfunc () packages, we find the in-
verse of the matrix (J -zpr l whose entries are fimctions ofz.
> restart: # TransientMC.rnws
> with (linalg) :
Warning, new definition for norm

Warning, new definition for trace

> with(genfunc);

[rgf_charseq, rgf_encode, rgf_expand, rgf.findrecur, rgf_hybrid,

rgf_norm, rgfyjrac, rgfJelate, rgf_sequence, rgf_simp, rgf_term,
termscale]
> Row{ 1] : 9: 0, 1 , 0] ;
Row I := [0, I, 0]
> Row{ 2] : 9: 1 / 2 , 0, 1 /2] ;
I I
RoW]. := [2' 0, 2]
> Row{ 3] : 9: 3/10, 5/10, 2/1 0] ;
3 I I
Row) := [10' 2' 5]
 7.4 Discrete-Time Markov Chains 303

> P:=matrix(3,3,[ seq (Row{ k] ,k=1._3)]);

1
!~~
01 01 01
[
P:=

> ID:=array(identity,1 .• 3,1 •. 3);

ID := array (identity, 1..3, 1..3, 0)

> IDminuszP:=evalm(ID-z*P);
1 -z
1
[ --z
IDminuszp := ;
I
- - z --z
10 2

> inv:=inverse(IDminuszp);

-20+4z+sz2 4 z(-S +z)

%1 %1
z(lO+z) 4 -S+z z
inv := -10%1
%1 %1
z(Sz+6) _2z(S+3z) -2+z2
%1 %1 10 %1
% 1 := -20 + 4 z + IS z2 + z3
By applying rgf _expand () to each element of the matrix (I -zP)-I, Maple
inverts the generating functions. This operation generates the exact expressions
for each of the nine entries of the p(n) matrix denoted by Pn:

> Pn:=map(rgf_expand,inv,z,n);
304 7. Stochastic Processes

> p[ 1,1] : =unapply (Pn[ 1,1] ,n);

 7.5 Continuous-Time Markov Chains 305

PI,I :=n ~
98 262 /'1i" 1 n 262 /'1i" 98 1 n
11 (n-407 vll )( 8-2ffi) + (407 vll +n)( 8+2ffi)
37 + 8 - 2ffi 8 +2ffi
> evalf (p[ 1,1] (5));
.218500000 1
Taking the limit as n ~ 00 of each entry gives the limiting probabilities of
the Markov chain. The reader would appreciate that performing these inversions
manually without the help of Maple would be very difficult, if not impossible.
> rnap(lirnit,Pn,n=infinity);

_55%1 80 50
--%1 --%1
37 37 37
_ 55 %1 _ 80 %1 _ 50 %1
37 37 37
55 80 50
--%1 --%1 --%1
37 37 37
1
%1 := ------::=---=_
(-4 + ffi) (4 + ffi)
> evalf(%);
.2972972971 .4324324322 .2702702701]
[ .2972972971 .4324324322 .2702702701
.2972972971 .4324324322 .2702702701

Thus, the long-run probability of finding the process in state j = 1, 2, 3 is

independent of the initial state.

7.5 Continuous-Time Markov Chains

As we discussed in Section 7.4, if a stochastic process (Xn, n = O,I, ... J is a
discrete-time Markov chain with state space S, it spends exactly one time unit
in some state i E S before making a transition. With probability Pii the process
returns to the same state and with probability 1 - Pi; it makes a transition to
another state j =I i. Hence, the probability distribution of time spent in state i is
geometric with parameter 1 - Pii. That is, if we define N; as the (random) number
of time units the process spends in i, then its density is given by Pr(N; = k) =
p~-I(1 - Pii), k = 1,2, ... with mean duration E(N;) = 1/(1 - Pii). We now
present a continuous-time analogue of this process.
Consider a continuous-time stochastic process (X(t), t 2: OJ that spends an
exponential length of time, T;, with parameter V; in state i E S; that is, the density
306 7. Stochastic Processes

ofT; is !r;(/) = v;e-v;t, I ~ 0 with mean duration E(T;) = I/v;. The parameter
V; of the exponential distribution is the transition rate out of state i per unit time.
When the process leaves state i, it enters state j with probability P;j where Pii =
o and L.j# P;j = I for all i. Hence, the process (X(/), I ~ O} goes from state
to state according to a Markov chain with transition matrix P = [p;j], but each
time it visits a state, it stays there a random length (exponential) of time that
is independent of the past behavior. Now consider two states i and j such that
i '" j. Since the process leaves state i with rate V; and goes into state j with
probability P;j, we define q;j = V; P;j as the trans ilion role from i to j. Note that
as Ljji P;j = I we have L.j# % = v;. The q;j values constitute the entries of
the i~nilesimal generalor matrix Q = [%] of the CTMC.
Such a process (X(/), I ~ O} is a continuous-time Markov chain (CTMC) with
state space S. The process has the property that for i, j E S,
Pr[X(1 + s) = j X(s) = i, X(u) = x(u), 0 su < s]
= Pr[X(1 +s) = j I X(s) = i)
for all s, I ~ 0 andx(u), 0 SuS s.
Analogous to a discrete-time Markov chain, in a CTMC the conditional distri-
bution of X(I + s) given the past history over 0 SuS s depends only on the
current state X(s) at time s. We say that (X(/), I ~ O} is a lime-homogeneous
CTMC if the conditional probability Pr[X(1 + s) = j I X(s) = i) is independent
of s. In this case we write

Pr[X(1 +s) = j I X(s) = i) = P;j(t)

where P;j(t) is called the transitionjunction from state i to state j. This quantity
is analogous to the transition probability P&)of a discrete-time Markov chain.
Many realistic problems arising in manufacturing and communication systems
are commonly modeled using continuous time Markov chains. The Markovian
queueing models that will be presented in Chapter 9, Queueing Systems, are all
CTMCs since, in those models, one assumes that interarrival and service times
are exponential, thus making the process Markovian.
We now discuss some examples of continuous-time Markov chains.
Example 89 Birth and dealh processes. Consider a CTMC that can make transi-
tions from state i only to its neighboring states i-lor i + I for i E S = {O, I, ... }.
When the process is in state i, the transition rates are given by q;,;+\ = At for
i = 0, I, ... and q;,;_1 = p; for i = 1,2, .... Usually, the state X(t) of the pro-
cess represents the size of some population (e.g., customers in a queueing system).
In the case of a queueing system with a single server this corresponds to the fol-
lowing: When there are i customers in the system, the customers arrive according
to a Poisson process with rate A; and the service time is exponential with rate p;,
meaning that the arrival and service rates can be, in general, state dependent. The
parameters A; and p; are called the birth rates and the death rates, respectively.
Note that since Lj# q;j = V;, the transition rate out of state i is
 7.5 Continuous-Time Markov Chains 307

Vi = { AO, for ~ = 0
Ai + Pi' for I = 1,2, ...
and the infinitesimal generator Q is given by

~ (:\ A?O AI
Q= 2 P2? A2
·· .. ..
· . .
The entries with question marks (?) in the infinitesimal generator matrix Q will
be identified shortly.
To compute the transition probabilities Pij we first define Xi as the time until
the next birth (exponential with rate Ai) and Yi as the time until the next death
(also exponential but with rate Pi)' Thus, pOI = Pr(a birth at some time in the
future) = Pr(Xo < 00) = 1. To compute PI2 = Pr(a birth before a death) =
Pr(XI < YI), we proceed by conditioning on the occurrence of the time of death,
Y\, which gives

PI2 = Pr(XI < YI) = 10 00

Pr(XI < YI I YI = t)Ple-Plt dt
= 10 00
{l - e-AI')Ple-PI ' dt.

We compute this integral with Maple as follows.

> restart: # p12.mws
> assume (lambda[ 1] >O,mu[ 1] >0) ;
Error, (in assume) indexed names cannot be assumed
Since Maple is unable to use the command assume () with the indexed vari-
ables, we temporarily remove the indices from AI and PI to compute the integral.
> assume(lambda>O,mu>O);
> p[ 1,2] : =Int ( (l-exp (-larnbda* t) ) *mu* exp (-mu* t) ,
t=O .• infinity) ;

PI.2:= 10 00
{l-e(-At>)pe(-pt>dt

> value(%);

Thus, in general, we obtain

Ai
Pi.i+1 = k, + Pi = 1 - Pi.i-I

as the transition probabilities of the birth-death processes.

308 7. Stochastic Processes

Example 90 Pure birth (Poisson) process. It is interesting to note that if no deaths

are possible (i.e., when #i = 0 for all i = 1,2, ... ) and if the arrival rate is a
constant (i.e., Vi = ..ti = ..t for i = 0, I, ...) the birth-death process reduces to the
ordinary Poisson process with state space S = to, 1,2, ... }. For this process, the
transition probabilities are Pi,i+1 = =
I for; 0, I, ...

Example 91 A self-service car wash and vacuum facility. Consider a small self-
service car wash and vacuum facility where the service on a car is perfonned in
two stages, (i) wash and (ii) vacuum. We assume that there is only space for one
car at the facility, i.e., that queues are not permitted. Potential customers arrive
according to a Poisson process with rate ..t. If the facility is empty, the customer
enters, washes his car (time required distributed exponentially with rate #1), vac-
uums it (time required distributed exponentially with rate #2) and then departs. If
the facility is busy, the arriving customer goes elsewhere since there is no space
for waiting.
The stochastic process X(t) representing the status of the facility at time t is
a CTMC with state space S = to, 1,2} where state 0 corresponds to the empty
system and states I and 2 correspond to car being washed and being vacuumed,
respectively. Thus, the transition rates are vo = ..t, VI = #1 and V2 = #2. Since
the operations are perfonned sequentially, the states are visited in the order 0 --+
I --+ 2, implying that the transition matrix P = [Pij] for this CTMC is

p-! (H !)
The infinitesimal generator Q = [qij] is then

Q= ~ (~#2
2
;
0
:1).
?

7.5.1 Kolmogorov Differential Equations

In discrete-time Markov chains the n-step transition probability matrix P<II) =
[Pt)] was computed using p(lI) = P<"-I)p where pt) = Pr(XII = j I Xo = i).
Using the Chapman-Kolmogorov equations, it is easy to show that pt+ lI ) =
Pr(Xm+1I = j I Xo = i) = LkeS P:;') p!j), implying that p<m+lI) = p<m)p<").
We now consider the continuous-time analogues of these quantities.
As we indicated at the start of this section, for a time homogeneous CTMC, we
have

Pr[X(t + s) = j I X(s) = i) = Pr[X(t) = j I X(O) = i) = Pij(t)

where Pij(t) is the probability that the process that is in state; will be in state
j after an additional time t. It follows that the continuous time version of the
 7.5 Continuous-Time Markov Chains 309

Chapman-Kolmogorov equations is given by

Pr[X(t +s) = j I X(O) = i) = Pij(t +s) = LPik(t)Pkj(S), (7.18)
keS
which follows from the Markovian property of the CTMC. Unfortunately, in its
present form, this result is not very useful as it cannot be used to solve for the
unknown functions Pij(t).
Before we develop differential equations whose solution would, in principle,
provide us with the probability functions Pij(I), we need to discuss a lemma.
Lemma 1 For state i in S, we have
. 1 - Pii(h)
Iun h = Vi,
h-+O

i.e., the limit is equal 10 the transition rate Vi oUl ofstate i. Additionally, for any
two slates i and j in S,
I .un -
Pij(h)
h-+O h- -q ..
- IJ,
i.e., the limit is equal 10 the transition rate qij from i 10 j.

Proof: Note that since Pii(h) = Pr[X(h) = i I X(O) = i), we find

1 - Pii(h) = Pr[X(h) is anywhere but i I X(O) = i)
= Pr[a transition in h I X(O) = i) = vih + o(h).
Dividing by h and letting h --+ 0 gives the first result. Similarly, since Pij(h) =
Pr[X(h) = j I X(O) = i) = VihPij +o(h), dividing by h and letting h --+ 0 gives
the second result. •

Kolmogorov's Backward Differential Equations

Using the Chapman-Kolmogorov equations (7.18), we can write Pij(h + I) =
L~ Pik(h)Pkj(t). This is the conditional probability that the process will be
in state j at time h + t given that it was in state i at time 0 and it is obtained
by conditioning on the state back at time h. Subtracting Pij(l) from both sides,
removing the term corresponding to k = i and collecting the terms we have,
00

Pij(h + I) - Pij(t) = LPik(h)Pkj(l) - Pij(l)

k=O
00

= LPik(h)Pkj(l) - [1 - Pii (h)]pij (t).

k=O
k#
Dividing by h and letting h --+ 0 gives
00

P;j(l) = Vi LPikPkj(t) - Vi Pij(t)

k#
310 7. Stochastic Processes

00

= LqikPkj(l) - ViPij(l) (7.19)

k#
as KoImogorov's backward differential equations with the initial conditions Pij (0)
= ~ij, for i,j e S where ~ij is the Kronecker delta. The solution of this infinite
system of differential equations, if available, would give the transient probabilities
Pij(l) for I ~ 0.7
Suppose we now define qi; = -Vi. Rewriting (7.19), we get
00

P;j(l) = LqijPkj(I),
k=O
or, using matrix notation,
P'(I) = QP(I), with P(O) = I
where I is the identity matrix. We see that with this definition, the diagonal ele--
ments of the ith row of the infinitesimal generator Q matrix is the negative of the
sum of all other elements in row i. For example, for the birth-death process of
Example 89, the Q matrix is
-AO AO

Q=:
( PI
7
-(PI + AI) AI
-(PI:+AI) ~2

Similarly, for the car wash model in Example 91, we have

Q= (~A _~I
P2 0
:1).
-P2
We first discuss a simple example with N = 2 states.
Example 92 A Markovian queue wilh no wailing space. Consider a machine that
works (stays ON in state I) for a random length oftime distributed exponentially
with rate p. When the machine breaks down and moves to the OFF state 0, it takes
a random length of time to repair it that is distributed exponentially with rate A.
This two state CTMC (N = 2) is thus a Markovian queue with a single server and
no waiting space. It is clear that for this model the transition rates from 0 and 1
are Vo = A and VI = p, respectively. It then follows that qOI = A and qlO = p.
For i = j = 0 the KoImogorov differential equation is written as

p~(I) = qOIPIO(I) - VOpOO(l)

= APto(l) -ApOO(l)

7For a CfMC with N states, this system has N 2 ordinary differential equations. Its solution, if
available, gives the transient probabilities Plj(/) for I ~ 0 and i, j e (1,2, ... , N) .
 7.5 Continuous-Time Markov Chains 311

with POO(O) = 1. The other three differential equations are obtained in a similar
manner, which gives

( Poo(l) PC)) (I) ) _ (-A. A.) (1'00(1) pOI (I) )

Plo(l) PI I (I) - # -# Pto(l) Pl1 (I) .

This system of N x N = 4 equations can be written as P'(I) = QP(I), with

P(O) = I where P(I) = [Pij(I)] and ~j(l) = [p~j(I)].
We now use Maple to solve these differential equations. 8
MaplehasamatrixDE () command that can solve a system oftime-dependent.
linear ODEs of the form X'(I) = A(I)X(I) + 8(1). This command returns a pair
of matrices [K(I), L(I)] of dimensions n x n and n x 1, respectively. If8(1) = 0,
= =
then we also have L(I) O. In our case since P'(I) QP(I), the term 8(1) is not
present and thus L(I) = O.
A particular solution of the system is then written in the form P(I) = K(I)Co +
L(I) where Co is an n x 1 vector to be determined. Using the condition at 1 = 0
r
that P(O) = I, we find Co = [K(O) I P(O) = [K(O) I. r
> restart: # MatrixDE.mws
> with (DEtools) : with (linalg) :
Warning, new definition for adjoint

Warning, new definition for norm

Warning, new definition for trace

> Q:=matrix (2, 2,[ -lambda, lambda,mu, -mu] );

Q := [-A.# -#A. ]
> Sol:=matrixDE(Q,t);

e(-(A+Jl)I) ] ~
e(-(A~)I) # ,[0, O]J

> K: =Sol[ 1] ;

8Since POO{I) + POI (I) = 1 and PIO{I) + PII (I) = I, we could have considerably simplified the
solution of this system by solving the lower dimensional system with the unknown functions POO{I)
and PIO{I) as

poo{t) = -).poo{/) + Ap\O{I)

Plo{t) = Jlpoo{l) - JlPIO{/).

The present Maple worksheet is more straightforward since it solves the system P' (I) = QP{I) di-
recdy.
312 7. Stochastic Processes

eC-CA+Jl)I) ]
K:= [ eC-C)'~)I) p

> q 0) :=simplify(subs(t=O,inverse(K)));

Co:= [ 1 ~P
A.+p
> P:=evalm(K &* q 0) );

p %1 A.
- A.- - -
%1-A. ]
P := [ A. + p + A. + p A.+p A.+p
p %Ip A. %1 p
-----
A.+p
- -
A.+p A.+p
+ --
A.+p
%1 := eC-CA+Jl)I)
To summarize, the explicit solution P(t) for the linear system of differential
equations P'(t) = QP(t), P(o) = I for this two state CTMC is obtained as fol-
lows:

As a check, we see that the solution found satisfies the original systems of
ODEs.
> RHS:=evalm(Q &* P);

RHS:=
[ -A.(-P-+ %U)+A.(_p__ %Ip),
A.+p A.+p A.+p A.+p
_A.(_A.__ %U)+A.(_A._+ %IP)]
A.+p A.+p A.+p A.+p
p %U p %Ip
[
p(A.+p + A.+p)-p(A.+p - A.+p)'
A. %U A. %1 p ]
p(A.+p - A.+p)-p(A.+p + A.+p)
%1 := eC-CA+Jl)I)
> seq (seq (normal (diff(P[ i,j) ,t)-RHS[ i,j)),
i=1..2),j=1..2);

0, 0, 0, °
 7.5 Continuous-Time Markov Chains 313

Kolmogorov's Forward Differential Equations

Again using the Chapman-Kolmogorov equations (7.18), we can write Pij(t +
h) = L~ Pik(t)Pkj(h). This is the conditional probability that the process will
be in state j at time t + h given that it was in state i at time O. This equation is
obtained by conditioning on the state at time t. Following steps similar to the ones
used in developing the backward equations, we find
00

P;j(t) = LPik(t)VkPkj - VjPij(t)

k::Fj
00

= LPik(t)qkj - VjPij(t)
k::Fj

as Kolmogorov'sforward differential equations with the initial conditions Pij(O)

= oij for i,j E S where Oij is the Kronecker delta. Defining qjj = -vj. these
equations can also be written in matrix form as P'(t) = P(t)Q with P(O) = 1.9

The Exponential Matrix

To find the transition functions pet) for the simple Markovian queue in Example
92 we had to solve a system of DEs that required us to type several lines of Maple
commands. We now describe a method based on the spectral representation of
matrices that considerably simplifies the process.
First, note that if we had a scalar differential equation given by p'(t) = qp(t),
with p(O) = 1, the solution would simply be pet) = tflt = L~o(qt)n In!. It can
be shown that for the matrix DE system P'(t) = QP(t), P(O) = I, the solution
assumes a similar form as pet) = e Qt , where e Q is the exponential matrix defined
by e Qt = L~(Qt)n In!. The computation of the exponential matrix requires
the use of spectral representation theorem of linear algebra that is discussed in
Karlin and Taylor [102, pp. 539-541] and Medhi [131, pp. 196-197]. Essentially,
if the generator matrix Q of a CTMC has distinct eigenvalues, then Q can be
expressed as Q = ADA -1 where A is a non-singular matrix (formed with the
right eigenvectors of Q) and D is a diagonal matrix having its diagonal elements
as the eigenvalues ofQ. Then the transient solution of the CTMC is obtained as
P(t) = AeDtA-1 where eDt is a diagonal matrix constructed with the eigenvalues
ofQ.
Maple can compute the exponential matrix pet) = eQt with the exponen-
tial (Q, t) command as we will now show by re-solving the Markovian queue
problem of Example 92:
> restart: *
QueueExpQt.mws
> with (linalg) :

9For the Markovian queue example with no waiting space, the forward equations look different but
their solution is the same as the backward equations.
314 7. Stochastic Processes

Warning, new definition for norm

Warning, new definition for trace

> Q:=rnatrix(2,2,[ -lambda,lambda,rnu,-rnu]);

Q := [-A. A.]
/l -/l
> P:=exponential(Q,t);

%U+/l _ A. (-I +%1) ]

A.+/l A.+/l
P := [ /l (-I +%1) /l%1 +A.
A.+/l A.+/l
%1 := e(-(HI')t)
By specifying that both A. and /l are positive, we also show that the probabilities
converge (as in a discrete time Markov chain) to the same values independent of
the initial state.
> assurne(lambda>O,rnu>O);
> PLirnit:=rnap(lirnit,P,t=infinity);

' '1
PL Iml '-
.-
[ A.:
/l /l ~
A. A. /l ]
- - --
A.+/l A.+/l
As another-and more impressive----example of the use of exponen tial ( ) ,
consider the car wash facility of Example 91 represented by a CTMC with three
states. In this case, following the same steps we compute the transient probability
matrix P(I) as follows.
> restart: # CarWashExpQt.rnws
> with(linalg): with(DEtools):
Warning, new definition for norm

Warning, new definition for trace

Warning, new definition for adjoint

> Q:=rnatrix(3,3,[ -lambda, lambda, 0,
0, -rnu[ 1] , rnu[ 1] ,
rnu[ 2] , 0, -rnu[ 2]] );

Q := [~A. _~I
/l2 0 -/l2
:1 ]
 7.5 Continuous-Time Markov Chains 315

In this problem the solution P(I) takes up several pages of output. In order
to obtain a view of the general structure of the solution, we use the sprint ()
facility in the \share folder. This utility allows the user to look at the ''toJrlevel''
structure of an expression to obtain a bird's-eye view of the expression. 10
> P:=exponential(Q,t):
> with(share): with (sprint) :
See ?share and ?share,contents for information about the
share library

Share Library: sprint

Author: Monagan, Michael.

Description: Utility routine 'short print' for displaying

large expressions, allowing the user to look at the
structure (top levels) of a large expression.

> sprint(p[ 1,1]); sprint(p[ 1,2]); sprint(p[ 1,3]);

«+16 »
(Ill + 112 +A + J« +6» )(Ill + 112 + A - J« +6» )J« +6»
-2 A« +11 »
(Ill + 112 + A + J < < +6 > > )(Ill + 112 + A - J < < +6 > > ) J < < +6 > >
2 A III < < +9 > >
(Ill + 112 + A + J < < +6> >)(Ill + 112 + A - J < < +6> » J < < +6> >
The exact solution for the transient probabilities pOj (I), j = 0, 1, 211 are very
large expressions. For example, the first term p[ 1, 1] corresponding to [JOO(I)
contains a sum in its numerator with 16 terms indicated by < < + 16 > >. The
probabilities in the first row add to 1.
> Sum(P[ 1, j] , j=l •. 3) ;

> is(value(%)=l);
true
Ifwe specify the parameter values by letting, say, A = 10, III = 5 and 112 = 30,
the conditional probability that the system is empty at time 1 given that it was

l~ote that the share library must be installed before this utility [and other share () utilities]
can be used. Once installed, entering with (share) ; followed by? share, contents gives a
complete list of all available share utilities.
IlNote that the states 0, 1,2 in the CfMC correspond to the rows 1,2 and 3 of the Q matrix.
316 7. Stochastic Processes

empty at time 0 is obtained as a complicated (but exact) expression involving

exponential functions. For 1 = 0.12, the transient probability is found as 0.396.
> lambda: =10; rnu( 1] : =5; rnu( 2] : =30;
A. := 10
#1 :=5

#2:= 30
> p[ 1,1] ;

...!..«-11500e(-1/2(45+JiS)I) + 11500e(-1/2(4s-JiS)I)
25
+ 700 v'25 e(-1/2 (4S-Ji5)I) + 600 v'25 + 700 v'25 e(-I/2 (45+JiS) I»
v'25) /«45 + v'25) (45 - v'25»
> evalf(subs(t=0 . 12,p[ 1,1] »;
.3962472752
Finally, as 1 -+ 00, the limiting probabilities are found as [0.3,0.6,0.1] for
each row.
> PLirnit:=evalf(rnap(lirnit,P,t=infinity»;

.3000000000 .6000000000 .1000000000]

PLimil:= [ .3000000000 .6000000000 .1000000000
.3000000000 .6000000000 .1000000000

Unconditional State Probabilities

Analogous to the unconditional probabilities 1r }n) of a discrete-time Markov chain,
we now define 1r j(/) = Pr[X(/) = j] to be the unconditional probability that the
CTMC will be found in state j at time I. First, conditioning on the state at time I
we have 1r j(1 + h) = L~ 1r k(/)Pkj(h). Next, using arguments similar to those
employed in the development of the forward DEs, we find that
00

1rj(/) = L1rk(/)qkj -Vj1rj(/)

k~j

is the system of differential equations representing the unconditional probabilities.

Note that for a CTMC with a finite nwnber of states, say N, this system has only
N differential equations. Again defining q jj = -v j, these differential equations
can be written using vector/matrix notation as

TI'(/) = TI(/)Q

with TI (0) as the vector of initial probabilities.

 7.5 Continuous-Time Markov Chains 317

Here is the Maple solution of the unconditional probabilities for the Markovian
queue problem of Example 92 where n(O) = [co, clF is not specified but co +
CJ = 1. We solve this system of ODEs using the dsol ve command.
> restart:# TwoStateUncond.mws
> Q:=matrix(2,2,[ -lambda,lambda,mu,-mu] );

Q:=
-p,
[-Ap,
A ]
> pi : =vector (2, [ seq (pi. j (t) , j =0 .. 1)] ) ;
tr := [trO(/), tr 1(I)]
> RHS:=evalm(pi&*Q);
RHS:= [-trO(/) A + tr 1(/) p" trO(/) A - 1C 1(1) p,]
> for j from 0 to 1 do
DE. j : =di ff (pi. j (t) , t) =RHS[ j +1] od;

DEO := i; trO(/) = -trO(/) A + tr 1(I) P,

DEI := i; tr 1(I) = tr 0(/) A - tr 1(I) P,

> DESys : ={ seq (DE. j , j =0 •. 1)} ;

DESys := (i; tr 1(/) = trO(/) A-tr 1(1) p" i; trO(/) = -trO(/) A+ tr 1(/) p,}
> for j from 0 to 1 do IC.j:=pi.j(O)=q j] od;
ICO:= trO(O) = co
ICI := tr 1(0) = CI
> ICSys:={ seq(IC.j,j=O .. 1)} ;
ICSys := (trO(O) = co, tr 1(0) = cd
> dsolve(DESys union

I -
ICSys,{ seq(pi.j (t) ,j=O .• 1)} );

P, (CI + co) (A Co -CI p,)ec-rJ.-IP)

trO(t) - A
+p,
+ A
+p,
'

Ap, (CI + co) (A Co -CI p,)e(-IJ.-IP) p,

.1 (t) = _ - 1+p + p 1+p }

> simplify (%) ;

-ACI - ACO +%1 ACO -%1 CI P,

(tr 1(I) = A + p,

trO(/) = CI P, + co p, + %1 Aco - %1 CI P, }
A+p,
%1 := e(-(Hp)I)
318 7. Stochastic Processes

> expand(%);

l C) l co
{1C 1(t=--+---
) l co CI P
+~--:---7:--:-:--:~
l+p l+p (l+p)e(IA.)e(lp) (l+p)e(tA.)e(lp)'
CI P COP lco CI P
1C
O()
t = --+--+
l+p l+p (l+p)e(IA.)e(lp) (l+p)e(IA.)e(lp)
}
To summarize, the explicit solution TI(/) for the linear system of DEs TI'(t)
= TI Q is obtained as

TI(t) -
o( Hp
+ co~e-(Hp)1
....iL
A.+P
- cl....iLe-(Hp)1 )
I+Ii
- I l~p - Co l~p e-(Hp)1 + C) l+p e-(Hp)1 ,

*),
which, as t ~ 00, becomes

lim TI(/) = 01 ( l+p

1-+00

as expected.
Although the three-state car wash model of Example 91 is more complicated,
Maple can still solve explicitly for the unconditional probabilities TI (I). However,
the solution is very long and does not reveal any insights. Thus, we leave the
analysis of this example to the reader who can easily solve the model by replacing
the definition of the Q matrix in the Maple worksheet TwoStateUncond.mws
to (3,3,[ -lambda, lambda, 0, 0, -mul,mul,mu2, 0, -mu2] ) ; and by
changing all the limits on the index j from 0. • 1 to 0. . 2.

7.5.2 Limiting Probabilities

As we discussed in Section 7.4.2, the limiting probabilities 1C j = limn-+oo prj>
of a discrete-time Markov chain can be found-when they exist~y solving the
linear system TI = TIP with TIe = 1. One approach that can be used to find
the limiting probabilities of a eTMe is the Kolmogorov differential equations.
Suppose we consider the forward equations
P:j(t) = LPiI,(/)qkj - VjPij(/), Pij(O) = dij.
k#j

Ifwe assume that the limiting probabilities 1C j = lim Hoo Pij(/) exist, 12 then the
derivative P:j(t) must converge to o. This gives

(7.20)

12They will exist provided that the transition matrix of the embedded Markov chain for the CfMC
is irreducible and positive recurrent
 7.5 Continuous-Time Markov Chains 319

The ''balance'' equations (7.20) have a very useful interpretation. Since Vj is

the rate of leaving state j and lC j is the limiting probability of being in state j,
the quantity vj lC j is the steady-state output rate from state j. Similarly, qkj is the
transition rate from state k into state j for k :I: j. Thus, the sum ~~j lC jqkj is
the steady-state input rate into state j. In summary, we have, for any state j E S,

Output rate from j = Input rate to j.

Before using the balance equation approach to find the limiting probabilities
for some of the models discussed previously, we note the following.
As before, if we define qjj = -v j, the system of equations in (7.20) can be re-
written more compactly as 0 = nQ where Q is the infinitesimal generator matrix
oftheCTMC.
It should be interesting to note that for discrete-time Markov chains, the limit-
ing probabilities are also found using a similar expression, i.e., by writing n =
np as 0 = nQ where Q = P - I is a matrix with rows adding up to 0 (as in the
infinitesimal generator matrix Q of the CTMC).
Example 93 A Markovian queue with no waiting space revisited We know from
our previous discussion that the limiting probabilities for this model are

[lCO, lC J1 = [). ~ Jl' ). ~ Jll

Using the balance equation approach we have the following linear equations
for each state j = 0, I:

--=-_~I Rate Out = Rate In

State 0: ).lCo = JllCl
State 1: JllC I = ).lCO.

Since the probabilities must sum to unity, we also include lC 0 + lC I = 1 as another

linear equation.
As in discrete-time Markov chains, one of the balance equations is always re-
dundant. Hence, we remove, say, the first equation before solving for the two
unknowns lC 0 and lC I.
> restart: # TwoStateCTMCLirnit.rnws
> eq 0] : =lambda*pi[ 0] =rnu*pi[ 1] ;
:= ).lCO = JllC I
eqo
> eq[ 1] : =rnu* pi[ 1] =lambda* pi[ 0] ;
:= JllCl = ).lCO
eql
> Unit y: =surn (pi[ j] ,j=O .. 1)=1;
320 7. Stochastic Processes

Unity:= 1r0 + 1r1 = I

> Eqns:={ seq(eq[ k] ,k=l)} union {Unity} ;
Eqns := {P1r I = A. 1r 0, 1r 0 + 1r I = I}
> Vars:={ seq(pi[ j] ,j=O .• 1)};
Vars := (1r0, 1r d
> with (linalg) :
Warning, new definition for norm

Warning, new definition for trace

> solve(Eqns,Vars); assign(%);
P A.
{1r0 = - - , 1r I = --I
p+A. p+A.
Naturally, the results agree with what we had found before using different meth-
ods.
Example 94 The self-service car wash and vacuum facility revisited In this prob-
lem there are three states and the balance equations are obtained as follows:

Rate Out = Rate In

State 0: A.1r0 = P2 1r 2
State I: PI 1r 1 = A.1r0
State 2: P2 1r 2 = PI 1r 1

Noting, as usual, that L]=o 1r j = I and following the steps used in the Marko-
vian queue example, Maple finds that limiting probabilities are given by

[1r0, 1r1, 1r2) = [

P2PI
,
P2A. ,
A.PI].
.
P2PI + P2A. + PIA. + P2A. + PIA. + P2A. + PIA.
*
P2PI P2#1
> restart: CarWashCTMCLimit.mws
> eq[ 0] : =larnbda* pi[ 0] =rou[ 2] * pi[ 2] ;
eqo := A. 1r 0 = P2 1r2
> eq[ 1] : =rou[ 1] * pi[ 1] =lambda* pi[ 0] ;
eql :=PI 1r 1 =A. 1r 0
> eq[ 2] : =rou[ 2] * pi[ 2] =rou[ 1] * pi[ 1] ;
eq2 := P2 1r 2 = PI1r I
> Unity:=sum(pi[ j] ,j=0 .. 2)=1;
Unity := 1r0 + 1r I + 1r2 = I
> Eqns: ={ seq (ecJ k] , k=1. .2)} union { Unity} ;
Eqns := {PIX' I = A.1r0, P2 1r2 = PI 1r I, 1r0 + X'I + X'2 = I}
 7.5 Continuous-Time Markov Chains 321

> Vars: ={ seq (pi[ j] ,j =0 .. 2)} ;

Vars := {no, 1C2, 1C I}
> with(linalg):

Warning, new definition for nonn

Warning, new definition for trace

> solve(Eqns,Vars); assign(%);

{1C - A. III 1C _ 112 III _ 112 A.}

2 - %1' 0 - %1 ,1C I - %1
%1 := 112 III + 112 A. + III A.
> pi[ 0]; pi[ 1] ; pi[ 2] ;
112 III

112 III + 112 A. + III A.

> normal (pi[ 0] +pi[ 1] +pi[ 2] ) ;

In this example, we entered each balance equation explicitly and then solved
the resulting equations after including L~=O n j = I to illustrate the use of the
"Rate Out = Rate In" approach. Some readers may feel that since the infinitesimal
generator Q of this CTMC is already available, it may be simpler to generate the
equation using the system 0 = IIQ. This is true, and the reader is invited to try
this approach to solve the problem with Maple's help.

Example 9S The self-service car wash and vacuum facility with extra capacity.
In order to increase the profitability of the car wash facility, the owner is con-
sidering increasing the capacity of the facility to two cars at a time-one at the
washing location, the other at the vacuuming location. \3 As before, there is no
possibility of forming a queue and the cars are first washed and then vacuumed.
With the additional capacity, a customer arriving when the wash space is free
can enter the system. After the customer finishes washing his car, he must wait
until the vacuuming space is free. This, of course, blocks the access to the wash
space for other potential customers. Once the vacuuming space is freed up, the
customer waiting in the wash space moves to the vacuum space and completes the

\3Cunendy, there is only one space at the facility.

322 7. Stochastic Processes

service. The arrival and service rates ..t, JlI and Jl2 have the same interpretation as
before. 14
In this more complicated problem, the states must be defined as a vector since
a knowledge of the number of customers in the facility, X(t), is not sufficient to
describe the system. Hence we define the system as (XI (t), X2(t» where XI (t)
is the state of the wash space (idle, busy or blocked) and X2(t) is the state of
the vacuuming position (idle or busy). With this definition of the state vector, we
obtain five possible values describing the system:

STATE DESCRIPTION
(0,0) Empty system
(1,0) Busy car wash, idle vacuum
(0,1) Idle car wash, busy vacuum
(1,1) Busy car wash, busy vacuum
(b, 1) Idle car wash blocked by busy vacuum

With this description of state space we can now generate the balance equations
by equating the "out" rate to the "in" rate.

Rate Out = Rate In

State (0,0): ..t1r 00 = Jl2 1r 01
State (0,1): (Jl2 + ..t)1r 01 = JlI1r 10 + Jl21r hi
State (1,0): JlI 1r IO = ..t1r00 + Jl21r II
State (1,1): (JlI + Jl2)1r II = ..t1r01
State (b, 1): Jl21r hi = JlI 1r 1l

Maple finds explicitly the solution of the system [minus the equation for State
(0,0)] with the equation 1r00 + 1r01 + 1r 10 + 1r II + 1r hi = I:
> restart: # CarWashExtraCTMCLimit.mws
> eq[ 0] : =lambda* piC 0,0] =mu[ 2] * piC 0, 1] ;
eqo := ..t 1ro, 0 = Jl2 1ro, I
> eq[ 1] := (mu[ 2] +lambda)*pi[ 0,1] =mu[ 1] *pi[ 1,0]
+mu[ 2] *pi[ b, 1] ;
eql := (Jl2 + ..t) 1r 0, I = JlI 1r 1,0 + Jl2 1r h, I
> eq[ 2] : =mu[ 1] * pie 1, 0] =lambda* pie 0,0]
+mu[ 2] *piC 1,1] ;
eq2 := JlI1r 1,0 = ..t 1ro,o + Jl21r 1,1
> eq[ 3] : = (mu[ 1] +mu[ 2] ) * piC 1,1] =lambda* pie 0,1] ;

14This could also be the description of a sboeshine shop or a barbershop with two chairs. In the
shoeshine shop, the first chair is used to polish shoes and the second chair used to buff them. In the
barbershop, the first chair is used to wash the customers' hair and the second chair is used to cut it;
see, Goodman [74, Section 11 4. ] and Ross [158, Section 3.3].
 7.5 Continuous-Time Markov Chains 323

eq3 := (J.l1 + J.l2) 1r I, I = ..t 1r O,1

> eq[ 4] : =mu{ 2] *pi[ b, 1] =mu{ 1] *pi[ 1,1] ;
eq4 :=J.l2 1r b,1 =J.lI 1r I,1
> Unity:=pi[ 0,0] +pi[ 0,1] +pi[ 1,0] +pi[ 1,1] +pi[ b,l] =1;
Unity := 1r 0, 0 + 1r 0, I + 1r 1,0 + 1r I, I + 1r b, I = I
> Eqns:={ seq(eq[ k] ,k=1. .4)} union {Unity};

Eqns := {(J.l2 +..t) 1rO,1 = J.l11r 1,0 + J.l21rb, " J.l11rI,O =..t 1rO,o + J.l21r 1, 1,
(.ul +J.l2) 1r I,1 =..t 1r O,I, J.l2 1r b, I =J.lI 1r I,I,
1rO,O+1rO,1 +1rI,O+1rI,1 +1rb,1 = I}
> Vars:={pi[ 0,0] ,pi[ 0,1] ,pi[ 1,0] ,pi[ 1,1] ,pi[b,l]};
Vars := {1rO,O, 1rO, " 1r b,l, 1r 1,1, 1r I, O}
> with (linalg) :
Warning, new definition for norm

Warning, new definition for trace

> solve(Eqns,Vars); assign(%);

{1r _ J.l2 2 J.l1 (J.l1 + J.l2) _ J.l 22 ..t (.ul + J.l2 + ..t)
0,0 - %1 ' 1rI,O - %1
1r _(J.l1+J.l2)..tJ.l2J.l1 _J.l1 2 ..t 2 _..t 2 J.l2J.l1}
0, I - %1 ' 1r b, I - %1 ,1r I, I - %1
%1 :=J.l22 J.l1 2 +J.l2 3 J.l1 +2J.ll..tJ.l1 +J.l23 ..t +J.l22 ..t 2 + J.l2 J.l1 2 ..t
+..t 2 J.l2J.l1 +..t 2 J.l1 2
> normal (pi[ 0, 0] +pi[ 0,1] +pi[ 1, 0] +pi[ 1,1] +pi[ b, 1] ) ;

Since Maple collects many of the tenns in the solution, we print the 1rOO value
in a sJightly more detailed fonn.
> pi[ 0,0];

J.l2 2 J.l1 (J.l1 + J.l2) /(J.ll J.l1 2 + J.l2 3 J.l1 + 2J.l22 ..t J.l1 + J.l 23 ..t + J.l22 ..t2

+ J.l2 J.l1 2 ..t +..t 2 J.l2 J.l1 +..t 2 J.l1 2)

Example 96 Response areas for emergency units. Consider two emergency units
i = 1,2 (e.g., ambulances) that serve two areas j = A, B, in a city. Nonnally,
units I and 2 respond to emergencies in their respective areas A and B. However,
if a call arrives from either area when only one unit is available the call is served
by the available unit. When both units are busy, the arriving calls are lost (i.e.,
alternative measures are taken to deal with the emergency).
324 7. Stochastic Processes

We assume that arrivals of calls from area j follow a Poisson process with rate
Aj. Service times in the areas are independent random variables where the time
required to serve a call from area j = A, B by unit ; = 1, 2 is exponential with
rate Jl;j.
As in Example 95, this model requires the state to be represented by a vector
(XI (I), X2(1» where

0 if uniti is idle
X;(I) = { 21 if unit ; is serving area 1
ifuniti is serving area 2.

Thus the vector process (XI (I), X2(1» is a eTMe with state space S = {(O, 0),
(0, 1), (0, 2), (1, 0), (1, 1), (1,2), (2,0), (2, 1), (2, 2)} that consists of nine states.
Let 1C;j, ;, j = 0, 1, 2 be the limiting probabilities of this eTMe. Using the
balance equation approach we find the nine equations for this process and enter
them in the Maple worksheet as follows: IS
>
>
restart: *
Ambulances.mws
eq[ 1] := (lambda( 1] +lambda( 2] )*pi[ 0,0]
=mu[ 2,1] *pi[ 0,1] +mu[ 2,2] *pi[ 0,2] +
mu[ 1,1] *pi[ 1,0] +mu[ 1,2] *pi[ 2,0] ;
eql := (AI + A2) 1C O,O = Jl2,I1CO,1 + Jl2,2 1C O,2 + JlI,I1CI,O + JlI,21C2,O
> eq[ 2] : = (lambda( 1] + lambda( 2] +mu[ 2, 1] ) * piC 0, 1]
=rou[ 1, 1] * piC 1, 1] +mu[ 1, 2] * piC 2, 1] ;
eq2 := (AI +A2 + Jl2,1) 1C O,1 = JlI,I1CI,1 + Jll,2 1C 2,1
> eq[ 3] : = (lambda( 1] +lambda( 2] +mu[ 2,2] ) * piC 0,2]
=lambda( 2] *pi[ 0,0] +mu[ 1,1] *pi[ 1,2]
+mu[ 1,2] *pi[ 2,2] ;
eq3:= (AI + A2 + Jl2,2) 1CO,2 = A2 1C O,O + JlI,I1C 1,2 + JlI,2 1C 2,2
> eq( 4] : = (lambda( 1] + lambda( 2] +mu[ 1, 1] ) * piC 1, 0]
=lambda( 1] * piC 0,0] +mu[ 2, 1] * piC 1,1]
+mu[ 2,2] * piC 1,2] ;

eq4:= (AI + A2 + JlI, I) 1C 1,0 = AI 1C O,O + Jl2,11C I, I + Jl2,2 1C 1,2

> eq 5] :=(mu[ 1,1] +mu[ 2,1] )*pi[ 1,1]
=lambda( 1] *pi[ 0,1] +lambda( 1] *pi[ 1,0] ;
eq5 := (JlI, I + Jl2, I) 1C I, I = A I lr 0, I + A I 1C I, 0
> eq( 6] :=(mu[ 1,1] +mu[ 2,2] )*pi[ 1,2]
=lambda( 1] * piC 0,2] +lambda[ 2] * piC 1,0] ;

:= (JlI, I + Jl2,2) 1C 1,2 = AI 1C O,2 + A21C 1,0

eq6
> eq 7] : = (lambda( 1] + lambda( 2] +mu[ 1, 2] ) * piC 2, 0]
=rou[ 2,1] *pi[ 2,1] +mu[ 2,2] *pi[ 2,2] ;

ISFor the infinitesimal generator matrix Q of this process, see KIlO [101, p. 245]
 7.5 Continuous-Time Markov Chains 325

eq7 := (A.I +.h + JlI,2) 1r2,O = Jl2,1 1r 2,1 + Jl2,2 1r 2,2

> eq[ 8] : = (mu{ 1,2] +mu{ 2,1] )*pi[ 2,1]
=lambda[ 2] * pi[ 0, 1] +lambda[ 1] * pi[ 2,0] ;
eqs := (JlI,2 + Jl2,1) 1r2,1 = A2 1r O,1 + AI 1r 2,O
> eq[ 9] : = (mu{ 1,2] +mu{ 2,2] ) * pi[ 2,2]
=lambda[ 2] *pi[ 0,2] +lambda[ 2] *pi[ 2,0];
+ Jl2,2) 1r2,2 = A2 1r O,2 + A2 1r 2,O
eq9 := (JlI,2

Next, we introduce the equation L:=O L]=O 1r ij = I, define the nine equations
of the system and solve the resulting system using sol ve () .
> Unity:=sum(sum(pi[ i,j] ,j=O . . 2),i=0 •• 2)=1;
Unity := 1rO,O + 1rO,1 + 1rO,2 + 1r 1,0 + 1r 1,1 + 1r 1,2 + 1r2,O + 1r2,1 + lr2,2 = 1
> Eqns: ={ seq (eq[ k] , k=1. .8)} union { Unity} i

Eqns := {(AI + A2) 1rO,O = Jl2,I1rO,1 + Jl2,21rO,2 + JlI,I1r 1,0 + JlI,2 lr 2,O,
(AI + A2 + Jl2,1) 1rO,1 = JlI,I1r 1,1 + JlI,2 lr 2,1,
(AI + A2 + JlI,2) 1r2,O = Jl2,1 1r 2,1 + Jl2,21r2,2,
(JlI,2 + Jl2, I) 1r2, I = A2 1rO, I + AI 1r2,O,
(AI + A2 + Jl2,2) 1rO,2 = A2 1r O,O + JlI,I1r 1,2 + JlI,2 lr 2,2,
(AI + A2 + JlI,I) 1r 1,0 = AI 1r O,O + Jl2,11r 1,1 + Jl2,2 1r 1,2,
(JlI, I + Jl2, I) 1r I, I = AI 1r O,1 + AI 1r I, 0,
(JlI, I + Jl2,2) 1r 1,2 = AI 1r O,2 + A21r 1,0,
1rO,O + 1rO,1 + 1rO,2 + 1rI,O + 1rI,1 + 1rI,2 + 1r2,O + 1r2,1 + 1r2,2 = I}
> Vars:={ seq(seq(pi[ i,j] ,j=0 •• 2),i=0 •• 2)};
Vars:= (1rO,O, 1rO,I, 1rO,2, 1rI,O, 1r2,O, 1r2,1, 1rI,)' 1rI,2, lr2,2}
> with (linalg) :
Warning, new definition for norm

Warning, new definition for trace

> solve(Eqns,Vars): assign(%):
This model is originally due to Carter, Chaiken and Ignall (40), but it is also
discussed in Tijms (186). After presenting this model, Tijms [186, p. 116) states
(somewhat pessimistically): "These linear equations must be solved numerically."
But Maple succeeds in solving the linear system symbolically (i.e., exactly)! How-
ever, the solution for each unknown 1rOO, 1r01 and so on takes up several pages of
output. To appreciate the nature of the solution, the reader is invited to replace the
colon (:) in front of the pi[ 0,0] by a semi-colon (;).
> pi[ 0,0] :
A check reveals that probabilities add up to I.
326 7. Stochastic Processes

> simplify (sum(sum(pi[ i,j] ,j=O •• 2) ,i=O •. 2));

We again use share utility sprint () in Maple's \share folder.

> with(share); with(sprint);
See ?share and ?share,contents for information about the
share library
o
Share Library: sprint

Author: Monagan, Michael.

Description: Utility routine 'short print' for displaying

large expressions, allowing the user to look at the
structure (top levels) of a large expression.
["sprint"]
> sprint (pi[ 0,0] ) ;
#2. 2#1.2#2.1#1.1« +79»
«+275 »
Thus, we see that 1rOO is a very large symbolic expressio~a fraction----that has
a sum of79 tenns in its numerator and a sum of275 tenns in its denominator!
We can, of course, perform numerical analysis once we find the symbolic s0-
lution for the unknowns. For example, setting the values of AI, A.2, #Ib #12,
#21, #22 as
> lambda[ 1] :=0.1; lambda[ 2] :=0.125;
A.I :=.1
A2 := .125
> mu[ 1,1] :=0.5; mu[ 1,2] :=0.25;
mu[ 2, 1] : =0 • 2; mu[ 2, 2] : =0 • 4 ;
#1.1 :=.5
#1.2:= .25
#2.1:= .2
#2.2 :=.4
the numerical value of the probability of the idle system is computed as 1r 00 =
0.569.
> pi[ 0,0];
.5692678429
 7.6 Summary 327

Finally, note that the sum L.T=I L.]=I1f;j is the probability that both units are
busy and thus the incoming call is lost to the system. This probability can now be
computed easily as 10.48%.
> Surn(Surn(pi[ i,j] ,j=1..2),i=1..2); value(%);
2 2
L(L 1f ;,j)
;=1 j=1

.1048372876

Using the model presented in this example, one can analyze decision problems
such as the optimal number of units to assign in each location in order to minimize
the probability of lost calls or the optimal number of designated areas to serve
within the city boundaries.

7.6 Summary
This chapter started with a detailed description of the exponentiaVPoisson dual-
ity. Building on this duality, we presented renewal theory where the renewal func-
tion was computed explicitly using Maple's invlaplace () function that in-
verts a large class of Laplace transforms. Next, discrete-time Markov chains were
discussed and their limiting and transient behavior was analyzed using Maple's
sol ver and generating function inverter rgf expand ( ). We completed the
chapter with an analysis of continuous-time M3rlcov chains. Limiting probabili-
ties were again computed analytically using the solve () function. To compute
the transient probabilities, we used Maple's rna trixDE ( ) ,dsol ve () and ex-
ponen tial () functions. We can now start our analysis of specific applications
in inventory management and queueing processes.

7.7 Exercises
1. Provide a reasonable definition of the memoryless property for a nonnega-
tive discrete random variable N. Based on your definition show that P {N =
n} = (1 - p)p"-I, n = 1, 2, .. . is the unique discrete memoryless distri-
bution.

2. Let X and Y be independent exponential random variables with means

E(X) = 1/ A. and E(Y) = 1/J.l .

(a) Find the distribution of Z = min(X, Y)'

(b) Now, compute Pr(X < Y).
328 7. Stochastic Processes

3. Show that the probability generating function GN(t)(Z) = E [zN(t)] of a

nonhomogeneous Poisson process (N(/), I ~ O} with rate A(/) is GN(t)(Z) =
e(=-I)M(t) where M(/) = f~ A(u)du.

4. Suppose we have a counting process (N(/); I ~ O} where N(/) denotes

the number of arrivals in the interval (0, I). Consider the following infinite
system of differential-difference equations for Pn(/) = Pr[N(/) = n):

PM/) = -APo(/), Po(O) =I

P~(t) = -APn(/) + APn-I(/), n= 1,2, ...
Solve this system using generating functions.
HINT: For fixed I, define the probability generating function P(z, I) =
L~zn Pn(/), Izl < I. Next find a solution for P(z, I) and extract its
coefficients.
5. Let N(/) be a Poisson process with rate A. Assume that T is a random
variable with density g(/) = p.e- llt and define N as the number ofrenewals
in a time interval of random length (0, T). Find the probability Pr(N = n),
forn=0,1,2, ...
6. In a renewal process let A(/) = 1- SN(t) and B(/) = SN(t)+1 - I.

(a) Find Pr{A(/) ::;a}andPr{B(/) ::; b} for finite I.

(b) Assume that interarrival times are exponential with p. = E(X). Find
the above quantities. Then let I ~ 00. Do the results agree with your
intuition?
7. Consider a renewal process where the interarrival times are i.i.d. with den-
sity f(/) = aIAle-).lt + a2A2e-).2 t for t ~ 0 with al + a2 = I. Assume
al = 0.3, AI = I and A2 = 2 and find the renewal function M(/).

8. Consider the following transition probability matrix for a Markov chain.

0 3/4 1/4]
P = [ 1/4 0 3/4 .
1/4 1/4 1/2
Using Maple's genfunc () function, find the transient probabilities for
this chain, i.e., find pen) for finite n. Next, find the stationary distribution
{1r it j = I, 2, 3} and compare it to above when n ~ 00.

9. Let {Xn , n = 0, I, ... } be a Markov chain with state space S = {I, 2, 3}.
The transition probability matrix for this chain is given as

p= [ !pp q! !q 0]
o p q
 7.7 Exercises 329

with p + q = 1 and 0 < p < 1. Find the transient probabilities for this
chain. Med, 103

10. Let {Xn, n = 1,2, ... } be an irreducible and aperiodic MC with state space
S = {t, 2, ... , m} and Pij = Pji, i,j E S. Find lim Pr(Xn = k). (This
n-+oo
is an example of a doubly-stochastic matrix.)

11. Consider a CTMC with three states 0, 1 and 2. This CTMC has the infinites-
imal generator

where the values assumed by the diagonal entries denoted by • are such
that the row sums are zero. Find the long-run probability that the system is
in state O.

12. The infinitesimal generator Q of a three-state CTMC assumes the form

Q= [-;1 !;1
Il 0 -Il
2~].
Let 1 =
1 and Il = 8 and determine the transient probabilities of this
CTMC. Next, let t -+ 00 and find the limiting probabilities.
8
Inventory Models

8.1 Introduction
According to recent estimates the total value of inventories in the United States is
in excess oUI trillion [135, p. 184]. Abnost 80% of this investment in invento-
ries is in manufacturing, wholesale and retail sectors. Many of the mathematical
models that we will discuss in this chapter have proved very useful in manag-
ing inventories and their applications have resulted in substantial savings for the
companies employing them.
The term "inventory" can be defined as a stock of commodity that will be used
to satisfy some future demand for commodity. Due to the random nature of cus-
tomer arrivals and the amount required by the customers, future demand is usually
not known in advance. Inventory processes are influenced by the customers gener-
ating the demand, the management making the inventory order decisions and the
supplier(s) providing the required amount of the commodity to the management.
In order to satisfy ·future demands, management has to answer two basic ques-
tions relating to timing and magnitude: (I) When should an inventory replenish-
ment order be placed? (2) How much should be ordered? In the last 50 years,
thousands of inventory models have been developed to answer these questions
under different assumptions on the demand process, cost structure and the char-
acteristics of the inventory system. For recent reviews, see Barancsi et al. [15],
Porteus [151] and Lee and Nahmias [120]. The objective in almost all of these
models is to minimize the total costs of operating the system. This is achieved by
finding the optimal values of timing and magnitude decision variables that mini-
mize the objective function. (When the model involves random components such

M. Parlar, Interactive Operations Research with Maple

© Birkhäuser Boston 2000
332 8. Inventory Models

as random demand or random yield, the objective is to minimize the expected

cost.)
There are several reasons why an organization may want to keep inventories.
First, it may be either impossible or economically unsound to have goods arrive
in a system exactly when the demands occur. Although in manufilcturing orga-
nizations the just-in-time system has made inventories almost obsolete, the retail
sector still needs goods on the shelves to attract customers. Even in a family unit,
we see a type of inventory management when the weekly food shopping excur-
sions result in purchases oflarge quantities of bread, milk, etc., that may last the
family more than a few days.) This is the transaction motive for ordering large
lots and keeping inventories in order to reduce the fixed cost of ordering.
A second reason for keeping inventories is the precautionary motive, which is
based on the fact that it is usually very difficult to predict the random demand, and
hence inventory is a hedge against uncertainty [120]. The randomness of demand
and lead time are the most common filctors of uncertainty, but as we saw in the
1970s during the OPEC oil embargoes, in some cases even the supply can be
random, thus adding to the uncertainty in the inventory system.
Another reason for keeping inventories is the specUlative motive. When the
price of a commodity is expected to increase, it may be more profitable to pur-
chase large quantities of the commodity and keep it for future use when the price
will be higher. There are other motives - such as long transportation times and
logistics reasons - for keeping inventories. We refer the reader to an early paper
by Arrow [9] and the recent book by Nahmias [135, Section 4.2] for a discussion
of these and other motives for keeping inventories.

8.2 Classification of Inventory Models

In the last 50 years thousands of inventory models have been developed by op-
erations researchers. These models have ranged from simple extensions of the
classical economic order quantity (EOQ) model [81] to complex multilevel con-
trol systems [168]. In recent years some researchers have started investigating the
possibility of classifying the inventory models. Review papers by Aggarwal [6],
Nahmias [134] and Silver [174] and a book by Barancsi etal. [15] provideclassifi-
cations of inventory models that emphasize the wide variety of problem situations.
There were also some attempts to incorporate artificial intelligence technology to
automate the selection of the correct inventory model in a given situation. The first
paper dealing with such a problem appears to be by Parlar [142], who developed
a knowledge-based expert system (EXPIM) that could identify and recommend
up to 30 production/inventory models.

1While in North America a weekly grocery list may include several loaves of bread that may last
the family many meals, in many other countries bread is still purchased daily from the local bakery.
 8.2 Classification of Inventory Models 333

Among the classifications published in recent years, the one by Silver [174] is
probably the most detailed. In his scheme, Silver lists a nwnber of factors that
can each take on two or more values. For example, inventory models are clas-
sified according to the nwnber of items involved (single vs. multiple), nature of
demand (deterministic vs. probabilistic), length of decision horizon (single pe-
riod vs. multiperiod), time variability of the demand process (stationary vs. time
varying), nature of the supply process (deterministic or random lead time), pro-
curement cost structure (discount possibility), backorders vs. lost sales when there
is a shortage, shelf-life considerations (obsolescence or perishability) and single
vs. multiple stocking points.
One can define a set of "standard problems" that have mathematically devel-
oped optimal solutions. These problems have found wide applicability or have
potential applications with minor modifications. Among these problems, EOQ-
type models are classified as single item with deterministic demand and stationary
conditions. In multi-items problems, deterministic and stationary demand under
budget or space constraints are normally solved using the Lagrange multiplier
technique. The single-item problem with deterministic but time-varying param-
eters (also known as the dynamic lot size problem) was solved by Wagner and
Whitin [191] using dynamic programming.
In continuous-review models it is asswned that the inventory level is known at
all times. In many supermarkets that have scanning devices at the checkout coun-
ters, it is possible to keep track of the inventory level of any item. Hence, in these
systems transaction information is easily available that makes the application of
continuous-review models possible. In periodic-review models the inventory level
is measured only in discrete time points when inventory is physically counted.
The continuous-review (Q, r) policy where r is the fixed order point and Q is
the fixed order quantity is used to control the single-item problem with stationary,
probabilistic demand with known distribution. Variants of this policy such as the
(R, S) periodic replenishment system where R is the review interval and S is
the order-up-to level, and the (R, r, S) policy are also used, for which efficient
solution techniques have been developed. Decision rules are also available for
multi-item probabilistic stationary problem under a budget constraint. The single-
item, single-product problem (i.e., the classical newsvendor model) is the simplest
of the probabilistic inventory models and its variants with multi-items with budget
constraints are solved using the Lagrangian technique. The coordinated control
of items under deterministic or probabilistic stationary demand with or without
discounts and multiechelon stationary situations are more complicated models
for which decision rules are available.
For a description of the solution of these and other inventory models, we refer
the reader to the books by Banks and Fabrycky [13], Hax and Candea [84], Silver
and Peterson [175], and Tersine [185].
334 8. Inventory Models

8.3 Costs Associated with Inventory Models

A majority of inventory models are built with the assumption that the objective is
to minimize the total (expected) cost of procurement, holding (carrying) cost and
penalty cost for shortages.

8.3.1 Procurement Cost

This cost corresponds to the dollar amount spent to procure (or produce) the com-
modity. If we denote by c(y) the cost of procuring y units, then

(y) _ { K + cy if y > 0
(8.1)
c - 0 ify = 0

is a procurement cost function that has a fixed setup cost component K that is
incurred when an order is placed. When demand is probabilistic and a periodic-
review system is in place, such a cost structure results in the optimality of the
well-known (s, S) policy, as proved by Scarf [166].2

8.3.2 Holding (or Carrying) Cost

This cost is the sum of all costs that are proportional to the level of inventory on
hand at any time. These costs include mainly the opportunity cost of alternative
investments (i.e., the loss in revenue that results in having cash tied up in inven-
tory), taxes and insurance and cost of providing the physical space to store the
items. If x(t) is the on-hand inventory at any time t and h is the holding cost
per unit per time, then the total holding cost incurred from time 0 to time T is
hIt x(t)dt.

8.3.3 Penalty Cost

An inventory system is operated in order to satisfY the customers' demands, and
thus if the demands are not met there must be some economic loss. In inventory
literature the penalty cost is assumed to be a nondecreasing function of time t,
H(t), to represent the cost of each unit short for which the backorder remain on
the books. Usually (as in Hadley and Whitin [81, Section 1-11)) the function
H(t) is specialized to a linear form as H(t) = Hit + H2, where HI is the time

21n inventory literature the symbol s appears under different disguises. In Hadley and Whitin's
[8\, Chapter 4] continuous-review (Q, r) model where Q is the lotsize and r is the reorder point, s
is used to denote the safety stock, which is defined a<I the expected value of the inventory level when
a procurement arrives. In Scarf's [\66] periodic-review (s, S) model, s denotes the reorder point. In
this chapter, in order to present a development parallel to those in Hadley and Whitin and Scarf, we
use s as the safety stock when we discuss the continuous-review models. But when a periodic-review
model is under consideration, s corresponds to the reorder point.
 8.4 Detenninistic Inventory Models 335

dependent per unit backorder cost with dimension $Iunit/time and 1C 2 is the per
unit backorder cost with dimension $Iunit. [Here 1C, plays a role similar to the
holding cost h (S/unit/time) that is used in the computation of the holding cost
component.]
Let b(t) be the number of units on backorder at time t during a shortage interval
of [0, T]. Then Jei 1r ,b(t) dt + 1r2b(T) is the total penalty cost incurred during
[0, T]. In practice both 1r, and 1r2 are difficult to estimate since they usually
represent intangible factors such as the loss of goodwill resulting from a delay in
meeting demands. In a recent paper, <;etinkaya and Parlar [42] consider the two
backorder costs indirectly by introducing two safety-stock type constraints. Thus
in their form the problem becomes one of minimizing the ordering and holding
costs subject to two appropriate constraints.

8.4 Deterministic Inventory Models

We will begin our presentation by discussing some inventory models where de-
mand and every other problem parameter are deterministic and stationary. The
first example of such a model dates back to the early 20th century when Ford
Harris [83] and R. H. Wilson (195] discovered the EOQ formula. The assump-
tions of deterministic and stationary demands may seem very restrictive but as we
will see, the solution to the EOQ model serves as a good starting point even for
more complex and realistic problems.
Before we develop the optimization models, it is worthwhile to briefly describe
an important theorem from renewal theory. The renewal reward theorem (RR1}-
as discussed in Section 7.3.5 of Chapter 7, Stochastic Processes-is a powerful
tool used in the optimization of stochastic systems. Once a regenerative cycle of
a controlled stochastic process is identified, one forms an average cost objective
function simply as the ratio of the expected cycle cost to the expected cycle time.
The resulting objective function is then optimized with respect to the decision
variables of the model. For example, in an inventory problem, these decision vari-
ables may be the quantity ordered Q and the reorder point r (the level of inventory
that triggers an order). The usefulness of the RRT is in reducing an infinite horizon
optimization problem to a static nonlinear program that involves the computation
of only the first moments of the cycle-related random variables.
It is fortunate that even if we have a deterministic problem, the renewal reward
theorem may still be useful in forming the objective function provided that we
identify the regenerative cycles of the process in a correct manner. Once the cycles
are identified, we can then develop the average cost per unit time as the total cycle
cost divided by the total cycle length.

8.4.1 The Basic EOQ Model

In this model we make the following assumptions:
336 8. Inventory Models

1. The demand rate is known and constant (stationary) at 1 units per unit
time. 3

2. Shortages are not permitted.

3. Order leadtime is zero, i.e., the delivery is instantaneous.

4. There are three types of costs.

(a) Purchase cost is c dollars per unit and setup (order) cost is K dollars
per order.
(b) Holding cost is h dollars per unit per unit time. This cost is usually
defined as h = i c where i is the inventory carrying charge, the cost in
dollars of carrying $1 of inventory per unit time.
(c) Since shortages are not allowed, the shortage cost is infinitely high.

It should be clear that since leadtime is zero and demand is known with cer-
tainty, it is optimal to order only when the inventory level 1(/) at time I reaches
zero.4 As each order must be of the same size, we assume that the order quantity
is always Q units.
Referring to Figure 8.1, we see that a regenerative cycle starts when the in-
ventory level is increased to Q units. Since demand is 1 units per time, it takes
T = Q/ 1 time units for the cycle to end. Thus the deterministic cycle length is
simply
=
E(Cycle length) Cycle length Q/1. =
The cycle cost computation is slightly more complicated as we need to account
for the total cost of ordering and holding in a given cycle. At any time I E [0, T),
the inventory level is 1(/) = Q -11. Thus, during each cycle the total holding
cost is

The total cycle cost is found by adding the order cost K to the holding cost: s

E(Cycle cost) = Cycle cost = K + !h{f /1.

Now we let Maple take over and perform the necessary computations to find
the optimal order quantity.

3In this chapter the "unit time" will normally correspond to one year.
4For a rigorous proof of this result, see ArrC1W, Harris and Marschak [II).
SSince the demands must be met ultimately (and there is no quantity discount for larger orders),
we do not include the procurement cost per cycle cQ in the cycle cost If this cost were included,
then dividing it by the cycle length Q/l would give d, a constant that would not affect the optimal
decision.
 8.4 Deterministic Inventory Models 337

Inventory /(t)

--T=Q -+ Timet
A.

FIGURE 8.1. Inventory level fluctuations in the basic EOQ model.

> restart: * EOQ.rnws

We use the cycle cost and the cycle length denoted by EC and E L, respectively,
to write the expression for the average cost G(Q) EC/EL = =
IK/Q + !hQ.
> EC:=K+(1/2)*h*Q A 2/1arnbda;

> EL:=Q/larnbda;

EL:= Q
I
> G:=expand(EC/EL);
I K I
G:=Q+'2 Qh
Differentiating G once and then twice to find GQ and GQQ, we see that the
unique positive solution of the necessary condition will be the global optimal
solution since the second derivative is always positive for Q > O.
> GQ:=diff(G,Q);
I K I
GQ:=-Q2+'2 h
> GQQ:=norrnal(diff(G,Q$2»;
IK
GQQ:=2 Q3
Solving the necessary condition and recalling that every parameter is positive,
we pick +J2Kl/h as the optimal solution.
338 8. Inventory Models

> sol : =solve(GQ,Q);

sol:= ,Ji..r:rx,
> QOpt : =simplify(sol[ 1] ,power,symbolic);

QOpt:= ,Ji ~,JK

The minimum value of the average cost denoted by GOpt is also obtained as a
square root expression.
> GOpt:=subs(Q=QOpt,G);
GOpt := ,.fh ,Ji ,.fX ,JK
To show that the optimal cost is not very sensitive to small deviations from the
true optimal value, we substitute Q = t5 x QOpt in the objective function G
where t5 is a positive constant. Simplifying gives
> GNonOpt:=normal(subs(Q=delta*QOpt,G)/GOpt);
I 1 +~
GNonOpt := "2 - 0 -
Here, (GNonOpt - I) x 100 is the percentage deviation from the minimwn
cost if the order quantity is chosen as t5 x QOpt. Note that even if we deviate
from the optimal solution by 100010, i.e., when t5 = 2, we find
> evalf(subs(delta=2,GNonOpt));
1.250000000
implying that the cost increases by only 25%. When 0 is very, very large, an ad-
ditionallOOOIo increase in 0 (that is, ONew = Oold + I) increases the cost by only
50010. This is shown by differentiating GNonOpt with respect to t5 and letting
o~ 00, which gives 1/2.
> limit(diff(GNonOpt,delta),delta=infinity);

-
2

8.4.2 The EOQ Model with Planned Backorders

We now consider an extension of the basic EOQ model and assume that it may
be permissible for the inventory system to be out of stock when a demand occurs.
Many businesses such as capital-goods firms that deal with expensive products
and some service industries that cannot store their services operate with substan-
tial backlogs. Thus planned backorders may be an acceptable policy to follow in
some cases.
 8.4 Detenninistic Inventory Models 339

In this model, we retain every assumption we made while discussing the basic
EOQ model except the requirement that there must be no shortages. Here we as-
sume that the demands occurring when the system is out of stock are backordered
until a new shipment arrives. At that time all backorders are satisfied before the
shipment can be used to meet other demands.
We let s ~ 0 be the number of backorders on the books when an order is placed
and a shipment of Q units arrives immediately.6 After satisfying the backorders,
the cycle will start with Q - S lDlits, which will be depleted to zero after TI =
(Q - s)/ A. time units. If T is the cycle length, then T2 = T - TI is the length of
a subcycle during which backorders will be incurred. The inventory process for
this model is depicted in Figure 8.2.

Invemay /(/)
Q-s

1
L
8

-8 .
Avenge W81t w = -
r
2AQ

FIGURE 8.2. Inventory and backorder levels.

The inventory holding cost per cycle is

h loo
TI h
(Q -s - A.1)dl = -(Q _s)2
2A.

and the backorder cost per cycle is

6This is essentially a continuous-review model with zero leadtime and deterministic demand.
Hence the reorder point s in this model is also the safety stock.
340 8. Inventory Models

since T2 = s /1. Noting that the procurement cost can again be ignored (since the
demands will ultimately be satisfied), the average cost is found as

G
1K
=Q 1
+ 2Qh(Q 2 1 (I 2
-s) + Q !1fIS + 1f21S
)
These expressions are entered in Maple and the optimal solution is computed
as follows:
> restart: # EOQBackOrder.mws
> T1:=(Q-s)/lambda;

Tl:= Q-s
1
> HoldingC:=normal(h*int(Q-s-lambda*t,t=O .. T1));

HoldingC := ~ h (-~ + s)2

> T:=Q/lambda;

> T2:=normal(T-T1);
s
T2:= ;:
> BackorderC:=pi[ 1] * int (lambda*t, t=O .. T2) +pi[ 2] * s;
1 1f 1 s2
BackorderC := 2: -1- + 1f2 s
> EC:=K+HoldingC+BackorderC;
EC K 1 h ( - Q + S )2 1 1f 1 s2
:= +2: 1 +2:-1-+ 1f2S
> EL:=T;
Q
EL:=-
1
> G:=expand(EC/EL);
1K 1 1 h s2 1 1f 1 s2 1 1f2 S
G:= Q+2:Qh-hs+2:Q+2:Q+Q
For ease of exposition, we consider the simpler case of this problem with 1f2 =
o.
> G:=subs(pU ~ =O,G);
1K 1 1 h s2 1 1f 1 s2
G:=Q+2: Qh - hs +2:Q+2:Q
The necessary conditions are found by differentiating the objective function.
 8.4 Detenninistic Inventory Models 341

> GQ:=diff(G,Q); Gs:=diff(G,s);

). K I I h s2 I 'rt s2
GQ:=- Q2 +2 h -2 Q2 - 2 7

hs ll'IS
Gs:=-h+ Q + Q
Here we check to see if the objective function is strictly convex in the decision
variables (Q, s). First we load the linalg () package and compute the Hessian
matrixofG.
> with (linalg) :
Warning, new definition for norm

Warning, new definition for trace

> H:=hessian(G,[Q,s]);

-~~-~:]
h 1r I
-+-
Q Q
Maple gives two conditions in order for G to be positive definite.
> definite(H,positive_def);

). K h s2 1r 1 s2
- 2 - - - - - - <Oand
Q3 Q3 Q3
h s2 1r 1 s2 ) ( h
(2 ). K ll' 1 ) ( hs ll' 1 s )2 0
- Q3+Q3+Qr Q+"Q+-Q2-Q2 <
A further simplification shows that the second term is also negative, indicating
that G is positive definite. 7
> Condition2 : =-(2*lambda*K/(Q A3)+h*sA2/(Q A3)
+pi[ 1] * sA2/ (Q A3) ) * (h/Q+pi[ 1] /Q)
+ (-h* s/ (QA2) -pi[ 1] * s/ (QA2)) A2;
.. ). K h s2 1f 1 s2 h 1f 1 hs 1f 1 S 2
Cond,t,on2:= -(2 Q3 + Q3 + Qr)(Q +"Q) + (- Q2 - Q2)
> sirnplify(Condition2);
-2 (h + ll' I»). K
Q4
Solution of the necessary conditions gives the values of Q and s.

7We should note that we have not reentered the second tenD manually but simply copied it to the
clipboard using the mouse and then pasted it to the next line.
342 8. Inventory Models

> sol : =solve({ GQ,Gs},{ Q,s} );

sol := (s = RootOf«h 1C I + 1C 12) _Z2 - 21 K h),

Q = RootOf«h 1C I + 1C 12) Z2 - 21 K h)(h + 1C I) }
h
However, this result is not very readable, so we tum it into a more familiar form
using the all va 1 ue s () function.
> OptSol:=allvalues(sol);

Op SoI '={Q= J2%2(h+1CI) = J2%2}

t . %1 h ' s %1'
(Q = J2%2(h + 1C1) = _ J20/02}
%1 h ' s %1
%1 :=h1C1 +1C1 2
0/02:= ,,/1C1 (h+11"I)lKh
> OptSol[ 1) ;

Q = J2,,/11" I (h + 1C J) 1 K h (h + 1C I), S = J2,,/1C I (h + 1C I) 1 K h}

{
(h1C1 +1C12)h h1C1 +1C1 2
> assign (OptSol[ 1) ) ;
> simplify(Q,power,syrnbolic);

J2 $+1iI../I,JK
Fi./h
> simplify(s,power,syrnbolic);

J2../I,JK./h
.;;rJ $+1iI
As the backorder cost 1C I approaches infinity, the solution should converge to
the one we found for the basic EOQ model, which is indeed the case.
> limit (Q, pi[ 1) =infinity) ;

~J2
h
> limit (s,pi[ 1) =infinity);
o
Note that since lim s = 0 since for very large backorder costs, it would not
lrl~OO

be desirable to have any backorders.

As a simple example consider the case with a demand of 1 = 2000 units per
year, order cost of K = $25 per order, holding cost of h = $10 per unit per
year and a backorder cost of 1C I = $30 per unit per year. Using the foregoing
 8.4 Detenninistic Inventory Models 343

solution, Maple computes the optimal values of the decision variables Q and s
as approximately 115 and 29, respectively. The minimum cost is computed as
$866.02.
> lambda:=2000; K:=25; pi[ 1] :=30; h:=10;
A := 2000
K :=25
1r 1 := 30
h:= 10
> Q:=evalf(Q); s:=evalf(s);
Q := 115.4700538
s := 28.86751346
> G;
866.0254037

8.4.3 Analysis ofImplied Backorder Costs

In practical problems it is usually very difficult to accurately estimate the value
of the backorder cost. In this section we redefine the simpler problem with 1r2 =
o presented in Section 8.4.2. The new formulation is a nonlinear programming
problem with a service measure constraint defined as the average time a demand
must wait until it is filled. Such a constraint may be easier to specify than the
backorder cost 1r I. With this different formulation of our new problem, we obtain
results that give implicit estimates of the backorder cost 1r 1•
Since management would normally be averse to increased backorder delays,
we introduce a service measure w as the average time a demand must wait until it
is filled. With this definition we refer to Figure 8.2 and find

w = total waiting time per cycle =..!..

demand per cycle
r At dt = ~.
Q Jo
T2
2A.Q
Now suppose that management specifies an upper limit W > 0 on w. Our problem
then reduces to
KA h(Q-s)2
min
Q"+ 2Q
s2
s.t. 2A.Q :5 W.

Note that in this formulation the backorder cost 1r 1 does not appear.
It is clear that W must be positive; otherwise the problem reduces to the basic
EOQ model with optimal s = 0 and optimal Q = J2Kl/h.
344 8. Inventory Models

The Lagrangian of this nonlinear program is written as

£=K). h(Q-s)2 (~-).W).

Q + 2Q +fJ 2Q

Since the objective function and the feasible set is convex, the Kuhn-Tucker (KT)
conditions are necessary and sufficient for optimality.
The KT conditions are o.c;os = 0, o.c;oQ = 0, o£/ofJ ~ 0, fJ· (o£/ofJ) =
and fJ ~ 0. The first three of these are easily obtained by Maple as follows.
°
> restart: # ImpliedBO.mws
> L:=K*lambda/Q+(1/2)*h* (Q-s)A2/Q
+mu* ((1/2)* sA2/Q-lambda*W);

L .= K)' + ~ h (Q - s)2 + fJ (~s2 -). W)

. Q 2 Q 2Q
> Ls:=diff(L,s);

Ls .- h(Q-s) fJS
.- Q +0-
> LQ:=diff(L,Q);
LQ .__ K ). + h (Q - s) _ ~ h (Q - s)2 _ ~ fJ s2
.- Q2 Q 2 Q2 2 Q2
> Lmu:=diff(L,mu);
1 s2
Lmu := 2" Q -). W
Solving for the first two equations we find the optimal values of the decision
variables (s, Q). Comparing this solution with the one we found in Section 8.4.2,
we see that they are the same with 1C I = fJ and that s / Q = h / (h + fJ).
> sol :=solve ({ Ls, LQ) ,{ s, Q) );

sol:= (s = RootOf«h fJ + fJ2) _Z2 - 2 K ). h),

Q = RootOf(h fJ + fJ2) Z2 - 2 K ). h)(h + fJ)}
h
> SolOpt:=allvalues(sol);

J2%1 (h + fJ) J2%1

SoIOpt:={Q= (h 2)h ,s=h 2}'
fJ+fJ fJ+fJ
{Q= J2%I(h+ fJ ),s= J2%I}
(h fJ + fJ2) h h fJ + fJ2
%1 :=JfJ(h+fJ)Klh
> SolOpt[ 1) ;
 8.4 Deterministic Inventory Models 345

{Q = ,J2 J p. (h + p.) K 1 h (h + p.) s = ,J2 J p. (h + p.) K 1 h }

(h p. + p.2) h ' h p. + p.2
> assign (SolOpt[ 1) ) ;
> s/Q;
h
h+p.
> eq: =sirnplify (subs (SolOpt[ 1) ,sA2/(2*Q)=larnbda*W));

e := ~ K ). h2 ,J2 = ). W
q 2 (h+ p.)Jp. (h + p.)Klh
We note that if p. = 0, no solution can exist for this problem since the first KT
condition aClas = 0 implies s = Q, thus reducing the second KT condition to
- K).I ~ = 0, which is impossible. Hence p. > 0 and the constraint s2 1(2). Q) ~
W must be satisfied as an equality.
Substituting the optimal values of sand Q into s2/(2).Q) = W gives a nonlin-
ear equation (a fourth-order polynomial) in p.. This equation has a unique solution
since the term in its left-hand side is a strictly decreasing function of p. that re-
duces from infinity to zero.
> LHSeq:=lhs(eq);

LHSe ._ ~ K ). h2 ,J2
q .- 2 (h + p.) J p. (h + p.) K ). h
> norrnal(diff(LHSeq,rnu));
1 (4 p. + h),J2 hJ ).2 K2
-4 (h + p.)(p. (h + p.) K ). h)(3/2)
> assurne(h>O, K>O, larnbda>O);
> lirnit(LHSeq,rnu=O,right);
00

> lirnit(LHSeq,rnu=infinity);
o

In summary, if the specification of the backorder cost 11: I is difficult, then an

alternate approach can be used where an upper limit W on w is specified. The new
problem has a solution with exactly the same structure as the original backorder
model. Given the value of W, the implied value p. of the backorder cost 11: I is
computed after finding the unique solution of a fourth-order polynomial.
A more general version of this approach is presented in <;etinkaya and Parlar
[42] where implied backorder costs are computed for the two types of backorder
costs 11: I and 11: 2.
346 8. Inventory Models

8.4.4 Quantify Discounts

In the models presented so far we made the assumption that the unit purchase
cost e is a constant regardless of the number of units procured. This meant that
the total procurement cost per cycle was eQ. Since the cycle length was Q/..l, the
average procurement cost per unit time was found as eQ/(Q/..l) = c..t which is
independent of the order quantity decision variable Q.
In many real-world applications the unit cost of an item is not always inde-
pendent of the quantity purchased. In some cases, larger purchase quantities of
a product may lead to reductions in the unit cost. A common example of such a
quantity discount is the case of the "baker's dozen" where some bakeries give their
customers 13 units of an item (such as bagels) for the price of 12. Other examples
of lower unit prices for bulk purchases include computer diskettes purchased in a
box of 10, office stationary purchased in bulk and soft drink cans purchased in a
case of24.
In the general quantity discount model we will analyze in this section, we
will define bl, In, ... , bn-I as the order quantity points (break points) where
a price change occurs. The order-quantity-dependent unit purchase price e(Q) is
described as

PI, if bo = 0 ::; Q < bl

e(Q) ={ Pk, < bk for k = 2, ... , n - 1
~f bk-I ::; Q (8.2)
Pn, Ifbn-I::; Q < bn = 00.

Since larger order quantities are associated with lower prices, we have PI > P2 >
... > Pn. Recalling the definition of the holding cost h = ie, we see that since
c is a function of Q, the order-quantity-dependent order cost would be h(Q) =
ic(Q). With this definition, and retaining every other assumption we made when
we introduced the basic EOQ model, the new objective function G (Q) is obtained
as
)'K 1
G(Q) = Q + "2h(Q)Q + ).c(Q). (8.3)

The objective is to compute the optimal Q that will minimize the average cost
function G(Q) in (8.3). Since the holding and procurement costs are both discon-
tinuous functions of Q, the objective function G(Q) is also discontinuous - see
Figure 8.3 - and its minimum value has to be obtained using a procedure that is
more complicated than the one we employed for the basic EOQ model.
Essentially, the procedure for computing the optimal order quantity involves
three steps.

I. For each region bk-I ::; Q < bk, k = I, . .. , n defined in (8.2), compute an
order quantity Q using the EOQ formula with the unit cost associated for
that category.

2. It is possible that some of the Q's found in Step 1 may be too small to
qualify for the related discount price; i.e., they may be infeasible. In that
 8.4 Detenninistic Inventory Models 347

case adjust the order quantity upward to the nearest order quantity point at
which the unit price will be reduced.
3. For each of the order quantities found in Steps 1 and 2, compute the average
cost G(Q) in (8.3) using the unit price that is appropriate for that quantity.
The order quantity yielding the lowest cost is the optimal one.

When there is a large number of break points, this procedure outlined above
may be cumbersome to implement manually. However, using Maple's graphical
and numerical capabilities, the optimal solution to the quantity discount problem
can be found very easily.
As an example, consider a problem with an annual demand of A. = 1000 units,
inventory carrying charge of i = 0.10 and order cost of K = $100 per order.
> restart: *
QuanDisc.mws
> lambda:=1000; i:=0.10; K:=100;
A. := 1000
i:= .10
K:= 100
There are two break points at hI = 100 and In = 300 with prices PI = 50,
Pl = 49 and 1'3 = 48.50. Thus, the procurement cost function is
50, if bo = 0 ::: Q < 100
c(Q) ={ 49, if 100 ::: Q < 300
48.50, if 300 ::: Q < 00.
> b[l]:=100; b[2]:=300;
p[ 1] :=50; p[ 2] :=49; p[ 3] :=48.50;
hI := 100
h2:= 300
PI := 50
Pl:= 49
P3 :=48.50
We enter this information in Maple using the piecewise () function and
form the objective function G(Q).
> c: =Q->piecewise (Q<b[ 1] ,p[ 1] ,Q<b[ 2] ,p[ 2], p[ 3] ) ;
c := Q -+ piecewise(Q < hI, PI, Q < In, Pl, 1'3)
> C (Q) ;

150
49
Q < 100
Q < 300
48.50 otherwise
> h : =Q-> i * c (Q) ;
348 8. Inventory Models

h:=
ic(Q) Q~

> G:=Q->K*lambda/Q+(l/2)*h(Q)*Q+lambda*c(Q);
K)' I
G:= Q ~ Q + 2h(Q) Q+).c(Q)
A visual inspection of the plot of G(Q) in Figure 8.3 reveals that the optimal
solution is at the last break point h]. = 300.
> iplot(G(Q),Q=50 .. 500,discont=true);

51500

51000

50000

FIGURE 8.3. The cost function G(Q) under quantity discounts.

Maple's ability to compute the derivatives of piecewise functions - such as

G(Q) - pennits us to simplify the first two steps of the procedw-e given on page
346. We differentiate G(Q), equate the result to zero and solve to find the only
feasible EOQ in the interval [bl, h].) = [100,300) as q = 202.03.
> q:=solve(diff(G(Q),Q),Q);
q := -200.,202.0305089
> q[ 2] ;

202.0305089
At this EOQ point, the cost function assumes the value G(202.03) = $49989.94.
> G (ct 2] ) ;

49989.94949
However, the order quantity at the next break point h]. = 300 gives a lower
cost as G(300) = $49560.83, indicating that the optimal order quantity must be
= =
Q In 300 units.
> G (b[ 2] ) ;

49560.83333
 8.5 Probabilistic Inventory Models 349

8.5 Probabilistic Inventory Models

A common assumption in all the models presented so fur was the deterministic
nature of every relevant factor such as demand. cost data and supplier availability.
In reality, many of these factors are not deterministic but random. For example,
due to the wcertainty present in consumer preferences the actual number of units
of a product a retailer hopes to sell in the coming month is usually not known in
advance. Future levels of interest rates, which playa crucial role in determining of
the holding cost h, are subject to random fluctuations. Due to unexpected failures
in production processes or wplanned work stoppages caused by labor unrest, a
supplier may sometimes be wable to deliver the product ordered by the retailer.
In this section we will develop several inventory models that will involve at
least one random factor (such as demand or supplier availability). We will assume
that although the actual outcome of the random variable of interest cannot be
predicted in advance, we have a probability distribution representing the possible
outcomes.
We will start by describing a continuous-review inventory model with random
demand where backorders are allowed. First, we will present an heuristic treat-
ment of the approximate fonnulation of this model. We will then briefly discuss
an exact version of the model for the case of Poisson demands and constant lead-
time. Periodic-review models with random demand (including the single-period
newsvendor model) will also be introduced. We will also discuss some models
where yield is random and where the supplier status is a random process.

8.5.1 The Continuous-Review Model: Approximate Formulation

Consider an inventory system that is controlled by selecting an order quantity (or
lot size) Q and a reorder point r . This policy calls for ordering Q units when the
on-hand inventory drops to the level r . As in the basic EOQ model, we make some
assumptions that will facilitate the development of the model.

I. Since the system is continuous review, the on-hand inventory level is known
at all times.

2. Demand is random but stationary with a constant expected value of A. units

per year.

3. There is never more than one order outstanding. This means that when the
reorder point r is reached. there are no outstanding orders so that the inven-
tory position (IP)s is equal to the inventory level (IL), implying that order
quantity Q will always be the same.

81nventory position is defined as the amount on hand plus the amOlDlt on order minus the number
of backorders. Inventory level is the 8I11OIDIt on hand minus backorders. Some authors (e.g., Hadley
and Whitin [SI, p. 163]) use the tenn "net inventory" instead of "inventory level."
350 8. Inventory Models

4. The reorder point r is positive.

5. There is a constant positive leadtime of L time units. 9 The random demand

during leadtime (DDLT) is denoted by X with probability density f(x) and
the expected value p = E(X) = )'L.

6. There are three types of costs:

(a) Purchase cost is c dollars per unit and order (setup) cost is K dollars
per order.
(b) Holding cost is h dollars per unit per year.
(c) There is a shortage cost of tr (= 1C I) per unit of unsatisfied demand.
The time-dependent part of the shortage cost is tr2 = O.

In deterministic inventory models such as the basic EOQ model, the inventory
level at the time of an order arrival can be determined with certainty. In stochastic
inventory models this is not possible since the demand during leadtime is also a
random quantity. The expected value of the inventory level at the time of an order
arrival is defined as the safety stock and it is denoted by s. Since the expected
demand during leadtime is p, we can write s = r - p.
In this model, the regenerative cycles are of random length and they commence
when the inventory position is raised to Q + r. Figure 8.4 depicts a sample real-
ization of the expected (average) behavior of the inventory position and inventory
level processes. The sample paths of these processes are drawn as lines since we
had asswne that the demand is stationary with a constant expected value of). units
per year. By using the renewal reward theorem we can construct the average cost
per year as follows.

Cycle Length
Since the IP fluctuates between r and Q + r, and Q units are used at an average
rate of). units per year, the expected cycle length is E (length) = Q / )..

Cycle Cost
We examine the cycle cost in three parts.

• Order Cost: Since each cycle starts with an order, the order cost in a cycle
is K dollars.

• Holding Cost: To compute this cost, we make an important simplifYing

approximation and asswne that backorders occur only a small fraction of
time. This has the implication that the on-hand inventory can now be ap-
proximated by the inventory level IL. (As Hadley and Whitin [81] indicate,

9 As we indicated previously, in this chapter the unit time is normally taken as one year.
 8.5 Probabilistic Inventory Models 351

Cyde - + - Cycle-+- Cycle - +-

Inventory position
Q+r
\/
\
\
1' .....
I ..........
I
\ I
\ I
\ I
I
I

-L- -L- -L_

Timet

FIGURE 8.4. A sample realization of the expected (average) behavior ofinventory position
and inventory level processes in the continuous-review model with backorders.

this is a valid approximation when the backorder cost 1C is large relative

to the holding cost h, which is usually the case in practical applications.)
Since the average demand rate was assumed to be a constant A, the expected
inventory level varies linearly between s and Q + s and the average inven-
!
tory during each cycle is approximately [(Q + s) + s] = Q + s where !
s = r - AL. Recalling that each cycle lasts Q / A years, the expected holding
cost per cycle is E(holding cost) = h(!Q + r - AL)Q/A.
• Backorder Penalty Cost: Defining ,,(X, r) as the random number ofbacko-
rders per cycle, we have
0, if x < r
,,(x, r) = { x _ r, if x ~ r.
Thus, the expected number of backorders in a cycle is E(backorder cost) =
1C ir(x -r)f(x)dx.
Combining the three types of costs described and using the renewal reward
theorem, the expected average cost G( Q, r) per year is
K +h(!Q+r - p)Q/A +1C ir(x -r)f(x)dx
G(Q,r) = Q/A

= AK 1CA
g+h(!Q+r-p)+Q 1r
00
(x-r)f(x)dx.
352 8. Inventory Models

The third tenn B(Q, r) = (1r AI Q) ir(x -r)f(x) dx in the objective function
is the expected backorder cost per year. As we showed in Chapter 5, Nonlinear
Programming, despite the claim made by Hadley and Whitin [81, p. 221, problem
4-6] this function is not, in general, jointly convex in Q and r. But, as we also in-
dicated, if t; is a number such that f(x) is nonincreasing for x ~ t;, then B(Q, r)
is convex in the region {(Q, r) : 0 < Q < 00 and t; ::; r < oo}. Thus, if r is
restricted to be suitably large, then B(Q, r) -and the objective function G(Q, r)
- is convex within such a region. For example, if f(x) is nonnal, then B(Q, r)
is convex when r exceeds the expected demand during lead time. When f(x)
is a strictly decreasing density such as the exponential, then B(Q,r) is convex
everywhere.
We now use Maple to find the necessary conditions minimizing G(Q, r) and
present an example using an exponential demand density f(x) = Oe-J:x, x ~ o.
> restart: # Qr.mws
The objective function G (Q, r) and its derivatives G Q and G r are entered and
evaluated to give the necessary conditions.
> G:=lambda*K/Q+h* (Q/2+r-mu)
+(pi*lambda/Q)*int«x-r)*f(x),x=r .. infinity
) ;

G :=
AK I
Q + h (2 Q + r - Jl) +
1r A 1r
00
(x - r) f(x) dx
Q

> GQ:=diff(G,Q);

AK
GQ := --Q-2
I
+ 2h -
1r )'1 00
(x - r) f(x) dx
_..::..;r:......-_-:2
,....---
Q
> Gr:=diff(G,r);

Gr:= h +
1r A1 00
- f(x)dx
_....::...:.r_ _ __
Q
An attempt to solve for the unknowns Q and r directly from the necessary
conditions fails. This is understandable since it is impossible to find a c1osed-fonn
solution for the unknowns from expressions involving integrals whose lower limit
includes r. Thus the necessary conditions must be solved numerically once the
density f(x) is specified.
> solve({ GQ,Gr} ,{ Q,r} );
Error, (in solve) cannot solve expressions with,
int(-(-x+r)*f(x),x =r .. infinity), for, r

> Qsol:=solve(GQ,Q);
 8.5 Probabilistic Inventory Models 353

v'2 h A. (K + [00 f(x) x dx It _[00 f(x)r dx It)

Qsol:=--~----------------------------
h

v'2 h A. (K + [00 f(x)x dx It _[00 f(x)r dx It)

h
Although we cannot solve the necessary conditions for the Q and r values
explicitly, we can at least express Q in tenns ofr (denoted in the Maple worksheet
by Qr) and r in tenns of Q (denoted by rQ) easily as follows:
> Qr: =Qsol[ 1] ;

Qr:~ ../i hl(K + 1 00

f(x)xdxx - [f(X)rdU)

h
> readlib(isolate);
p~(etpr, x, n) ... end
> rQ:=isolate(Gr,int(-f(x),x=r .. infinity»:
00 hQ
rQ:= [
r
- f(x)dx =--
ltA.
Following is an example illustrating the solution of the approximate (Q, r)
model with an exponential density for the demand during lead time.
> K:=100: h:=.15; pi:=l; lambda:=10000;
delta:=1/1000; mu:=l/delta;
fIx) :=delta*exp(-delta*x);
K:= 100
h:= .15
It := I
A. := 10000
I
t5:= 1000
p. := 1000
I
f(x) := 1000 e(-I/IOOOx)

The cost function G and its partials now assume specific fonns.
> G;
I e(-I/IOOOr)
1000000 Q + .07500000000 Q + .15r - 150.00 + 10000000 Q
354 8. Inventory Models

> GQi
1 e(-I/IOOOr)
-1000000 Q2 + .07500000000 - 10000000 Q2
> Gri
e(-I/IOOOr)
.15 -10000 Q

Since the data and the density f(x) are specified. Maple easily finds the opti-
mal Q and r values as 4785 and 2634, respectively. The minimum value of the
objective fimction is obtained as $962.99.
> sol:=solve({ GQ,Gr},{ Q,r} ) i

so/:= {Q = -2785.938897, r = 3175.120135 - 3141.592654 I},

{r = 2634.022855, Q = 4785.938897}
> sol[ 2] i GOpt : =eval f (subs (sol[ 2] , G) ) i
{r = 2634.022855, Q = 4785.938897}
GOpt := 962.9942630

8.5.2 The Continuous-Review Model: Exact Formulation

In the exact formulation of the (Q, r) model it is assumed that unit demands are
generated by a Poisson process with rate l. Thus, all variables of interest such Q,
r and the inventory levels are discrete integers. There is a constant lead-time of L
years and, of course, the shortages are backordered. In this model, the two types
of backorder costs 1r I and 1r2 assume positive values.
Development of the objective fimction of this model requires the stationary
distribution of the inventory position. This is achieved using tools from stochastic
processes (more specifically, continuous-time Markov chains); see Hadley and
Whitin [81, Chapter 4]. The average cost G(Q, r) is the sum of expected order
cost lK/Q, expected holding cost hS(Q,r) and two types of backorder costs
[1rlb(Q,r) + 1r2B(Q,r)], i.e.,
lK
G(Q,r) = Q + hS(Q,r) + 1Clb(Q,r) + 1C2B(Q,r)
where
1
S(Q,r) = 2(Q+l)+r-lL+B(Q,r)
1
B(Q,r) = QlP(r) - ft(r + Q)]
l
b(Q,r) = Q[a(r) - a(r + Q)]
 8.5 Probabilistic Inventory Models 355

1 2- - 1 -
P(v) = 2(AL) F(v - 1; AL) - ALvF(v; AL) + 2v(v + I)F(v + 1; AL)
a(v) = ALF(v; AL) - vF(v + 1; AL).
In these expressions,
00 u-I
F(u; AL) = LfV; AL) = 1 - LfV; AL)
j=u j=O

is the complementary cumulative distribution of the Poisson process with density

f(x; U) = e-J.L(AL)X, x = 0,1, ...

x!
Optimization of G(Q, r) over the nonnegative integers appears to be a very
complicated task. However, as usual, Maple's ability to plot graphs and perfonn
symbolic and numeric computations simplifies this problem considerably as we
demonstrate below.
> restart: # QrExact.mws
The Poisson density f(x; AL) and its complementary cumulative distribution
F(x; AL) are entered using the unapply () ftmction.
> f:=unapply(exp(-lambda*L)* (lambda*L)Ax/x!,X)i
e(-H) (A L)X
f := x -+ ------'--
x!
> FBar:=unapply(l-Sum(f(j),j=O .• (u-l»,U)i

FBar:= u -+ 1 - C
L -I e(-H) (A
.f
LY)
.=0 }.

The expressions for a(v), P(v), b(Q, r), B(Q, r) and S(Q, r) are also defined
using the unapply () ftmction so that they may be evaluated when we compute
the G (Q, r) ftmction.
> alpha:=unapply(' lambda*L*FBar (v)-v* FBar (v+l)' ,v) i

a := v -+ A L FBar(v) - v FBar(v + I)
> beta:=unapply(' ((lambda*L)A2/2)*FBar(v-l)
-lambda* L*v* FBar (v)
+(1/2)*v* (v+1)*FBar(v+1)' ,v) i
1 1
P := v -+ 2 A2 L2 FBar(v - 1) - A L v FBar(v) + 2 v (v + 1) FBar(v + 1)
> b:=unapply(' (lambda/Q)* (alpha (r)-alpha (Q+r) )' ,
Q,r)i

._ (Q)
b .- A(a(r) - a(Q + r»
,r -+ Q
> B:=unapply(' (l/Q)* (beta(r)-beta(Q+r»' ,Q,r)i
356 8. Inventory Models

B := (Q, r) -+ p(r) - P(Q +r)

Q
> S:=unapply(' (1/2)* (Q+1)+r-lambda*L+B(Q,r)' ,Q,r);
1 1
S:= (Q, r) -+ 2" Q+ 2" +r -)'L +B(Q, r)
Next we define the objective function G(Q, r) in tenns of the expressions in-
troduced earlier and enter the data values for )., L, K, h, 7C 1 and 7C2. Although the
variables Q and r are discrete, we plot the continuous contour curves of G in or-
der to obtain some insights for its convexity. We see in Figure 8.5 that G appears
to be a functionjointiy convex in (Q, r).
> G: =unapply (' (lambda/Q) * K+h* S (Q, r)
+pi[ l]*b(Q,r)+pi[ 2]*B(Q,r)' ,Q,r);
)'K
G:= (Q, r) -+ Q +hS(Q, r) +7CI b(Q, r) +7C2B(Q, r)

> lambda:=400; L:=0.25; K:=0.16; h:=2; pi[ 1] :=0.10;

pi[ 2] : =0 . 30;
). := 400
L:= .25
K := .16
h =
: 2
7CI :=.10
7C2 := .30
> with(plots):
> contourplot(G(Q,r),Q=5 .. 35,r=5 .. 120,color=black);
Based on the information extracted from the graph of the contour curves, the
optimal solution appears to be in the region [15,35] x [80,120]. Ideally one
should implement a discrete optimization method to locate the optimal solution
for this problem. But in this case we will use an exhaustive search to compute the
optimal value that minimizes G(Q, r). In this way, we will have an opportunity
to introduce the Maple time () function that sets the current clock time and that
can be used to compute the CPU time of an operation.
> settime:=time():
> GL:=[ seq(seq([ Q,r,evalf(G(Q,r))] ,
Q=15 .. 35) ,r=80 .. 120)] :
> nops(GL);
861
On a Pentium-based desktop computer with a clock speed of 200MHz, Maple
took slightly over two minutes of CPU time to evaluate the objective function at
 8.5 Probabilistic Inventory Models 357

120

100

80

reo
40

20

o 5 10 15 a 20 25 30

FIGURE 8.5. Contour curves of the G(Q, r) function in the exact formulation of the con-
tinuous-review model. Note that although G (Q, r) is a discrete function of discrete vari-
ables Q and r, the graph is drawn using continuous variables.

all 861 integer points in the [15,35] x [80,120] region. The optimal solution is
found as Q = 18, r = 96 and G = $32.89.

> cpu_time: = (time ()-settime)*seconds;

cpu_lime := 143.474secomU
The list GL consists of lists of triples with values of [Q, r, G). The next set of
statements extracts the triple with the smallest third element G.

> for i from 1 to nops(GL) do if

GU i) [ 3) =min (seq (GU i) [ 3) , i=1. . nops (GL) ) )
then print (GL[ i) fi od;
[18, 96, 32.8953311)

> evalf(G(19,96»;
32.9419609
We should note in closing that the same problem was also solved by Hadley
and Whitin [81, Section 4-10), who obtained a slightly different-and incorrect-
"optimal" solution as (go, rO, G(go, rO» = (19,96, $31.75). Although this is
an incorrect result IO due to the flatness of the objective function, using Hadley
and Whitin's solution (19,96) would give G(l9, 96) = $32.94 - an increase of
only 4 cents, or 0.15%.

lOA Quick BASIC program written to check the results found that Maple's solution was correct.
358 8. Inventory Models

8.5.3 One-Period (Newsvendor) Model with Random Demand

In this section we treat the one-period inventory model in which only a single in-
ventory replenishment decision is made in anticipation of the random demand for
the product. This is the simplest probabilistic inventory model but it is important
for two reasons. First, many real-life inventory decision problems (such as the
order decisions for newspapers, Christmas trees and bakery products) can be rep-
resented using the one-period model. Second, insights gained from the analysis
of the one-period model provide a useful starting point for the analysis of more
complicated - and more realistic - multiperiod dynamic models.
To motivate the discussion let us assume that a newsvendor selling the local
newspaper Hamilton Spectator in Hamilton, Ontario, must choose the order quan-
tity Q of newspapers he will order from the publisher. Each unit he orders will
cost him c cents per unit and he sells it for s cents per unit with c < s. Ifhe overes-
timates the demand and ends up with a surplus at the end of the day, he can return
the unsold units for a "salvage" value ofv cents per unit where v < c. If, on the
other hand, he underestimates the demand and finds that more customers wanted
to purchase the newspapers than he had available, then he would incur a penalty
cost of p cents per unit. Such a penalty cost is usually difficult to measure but it
may correspond to a cost of lost "goodwill" or the marginal cost of satisfying the
demand through, say, an expedited order.
If the daily demand is represented by a random variable X with density /(x),
and if we define n as the random profit per period, then the realized value 'If of
profit can be written as

_ { sx + v(Q - x) - cQ ifx ~ Q (surplus)

'If - sQ - p(x - Q) - cQ ifx> Q (shortage).

Thus, the expected profit E(n) = P(Q) is obtained as

P(Q) = Io Q
[sx+V(Q-X)-c Q]/(X)dX

+ koo[sQ - p(x - Q) -cQ]dx

= (s - v)p- (c - v)Q - (s +p - v) 10
00
(x - Q)/(x)dx

where p = E(X) is the mean demand.

Using Maple (which in turn uses Leibniz's rule of differentiation under the
integral sign), we find the first and second derivatives of P(Q) as follows:
> restart: # Newsvendor.mws
> P:=(s-v)*mu-(c-v)*Q
-(s+p-v)*int((x-Q)*f(x),x=Q •. infinity);

P := (s - v) p - (c - v) Q - (s +p - v) 10
00
(x - Q) f(x) dx
 8.5 Probabilistic Inventory Models 359

Since PQQ < 0, the expected profit function P(Q) is a strictly concave. Thus
the solution of the necessary condition PQ = 0 would give the unique maximizing
value.
> PQ:=diff(P,Q)i

PQ := -c + v - (s +p - v) 10
00
- f(x) dx

> PQQ:=diff(P,Q$2)i
PQQ:= -(s +p - v)f(Q)
In order to isolate the term containing the integral term, we use the iso-
late () function.
> i:=op(3,op(3,PQ))i

; := 1000
- f(x)dx

> readlib(isolate)i
proe(expr, x, n) ... end
> isolate(PQ,i)i

fOO _ f(x) dx = c- v
1Q -s - p+v
The necessary condition is thus obtained as

- =1
F(Q)
Q
00
f(x)dx = s+p-v
c-v . (8.4)

Since v < c and c < p, the right-hand side of this expression is always less than
unity.
We now solve a numerical example assuming that the demand is a two-stage
Erlang random variable with a mean of 100 units.
> c:=15i s:=55i V:=lOi p:=20i
lambda:=l/lOOi rnu:=l/lambda:
f:=x->2*lambda* (2*lambda)*x*exp(-2*lambda*x)i
c:= 15
s:= 55
v:= 10
p:=20
1
1:= 100

f :=x -+ 412 xe(-2Ax)

> PQi
360 8. Inventory Models

-S + ~ Qe(-I/SOQ) + 6Se(-I/soQ)
10
Solving the necessary condition gives two solutions, but the correct one is the
positive solution that is equal to 210.82. The maximum value of the expected
profit is found as $3,147.86.
> sol:=evalf(solve(PQ,Q));
sol := -48.S4324892, 210.84207S0
> evalf (subs (Q=sol[ 2) , P) ) ;
3147.867908

It is useful to note that the same problem can be restated in terms of expected
cost minimization as follows. Define Co = C- v to be the cost of overage, i.e., the
loss per unit on any remaining units. Similarly, let Cu = (s - c) + p as the cost of
underage, i.e., cost per unit incurred if a demand occurs when the system is out
of stock. Here Cu is the sum of the potential profit lost s - c, and the cost of lost
"goodwill" p . With these definitions, the condition in (8.4) can be rewritten as

F(Q) = C - v = C - v = co. (8.S)

s+p-v [(s-c)+p]+(c-v) cu+co

This suggests that minimizing a relevant expected cost function (of overage
plus underage) is equivalent to maximizing the expected profit function. This fol-
lows since the realized value of cost is

ost {co(Q - x) if x ~ Q (overage)

c = cu(x - Q) if x > Q (underage)

and hence

E(Cost) = G(Q) = Io Q co(Q -x)f(x)dx + 1000

cu(x - Q)f(x)dx.

Differentiating G(Q) and equating the result to zero gives C'(Q) = co-coF(Q)-
cuF(Q) = 0 so that
F(Q) = Co .
Cu +co
This is the same result as in (8.5).
A review by Khouja (107] lists nearly 100 papers that deal with applications
and ramifications of the single-period inventory model. Some of the recent work
on coordination in supply chain management also involves the use of the single-
period model; see, Parlar and Weng [147] and Weng and Parlar (193].
 8.5 Probabilistic Inventory Models 361

8.5.4 Dynamic Inventory Models

In the previous section we considered the single-period inventory model with
stochastic demand and showed that the optimal order quantity can be obtained
in terms of the cumulative distribution of the demand random variable. In this
section we generalize the model and assume that the inventory system is to be
managed over a finite horizon ot: say, N periods. Our exposition here follows
closely that of Bertsekas [31, Section 4.2).
At the start of each period, the inventory level is observed and a decision is
made on the order quantity. It: at the end of a period, there are some unsold items,
the system incurs a cost of h dollars per unit held in inventory. It: on the other
hand, the period ends with a shortage and the shortages are backordered, then a
cost of b dollars per unit is incurred. Clearly, in this model h is the same as the
cost of overage Co, and b is the same as the cost of underage cu introduced in the
previous section where we argued that cost minimization and profit maximization
are equivalent. The objective is to minimize the total expected cost of operating
the inventory system over N periods.
The dynamic inventory problem is formulated naturally as a dynamic program-
ming (DP) problem. Thus, in this section we adapt the state-equation approach
introduced in the Chapter 6, Dynamic Programming, and define state X/c as the
inventory available at the start of period k. Similarly, we define control u/c as the
quantity ordered (and immediately delivered) at the start of period k. The random
demand in each period is defined as W/c with a given density function !k(w/c).
We assume that roo, WI, ••• , WN-I are i.i.d. and that the initial inventory xo is a
known quantity. The inventory level process is represented by
Xk+1 = X/c + u/c - W/C, k = 0,1, . .. , N - I. (8.6)
Although the purchase cost function c( u) can be defined as in (8.1), in this sec-
tion we will assume that this cost is linear, i.e., c(u/c) = CU/c. With this definition,

I
the expected cost that must be minimized is

E{ ~ [cu, + h f.'" (yt - 10,)It(10') dw, + b L~ (ID, - y,)f,(1Dt) dID,]

where y/c = X/c + U/c is the beginning inventory immediately after receiving an
order of U/c ~ 0 units. This functional must be minimized with respect to the state
dynamics given by (8.6).
In order to apply the DP algorithm, we define V/c(x/c) as the minimum expected
cost for periods k, k + 1, ... , N - 1 when starting at period k and with an initial
inventory of X/c units. This gives

min {cu/c
Uk~O
+ L(x/c + U/c) + E[Vk+I(x/C + U/c - W/c)]}

= min {CU/c
Uk~O
+ L(x/c + U/c) +
Jotx) Vk+I(X/c + u/c - w/c)!/c(W/c)dW/c}
362 8. Inventory Models

where the function L (y) is defined as

L(y) = hE[max(O,y - w)] + bE[max(O, w - y»

= h loy (y - w)f(w)dw + b 1,00 (w - y)f(w)dw.

The L (y) function can be interpreted as the expected loss per period due to pos-
sible overages or underages. It can be shown that L (y) is strictly convex since
L"(y) = (h + b)f(y) > O.
We now state an important theorem concerning the structure of the optimal
policy for this problem.
Theorem 14 Assuming c > 0, h ~ 0, b > c and linear order cost c(u) = CU, the
optimal policy p (x) is ofthe form

(X ) _ ( Sic - Xlc ifxlc < Sic

Pic Ic - 0 ifxlc ~ Sic .

Such a policy is said to be "base stock"-ifthe starting inventory level in pe-

riod k is below Sic, a sufficient number of units are ordered to increase it to Sic;
otherwise no action is taken.
Since this is one of the central results in stochastic inventory theory, we will
provide a brief sketch of the proof. For details, we refer the reader to Bertsekas
[30, Section 2.2].
We first define

for each period k = 0, I, ... , N - I.

For the tenninal period N, we have VN(XN) = 0, which is convex in XN. For
period N - 1, the value function takes the fonn

min
YN-I~XN-I
[CYN-l + L(YN-l)] -CXN-l
= min GN-l(YN-l) - CXN-l
YN-I~XN-I

since E[VN(YN-l - WN-l)] = O. This is a single-period problem and it can be

shown that the value minimizing the convex ll function GN-l(YN-l) is unique
and is found by solving
- c+h
F(YN-l) = b + h

where F(y) is the cumulative distribution function of f(w) evaluated at y . We

denote the minimizing value by SN-l.
To detennine the fonn of the policy, we must examine two cases:

II Convexity ofG(y) = cy + L(y) follows from the fact that cy is linear and L(y) is convex.
 8.5 Probabilistic Inventory Models 363

• If the initial inventory XN-I < SN-I, thenYN-1 = XN-I +UN-I = SN-I;
i.e., we must order up to SN-I. This implies that the optimal order quantity
must be UN-I = SN-I-XN-I. Note that in this case GN-I (SN-J) 5 G(y)
for all Y ~ XN-I .
• If the initial inventory XN-I ~ SN-I, then YN-I = XN-I + UN-I = XN-I;
i.e., UN-I = 0, thus no units should ne ordered.

In summary, for period N - 1, the optimal policy J-l N _I (x N -I) is state depen-
dent and it assumes the form
SN-I -XN-I ifxN-1 < SN-I
(
J-lN-1 XN-I
)
={ 0 ifxN-1 ~ SN-I.
The value function now becomes

V ( ) C(SN-I - XN-I) + L(SN-I) ifxN-1 < SN-I

N-I XN-I ={ L(
XN-I
)
ifxN-1 ~ SN-I.
It is easy to see that VN _I (x) is a convex function since L is convex. Thus, we
have shown that given the convexity of VN, the value function VN _I is also con-
vex.
Stepping back one more period, we arrive at N - 2 for which the value function
assumes the form

where GN-2(Y) = cy+ L(y)+ E[VN-I (y - w)). We recall that since VN-I was
convex, so is E[VN-I(Y - w)], implying that GN-2(Y) is also convex. 12 Thus,
there exists an SN-2 that minimizes GN-2(Y) that results in the same policy, i.e.,

(X _ ) = { SN-2 - XN-2 ~fxN_2 < SN-2

J-lN-2 N 2 0 IfxN-2 ~ SN-2.

Hence,

C(SN-2 - XN-2) + L(SN-2) + E[ VN-I (~N-2 - WN-2)],

VN-2(XN-2) ={ IfxN-2 < SN-2
L(XN-2) + E[VN-I(XN-2 - WN-2»), ifxN-2 ~ SN-2

Continuing with induction we prove that for all k = N - 1, N - 2, ... , I, 0, the

order-up-to (or base-stock) policy is optimal for the stochastic dynamic inventory
problem with the properties stated above.
We should note that although the structure of the optimal policy is available
from Theorem 14, we still do not know how to compute the optimal base-stock

12It is easy to see that if, say, g(y) is convex, then t/J(y) = £[g(y - w)) is also convex since
t/J"(y)=.roo g"(y - w)dF(w) ~ o.
364 8. Inventory Models

levels .so,
SI, .. . ,SN-I for each period. The exact computation of these values
is usually very difficult since they require the analytic solution of the dynamic
programming functional equations for each period. We now describe an example
where Maple is used to compute the exact values of the base-stock levels.
We assume that N = 2 and the demand in each period is uniformly distributed
with density f(w) = 1/15. The relevant cost parameters are c = 4, h = 7 and
b= 10.
We start by defining the input data.
> restart : *
BaseStock . mws
> c:=4; h : =7; b:=10;
Low:=O; Hi:=15; f:=w->1/(Hi-Low);
c :=4
h :=7
b:= 10
Low:=O
Hi:= 15
I
f:=w-+ - - -
Hi-Low
=
The value function for the terminal period N 2 is Y2(X2) = O. The function
L (y) is defined in the usual manner for each period.
> EV[ 2] : =unapply(O,x);
EY2 :=0
> L:=unapply(h*int((y-w)*f(x),w=O •• y)
+b*int((w-y)*f(x),w=y .• 10),y);

L :=y-+ -110;+ I~ -~y(lO-Y)

For period k = I, we compute symbolically the GI(Y) function and find the
optimal base-stock level as SI = 40/17 ~ 2.35. Using this information, the
value function YI (XI) for the current period is defined with the piecewise ( )
function.
> G( 1] : =unapply(c*y+L(y)+EV[ 2] (y),y);

GI :=Y-+ 4y- 110;+ I~ -~y(lO-Y)

> ~ ~ :=solve (diff (G( ~ (y),y),y); evalf(%);
40
SI :=T7
2.352941176
> V[ 1] : =unapply(piecewise(x<S[ 1] ,c* (S[ 1] -x)+L(S[ 1]),
simplify(L(x))),x);
 8.5 Probabilistic Inventory Models 365

. . 40 1540 17 100 20
VI := X ~ Plecewlse(X < - - - - 4 x - X2
17' 51 ' 30
+ - 3 - -3 X)

> V[ 1] (x);

1540 -4x 40
X <-
{ 51 17
17 x2 + 100 _ 20 X otherwise
30 3 3
> Vyw:=V[ 1] (y-w);

1540 -4y+4w 40
y-w<-
JF.yw ._ { 51 17
r .- 17 100 20 20
_(y-w)2+_ --y+-w otherwise
30 3 3 3
> op(l,Vyw); op{2,Vyw); op(3,Vyw);

40
y-w<-
17
1540
5l- 4y + 4w
.!2 (y _ wi + 100 _ 20 Y + 20 w
30 3 3 3
The expectation of the value function E[ VI (y - w)] is now evaluated as a swn
of two integrals which results in a cubic equation.
> EV[ 1] :=unapply(int(op(3,Vyw)*f(w),w=O •• y-S[ 1])
+int(op(2,Vyw)*f(w),w=y-S[ 1] •• 10),y);

EV .-
I .-y~ 1350
.1Z... (y _40 3 ~ _E..
~
17) +2( 225 Y +9)(Y
_ 40 2 E.. ..2 (y _
17) + 450 Y
40)
17
32 85760 4 40 4 210 2 40 2
+ 153 Y + 2601 -9 Y (Y-17)-TS Y (17- Y )-TS(Y-17)
> simplify (EV[ 1] (y»;

17 3 4 . .2 376 259840
1350 y - 45 Y - 153 Y + 7803
For period k = 0, the Go(y) function is computed explicitly. Minimizing this
function gives the optimal base-stock level as So = 4.54.
> G[ 0] :=unapply(c*y+L(y)+EV[ 1] (y) ,y);
366 8. Inventory Models

> simplify (G[ 0] (y));

784 43 2 519940 17 3
- 153 Y + 90 Y + 7803 + 1350 Y
> evalf(solve(diff(G[O] (y),y),y)); seO] :=%[1];
4.54562088, -29.83973852
So := 4.54562088

To summarize, the optimal base-stock levels are obtained as So = 4.54 and

S) = 2.35. The result that So > S) should be intuitively clear: When there are
two periods left we can be adventurous and afford to start with a larger initial
inventory. But when there is only one period left, the optimal starting inventory
must be somewhat smililer since the unsold items have no terminal value, i.e.,
V2(X2) = O. As in this example, it can be shown that, in general, So 2: S) 2: .•. 2:
SN-).
When the order cost function c( u) has a fixed cost component K as in (8.1), the
problem becomes much more difficult to solve. However, Scarf [166] has shown
using the concept of K -convexity that in this case the optimal policy is of the
(s, S)-type, i.e.,
( ) _ { Sk - Xk ifxk < Sk
Ilk Xk - 0 ifxk 2: Sk.

For an up-to-date discussion of K -convexity and an enlightening proof of the

optimality of the (s, S) policy, see Bertsekas [30, Section 2.2).
Although the optimal policy assumes this simple form, actual computation of
the parameters (so, So), (S), S), ... , (SN-) , SN-) for each period isa nontrivial
task. However, assuming stationarity of the cost and demand data, one can reduce
the problem to the computation of a single pair of numbers (s, S), which simplifies
the problem considerably. For a discussion of these issues, we refer the reader to
Porteus [150] and references therein.

8.5.5 Diversification Under Yield Randomness

In many problems what is received from the supplier may not be exactly equal
to what is ordered by the inventory manager. This discrepancy may be due to
shipments that perish during transportation or to clerical errors. Such randomness
in the ''yield'' (the actual amount received) also arises in other contexts including
electronic fabrication and assembly, agriculture and chemical processes.
The importance of incorporating the yield randomness into production and in-
ventory models is now well established and in recent years several papers dealing
with this issue have been published. Authors such as Silver [173], Shih [172]
and Lee and Yano [119] have assumed that when a batch is produced by a single
supplier, a random fraction may be defective. Gerchak and Parlar [69] have gener-
alized these models by assuming that yield randomness can be optimally selected
 8.5 Probabilistic Inventory Models 367

jointly with lot sizing decisions. Recently Yano and Lee [197] have published a
survey in which they review these and other random yield models.
In this section we consider the issue of source (supplier) diversification under
yield randomness and examine the conditions for choosing two suppliers with
different yield distributions over a single source. Our exposition parallels the dis-
cussion of the second model presented in Gerchak and Parlar (69).
As in the basic EOQ model, we let h be the holding cost per unit per time and A.
be the known and constant demand per time. When Q units are ordered, a random
number of units are received, denoted by YQ. It is assumed that E(YQ) = #Q
and Var(YQ) = a 2 (f for Q 2: O. In production-related applications, YQ usually
represents the number of usable units in a lot of Q units. However, for the sake of
generality, we assume that # does not have to be less than unity and we let # 2: O.
First let us suppose that two independent sources are available with K as the
cost of setup (ordering) when both sources are used simultaneously. Hence, if
Qi > 0 units are ordered and YQ; are received from source i = I, 2, we have

(~ +~ )2
Cost per cycle = K + QI Q2
2A.

and
~Q +~Q
Length of cycle = 1 2•
A.
If the moments are specified as #i, a~, i = 1,2, then using the renewal reward
theorem, we obtain the average cost G(QI, Q2) = E(Cost per Cycle)/ E(Length
of Cycle) as

G(QI, Q2) = 2KA. + h[Qt(a~ + #~) + 2QI Q2#1#2 + ~(a~ + #~)].

2(#1 QI + #2 Q2)
We demonstrate the minimization of G(QI, Q2) using Maple.
> restart: # Diversify.mws
The objective function G and its partial derivatives G QI , G Q2 are as follows.
> G:=(2*K*lambda+h* (l1] "2* (sigma[ 1] "2
+mu[ 1] "2) +2* (l 1] * (l 2] * rou[ 1] *mu[ 2]
+(l 2] "2* (sigma[ 2] "2+mu[ 2] "2)))
/ (2* (mu[ 1] *(l1] +mu[ 2] *(l 2] ));

G:= (2K A.+

h (Q1 2 (a1 2 + #1 2) + 2 QI Q2 #1 #2 + Q22 (a2 2 + #22)))
/(2#1 Ql + 2#2 Q2)
> GQ1:=normal(diff(G,(l 1] ));
368 8. Inventory Models

I
GQ}:= "2(h QI 2 0"12 PI +2h QI 0"1 2 P2 Q2 +h QI2 PI 3

+2h QI PI 2 P2 Q2 +h Q2 2 PI P2 2 -2PI K A
- PI h Q2 2 0"22) /(PI QI + P2 Q2)2

> GQ2:=norrnal(diff(G,cr 2] ));

122 2
GQ2:=-"2(-hQI PI P2- 2h QIPIP2 Q2
- 2 h Q2 0" 22 PI QI - h Q2 2 0"22 P2 - h Q2 2 P2 3

+ 2 P2 K A + P2 h QI2 0" 12) / (PI QI + P2 Q2)2

Solving the necessary conditions results in a complicated-looking expression
which we simplify using the all values () function.

> Sol: =501 ve ({ GQ1, GQ2} ,{ cr 1] ,cr 2] } ) ;

Sol := {QI = 2K A PI 0" 22 / «P2 4 0" 14 + 0" 22 0" 14 P2 2

O"i pl 0"1 2 + PI 2 0"24 0"1 2 + 0"24 PI 4)RootOf«
+ 2 PI 2
4 4
hp2 0"1 +h0"220"1 4 P2 2 +2hp1 2 0"22 P2 2 0"1 2
+hp1 2 0"2 4 0"1 2 +h0"24 PI 4LZ2 -2K A)h), Q2 =
RootOf«h P2 4 0" 14 + h 0" 22 0" 14 P2 2
+2hp1 2 0"22 P2 2 0"1 2 +hp1 2 0"2 4 0"1 2 +h0"2 4 PI 4LZ2
- 2 K A)0"12 P2}

> Sol:=allvalues(Sol);

Sol:= {Q = -J2Jho/02K AO"I2 P2 Q = K API 0"22%1-J2

2 %1 ' I %2Jh%2KAh
K A PI 0" 22 %1 -J2
},{QI =- %2Jh%2KAh'

Q = _ -J2Jh%2K AO"I2 P2}

2 %1
%1 := hp2 4 0"1 4 +h0"220"1 4 P2 2 +2hp1 2 0"22 P2 2 0"1 2
+hp1 2 0"2 4 0"1 2 +h0"24 PI 4
%2 := P2 4 0" 14 + 0" 22 0" 14 P2 2 + 2 PI 2 0" 22 P2 2 0" 12 + PI 2 0"2 4 0" 12
+ 0"2 4 PI 4
 8.5 Probabilistic Inventory Models 369

The results still look very unappealing, but we note that there is a relationship
between the complicated terms defined by %1 and %2. Maple informs us that these
terms are a constant multiple of each other. \3
> sirnplify(%1/%2);
h
The solution for QI and Q2 are assigned in order to manipulate them.
> Sol[ 1] ;

{Q2 = ,J2Jh%1 K A.0"12 #2 /(h#2 4 0"1 4 +h0"220",4 #22

+ 2h #,2 0" 22 #22 0",2 +h #1 2 0"2 4 0",2 + h0"2 4 #,4), Q,
4
= K A. #, 0"22(h #2 0",4 + h 0" 22 0",4 #22
+ 2h #,2 0" 22 #22 0",2 + h #1 2 0"2 4 0",2 + h0"2 4 #,4),J2
/(%1 Jh%1 K A.h)}
%1 := #2 4 0",4 +0" 220",4 #22 +2#120"22 #lO"12 + #,20"2 4 0",2
+0"24 #,4
> assign (Sol[ 1] ) ;
> Q1So1:=sirnplify(cr 1]);

Q1801:= ,J20"22 #, K A. /sqrt(h(#2 4 0",4 + 0" 22 0",4 #22

+2#120"22 #2 2 0",2 + #1 2 0"2 4 0",2 +0"2 4 #,4)K A.)

> Q2So1:=simplify(cr 2]);

Q2801:= ,J2sqrt(h(#2 4 0",4 +0" 220",4 #l +2#120"22 #220",2

+ #,2 0"2 4 0",2 + 0"2 4 #,4)K A.)0",2 #2 /(h(#2 4 0",4
+ 0" 22 0",4 #22 + 2 #,2 0"2 2 #2 2 0",2 + #,2 0"2 4 0",2
+ 0"2 4 #,4»
We find after some manipulations that

h[(#~O"~ + #~0"~)2 + #~O"to"~ + #~O"~O"~]

However Q2 appears a little different from Q I, so we attempt to discover a
simple relationship between these two variables and see that Q2/ Q, is a constant
given by 0"~#2/(0"~#').
> Ratio:=sirnplify(Q2So1/Q1So1);

13Note that in order \0 see the output in terms of %1 and %2, one should set \Option \Output
Display \Typeset Notation. Otherwise Maple gives an error message.
370 8. Inventory Models

. U)2 P2
Ratlo := - 2 -
U2 p)
This discovery helps in rewriting the expression for Q2 in a fonn that is a mirror
imageofQ) .
> Q2Sol:=simplify(Ratio*Q1Sol);

Q2801 := U)2 P2../2 K 1 /sqrt(h(P2 4 U)4 + U2 2 U)4 P2 2

+2p)2 U2 2 P2 2U )2 + p)2U24 U)2 +U2 4 p)4)K 1)
i.e.,

h[(p~u~ + p~u~)2 + p~u~u~ + p~u1u~]

The optimal value of the objective function is also easily calculated using the
simplify () function.
> GOpt:=simplify(G,power,syrnbolic);

GOpt :=,JK,Ji ../2"fhsqrt(P2 4 U)4 + U2 2 UI 4 P2 2

+2P1 2u 22 P2 2U )2 + p)2 U2 4 U1 2 +U24 P1 4) /(
pluI2 + p)2 U2 2)
To further simplify this expression, we dissect it using the nops ( ) function.
Each of the six operands are listed and we observe, after some manipulations of
the fifth and sixth operands, that the optimal cost can be written as

(8.7)

> nops(GOpt);
6
> seq(op(i,GOpt),i=l • . nops(GOpt));

../2, "fh,sqrt(P24U)4 +U2 2 U)4 pl

,JK, ,Ji,
+2p)2 U2 2 P2 2U )2 + p)2U2 4 U)2 +U24 p)4),
1
P2 2 U)2 + PI 2u 22
> InsideSqrt:=simplify(op(5,GOpt)A2*op(6,GOpt)A2);
. U)2U2 2 +P2 2 U)2+p)2U22
2 2
InsideSqrt :=
P2 U) + PI 2 U2 2
 8.6 Summary 371

Gerchak and Parlar [69] show that if a single source were used - where the
setup cost would be lower - the corresponding optimal costs for each source
i = 1, 2 would be
(8.8)

Thus, depending on the relative magnitudes of G·, Gj and Gi, it would be

optimal to use either both sources (if G· is the smallest), or any of the single
sources (if Gj or Gi is the smallest).

8.6 Summary
Inventory management is one of the oldest and most thoroughly studied areas
of operations research. In this relatively short chapter we covered deterministic
and probabilistic inventory models. These models ranged from the simplest de-
tenninistic demand economic order-quantity model to dynamic inventory mod-
els with random demand. Maple's ability to manipulate expressions symbolically
simplified the analysis of many nontrivial inventory problems such as the optimal
base-stock levels that are obtained by solving a dynamic programming problem.
Maple's numerical analysis facilities also helped in computing the optimal deci-
sions for the exact fonnulation of the continuous-review model with backorders.
Maple's graphics were also useful in visualizing some of the objective functions
including the discontinuous cost function of the quantity discount problem.

8.7 Exercises
1. Consider the basic EOQ model presented in Section 8.4. Now assume that
the demand is an integer quantity; i.e., the units are demanded one at a time
and the time between demands r is deterministic implying that the demand
rate is A = 1/ r. Show that the optimal integer order quantity is the largest
positive integer Q such that

UK
Q(Q-l) < T'
2. In the EOQ models we have discussed, it was implicitly assumed that the
supplier was always available, i.e., we would always receive our order (after
a possibly positive lead-time). In the classical EOQ model now assume that
the supplier is not always available and he can be ON (available) or OFF
(unavailable) for random durations.

(a) Without loss of generality, let the demand rate Abe equal to 1 unit/time.
Let us assume that the exponential random variables X and Y corre-
spond to the lengths of the ON and OFF periods, with parameters a
372 8. Inventory Models

and p, respectively. When an order is placed and the state is ON, an or-
der cost ofSKlorder is incurred. The holding cost is Shlunitltime and
the shortage (backorder) cost is SIr lunit when the state is OFF and the
demand cannot be met. For the sake of generality, also assume that
the time dependent part of the backorder cost is Si lunitltime.
When the inventory level reaches the reorder point r and if the sup-
plier is ON an order for Qunits is placed. This brings the inventory up
to Q+ r. If the supplier is OFF when the inventory reaches r, then the
decision maker has to wait until the supplier becomes available (ON)
before an order can be placed which increases the inventory to Q+ r.
Note that, with Q as the order quantity (which is received immedi-
ately if the state is ON), inventory will reach the reorder point rafter
exactly Q time units since demand rate A = I as we had assumed.
Develop the objective function for this model. Find the optimal solu-
tion for the problem with data given as K = 10, h = 5, 1C = 250,
i = 25, a = 0.25, P = 2.50. Compare the results you would obtain
for the classical EOQ model where the supplier is always available.
HINT: To compute the expected cycle length and expected cycle cost,
condition on the state of the system when inventory reaches the level
r.
(b) For the case of two suppliers suggest ways of dealing with the prob-
lem.

3. This question considers a generalized version of the single period newsven-

dor problem. Assume that there are two products which may be substi-
tutable if one of them is out of stock during the period. For simplicity, as-
sume that the beginning inventory for both products is zero, and there is no
salvage value for unsold units nor a penalty cost for unsatisfied demand.
Use the following notation in your answer:

• u : order quantity of item I (decision variable),

• v : order quantity of item 2 (decision variable),
• X: random demand for item I with density f(x),
• Y: random demand for item 2 with density g(y),
• S; : sale price/unit of item i (i = I, 2),
• C; : purchase costlunit of item i (i = 1,2),
• a : fraction of customers who will accept a unit of item 2 when item
I is out of stock
• b : fraction of customers who will accept a unit of item I when item
2 is out of stock
• n profit
: (random).
 8.7 Exercises 373

Write down the expressions for expected profit per period. i.e., E(n).
HINT: For given values of U and v, and depending on the values taken by
the random variables X = x, Y = y, one has to consider four different
possibilities.
4. Consider a stochastic inventory system where unit demand for items arrive
in a renewal process at time points To, T\, ... Let Tn - Tn-\ (n = 1,2, ... )
be i.i.d. random variables with mean 1/ A., i.e., the demand rate is A.. Assume
that we use an (s, S) policy, that is, when the inventory level falls to s, we
order S - s units which are received immediately. Order cost is $K per
order and the purchase cost is $c per unit. The inventory holding cost is $h
per unit per time.
Find the best values of s and Q so that the average cost is minimized.
5. Consider the last period of the dynamic inventory problem discussed in
Section 8.5.4. Now assume that the cost of purchasing U units is

( ) _ { K + CU, if U > 0
C U - 0, 1·f U= 0.
Show that the optimal policy is now of the (s, S)-type, i.e., it is given in the
form
S-x, if X < s
Jl(x) = { 0, ifx ~ s.
6. Consider the optimization problem
N-\
min E
Ilk 2:0
L
Ie=O
CUie + h max(O, Xle + Ule - WIe) + p max(O, Wle - Xle - UIe)

where Xle is the inventory at the start of period k, Ule is the order quantity
at the start of period k (lead-time is assumed zero) and Wle is the random
demand with given c.d.f. F(wIe) for k = 0, I, ... , N - I. The parameters
c, h and p are the unit costs of purchase, holding and shortage penalty,
respectively.
(a) Find the optimal solution for this problem with f(w) = F'(w) = 1,
o ~ W ~ 1, C = 1, h = I, P = 2, N = I.
(b) Find the optimal solution for the problem
N-\
min lim E
IIk2:0N-+oo
L
Ie=O
(i[cule + h max(O, Xle + Ule - WIe)

+pmax(O, Wle - Xle - UIe)]

with f(w) = F'(w) = I, 0 ~ W ~ 1; C = I, h = I, P = 2 and
a =0.9.
9
Queueing Systems

9.1 Introduction
The English poet Milton once wrote, "They also serve who only stand and wait,"
[178, Sonnet 19] but he never elaborated on the average number of people who
would wait and the average time spent waiting for service. Of course, as a poet
he had no interest in such technicalities and even ifhe did, the mathematical tools
available to him and his contemporaries in the 17th century would not have been
of much help in analyzing the queueing processes arising in his day. I
Modern queueing theory had its origins in the early 1900s when the Danish
engineer A. K. Erlang published his seminal work on congestion in telephone
traffic. Since Erlang's time, the number of published (and unpublished) papers
and books on queueing theory has shown a tremendous growth. In 1987, Disney
and Kiessler [62] estimated at least 5000; given the high growth rate we would
estimate the present number around 10,000.
When does a customer requiring service have to "stand and wait"? Queues
form when current demand for a service exceeds the current capacity to provide
the service. As an example let us consider a simple (and naiVe) deterministic
queueing system, e.g., an assembly line operated by a robot. If we assume that

1 Milton, of course, WM not referring to a physical queueing system when he wrote his Sonnet 19
that mentions "[those] ... who ... wail" Instead, he WM comparing angels who serve God and devout
people on earth who wait with patience for the fulfillment of God's purposes. We have included the
reference to Milton since this famous quotation is a favorite of many queueing theorists; see Cox and
Smith [54, p. vii).

M. Parlar, Interactive Operations Research with Maple

© Birkhäuser Boston 2000
376 9. Queueing Systems

the time between arrivals of items requiring processing is, say, a = 3 minutes
and the service (i.e., assembly) time is s = 2 minutes, then there will never be
any queue in this system and for s :::; a, the robot will be idle for (a - s)/a =
I - sla fraction of the time. However, if s > a, then due to demand exceeding
the capacity, arriving items will pile up and the queue size will grow to infinity.2
If the arrival and service processes are stochastic, then even when the average
time s to serve a customer is shorter than the average time ii between the arrivals
of customers, due to the variability in the system, a queue may form. Such con-
gestion phenomena are commonly observed in daily life where customers have to
wait for service at a bank, at a restaurant or at the airport checkout counters or
jobs arriving at a job shop have to wait for processing or telephone calls arriving
at a switchboard may have to wait until the busy signal is cleared.
A shorthand notation has been developed by Kendall [106] to describe a queue-
ing process. The notation involves a series of symbols and slashes such as A I B I X
I YI Z. Here, A indicates the interarrival distribution [e.g., M for Memoryless (or
exponential or Markovian), D for deterministic]; B indicates the service-time dis-
tribution (e.g., G for general, Elc for Erlang with k stages); X indicates the number
of (parallel) servers which could be any integer between I and 00; Y indicates the
system capacity and Z indicates the queue discipline [e.g., first come, first served
(FC FS), priority (P R)). For example, MI MI I 1001 FCFS is the simplest queue-
ing system with exponential interarrival times, exponential service times, with a
single server, no restrictions on the system capacity and a FCFS service disci-
pline. As another example, MI E2/3/151 PR may represent the emergency de-
partment of a hospital with three doctors and a capacity of 15 where the patients'
interarrival times is exponential and they have to go through two stages (Erlang
with k = 2) before service is completed. Normally, if Y = 00 and Z = FCF S,
these symbols are dropped from the notation and only the first three are used.
Thus, M 1MI I would be used to represent the single-server Markovian system
MI MI I 1001 FCFS.
Queueing processes are studied in order to gain an understanding of their oper-
ating characteristics such as the average number of customers in the system or the
average time a customer must spend in the system. Once these descriptive analy-
ses are complete, a normative analysis of the system may also be conducted in or-
der to analyze such questions as "How many servers should be available?" "How
fast should the servers be?" or "How should the system be designedr' Queueing
theory as it has been developed in the last several decades attempts to answer such
questions using detailed mathematical analyses. In this chapter we will provide a
description of some types of queueing models and demonstrate the usefulness of
Maple in solving several queueing problems.

2The reader who is familiar with the I Love Lucy TV shows of the 1950s will recognize dial the
'Chocolate Factory' episode corresponds to the C$e where $ > Q.
 9.2 Markovian Queueing Systems 377

9.2 Markovian Queueing Systems

In this section we present queueing models where the arrival and service pro-
cesses are both exponential and where only the transitions to neighboring states
are pennissible; i.e., they are based on birth and death processes.

9.2.1 Birth and Death Processes

Before we present the simple Mj Mjl queueing system, it would be useful to de-
velop expressions representing the evolution of the time-dependent probabilities
of the birth and death processes.
We assume that when there are n customers in the system, the arrival rate of the
customers is A" and the service rate is 1-',,; i.e., the arrival and service rates are, in
general, state dependent. Thus, when the system is in state n, the probability of a
birth (arrival) in a short time interval of length M is A"M + o(a/), n ~ 0, and
the probability ofa death (departure) is I-'"al + oeM), n ~ 1. The queue forms
if the server is busy and a new customer arrives and must wait for service. When
the service of a customer is completed, he leaves and the customer who has been
in the system longest enters the service facility.
Due to the memorylessness of the interarrival and service times, the stochastic
process {N(/); I ~ O} where N(/) is the number of customers in the system at
time I is a continuous-time Markov chain.
If we define p,,(/) = Pr{N(/) = n} as the probability that there will be n
customers in the system at time I, an important problem is that of computing
the transient solution of the probability p,,(/) for finite I. In principle, using the
birth-death processes that arise from the memorylessness of the arrival and ser-
vice processes, one can develop an infinite system of first-order linear differen-
tial equations (KoImogorov equations) whose solution would provide us with the
transient probabilities p,,(/).
Since the state n at time 1+ M can be reached either from state n (if there is
neither a birth nor a death) or from state n - I (if there is a birth and no deaths)
or from n + I (if there is a death and no births), we can wriW for n ~ I
p,,(1 + a/) = p,,(/)(I - A"M)(I - I-'"M) + P,,+I (1)1-',,+1 M(1 - A,,+I M)
+P,,-I (/)A,,_I M(I - 1-',,-1 a/) + oeM). (9.1)
Similarly, for n = 0, we get
Po(1 + a/) = Po(/)(I - loM) + PI (/){J.l1 M)(I - AI a/) + oeM). (92)
Following the standard steps of simplification, carrying the term involving p" (I)
to the left of the equality, dividing both sides by M and taking the limit as M ~ 0

3We could also have included the probability of I birth and I death in the time interval At as
p"(t)(1,,6t){Jl,,At). But the result is of the order o(At), and thus it can be safely ignored in the
subsequent development.
378 9. Queueing Systems

gives

p~(I) = -O.n + Iln)Pn(l) + Iln+IPn+1 (I) + ,tn-lPn-I (I), n ~ 1(9.3)

PO(I) = -AOPO(I) + III PI (I), n = O. (9.4)

The explicit analytical solution of this infinite system of differential equations

(9.3)-{9.4) for finite values of 1 is impossible to obtain. However, if the limiting
solution Pn = lim Pr{N(I) = n} is assumed to exist, then lim p~(I) = 0 and
1-+00 1-+00
the system can be written as an infinite system of difference equations

o = -(An + Iln)Pn + Iln+IPn+1 + ,tn-lPn-I, n ~ 1 (9.5)

o = -AOPO + IlIPt. n 0 = (9.6)
with the condition L~ Pn = 1. We will postpone the general solution of this
system to Section 9.2.4.

9.2.2 MI Mil Queueing System

We now use Maple to solve the system of difference (recurrence) equations (9.5}-
(9.6) for the special caseofA n = A and Iln = Il, which reduces the birth and death
processes to the M 1M11 queue. Note that before using the r sol ve () command,
we inform Maple of the initial condition that for n = 1, PI = (AI Il)PO (since at
n = 0, we have 0 = -APO + IlPI).
We first input the difference equations (9.S)-{9.6) using Maple's syntax:
> restart: # MMl.MWS
> mrnl:={ O=-(lambda+mu)*p(n)
+lambda*p(n-l)+mu*p(n+l),p(n=O •• l)
= (lambda/mu) "n*p[ Ol};

A
mml := {O = -(A + Il)p(n) + A p(n - 1) + Il p(n + 1), p(n = 0.. 1) = (_)n PO
Il
)
Solving the system for the unknown Pn gives
> sol:=rsolve(mrnl,p(n));

> p : =unapply(sol,n);
A
P := n ~ (_)n PO
Il
Thus, Pn is obtained as a function of n and the model parameters A and Il and
the (as yet) unknown po. Let us now check the solution for the first few values of
n:
> p(O); p(l); p(2);
 9.2 Markovian Queueing Systems 379

po
ApO
J.l
A2 po
7
Before finding the value of the unknown po in tenns of the model parameters,
we develop fonnulas for some of the operating characteristics of the M 1M11
queue, e.g., the expected number of customers in the system, L = L~ npn, and
the expected queue size Lq = L~I (n - 1)Pn. Since the initial state probability
po is not yet computed, the results for L and Lq are obtained in tenns of this
unknown value.
> L:=sum(n*p(n),n=O .• infinity);
Lq:=sum((n-l)*p(n),n=l •. infinity);
L.- po J.l A
.- (-A+J.l)2

POA2
Lq:= (-A + J.l)2
Now, using the fact that L:;O=o p" = 1, we can finally compute the unknown
po as
> p[ 0] :=solve(l=sum(p(n),n=O .• infinity),p[ 0]);
-A + J.l
po := ----''-
J.l
The expressions for Land Lq can now be obtained in tenns of the parameters
Aand J.l.
> L; Lq;

(-A+J.l)J.l
Since the quantity AI J.l appears frequently in many queueing fonnulas, we in-
troduce a new symbol p, which is defined as AI J.l. This dimensionless quantity is
known as the "traffic intensity" or the "utilization factor" of the service facility.4
Note that for the M 1M11 queue, the expected system size L and the expected
queue size Lq are related through the utilization factor p, i.e., L = p + Lq.

4In multiple-server queueing systems the utilization factor is defined as p = J./(cp) where c is the
number of servers present in a service facility and p is the service rate of each server.
380 9. Queueing Systems

Having developed the expressions for the expected number ofcustomers present
in the system and in the queue, we now present the formulas for the distributions
and the means of the waiting time random variables of the M / M / I model.
Let Tq be the random time a customer spends waiting in the queue and Wq{t)
its distribution function. To develop Wq{t), we argue as follows: It is possible that
a customer may not have to wait in the queue at all if the system is empty at the
time of his arrival. Thus, for t = 0,

Wq{O) = Pr{Tq = 0)
= Pr{system empty at time of arrival)
= pO = I-p.
For t > 0, we condition on the number of customers found in the system at the
time of the arrival and write

Wq{t) = Pr{Tq::S t)

L Pr{n service completions in ::s t I arrival finds n

00

=
n=l
customers in system) x Pn + Wq{O)
Since each service time is exponential, the distribution of the completion of n
services is Erlang with n stages. Hence, as pn = (I - p )pn, the distribution
Wq{t) can be written as follows:

Wq{t) = t; [f'10
00
Jl
(Jl~)n-l
_ I;!
-JJX
dx
]
x {I - p)pn + (l - p)

f'
= (1- p)p 10 Jle-JJX t;
00 {JlX P)n-l
(n _ I)! dx + (l - p).

Now, using Maple's help we compute the foregoing expression.

> restart: *
wqt.mws
> cdfTq:=(l-rho)*rho* Int (mu*exp(-mu*x)
* Sum ( (mu* x* rho) " (n-l) / (n-l) ! , n=1. . infinity) ,
x=O .. t)+(l-rho);

cdjTq := (I - p) p f' Jl e(-JJx) {~ (Jl x p)(n-l») dx + I _ p

10 \.~ (n -I)!
> Wqt:=normal(value(cdfTq));
Wqt := -p e(-JJ t+JJ t p) + I
Thus, the distribution function of Tq is obtained as

I - p, t = 0
Wq{t) ={ I _ pe-JJ(l-p)t, t > o.
 9.2 Markovian Queueing Systems 381

The probability density function Wq(t) of Tq is found by differentiating Wq(t)

with respect to t as
> wqt:=diff(Wqt,t);
wqt := -P (-/l + /l p) e(-p t+p t p)
Using this result (and infonning Maple that 0 < P < 1 and /l > 0) we can
compute the expected value E(Tq) = fooo t Wq(t) dt.
> assume(rho<l,rho>O,mu>O);
> ETq:=O* (l-rho)+Int(t*wqt,t=O •. infinity);

ETq:= 10 00
-tp(-/l+/lp)e(-Pt+ptp)dt

> simplify(value(ETq));
P
/l(-l+p)
Finally, substituting the definition of p = )./ /l, the expected time spent waiting
in the queue is obtained in terms of the model parameters.
> Wq:=normal(subs(rho=lambda/mu,%));
).
Wq := ----:-~
/l (/l - ).)
So far we have found expressions for L, Lq and Wq as the expected number of
customers in the system, the expected number of customers in the queue and the
expected time spent in the queue, respectively. It should now be easy to see that
the expected time spent in the system is W = Wq + 1//l = 1/ (Jl - ).) since Wq
is the expected time spent in the queue and 1/ /l is the expected time spent in the
service. (Recall that the service time distribution was assumed exponential with
parameter /l; hence its mean is 1/ /l.)
By this time the observant reader may have noticed that there is a certain rela-
tionship among the expressions for L, L q , Wq and W. In fact, it turns out that, as
we see in the following Maple output, L =).W (and Lq = ).Wq).
> W:=normal(Wq+l/mu);
1
W:= /l-).
> L:=lambda/(mu-lambda);
Lq:=lambda A 2/(mu* (mu-lambda));

L := /l ~).
).2
Lq'---
.- /l (/l - ).)
> is(L=lambda*W);
382 9. Queueing Systems

true
> is(Lq=larnbda*Wq);
true

This is known as Little's fonnula [125] and it can be shown that the fonnula is
not specific to the M / M /1 queue, but under very mild assumptions, it is true in
general for any queue; see [180] and Wolff [196, p. 236].

9.2.3 Finite Capacity Markovian Queue: MI MIll K

In this section we assume that the queueing system is identical to the M / M /1 sys-
tem of Section 9.2.2 except that when there are K customers present, the new ar-
rival(s) turn away and do not enter the system. Such finite capacity queues are en-
countered frequently, e.g., at barber shops, drive-ins, and telephone switchboards.
With the limit of K imposed on the system capacity, the infinite system of dif-
ference equations given by (9.5)-(9.6) reduces to the following finite system of
linear equations, which can be solved easily by Maple.

o = + p.n)Pn + P.n+IPn+1 + A.n-IPn-l, n = I, ... , K

-(A.n
o =
-A.OPO + P.IPI, n = 0
Assuming again that A.n = A. and P.n = p. and using the same set of commands
but keeping in mind the fact that the system capacity is K, we solve this problem
and compute the expected number of customers L as follows.
> restart: # mmlcap.mws
> MMlcap:9 0=-(larnbda+mu)*p(n)+larnbda*p(n-l)
+mu*p(n+l),p(n=O .. l)=
(larnbda/mu) "n* p[ 0] } ;

MMlcap:= {
A.
0= -(A. + p.) p(n) + A. p(n - I) + p. p(n + I), p(n = 0 .. 1) = (_)n
p.
po}
> sol:=rsolve(MMlcap,p(n));
A.
sol:= (_)n po
p.
> p:=unapply(sol,n);

> L:=sum(n*p(n),n=O .. K);

A.
po (_)(K+I) p. (-(K + l)A. + (K + I)p. + A.) A.
L .-
·-
p. + pop.
(-A. + p.)2 (-A. + p.)2
 9.2 Markovian Queueing Systems 383

> p[ 0] :=solve (l=sum(p (n) ,n=O •• K), p[ 0] ) ;

-A + Jl
.-
pO ' -
A
Jl «_)(K+I) - 1)
Jl
> L:=factor(L);

_(~)(K+I) A K + (~)(K+I) Jl K + (~)(K+1) p- A

'-
L .- Jl P Jl
A
(-A + Jl)«_)(K+I) - 1)
P
As a final check of the results, we asswne that A < Jl and take the limit of L to
obtain the original result found in the M 1MI I system with infinite capacity.
> assume(larnbda<mu,lambda>O);
> limit(L,K=infinity);

-A + Jl
It should be noted that since the system capacity is finite (Le., K) and the queue
size cannot grow without bound, the results hold not only for A < Jl but also for
A > Jl. When A = Jl, it is easy to show that pn = I/(K + I), n = 0, ... , K and
L =!K.

9.2.4 Multiserver Queue M / M / c

So far we have asswned that the queueing system consists of a single server. In this
section we generalize the model to allow for the possibility of multiple servers.
To do this we return to the general birth-death process equations (9.5)-(9.6) and
attempt to solve them for the steady-state probabilities Pn, n ~ O.
In this case, Maple's rsol ve () procedure fails to find a solution since the
parameters An and Jln are functions of the state n. We thus proceed in a different
manner and solve the system for the first few values of n in order to see a possible
pattern in the solution.
> restart: * mmc.mws
> eq[ 0] : =-lambda{ 0] * p[ 0] +mu{ 1] * p[ 1] ;
eqo := -AO pO + JlI PI
> assign (solve ({ eq[ O]} ,{ p[ I]} »;
> p[ 1] ;

AOpO
JlI
> eq[ 1] : =- (lambda{ 1] +mu{ 1] ) * p[ 1] +mu{ 2] * p[ 2]
+ lambda{ 0] * p[ 0] ;
384 9. Queueing Systems

eql := -
0·1 + JlI) AO pO + Jl2 P2 + AO pO
JlI
> assign (sol ve ({ eq[ 1) } ,( p( 2) } ) ) ;
> p( 2) ;

AOpOAI
JlIJl2
> eq[ 2) : =- (lambda[ 2) +mu( 2) ) * p( 2) +mu( 3) * p( 3)
+ lambda[ 1) * p( 1) ;

eq2 := -
(A2 + Jl2) AO pO AI + Jl3 P3 + .-;....::....;;.....;.
AO pO AI
JlIJl2 JlI
> assign (solve ({ eq[ 2) } ,( p( 3) } ) ) ;
> p( 3) ;

AOpOAI 12
JlIJl2Jl3
For the first few values of n, there does appear to be a structure to the solution
of this system. Next. we automate this solution procedure to compute the p" for
higher values of n.
> for n from 3 to 10 do
> eel n) :=- (lambda[ n) +mu( n) )*p( n) +mu( n+1) *p( n+1)
+lambda[ n-1) *p( n-1) ;
> assign (solve ({ eq[ n)} ,( p( n+1)} »;
> p( n) ;
> od:
> p( 2); p( 3); p( 4); p( 5); p( 6) ;
10pOAI
JlIJl2
10pOl112
JlIJl2Jl3
10pOl112 1 3
JlIJl2Jl3Jl4
10 pO 1112 1 3 1 4
Jl I Jl2 Jl3 Jl4 Jls
AOpOAI A2 A3 A4 AS
JlIJl2Jl3Jl4JlSJl6
 9.2 Markovian Queueing Systems 385

Again, we note that the interesting pattern that has emerged so far suggests that
the probabilities p,. are in the fonn

P,.=PO n,.
;=1
,.1.;_1
-.-,
p,
n ~ I. (9.7)

Naturally, this result must be proved using mathematical induction (for which
we refer the reader to Gross and Harris [76, p. 85]). Since L~ I p,. = I, i.e.,

1= PO(I + t.il A~~I)

the existence of the steady-state solution requires the convergence of the infinite
series L:'I 07=1 ,.1.;_11 p;. For example, when ,.1.,. = ,.1. and p,. = p, the model
reduces to the simple MIMII queue and we obtain p,. = (Alp)n(l - Alp),
n ~ 0 provided that ,.1. < p. When A. > p, customers arrive faster than they can
be served and thus the queue size diverges to infinity. When ,.1. = p, the queueing
system behaves like a symmetric random walk that is null recurrent and has no
limiting probabilities [157, p. 106).
Returning to the multiple-server queue M 1MI c with c servers, we asswne that
each server's service time is i.i.d. exponential and the arrival process is Poisson
with rate ,.1.. Hence An = ,.1., and

np, n = I, ... , c
Pn = { cp, n = c, c + I , ....

Using An and Pn in (9.7), one obtains [76, p. 86] the following expressions as the
solution for Pn.

~={
n=I, ... ,c

n=c,c+I, ....

After some lengthy algebra PO is computed as

(9.8)

thus completing the analysis of the steady-state probabilities of the M 1MI c model. S
We now compute the expected queue length Lq = L~=c(n - c)Pn using
Maple's sum () function.

Sit should be noted here that the condition for the existerK:e of PO is p = A./(cp) < I, implying
that the mean arrival rate A. must be less than the maximum potential service rate cp o
386 9. Queueing Systems

> restart: # mmcLq.mws

> (n-c)* (lambda/mu) "n/ (c" (n-c)*c! )*p[ 0] ;
).
(n-c)(_)npO
Jl
c(n-c) c!
> Lq:=sum(%,n=c .. infinity);
).
( - )(I+c ) cpO Jl2
Lq ._ ....!.Jl~_ _---=-
.- c! (-). + Jlc)2
Since L = AIJl + Lq , the expected number of customers in the system is found
as
> L:=lambda/mu+Lq;

where pO is found using (9.8).

As an example, we set c = 5, ). = 4 per hour and Jl = 2 per hour and compute
the values of pO, L and W using the following Maple commands:
> restart: # mmcpO.mws
> pO:=(' swrr «l/n!)* (lambda/mu)"n,n=O .. c-l)
+(l/c!)* (lambda/mu)"c
* (c*mu/(c*mu-lambda»)"(-l);

> c:=5; lambda:=4; mu:=2;

c :=5
).:= 4
Jl :=2
> evalf (pO) ;
.1343283582
> Lq:~ (lambda/mu)" (1+c)*c*pO*mu"2
/ (c!* (mu*c-lambda) "2)' ;

(~)(I+C) cpO Jl2

_ _ _-=--
Lq ._ ....!.Jl
.- c! (c Jl - ).)2
 9.2 Markovian Queueing Systems 387

> evalf(Lq);
.03980099502
> L:=evalf(Lq+lambda/mu);
L := 2.039800995
> W:=evalf(L/larnbda);
VV:=.5099502488

9.2.5 Markovian BulkA"ival System: M X I MIl

In many realistic applications, customers arrive at a queueing system in groups
rather than singly. Obvious examples are the arrival of a family of, say, four people
at a restaurant, groups of travelers at customs or a box of parts at an assembly line.
In this section we consider a single-server queue in which customers arrive in
groups according to a Poisson process with rate A.. VVe will assume that the group
size X is a random variable with the density Pr(X = k) = Ok, k = 1,2, .... In
other words, the probability that a group of k customers arrives in an infinitesimal
interval oflength o(M) is A.OkM + o(M). If the service time is exponential with
parameter p, then the time-dependent balance equations for the resulting system
can be written as
n
p~(t) = -(A. + P)Pn(t) + PPn+l(t) + ~A.OkPn-k(t), n~k~ 1
k=1
Po(t) = -A.pO(t) + PPI(t).
VVe note here that Lk=1 A.OkPn-k(t)M is the probability that the system state is
increased to n from n - k with the arrival of k customers (k = 1, ... , n), which
has a probability of Ok.
Assuming that the steady-state solution exists, we have
n
o = -(A. + P)Pn + PPn+1 + ~A.OkPn-k' n~k~ 1 (9.9)
k=1
o = -A.pO + PPI. (9.10)

To solve this system, we introduce the generating ftmction G(z) = ~=O pnzn.
Multiplying (9.9) by zn for n = 1,2 ... , adding the result to (9.10) and simplify-
ing (see, [129, p. 3] and [130, p. 182]) we obtain

G(z) = p(I - z)pO

p(I - z) - A.z[1 - A(z)]
where A(z) = L~onzn is the generating ftmction of the random variable X.
Since G(I) = 1, it can be shown that pO = 1 - p where p = A.E(X)/ P < 1 is
the traffic intensity.
388 9. Queueing Systems

We now apply these results to a specific model where it is assumed that the
random variable X has the truncated geometric density, i.e., Ok =
Pr(X =
k) =
c(l - c)k-l, 0 < c < I, k = 1,2, ....
> restart: # bulkgeom.mws
We define the density of X and check that the probabilities add up to unity.
> pdfX:=c* (l-c)A(n-I);
pdjX:= c (l - c)(n-l)
> sum(pdfX,n=I .. infinity);

The next three commands define the generating function A(z) of X, compute the
mean of X using the property that E (X) = L~ 1 kOk = A' (z) 1== \' and write the
generating function G(z) of the steady-state probabilities Pn.
> A:=sum(pdfX*zAn,n=I .. infinity);
cz
A·-----
.- -z+zc+ 1
> EX:=normal(subs(z=l,diff(A,z)));
1
EX:=-
c
> G:=mu*p[ 0] * (I-z) / (mu* (I-z) -lambda* z* (I-A));

G ._ p pO (l - z)
.- cz
p(l-z)-lz(l- )
-z+zc+ 1
Since G(z) is given in terms of the unknown po, we find this value using the
property that po = lim G(z).
=~l

> pO:=limit(G,z=I);
cpop
pO:=
-pc+l
> p[ 0] :=normal (solve (l=pO,p[ 0] ));
-pc+l
po .-
.-
cp
Once po is found, the traffic intensity p can be computed as
> rho:=lambda/(c*mu);
1
p:=-
pc
In order to invert the resulting generating functions we load the genfunc li-
brary and display G(z), which no longer depends on the probability po.
> with(genfunc);
 9.3 Transient Solutions 389

[rgf_charseq, rgf_encode, rgf_expand, rgf.flntirecur, rgf_hybrid,

rgf_norm, rgf"'pjrac, rgfJelate, rgf_sequence, rgf_s;mp, rgf_term,
termscale]
> G;
(-pc+)')(l-z)
c (p (1 - z) - ). z (1 -
cz
-z+zc+
1 »
Finally, the steady-state probabilities Pn are computed explicitly for each n.
> pn:=rgf_expand(G,z,n);

).(-pc+).)(p - PC+).)n
pn:=-------------~p-----
c (p - p c +).) p
As a check of the results found, the next command adds all the probabilities and
finds that they sum to one, i.e., 1:~ Pn = I.
> norrnal(R 0] +surn(pn,n=l .. infinity));

The expected number of customers in this bulk queueing system, L, can now
be easily computed using the property that L = E(customers in system) =
1::'1 nPn = G'(z)I==I·
> L:=lirnit(diff(G,z),z=l);
).
L .-
·-
(-pc+).)c

9.3 Transient Solutions

The results obtained in Section 9.2 assumed that the system was in operation for
long enough so that the steady state was reached. This assumption simplified the
model as the time-dependent differential-difference equation system (9.3)-(9.4)
reduced to a time-independent system of difference equations.
However, in many realistic problems, one may be interested in the finite-time
(transient) solution of the probabilities. Thus, we now describe the transient solu-
tion of some special queueing systems.

9.3.1 Finite Capacity M / M /1/ K Queue with K =1

In this simple system there is only a single customer, e.g., a machine, that may
require service (e.g., repair) at random times. We will let state 1 correspond to the
state of the busy server (machine in repair) and state 0 correspond to the state of
the idle server (machine operating). Thus, if ). is the arrival rate of the machine
390 9. Queueing Systems

(i.e., the rate at which the process departs state 0) and P is the service rate of the
server (i.e., the rate at which the process departs state 1), then 1I A is the average
time the machine is in operation and 1I P is the average time it takes to repair
the machine. Using the steps for developing differential equations for birth-death
processes, it is easy to show that we have two differential equations representing
the evolution of the transient probabilities for the finite capacity M1M11 I K queue
with K = 1.

poet) = -ApO(t) + PPI (t)

PI(t) = -PpO(t) + API(t)

Maple's deep knowledge of the solution of systems of differential equations

makes it almost trivial to find the solution for this problem where we will assume
that initially the system is empty, i.e., pOCO) = 1 and PI (0) = o.
> restart: * MMll-tra.MWS
> deO:=diff(p.O(t),t)=-larnbda*p.O(t)+mu*p.l(t);
deO := f, pOet) = -A pOet) + P pI(t)
> del:=diff(p.l(t),t)=-mu*p.l(t)+larnbda*p.O(t);
del := f, pI(t) = -P pI(t) + A pOet)
> inO:=p.O(O)=l;
inO := pOCO) = 1
> inl:=p.l(O)=O;
inl := pI (0) = 0
> sol:=dsolve({deO,del, inO,inl} ,{p.O(t),p.l(t)});

(-tA-tp) ~_ e(-tA-tp) AJl}

so/:= { pOet) = _P_ + e A, pI(t) = A + P A+ P
A+P A+P P

> assign(sol);
> p{ 0] :=unapply(simplify(pO(t)) ,t);
p{ 1] : =unapply (simplify (pI (t) ) , t) ;
P + Ae(-().+P)t)
pO := t -+ ~-:----­
A+P
A (-1 + e(-().+p) t»
PI := t-+
A+P
The last line of the Maple output gives the transient solution for the M1M11/1
system. To find the steady-state solution, we let t -+ 00 and obtain the steady-
state probabilities as pO = pl(A + p) and PI = AI(A + p).
> assume (larnbda>O, mu>O); interface(showassumed=2);
 9.3 Transient Solutions 391

> limit (p( 0] (t), t=infinity);

limit (p( 1] (t), t=infinity);

Jl
1+Jl
with assumptions on Jl and 1

1
1+Jl
with assumptions on 1 and Jl

9.3.2 Ample Server System M / M /00

Such systems are encountered in self-service-type environments where the cus-
tomers do not have to wait at all for service. The service rate is assumed to be
linearly increasing with the number of customers in the system (Le., each cus-
tomer is his own server); thus Jln = nJl. If we also assume that 1n = 1, the
differential-difference equation system (9.3)-{9.4) takes the form

p~(t) = -(1 + nJl)Pn(l) + (n + I)JlPn+1 (I) + 1pn-1 (I), n ::: 1(9.11)

Po(l) = -11'0(1) + JlPI(I), n = O. (9.12)

To solve these equations, we make use of the generating function of the tran-
sient probabilities defined by
00

G(z, I) = LPn(I)?
n=O
Multiplying both sides of (9.11) by z", summing from 1 to 00 and adding the
equation in (9.12) gives
00 00 00 00

LP~(I)? = - L(1+nJl)Pn(t)?+Jl L(n+l}Pn+I(I)?+1 LPn-I (I)?

n=O n=O n=O n= I

Since L:'.
Pn(l)nz"-1 = fJG(z, 1)/fJz and L:;O=o p~(I)z" = oG(z, 1)/01, these
equations can be rewritten as a partial differential equation (PDE) in terms of the
unknown generating function G(z, I); see Gross and Harris [76, Section 2.10.3]:

oG(z, I) = Jl(1 _ z) fJG(z, I) _ 1(1 - z)G(z, I).

01 oz
At this stage we request Maple's help in solving this PDE.
> restart: # ample.mws
> PDEGenFunc:=diff(G(z,t),t)=mu* (l-z)*diff(G(z,t),z)
-lambda* (l-z)*G(z,t);
392 9. Queueing Systems

PDEGenFunc := f, G(z, I) = p (1 - z) (t G(z, I» - A. (1 - z) G(z, I)

> pdsolve(PDEGenFunc,G(z,t));

G(z, I) = YI( -I P + In(-I +z»e<Aj)

p
The solution is obtained in tenns of an arbitrary function _Fl. To detennine the
fonn of this function, we assume that the system is initially empty, i.e., 1'0(0) = I
and Pn(O) = 0 for n > O. Thus,

G(z,O) = 1 = _FI ( -In(-I+Z))

p exp (A.Z)
-; .

The next four Maple statements use this infonnation and detennine the fonn of
the unknown function.
First, we give a name to the tenn inside _Fl.
> inside:=(t*mu-ln(-l+z))/mu;
"de Ip-ln(-I+z)
Insl := ......:....----=---...:...
p
At 1 = 0, this tenn takes the value
> subs(t=O,inside);
In(-I +z)
p
which, when equated to, say, y and solved for z gives
> solve(%=y,z);
e(-YIl) +1
Substituting the expression for z into inside, we obtain the functional fonn of
FI:
> fl:=y->exp(-lambda* (exp(-y*mu)+l)/mu);
_1 (e(-Y P)+I))
(
fJ :=y ~ e I'

i.e.,
_FI(y) = exp ( A.exP(-;p) + I).
Now that the fonn of the unknown function _ F I is detennined, we can explic-
itly compute the generating function G(z, I) as follows.
> fl(inside)*exp(lambda/mu*z);
l(e(-II'+ln<-I+:»+I)) 1:
(
e I' e(p)

> G(z,t) :=simplify(%);

 9.3 Transient Solutions 393

. (A(-I+=><;<-I/ll_11)
G(Z, t).= e
With the explicit fonn of the generating function available, the transient proba-
bilities (p,,(t)} can be obtained by expanding G(z, t) in a power series L~=oa"z"
where, of course, the a" coefficients are the required transient probabilities p,,(t)
since the generating function was defined as G(z, t) = L~=O p,,(t)z".
Using the Maclaurin series, i.e.,

a" = !.. a"G(z, t) I '

n! az" ==0

the first few coefficients are computed with Maple's help as follows.
> a[ 0] :=(1/0!)*subs(z=0,G(z,t));

( A<e<-I/ll-n)
ao :=e /I

> a[ 1] :=(l/1!)*subs(z=O,diff(G(z,t),z$l));
a[ ~=(1/2!)*subs(z=0,diff(G(z,t),z$2));
a[ 3]=(1/3!)*subs(z=0,diff(G(z,t),z$3));

(
A lel-I/ll-n)
l (e(-t II) - 1) e /I
a) :=
Jl

These results imply-after using induction-that the transient probabilities of

the ample server M/ M /00 system are obtained as

1
p,,(t) = n! [l;(1 - J" [l
-;(1 -
e-Il t ) exp e-lit ),
] n ~ O.

Finally, as t ~ 00, we see that the steady-state probabilities reduce to

(l/ Jl)" exp(-l/ Jl)

p" = , n ~ 0,
n!

which is, of course, the Poisson density with parameter l/ Jl.

394 9. Queueing Systems

9.4 Queueing Networks

Many realistic queueing systems consist of several single queues (nodes) whose
output becomes the input to another queue. Manufacturing or assembly line pro-
cesses where units must be processed in a sequence of workstations (e.g., drill
press, paint shop and packaging) and a university registration process where stu-
dents must stop at several desks (e.g., advisor, cashier) are examples of serial
queueing networks. In the more general - nonserial - case customers may ar-
rive from outside to any node that may have any number of servers and may depart
from any node and even return to nodes previously visited (e.g., for rework).
In this section we will examine some models representing such queueing net-
works. First, we will describe a simple serial queue with blocking and then we will
consider a Jackson network, which is a queueing network with a special structure.

9.4.1 Serial Queue with Blocking

Consider a very simple serial queueing network with two nodes (stations) and a
single server at each node [76, p. 226]. Customers must sequentially visit node 1
and then node 2.6 It is assumed that no queues are allowed at any node. If station
2 is occupied and service is completed in node I, the customer in node I must
wait (i.e., is blocked) until the node 2 customer is cleared. When the system is
blocked, any arrivals to node 1 are turned away even if node 2 is empty.
The state of the queueing system is a vector (m, n). Here, (0, 0) is the empty
system, (I, 0) is the system with a customer in node I only, (0, I) is the system
with customer in node 2 only, (I, 1) is the customers in both nodes and (b, I) is
the blocked system.
We define Pmn(/) as the probability that the state of the system is (m, n) and
assume that arrivals to the system are Poisson with rate l and service time is
exponential with parameters PI and P2 in nodes I and 2, respectively. The usual
procedure gives the following system of five differential equations:

Poo(t) = + P2POI (I)

-lpoo(/)
p~o(t) = -PIPIO(/) + P2PII (I) + lpOO(/)
PCB (I) = -(l + P2)POI (I) + PI PIO(/) + P2Pbl (I)
p~ 1(/) = -(PI + P2)PII (I) + lPOI (I)
Pbl (I) = -P2Pbl(/) + PIPII(/)

We will assume that the system is initially empty, i.e., poo(O) = 1.

This is a linear system of differential equations in the form P'(/) = P(/)A,
and hence, in principle, it can be solved analytically using the spectral lheorem

6An example of such a system is a car wash with two stages-in the first stage the car is washed,
and in the second stage it is vacuumed. This was first discussed in Example 91 of Chapter 7, Stochastic
Processes.
 9.4 Queueing Networks 395

([102, p. 541], [64, p. 113]) which is implemented by Maple's exponen tial ( )

procedure. However, the results are several pages long and not very instructive.
For this reason, we choose to present a discussion of the numerical solution of
this system.
The five differential equations are entered in the usual way using Maple's syn-
tax and the system is defined in the following lines. The initial condition that the
system is empty, i.e., that pOO(O) = 1, is indicated by the ini: = statement.
> restart: i serial.rnws
> de[ 0,0] :=diff(P[ 0,0] (t),t)=
-lambda* p[ 0, 0] (t) +rnu( 2] * p[ 0, 1] (t);

deo,o := §; pO,o(l) = -A pO, 0(1) + 112 pO, I (I)

> de[ 1,0] :=diff(P[ 1,0] (t),t)=
-rnu( 1] * p[ 1, 0] (t) +rnu( 2] * p[ 1, 1] (t)
+lambda*p[ 0,0] (t);

del,O := §; PI,o(l) = -Ill PI,o(l) + 112 PI,I (I) + ApO,O(I)

> de[ 0,1] :=diff(p[ 0,1] (t),t)=
- (lambda+rnu( 2] )*P[ 0,1] (t)
+rnu( 1] * p[ 1, 0] (t) +rnu( 2] * p[ b, 1] (t);

deo, I := §; pO, I (I) = -(A + 1l2) pO, I (I) + III PI, 0(1) + 112 Pb, I (I)
> de[ 1,1] :=diff(P[ 1,1] (t),t)=
- (rnu( 1] +rnu( 2] ) * p[ 1, 1] (t)
+lambda*p[ 0,1] (t);

del, I := §; PI, I (I) = -(Ill + 1l2) PI, I (I) + A pO, I (I)

> de[ b, 1] : =dif f (P[ b, 1] (t), t) =
-rnu( 2] * p[ b, 1] (t) +rnu( 1] * p[ 1, 1] (t);

deb,l := §; Pb,I(I) = -1l2Pb, 1(1) + III PI,I(I)

> sys:=de[ 0,0] ,de[ 1,0] ,de[ 0,1] ,de[ 1,1] ,de[b,l]:
> ini:=p[ 0,0] (0)=1,P[ 1,0] (0)=0,
p[ 0,1] (O)=O,P[ 1,1] (O)=O,P[ b,l] (0)=0;
;n;:= pO,o(O) = 1, PI,O(O) = 0, pO,I(O) = 0, PI, 1(0) = 0, Pb,I(O) = 0
Numerical values for the parameters A, III and 112 are specified as 5, 2 and 4,
respectively.
> lambda:=5; rnu( 1] :=2; rnu( 2] :=4;
A := 5
III :=2
112 :=4
Maple's dsol ve () procedure computes the solution of the system numeri-
cally using the Runge-Kutta method.
396 9. Queueing Systems

> F:=dsolve({ sys,ini},

{ p( 0, 0] (t), p( 1, 0] (t), p( 0, 1] (t), p( 1, 1] (t),
p( b, 1] (t)} ,type=numeric);
F := proc(rlrf45_x) ... end
The solution is displayed at t = 0, 1, 2.5 and 20. We note that the steady-state
levels have been reached as early as t = 2.5.
> F(O); F(l); F(2.5); F(20);
[I = 0, Pb,l(t) = 0, fJO,I(I) = 0, fJO,o(l) = 1., PI,o(l) = 0, PI,I(I) = 0]
[I = 1, Pb,I(I) = .05192669369649984, fJO,I(I) = .1471821386619817,
fJO,o(l) = =
.1197361520137099, PI,O(I) .5616488539244483,
PI,I(I) = .1195061617033598]

[I = 2.5, Pb,I(I) = .06197178680874423, fJO,I(I) = .1488757203993332,

fJO,O(I) = .1190881473988159, PI,O(t) = .5460107337664787,
PI,I(I) = .1240536116266276]

[I = 20, Pb,l(t) = .06203473911889895, fJO,I(I) = .1488833747939046,

fJO,o(t) = .1191067001829406, PI,o(l) = .5459057062649429,
PI,I(t) = .1240694796393126]
To see how the transient probabilities behave for, say, the empty system and the
blocked system, we assign the results for pOo(l) and Pbl(t) to Sol[ 0,0] and
Sol[ b, 1] using Maple's op () procedure. 7
> Sol[ 0,0] :=t->rhs (op (2, F(t»);
Sol[b,l] :=t->rhs(op(6,F(t»);
Solo,o := 1 -+ rhs(op(2, F(I)))

Solb, I := 1 -+ rhs(op(6, F(I)))

The results are plotted in Figure 9.1.
> ftplot ({ Sol[ 0,0] ,Sol[ b, 1]} ,0 •• 6, O•• 1);
The steady-state probabilities can be easily computed symbolically since as
1 -+ 00, P:"n(t) -+ O. The resulting system oflinear algebraic equations is easily
solved by Maple as follows:
> restart: ft serial.rnws (Steady State)
> eq[ 0,0] :=-larnbda*p( 0,0] +rnu( 2] *P( 0,1] ;
eqo,o := -.t fJO,o + #2 fJO,l
7ff nonadaptive plotting-where graphs may not be as smooth as possible-is acceptable, then
there is a simpler way to plot the transient probabilities: After generating the solution to Fusing
dsol ve as above,load the plots graphics package using wi th (plots I and apply the command
odeplot(F,11 t,F{ 0,0) (tl),1 t,F{ b,l) (tl)) ,0 .. 61.
 9.4 Queueing Networks 397

0.8

0.6

0.4

0.2

o 2 3 4

FIGURE 9.1. Transient solution for the empty system and blocked system probabilities
= I and Pbl (0) = O.
POO(I) and Pbl (I), respectively, with POO(O)

> eq[ 1, 0] : =-rnu{ 1] * p[ 1, 0]

+rnu{ 2] * p[ 1, 1] + larnbda* p[ 0, 0] ;

eql,o := -141 PI,O + 142 PI,I + A. po,O

> eq[ 0,1] :=- (larnbda+rnu{ 2] )*p[ 0,1]
+rnu{ 1] * p[ 1,0] +rnu{ 2] * p[ b, 1] ;

eqo,l := -(A. + 142) PO,I + 141 PI,O + 142 Pb,l

> ecj 1,1] :=- (rnu{ 1] +rnu{ 2] )*p[ 1,1]
+ larnbda* p[ 0, 1] ;

eql,l := -(141 + 142) PI,I + A. PO,I

> eq b, 1] : =-rnu{ 2] * p[ b, 1]
+rnu{ 1] * p[ 1, 1] ;
eqb,l := -142 Pb,l + 141 PI,I
> eq[ unity] : =
p[ 0,0] +p[ 1,0] +p[ 0,1] +p[ 1,1] +p[ b, 1] =1;

eqlD/ity := po,o + PI,O + PO,I + PI,I + Pb,l = 1

> sol ve ({ eq 0, 0] ,eq[ 1, 0] ,eq 0, 1] ,
eq 1,1] ,eq[ b, 1] ,eq unity] } ,
{ p[ 0, 0] ,p[ 1, 0] ,p[ 0, 1] ,p[ 1, 1] ,p( b, 1] } ) ;
398 9. Queueing Systems

;.2p,2P" P, 22;'(;'+P,,+P,2) p,,2;.2

{PI, ,= %1 'P',O = %1 ' Pb,' = %1 '

P _ P, 22 P" (p" + P,2) _ (p" + P,2);' P,2 P" }

0,0 - %1 ' PO,' - %1
%1 := 2 p,l;. P" + P, 22 p,,2 + P,2 3 P" + P,22;.2 + P,2 3 ;. + P,2 p,,2;. +;.2 P,2 P"
+;.2p,,2
When the servers have the same service rate, i.e., when p, = P,2, the solution
simplifies and reduces to the fonn that was reported in Gross and Harris [76, p.
225].
> assign(%);
> mu[ 1] : =mu; mu[ 2] : =mu;
P" := P,
P,2 := P,
> normal (p[ 0,0] ); normal (p[ 1,0] ) ;
normal (p[ 0, 1] ); normal (p[ 1, 1] ) ;
normal (p[ b, 1] ) ;
p,2
2 ----=---."...----".
4 P, ;. + 2 p,2 + 3;.2
;. (;. + 2 p,)
4 P, ;. + 2 p,2 + 3;.2

2 P,;'
4 P, ;. + 2 p,2 + 3;.2
;.2

9.4.2 Jackson Networks

Jackson networks [98] are queueing networks with a special structure that con-
sist of k nodes (queues) where the outside customers arrive at node i according
to a Poisson process with mean arrival rate Yi' i = I, ... , k. Each node with an
unlimited waiting capacity may have multiple identical servers where each server
has an exponential service time with parameter P, i' When a customer leaves node
i, it goes to node j with probability Pij independent of that customer's past his-
tory. Thus, one may think of the sequence of nodes visited in a Jackson network
as a discrete-time Markov chain with one absorbing state.
 9.4 Queueing Networks 399

The first step in analyzing a Jackson network is the computation of the total
input rate A; into node i. Before their final departure, the customers may be cycled
between different nodes within the network as a result of, say, rework of an item in
a manufacturing process. Hence, the total input rate into node i may be different
from the arrival rate from outside into node i. Thus, we have

A; = y; + "'2::/jPji
j

since y; is the arrival rate to node i from outside the network and Pj; is the
probability that the customer leaving node j enters node i . (If node i has C; servers
and if p; = A;/(C;jJ.;) < I for all nodes, then Jackson has shown that node i can
be treated as a multiple server Mj Mjc; system [98J.)
This system may be rewritten in vector/matrix form as .oX = 'Y + .oXP and when
solved for .oX gives

where P is the k x k (sub-Markov) switching probability matrix.

Let us now consider a Jackson-type manufacturing process where customers
arrive at any node of the network with three nodes. The arrival rate vector is
'Y = (y I , )' 2, Y3) and the switching matrix is

P= [~I ~12 ~~].

P31 Pl2 0
We use Maple to perform symbolically the necessary computations and find the
total arrival rates at each of the three nodes as a function of model parameters 'Y
andP.
> restart: * JACKSON.MWS
> with (linalg) :
Warning, new definition for norm

Warning, new definition for trace

> gammaVector:=array(1 .. 3,
[ gamma[ 1] ,gamma[ 2] ,gamma[ 3] ] ) ;

gamma Vector := [y I, Y2, Y3]

> PMatrix:=matrix(3,3,[ O,p[ 1,2] ,p[ 1,3],
p[ 2, 1] ,0, p[ 2, 3] ,p[ 3, 1] ,p[ 3, 2] ,0] ) ;

0 PI.2 PI.3]
PMatrix:= [ Pl. I 0 Pl. 3
Pl. I P3.2 0
> IdentityMatrix:= array(identity, 1 .. 3,1 •. 3);
IdentityMatrix := array (identity, 1.. 3, 1.. 3, m
400 9. Queueing Systems

> IminusP:=evalm(IdentityMatrix-PMatrix);

IminusP := [ -~. I I
-PI.2
-Pl. 3
-PI.3]

-P3. 1 -PJ.2 I

> IminusPinv:=inverse(IminusP);

IminusPinv :=
-I +%3 PI.2 + P1,3 P3.2 PI.2 Pl. 3 + PI.3
0/04 0/04 0/04
Pl. I + Pl. 3 P3. 1 -I +%1 Pl. 3 + PI . 3 Pl. I
0/04 0/04 0/04
Pl. I P3.2 + P3.1 P3.2 + P1.2 P3. I -I +%2
0/04 0/04 0/04
%1 := P3.1 PI . 3
%2:= Pl. I PI . 2
%3 := Pl. 3 P3.2
0/04 := -I +%3 +%2 + Pl,I P1,3 P3,2 + P3.1 PI,2 Pl,3 +%1
The total input rate vector ~ is found as follows:

> lambdaVector:=evalm(gammaVector&*IminusPinv);

lambdaVector:= [

)/ I (-I + %3) )/2 (Pl,1 + Pl,3 P3,1) )/3 (Pl. I P3, 2 + PJ.I)
0/04 0/04 0/04
_)/1 (PI,2+PI,3P3 , 2) + )/2(-1 +%1) _ )/3(P3,2+PI.2P3.1)
0/04 0/04 0/04
_)/dPI.2Pl.3+PI,3) _ )/2(Pl.3+PI,3Pl, I) + )/3(-1+%2)]
0/04 0/04 0/04
%1 := P3.1 PI.3
%2:= Pl. I PI,2
%3 := Pl. 3 P3.2
0/04 := -I + %3 + %2 + Pl. I PI. 3 P3,2 + P3. I PI. 2 Pl. 3 + % I
As an example, let us now compute numerically the value of ~ assuming that
"y = (4,2, I) and

0 0 5. 5 0.25]
P = [ 0.3 0 0.65.
0.05 0.1 0

Substituting these values into the formulas for ~ we developed, Maple gives
 9.5 Optimization of Queueing Systems 401

> subs({ gamma[ 1] =4,gamma[ 2] =2,gamma[ 3] =1,

p{ 1,2] =.55,p{ 1,3] =.25,p{ 2,1] =.3,
p{ 2,3] =.65,p{ 3,1] =.05,p{ 3,2] =.1},
evalrn(lambdaVector));
[6.126003073, 6.013317398, 6.440157077]

i.e., AI = 6.126, A2 = 6.013 and A3 = 6.440. These values can now be used
in conjunction with the service rate values Pi to compute the traffic intensities
Pi = A;/(CiPi)·
Assuming that (PbP2, P3) = (10,4,12) and (CI,C2,C3) = (1,2,1), we ob-
tainpl = 0.6126, P2 = 0.751 and P3 = 0.5366. Using these values, the expected
number of customers in the network can be computed as a sum of the individual
Li values for each node. Since the first and third nodes are M / M /1 and the second
node is an M/M/2queue, it can be shown that (LI, L2, L3~ = (1.58,3.45,1.15);
thus the expected number of customers in the system is Li=1 Li = 6.18.

9.5 Optimization of Queueing Systems

10 this chapter we have so far considered the descriptive aspects of queues and de-
veloped expressions for operating characteristics of different queueing processes.
We now turn our attention to an important aspect of queueing theory that deals
with prescriptive (i.e., normative) issues where the objective is to optimally de-
sign and control a queueing system.
Normative queueing models are concerned with finding the optimal system pa-
rameters, e.g., optimal number of servers, mean service rate, queue discipline and
queue capacity. In general, a design model is static in nature in the sense that op-
erating characteristics of a queue are used to superimpose a cost structure in order
to compute the optimal value of a model parameter such as c, P or K. These mod-
els are usually optimized using classical optimization techniques such as calculus
and nonlinear programming [92, Ch. 17]. As a simple example, suppose it is pos-
sible to choose a service rate P > A for an M/ M/1 queue to minimize the cost
C(p) = CI L + c(p) where CI is the waiting cost per customer per unit time and
c(p) is the operating cost per unit time as a function p. Ifwe assume for simplic-
ity that c(p) = C2P, since L = A/(p - A) we obtain C(p) = CIA/(p - A) + C2P
as the cost function to be minimized. Following the standard steps of optimization
we obtain the optimal value of P as follows.
> restart: # mmlrnuopt.rnws
> C:=c{ 1] * lambda/ (rnu-lambda) +c[ 2] *rnu;
CIA
C:=--+C2P
P-A
> diff (C, rnu) ;
402 9. Queueing Systems

> sol:=solve(%,mu);

sol := ~ 2 C2 A. + 2 .JC2CII, ~ 2 C2 A. - 2 .JC2CII

2 C2 2 C2

> expand (sol[ 1] ) ;

Thus, the optimal service rate is obtained as Jl = A. + FtJJCi.

Dynamic control models may involve the determination of a decision variable
as a function of the nwnber of customers in the system or (as in the transportation
queueing example to be discussed) as a function of the customers who are lost
to the system. A dynamic control model may also deal with the determination of
an optimal policy such as "provide no service (tum the server oft) if there are m
or fewer customers in the system; but when the nwnber of customers increases
to M (M > m), tum the server on and continue service until the nwnber in
the system drops to m" (Sobel [179]). The first type of dynamic control models
may be analyzed using calculus. The second type of dynamic control models are
normally analyzed using stochastic dynamic programming techniques.
For a comprehensive bibliography of papers published before 1977 that deal
with the static design and dynamic control of queues, see Crabill, Gross and Mag-
azine [55].

9.5.1 A Transportation Queueing Process

As an example of a queueing design problem, we consider a transportation queue-
ing process studied by Bhat (32). It is asswned that the arrival process of taxis at a
taxi stand follows a Poisson process and the arrival process of customers follows a
renewal process that is independent of the taxi arrivals. This transportation queue-
ing process (an MIGII queue) is controlled by calling extra taxis whenever the
total nwnber of customers lost to the system reaches a certain predetermined nwn-
ber. If w is the predetermined upper limit of the nwnber of lost customers, then
in the system described an optimal choice of the nwnber w becomes essential.
Bhat finds the transient and steady-state behavior of the process by discovering
regeneration points and using renewal theoretic argwnents.
If Q(t) is the nwnber of taxis waiting at time I with Q(O) = i ~ 0, A(I) is the
nwnber of taxis arriving in (0, I] and D(I) is the nwnber of customers arriving
in (0, I], then X(I) = Q(I) + A(I) - D(I) where -X(I) denotes the nwnber of
customers lost at time I .
 9.5 Optimization of Queueing Systems 403

For the general Q(/) process Bhat identifies the renewal periods, and using
renewal arguments he finds that the limiting expected number of taxis waiting is

where Q = lim Q(/),,t is the rate of taxi arrivals and V is the interarrival time
t~oo

of consecutive customers with p = ,t E (V) < I. Next, by using the concept of

forward recurrence time of renewal theory, he obtains the limiting probability of
unsatisfied customers tk, where

3
to= w+2' o <k:::::w-1.
The following Maple commands compute the expected number of lost cus-
tomers in a renewal period (ELos t). Using the expression for the expected num-
ber of taxis waiting (ETaxis) with CI as the cost of making a taxi wait and C2
as the cost of losing a customer, the total cost C to the system is obtained and
optimized to find the optimal value of w:
> restart: # bhat.mws
> Sum(k/(w+2),k=1 •. w-l);
w-I k
~w+2
> ELost:=normal(value(%»;
E'Lo ._.!. w(w - I)
'SI.- 2 w +2
> ETaxis:=3/(w+2)+lambda A 2*EV2
/ (2*rho* (I-rho»;
I 1,t2 EV2
ETaxi'S := 3 w + 2 + 2 p (I _ p)
> C:=c[ 1] *ETaxis+c[ 2] *ELost;

C := CI (3 _1_ +.!.,t2 EV2 ) + .!. C2 w (w - I)

w+2 2p(l-p) 2 w+2
> diff (C, w) ;
-3 CI + .!. C2 (w - I) +.!. C2 w _.!. C2 w (w - I)
(w + 2)2 2 w + 2 2 w +2 2 (w + 2)2
> sol:=solve(%,w);

sol:=.!. -4C2 + 2 J6C2 2 + 6C2 CI , .!. -4C2 - 2 J6C2 2 + 6C2 CI

2 C2 2 C2
> wOpt: =collect (normal (sol[ 1] ) ,c[ 2] ) ;
404 9. Queueing Systems

Thus, the optimal solution is w· = J6(I + CI/C2) - 2. Here we note that as the
cost oflosing a customer approaches infinity (C2 ~ 00), then w· is obtained as a
value close to zero, as expected:
> evalf (limit (wOpt, c{ 2] =infinity) ) ;
.449489743

9.5.2 An MI Mil System with Controlled A"ivals

In dynamic control of some queues, it may be possible to deny admission to en-
tering customers by physically controlling the arrival rate by rejecting every few
customers when the system becomes crowded. In [149], Pliska considers such a
system for an M / M / I queue with arrival rate A and service rate p. It is assumed
!
that arriving customers are turned away with probability whenever k or more
customers are present in the system. Thus, when the number of customers in the
system is k or more, the actual arrival rate is reduced to A/2. In such a case, the
queueing system can be modeled as a general birth-death process with a state-
dependent arrival rate ).n such that

A _( A forn<k (9.13)
n- A/2 for n 2! k .

Using the results presented for the steady-state solution of the birth-death pro-
cesses in Section 9.2.4 with c = I, An as in (9.13) and Pn = P for all n and
assuming that p = A/(2p) < I, we obtain the steady-state probabilities Pn as

(2p)n pO for n = 0, ... , k

{
Pn = 2" pn pO for n = k, k + I, ....

Since L~ Pn = I, the unknown parameter pO is obtained as

Now we suppose that each time a customer is served a reward of r is collected.

but for each customer in the system a waiting cost of c is incurred. The problem is
then to compute the optimal value of k that will maximize the profit per unit time
f(k) given as
f(k) = Or - cL.
 9.5 Optimization of Queueing Systems 405

Here, (} = Jl(1 - pO) is the steady-state processing rate that is equal to the rate at
which customers depart the system and

L = ~np.=p<>[~n(2p)'+2'~np.]
*-1 2* *-1]
= pO [ Ln(2p)n + (1 _ )2 - 2* Lnpn
n=O p p n=O
is the average number of customers in the system which, according to Pliska, "is
not a pretty expression but is straightforward to compute."
Since f is not known to be necessarily concave with respect to k, Pliska sug-
gests an exhaustive search procedure to find the optimal value of k. The develop-
ment of the model and the solution of an example with A. = 16, Jl = 14, r = 10,
e = 3 using an exhaustive search is presented in the following Maple worksheet.
> restart: # pliska.rnws
> p{ 0] :=1/(' sum' ((2*rho)"n,n=0 •. k-l)
+(2*rho)"k/(l-rho))i

*
pO:=

n=O
(i: 1
(2p)n) + (2p)*
1- p
> L:='p{ 0]'* (' sum' (n* (2*rho)"n,n=0 •• k-l)
+2"k*rho/(l-rho)"2
-2"k*' sum' (n*rho"n,n=O .• k-l)) i
*-1 2* *-1
L :=pO«Ln(2p)n)+ (1_~)2 -2* (Lnpn»
n=O n=O
> theta:=rnu*' (l-P{ 0] )' i
(} := Jl (1 - pO)
> f: =' r* theta -c* L' i
f :=r(} -e L
> larnbda:=l6i rnu:=l4i r:=lOi
C:=3i rho:=(larnbda/2)/rnui
A. := 16
Jl := 14
r:= 10
e :=3
4
p '--
'-7
> Digits:=6i
Digits:= 6
406 9. Queueing Systems

> for k from 0 to 10 do

print(k,evalf(f)) od;
0,76.
1, 96.7273
2, 106.495
3, 111.647
4, 114.402
5, 115.743
6, 116.168
7, 115.961
8, 115.294
9, 114.278
10, 112.989
Thus, the optimal solution is k* = 6 with a maximum expected profit of 116.168.
It is worth noting here that if p > ! (as in this example), then L -+ 00 as
k -+ 00 and thus a finite value of (k* = 6) maximizes the objective ftmction. The
reason for this policy is as follows: If the traffic intensity is still high even with
controls, it is best to start turning away some of the customers as soon the system
!,
size reaches some critical but finite value. However, if p < then L is bounded
by 2p / (1 - 2p), in which case the optimal value of k may be very large.

9.5.3 Optimal Dynamic Service Rate Control

We now describe an application of deterministic optimal control theory to find the
optimal time-dependent service rate p(/) in an S-server, finite-capacity (N units),
Markovian queue M(/) / M(/) / S / N with nonhomogeneous (Le., time-dependent)
Poisson arrivals. As discussed in Parlar [141], for this process there are N + 1
Kolmogorov differential equations representing the time-dependent probabilities
Xn(/), n = 0, . .. , N , and the service rate U(/) is the control ftmction that must
be found to minimize a cost ftmctional. [In this section we will use a notation
that is standard in optimal control theory and denote the states Pn(/) by xn(/)
and the service rate control ftmction p(/) by U(/).] The objective ftmctional to be
minimized includes the cost of waiting customers plus the cost of service over a
specified time interval [0, T] and a final time penalty cost of deviations from an
expected queue length.
We assume that the customers arrive according to a Poisson process with time-
dependent mean arrival rate A(/) and the service facility has S servers each with
an exponential service time of rate u(/) and a maximum of N customers can
be in the system. Aqueueing model fitting this description has been applied to
forest fire fighting by Bookbinder and Martell [35], landing clearance of aircraft
 9.5 Optimization of Queueing Systems 407

by Koopman [113] and scheduling police patrol cars in New York City by Kolesar
et al. [112]. As described in Gross and Harris [76], the state probabilities for this
system are obtained from the Kolmogorov equations (9.1 }-{9.2) of the birth-death
processes as follows: For n = 0,

xO(I) = -A,(I)Xo(l) + U(I)Xl (I),

for n = 1, ... , S - 1,

Xn(l) = A, (I)xn-l (I) - [A,(I) + nU(I)]xn(l) + (n + I)u(I)Xn+l (I),

for n = S, ... , N - 1,

Xn(l) = A,(I)xn-l (I) - [A,(I) + SU(I)]xn(l) + Su (I)Xn +I (I),

and finally, for n = N,
Xn(l) = A,(I)Xn-l (I) - SU(I)XN(I).

For this system, the initial condition is 'L:=o Xn (0) = 1. Also, since Xn (I) are
probabilities, we also have the point constraints 'L:=oXn(l) = 1, xn(l) ~ 0, n =
°
0, ... , N for all 1 E [0, T]. However, if U(I) ~ then the point constraints are sat-
isfied automatically; see Klein and Gruver [109]. Defining P'(I) = [Xo(I), ... ,
Xn(I)] where prime denotes transposition, we can write the system more com-
pactly as P(I) = u(I)F P(I) + A, (I)G P(I) with the constant matrices F and G
having dimension (N + 1) x (N + 1). (For details, see Parlar (141).)
A performance criterion incorporating the cost of waiting customers (for exam-
ple, fires burning in Bookbinder and Martell's forestry model or aircraft awaiting
landing clearance in Koopman's model) and the cost of service rate over [0, T]
seems to be a reasonable objective for M(I)/M(I)/S/N queues. Since the state
variable xn(t) is the probability of n customers in the system at time I, and since
the system capacity is N units, the expected number of customers L(I) at time 1
is
N
L(I) = LnXn(l) = C' P(I)
n=O

where C' = (0, 1, 2, ... ,N). We now introduce a service rate cost function
q[u(I)] and add a desired (target) expected queue size at the final time T and pe-
nalize deviations from this level. Thus, the objective functional to be minimized
with a proper choice of U(I) is

J = loT [C' P(I) + q(u)] dl + !b[C' P(T) - k]2

where k is the desired end-of-day expected queue length and b gives the relative

°
cost of deviating from desired k. Although each state variable xn(l) is restricted
to take values between and 1, it is not necessary to introduce these constraints
408 9. Queueing Systems

explicitly into our fonnulation since they will be automatically satisfied-as ex-
plained above-if u(t) ~ 0, for t e [0, T). Since it may be physically impossible
u
to exceed some upper bound on the control, this implies that we must add to
the fonnulation constraints on the control function in the fonn 0 ::: u(t) ::: u,
t e [0, T]. Now the optimal control problem is given as

10r [C' pet) + 0" (u)] dt + !b[C' peT) -

T
min J = k]2
"(I)

subject to
Pet) = u(t)F pet) + ).(t)G pet)
e' P(O) = I, e' = (I, I, ... , I)
o ::: U(I) ::: u.
Introducing a new (N + 2)nd state variable XN+l as XN+l (t) J~[C' per) + =
dr so that XN+l (t) = C' pet) + O"(u) and XN+l (0) = 0, the problem can
0" (u)]
be refonnulated in Mayer fonn [162, p. 28] as follows:

min J
U(I)
= !b[C' peT) - k]2 + XN+l (T)

subject to
= u(I)F pet) + ).(I)G pet), e' P(O) = I
P(I)
XN+l (I) = C' P(I) + 0" (u), XN+l (0) = 0
o ::: u(t) ::: U.
Using Pontryagin's maximum principle [169], it can be shown that [141] to find
the optimal service rate u(t) in an M(I)/M(I)/S/N queue one has to solve the
two-point boundary-value problem (TPBVP) given by the following differential
and algebraic equations:

Wet) = -[u(t)F'W(I)+).(I)G'W(I)+C], WeT) = bC[C' P(T)-k] (9.14)

p(t) = u(I)F pet) + ).(I)G P(I), e' P(O) = I, (9.15)

XN+l (I) = C' pet) + 0" (u), XN+l (0) = 0 (9.16)
0, ifg(-P'(t)FW(I» <0
u(l) = { g(-P'(/)FW(/», if 0 ::: g(-P'(/)FW(/» ::: u (9.17)
u, ifg(-P'(/)FW(/» > u
where g(Z) is the inverse function of dO" /du evaluated at Z. This TPBVP has
2(N + I) + I differential equations where the first N + I for W(/) have known
final time conditions and the last N + 2 for [P(I), XN+l (I)] have known initial
time conditions. Also, Wet), p(t) and XN+l (I) all depend on U(I), which in turn
depends on the values of W(I) and pet). Thus, like many optimal control prob-
lems arising in engineering applications, this one also requires the application of
a numerical technique for its solution.
 9.5 Optimization of Queueing Systems 409

As an example consider now the case with N = S = 1. Letting q(u) = !au2,

u = 1 and noting that Xo(/) + XI(/) = I, the conditions (9.14}-(9.17) reduce to
the following TPBVP with three differential equations and one algebraic relation:

WI (t) = WI (/)[).(/) + U(/)] - I, WI (T) = b[XI (T) - k]

Xl (t) = ).(/)[1 - Xl (t)] - U(I)XI (I), Xl (0) = XIO

I
X2(/) = XI (I) + !a[U(/)]2, X2(0) =0
0, if WI (/)XI (I)/a < 0
U(/) = WI (t)XI (t)/a, if 0 :s WI (t)XI (t)/a :s 1
1, if WI (t)XI (t)/a > 1.

For the time-dependent arrival rate we choose ).(t) = 20t 2(1 - t)3. Note that as
t ~ 0 or t ~ 1 the arrival rate approaches zero. The values of other parameters
are given as a = 0.4, b = 2.5, k = 0.1, T = 1, Xl (0) = 0.8 and X2(0) = 0.0.
We now solve this TPBVP numerically using Maple's piecewise () func-
tion and its knowledge of numerical solution of differential equations.
> restart: # mmsncont.rnws
We first define u(t) as a piecewise function in terms of the unknown functions
and Xl (t):
WI (t)
> u: =unapply (piecewise (w{ 1] (t) * x[ 1] (t) / a<O,
O,w{ 1] (t)*x[ 1] (t)/a>=O and w{ 1] (t)*x[ 1] (t)/a<=l,
w{ 1] (t) * x[ 1] (t) / a, w{ 1] (t) * x[ 1] (t) / a> 1, 1) , t) ;

• • (WI (I) XI (I)

U := t ~ Piecewise < 0, 0,
a
WI (t) XI (t) :s 0 and WI (t) XI (t) _ 1 :s 0, WI (I) XI (t), 1 < WI (I) XI (t) ,
a a a a
I)
Next, the system of differential equations is defined:
> adjoint[ 1] :=diff(w{ 1] (t) ,t)
=w{ 1] (t)* (lambda (t) +u (t)) -1;

<><!ioint I := * WI (t) = WI (t) (l(t l+ ({ ~I -%1< :s0 0 and %1 - 1 :s 0J.~ - 1

%1
1 < %1
%1 := WI (t) XI (t)
a
> stater 1] :=diff (x[ 1] (t), t)
=lambda(t)* (I-x[ 1] (t) )-u(t)*x[ 1] (t);

statel:= f, XI (t) = l(t)(1 - XI (I)) - ({ ~I %1 < 0

-%1 :s 0 and %1 - I :s 0
)
Xl (t)
1<%1
%1 := WI(I)XI(t)
a
410 9. Queueing Systems

> state[ 2] :=diff(x[ 2] (t) ,t)

=x[ 1] (t)+(1/2)*a*u(t)"2;

state, :~ £-'2(t) =X1(t) + ~a ({ ~l

%1 < 0
-%1 :5 Oand%I-1 :5 0
)2
1 < %1
%1 := WI(I)XI(I)
a
> desystem:=adjoint[ 1] I state[ 1] I

state[ 2] ;

desyslem := -I; WI (I) = WI (I) 0.(1) + 0/02) - I,

-I; XI (I) = l(l) (I - XI (I» - 0/02 XI (I), -I; X2(1) = XI (I) + 2"1 a 0/022
%1 := WI(I)XI(I)
a
o %1<0
0/02:= { %1-%1:5 0 and %1 - 1 :5 0
1 1 < %1
> funes:={ wi: 1] (t) I x[ 1] (t) I x[ 2] (t)} ;
Junes := (X2(1), WI (I), XI (I)}
Our TPBVP where WI (T) = b[xl (T) - kJ is nonnally solved using the "initial
value shooting method," which starts with a guess for the value of WI (0), say
WIO. With this method, using the initial conditions WI (0) = WIO, XI (0) = 0.8 and
X2(0) = 0, the differential equation system is solved numerically. (Maple con-
veniently incorporates the piecewise algebraic relation for U(I) in the numerical
solution of the system.) If at T we observe that WI (T) = b[xl (T) - k J, then the
solution of the TPBVP has been found; otherwise the initial guess is modified
and the system is re-solved. For a detailed description of shooting methods see
Roberts and Shipman [155J. Sethi and Thompson [169, Ch. IIJ provide a sum-
mary description of the initial value shooting method for problems that require
guessing for a single state variable (which is the case in this example).
In our problem, we manually implemented the initial value shooting method
and found the value of WI (0) as .7475 for which the final time difference is
WI(T) - b[xl(T) - kJ = .00074165-an acceptable error. With the definition
of l(I), the system is solved numerically using the output=listproeedure
option so that the resulting numerical trajectory can be used in plotting the re-
quired graphs.
> init : =w[ 1] (0) =. 7475 I x[ 1] (0) = . 8 I
x[ 2] (0) =0;
inil := WI (0) = .7475, XI (0) = .8, X2(0) = 0
> lambda:=t->20*t"2* (1-t)"3;
a:=.4; b:=2.5; k:=.1; T:=1;
 9.5 Optimization of Queueing Systems 411

A. := 1-4201 2 (I - 1)3
a:= .4
b:= 2.5
k:= .1
T:= I
> Sol:=dsolve({desystern,init} ,funes,
type=nurnerie,output=listproeedure);

Sol := [I = (proc(l) '" end), X2(1) = (proc(l) ... end),

WI (I) = (proc(l) ... end), XI (I) = (proc(l) .. . end)]
> wSol[ 1] : =subs (Sol, wi: 1] (t));
xSol[ 1] : =subs (Sol, x[ 1] (t));
xSol[ 2] : =subs (Sol, x[ 2] (t));
wSoIt := proc(l) ... end
xSo/1 := proc(l) ... end
xSoh := proc(l) ... end
> init; wSol[ 1] (T) -b* (xSol[ 1] (T) -k) ;
WI (0) = .7475, XI (0) = .8, X2(0) = 0
.0007416587
The trajectories for the functions WI (I) and X2(1) for I E [0, T] are presented in
Figure 9.2. Note here that WI (0) = .7475 and X2(0) = o.

0.8

0.6

0.4

0.2

o 0.2 0.4 0.6

FIGURE 9.2. Optimal trajectories of WI (I) and X2(t) for 1 e [0, IJ with WI (0) = 0.7475
and X2 (0) = o.
412 9. Queueing Systems

> with (plots) :

> odeplot(Sol,[[ t,w( 1) (t)),[ t,x( 2) (t))),
O•• T, view=[ O.. T, O.. T) , color=black) ;
The other trajectories of importance are those that correspond to the control func-
tion u(t) and the states [xo(t), XI (t», which are plotted in Figure 9.3 where
u(O) = 1, xo(O) = 0.2 and XI (0) = 0.8.
> x( 0) (t) :=1-x( 1) (t);
xo(t) := 1 - XI (t)
> odeplot(Sol,[[ t,u(t)),
[ t, x( 0) (t)) , [ t, x( 1) (t))) , 0 •• T ,
view=[ o.. T, o.. T) , color=black) ;

0.2 0.4 0.6 0.8

FIGURE 9.3. Optimal trajectories of U(/), XO(/) and XI (I) for 1 E [0, I] with u(O) = I,
xo(O) = 0.2 and XI (0) = 0.8.

Finally, since J = !b[xl (T) - k]2 +x2(T), this value of the objective functional
is computed as follows:
> J:=(1/2)*b* (xSol[ 1) (T)-k) "2+xSol[ 2) (T);
J := .8562668471

We thus see that an otherwise difficult-to-solve dynamic optimization problem

can be analyzed relatively easily using Maple's dsol ve () and piecewise ( )
functions.8

8For details of an alternative solution method for this control problem, the "Newton-Raphson
boundary condition iteration,» see Parlar [141].
 9.6 Summary 413

9.6 Summary
The main goal of this chapter was to present a survey of Markovian queueing
systems.9 The limiting distribution and the related operating characteristics of
different types of Markovian queues were discussed. Using Maple's ordinary and
partial differential equation solvers [i.e., dsol ve () and pdsol ve ( )] we were
able to find the transient solutions for a finite capacity queue and for the am-
ple server queue. Next, a serial queue and Jackson networks were analyzed. The
chapter ended with a detailed discussion of optimization of some types of queue-
ing systems. In particular, Maple's ability to solve differential equations defined
by piecewise functions was helpful in determining the optimal control (i.e., ser-
vice rate) trajectory in a time-dependent queueing system.

9.7 Exercises
I. Yule process is a pure birth process with state-dependent birth rate An = nA,
n = 0, I, .... This process represents the growth of a population where each
member gives birth at an exponential rate A. Let X(t) denote the population
size at time t and assume that the population starts with a single member,
i.e., X(O) = I. Find the transient solution Pn(t) = Pr[X(t) = n] for this
process.

2. Consider a queueing system where both arrival and service rates are state
dependent. Assume that the arrival rate An decreases and service rate Jl.n
increases as the number n in the system increases, i.e.,

An = { (N - n)A, n:::: N
0, n> N,

_ (n p , n:::: N
Pn - 0, n> N.
What are the differential equations for the probability Pn(t) = Pr(n in the
system at time t)?

3. Consider a Markovian queue with two servers where the arrival and service
rates are given as

An = A, n ~ 0
= (p, n=1
Jl.n 2p, n ~ 2.

Find the steady-state probabilities pn = limHoo Pn(t).

9An example ofa non-Markovian queue (M/G/I) was presented in Section 7.4.4 of Chapter 7
where the queue was analyzed using the imbedded Markov chain technique.
414 9. Queueing Systems

4. The MI Mlclc model: Consider a c-server Markovian queue with a capac-

ity of c spaces. When all servers are busy an arriving customer leaves the
system without waiting for service. Thus, the arrival and service rates are
An = A,J.ln = nJ.l forn = 0,1,2, ... ,c-1 and An = O,J.ln = CJ.l
for n ::: c. Show that the steady-state probability Pc(P) that an arriving
customer is lost to the system is given by
pClc!
Pc(P) = Lk=opklk!

where p = AI J.l. Note that Pc(P) is a ftmction of both the traffic intensity p
and the number of servers c.
5. The transient solution Pn(t), t ::: 0, n = 1,2, . .. for the MI Mil queue
with an initially empty system is given as

Pn(t) = e-(A+Jl)t {pn/2In(at) + p<n-I)/2In+1 (at)

+(1- p)pn f
k=n+2
p-k/2!t(at)!

where In(x) is the modified Bessel ftmction of the first kind, p = AI J.l and

and J.l = 2 and plot the transient probability trajectory for

n=I,2,3.
°
a = 2,.fJ:;; see Medhi [130, p. 120] and Saaty [161, p. 340]. Let A = I
~ t ~ 10 and

HINT: In Maple the modified Bessel ftmction of the first kind is represented
by Bessell.
6. Let Wn, n ::: I be the waiting time of the nth customer in a GIGII queue.
Show that Wn+1 = max(O, Wn + Vn - Un) where Vn is the service time
of the nth customer and Un is the interarrival time between the nth and
(n + I)st customer.

7. Consider a closed queueing network with three nodes and two customers.
Assume that a customer chooses the next node which has the fewest num-
ber of customers and ties are broken by flipping an unbiased coin. Departure
rate from node j is J.l j' j = I, 2, 3. The state space of the stochastic process
representing this queue can be described by a vector with three components
(nl, n2, n3). Write down the infinitesimal generator for this stochastic pro-
cess and compute the steady state probabilities.
8. Consider the MI Mlclc model presented in Exercise 4. Since Pc(P) is the
probability that an arriving customer is lost to the system, the effective ar-
rival rate is Aetf = [I - Pc(P»)A. If each arriving customer generates an
average ofr dollars of profit and each server costs kJ.l dollars per hour, the
average profit rate can be written as A = r..tetf - ckJ.l. Let p = 1 (i.e.,
 9.7 Exercises 415

A. = J.l) and determine whether it is preferable to have two servers instead

of one.
HINT: The results should be found in terms of the parameters rand k.
10
Simulation

10.1 Introduction
Simulation I is a computer-based probabilistic modeling tool that is used to imitate
the behavior of an existing or a proposed system. If an existing system such as a
bank branch is experiencing long queues and long customer waiting times, the
manager of the branch may use simulation to examine different alternatives for
improving service without actually making any physical changes in the system.
For example, the effect of increasing the number of available tellers and/or
changing the queue design (e.g., combining parallel queues into a single queue) on
the average queue length and the average waiting times can be examined without
physically altering the system. Behavior of the proposed system can be observed
by running computer simulations of different alternatives over time and measuring
the operating characteristics of the system.
Of course, in some cases the system under study may be simple enough so
that the queueing models that were presented in Chapter 9 may be used to model
it. But, if the random variables representing the arrival and/or service processes
are not exponential (thus making the system non-Markovian) and if the proposed
design is complex (e.g., the bank branch may be a complicated queueing network)
none of the available queueing models may be useful in modeling the problem.
In those cases, simulation becomes a necessity in order to model and analyze the
system.

I "Simulation" is derived from the Latin word simulare which means "10 copy, imitate or represent."

M. Parlar, Interactive Operations Research with Maple

© Birkhäuser Boston 2000
418 10. Simulation

Along with linear programming and statistical methods, simulation has become
one of the most important operations research techniques used to solve practical
problems.2 There are now several very powerful and flexible simulation software
programs-such as Arena [104] and Visual SLAM [I 53]-that simplifY the mod-
eling and simulation of almost any real-life system that exhibits probabilistic be-
havior.
The most basic computational ingredient in simulation involves the generation
of random variables distributed uniformly between 0 and I. These are then used
to generate other random variables-such as the Weibull or Erlang random vari-
ables that may represent a teller's service time-and the simulation is run long
enough to obtain an estimate of the operating characteristics of the system, e.g.,
average waiting time of customers and the percentage of time the fucility is idle.
Since the result of each run is a random sample, a simulation study is a statistical
experiment that must be conducted using statistical tools such as point estimation,
confidence intervals, hypothesis testing. For a discussion of these important sta-
tistical issues, the reader should consult specialized simulation textbooks such as
Law and Kelton [118, Chs. 8-9] and Banks, Carson and Nelson [12, Ch. 12].
Clearly, Maple is not designed to be used as a full-fledged simulation language.
However, Maple's ability to generate random variates from nearly 20 continuous
and discrete random variables (such as the exponential, gamma, Weibull, normal,
binomial and Poisson) makes it a useful teaching tool for presenting many inter-
esting simulation examples.

Example 97 Port operations. As a motivating example, consider the day-t~day

operations of a port that is used to unload tankers containing imported goods. The
tankers arrive very early in the morning and with the available workforce, it is
possible to unload a maximum of four tankers in one day. Based on historical
data, the number of tankers arriving, X, is binomially distributed with parameters
n = 5 and p = 0.7, i.e., Pr(X = k) = a)(O.7)k(O.3)5-k, k = 0, 1, ... ,5.
Since the workers can unload a maximum of four tankers per day, any tankers
still unloaded must wait until the next day for unloading. The problem is to find
the long-run average number of tankers delayed and propose improvements if
necessary.
This appears to be a queueing problem with a random number of tankers arriv-
ing with an average arrival rate of A. = np = 3.5 per day and a maximum service
rate of m = 4 tankers per day. However, since arrival events take place at discrete
time (i.e., during early mornings) and the number of arriving tankers is binomial,
none of the queueing models presented in Chapter 9, Queueing Systems, seems
to be suitable for this problem. Thus, we decide to simulate this process to find
out the long-run average number of tankers delayed.

2See the paper by Lane, Mansour and Harpell [116] that describes the results of a survey of ed-
ucators and practitioners who teach and use operations research techniques. The survey found that
the three techniques mentioned in this paragraph were consistently rated by the OR educators and
practitioners as the most important
 10.1 Introduction 419

In this case, it is not too difficult to see that if we define X t as the total number
of delayed tankers at the beginning of day t, then we can write

Xt+1 =Xt + At - min(4, X t + At). t = 1,2, ...

where XI = XI is the number of tankers waiting when simulation starts at the
beginning of day 1 and At is the number of tankers arriving on day t. This equation
follows since X t + At is the total number of tankers that need to be unloaded on
day t, and since the workers cannot unload more than four tankers per day, the
number of tankers delayed today t+ 1 is thedifferenceXt+A t -min(4, Xt+A t )·
In this problem, even though we have a mathematical model of the process,
it may not be easy to find analytical expressions for the average number of de-
layed tankers. The fact that the arrivals At are binomially distributed and the state
equations involve the mine .) operator makes this a nontrivial problem.)
The following Maple worksheet simulates this problem for 25 days, computes
the average number of delayed tankers and displays the histogram plot of the
distribution of the number delayed.
> restart: # Tankers.mws
> with(stats): #readlib(randomize): randomize():
We first set the values of m = 4 and the number of days of simulation T = 25.
> m:=4; T:=25;
m :=4
T :=25
The binomial arrival process has parameters (n, p) = (5,0.7).
> n:=5; p:=.7;
n :=5
p:=.7
> pb:=(n,k)->binomial(n,k)*pAk* (l-p)A(n-k);
pb:= (n, k) -+ binomial(n, k) pk (l _ p)(n-k)
We compute the binomial probabilities p(k), k = 1, ... , n = 5 and check to
see that Lk=1
p(k) = 1.
> seq([ k,pb(n,k)] ,k=O .. n); sum(pb(n,k),k=O .. n);
[0, .00243], [1, .02835], [2, .13230], [3, .30870], [4, .36015], [5, .16807]
1.
How would we go about simulating the binomial random variable X with pa-
rameters (n, p) = (5, 0.7)? Since the density function p(k) of X is available, we

3 Actually, it is also possible to model this problem as a discrete-time Matkov chain. Once the tran-
sition probability matrix of the chain is identified, the steady-state distribution of XI can be computed
numerically.
420 10. Simulation

could first compute the cumulative distribution function P(k) = L:=O p(i) of X
as given in the following table.

k 0 2 3 4 5
p(k) 1.00243 .02835 .1323 .3087 .3603 .1681
P(k) .00243 .03078 .1631 .4718 .8319 1.0

Next, generating a uniform random number U between 0 and 1, we could sim-

ulate the binomial random variable X using the fonnula

0, if 0.00000 < U < 0.00243

1, if 0.00243 ::; U < 0.03078
2, if 0.03078 ::; U < 0.16310
X= 3, if 0.16310 ::; U < 0.47180
4, if 0.47180 ::; U < 0.83190
5, if 0.83190 ::; U < 1.00000.

For example, if we happen to generate a unifonn random number U = 0.45134,

then this would correspond to a simulated value of X = 3 tankers arriving.
This method is known as the discrete inverse transform method since, after
generating a U, we find the value of X = k by finding the interval [P(k-I), P(k»
in which U Iies--that is, we find the inverse of p(U).4
Fortunately, Maple users do not have to be concerned about the mechanics of
this method since it has already been implemented in Maple. Abinomial random
variate for the number of tankers arriving, a, can be generated easily using the
randon( binomiald] command that implements essentially same the proce-
dure we have just described.
> a : =randon( binomial<={ n, p)) (1);
a :=3.0
We assume that at the start of day I, there are XI = 2 tankers waiting to be
unloaded.
> x[ 1) :=2;
XI :=2
> for t from 1 to T do
a[ t) : =randon( binomiald[ n, p)) (1);
x[ t+1) : =x[ t) +a[ t) -min (m, x[ t) +a[ t] ) ;

> od:
> seq([ t,l x[ t) ,a[ t)) ,l x[ t+1))) ,t=1. .T);

4There are more efficient methods for generating binomial random variates; see, for example, Ross
[160, pp. SO-51).
 10.1 Introduction 421

[1, [2, 3.0], [1.0)), [2, [1.0, 3.0], [0)), [3, [0, 4.0], [0)), [4, [0, 4.0], [0)),
[5, [0, 4.0], [0)), [6, [0, 2.0], [0)), [7, [0, 4.0], [0)), [8, [0, 4.0], [0)),
[9, [0, 4.0], [0]], [10, [0, 3.0], [0)), [II, [0, 3.0], [0)),
[12, [0, 4.0], [0)), [13, [0, 2.0], [0)), [14, [0, 2.0], [0]],
[15, [0, 5.0], [1.0)), [16, [l.0, 4.0], [1.0)), [17, [1.0,3.0], [0)),
[18, [0, 4.0], [0)), [19, [0, 5.0], [1.0)), [20, [1.0, 5.0], [2.0)),
[21, [2.0,2.0], [0)), [22, [0, 2.0], [0)), [23, [0, 3.0], [0)),
[24, [0, 4.0], [0]], [25, [0, 3.0], [0))
The simulated number of delayed tankers for T days is as follows.
> Delayed:={ seq(x[ t] ,t=1..T)];

Delayed := [2, 1.0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1.0, 1.0, 0, 0, 1.0, 2.0,

0, 0, 0, 0]
The average of the simulated values over 25 days is found as 0.32 tankers.
> describe[ mean] (Delayed);
.3200000000
Finally, we plot the histogram of the number of delayed tankers in Figure 10.1.
> Ranges: ={ O.. 1,1. .2,2 .. 3, 3 .. 4,4 •• 5,5 .. 6, 6 .• 7] ;
Ranges := [0 .. 1, 1..2, 2 ..3, 3..4, 4 ..5, 5.. 6, 6. 7]
.
> with(transform):
> List:=tallyinto(Delayed,Ranges);

List := [Weight(O.. I, 19), Weight(1..2, 4), Weight(2 .. 3, 2),

Weight(3 ..4, 0), Weight(4 ..5, 0), Weight(5 .. 6, 0),
Weight(6 .. 7, 0)]
> with(statplots):
> histogram(List);
Naturally, running the simulation for a "run length" of only T = 25 days and
only once--"number ofruns"-would not provide a very accurate estimate of the
average number of delayed tankers. It is important to emphasize that the proper
choice of the run length and the number of runs is a crucial factor in understanding
the system behavior.
Since the result of each run is a statistical sample, it is advisable to have several
runs of sufficient length to obtain accurate results from the simulation study. In
fact, there are procedures for determining the correct number of runs, R, in order
to obtain a specified precision to estimate the performance measures of the system
under study. A rough estimate for determining R is as follows:
A (I - a)% confidence interval for the true mean Jl of a performance measure
is given by
Pr(X - h s. Jl S. X + h) = I-a.
422 10. Simulation

FIGURE 10.1. Distribution of delayed tankers obtained by simulating the process for a run
length of T = 25 days.

Here, X is the sample mean obtained from an initial sample of, say, Ro runs and
h is the half-width of the desired confidence interval

h _ Za/2 S (10.1)
- ,JR'
In equation (10.1), s is the sample standard deviation obtained from the initial
runs and (for a large R) Za/2 is the Z-score, which bounds a right tail equal to
a /2. If we want to choose R such that h :s t5, then

R ~ ea~2S)2 (10.2)

is the minimum number of runs that should be conducted.

For this example, we increase the value of the run length T to 1000 days and
simulate the system for Ro = 5 times. This gave 0.518, 0.383, 0.445, 0.513 and
0.458 as the average number of delayed tankers. Taking the average of these five
samples gives X = 0.463.
> restart: *
NOR.mws
> with(stats,describe,statevalf);
[describe, statevalj]
> Means:={ .518, .383, .445, .513, .458] ;
~ 0] :=nops(Means);

Means := [.518, .383, .445, .513, .458]

Ro :=5
> XBar: =describe[ mean] (Means);
XBar := .4634000000
 10.2 Generating (Pseudo-) Random Numbers 423

We choose ~ = 0.01, a = 0.05 and find that z = 1.9599. The sample standard
deviation is s = 0.0495. Using (10.2) gives R ~ 94 runs.
> delta:=.Ol;
.01 ~:=
> s:=describe[ standarddeviation] (Means);
s := .04954432359
> alpha:=.05;
a:= .05
> z:=stateval~ icdf,normal~ (1-alpha/2);
z := 1.959963985
> R:=(s*z/delta)A2;
R := 94.29398485

For a careful discussion of determining the number of runs, see Banks, Carson
and Nelson [12, Sec. 12.4] and Law and Kelton [118, Chapter 8].
We continue this chapter with a discussion of the techniques for generating
(pseudo-) random numbers and the "quality" of Maple's unifonn random number
generator. This is followed by some examples of sIalic Monte Carlo simulation
where time plays no role. The chapter ends with examples of dynamic simulation
models that represent systems evolving over time (such as the port operations in
Example 97).

10.2 Generating (Pseudo-) Random Numbers

As we indicated, the most basic computational ingredient in a simulation study is
the generation of unifonn random variables U distributed between 0 and I with
the probability density function

f(u) = { I, O::s U ~ I
0, Otherwise.

It is easy to show that this r.v. has mean £(U) = !, second moment £(U 2) = l,
!
third moment £(U 3 ) = and variance Var(U) = -b.
> restart: # Uniform.mws
> f:=u->l;
f:= I
> EU:=int(u*f(u),u=O .. l);
I
£U:=-
2
424 10. Simulation

> EU2:=int(u A 2*f(u),u=O .. 1);

1
EU2:= -
3
> EU3:=int(u A 3*f(u),u=O .. 1);
1
EU3:= -
4
> VarU:=int(u A 2*f(u),u=O .. 1)-EU A 2;
1
VarU:= 12
We now discuss a widely used method for generating uniform random numbers.

10.2.1 Mixed-Congruential Method

A sequence of random numbers generated by a computer is not truly random
since the computer uses a deterministic equation rn+1 = f(rn) to generate the
(n + I)st random number rn+1 from the nth random number rn. For this reason,
the computer-generated random numbers are ''pseudo-random'' numbers that only
give the "appearance" of randomness.
There are several methods to generate random numbers and the most widely
used among these is the mixed-congruential method. This method starts with an
initial seed value zo and then recursively computes a sequence of integers Zn+ I,
n ~ 0 using
Zn+1 = (az n +b) modm, n=0,1,2, ...
where a (the multiplier), b (the increment) and m (the modulus) are given non-
negative integers. Note here that to find Zn+l, we divide (azn + b) by m and the
remainder is taken as the value of Zn+ I. For example, if a = 5, b = 1, m = 16
and zo = 4, then

ZI = (5 x 4 + 1) mod 16
= 21 mod 16
= 5

and

Z2 = (5 x 5 + 1) mod 16
= 26 mod 16
= 10,

etc. Since each Zn value is an element of the set to, 1, ... , m - I}, the (n + I)st
random number rn+1 that is in the interval [0, I) is computed as rn+1 = Zn+l/m,
n = 0,1, ... For example, using the above data, we would find rl = zl/m =
5/16 = .313 and~ = Z2/m = 10/16 = .625.
 10.2 Generating (Pseudo-) Random Numbers 425

Such methods produce random numbers that always start cycling after some
finite number p (called the period of the generator) not exceeding m. Since the
selection of the parameters zo, a, b and m affects the cycle length, it is important
to choose these parameters carefully in designing random number generators.

Example 98 Random numbers and cycle length. As a simple example of early

cycling (after generating the 16th random number), consider the case with a = 5,
b = I, m = 16 andzo = 4.

> restart: * Congruential.mws

> Digits:=3;
Digits:= 3

> a:=5; b:=l; m:=16; ~ m :=4;

a :=5
b:= I
m:=16
zo :=4
Generating the Zn values for n = I, ... , 25 suggests that the random numbers
r n will start cycling after the 16th one.

> for n from 0 to 25 do z[ n+l] :=(a*z[ n] +b) mod m;

r[ n+l] :=z[ n+l] 1m od:

> seq(~ ~ ,n=1 .. 25);

5, 10, 3, 0, I, 6, 15, 12, 13, 2, 11, 8, 9, 14, 7, 4, 5, 10, 3,0, 1, 6, 15, 12, 13
Indeed, we see that the 17th random number is the same as the 1st, the 18th is
the same as the 2nd, etc.

> seq([ n,evalf(r[ n] )] ,n=1. .25);

[1, .313], [2, .625], [3, .188], [4,0], [5, .0625], [6, .375], [7, .938],
[8, .750], [9, .813], [10, .125], [11, .688], [12, .500], [13, .563],
[14, .875], [15, .438], [16, .250], [17, .313], [18, .625], [19, .188],
[20, 0], [21, .0625], [22, .375], [23, .938], [24, .750], [25, .813]
It has been shown by Fishman [66] that when c t- 0, for a B-bit word computer
the maximal (full) period of Pmax = m = 2 8 can be achieved provided that b is
relatively prime to m (in other words, I is the only positive integer that exactly
divides both b and m) and a = I + 4k where k is an integer. When c = 0, for a
32-bit word computer a satisfactory choice for the modulus and the multiplier are
m = 231 - I and a = 75 . These values have been tested and used extensively;
see, Banks, Carson and Nelson [12, p. 295].
426 10. Simulation

10.2.2 Maple's Uniform Random Number Generator

It is easy to generate uniformly distributed random variables using Maple. Af-
ter loading the random subpackage of the stats package, entering random-
[ uniform] (k) generates k uniform random variables.
> restart: # MapleUniformRN.mws
> with(stats,random);
[random]
> randon( uniform] (10);

.4274196691, .3211106933, .3436330737, .4742561436, .5584587190,

.7467538305, .03206222209, .7229741218, .6043056139,
.7455800374

10.2.3 Kolmogorov-Smirnov Test for Uniformity

A sequence of uniform random variables UI, U2, ... , UN generated for a simula-
tion study must be-naturally-uniformly distributed and independent. There are
several tests available for testing these properties. For example, frequency tests
are used for testing uniformity, and run tests and autocorrelation tests are used for
testing independence; see Banks, Carson and Nelson [12, Section 8.4].
In this section we describe the Kolmogorov-Smimov test for testing uniformity
and use it to test Maple's random number generator. This test compares the the-
oretical distribution F(u) = IoU 1 dx = u,O < u < 1, of the uniform random
variable U, and the empirical distribution Ge(u) of the sample of N observations
obtained by generating uniform random numbers. The test involves a few simple
steps that can be easily implemented with Maple.
Suppose we generate N sample observations UI, U2, ... , UN for which the
empirical distribution is defined as
G ( ) _ number of i's such that Ui ~ U
e U - N .

We start by ranking the observations UI, U2, .. " UN in increasing order, i.e.,
U[I], U[2], .. . , UrN] where Uri] ~ U[i+I]. The test statistic dmax measures the
deviation of the random numbers from the theoretical distribution and it is defined
as
dmax = max(tti" , a)
where

d+ = max
19~N
(.!....
N
- Uri]) and a = max (U[i] _
l~i~N
i-I).
N
Note that i / N - Uri] is the deviation of Ge(u) above F(i / N) = i / N and Uri] -
(i - 1)/ N is the deviation of Ge(u) below F ((i - 1)/N) = (i - 1)/N. Thus,
d+ and d- correspond to the largest deviation of Ge(u) above and below F(u),
 10.2 Generating (Pseudo-) Random Numbers 427

respectively. Intuitively, if the maximum dmax of the largest deviations is greater

than a critical value, then we reject the hypothesis that the samples are drawn from
a uniform distribution.
More specifically, the test statistic dmax is compared to a critical value da for
a given significance level of u. If dmax > da , this indicates that the deviations
from the theoretical distribution are large, thus the hypothesis that the sample
data are from a uniform distribution is rejected. If dmax ~ da , the hypothesis is
not rejected. For N > 35, the approximate critical values are given by do.OJ =
1.63j,.[N, do.os = l.36j,.[N and do. 10 = 1.22j,.[N. For a detailed discussion of
the Kolmogorov-Smimov test; see Banks, Carson and Nelson [12, Sec. 8.4].
Example 99 Kolmogorov-Smirnov uniformity test of Maple's random number
generator. We generate 40 uniform random numbers with Maple's randorr( uni-
form] command and apply the Kolmogorov-Smimov test to see if these samples
come from the true uniform distribution. Using a significance level of u = 0.05,
we would reject the hypothesis that the sample is from the true uniform if the test
statistic dmax > da = 1.36jJ40 = 0.215.
> restart: # KSTest.mws
> with(stats): #readlib(randomize): randomize():
> N:=40; ~ .O~ :=evalf(1.36/sqrt(N»;
N:=40
d.os := .2150348809
> U: ={ randorr( uniform] (N)] :
> USort:=sort(U);

USort := [.005862664913, .03206222209, .03916959416, .07481365622,

.08843057167, .1319057546, .1464863072, .1555907635,
.2197600994, .2598119527, .2726006090, .2813387792,
.3100754872, .3137460865, .3211106933, .3436330737,
.4274196691, .4293926737, .4537470195, .4742561436,
.5254285110, .5584587190, .6043056139, .6283634430,
.6438424438, .6440313953, .6720753584, .6759829338,
.6764707883, .7229741218, .7455800374, .7467538305,
.7512095393, .7924959004, .7971794905, .8129204579,
.8454735095, .9206249473, .9510535301, .9604988341]
> ~ a] :=max(seq(i/N-USort[ i] ,i=1. . N»;
da := .0870795421
> ~ b] :=max(seq(USort[ i] -(i-i) /N,i=1. .N»;
db := .0543056139
But we find dmax = 0.087 < da ; thus we do not reject the hypothesis and
conclude that no difference has been detected between the theoretical uniform
distribution and the distribution of the sample {VI, U2, ... , UN}.
428 10. Simulation

> clrnax: =rnax (d[ a] , d[ b] ) i

d",ax:== .0870795421
> if clrnax>c{ .05] then "Reject" else "Don't Reject"
fii
"Don't Reject"
The next few Maple statements generate and plot the empirical distribution
Ge(u) and the theoretical uniform distribution F(u)== u, 0 < u < I. Both
graphs are displayed in Figure 10.2.
> G:=seq([ i/N,USor~~] ,i=l .. N):
> with (plots) :
> GPlot:=pointplot([ ~ ,style=line,
connect=true,linestyle=1,color=black,thickness=2) :
> FPlot:=plot(u,u=0 .. 1,linestyle=2,
color=black,thickness=2):
> display ({ GPlot, FPlot} ) i

.....

0.8

0.6

FIGURE 10.2. Comparison of the empirical distribution Ge(u) drawn as a solid curve and
the theoretical unifonn distribution F(u) drawn as a dotted line.

10.3 Generating Random Variates from Other

Distributions
Assuming that a uniform random number has been generated using the method(s)
described, it can then be used to generate another random variable with a given
distribution. (This is known as generating a rando", variate.) In this section, we
discuss the inverse transform method and also explain the use of Maple commands
that can generate random variates from 18 continuous and discrete distributions.
 10.3 Generating Random Variates from Other Distributions 429

Consider a random variable X with density f(x) and distribution F(x). If the
inverse F-I of the distribution function can be easily computed, then the inverse
transform method can be used to generate a random variate from the distribution
of X. This is done by setting F(X) = U where U is a uniform number and solving
for X = F-1(U).

10.3.1 Exponential Random Variates

As an example, we apply this method to generate random variates from the expo-
nential distribution with parameter A.. Recall that an exponential random variable
X with parameter A. has the density function f(x) = A.e- b and the distribution
function F(x) = I; A.e-).t dt = 1 - e- b , x ~ o. Since the inverse F-1ofthe
distribution function of exponential is available, the inverse transform method can
be easily used to generate exponential random variates.

Example 100 Generation of exponential random variates. We start by defining

the density and the distribution of the exponential.
> restart: * ITMExponential.mws
> f:=x->larnbda*exp(-lambda*x);

f:= x ~ A.ehh )
> F:=x->int(f(t),t=O .. x);

F:= x ~ lie f(t)dt

After inverting the distribution (i.e., solving F(X) = U for X), we find that
X = -log(1 - U)/ A.. Letting A. = 3 and generating uniform random numbers,
we can now sample from the exponential distribution.
> X:=unapply(solve(F(X)=U,X),U);

X := U ~ _ In(1 - U)
A.
> larnbda:=3;
A. := 3
> X(.3562);
.1467890534
> with(stats);
[anova, describe, fit, importdata, random, stateva/f, statplots, transform]
Maple's randorr( uniform] (10) command generates 10 uniform random
numbers. After mapping X to them, we obtain 10 samples from the exponential
distribution.
> U:={ randorr( uniform] (10)] ;
430 10. Simulation

U := [.3211106933, .3436330737, .4742561436, .5584587190,

.7467538305, .03206222209, .7229741218, .6043056139,
.7455800374, .2598119527]
> map (X, U) ;

[.1290990629, .1403451025, .2143137166, .2724945872, .4577977537,

.01086249093, .4278814513, .3090377060, .4562563270,
.1002836690]
The inverse transfonn method is useful in generating random variates from the
exponential, unifonn defined over the interval (a, b), Weibull, triangular and em-
pirical continuous distributions since the inverse F- 1 of these distributions can
be found with relative ease. But this method fails for a number of important con-
tinuous distributions including the nonnal, gamma and beta. There are, of course,
other methods that can be used to sample from these distributions and these are
discussed in textbooks on simulation, e.g., Banks, Carson and Nelson [12], Ross
[160], and Pritsker, O'Reilly and LaVal [153]. We should also mention that as we
demonstrated in Example 97 on port operations, the inverse transfonn method is
also applicable to sampling from a wide variety of discrete distributions including
the binomial and Poisson.

10.3.2 Maple's Random Variate Generators

Maple can generate random variates from a total of 18 distributions (13 continu-
ous, 4 discrete and 1 empirical discrete) using the command randorr( distr-
ibu t i 0 n _name] . Additional infonnation on these distributions can be obtained
by entering? stats, distributions. The list of 13 continuous and 5 dis-
crete distributions and the corresponding Maple syntax is provided in Tables 10.1
and 10.2, respectively.
We now present an example where Maple generates 5 random variates from
some of the distributions that arise frequently in operations research applications.

Example 101 Random variates from some distributions. The next set of com-
mands generates random variates from beta, exponential, gamma, nonnal, uni-
fonn and Weibull continuous distributions.
> restart: # MapleRandornVariates.rnws
> with(stats); #with(randorn); nops(with(randorn));
[anova, describe, fit, importdata, random, statevaJf, statp/ots, transform]
> randorr( beta[ 1,2]] (5);
.1586913674, .4696943744, .01327905798, .4781283152, .04138715231
> randorr( exponential[ 2]] (5);
.9336948690, .5642329350, .1651826067, .7863020910, .6955721300
> randorr( garnrna[ 2, 4]] (5);
 10.3 Generating Random Variates from Other Distributions 431

Distribution Maple Command Parameters

Beta beta[ n), n2] n), n2 e N+
Cauchy cauchy[ a, b] a e lR, b e lR.+
Chi-square (x2) chisquare[ n] neN+
Exponential exponential[ .A.] .A. e lR.+
Fisher's F fratio{ nl,n2] n),n2 e N+
Gamma garruna[ a, b] a,b e R+
Laplace laplaced{ a, b] a e lR, be R+
Logistic logistic[ a, b] a e lR, b e lR.+
Lognonnal lognormal[ /J,O'] /J e lR, 0' e R+
Nonnal normald{ /J,O'] /J e lR, 0' e lR.+
Student's t studentst[ n] n eN+
Unifonn uniforrr( a, b] a,b e R, a ~ b
Weibull weibull[ a, b] a,b e R+
TABLE 10.1 . Maple can generate random variates from this list of 13 continuous distribu-
tions.

Distribution Maple Command Parameters

Binomial binomiald{ n, p] n e N+, p E (0, I)
Discrete Unifonn discreteuniforrr( a, b] a,b e N,a ~ b
Negative Binomial negati vebinomial[ n, p] n e N+, p e (0, 1)
Poisson poisson[ .A.] .A. e lR.+
Empirical empirical[ Probs] LProbs= 1
TABLE 10.2. Maple can generate random variates from this list of 5 discrete distributions.

10.83376082, .4130222104, 11.26025842, 12.10647440, 14.76067790

> randorr( normalc{ 1,2]] (5);
-2.199385336, 2.636270022, 1.170950535, 1.189344489, -1.815978260
> randorr(uniforrr(2,7]] (5);
5.147399353, 5.682254920,4.007973530,2.891574531,6.574125685
> randorr( weibull[ 1,5]] (5);
1.651659671, 3.026604177, 7.327432705, .4207307627,4.645956875
In the next group, we generate random variates from binomial, discrete unifonn
and Poisson discrete distributions.
> randorr( binomiald{ 5, .5]] (5);
3.0, 4.0, 3.0, 4.0, 3.0
> randorr( discreteuniforrr( 1,5]] (5);
3.0, 5.0, 3.0, 5.0, 1.0
> randorr( poisson[ 5]] (5);
8.0, 4.0, 2.0, 2.0, 5.0
432 10. Simulation

Maple can also generate random variates from discrete empirical probability
distributions defined over the integers I, . . . , N. Once these distributions are de-
fined, their random variates can be generated using the random command. In
the next example, we define an empirical distribution p(k) that can take 6 values
k=I, ... ,6:
0.30, fork = I
0.20, fork=2
0.10, fork = 3
p(k) =
0.25, fork=4
0.05, fork=5
0.10, fork = 6.
> Probs:=.3, .2, .1, .25, .05, .1;
sum ([ Probs] [ i) , i=l . . nops ([ Probs] ) ) ;
Probs := .3, .2, .1, .25, .05, .I
1.00
> p:=ernpirical[ Probs] ;
p := empirical.3, .2,.1, .25, .05,.1
The next command generates 10 random variates from this empirical distribu-
tion.
> randorr( p] (10);
1.0, 3.0, 1.0, 2.0, 4.0, 4.0, 1.0, 4.0, 1.0, 1.0

10.4 Monte Carlo Siniulation

A static simulation where time does not playa role is known as Monte Carlo sim-
ulation. This method has been found useful in the numerical evaluation of definite
integrals. It can also be used in the simulation of some one-period probabilistic
problems.

10.4.1 Numerical Evaluation of Definite Integrals

Suppose we wish to evaluate

J.l = 10 1 g(x) dx
where g(x) is a given function-such as g(x) = log(x2 + e-x 2 )-whose integral
cannot be computed directly. To evaluate this integral by simulation, we proceed
as follows.
 10.4 Monte Carlo Simulation 433

First note that if V is uniform over (0, I}-Le., if V's density is fu(u) = 1,
o< u < I-then we can write

J.l = E[g(U)] = 10' g(u)fu(u)du = 10' g(u)du.

Thus, if we can generate independent and uniform random variables V" ... , Vk
defined over (0, 1), then we can approximate the integral Jd
g(x) dx using

1 k
lim -k Lg(V;) = E[g(U)] = J.l
k-.+oo ;=,
where the g(VJ), ... , g(Vk) are also independent and identically distributed ran-
dom variables. The result follows by using the strong law of large numbers.
Generating a sufficiently large number of uniform random numbers u" ... , Uk
and taking the average of g(u,), ... ,g(Uk) gives the value of the integral.

Example 102 Evaluation ofa definite integral. Consider the problem of evaluat-
ing the definite integral Jo' g(x) dx where g(x) = log(x2 + e-x \ We generate
1000 uniform random numbers and evaluate the integral using Monte Carlo.
> restart: # MonteCarloIntegral.mws
> with(stats,random); with(stats,describe);
[random]
[describe]
> u: ={ randorr( uniform] (1000)]
> g:=u->log(u A 2+exp(-u A 2));
g := u ~ log(u2 + e<-u 2»
> #plot(g(x),x=O .. l);
> mu:=map(g,u):
We find 0.07401 as the simulated value of the integral.
> describe[ mean] (mu);
.07401925167
Maple is unable to evaluate the integral using the int () command. But nu-
merical integration of the function-using evalf (int ( ) ) -gives a result that
is close to what we found using Monte Carlo simulation.
> int(g(x),x=O .. l);

10 'In(x 2 + e< _x2» dx

> evalf(int(g(x),x=O .. l));

.07296304693
434 10. Simulation

10.4.2 Simulation ofa Static (Single-Period) Problem

In our next example, we use simulation to solve the (in)famous "car and goats"
problem.

Example 103 The car and goats problem-The Monty Hall Dilemma. Suppose
you are on Monty Hall's TV game show "Let's Make a Deal," and Monty gives
you a choice of three doors. Behind one door is a shiny new car, and behind each
of the other two doors is a smelly goat. You choose a door, say, number I, and
Monty, who knows what is behind the doors, opens another door, say number 3,
which has a goat. He says to you, "Do you want to pick door number 2?" Is it
to your advantage to switch your choice of doors? That is, do you have a better
chance of winning the car if you switch?
This problem created a lot of an excitement in 1991 when it appeared in Mar-
ilyn vos Savant's column "Ask Marilyn" in Parade magazine and received many
wrong answers from readers--many people, including some professional mathe-
maticians, thought that after Monty reveals the goat, the probability of winning is
112; see vos Savant's book The Power ofLogical Thinking [165] for transcripts of
the readers' letters. Vos Savant answered the question correctly by stating that if
the contestant switches doors then the probability of winning is 213, not 112. 5
To solve this problem using simulation we will assume, without loss of gener-
ality, that you pick door I. Monty knows where the car is and will open an empty
door. Here are the outcomes if you always switch.

• If the car is really behind door I, Monty is equally likely to open door 2 or
3.

- Ifhe opens door 2 and if you switch, you lose.

- Ifhe opens door 3 and if you switch, you again lose.

• If the car is really behind door 3, Monty will open door 2.

- In this case, if you switch, you win.

• If the car is really behind door 2, Monty will open door 3.

- In this case, if you switch, you win.

Thus, using the policy of always switching, you would win the car if it is behind
door 2 or 3. With the policy of never switching, you would win the car if it is
behind door 1.6

SShe argued as follows: Suppose there are 1 million doors and the contestant picks door I. Monty,
who knows what is behind all the doors and always avoids the one with the car, opens all except door
number 777,777. In this case, the contestant would surely pick door number 777,777.
6ft is perhaps now obvious (even without using simulation) that, if you switch, the probability of
winning the car is 213.
 10.5 Dynamic Simulation Models 435

At the start of the game, the car is equally likely to be behind any of the three
doors. We generate unifonn random numbers using discreteuniforrrf 1,3)
to simulate the actual location of the car.
> restart: # MontyHall.mws
> with(stats); #readlib(randomize): randomize():
Canova, describe, fit, importtiata, random, statevalJ, statp/ots, transform]
> p:=discreteuniform 1,3) ;
P := discreteunijormt. 3
We simulate the game 1000 times and find that 67.5% of the time the contes-
tant wins the car under the policy of always switching and 34% of the time the
contestant wins under the policy of never switching. Marilyn was right!
> N:=1000;
N:= 1000
> # Policy 1: Always Switch
> Win:=O:
> for n from 1 to N do
CarBehindDoor: =random p) (1);
if CarBehindDoor=1.0 then Win:=Win+O;
elif (CarBehindDoor=2.0 or CarBehindDoor=3.0)
then Win:=Win+1;
fi od:
> evalf (Win/N) ;
.6750000000
> # Policy 2: Never Switch
> Win:=O:
> for n from 1 to N do
CarBehindDoor: =random p) (1);
if CarBehindDoor=1.0
then Win:=Win+1;
elif (CarBehindDoor=2.0 or CarBehindDoor=3.0)
then Win:=Win+O;
fi od:
> evalf(Win/N);
.3400000000

10.5 Dynamic Simulation Models

Many real-life systems encountered in operations research applications exhibit
probabilistic behavior as they evolve over time. For example, inventory level in
a periodic-review inventory control problem is a stochastic process that is influ-
enced by the randomness of demand and the choice of the inventory ordering
436 10. Simulation

policy. In a queueing system, the waiting time of consecutive customers is also

a stochastic process that is influenced by the randomness in arrival and service
patterns.
For problems of this type, it may be important to determine the long-run be-
havior of the system (e.g., average inventorylbackorders or the average time spent
in the queue) in order to choose policies that optimize the performance of the
system. Naturally, if the problem is simple enough, the system may be analyzed
using one of the available analytic models. But in many cases this may not pos-
sible and simulation becomes an important tool in understanding the behavior of
the system.
In this section we present two such examples-one from inventory and one
from queueing-and determine the long-run average behavior of the systems us-
ing simulation.

10.5.1 Simulation ofan Inventory System with Random Yield

Example 104 The periodic-review (s, S) inventory policy with random yield We
consider a single product for which demand Yn during successive weeks n =
1,2, ... are i.i.d. random variables with density ak = Pr(Yn = k), k = 0, I, ....
The inventory level Xn at the end of week n is observed and an order decision is
made according to the following (s, S) policy: If Xn < s, then we order Qn =
S - Xn units; otherwise (i.e., if s ~ Xn ~ S), no order takes place. We asswne
that deliveries are instantaneous and that any shortages are backordered.
To complicate matters, we will asswne that the amount actually received, Rn ,
after ordering Qn units is a random fraction of the order quantity given by Rn =
lPQnJ where P is a continuous random variable with density g(p) defined over
(0, 1) and LzJ is the greatest integer less than or equal to z.7 With these asswnp-
tions, the inventory level Xn can be written as

X n+1 = { lP(S - Yn+I)J ifXn < s

(10.3)
Xn - Yn+1 ifs ~ Xn ~ S.
The fact that the yield is random makes this a nontrivial problem. If the yield
were deterministic (i.e., if P == 1), then the problem could conceivably be solved
as a Markov chain after specifying the transition probability matrix of the in-
ventory level process. But since P and Yn (for given n) are random variables,
simulation appears to be a convenient way to analyze this problem.
> restart: # SsRandomYield.mws
> with(stats): #readlib(randomize); randomize();
We start by asswning that (s, S) = (5, 15) and Xo = 15. The weekly demands
are asswned Poisson with rate A. = 10, i.e., ak = Pr(Yn = k) = e- IO lOk /k!,

7Modeling the randomness of yield has recently become an important research area in operations
management; see Gerchak and Parlar [69] and Yano and Lee [197].
 10.5 Dynamic Simulation Models 437

and the random yield fraction P is taken as beta with parameters (a, b) = (6,2),
i.e., g(p) = 42 p 5(1 - p), 0 < p < 1. In the simulation of the inventory process,
random variates from both the Poisson and beta distributions are generated using
Maple's random function.
> S:=15; s:=5; lambda:=10; x[ 0] :=15;
S:= 15
s:= 5
A. := 10
xo := 15
We perform the simulation for a total of 100 weeks and generate the values of
Xn in equation (l0.3) using the following Maple commands.
> N:=100;
N:= 100
> #plot (statevalff pdf, beta[ 6,2]] (x), x=O .. 1) ;
> for n from 0 to N do
y[ n+1] :=randon( poisson[ lambda]] (1);
if x[ n] <s then
b:=randon( beta[ 6,2]] (1);
x[ n+1] :=floor (b* (S-y[ n+l] ) )
elif x[ n] >=s then
x[ n+ 1] : =x[ n] -y[ n+l]
fi od:
The simulated inventory levels (and demand) for the first 10 weeks are found
as follows.
> seq([ n,[ x[ n] ,y[ n+1]] ,x[ n+1]] ,n=1. .10);

[I, [6.0, 8.0], -2.0], [2, [-2.0, 9.0], 4], [3, [4, 12.0], 2],
[4, [2, 13.0], I], [5, [I, 15.0],0], [6, [0, 7.0], 6],
[7, [6, 13.0], -7.0], [8, [-7.0, 12.0], 2], [9, [2, 11.0], 2],
[10, [2, 10.0], 3]
> IL:={ seq (x[ n] ,n=1. .N)] : nops (IL):
In order to compute the average number of units in inventory and the average
number backordered, we need to isolate the nonnegative and negative inventory
levels. This is done easily with the select () command.
> ILNonNegative:=select(type,IL,nonneg);

ILNonNegative := [6.0, 4, 2, I, 0, 6, 2, 2, 3, 3, 6, 0, 6, 3, 6, 2.0, 7,4,

4, 0, 4, 2, 2, 4, 6, 2.0, 6, 5, 3, 5, I, I, 6, 2, 0, 2, 0, 1, 4, 2, 2,
2, 7, 0, 4, 5, 1, 4, 3, I, 2, I, 5, 0,4, 3, 3, 6, 3, 9, 1.0, 5,4, 5,
4, 1,4,6, 3, I, 2, 1,3, 3, 10, 5, 6]
> ILNegative:=select(type,IL,negative);
438 10. Simulation

ILNegative:= [-2.0, -7.0, -4.0, -2, -3.0, -5.0, -5.0, -3.0, -4.0, -5.0,
-2.0, -2, -2.0, -1.0, -4.0, -6.0, -5.0, -3, -8.0, -4, -1.0, -11.0,
-1.0]
The average inventory level and the average number backordered are found as
3.29 and 3.91, respectively.
> with(describe):
> describe[ mean] (ILNonNegative):
describe[ mean] (ILNegative):
3.298701299
-3.913043478
The distribution of the inventory level is found as follows.
> for n from min(op(IL)) to max(op(IL)) do
k[ n] : = (n. . (n +1)) od :
> Intervals:~ seq(k[n] ,n=min(op(IL)) .. max(op(IL)))];

Intervals := [-11.0 .. - 10.0, -10.0 .. - 9.0, -9.0 .. - 8.0, -8.0 .. - 7.0,

-7.0 .. - 6.0, -6.0 .. - 5.0, -5.0 .. - 4.0, -4.0 .. - 3.0, -3.0 .. - 2.0,
-2.0 .. - 1.0, -1.0 .. 0, 0 .. 1., 1...2., 2 .. .3., 3.. .4., 4 ... 5.,
5... 6.,6 ... 7., 7 ... 8., 8 ... 9., 9 ... 10., 10... 11.]
> with (transform) :
> Distribution:=statsort(tallyinto(IL,Intervals)):

Distribution := [-11.0 .. - 10.0, Weight(-IO.O .. - 9.0, 0),

Weight(-9.0 .. - 8.0,0), -8.0 .. -7.0, -7.0 .. - 6.0, -6.0 .. - 5.0,
Weight( -5.0 .. - 4.0, 4), Weight( -4.0 .. - 3.0, 4),
Weight( -3.0 .. - 2.0, 3), Weight( -2.0 .. - 1.0, 5),
Weight(-1.0 .. 0, 3), Weight(O .. I., 7), Weight(I...2., 11),
Weight(2 ... 3., 14), Weight(3 .. .4., II), Weight(4 ... 5., 12),
Weight(5 ... 6., 7), Weight(6 ... 7., II), Weight(7 ... 8., 2),
Weight(8 ... 9., 0), 9 .. .10., 10... 11.]
Thus, there is one observation in the interval [-II, -10), zero observations in
the interval [-10, -9), etc.
The complete distribution of the inventory level Xn is given in Figure 10.3.
> with (statplots) :
> histogram(Distribution):

10.5.2 Simulation ofa Non-Markovian Queue

Example 105 Distribution of the waiting time in the queue for a GIG I I queue.
As we saw in Chapter 9, Queueing Systems, the average waiting time in the
queue, W, for the simple Markovian MI Mil model is found analytically as
 10.5 Dynamic Simulation Models 439

FIGURE 10.3. Distribution of the inventory level Xn in the periodic-review inventory prob-
lem random yield for a simulation length of N = 100 weeks. Weekly demands are assumed
Poisson with parameter A = 10 and the random yield fraction p is beta sitributed with pa-
rameters (a, b) = (6,2).

W = 2/[.u(.u - 2)] where ;. is the rate of the Poisson arrival process and .u
is the rate of the exponentially distributed service times. s When Markovian as-
sumptions are no longer valid, i.e., when the arrival process is no longer Poisson
and/or service time is not exponential, such simple formulas no longer apply.
However, there is a very general set of recursive equations that relate the ran-
dom queue waiting time Wn of the nth customer to the interarrival time An be-
tween the (n - 1)st and nth customer and the service time Sn-I of the (n - l)st
customer.9 As shown originally by Lindley (122] (see also, Kleinrock [110, p.
277] and Gaver and Thompson [68, p. 572]), we can write

Wn+1 = max(Wn - An+1 + Sn, 0), n = 0, I, ... .

The waiting time in the queue process can now be easily simulated for customers
n = 1,2, ... , and the distribution of the time in the queue can be developed.
We assume that the interarrival times are distributed as uniform with parameters
(6,8) having density /A(x) = 1/2, 6 ~ x ~ 8 and the service time as Weibull
with parameters (2,6) having density fs(x) = xe-(x/6)2 /18, x ~ O.
The Maple commands used to simulate the Wn process are almost identical to
the ones we described in Example 104.
> restart: # GGl.mws
> with(stats):

8Generally, waiting time in the queue is denoted by Wq . In this example we will use the notation
W in order not to complicate the notation.
9Note that we do not make any probability assumptions about the distributions of An and Sn.
440 10. Simulation

> N:=100; ~ 00 :=0;

N:= 100
W() := 0

> #plot (statevalf[ pdf, weibull[ 2,6]] (x), x=O •• 50) ;

> for n from a to N do
a[ n+1] :=randon( uniforn( 6,8]] (1);
s[ n] : =randon( weibull[ 2,6]] (1);
w[ n+ 1] : =max (~ n] -a[ n+1] +s[ n] , 0)
od:
> seq([ n,[ a[ n+1] ,s[ n]],~ n+1]] ,n=0 .• 5);

[0,
[6.854839338, 3.733992340], 0], [I, [6.687266147,4.811016670],0],
[2,
[7.116917438, 7.031511744], 0],
[3,
[6.064124444, 6.797881782], .733757338],
[4,
[7.208611228, 7.019664048], .544810158],
[6.519623905, 3.655438579], 0]
[5,
> wList:=[ seq(w[ n] ,n=O .. N)] :
> for n from min(op(wList)) to max(op(wList)) do
k[ n] : = (n .. (n + 1)) od:
> Intervals:=[ seq(k[ n] ,n=min(op(wList)) .•
max (op(wList) ))] :
> with(describe):
We see that the mean waiting time in queue is obtained as 2.004 with a standard
deviation of3.1 0 1.
> describe{ mean] (wList);
describe[ standarddeviation] (wList);
2.004485394
3.101872397
> with (transform) :
Following is the complete distribution of the waiting times where we find that
out of 100 observations, 64 fall in the interval [0,1), 7 fall into interval [1,2) and
soon.
> Distribution:=statsort(tallyinto(wList,
Intervals));

Distribution:= [Weight(O .. I, 64), Weight(1..2, 7), Weight(2 .. 3, 3),

Weight(3 . .4, 3), Weight(4 . .5, 6), Weight(5 .. 6, 5),
Weight(6 .. 7, 2), Weight(7 .. 8, 3), Weight(8 ..9, 5), 9 . .10,
Weight(IO .. II,O), 11..12, Weight(I2 .. 13, 0),
Weight(13 . .14,O), 14 .. 15]
> with(statplots):
A histogram of the distribution of the waiting times is presented in Figure 10.4.
> histogram(Distribution);
 10.6 Optimization by Random Search 441

FIGURE 10.4. Distribution of the waiting time in queue, Wn , for the G / G / I queue with
uniform interarrival times and Wei bull service times for a simulation length of N = 100
customers. The uniform distribution has parameters (6, 8) and the Weibull has parameters
(2,6).

It is also possible to obtain a running average of the simulated waiting times,

~ L7=1 Wi, and plot the result using pointplot ( ). The Maple commands to
compute these averages and plot them follow. They can be executed after remov-
ing the "#" symbol before each command. Note that as the simulated number
of customers (n) approaches the run length N, the graph of the running average
should converge to the simulated final average of2.004.
> #for n from 1 to N do wRun[ n] :=sum(w[ i] ,i=1. .n) /n
od:
> #L:=seq([ n,wRun[ n]] ,n=1. .N):
> #with (plots) :
> #pointplot({ U ,symbol=point);

10.6 Optimization by Random Search

The optimization techniques presented in Chapter 5, Nonlinear Programming, re-
quired the differentiability of the function that was to be optimized. In certain
cases the function is not differentiable or it may be discontinuous or even speci-
fied as a table of values, or worse, it may have a large number of local optima In
those cases, the classical optimization techniques may have to be supplemented
by alternative methods in order to optimize a function of several variables.
One of the most useful of such techniques relies upon generation of random
numbers to search for the (global) optimum. Adaptive random search (ARS) is a
simpler version of the global optimization method known as simulated annealing;
442 10. Simulation

see, e.g., Gottfried and Weisman [7S, Section 3.4] for a description of ARS and
Laarhoven and Aarts [lIS] for a detailed account of simulated annealing.
For a problem With, say, two decision variables x and y, starting with a feasible
point in the two-dimensional region that is known to contain the optimal solution
(x·, y.), ARS randomly generates the next feasible point and compares it to the
previous point. If the new point is better than the previous one and it is feasible,
it is kept; otherwise, the new point is discarded. lo For example, if x· is known to
be in the interval [Xmin, xmax] and if Xold is the previous point, then the new point
is generated using the formula Xnew = Xold + (xmax - x min)(2r - I)V where r is
a random number between 0 and I and v is an odd integer. As recommended by
Gall [67], initially, v is chosen as 3 or S and later, when the improvements in the
objective function become smaller, it is increased to 7 or 9. Note that the larger the
interval of uncertainty (xmax - Xmin), the larger the step size (xmax - xmin)(2r -l)v .
But increased values of v have the effect of shortening the step size since -1 <
2r-I<1.
Example 106 Minimization of a discontinuous and nondifferentiable junction
with two local minima. Consider a function z(x, y) defined as

z(X, ) = { S.)(x - 2)2 + (y - 2)2, for (x - 2)2 + (y - 2)2 :5: 4

y 31x - SI + Iy - SI + 3, otherwise.
As the three-dimensional graph in Figure 10.5 indicates, z(x, y) is a discontin-
uous and nondifferentiable function that has two local minima, one at (2,2) and
the other at (S,S).
> restart: # TwoMinima.mws
> z:=proc(x,y) if (X-2)A2+(y-2)A2<=4 then
S*sqrt((x-2)A2+(y-2)A2) else
3*abs(x-S)+abs(y-S)+3 fi end;

z := proc(x, y)
if(x - 2)2 + (y - 2)2 :5: 4 then S x sqrt«x - 2)2 + (y - 2)2)
else 3 x abs(x - S) + abs(y - S) + 3
fi
end
> z(2,2); z(S,S);
o
3
> plot3d(z,0 .. 10,0 .. 10,axes=boxed,
orientation={ 140,68] , shading=none) ;
While a visual inspection in this case reveals that the global minimum of z(x, y)
is at (x, y) = (2,2) with z(O, 0) = 0, the complicated nature of this type of

lOIn simulated annealing, the new point that is worse than the old one is not immediately discarded,
but it is kept with a specific probability.
 10.6 Optimization by Random Search 443

function nonnally makes it very difficult to find the global optimum with the
standard optimization methods.

20

15

10

FIGURE 10.5. Graph of the z(x,y) function.

To minimize z(x, y), we now use ARS.

> restart: # ARSTwoMinima.mws
> with(stats): with (student) :
#readlib(randomize); randomize();
We let N = 10000 be the maximum number of random points that will be
generated. Clearly, many of these points will be quickly discarded because they
could be either infeasible or inferior to the last "good" point. The convergence
criterion is IZn - zn-II < e where Zn is the value of the objective function at the
nth good point that improves the objective and e = 10-6 .
> N:=10000; epsilon:=10.A(-6);
N:= 10000
e := .1000000000 10-5
The search is perfonned over the set E = [0,5] x [0,5] which is (presumably)
known to contain the optimal solution. The initial point is generated randomly in
the set E and the initial value of the objective is set at Z = 999.
> xMin:=O: xMax:=S: yMin:=O: yMax:=S:
> r:=randorr( uniform] ();
r := .4274196691
> xOpt:=xMin+r* (xMax-xMin);
xOpt := 2.137098346
> r: =randorr( uniform] ();
r := .3211106933
444 10. Simulation

> yOpt:=yMin+r* (yMax-yMin);

yOpt := 1.605553467
> zOpt:=999;
zOpt:= 999
The list Ln = [XOpt. YOpt, zOpt] is used to keep track of the points that corre-
spond to progressively better values of the objective function.
> n: =0; I( 0] : ={ xOpt, yOpt, zOpt] ;
n:=O
Lo := [2.137098346, 1.605553467, 999]
The function z(x, y) is defined as a procedure and we generate the new points
randomly using v = 3. The search stops either (i) when the differences in the im-
proved objective function values become smaller than £ or (ii) when N is reached.
> z:=proc(x,y) if (X-2)A2+(y-2)A2<=4 then
5*sqrt((x-2)A2+(y-2)A2) else
3*abs(x-5)+abs(y-5)+3 fi end;

z := proc(x, y)
if(x - 2)2 + (y - 2)2 ~ 4 tben 5 x sqrt«x - 2)2 + (y - 2)2)
else 3 x abs(x - 5) + abs(y - 5) + 3
fi
end
> for i from 1 to N do:
r:=randorr( uniform] () :
> x:=xOpt+(xMax-xMin)* (2*r-1)A3:
r: =randorr( uniform] ():
> y:=yOpt+(yMax-yMin)* (2*r-1)A3:
> if (x<xMin or x>xMax or y<yMin or y>yMax) then
next fi:
> if z(x,y»zOpt then next fi:
> xOpt:=x; yOpt:=y; zOpt:=z(x,y);
n:=n+1:
I( n] : ={ xOpt, yOpt, zOpt] ;
Delta{ n] :=I( n] -I( n-1] ;
if abs(Delta{ n][ 3] )<epsilon then break fi;
> od:
Once the iterations end, we check the final values of n and i. The value of
n = 23 indicates that out of 10000 iterations only 23 "good" points were found
that improved the objective function. However, the value of i = 1000 1 signals
that the convergence criterion was not yet satisfied. But, examining the values of
Ln and Ln - Ln-I, we note that the solution found by ARS, i.e.,

x = 2.000044975, Y = 2.000078133, and z = .00045076369,

 10.7 Summary 445

is reasonably close to the global optimum (2, 2, 0).

> n; i;
23
10001
> I..{ n] ;
[2.000044975, 2.000078133, .0004507636940]
> I..{n]-I{n-l];
[.000122649, -.000112829, - .0005800095080]
Following are some values of [k, xOpt. YOpt, zOpt] for k = l3n/4J to n where
we observe the quick convergence of the points generated by ARS.
> ListxOptyOptzOpt:=seq([ k,[ I{ k] [ 1] ,I{ k] [2]] ,
I{ k][ 3]] , k=floor (3* n/ 4) . . n) ;

ListxOptyOptzOpt:= [17, [1.999697073, 2.000495347], .002903159562],

[18, [1.999688219, 2.000251433], .002002660390],
[19, [1.999688006, 2.000219307], .001906802404],
[20, [1.999848550, 2.000217551], .00 1325382399],
[21, [1.999892000, 2.000207601], .001170065973],
[22, [1.999922326, 2.000190962], .001030773202],
[23, [2.000044975, 2.000078133], .0004507636940]
> X:=' x'; y:=' y' ;
x :=X
Y:=Y
Improving values of the objective function can also be plotted using point-
plot (). [The graph can be displayed after removing the * sign preceding the
pointplot () command.]
> with (plots) :
> ListzOpt : =seq([ k,I{ k] [ 3]] , k=floor(n/2) . • n);

ListzOpt := [II, .006973344915], [12, .004455756688],

[13, .004023786549], [14, .003955266465], [15, .003893983199],
[16, .003338421731], [17, .002903159562], [18, .002002660390],
[19, .001906802404], [20, .001325382399], [21, .001170065973],
[22, .001030773202], [23, .0004507636940]
> *pointplot ([ ListzOpt] , view=[ 1. . n, -1. .1] ) ;

10.7 Summary
Many interesting and important problems in operations research cannot (yet) be
solved analytically. For example, a large number of problems in queueing net-
446 10. Simulation

works are not amenable to solution since the available mathematical theory is
not sophisticated enough to find a closed-form solution for the operating char-
acteristics of such systems. Simulation is a very flexible (albeit time-consuming)
solution technique that has found wide applicability. We started this chapter by
describing Maple's uniform random number generator. We also gave examples
of using Maple's nearly 20 continuous and discrete random variate generators
including the exponential, normal, binomial and Poisson. Next, Monte Carlo sim-
ulation was used to estimate the definite integral of a complicated function and to
simulate the optimal policy in the "car and goats" TV game. Maple's random vari-
ate generators were then used to simulate an inventory system with random yield
and a non-Markovian queue. The chapter ended with an optimization application
where the global optimum of a discontinuous and nondifferentiable function was
located using the method of adaptive random search.

10.8 Exercises
I. Two friends who have unpredictable lunch hours agree to meet for lunch
at their favorite restaurant whenever possible. Neither wishes to eat at that
particular restaurant alone and each dislikes waiting for the other, so they
agree that:

• Each will arrive at a random time between noon and I :00 p.m.
• Each will wait for the other either for 15 minutes or until I :00 p.m. at
the latest. For example, if one arrives at 12:50, he will stay until 1:00
p.m.

(a) Use simulation to find the probability that the friends will meet.
(b) Use your knowledge of probability to find the exact solution for this
problem.

2. Ifzo = 5 andzn+1 = tzn +3 mod 100, findzl.z2, ...• Z20.

3. Consider two independent standard normal random variables Z 1 "" N (0, I)
and Z2 "" N(O, I) and let R and a denote the polar coordinates of the
vector (Z I, Z2), i.e.,

R2 = Zf + zi
Z2
tana = ZI.

(a) Show that the joint density of ZI and Z2 is given by

 10.8 Exercises 447

(b) To detennine the joint density of (R 2 , 9) make the change of vari-

ables

s = zT +~
o = tan-I (;~)
and show that

iR2,e(s,O) = (~e-!s) x 2~' 0<s< 00, 0 < 0 < 21f .

(c) Since R2 is exponential with mean 2, and 9 is unifonn over (0, 21f),
argue that the vector (Z I, Z2) can be generated using the fonnula
ZI = Rcos(9) = J-210gUI COS(21fU2)
Z2 = Rsin(9) = J-210gUI sin(21fU2)
where UI and U2 are random numbers between 0 and 1. This method
is due to Box and Muller [36].
4. Consider the function f(x) = 20x 2 (1 - x)3.

(a) Find the area under f(x) between 0 and 1.

(b) Use Maple's int () function to find the exact solution for this prob-
lem.
5. Daily demand for the National Post newspaper at a particular newsstand in
Toronto is randomly distributed as follows:
Number Sold Probability
150 .05
175 .15
200 .35
225 .25
250 .20
The owner of the newsstand makes a profit of 40 cents on every paper sold,
but loses 15 cents on every paper unsold. Use 100 days to simulate the
owner's expected profit ifhe orders 150, 175,200,225 or 250 newspapers.
What is the maximum order quantity and the maximum expected profit?
6. The standard Brownian motion (BM) (Z(t); t ~ O} is a continuous-time,
continuous state-space stochastic process with the property that, (i) Z(O) =
0, (ii) Z(t) has stationary and independent increments, and (iii) for every
t ~ 0, Z(t) ""' N(O, t), i.e., Z(t) is nonnal with zero mean and vari-
ance t. This implies that the increment ~Z(t) of the standard BM can
448 10. Simulation

be written as llZ(t) =
y,JKi where Y ' V N(O, I), E[llZ(t)] 0 and =
Var[llZ(t)] = llt. Simulate the standard Brownian motion over the unit
time interval [0, I] by taking a step size of t:.t = 0.00 I and plot the result-
ing sample path.
7. The function/(x, y) = lOO(y-x 2)2+(I _x)2 is known as Rosenbrock's
banana function since its contours near the minimum point resemble a ba-
nana.

(a) Plot the surface of this function and its contours.

(b) Use the adaptive random search method to find the minimum.

8. Consider a dam that generates electricity and supplies water for irrigation.
Define X n as the water level at the start of period n and let r n be the amount
of water released in period n. Let t; n be the random amount of rainfall
during period n distributed as Poisson with parameter..t, i.e., Pr(t;n = k) =
e-). At / kL Assume that there is a dam capacity of M units.

(a) Write a difference equation that relates Xn+l to X n, r n, t;n and M.

(b) Let..t = 5, M = 10 and use the release policy as
Xn - m, ifm ~ Xn < M
rn ={ 0, if 0 ~ Xn < m

with m = 4. [What is the relationship between this (m, M) policy and

the (s, S) policy used in inventory modeling?] Assuming Xo = 10,
simulate the water level process for the next 100 days. What is the
average water level in the dam using this policy?
References

[I] 1. Abate and W. Whitt. Numerical inversion of probability generating func-

tions. Operations Research leiters, 12:245-251, 1992.
[2] J. Abate and W. Whitt. Numerical inversion of Laplace transfonns ofprob-
ability distributions. ORSA Journal on Computing, 7:36-43,1995.
[3] R. L. Ackotf. The meaning, scope and methods of operations research.
In R. L. Ackotf, editor, Progress in Operations Research, volume I. John
Wiley, New York, 1961.
[4] R. L. Ackotf and M. W. Sasieni. Fundamentals of Operations Research.
John Wiley, New York, 1968.
[5] R. M. Adelson. Compound Poisson distributions. Operational Research
Quarterly, 17:73-75, 1966.
[6] S. C. Aggarwal. A review of current inventory theory and its applications.
International Journal ofProduction Research, 12:443482, 1974.
[7] H. Anton. Elementary Linear Algebra. John Wiley, New York, 6th edition,
1991.

[8] T. M. Apostol. Mathematical Analysis. Addison-Wesley, Reading, Mass.,

1969.

[9] K. 1. Arrow. Historical background. In K. J. Arrow, S. Karlin, and H. Scarf,

editors, Studies in the Mathematical Theory of Inventory and Production,
pages 3-15. Stanford University Press, Stanford, California, 1958.
450 References

[10] K. J. Arrow. Essays in the Theory of Risk-Bearing. Markham, Chicago,

1971.

[11] K. J. Arrow, T. Harris, and J. Marschak. Optimal inventory policy. Econo-

metrica, 19:250-272, 1951.
[12] J. Banks, J. S. Carson, and B. L. Nelson. Discrete-Event System Simula-
tion. Prentice Hall, Upper Saddle River, N.J., 2nd edition, 1996.
[13] J. Banks and W. J. Fabrycky. Procurement and Inventory Systems Analysis.
Prentice-Hall, Englewood Cliffs, N.J., 1987.
[14] S. K. Bar-Lev, M. Parlar, and D. Perry. Optimal sequential decisions
for incomplete identification of group-testable items. Sequential Analysis,
14(1):41-57, 1995.

[15] E. Barancsi, G. Banki, R. Borloi, A. Chikan, P. Kelle, T. Kulcsar, and

G. Meszena. Inventory Models. Kluwer Academic Publishers, Dordrecht,
1990.

[16] D. J. Bartholomew. Stochastic Models for Social Processes. John Wiley,

Chichester, 1982.

[17] M. Baxter and A. Rennie. Financial Calculus: An Introduction to Deriva-

tive Pricing. Cambridge University Press, Cambridge, 1996.
[18] M. S. Bazaraa and C. M. Shetty. Nonlinear Programming: Theory and
Algorithms. John Wiley, New York, 1979.
[19] M. J. Beckmann. Dynamic Programming of Economic Decisions.
Springer-Verlag, Berlin, 1968.
[20] S. Beer. Decision and Control. John Wiley, New York, 1966.

[21] R. Bellman. On the theory of dynamic programming. Proceedings of

National Academy ofSciences, 38:716-719,1952.
[22] R. Bellman. Dynamic progamming and a new formalism in the calculus
of variations. Proceedings ofthe National Academy ofSciences, 39: 1077-
1082, 1953.

[23] R. Bellman. Dynamic Programming. Princeton University Press, Prince--

ton, N.J., 1957.

[24] R. Bellman. Adaptive Control Processes: A Guided TOUT. Princeton Uni-

versity Press, Princeton, N.J., 1961.

[25] R. Bellman and K. L. Cooke. Modern Elementary Differential Equations.

Dover, New York, 2nd edition, 1971.
 References 451

[26] R. Bellman and S. Dreyfus. Dynamic programming and the reliability of

multicomponent devices. Operations Research, 6:200-206,1958.
[27] R. Bellman and R. E. Kalaba. Dynamic programming and statistical com-
munication theory. Proceedings of the National Academy of Sciences,
USA, 43:749-751, 1957.
[28] R. Bellman and R. E. Kalaba. On the role of dynamic programming in sta-
tistical communication theory. IRE Transactions on Information Theory,
IT-3:197-203,1957.
[29] A. Bensoussan, E. G. Hurst, and B. Naslund. Management Applications of
Modern Control Theory. North-Holland, Amsterdam, 1974.
[30] D. P. Bertsekas. Dynamic Programming: Deterministic and Stochastic
Models. Prentice-Hall, Englewood Cliffs, 1987.
[31] D. P. Bertsekas. Dynamic Programming and Optimal Control, volume I.
Athena Scientific, Belmont, Massachusetts, 1995.
[32] U. N. Bhat. A controlled transportation queueing process. Management
Science, 16(7):446-452, 1970.
[33] U. N. Bhat. Elements ofApplied Stochastic Processes. John Wiley, New
York, 2nd edition, 1984.
[34] F. Black and M. Scholes. The pricing of options and corporate liabilities.
Journal of Political Econamy, 81 :637-659, May-June 1973.
[35] J. H. Bookbinder and D. L. Martell. Time-dependent queueing approach to
helicopter allocation for forest fire initial attack. INFOR, 17:58-70, 1979.
[36] G. E. P. Box and M. F. Muller. A note on the generation of random normal
deviates. Annals ofMathematical Statistics, 29:610-611, 1958.
[37] L. Breiman. Stopping-rule problems. In E. F. Beckenbach, editor, Applied
Combinatorial Mathematics, pages 284-319. John Wiley, New York, 1964.
[38] R. S. Brooks and J. Y. Lu. On the convexity of the backorder function for
an EOQ policy. Management &ience, 15(7):453-454, 1969.
[39] A. E. Bryson and Y.-C. Ho. Applied Optimal Control. Halstead, New
York,1975.
[40] G. Carter, J. M. Chaiken, and E. J. Ignall. Response areas for two emer-
gency units. Operations Research, 20:571-594,1972.
[41] S. <;etinkaya and M. Parlar. Optimal nonmyopic gambling strategy for
the generalized Kelly criterion. Naval Research Logistics, 44(4):639-654,
1997.
452 References

[42] S. Cetinkaya and M. Parlar. Nonlinear programming analysis to estimate

implicit inventory backorder costs. Journal of Optimization Theory and
Applications, 97(1):71-92, 1998.
[43] Y. S. Chow, H. Robbins, and D. Siegmund. The Theory of Optimal Stop-
ping. Houghton MitHin, Boston, 1971.
[44] C. W. Churchman. The Systems Approach. Delta, New York, 1968.
[45] C. W. Churchman, R. L. Ackoff, and E. L. Arnoff. Introduction to Opera-
tions Research. John Wiley, New York, 1958.
[46] V. Chvatal. Linear Programming. W. H. Freeman and Company, New
York,1983.
[47] E. <;mlar. Introduction to Stochastic Processes. Prentice-Hall, Englewood
Cliffs, 1975.
[48] C. W. Cobb and P. H. Douglas. A theory of production. American Eco-
nomic Review, 18(Supp. No. 2):139-165, 1928.
[49] L. Cooper and M. W. Cooper. Introduction to Dynamic Programming.
Pergamon Press, New York, 1981.
[50] R. B. Cooper. Introduction to Queueing Theory. CEE Press, Washington,
D.C., 1990.
[51] R. M. Corless. Essential Maple. Springer-Verlag, New York, 1995.
[52] D. R. Cox. The analysis of non-Markovian stochastic processes by the
inclusion of supplementary variables. Proceedings of Cambridge Philo-
sophical SOCiety, 51 :33-41, 1955.
[53] D. R. Cox and H. D. Miller. The Theory ofStochastic Processes. Chapman
and Hall, London, 1965.
[54] D. R. Cox and W. L. Smith. Queues. Chapman and Hall, London, 1961.
[55] T. B. Crabill, D. Gross, and M. J. Magazine. A classified bibliography of
research on optimal design and control of queues. Operations Research,
25:219-232, 1977.
[56] M. A. Crane and A. J. Lemoine. An Introduction to the Regenerative
Method for Simulation Analysis. Springer-Verlag, New York, 1977.
[57] S. DaruJ. Nonlinear and Dynamic Programming. Springer-Verlag, New
York,1975.
[58] G. B. Dantzig. Maximization of a linear function of variables subject to
linear inequalities. In T. C. Koopmans, editor, Activity Analysis ofProduc-
tion and Allocation, pages 339-347. John Wiley, New York, 1951.
 References 453

[59] G. B. Dantzig. Linear Programming and Extensions. Princeton University

Press, Princeton, N.J., 1963.
[60] M. D. Davis. Game Theory: ANontechnical Introduction. Dover, Mineola,
N.Y.,1983.

[61] N. Derzlm, S. P. Sethi, and G. L. Thompson. Dirtributed parameter sys-

tems approach to the optimal cattle ranching problem. Optimal Control
Applications and Methods, 1:3-10, 1980.
[62] R. L. Disney and P. C. Kiessler. Traffic Processes in Queueing Networks:
A Markov Renewal Approach. John Hopkins University Press, Baltimore,
MD,1987.
[63] W. Edwards. Dynamic decision theory and probabilistic information pro-
cessing. Human Factors, 4:59-73, 1962.
[64] R. M. Feldman and C. Valdez-Flores. Applied Probability and Stochastic
Processes. PWS Publishing Company, Boston, 1996.
[65] W. Feller. An Introduction to Probability Theory and its Applications, vol-
ume II. John Wiley, New York, 2nd edition, 1971 .
[66] G. S.Fishman. Principles ofDiscrete Event Simulation. Wiley, New York,
1978.
[67] D. A. Gall. Apractical multifactor optimization criterion. In A. Lavi and
T. P. Vogel, editors, Recent Advances in Optimization Techniques. John
Wiley, New York, 1966.
[68] D. P. Gaver and G. L. Thompson. Programming and Probability Models
in Operations Research. Brooks/Cole, Monterey, Calif., 1973.
[69] Y. Gerchak and M. Parlar. Yield randomness, cost tradeoffs and diversifi-
cation in the EOQ model. Naval Research Logistics, 37:341-354,1990.

[70] Y. Gerchak and M. Parlar. Investing in reducing lead-time randomness in

continuous-review inventory models. Engineering Costs and Production
Economics, 21:191-197,1991.
[71] Y. Gerchak, M. Parlar, and S. S. Sengupta. On manpower planning in
the presence of learning. Engineering Costs and Production Economics,
20:295-303,1990.
[72] A. Ghosal, S. G. Loo, and N. Singh. Examples and Exercises in Operations
Research. Gordon and Breach, London, 1975.
[73] P. E. Gill, W. Murray, and M. H. Wright. Practical Optimization. Aca-
demic Press, New York, 1981.
454 References

[74] R. Goodman. Introduction to Stochastic Models. Benjamin/Cummings,

Menlo Park, 1988.

[75] B. S. Gottfried and J. Weisman. Introduction to Optimization Theory.

Prentice-Hall, Englewood Cliffs, New Jersey, 1973.

[76] D. Gross and C. M. Harris. Fundamentals of Queueing Theory. John

Wiley, New York, 2nd edition, 1985.
[77] R. L. Gue and M. E. Thomas. Mathematical Methods in Operations Re-
search. Macmillan, London, 1968.
[78] O. GOrier and M. Parlar. An inventory problem with two randomly avail-
able suppliers. Operations Research, 45(6):1-15, 1997.
[79] G. Hadley. Linear Programming. Addison-Wesley, Reading, Mass., 1962.
[80] G. Hadley. Nonlinear and Dynamic Programming. Addison-Wesley,
Reading, Mass., 1964.
[81] G. Hadley and T. M. Whitin. Analysis ofInventory Systems. Prentice-Hall,
Englewood Cliffs, N.J., 1963.
[82] B. Harris. Theory ofProbability. Addison-Wesley, Reading, Mass., 1966.
[83] F. W. Harris. How many parts to make at once. Factory, The Magazine of
Management, 10:135-136, 1913.
[84] A. C. Hax and D. Candea. Production and Inventory Management. Pren-
tice Hall, Englewood Cliffs, N.J., 1984.
[85] K. M. Heal, M. L. Hansen, and K. M. Rickard. Maple V Learning Guide.
Springer-Verlag, New York, 1998.
[86] A. Heck. Introduction to Maple. Springer-Verlag, New York, 2nd edition,
1996.

[87] J. M. Henderson and R. E. Quandt. Microeconomic Theory: A Mathemati-

cal Approach. McGraw-Hili, New York, 1958.
[88] D. P. Heyman and M. J. Sobel. Stochastic Models in Operations Research,
Volume I: Stochastic Processes and Operating Characteristics. McGraw-
Hill, New York, 1982.
[89] D. P. Heyman and M. J. Sobel. Stochastic Models in Operations Research,
Volume II: Stochastic Optimization. McGraw-Hili, New York, 1984.
[90] D. P. Heyman and M. 1. Sobel, editors. Stochastic Models. Handbooks
in Operations Research and Management Science. North-Holland, Ams-
terdam, 1990.
 References 455

[91] F. B. Hildebrand. Methods ofApplied Mathematics. Prentice-Hall, Engle-

wood Cliffs, N.J., 2nd edition, 1965.

[92] F. S. Hillier and G. J. Liebennan. Introduction to Operations Research.

Holden-Day, Oakland, Calif., 4th edition, 1986.

[93] C. C. Holt, F. Modigliani, J. F. Muth, and H. A. Simon. Planning Produc-

tion, Inventories, and Work Force. Prentice-Hall, Englewood Cliffs, N.J.,
1960.

[94] I. Horowitz. Decision Making and the Theory of the Firm. Holt, Rinehart
and Winston, New York. 1970.

[95] R. A. Howard. Dynamic programming. Management Science, 12:317-

348,1966.

[96] J. P.Ignizio. Linear Progamming in Single- and Multiple Objective Sys-

tems. Prentice-Hall, Englewood Cliffs, N.J., 1982.

[97] R. B. Israel. Calculus the Maple Way. Addison-Wesley, Don Mills, On-
tario, 1996.

[98] J. R. Jackson. Networks of waiting lines. Operations Research, 5:518-

521, 1957.

[99] M. I. Kamien and N. L. Schwartz. Dynamic Optimization: The Calculus

of Variations and Optimal Control in Economics and Management. North-
Holland, New York, 1981.

[l00] L. V. Kantorovich. Mathematical Models of Organizing and Planning

Production. Leningrad State University, Leningrad, 1939. In Russian-
English translation appeared in Management Science, 6, pp. 366-422,
(1959-60).

[101] E. P. C. Kao. An Introduction to Stochastic Processes. Duxbury, Belmont,

Calif., 1997.

[102] S. Karlin and H. M. Taylor. A First Course in Stochastic Processes. Aca-

demic Press, San Diego, 2nd edition, 1975.

[103] J. L. Kelly. A new interpretation ofinfonnationrate. Bell System Technical

Journal, 35:917-926,1956.

[104] W. D. Kelton, R. P. Sadowski, and D. A. Sadowski. Simulation with Arena.

McGraw-Hill, Boston, 1998.

[105] D. G. Kendall. Some problems in the theory of queues. Journal of the

Royal Statistical Society, Series B, 13:151-185, 1951.
456 References

[106] D. G. Kendall. Stochastic processes occmring in the theory of queues and

their analysis by the method of imbedded Markov chains. Annals ofMath-
ematical Statistics, 24:338-354, 1953.

[107] M. Khouja. The single-period (news-vendor) problem: Literature review

and suggestions for future research. Omega, 27:537-553, 1999.

[108] D. E. Kirk. Optimal Control Theory. Prentice-Hall, Englewood Cliffs,

New Jersey, 1970.

[109] C. F. Klein and W. A. Gruver. Dynamic optimization in Markovian queue-

ing systems. In E. Roxin and P. T. Liu, editors, Kingston Conference on
Differential Games and Control Theory, pages 95-118. Marcel Dekker,
New York, 1978.

[110] L. Kleinrock. Queueing Systems, Volume J: Theory. John Wiley, New

York,1975.

[Ill] G. Klimek and M. Klimek. Discovering Curves and Surfaces with Maple.
Springer-Verlag, New York, 1997.

[112] P. J. Kolesar, K. L. Rider, T. B. Crabill, and W. E. Walker. A queueing-

linear programming approach to scheduling police patrol cars. OperatiOns
Research, 23:1045-1062,1975.

[113] B. O. Koopman. Air-terminal queues under time-dependent conditions.

Operations Research, 6:1089-1114,1972.

[114] H. W. Kuhn and A. W. Tucker. Nonlinear programming. In J. Neyman,

editor, Proceedings of2nd Berkeley Symposium on Mathematical Statistics
and Probability, Berkeley, 1951. University of California Press.

[115] P. J. M. Laarhoven and E. H. L. Aarts. Simulated Annealing: Theory and

Applications. Kluwer Academic, Boston, 1987.

[116] M. S. Lane, A. H. Mansour, and 1. L. Harpell. Operations research tech-

niques: A longitudinal update 1973-1988. Interfaces, 23(2):63-68, 1993.

[117] R. E. Larson and 1. L. Casti. Principles of Dynamic Programming. Part

I: Basic Analytic and Computational Methods. Marcel Dekker, New York,
1978.

[118] A. M. Law and W. D. Kelton. Simulation Modeling and Analysis.

McGraw-Hili, New York, 1982.

[119] H. Lee and C. A. Yano. Production control for multi-stage systems with
variable yield losses. Operations Research, 36:269-278, 1988.
 References 457

[120] H. L. Lee and S. Nahmias. Single-product, single-location models. In

S. C. Graves, A. H. G. Rinnooy Kan, and P. H. Zipkin, editors, Logistics of
Production and Inventory, pages 3-55. North-Holland, Amsterdam, 1993.
[121] B. Leonardz. To Stop or Not to Stop: Some Elementary Optimal Stopping
Problems with Economic interpretations. Almqvist and Wtksell, Stock-
holm,1974.
[122] D. V. Lindley. The theory of queues with a single server. Proceedings of
Cambridge Philosophical Society, 48:277-289,1952.
[123] S. A. Lippman and J. J. McCall. The economics of job search: A survey.
Part I. Economic Inquiry, 14:155-189, 1976.
[124] S. A. Lippman and J. J. McCall. The economics of job search: A survey.
Part II. Economic Inquiry, 14:347-368, 1976.
[125] J. D. C. Little. A proof for the queueing formula L = ..tW. Operations
Research,9:383-387, 1961.
[126] R. F. Love, J. G. Morris, and G. o. Wesolowsky. Facilities Location: Mod-
els and Methods. North-Holland, New York, 1988.
[127] D. G. Luenberger. Linear and Nonlinear Programming. Addison-Wesley,
Reading, Mass., 1984.
[128] D. G. Luenberger. Investment Science. Oxford University Press, New
York,I998.
[129] J. Medhi. Recent Developments in Bulk Queueing Models. Wiley Eastern
Limited, New Delhi, 1984.
[130] J. Medhi. Stochastic Models in Queuing Theory. Academic Press, New
York,I991.
[131] J. Medhi. Stochastic Processes. John Wiley, New York, 1994.
[t 32] M. B. Monagan, K. O. Geddes, G. Labahn, and S. Vorkoetter. Maple V
Programming Guide. Springer-Verlag, New York, 1996.
[133] J. Mossin. Optimal multi-period portfolio policies. Journal of Business,
41:215-229,1968.
[134] S. Nahmias. Inventory models. In A. Holzman J. Belzer and A. Kent, ed-
itors, The Encyclopedia of Computer Sciences and Technology, volume 9,
pages 447-483. Marcel Dekker, New York, 1978.
[135] S. Nahmias. Production and Operations Analysis. Irwin, Homewood, III.,
2nd edition, 1993.
458 References

[136] S. N. Neft~i. An Introduction to Financial Derivatives. Academic Press,

San Diego, 1996.

[137] R. Nelson. Probability, Stochastic Processes, and Queueing Theory: The

Mathematics of Computer Performance Modeling. Springer-Verlag, New
York,I995.

[138] G. L. Nemhauser. Introduction to Dynamic Programming. John Wiley,

New York, 1966.

[139] Operations Research Center, MIT. Notes on Operations Research 1959.

The Technology Press, Cambridge, Massachusetts, 1959.

[140] M. Parlar. A decomposition technique for an optimal control problem

with "PQDZ" cost and bounded controls. IEEE Transactions on Automatic
Control, AC-27(4):947-951, 1982.

[141] M. Parlar. Optimal dynamic service rate control in time dependent

MlM/S/N queues. International Journal of Systems Science, 15( 1): 107-
118,1984.

[142] M. Parlar. EXPIM: A knowledge-based expert system for produc-

tion/inventory modelling. International Journal of Production Research,
27(1):101-118,1989.

[143] M. Parlar. Probabilistic analysis of renewal cycles: An application to a

non-Markovian inventory problem with multiple objectives. Operations
Research, 48(2), 2000.

[144] M. Parlar and Y. Gerchak. Control of a production system with vari-

able yield and random demand. Computers and Operations Research,
16(4):315-324, 1989.

[145] M. Parlar and R. Rempala. Stochastic inventory problem with piecewise

quadratic holding cost function containing a cost-free interval. Journal of
Optimization Theory and Applications, 75(1): 133-153, 1992.

[146] M. Parlar and R. G. Vickson. An optimal control problem with piecewise-

quadratic cost functional containing a "dead-zone". Optimal Control Ap-
plications and Methods, 1:361-372, 1980.

[147] M. Parlar and Z. K. Weng. Designing a firm's coordinated manufacturing

and supply decisions with short product life-cycles. Management Science,
43(10):1329-1344,1997.

[148] A. L. Peressini, F. E. Sullivan, and J. J. UbI. The Mathematics ofNonlinear

Programming. Springer-Verlag, New York, 1988.
 References 459

[149] S. R. Pliska. Management and optimization of queueing systems. In

S. Ozekici, editor, Queueing Theory and Applications, pages 168-187.
Hemisphere Publishing Corporation, New York, 1990.
[150] E. L. Porteus. Numerical comparisons of inventory policies for periodic
review systems. Operations Research, 33( I): 134--152, January/February
1985.
[151] E. L. Porteus. Stochastic inventory theory. In Heyman and Sobel [90],
pages 605-652.
[152] N. U. Prabhu. Stochastic Processes. Macmillan, New York, 1965.
[153] A. A. B. Pritsker, J. J. O'Reilly, and D. K. LaVal. Simulation with VISual
SLAM and AweSim. John Wiley, New York, 1997.
[154] D. Redfern and E. Chandler. Maple O.D.E. Book. Springer-Verlag, New
York,1996.
[155] S. M. Roberts and J. S. Shipman. Two-Point Boundary Value Problems:
Shooting Methods. American Elsevier, New York, 1972.
[156] S. Ross. Applied Probability Models with Optimization Applications.
Holden Day, San Francisco, Calif., 1970.
[157] S. Ross. Stochastic Processes. John Wiley, New York, 1983.
[158] S. Ross. Introduction to Probability Models. Academic Press, Orlando,
Fla., 3rd edition, 1985.
[159] S. M. Ross. Introduction to Stochastic Dynamic Programming. Academic
Press, New York, 1983.
[160] S. M. Ross. A Course in Simulation. Macmillan, New York, 1991.
[161] T. L. Saaty. Mathematical Methods of Operations Research. McGraw-
Hill, New York, 1959.
(162] A. P. Sage and C. C. White. Optimum Systems Control. Prentice Hall,
Englewood Cliffs, N.J., 2nd edition, 1977.
[163] i. ~ahin. Regenerative Inventory Systems: Operating Characteristics and
Optimization. Springer-Verlag, New York, 1990.
[164] M. Sasieni, A. Yaspan, and L. Friedman. Operations Research: Methods
and Problems. John Wiley, New York, 1959.
(165] M. Vos Savant. The Power of Logical Thinking. St. Martin's Griffin, New
York,I996.
460 References

[166] H. Scarf. The optimality of (S, s) policies in the dynamic inventory prob-
lem. In K. J. Arrow, S. Karlin, and P. Suppes, editors, Mathematical
Methods in the Social Sciences. Stanford University Press, Stanford, Calif.,
1960.

[167] J. W. Schmidt and R. P. Davis. Foundations of Analysis in Operations

Research. Academic Press, New York, 1981.
[168] L. B. Schwarz, editor. Multi-Level Production/Inventory Control Systems:
Theory and Practice. Studies in the Management Sciences, Volume 16.
North-Holland, Amsterdam, 1981.

[169] S. P. Sethi and G. L. Thompson. Optimal Control Theory: Applications to

Management Science. Martinus NijhoffPublishing, Boston, 1981.
[I70] M. Shaked and J. G. Shantikumar. Reliability and maintainability. In Hey-
man and Sobel [90], pages 653-713.

[171] C. Shannon. A mathematical theory of communication. Bell System Tech-

nical Journal, 27:379--423, 1948.
[172] W. Shih. Optimal inventory policies when stockout results from defec-
tive products. International Journal of Production Research, 18:677--{)86,
1980.

[173] E. A. Silver. Establishing the reorder quantity when the amount received
is uncertain. INFOR, 14:32-29, 1976.

[174] E. A. Silver. Operations research in inventory management: A review and

critique. Operations Research, 29:628--{)45, 1981.

[175] E. A. Silver and R. Peterson. Decision Systems for Inventory Management

and Production Planning. John Wiley, New York, 2nd edition. 1985.
[176] J. Singh. Operations Research. Penguin, Hannondsworth, England, 1968.

[177] B. D. Sivazlian and L. E. Stanfel. Analysis of Systems in Operations Re-

search. Prentice-Hall, Englewood Cliffs, 1975.
[178] J. S. Smart. The Sonnets ofMilton. Maclehose, Jackson. Glasgow, 1921.

[I79] M. J. Sobel. Optimal average cost policy for a queue with start-up and
shut-down costs. Operations Research, 17:145-162, 1969.

[180] S. Stidham. L = AW: A discounted analogue and a new proof. Operations

Research, 20:1115-1126,1972.
[181] D. Stirzaker. Elementary Probability. Cambridge University Press, Cam-
bridge, 1994.
 References 461

[182] H. A. Taha. Operations Research: An Introduction. Macmillan, New York,

1987.

[183] L. Takacs. Stochastic Processes. Methuen, London, 1966.

[184] H. M. Taylor. Optimal replacement under additive damage and other fail-
ure models. Naval Research Logistics Quarterly, 22:1-18,1975.
[185] R. J. Tersine. Principles of Inventory and Materials Management. North
Holland, New York, 3rd edition, 1988.
[186] H. C. Tijms. Stochastic Modeling and Analysis. John Wiley, Chichester,
1986.
[187] H. C.Tijms. Stochastic Models: An Algorithmic Approach. John Wiley,
Chichester, 1994.
[188] A. F. Veinott. Optimal policy for a multi-product, dynamic nonstationary
inventory problem. Management Science, 12:206-222, 1965.
[189] R. G. Vickson. Lecture notes in MS 635, 1993. Department of Manage-
ment Sciences, University of Waterloo.
[190] H. M. Wagner. Principles of Operations Research with Applications to
Managerial Decisions. Prentice-Hall, Englewood Cliffs, N.J., 1969.
[191] H. M. Wagner and T. M. Whitin. Dynamic version of the economic lot size
model. Management Science, 5:89-96, 1958.
[192] A. Waldo Sequential Analysis. John Wiley, New York, 1947. Republished
by Dover in 1973.
[193] Z. K. Weng and M. Parlar. Integrating early sales with production deci-
sions: Analysis and insights. liE 7ransactions on Scheduling and logis-
tics, 31(11):1051-1060,1999.
[194] P. Wilmott. Derivatives: The Theory and Practice of Financial Engineer-
ing. John Wiley, Chichester, 1998.
[195] R. H. Wilson. A scientific routine for stock control. Harvard Business
Review, 13:116-128, 1934.
[196] R. Wolff. Stochastic Modeling and the Theory of Queues. Prentice-Hall,
Englewood Cliffs, N.J., 1989.
[197] C. A. Yano and H. L. Lee. Lot sizing with random yields: A review. 0p-
erations Research, 43:311-334,1995.
[198] S. Zionts. Linear and Integer Programming. Prentice-Hall, Englewood
Cliffs, N.J., 1974.
462 References

[199] P. H. Zipkin. Foundations of Inventory Management. McGraw-Hili,

Boston, 2000.
Index

->,8 bivariate normal

:,6 plot of density, 25
:=,8
;,6 Chapman-Kolmogorov equations, 276
charpoly,62
%1,369 Cholesky factorization, 164
%2,369 Cobb-Douglas production function,
179
adaptive random search coeff,217
optimization using, 441 collect, 13
advertising policies, 127 combine,7
algsubs, 217 complementary slackness, 171
all values, 41, 342, 368 constraint qualification, 174
alternating renewal process, 265 continuous-time Markov chain, 306
arrays, II balance equations, 319
assign, 11,369 exponential matrix, 313
assume, 8, 307 infinitesimal generator, 306
augment, 59 Kolmogorov differential equa-
tions, 308
balance equations, 319 limiting probabilities, 318
basic solution, 105 contourplot,22
basic solution set, 105 convert, II
basis,63 convex function, 68, ISS
basis matrix, 105 convex set, 101, 154
birth and death process, 306, 377 convexity
464 Index

relation to Hessian, 160 policy, 194

cycle length, 425 stage, 194
state, 193
D,45 transformation, 194
definite, 67, 159 dynamic simulation
derivatives, 45 inventory model
describe, 17 random yield, 436
det, 56
diagonalization, 61 eigenvalUes, 61, 159
diff,45 eigenvectors, 61
difference equation eigenvectors, 62
nonlinear, 229 elementary renewal theorem, 261
differential equation, 15 empirical,91
analytic solution, 69 EOQ model, 335
infolevel,31 planned backorders, 338
numerical solution, 22 equation
partial,77 analytic solution, 14
savings account, 69 difference (recurrence), 79
system numerical solution, 16
in optimal control, 73 solution of cubic, 38
Lancaster's, 72 solution of quadratic, 38
numerical solution, 74 equations
Van der Pol's differential-difference, 244
numerical solution, 70 homogeneous linear system, 60
Digits, 277 linear system, 58
display, 101 nonlinear system, 41
dsol ve, 23, 72, 76, 241, 395 eval,8
dual,139 evalf,16
dual price, 131 int, 433
duality, 136 evalm, 12,55
economic interpretation, 138 expand,6
in game theory, 143 exponential,313
main results, 140 exponential matrix, 313
dynamic inventory model exporting to
dynamic programming formu- FORTRAN,32
lation, 361 LaTeX, 32
optimal base-stock policy, 362 extrema, 68
optimality of (s, S) policy, 366 extreme point, 101
dynamic programming
backward recursion, 195 factor, 6
continuous time, 211 feasible points, 105
cost-to-go (value) function, 196 Fibonacci sequence, 79
decision, 194 finance, 33
discount factor, 211, 213 fit, 64
forward recursion, 195 fsolve, 17,40, 170
 Index 465

futurevalue,29 inttrans, 242, 258

inventory, 331
G/G/l classification, 332
simulation, 438 reasons for keeping, 332
gambling model, 225 inventory cost
geneqns,58 holding, 334
generating function, 81, 85,296,302 penalty, 335
as opposed to z-transfonn, 85 procurement, 334
numerical inversion, 87 inventory model
genfunc,298,302,388 backorder cost function, 161
genmatrix,107 backorders, 24
grad,156 constrained multi-item, 41
gradient, 67 continuous-review, 49
gradient vector, 156 lost sales, 46
graphics periodic review, 274, 277
three-dimensional, 24 periodic-review
two-dimensional, 20 stationary distribution, 279
quantity discounts, 27
Hamilton-Jacobi-Bellman equation, random yield
213 simulation, 436
hazard rate, 240 stochastic, 14
Hessian, 66, 156 inverse, 56
hessian, 157 invlaplace,81
histogram, 18 is,68
iso-profit lines, 103
ifactor, 16 isolate, 359
ilp,146
implicitdiff,48 Jackson network, 398
implici tplot, 22 Jacobian, 67, 95
implied backorder costs, 343
indefinite, 157 key renewal theorem, 264
inequal, 101 Kolmogorov differential equations
inequality backward, 310
solution, 39 forward, 313
infeasible basic solution, 106 Kolmogorov-Smimov test, 426
infinite series, 54 Kuhn-Tucker conditions, 171
infolevel, 113 geometric interpretation, 173
infolevel,30
integer programming, 145 I'Hopital's rule, 296
integral, 50 Lagrange multipliers, 171
multiple, 53 economic interpretation, 181
integration Lagrangian dual, 186
infolevel,30 Lagrangian function, 176
interface, 29 laplace, 82
intersect, 10 Laplace transfonn, 81
466 Index

numerical inversion, 83 Internet resources, xiii

least squares, 64 knowledge base, xi
leastsquare,64 Release 5.1 and 6, xiii
Leibniz's rule, 47, 358 users, xi
limit, 44 Markov chain, 61, 272
limiting (steady-state) probabilities, aperiodic, 288
278 communicating states, 287
linalg, 55, 302 equivalence class, 288
linear programming, 19 ergodic, 290
degeneracy, 119 first passage time probability, 288
graphical solution, 100 imbedded, 293
infeasibility, 118 irreducible, 288
mixed constraints, 116 machine maintenance, 283
number of iterations, 107 Markovian property, 273
simplex, 104 mean recurrence time, 290
transportation model, 121 null recurrent, 290
unbounded solution, 119 pathological case, 286
linear system period of a state, 288
homogeneous, 60 positive recurrent, 290
solution, II reachable states, 287
linear-quadratic control, 187,201 recurrent state, 288
infinite stage, 207 transient behavior, 301
relation to nonlinear program- transient state, 288
ming,204 matrix multiplication, 55
linsol ve, 58 matrix operations, 55
lists, to matrixDE,311
location problem, 163, 169 Milton's sonnets, 375
minus, 280
M(t)/M(t)/S/N mixed-congruential method, 424
dynamic service rate control, 406 Monte Carlo simulation, 432
M/Ek/ I,299 numerical evaluation ofdefinite
M/ M/l, 298, 378 integrals, 432
controlled arrivals, 404 multiply, 129
M/M/I/I M X /M/I,387
transient solution, 390 myopic solution, 227
M/M/l/K,382
M/M/c,383 negative definite, 157
M/M/oo negative semidefinite, 157
transient solution, 391 nonbasic variables, 105
M/M/oo,78 nonhomogeneous Poisson process, 247
makeproc,79 nonlinear programming, 153
map, 12 duality theorem, 186
Maple NONNEGATIVE, 126
books on, 6 nops,11
help facility, 5 normal,7
 Index 467

odetest, 69, 72 Pushkin, 273

op, 10,30
open-loop decision, 220 quadratic form, 157
operations research quantity discounts, 346
definition, 2 queueing networks, 394
journals, 2 queueing notation, 376
origins, 1 queueing system
without mathematics, 37 optimization, 401
optimal stopping, 228
infinite stage, 230 random, 420
option pricing uniform, 426
blackscholes,34 random variable
orientation beta, 231
in plot3d, 26 binomial, 418
output=listprocedure, 410 moment generating function,
55
partial differential equation, 77 simulation, 420
partial fraction expansion, 82 Cauchy, 94
pdsolve,77 continuous
piecewise, 27, 222, 347, 409 pdf, cdf, icdf, 89
piecewise-quadratic cost, 189 discrete
plot,249 pf, dcdf, idcdf, 89
plot3d, 25, 282 Erlang, 242
Poisson process, 243,308 density, 246, 299
compound, 252 exponential, 14,52,238
nonhomogeneous, 247 memory less property, 239
Pollaczek-Khinchine formula, 298 geometric, 55
Pontryagin's principle, 408 normal, 16, 89
portfolio selection, 154, 171 mean and variance, 92
positive definite, 157 Poisson, 90
positive definiteness, 67 triangular, 28
positive semidefinite, 157 uniform, 423
principal minor, 158 unit normal, 52
principle of optimality, 195 Weibull, 241
probabilistic inventory model plot of density, 20
continuous review with backo- random variate generation, 18
rders exponential, 429
approximate formulation, 349 inverse transform method, 430
exact formulation, 354 random, 430
one period (newsvendor), 358 random vector
yield randomness, 367 joint density, 95
probability random walk, 272
conditional, 277 randvector, 129
unconditional, 277 rank,58
programming, 32 renewal density, 259
468 Index

renewal equation, 262 successive approximations, 207

renewal function, 259 sum, 170,385
renewal process, 256
renewal reward theorem, 268 tallyinto, 18
use in inventory models, 335 time~ependentarrivalrate, 75
rgf _encode, 85 transform, 18
rgf _expand, 85 transportation queueing process, 402
rgf _expand, 303 type=nurneric, 23, 70, 76
rsolve, 79, 229, 383
unapply,8, 10,304,355
Runge-Kutta method, 395
union, 10,280
saddle-point conditions, 186
work force planning model, 219
select, 437
sensitivity analysis zero-sum game
adding a new decision variable, solution using linear program-
135 ming, 124
change in objective function, 134 ztrans,85
change in RHS values, 131
serial queue with blocking, 394
sets, 10
simplex, 19
final table, 115
simplex, 113
not useful in integer program-
ming,145
simplify, 7, 370
simulation, 417
number of runs, 421
programs, 418
runlength,421
slack variable, 104
solve, 16,171,181
sprint, 315
stagecoach problem, 197
statevalf,89
static simulation
car and goats problem, 434
s ta ts, 17, 64,426
distributions, 430
statsplots,18
stochastic matrix, 272
stochastic process, 237
strong law oflarge numbers, 433
student, 248
subs,8