Introduction to

Discrete Event Systems

Second Edition

by

Christos G. Cassandras Stephane Lafortune

Boston University The University of Michigan

fyA Springer
Table of Contents

Preface - Second Edition xv

Preface xvii
Organization of Book xxi

Systems and Models l

1.1 INTRODUCTION . . 1
1.2 SYSTEM AND CONTROL BASICS 2
1.2.1 The Concept of System 2
1.2.2 The Input-Output Modeling Process 2
1.2.3 The Concept of State 6
1.2.4 The State Space Modeling Process 8
1.2.5 Sample Paths of Dynamic Systems 13
1.2.6 State Spaces 15
1.2.7 The Concept of Control 20
1.2.8 The Concept of Feedback 22
1.2.9 Discrete-Time Systems 25
1.3 DISCRETE EVENT SYSTEMS 26
1.3.1 The Concept of Event 27
1.3.2 Characteristic Properties of Discrete Event Systems 30
1.3.3 The Three Levels of Abstraction in the Study of Discrete Event Systems 33
1.3.4 Examples of Discrete Event Systems 35
1.3.5 Hybrid Systems 43
1.4 SUMMARY OF SYSTEM CLASSIFICATIONS 44
1.5 THE GOALS OF SYSTEM THEORY 46
SUMMARY : . . . . 48
PROBLEMS .48
SELECTED REFERENCES 50

Languages and Automata 53

2.1 INTRODUCTION 53
2.2 THE CONCEPTS OF LANGUAGES A N D A U T O M A T A 54
2.2.1 Language Models of Discrete-Event Systems 54
2.2.2 Automata . ". 59
2.2.3 Languages Represented by Automata 62
2.2.4 Nondeterministic Automata 69
2.2.5 Automata with Inputs and Outputs 72
viii | Table of Contents

2.3 OPERATIONS ON AUTOMATA 74

2.3.1 Unary Operations 75
2.3.2 Composition Operations 77
2.3.3 State Space Refinement 85
2.3.4 Observer Automata 87
2.3.5 Equivalence of Automata 90
2.4 FINITE-STATE AUTOMATA 92
2.4.1 Definition and Properties of Regular Languages 92
2.4.2 Regular Expressions 95
2.4.3 State Space Minimization 96
2.5 ANALYSIS OF DISCRETE-EVENT SYSTEMS 100
2.5.1 Safety and Blocking Properties 101
2.5.2 Partially-Observed DES 102
2.5.3 Event Diagnosis 108
2.5.4 Software Tools and Computational Complexity Issues 117
2.5.5 Formal Verification and Model Checking 118
SUMMARY 119
PROBLEMS 120
SELECTED REFERENCES 130

3 Supervisory Control 133

3.1 INTRODUCTION 133
3.2 FEEDBACK CONTROL WITH SUPERVISORS 135
3.2.1 Controlled Discrete Event Systems 135
3.2.2 Control Under Partial Observation 137
3.3 SPECIFICATIONS ON CONTROLLED SYSTEM 139
3.3.1 Modeling of Specifications as Automata 140
3.3.2 The Need for Formal Methods 143
3.4 CONTROL WITH PARTIAL CONTROLLABILITY 145
3.4.1 Controllability Theorem 145
3.4.2 Realization of Supervisors 148
3.4.3 The Property of Controllability 151
3.4.4 Some Supervisory Control Problems and Their Solutions 156
3.4.5 Computation of K^c: Prefix-Closed Case 159
3.4.6 Computation of KiC 161
3.5 NONBLOCKING CONTROL 163
3.5.1 Nonblocking Controllability Theorem 163
3.5.2 Nonblocking Supervisory Control 164
3.5.3 Computation of K^c: General Case 167
3.5.4 Dealing with Blocking Supervisors 170
3.6 CONTROL WITH MODULAR SPECIFICATIONS 174
3.7 CONTROL UNDER PARTIAL OBSERVATION 178
3.7.1 Controllability and Observability Theorem 178
3.7.2 Realization of P-Supervisors 185
3.7.3 The Property of Observability 188
3.7.4 Supervisory Control Problems Under Partial Observation 193
3.7.5 The Property of Normality 195
 Table of Contents IX

3.8 DECENTRALIZED CONTROL 199

3.8.1 Conjunctive Architecture 201
3.8.2 Disjunctive Architecture 205
3.8.3 Combined Architecture 208
3.8.4 Realization of Decentralized Supervisors 210
3.8.5 The Property of Coobservability 210
3.8.6 Undecidability in Decentralized Control 211
SUMMARY 212
PROBLEMS 213
SELECTED REFERENCES 219

Petri Nets 223

4.1 INTRODUCTION 223
4.2 PETRI NET BASICS 224
4.2.1 Petri Net Notation and Definitions 224
4.2.2 Petri Net Markings and State Spaces 226
4.2.3 Petri Net Dynamics 227
4.2.4 Petri Net Languages 231
4.2.5 Petri Net Models for Queueing Systems 233
4.3 COMPARISON OF PETRI NETS AND AUTOMATA 236
4.4 ANALYSIS OF PETRI NETS 239
4.4.1 Problem Classification 239
4.4.2 The Coverability Tree 244
4.4.3 Applications of the Coverability Tree 247
4.4.4 Linear-Algebraic Techniques 250
4.5 CONTROL OF PETRI NETS 253
4.5.1 Petri Nets and Supervisory Control Theory 254
4.5.2 State-Based Control of Petri Nets 257
SUMMARY 260
PROBLEMS 261
SELECTED REFERENCES ' 266

Timed and Hybrid Models 269

5.1 INTRODUCTION 269
5.2 TIMED AUTOMATA 270
5.2.1 The Clock Structure 271
5.2.2 Event Timing Dynamics 275
5.2.3 A State Space Model 278
5.2.4 Queueing Systems as Timed Automata 283
5.2.5 The Event Scheduling Scheme 285
5.3 TIMED PETRI NETS 286
5.3.1 Timed Petri Net Dynamics 288
5.3.2 Queueing Systems as Timed Petri Nets 290
5.4 DIOID ALGEBRAS 292
5.4.1 Basic Properties of the (max, +). Algebra 292
5.4.2 Modeling Queueing Systems in the (max, +) Algebra 294
5.5 ALTERNATIVE TIMED MODELS 297
 Table of Contents

5.6 TIMED AUTOMATA WITH GUARDS 299

5.6.1 Model Definition 300
5.6.2 Model Execution 303
5.6.3 Parallel Composition 305
5.6.4 Untiming 307
5.7 HYBRID MODELS 311
5.7.1 Hybrid Automata 312
SUMMARY , 318
PROBLEMS 319
SELECTED REFERENCES 324

6 Stochastic Timed Automata 327

6.1 INTRODUCTION 327
6.2 STOCHASTIC PROCESS BASICS 328
6.2.1 Continuous-state and Discrete-state Stochastic Processes 329
6.2.2 Continuous-time and Discrete-time Stochastic Processes 329
6.2.3 Some Important Classes of Stochastic Processes 329
6.3 STOCHASTIC CLOCK STRUCTURES 333
6.4 STOCHASTIC TIMED AUTOMATA 334
6.5 THE GENERALIZED SEMI-MARKOV PROCESS 336
6.5.1 Queueing Systems as Stochastic Timed Automata 339
6.5.2 GSMP Analysis 340
6.6 THE POISSON COUNTING PROCESS 341
6.7 PROPERTIES OF THE POISSON PROCESS 347
6.7.1 Exponentially Distributed Interevent Times 347
6.7.2 The Memoryless Property 348
6.7.3 Superposition of Poisson Processes 351
6.7.4 The Residual Lifetime Paradox 353
6.8 AUTOMATA WITH POISSON CLOCK STRUCTURE 355
6.8.1 Distribution of Interevent Times 356
6.8.2 Distribution of Events 357
6.8.3 Markov Chains 359
6.9 EXTENSIONS OF THE GSMP 360
SUMMARY 362
PROBLEMS 364
SELECTED REFERENCES 367

7 Markov Chains 369

7.1 INTRODUCTION 369
7.2 DISCRETE-TIME MARKOV CHAINS 370
7.2.1 Model Specification 370
7.2.2 Transition Probabilities and the Chapman-Kolmogorov Equations . . 371
7.2.3 Homogeneous Markov Chains 372
7.2.4 The Transition Probability Matrix 374
7.2.5 State Holding Times 377
7.2.6 State Probabilities 378
7.2.7 Transient Analysis 378
7.2.8 Classification of States 382
 Table of Contents | xi

7.2.9 Steady State Analysis 391

7.2.10 Irreducible Markov Chains 392
7.2.11 Reducible Markov Chains 397
7.3 CONTINUOUS-TIME MARKOV CHAINS 399
7.3.1 Model Specification : 400
7.3.2 Transition Functions 400
7.3.3 The Transition Rate Matrix 401
7.3.4 Homogeneous Markov Chains 402
7.3.5 State Holding Times 402
7.3.6 Physical Interpretation and Properties of the Transition Rate Matrix . 403
7.3.7 Transition Probabilities 405
7.3.8 State Probabilities 407
7.3.9 Transient Analysis 407
7.3.10 Steady State Analysis 410
7.4 BIRTH-DEATH CHAINS 412
7.4.1 The Pure Birth Chain 414
7.4.2 The Poisson Process Revisited 415
7.4.3 Steady State Analysis of Birth-Death Chains 415
7.5 UNIFORMIZATION OF MARKOV CHAINS 417
SUMMARY 421
PROBLEMS 422
SELECTED REFERENCES 427

8 Introduction to Queueing Theory 429

8.1 INTRODUCTION 429
8.2 SPECIFICATION: OF QUEUEING MODELS 430
8.2.1 Stochastic Models for Arrival and Service Processes 430
8.2.2 Structural Parameters 431
8.2.3 Operating Policies 431
8.2.4 The A/B/m/K Notation 432
8.2.5 Open and Closed Queueing Systems 434
8.3 PERFORMANCE OF A QUEUEING SYSTEM 434
8.4 QUEUEING SYSTEM DYNAMICS . 437
8.5 LITTLE'S LAW 439
8.6 SIMPLE MARKOVIAN QUEUEING SYSTEMS 442
8.6.1 The M/M/l Queueing System 444
8.6.2 The M/M/m Queueing System 448
8.6.3 The M/M/oo Queueing System 452
8.6.4 The M/M/l/K Queueing System 454
8.6.5 The M/M/m/m Queueing System 458
8.6.6 The M/M/l//N Queueing System 459
8.6.7 The M/M/m/K/N Queueing System 461
8.7 MARKOVIAN QUEUEING NETWORKS ' . . . 462
8.7.1 The Departure Process of the M/M/l Queueing System ,. 464
8.7.2 Open Queueing Networks 467
8.7.3 Closed Queueing Networks 471
8.7.4 Product Form Networks 476
xii | Table of Contents

8.8 NON-MARKOVIAN QUEUEING SYSTEMS 478

8.8.1 T h e M e t h o d of Stages 479
8.8.2 Mean Value Analysis of t h e M/G/l Queueing System 482
8.8.3 Software Tools for t h e Analysis of General Queueing Networks . . . . 488
SUMMARY 490
PROBLEMS 491
SELECTED REFERENCES 496

9 Controlled Markov Chains 499

9.1 INTRODUCTION 499
9.2 APPLYING "CONTROL" IN MARKOV CHAINS 500
9.3 MARKOV DECISION PROCESSES -. . 502
9.3.1 Cost Criteria 503
9.3.2 Uniformization 504
9.3.3 The Basic Markov Decision Problem 506
9.4 SOLVING MARKOV DECISION PROBLEMS 510
9.4.1 The Basic Idea of Dynamic Programming 510
9.4.2 Dynamic Programming and the Optimality Equation 514
9.4.3 Extensions to Unbounded and Undiscounted Costs 524
9.4.4 Optimization of the Average Cost Criterion 532
9.5 CONTROL OF QUEUEING SYSTEMS 535
9.5.1 The Admission Problem 537
9.5.2 The Routing Problem 542
9.5.3 The Scheduling Problem 546
SUMMARY 552
PROBLEMS 553
SELECTED REFERENCES 554

10 Introduction to Discrete-Event Simulation 557

10.1 INTRODUCTION 557
10.2 THE EVENT SCHEDULING SCHEME 558
10.2.1 Simulation of a Simple Queueing System 561
10.3 THE PROCESS-ORIENTED SIMULATION SCHEME 573
10.4 DISCRETE-EVENT SIMULATION LANGUAGES 574
10.5 RANDOM NUMBER GENERATION 576
10.5.1 The Linear Congruential Technique 577
10.6 RANDOM VARIATE GENERATION 578
10.6.1 The Inverse Transform Technique 579
10.6.2 The Convolution Technique 582
10.6.3 The Composition Technique 583
10.6.4 The Acceptance-Rejection Technique 583
10.7 OUTPUT ANALYSIS 587
10.7.1 Simulation Characterizations 587
10.7.2 Parameter Estimation 589
10.7.3 Output Analysis of Terminating Simulations 595
10.7.4 Output Analysis of Non-Terminating Simulations 598
SUMMARY 604
PROBLEMS 605
SELECTED REFERENCES 614
 Table of Contents | xiii

11 Sensitivity Analysis and Concurrent Estimation 617

11.1 INTRODUCTION 617
11.2 SAMPLE FUNCTIONS AND THEIR DERIVATIVES 619
11.2.1 Performance Sensitivities 620
11.2.2 The Uses of Sensitivity Information 621
11.3 PERTURBATION ANALYSIS: SOME KEY IDEAS 623
11.4 PA OF GI/G/1 QUEUEING SYSTEMS .629
11.4.1 Perturbation Generation 630
11.4.2 Perturbation Propagation 634
11.4.3 Infinitesimal Perturbation Analysis (IPA) 639
11.4.4 Implementation of IPA for the GI/G/1 System 649
11.5 IPA FOR STOCHASTIC TIMED AUTOMATA 650
11.5.1 Event Time Derivatives 652
11.5.2 Sample Function Derivatives 655
11.5.3 Performance Measure Derivatives 657
11.5.4 IPA Applications 665
11.6 SENSITIVITY ESTIMATION REVISITED 670
11.7 EXTENSIONS OF IPA 673
11.7.1 Discontinuities due to Multiple Customer Classes 673
11.7.2 Discontinuities due to Routing Decisions 678
11.7.3 Discontinuities due to Blocking: IPA with Event
Rescheduling (RIPA) 680
11.8 SMOOTHED PERTURBATION ANALYSIS (SPA) 681
11.8.1 Systems with Real-Time Constraints 685
11.8.2 Marking and Phantomizing Techniques 687
11.9 IPA FOR STOCHASTIC HYBRID AUTOMATA 691
11.9.1 Stochastic Fluid Models (SFMs) 693
11.9.2 Sample paths of SFMs 695
11.9.3 Comparing SFMs to Their DES Counterparts 697
11.9.4 IPA for a Single-Class Single-Node SFM 700
11.9.5 IPA for SFMs with Multiple Classes, Multiple Nodes
and Feedback 705
11.10 PA FOR FINITE PARAMETER CHANGES 705
11.11 CONCURRENT ESTIMATION 706
11.11.1 The Sample Path Constructability Problem 707
11.11.2 Uses of Concurrent Estimation: "Rapid Learning" 709
11.11.3 Sample Path Constructability Conditions 710
11.11.4 The Standard Clock Approach 714
11.11.5 Augmented System Analysis 718
11.11.6 The "Time Warping" Algorithm 725
SUMMARY 730
PROBLEMS 732
SELECTED REFERENCES 736

I Review of Probability Theory 741

1.1 BASIC CONCEPTS AND DEFINITIONS 741
1.2 CONDITIONAL PROBABILITY 743
1.3 RANDOM VARIABLES 744
xiv | Table of Contents

1.4 CONDITIONAL DISTRIBUTIONS 745

1.5 FUNCTIONS OF RANDOM VARIABLES 746
1.6 EXPECTATION 747
1.7 CHARACTERISTIC FUNCTIONS 748
1.8 RANDOM SEQUENCES AND RANDOM PROCESSES 751

I I IPA Estimator 755

Index 761

About the Authors 771