to solve practical problems will be discussed.

An introduction , expectation and other fundamental issues are covered with the focus on their applications in the study of industrial systems. Stochastic models such zation, analysis, interpretation and presentation

mous, 1722), but a memoir prepared by models the operation of a system as a discrete sequence of events in time. Each event occurs at a particular instant in time and marks a change of state in the system.[1] Between consecutive events, no change in the system is assumed to occur; thus the simulation can directly jump in time from one event to e relates the odds of event to event , before (prior to) and after (posterior to) conditioning on another event . The odds on to event is simply the ratio of the probabilities of the as posterior is proportional to prior times likelihood, proportionality symbol means that the left hand side is proportional to (i
in continuous time, see continuous-time Markov chain.

where the

A simple two-state Markov chain

A Markov chain (discrete-time Markov chain or DTMC[1]) named after Andrey Markov, is a mathematical system that undergoes transitions from one state to another, between a finite or countable number of possible states. It is a random process usually characterized as memoryless: the next state depends only on the current state and not on the sequence of events that preceded it. This specific kind of "memorylessness" is called the Markov property. Markov chains have many applications as statistical models of real-world processes.

theory.

Andrey Markov introduced[citation needed] the notion of Markov chains (1906), which played an important r ernational Association for the Evaluation of Educational Achievement (IEA). "It is designed to measure childrens reading literacy achievement, to provide a baseline for future studies of trends in achievement, and to gather information about childrens home and school experiences in learning to read."[1] PIRLS 2006 tested 215,000 students from 46 educational systems.[1] PIRLS 2011 testing has been done and the results will be ed in discrete-event simulation is that the steady-state distributions of event times may not be known in advance. As a result, the initial set of events placed into the pending event set will not have arrival times representative of the steady-state distribution. This problem is typically solved by bootstrapping the simulation model. Only a limited effort is made to assign realistic times to the initial set of pending events. These evem's future can be predicted. In many applications, it is these statistical properties that are important. A famous Markov chain is the so-called "drunkard's walk", a random walk on the number line where, at each step, the position may change by +1 or 1 with equal probability. From any position there are two possible transitions, to the next or previous integer. The transition probabilities depend only on the current position, not on the manner in which the position was reached. For example, the transition probabilities from 5 to 4 and 5 to 6 are both 0.5, and all other transition probabilities from 5 are 0. These probabilities are independent of whether the system was previously in 4 or 6. Another example is the dietary habits of a creature who eats only grapes, cheese, or lettuce, and whose dietary habits conform to the following rules:

It eats exactly once a day. If it ate cheese today, tomorrow it will eat lettuce or grapes with equal probability. If it ate grapes today, tomorrow it will eat grapes with probability 1/10, cheese with probability 4/10 and lettuce with probability 5/10. If it ate lettuce today, it will not eat lettuce again tomorrow but will eat grapes with probability 4/10 or cheese with probability 6/10.

This creature's eating habits can be modeled with a Markov chain since its choice tomorrow depends solely on what it ate today, not what it ate yesterday or even farther in the past. One statistical property that could be calculated is the expected percentage, over a long period, of the days on which the creature will eat grapes. A series of independent events (for example, a series of coin flips) satisfies the formal definition of a Markov chain. However, the theory is usually applied only when the probability distribution of the next step depends non-trivially on the current state. Many other examples of Markov chains exist.

Formal definition
A Markov chain is a sequence of random variables X1, X2, X3, ... with the Markov property, namely that, given the present state, the future and past states are independent. Formally,

The possible values of Xi form a countable set S called the state space of the chain. Markov chains are often described by a directed graph, where the edges are labeled by the probabilities of going from one state to the other states.
Variations

a finite expectation. The mean recurrence time at state i is the expected return time Mi:

State i is positive recurrent (or non-null persistent) if Mi is finite; otherwise, state i is null recurrent (or null persistent). Expected number of visits It can be shown that a state i is recurrent if and only if the expected number of visits to this state is infinite, i.e.,

Absorbing states A state i is called absorbing if it is impossible to leave this state. Therefore, the state i is absorbing if and only if

If every state can reach an absorbing state, then the Markov chain is an absorbing Markov chain.
Ergodicity

A state i is said to be ergodic if it is aperiodic and positive recurrent. In other words, a state i is ergodic if it is recurrent, has a period of 1 and it has finite mean recurrence time. If all states in an irreducible Markov chain are ergodic, then the chain is said to be ergodic.

nts, however, schedule additional events, and with time, the distribution of event times approaches its steady state. This is called bootstrapping the simulation model. In gathering statistics from the running model, it is important to either disregard events that occur before the steady state is reached or to run the simulation for long enough that the bootstrapping behavior is overwhelmed by steady-state behavior. (This use of the term bootstrapping can be contrasted with its use in both statistics and computing.)
Statistics

The simulation typically keeps track of the system's statistics, which quantify the aspects of interest. In the bank example, it is of interest to track the mean waiting times.
Ending condition

Because events are bootstrapped, theoretically a discrete-event simulation could run forever. So the simulation designer must decide when the simulation will end. Typical choices are at time t or after processing n number of events or, more generally, when statistical measure X reaches the value x.

Simulation engine logic

ed in discrete-event simulation is that the steadyand Artemas Martin).[citation needed]