Scribd Logo
Upload

Welcome to Scribd!

  • 24 views

    Worksheet 1 - Stoch Mod - ENSIA 2024-2025

    Uploaded by

    abdelizohr

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    Worksheet 1 - Stoch Mod - ENSIA 2024-2025

    Uploaded by

    abdelizohr
    24 views2 pages

    Original Title

    Worksheet 1- Stoch Mod - ENSIA 2024-2025

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    24 views2 pages

    Worksheet 1 - Stoch Mod - ENSIA 2024-2025

    Uploaded by

    abdelizohr

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    You are on page 1of 2

    Stochastic Modeling and Simulation

    Semester 5
    2024/2025

    Worksheet 1
    Exercice 1 Suppose that the weather from one day to the next is described by a Markov chain on
    the 0; 1; 2 (0 for sunny, 1 for cloudy, 2 for rainy) whose transition matrix is given by
    0 1
    1=2 1=2 0
    @1=4 1=2 1=4A :
    0 1=2 1=2

    It’s Tuesday and cloudy. Determine

    1. The probability that the next three days will be sunny.

    2. The probability that next Friday will be sunny.

    Exercice 2 A Markov chain on three states, say 0; 1 and 2, has as transition matrix
    0 1
    1=4 3=4 0
    P = @ 0 1 0 A:
    1=2 0 1=2

    Determine

    1. State classes and their types;

    2. The n-step transition matrix;

    3. The limit of this matrix when n goes to in…nity.

    Exercice 3 Consider a four-state Markov chain S = f1; 2; 3; 4g whose transition matrix is


    0 1
    0 1=2 1=2 0
    B1=2 0 0 1=2C
    P =B@ 0 0
    C:
    1 0 A
    0 0 0 1

    1. Draw the transition graph associated with this Markov chain.

    2. Show that states 3 and 4 are absorbing. Are the others transient or recurrent?

    3. Calculate the probabilities of the following trajectories as a function of the initial probabilities
    of each state (p0 ; q0 ; r0 ; s0 ):
    (X0 = 1; 8n 1 Xn = 3) ; (X0 = 1; X1 = 2; 8n 2 Xn = 4),
    (Xn = 1 if n is even and Xn = 2 if n is odd).
    Show that the path (Xn = 1 if n is even and Xn = 2 if n is odd) has probability zero.

    1
    4. Assume that the distribution between the four states is uniform at initial time t = 0. Calculate
    the distribution at time t = 1 and then at time t = 2. Same question if we start from an initial
    distribution 0 = 21 ; 12 ; 0; 0 .

    5. Show that any initial distribution of the form 0 = (0; 0; r0 ; s0 ) with r0 + s0 = 1 is a stationary
    distribution. Are there any others?

    Exercice 4 A rat runs through the maze shown below. At each step it leaves the room it is in by
    choosing at random one of the doors out of the room.

    1. Give the transition matrix P for this Markov chain.

    2. Show that it is irreducible but not aperiodic.

    3. Find the stationary distribution

    4. Now suppose that a piece of mature cheddar is placed on a deadly trap in Room 5. The mouse
    starts in Room 1. Find the expected number of steps before reaching Room 5 for the …rst time,
    starting in Room 1.

    5. Find the expected time to return to room 1.

    Exercice 5 A slot machine in a casino gives you 30 DZD if you win a game, but you have to pay
    20 DZD per game to play. The machine advertises that there is always at least a 50=50 chance of
    k+1
    winning each game. In fact, the probability of success in a game is equal to k+2 , where k represents
    the number of successes in the two previous games. Is this slot machine pro…table for the casino in
    the long term?

    You might also like