Scribd
Upload
2 views3 pages

P Is A Markov Transition Matrix

The document describes a Markov transition model for a machine that can be in three states: Operating, Idle, and Broken, with defined probabilities for transitioning between these states. It outlines the setup for a simulation that runs for 10,000 steps to track the time spent in each state and calculates the empirical steady-state probabilities. The results indicate that the machine spends approximately 59.78% of its time in the Operating state, reflecting the estimated process efficiency.

Uploaded by

pachkhandeprajwal
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
2 views3 pages

P Is A Markov Transition Matrix

The document describes a Markov transition model for a machine that can be in three states: Operating, Idle, and Broken, with defined probabilities for transitioning between these states. It outlines the setup for a simulation that runs for 10,000 steps to track the time spent in each state and calculates the empirical steady-state probabilities. The results indicate that the machine spends approximately 59.78% of its time in the Operating state, reflecting the estimated process efficiency.

Uploaded by

pachkhandeprajwal
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 3

This setup could model a real-world machine like this:

 It usually keeps working, but sometimes goes idle or breaks.

 From Idle, it's fairly likely to start working again.

 From Broken, it has a good chance of being repaired.

P is a Markov transition matrix—it shows the probabilities of moving between states in a


single time step.

Each row in the matrix represents the current state, and each column represents the next
state.

🔹 Row 0: From Operating

 70% chance the machine stays Operating

 20% chance it transitions to Idle

 10% chance it breaks and moves to Broken

 🔹 Row 1: From Idle

50% chance it starts Operating again

40% chance it stays Idle

10% chance it breaks while idle

🔹 Row 2: From Broken

 60% chance it gets fixed and goes back to Operating

 30% chance it goes to Idle (maybe waiting for parts)

 10% chance it stays Broken

 In a Markov matrix, each row must sum to 1, because it represents all possible next
states from the current state.
 # Initial state distribution (start in Operating)
 state = 0
 n_steps = 10000
 state_counts = np.zeros(len(states))

1. state = 0

 This sets the initial state of the system.


 We are starting the simulation with the machine in state 0, which corresponds to
"Operating".
 In our states = ['Operating', 'Idle', 'Broken'], index 0 = "Operating".
 A simple list to give names to the states: index 0 = Operating, index 1 = Idle, index 2
= Broken.

In a more advanced version, you could choose this initial state randomly using a probability
distribution, but here we start with it always Operating.

2. n_steps = 10000

 This defines how many time steps (or transitions) we will simulate.
 A higher number gives more stable and reliable results, especially when
approximating steady-state probabilities.
 state = 0: We start with the machine Operating
 n_steps = 10000: Number of steps (or "transitions") to simulate
 state_counts: A counter to track how often we land in each state

3. state_counts = np.zeros(len(states))

 We initialize a counter array to track how often the system is in each state.
 len(states) is 3 in this case (Operating, Idle, Broken), so:

state_counts = [0.0, 0.0, 0.0]

 During the simulation, we'll increment the count for the current state at each step. At
the end, we’ll divide each count by n_steps to get the proportion of time spent in
each state (aka empirical steady-state probabilities).

for _ in range(n_steps):

state_counts[state] += 1

state = np.random.choice([0, 1, 2], p=P[state])

For each step:

Add 1 to the count of the current state

Use np.random.choice to pick the next state, based on the current row of P
E.g., if you're in state 0 (Operating), P[0] = [0.7, 0.2, 0.1], so there's a 70% chance to stay
Operating, etc.

This models how the machine transitions over time.

state_probs = state_counts / n_steps

Converts the total counts into percentages (or proportions).

This gives us the empirical steady-state probabilities—i.e., how much time the machine
spends in each state on average.

for i, s in enumerate(states):

print(f"Time in '{s}': {state_probs[i]*100:.2f}%")

Loops through each state and prints the % of time spent there.

Calculating Efficiency

efficiency = state_probs[0]

print(f"\nEstimated Process Efficiency: {efficiency*100:.2f}%")

 Defines process efficiency as the proportion of time the machine is in the Operating
state (index 0).
 Prints the final result.

Time in 'Operating': 59.78%

Time in 'Idle': 27.61%

Time in 'Broken': 12.61%

Estimated Process Efficiency: 59.78%

You might also like