Bayes's Theorem for Conditional Probability

Bayes's Theorem for Conditional Probability: Bayes's Theorem is a fundamental result in probability theory that describes how to update the probabilities of hypotheses when given evidence. Named after the Reverend Thomas Bayes, this theorem is crucial in various fields, including engineering, statistics, machine learning, and data science. This article explores Bayes's Theorem, its mathematical formulation, proof, and applications in engineering.

Read: Conditional Probability

What is Bayes's Theorem?

Bayes's Theorem provides a way to update the probability of a hypothesis based on new evidence. Mathematically, it is expressed as:

P(A|B) = \frac{P(B|A)P(A)}{P(B)}

where:

• P(A∣B) is the posterior probability of event A given event B.
• P(B∣A) is the likelihood of event B given event A.
• P(A) is the prior probability of event A.
• P(B) is the marginal probability of event B.

Mathematical Formulation

To derive Bayes's Theorem, we start with the definition of conditional probability:

P(A \mid B) = \frac{P(A \cap B)}{P(B)}

P(B \mid A) = \frac{P(A \cap B)}{P(A)}

Rearranging the second equation to express P(A∩B):

P(A∩B)=P(B∣A)⋅P(A)

Substitute this into the first equation:

P(A|B) = \frac{P(B|A)P(A)}{P(B)}

This is the formula for Bayes's Theorem.

Solved Examples of Bayes's Theorem

Simple Disease Test

Problem 1: A disease affects 1% of the population. A test is 95% accurate for both positive and negative results. If a person tests positive, what's the probability they have the disease?

Solution:

P(D) = 0.01, P(T|D) = 0.95, P(T|not D) = 0.05

P(T) = 0.95 * 0.01 + 0.05 * 0.99 = 0.0585

P(D|T) = (0.95 * 0.01) / 0.0585 ≈ 0.1624 or 16.24%

Factory Quality Control

Problem 2: A factory has two production lines. Line A produces 60% of the items and has a 3% defect rate. Line B produces 40% of the items and has a 2% defect rate. If an item is defective, what's the probability it came from Line A?

Solution:

P(A) = 0.6, P(D|A) = 0.03, P(D|B) = 0.02

P(D) = 0.03 * 0.6 + 0.02 * 0.4 = 0.026

P(A|D) = (0.03 * 0.6) / 0.026 ≈ 0.6923 or 69.23%

Weather Forecast

Problem 3: The weather is sunny 70% of the time. A weather app is 80% accurate when predicting sunny days and 60% accurate for non-sunny days. If the app predicts a sunny day, what's the probability it will actually be sunny?

Solution:

P(S) = 0.7, P(P|S) = 0.8, P(P|not S) = 0.4

P(P) = 0.8 * 0.7 + 0.4 * 0.3 = 0.68

P(S|P) = (0.8 * 0.7) / 0.68 ≈ 0.8235 or 82.35%

Email Classification

Problem 4: 20% of emails are important. A filter correctly identifies 90% of important emails and 85% of unimportant emails. If an email is marked as important, what's the probability it's actually important?

Solution:

P(I) = 0.2, P(M|I) = 0.9, P(M|not I) = 0.15

P(M) = 0.9 * 0.2 + 0.15 * 0.8 = 0.3

P(I|M) = (0.9 * 0.2) / 0.3 = 0.6 or 60%

Drug Test

Problem 5: 5% of athletes use performance-enhancing drugs. A drug test is 99% accurate for users and 95% accurate for non-users. If an athlete tests positive, what's the probability they're actually using drugs?

Solution:

P(D) = 0.05, P(T|D) = 0.99, P(T|not D) = 0.05

P(T) = 0.99 * 0.05 + 0.05 * 0.95 = 0.0945

P(D|T) = (0.99 * 0.05) / 0.0945 ≈ 0.5238 or 52.38%

Customer Loyalty

Problem 6: 30% of customers are loyal. Loyal customers have a 80% chance of making a purchase, while non-loyal customers have a 20% chance. If a customer makes a purchase, what's the probability they're loyal?

Solution:

P(L) = 0.3, P(P|L) = 0.8, P(P|not L) = 0.2

P(P) = 0.8 * 0.3 + 0.2 * 0.7 = 0.38

P(L|P) = (0.8 * 0.3) / 0.38 ≈ 0.6316 or 63.16%

Fire Alarm

Problem 7: The probability of a fire in a building is 0.1%. The fire alarm has a 99% chance of detecting a fire and a 0.5% false alarm rate. If the alarm goes off, what's the probability there's actually a fire?

Solution:

P(F) = 0.001, P(A|F) = 0.99, P(A|not F) = 0.005

P(A) = 0.99 * 0.001 + 0.005 * 0.999 = 0.005994

P(F|A) = (0.99 * 0.001) / 0.005994 ≈ 0.1652 or 16.52%

College Admissions

Problem 8: 40% of applicants have high test scores. Among those with high scores, 80% are admitted. Among those without high scores, 30% are admitted. If a student is admitted, what's the probability they had high test scores?

Solution:

P(H) = 0.4, P(A|H) = 0.8, P(A|not H) = 0.3

P(A) = 0.8 * 0.4 + 0.3 * 0.6 = 0.5

P(H|A) = (0.8 * 0.4) / 0.5 = 0.64 or 64%

Car Insurance

Problem 9: 15% of drivers are high-risk. High-risk drivers have a 20% chance of an accident in a year, while others have a 5% chance. If a driver has an accident, what's the probability they're high-risk?

Solution:

P(H) = 0.15, P(A|H) = 0.2, P(A|not H) = 0.05

P(A) = 0.2 * 0.15 + 0.05 * 0.85 = 0.0725

P(H|A) = (0.2 * 0.15) / 0.0725 ≈ 0.4138 or 41.38%

Software Bug Detection

Problem 10 : A software has a 10% chance of containing a critical bug. A testing tool detects 95% of critical bugs and has a 8% false positive rate. If the tool reports a critical bug, what's the probability the software actually has one?

Solution:

P(B) = 0.1, P(D|B) = 0.95, P(D|not B) = 0.08

P(D) = 0.95 * 0.1 + 0.08 * 0.9 = 0.167

P(B|D) = (0.95 * 0.1) / 0.167 ≈ 0.5689 or 56.89%

Practice Problems on Bayes's Theorem for Conditional Probability

1).In a city, 30% of residents own dogs. Among dog owners, 80% walk their dogs daily. Among non-dog owners, 10% walk daily for exercise. If you see someone walking, what's the probability they're a dog owner?

2).A rare genetic disorder affects 1 in 10,000 people. A new test for this disorder is 99% accurate for both positive and negative results. If someone tests positive, what's the probability they actually have the disorder?

3).A company produces widgets using two machines. Machine A produces 70% of the widgets and has a 3% defect rate. Machine B produces the remaining 30% and has a 1% defect rate. If a widget is defective, what's the probability it was produced by Machine A?

4).In a school, 60% of students play sports. Among athletes, 80% pass their exams. Among non-athletes, 60% pass their exams. If a student passes their exam, what's the probability they play sports?

5).A spam filter has a 98% chance of correctly identifying spam emails and a 95% chance of correctly identifying non-spam emails. About 40% of all emails are spam. If an email is flagged as spam by the filter, what's the probability it's actually spam?

6).In a certain population, 5% of people have a specific medical condition. A new screening test is 90% accurate for those with the condition and 95% accurate for those without. If a person tests positive, what's the probability they have the condition?

7).A factory has three production lines: A, B, and C. Line A produces 50% of the items, B produces 30%, and C produces 20%. The defect rates are 2%, 3%, and 4% respectively. If an item is found to be defective, what's the probability it came from Line B?

8).In a card game, 30% of players are experts, 50% are intermediate, and 20% are beginners. The probability of winning for each group is 70%, 40%, and 20% respectively. If a player wins a game, what's the probability they're an expert?

9).A security system has a 99.9% chance of detecting an intruder and a 0.1% false alarm rate. The probability of an actual intrusion on any given day is 0.01%. If the alarm goes off, what's the probability there's a real intruder?

10).In a dating app, 40% of users are actively seeking a relationship. Among these users, 70% respond to messages within a day. Among those not actively seeking, only 20% respond within a day. If a user responds to a message within a day, what's the probability they're actively seeking a relationship?

Applications of Bayes's Theorem in Engineering

1. Signal Processing: In signal processing, Bayes's Theorem is used in filtering and estimation techniques, such as the Kalman filter, to update the state estimates of a system based on new measurements.

2. Machine Learning: In machine learning, Bayes's Theorem underpins various algorithms, including Naive Bayes classifiers. These algorithms are used for classification tasks in text analysis, spam detection, and sentiment analysis.

3. Reliability Engineering: In reliability engineering, Bayes's Theorem helps in updating the probability of system failures based on observed data. It is used to improve maintenance schedules and predict future failures.

4. Medical Diagnosis: In biomedical engineering and healthcare, Bayes's Theorem assists in diagnostic tests by updating the probability of a disease based on test results, leading to more accurate diagnoses.

5. Robotics: In robotics, Bayes's Theorem is employed in probabilistic robotics to perform tasks such as localization and mapping. It helps robots update their position estimates based on sensor data.

Related Articles:

• Bayes Theorem | Statement, Formula, Derivation
• Bayes Theorem in Machine learning
• Real Life Applications of Bayes' Theorem

Conclusion

Bayes's Theorem is a powerful tool for updating probabilities based on new evidence, with wide-ranging applications in engineering, statistics, machine learning, and many other fields. Understanding and applying Bayes's Theorem is essential for making informed decisions in the presence of uncertainty.