Probability and

Statistics
EEF 271 E
Week 2
Tuba Yılmaz Abdolsaheb
[email protected]
Associate Professor
Instructor
• Tuba Yılmaz Abdolsaheb ([email protected])
 Research Interests
• Bio electromagnetics
• Dielectric Spectroscopy
• Optimization Algorithms
• Antennas – design and testing
• Application of ML algorithms to EM
• Office: 2415
• Office Hours:
• Tuesday – 11:30 - 12:30
• Thursday– 14:00 – 15:00
• All course material and announcements will be uploaded to ninova. It is your
responsibility to check the updates on ninova!
• http://ninova.itu.edu.tr/tr/dersler/elektrik-elektronik-fakultesi/33782/eef-271e/
Exams and Grades for 10871
• 1 Midterm : %35 (WEEK 9 December 3rd 2024 – IN CLASS HOURS
• Homework - 1 Project HW: %15 (One project homework graded 15% and
suggested homeworks with no grading)
• 3 POP Quizes: %10 in total (2 of them 3%, 1 of them 4%)
• Final prerequisite: 18 points out of 60 points (YOU WILL FAIL THE COURSE
WITH VF IF YOU DO NOT MEET THIS REQUIREMENT!!!)
• Final exam: %40
• Other Notes
• The attendance is encouraged but not compulsory
• Please sit down and spare weekly (undistracted) block of time to understand the
material presented here. Do review/solve the practice questions without looking
into the solutions both from the class and from recommended or other books.
What will be covered during the
course?
• TEXTBOOK
 Montgomery, Douglas C., and George C. Runger. Applied statistics and
probability for engineers. John Wiley & Sons, 2010
 Sheldon Ross, 1998, A First Course in Probability, Prentice-Hall International.
• Other Resources
 Murray R. Spiegel, 1961, Theory and Problems of Statistics, McGraw-Hill Book
Company.
A detailed course catalogue is available in the ninova web site. Make sure you are
added to the ninova web site.
What will be covered during the
course?
PREQUISITE: FAMILIARITY WITH SETS AND SET OPERATIONS
1. Counting techniques, permutation and combination problems
2. Probability and conditional probability
3. Random variables, Probability density function
4. Distribution function
5. Discrete distributions: Bernolli, Binom, Poisson
6. Continuous distributions: Normal, Gamma, Exponential
7. Expectation, Moment generating function
8. Measures of central tendency and dispersion, Chebchev’s inequality
9. Estimators and properties
10. Maximum likelihood estimators
11. Confidence intervals and hypothesis testing
12. One and two sample test for means
13. One and two sample test for means
14. Regression analysis
What will be covered during the
course?
Students who pass the course will be able to:
1. Solve permutation and combination problems.
2. Grasps the probability, conditional probabilities and random variables
concepts.
3. Solving real life problems in the engineering by using statistical methods.
4. Understand the details about the probability functions and apply them to
engineering problems.
5. Compute the measures of central tendency and dispersion.
6. Understand the statistical tests.
7. Construct One and two sample test for means and do the test.
8. Learn and apply regression analysis
What is probability?
• In layman terms probability is a way of expressing how likely something is to
happen based on the knowledge of the situation.
• Imagine you have a big jar filled with 100 marbles: 50 are red and 50 are blue. If
you close your eyes and pick one marble out at random, the chance or likelihood
that you'll get a red marble is what we call "probability".
• In this example, since half of the marbles are red, the probability of picking a red
marble is 50%. Similarly, the probability of picking a blue marble is also 50%.
• Mathematically: The probability of event E occurring probability defined as
• P(E)=
 What is probability?
• A "random experiment" (or sometimes just referred to as an "experiment") in probability theory is a process or procedure that
produces one of several possible outcomes, and the outcome cannot be determined in advance. The unpredictability of the
result is the essence of the randomness.
• Here are some key characteristics of a random experiment:
• Multiple Outcomes: A random experiment can have multiple possible outcomes.
• Unpredictability: The outcome of the experiment is not known in advance and cannot be determined until the experiment is
performed.
• Repeatability: The experiment can be repeated under the same conditions, but the outcomes can vary from one repetition to
another due to its random nature.
• Examples of random experiments include:
• Coin Toss: Tossing a coin can result in either a 'Heads' or 'Tails'. You can't predict the outcome beforehand.
• Rolling a Die: When you roll a standard six-sided die, any number between 1 and 6 can come up, but you don't know which one
will until you roll.
• Drawing a Card: Drawing a card from a deck can result in any one of the 52 cards, but you won't know which card it will be until
you draw.
• In each of these cases, the outcome is uncertain and unpredictable until the experiment is conducted.
Random Experiments
• Experiments are
• Performed under controlled conditions
• Executed to discover a previously unknown result or verify the results
obtained from theory.
• An experiment that can result in different outcomes, even though it is
repeated in the same manner every time, is called a random
experiment.
• Example: most science experiments
Sample Spaces
• The set of all possible outcomes of a random experiment is called the
sample space of the experiment. The sample space is denoted as S.
• A sample space is discrete if it consists of a finite or countable infinite set of
outcomes.
• A sample space is continuous if it contains an interval (either finite or infinite) of
real numbers
• EXAMPLE: roll a die, toss a coin, roll two dice, toss two coins, subway
sandwich, draw a card from a deck of cards (SHOW TREE DIAGRAM)
• EVENTS: SETS OF OUTCOMES
• The sum of rolls are 12
• Both tosses are heads
• Both cards are clubs
• The sandwitch is tuna melt with swiss cheese and rye bread
 What is probability?

• In probability and statistics, an "outcome" refers to a single possible result of a random experiment or process. It represents one specific
realization of the experiment.
• The "sample space" of a random experiment is the set of all possible outcomes of that experiment. It encompasses every outcome that could
possibly occur when the experiment is conducted. The sample space is often denoted by the letter S or Ω in mathematical notation.
• In the context of probability theory, an "event" is a set of outcomes of a random experiment. It describes a specific situation or set of
situations that we're interested in when a random experiment is performed. An event can consist of one or multiple outcomes.
• Coin Toss:
• Experiment: Tossing a coin.
• Sample Space (S): S={Heads (H),Tails (T)}
• Possible Outcomes: Heads (H) or Tails (T).
• If you toss the coin and it lands heads-up, then the outcome of that particular toss is "Heads.“
• An Event: The coin showing "Heads". This event consists of just one outcome, "Heads".
• Rolling a Die:
• Experiment: Rolling a standard six-sided die.
• Sample Space (S): S={1,2,3,4,5,6}
• Possible Outcomes: 1, 2, 3, 4, 5, or 6.
• If you roll the die and it shows a 4, then the outcome of that particular roll is "4.“
• An Event: The die showing an even number. This event consists of the outcomes 2, 4, and 6.
• Drawing a Card from a Deck:
• Experiment: Drawing a card from a standard deck of 52 cards.
• Sample Space (S): S consists of all 52 cards in the deck.
• Possible Outcomes: Any of the 52 cards (e.g., Ace of Spades, 10 of Hearts, etc.).
• If you draw a card and it's the Queen of Diamonds, then the outcome of that particular draw is "Queen of Diamonds.“
• An Event: Drawing a "Heart". This event consists of all the heart cards in the deck (13 outcomes).
 Basic Rules of Probability
• A probability on the sample space S is a function • addition rule
from events to numbers. P(A U B) = P(A) + P(B) - P(AnB)
• The probability of event A is denoted P(A).
• Addition of mutually exclusive events:
• We have some basic requirements for the function.
P(A [ B) = P(A) + P(B) A n B = {},
(i) The whole sample space S has probability P(S) = 1
(ii) For every event A we have 0 <P(A) < 1
Probability of complement and difference:
(iii) Additivity: For any collection of mutually • P(A c
) = 1 - P(A);
exclusive events of A1, A2, …. AN we have • P(B \A) = P(B) - P(A n B):
P(A1 U A2 U... ) = P(A1) + P(A2) + .... + P(AN)
SET OPERATIONS
Some of the basic set operations are summarized below in terms of events:
• The union of two events is the event that consists of all outcomes that are
contained in either of the two events. We denote the union as E1UE2 .
• The intersection of two events is the event that consists of all outcomes
that are contained in both of the two events. We denote the intersection
as E1ՈE2.
• The complement of an event in a sample space is the set of outcomes in
the sample space that are not in the event. We denote the component of
the event E as E’.
• Two events, denoted as E1 and E2, E1ՈE2={ } such that are said to be
mutually exclusive. Example: rolling a die.
Combined Events
• When combining events use the logical operators and relate them with the
previously introduced set operations
• Consider a group of pet owners when asked A is the event that a random per
owner is keeping a dog, B is the event that a random per owner is keeping a cat,
and some of them are keeping both dog and cat.
• A and B  A n B (remember that the mutually exclusive events can not happen
at the same time so no intersection for mutually exclusive events!)
• A or B  A U B
• A does not occur Ac
• B occurs but A does not B/A
• Draw the diagrams!
Set operations examples
Venn Diagrams
Counting Techniques
• Rule of Sum
• Event A occurs in ‘m’ ways and event B occurs in ’n’ ways. If two events cannot happen
simultaneously then event A and B can occur m+n ways.
n(A U B)=n(A)+n(B)
• Suppose Ali has 5 science fiction books, 6 comic books, and 4 personal development books. How
many different ways can Ali select one book out of all his books.
n = 5+6+4
n = 15 different ways Ali can select one book out of all books. In other words this event has 15
different outcomes.
• Probability and Statistics course is offered by mathematics, computer engineering, and electronics
engineering departments. How many different ways can a student undertake the course?
n = 1+1+1
n = 3 different ways a student can undertake this class.
Counting Techniques
• Product Rule
• Event A occurs in ‘m’ ways and independent of event A event B occurs in ’n’ ways. Combination of event A and B can occur mn
ways.
• n(A X B)=n(A)*n(B)
• A pizza shop is offering customers the option of putting together their own pizza. The customers can choose one from each
option among sauce, vegetable and meat topping. The sauces are white or tomato sauce, vegetable options are mushrooms,
peppers, olives, and corn, meat toppings are chicken, beef and shrimp. How many different pizzas can customers order?
n = 2*4*3
n = 24 different pizza types can be ordered by the customers. In other words this event has 24different outcomes.
• Joan is packing for a holiday with 5 shirts, 3 pants, and 2 shoes. How many different outfits can she wear during the holiday.
n = 5*3*2
n = 30 outfits joan can wear during the holiday. In other words this event has 30 different outcomes.
• Suppose Airline A has two-way 3 daily flights between Istanbul and Ankara where airline B has 2 two-way daily flights between
Istanbul and Ankara. How many different ways can Ali travel from Istanbul to Ankara and Back to Istanbul.
The number of flights from Istanbul to Ankara is 3+2=5
The number of flight from Ankara to Istanbul is 3+2=5
Total number of ways Ali can travel from Istanbul to Ankara and back n = 5*5, n = 25.
Counting Techniques
• Permutations
• Arrangement of n objects with a particular order is called permutation of the objects.
• Sampling without replacement: P(n, r)=
• Sampling with replacement: nr
• 3 cards will be selected from a deck of 52 cards find the total number of outcomes when the cards are chosen
a. with replacement 523
b. without replacement P(52, 3)=
• Consider the word PLENTY how many different words can be written using the same letters regardless of meaning?
n=6!, n=720
• Consider the word ROLLER how many different words can be written using the same letters regardless of meaning?
If the letters were unique n’=6!, n’=720 but we have 2 letters repeated twice!
Therefore we divide the n’ with 2!*2!, n= 6!/(2!*2!), n=180 permuttations.
• 6 students are placed in a row 2 of them are best friends and want to stand together in how many different variations
can these students can be placed in a row.
Normally the 6 students can be placed in a row in 6! ways. But since 2 of them wants to be together we can treat them
as a single student. Therefore, we have n=5!*2, n=240 different ways to put them in a line.
Counting Techniques
• Combinations
• Selection of r objects out of n objects regardless of order can be in combination of,
C(n, r)=
• Ali is packing his luggage for a trip. He has 9 books but he can only fit 4 books to his
luggage. How many different set of books he can pack?
C(9, 4)= , C(9, 4)= 126
• Find the number of ways that three teams can be formed out of 12 students.
• There is two ways to think about this

C(11, 3)* C(7, 3)=165*35=5775

• Tree Diagrams
Recap: Counting Techniques
• The fundamental counting principle states that if there are m ways to do one thing, and n ways to do
another, then there are m×n ways of doing both. The principle extends to multiple events and is
foundational in calculating permutations and combinations.
• Permutations: In probability and combinatorial mathematics, permutations are used when the arrangement or
order of items is important.
• Specifically, a permutation is an ordered arrangement of items. The number of permutations of n objects taken
r at a time is denoted and calculated as follows:
P(n, r)=

• When to Use Permutations:

1. Selection Without Replacement:
• Permutations are often used when you are choosing items without replacement.
2. Order Matters:
• Permutations are used when the order in which the items are selected is important.
Recap: Counting Techniques
• Example Scenarios:
1. Race Finishing Order:
• If you are trying to determine the number of different ways that 3 runners can finish a race and win
1st, 2nd, and 3rd place, you would use a permutation because the order in which they finish is
important.
2. Code or Password Cracking:
• When figuring out the different ways numbers or letters can be arranged for a lock or a password,
order is crucial (since "1234" is different from "4321"). So, this is a permutation problem.
3. Seating Arrangement:
• If you want to know the number of ways in which 5 guests can be seated in 5 chairs, you would
use a permutation because the arrangement or order of the guests is important.
In each of these scenarios, the order in which the items are arranged or selected is significant, which is
why permutations (not combinations) are the appropriate counting technique to use. If you have any
further questions or need additional examples, feel free to ask!
Recap: Counting Techniques- Subset Permutations and an
example
• For a sequence of r items from a set of n items:
n n!
P n(n  1)(n  2)...(n  r  1) 
r
(n  r )!
• Example: Printed Circuit Board
• A printed circuit board has four different locations in which a component can
be placed. If 8 different components are to be placed on the board, how many
designs are possible?
• Answer: Order is important, so use the permutation formula with n = 8, r = 4.

8 8! 8 7 6 5 4!
P 
4  8 7 6 5 1,680
(8  4)! 4!
Sec 2-1.4 Counting Techniques 26
Recap: Counting Techniques-Hospital
Schedule
• In a hospital, a operating room needs to schedule three knee
surgeries and two hip surgeries in a day. The knee surgery is
denoted as k and the hip as h.
• How many sequences are there?
Since there are 2 identical hip surgeries and 3 identical knee surgeries,
then
5! 5 4 3!
 10
2!3! 2 1 3!

• What is the set of sequences?

{kkkhh, kkhkh, kkhhk, khkkh, khkhk, khhkk, hkkkh, hkkhk, hkhkk, hhkkk}
Counting Techniques
• Combinations: In probability and statistics, the concept of combinations is
used in counting techniques to determine the number of ways of selecting
items from a larger set, where the order of selection does not matter.
• Selection of r objects out of n objects regardless of order can be in
combination of,  n n!
n
C   
C(n, r)= r
 r  r !(n  r )!
• If S = {a, b, c}, n =3, then
• If r = 3, there is 1 combination, namely: abc
• If r = 2, there are 3 combinations, namely ab, ac, and bc
• # of permutations ≥ # of combinations
• Since order does not matter with combinations, we are dividing the # of
permutations by r!, where r! is the # of arrangements of r elements
Counting Techniques
• When to Use Combinations?
1.Selection Without Replacement:
* When you choose items from a set and do not replace them, meaning
the items cannot be selected again.
2.Order Does Not Matter:
* When the order in which the items are selected is not important. That
is, choosing items A, B, and C is the same as choosing items B, C, and A.
3.Homogeneous Set (optional):
* Often, combinations are used when the items in the set are
homogeneous, or of the same type, though this isn't a strict rule.
 Counting Techniques
• Team Selection:
• If you have 10 people and want to select a team of 3, the order in which you select team members
does not matter. This is a combination.
• Hand of Cards:
• When you are dealt a hand of cards (say, 5 cards from a deck of 52), you might be interested in the
combination, not the order in which you received the cards.
• Choosing Desserts:
• If you have 8 desserts and can choose 2 to have after dinner, and it doesn’t matter which one you
choose first, you would use combinations to figure out how many ways you could choose desserts.
In all these scenarios, you have a larger set and you're choosing a subset where order doesn't matter,
making combinations the appropriate counting technique to use. If you have any further questions or need
additional examples, feel free to ask!
Counting Techniques
• Ali is packing his luggage for a trip. He has 9 books but he can only fit 4 books to his luggage. How many different set of books he can pack?
C(9, 4)= , C(9, 4)= 126
• Find the number of ways that three teams can be formed out of 12 students.
• There are three ways to think about this

• 12! is the total number of ways to arrange all 12 students.

• Dividing by 4! three times accounts for the arrangements within each of the three teams (since within a team, we don't care about the
order).

1. Select the first team: Choose 3 students from 12:C(12,3). But since we are considering the first student as fixed (as a team leader or a
reference), we actually choose 3 from the remaining 11: �(11,3)C(11,3).
2. Select the second team: Now, choose 3 students from the remaining 8 (since 3 have already been chosen for the first team and 1 is
considered fixed): C(8,3). However, one student from the first team of 3 has already been chosen, leaving us 7 to choose from: C(7,3).
• =5775
• If we are forming three teams, we need to consider how many members will be in each team from the 12 students. The question
doesn’t specify, so let’s assume the teams are of equal size, meaning 4 students per team.
• In this case, we choose the first team, then from the remaining students, we choose the second team, and the remaining students form
the third team. Since the teams are not labeled (i.e., there is no "team 1", "team 2", etc.), we need to divide by the number of ways the
teams can be arranged to avoid over-counting.
Example: Sampling w/o Replacement-1
• A bin of 50 parts contains 3 defectives and 47 non-defective
parts. A sample of 6 parts is selected from the 50 without
replacement. How many samples of size 6 contain 2 defective
parts?
• First, how many ways are there for selecting 2 parts from the 3
defective parts?
3 3!
C 
2 3 different ways
2!1!
• In Excel: 3 = COMBIN(3,2)
Example: Sampling w/o
Replacement-2
• Now, how many ways are there for selecting 4 parts from the 47
non-defective parts?
47 47! 47 46 45 44 43!
C 4   178,365 different ways
4!43! 4 3 2 1 43!

• In Excel: 178,365 = COMBIN(47,4)

Example: Sampling w/o
Replacement-3
• Now, how many ways are there to obtain:
• 2 from 3 defectives, and
• 4 from 47 non-defectives?
C23C447 3 178,365 535,095 different ways
• In Excel: 535,095 = COMBIN(3,2)*COMBIN(47,4)