Scribd
Upload
2 views

probability lecture notes

The document outlines key concepts in probability and stochastic processes, including De-Morgan's Laws, axioms of probability, conditional probability, and Bayes' Theorem. It discusses various distributions such as Gaussian and Poisson, as well as the properties of random variables and their independence. Additionally, it covers the Central Limit Theorem and the implications of joint distributions in machine learning contexts.

Uploaded by

yg3481
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
2 views

probability lecture notes

The document outlines key concepts in probability and stochastic processes, including De-Morgan's Laws, axioms of probability, conditional probability, and Bayes' Theorem. It discusses various distributions such as Gaussian and Poisson, as well as the properties of random variables and their independence. Additionally, it covers the Central Limit Theorem and the implications of joint distributions in machine learning contexts.

Uploaded by

yg3481
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 19

Probability and Stochastic Process

Lecture 1
Scarlett [email protected]

De-Morgan's Laws

Axioms of probability

1.

2.

3. , which means and are mutually exclusive, then

Conditional probability and Bayes' Theorem


What means ? Probability of that is given, or to say has occurred.

Partition

Several makes the full-set parts. If , then .

example 1.

black balls and white balls. Assume , then the probability of "first pick black, second pick white" is

If events are independent, .

In a coin tossing game,

where
therefore

Lecture 2
Review on .

Assume we have a constant , and , then , we call event is a rare event.

If obeys binomial distribution, then

When ,

Negative-Binomial
success at trial

Team and team are in a game. The team wins 4 in 7 games wins the game.

Which means , can win at , , , game.

Random variables

Distribution Function Properties

1.

2. Non-decreasing

3. Right-continuous

. .
Actually,

Continuous PDF example: Gaussian/normal distribution

Density Function Properties

1.

2.

Linear Filter

Lecture 3
Calculate the conditional probability ,

Since
Then

a-posteriori pdf

For conditional event ,

Expected value of .

is dispersion
measure of spread of observation around mean

Characteristic Function

Which means

Another example
We can make deduction of the Chebyshev's inequation

Make as

Then

Lecture 4
Characteristic Function

For Gaussian

Specifically, For

De-Mourne Laplace Theorem For

We can proof that follows Gaussian distribution


Notice that

Then

The notation is the same as in mathematical proof.

Leibnitz's Rule

Random Variables Joint Representation

Properties:

1.

2.
Obviously, there is a character

Lecture 5
A Function of Two Random Variables
Given , find .

For a rectangular region

If the region is of a random shape


Therefore

If

We say and are independent random variables.

Leibnitz Rule

Proof:

When two random variables are independent, we can use convolutional operation

For example,

Sum of any 2 Gaussian random variables (i.i.d. or not) is still Gaussian

Another example, .

, assume as , as ,
If

jointly Gaussian

If , they are independent.

Lecture 6
Review
, given, find .

The followings are on Brightspace written lecture notes.

Lecture 7
Gaussian
Joint Gaussian

For

If

If , of course are independent.

If , then how to measure their dependence?

Since

then
If , then

Here we have the correlation,

Also,

What if are independent, what about

then

So independent can deduce that are uncorrelated( ).

But reversely, it doesn't apply, except when are jointly Gaussian random variables.

Example:

Are uncorrelated? Are they independent?

For , you may calculate it as practice.

, assume ,
Theorem: Any two linear combinations of two jointly Gaussian random variables are still jointly Gaussian random variables.

For example, , are not independent. Then how about ? Still


jointly Gaussian random variables.

This is really strange, since are not independent, but they are able to generate that are independent.

Joint Characteristic Function of and

Assume

For jointly Gaussian

Let
Take as , as . Then

For Poisson like

Joint Poisson

, assume

For ,

Let

Central Limit Theorem: Large sum of any random variables behave like Gaussian.

Let
As ,

Proof:

Since and , we retain the first three terms

When

For the normalized sum

The characteristic function is

Accordingly, we have

That following Gaussian distribution, would be a good proof of central limit theorem.

Lecture 8
Conditional distributions and conditional p.d.f.s

In joint p.d.f. we can have


When that

Leads to

The joint conditional p.d.f. is therefore conveyed as upon.

Since

The Bayes' theorem is

In machine learning, it means given data , we can deduce the likelihood .

Conditional expectation is

Obviously, expectation is a function of instead of a constant.

has an alternative formula as

Which can be simplified as

Example: Poisson's eggs

eggs each arrived follows Possion( )


eggs may or maynot arrive

Which proves that and are actually independent, rather than instinctly feeling dependent.

Sequential vs. Batch Processing


With data , we can have , with data , we can have , and so on.

Two experiments:

We can Toss n times and observe k times, then get .

We can also keep tossing till we get .

. , .

Any of and 's linear combination is still Gaussian

Where
Best estimator for the unknown ? Collect data (Design experiments), find independent

Assume .

Likelihood function

Fisher: Go for that is most-likely value of suggested by data.

is unbiased estimation

Assume 𝕦 , where refers to maximum rain fall (unknown).

We can find

You might also like