Scribd
Upload
22 views3 pages

Linexpect

Uploaded by

boretaj297
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
22 views3 pages

Linexpect

Uploaded by

boretaj297
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 3

Linearity of Expectation: An In-depth Exploration

Introduction
Linearity of expectation is a fundamental property in probability theory and statistics that
simplifies the calculation of expected values. This property holds regardless of whether the random
variables involved are independent or not, making it an extremely powerful tool in various fields
such as finance, computer science, operations research, and more. Linearity of expectation states
that the expected value of a sum of random variables is the sum of their individual expected values,
even when the random variables are not independent.
This property simplifies many otherwise complex calculations and helps in deriving solutions in
problems involving random processes, stochastic variables, and decision-making under uncertainty.

Formal Definition
Mathematically, the linearity of expectation is stated as follows:
For any two random variables X1,X2,…,XnX_1, X_2, \dots, X_n, the expected value of their sum
is:
E[X1+X2+⋯+Xn]=E[X1]+E[X2]+⋯+E[Xn]E[X_1 + X_2 + \dots + X_n] = E[X_1] + E[X_2] + \
dots + E[X_n]
This equation holds true for any kind of random variables, whether they are discrete or continuous,
and whether or not they are independent. The expectation operator E[⋅]E[\cdot] denotes the
expected (mean) value of the random variable.
The remarkable aspect of the linearity of expectation is that it holds even when the random
variables are dependent. This is in contrast to other properties, such as the product rule for
independent random variables, which only holds when the variables are independent.

Understanding Linearity of Expectation with Examples


1. Simple Case: Sum of Two Random Variables
Suppose you have two random variables X1X_1 and X2X_2, where the expected values are
E[X1]=3E[X_1] = 3 and E[X2]=5E[X_2] = 5. According to the linearity of expectation:
E[X1+X2]=E[X1]+E[X2]=3+5=8E[X_1 + X_2] = E[X_1] + E[X_2] = 3 + 5 = 8
Even if X1X_1 and X2X_2 are dependent, the result still holds true. This is a key insight, as
it eliminates the need for considering their dependence structure when calculating the sum of
their expectations.
2. Application in Random Walks
The linearity of expectation is particularly useful in problems involving random walks. A
random walk is a process where each step is determined by a random variable, and it is
commonly used in modeling scenarios in physics, economics, and computer science. For
instance, in a simple random walk on a number line, where each step is either +1+1 or −1-1,
the expected position after nn steps is the sum of the expected positions at each step. Using
linearity of expectation, the expected position after nn steps can be computed as:
E[Position after n steps]=E[X1+X2+⋯+Xn]=E[X1]+E[X2]+⋯+E[Xn]E[\text{Position after
} n \text{ steps}] = E[X_1 + X_2 + \dots + X_n] = E[X_1] + E[X_2] + \dots + E[X_n]
This is true even if the steps are correlated or have some dependency between them, as long
as each step has a known expected value.
3. Example: Coin Tossing
Consider a scenario where you flip a fair coin 10 times, and let XiX_i be the indicator
random variable for the ii-th toss, where Xi=1X_i = 1 if the toss results in heads, and
Xi=0X_i = 0 if it results in tails. The expected value of each XiX_i is:
E[Xi]=1⋅P(Heads)+0⋅P(Tails)=0.5E[X_i] = 1 \cdot P(\text{Heads}) + 0 \cdot P(\text{Tails})
= 0.5
The total number of heads in 10 tosses is:
S=X1+X2+⋯+X10S = X_1 + X_2 + \dots + X_{10}
Using linearity of expectation, the expected number of heads is:
E[S]=E[X1+X2+⋯+X10]=E[X1]+E[X2]+⋯+E[X10]=10⋅0.5=5E[S] = E[X_1 + X_2 + \
dots + X_{10}] = E[X_1] + E[X_2] + \dots + E[X_{10}] = 10 \cdot 0.5 = 5
Even though the coin tosses are independent, the linearity of expectation holds regardless of
the independence of the random variables.

Why Linearity of Expectation is So Powerful


The power of the linearity of expectation lies in its simplicity and the fact that it applies universally
to both independent and dependent random variables. This makes it a go-to tool in problems where
calculating the expected value of sums of random variables is required.
In situations where the random variables are not independent, such as in some Markov chains,
Monte Carlo simulations, or complex systems modeling, the linearity of expectation still allows
us to compute the sum of expected values without needing to account for the correlation or
dependence between the variables.

Some Advanced Applications


1. Variance Decomposition: The linearity of expectation also plays a role in calculating
variance. For independent random variables X1,X2X_1, X_2, the variance of their sum is
the sum of their variances:
Var(X1+X2)=Var(X1)+Var(X2)\text{Var}(X_1 + X_2) = \text{Var}(X_1) + \text{Var}(X_2)
While the variance does not follow the same rules of linearity as expectation, understanding
how expectation behaves can still help with decomposing the behavior of sums of random
variables.
2. Random Variables with Different Distributions: In cases where random variables follow
different distributions, such as in a mixture of distributions, linearity of expectation still
applies. For example, in a scenario where a random variable can take values from several
different distributions with respective probabilities, you can compute the expected value as
the sum of the expectations from each distribution, weighted by their probabilities.
3. Applications in Game Theory and Decision-Making: In game theory, auction theory,
and decision theory, linearity of expectation is useful when assessing strategies under
uncertainty. For example, if a player’s payoff depends on several uncertain outcomes, the
expected value of their total payoff can be computed by summing the expected payoffs from
each outcome, thanks to linearity.
4. Indicator Random Variables: Linearity of expectation is often applied in problems
involving indicator random variables. These are binary random variables (0 or 1), and
their sum can often represent a total count of some desired event. The expectation of this
sum can be easily calculated using linearity.

Limitations and Common Misconceptions


While the linearity of expectation is a robust property, it does not extend to other operations in the
same way. For instance:
• Products of random variables: Unlike sums, the expectation of the product of two random
variables X1X_1 and X2X_2 is not simply E[X1]⋅E[X2]E[X_1] \cdot E[X_2] unless the
variables are independent. This is a key distinction from the linearity of expectation.
• Conditional expectation: Linearity also extends to conditional expectations. If X1,X2X_1,
X_2 are random variables and F\mathcal{F} is a sigma-algebra, then:
E[X1+X2∣F]=E[X1∣F]+E[X2∣F]E[X_1 + X_2 \mid \mathcal{F}] = E[X_1 \mid \
mathcal{F}] + E[X_2 \mid \mathcal{F}]

Conclusion
Linearity of expectation is a powerful and versatile property that simplifies the calculation of
expected values in complex stochastic processes. Its applicability to both independent and
dependent random variables makes it a cornerstone of probability theory. By understanding this
concept, one can solve problems more efficiently in fields such as economics, finance, operations
research, and machine learning. Whether dealing with sums of random variables, indicator
variables, or even mixtures of distributions, linearity of expectation provides an elegant way to
decompose the expected value into manageable components.

You might also like