Random Variables
A random variable (r.v.) X is set-valued function from the sample space into the real numbers.
Example 1
We roll a fair die, X is the number shown on the die
Example 2
We roll a fair die, X is 1 if the die shows a six, 0 otherwise.
Example 3
We roll a a fair die until the the first “6”, X is the number of rolls needed.
Example 4
We randomly pick a time between 10am and 12 am, X is the minutes that have passed since 10am.
There are two basic types of r.v.’s:
If X takes countably many values, X is called a discrete r.v.
If X takes uncountably many values, X is called a continuous r.v.
There are also mixtures of these two.
Aboves examples 1, 2 and 3 above X is discrete, example 4 X is continuous.
There are some technical difficulties when defining a r.v. on a sample space like \(\mathbb{R}\), it turns out to be impossible to define it for every subset of \(\mathbb{R}\) without getting logical contradictions. The solution is to define a \(\sigma\)-algebra on the sample space and then define X only on that \(\sigma\)-algebra. We will ignore these technical difficulties.
Almost everything to do with r.v.’s has to be done twice, once for discrete and once for continuous r.v.’s. This separation is only artificial, it goes away once a more general definition of “integral” is used (Rieman-Stilties or Lebesgue)
(Commulative) Distribution Function
The distribution function of a r.v. X is defined by
\(F(x) = P(X \le x) \text{ }\forall x \in \mathbb{R}\)
Example 1
say x=2.2, then
\(F(2.2) = P(X \le 2.2) = P({1,2}) = 2/6 =1/3\)
Example 4
say x=67.5, then
\(F(67.5) = P(X \le 67.5)\) = P(we chose a moment between 10am and 11h7.5min am) = 67.5/120 = 0.5625
Some features of cdf’s:
- cdf’s are standard functions on \(\mathbb{R}\)
- \(0 \le F(x) \le 1\)
- cdf’s are non-decreasing
- cdf’s are right-continuous
\[ \begin{aligned} &F(x) \rightarrow 0 \text{ as } x \rightarrow - \infty\\ &F(x) \rightarrow 1 \text{ as } x \rightarrow \infty\\ \end{aligned} \]
Example :
find the cdf F of the random variable X in example 3 above.
Solution: note \(X \in \{1,2,3,...\}\)
let \(A_i\) be the event “a six on the \(i^{th}\) roll”, i=1,2,3, …. Then
and
so for \(k \le x < k+1\) we have \(F(x)=1-(5/6)^k\)
Probability Density Function (pdf)
The probability density function of a discrete r.v. X is defined by \(f(x) = P(X=x)\)
Note:
\(f(x)=P(X=x)=P(X\le x)-P(X \le x-1) = F(x)-F(x-1)\)
Example
the pdf of X in example 3 is given by
\(f(x) = 1/6*(5/6)^{x-1}\) if \(x \in \{1,2,..\}\), 0 otherwise.
Note that it follows from the definition and the axioms that for any density f we have
\[ \begin{aligned} &f(x) \ge 0 \\ &\sum_x f(x) = 1\\ \end{aligned} \]
f is the density of a continuous random variable with cdf F if
\(F(x)=\int_{-\infty}^x f(t) dt\)
Again it follows from the definition and the axioms that for any pdf f we have
\[ \begin{aligned} &f(x) \ge 0 \\ &\int_{-\infty}^{\infty} f(x) dx = 1\\ \end{aligned} \]
Example
Show that \(f(x)= \lambda \exp(- \lambda x)\) if x>0, 0 otherwise defines a pdf, where \(\lambda >0\).
clearly \(f(x) \ge 0\) for all x.
\[ \begin{aligned} & \int_{-\infty}^{\infty} f(x) dx = \\ & \int_{0}^{\infty} \lambda \exp(-\lambda t) dt = \\ & -\exp(-\lambda t)|_{0}^{\infty} = 0-(-1) = 1\\ \end{aligned} \]
This r.v. X is called an exponential r.v. with rate \(\lambda\).
Random Vectors
A random vector is a multi-dimensional random variable.
Example
we roll a fair die twice. Let X be the sum of the rolls and let Y be the absolute difference between the two roles. Then (X,Y) is a 2-dimensional random vector. The joint density of (X,Y) is given by:
| 0 | 1 | 2 | 3 | 4 | 5 | |
|---|---|---|---|---|---|---|
| 2 | 1 | 0 | 0 | 0 | 0 | 0 |
| 3 | 0 | 2 | 0 | 0 | 0 | 0 |
| 4 | 1 | 0 | 2 | 0 | 0 | 0 |
| 5 | 0 | 2 | 0 | 2 | 0 | 0 |
| 6 | 1 | 0 | 2 | 0 | 2 | 0 |
| 7 | 0 | 2 | 0 | 2 | 0 | 2 |
| 8 | 1 | 0 | 2 | 0 | 2 | 0 |
| 9 | 0 | 2 | 0 | 2 | 0 | 0 |
| 10 | 1 | 0 | 2 | 0 | 0 | 0 |
| 11 | 0 | 2 | 0 | 0 | 0 | 0 |
| 12 | 1 | 0 | 0 | 0 | 0 | 0 |
where every number is divided by 36.
All definitions are straightforward extensions of the one-dimensional case.
Example
for a discrete random vector we have the density \(f(x,y) = P(X=x,Y=y)\).
Say above
f(4,0) =
P(X=4, Y=0) =
P({(2,2)}) = 1/36
or
f(7,1) =
P(X=7,Y=1) =
P({(3,4),(4,3)}) = 1/18
Example
Say \(f(x,y)=cxy, 0\le x<y \le 1\) is a pdf. Find c.
so c=8.
Say (X,Y) is a discrete (continuous) r.v. with joint density (pdf) f. Then the marginal density (pdf) \(f_X\) is given by
\[ \begin{aligned} &f_X(x) =\sum_y f(x,y)\text{ if Y is discete} \\ &f_X(x) =\int_{-\infty}^\infty f(x,y)dy\text{ if Y is continuous} \\ \end{aligned} \]
Example
For the discrete example above we find
\(f_X(2) = f(2,0) + f(2,1) + .. + f(2,5) = 1/36\)
or
\(f_Y(3) = 6/36\)
Example
Say \(f(x,y)=8xy, 0 \le x<y \le 1\), find \(f_Y(y)\)
Note that \(f_Y\) is s proper pdf: \(f_Y(y) \ge 0\) and
Conditional R.V.’s
let (X,Y) be a discrete r.v. with joint density f(x,y) and marginals density \(f_X\) and \(f_Y\). For any x such that \(f_X(x)>0\) the conditional density \(f_{Y|X=x}(y|x)\) is defined by
\[f_{Y|X=x}(y|x)=\frac{f(x,y)}{f_Y(y)} \]
Example
find \(f_{X|Y=5}(7|5)\) and \(f_{Y|X=3}(7|3)\)
For continous r.v. everything works the same:
Example
Find \(f_{X|Y=y}(x|y)\)
\[ \begin{aligned} &f_{X|Y=y}(x|y) = \\ &\frac{f(x,y)}{f_Y(y)} = \\ &\frac{8xy}{4y^3} = \frac{2x}{y^2}\\ \end{aligned} \] for \(0 \le x\le y\).
Here y is a fixed number!
Again, note that a conditional pdf is a proper pdf:
Note that a conditional density (pdf) requires a specification for a value of the random variable on which we condition, something like \(f_{X|Y=y}\). An expression like \(f_{X|Y}\) is not defined!
Independence
Two r.v. X and Y are said to be independent iff
\(f_{X,Y}(x,y)=f_X(x)f_Y(y)\)
Example
in the example above we found \(f_{X,Y}(7,1) = 1/18\) but \(f_X(7)f_Y(1) = 1/6*10/36=5/108\), so X and Y are not independent
Mostly the concept of independence is used in reverse: we assume X and Y are independent (based on good reason!) and then make use of the formula:
Say we use the computer to generate 10 independent exponential r.v’s with rate \(\lambda\). What is the probability density function of this random vector?
We have \(f_{X_i}(x_i)=\lambda \exp(-\lambda x_i)\) for i=1,2,..,10 so
\[ \begin{aligned} &f_{(X_1,..,X_{10})}(x_1, .., x_{10}) = \\ &\prod_{i=1}^{10} f_{X_i}(x_i) = \\ &\prod_{i=1}^{10} \lambda \exp(-\lambda x_i)= \\ &\lambda^{10} \exp(-\lambda \sum_{i=1}^{10} x_i) = \\ \end{aligned} \]
Notation: we will use the notation \(X \perp Y\) if X and Y are independent.