A random variable (r.v.) X is set-valued function from the sample space into the real numbers.
We roll a fair die, X is the number shown on the die
We roll a fair die, X is 1 if the die shows a six, 0 otherwise.
We roll a a fair die until the the first “6”, X is the number of rolls needed.
We randomly pick a time between 10am and 12 am, X is the minutes that have passed since 10am.
If X takes countably many values, X is called a discrete r.v.
If X takes uncountably many values, X is called a continuous r.v.
There are also mixtures of these two.
In the first three examples above X is discrete, in the last one X is continuous.
There are some technical difficulties when defining a r.v. on a sample space like \(\mathbb{R}\), it turns out to be impossible to define it for every subset of \(\mathbb{R}\) without getting logical contradictions. The solution is to define a \(\sigma\)-algebra on the sample space and then define X only on that \(\sigma\)-algebra. We will ignore these technical difficulties.
Almost everything to do with r.v.’s has to be done twice, once for discrete and once for continuous r.v.’s. This separation is only artificial, it goes away once a more general definition of “integral” is used (Riemann-Stilties or Lebesgue)
The distribution function of a r.v. X is defined by
\(F(x) = P(X \le x) \text{ }\forall x \in \mathbb{R}\)
We roll a fair die, X is the number shown on the die. Say x=2.2, then
\(F(2.2) = P(X \le 2.2) = P({1,2}) = 2/6 =1/3\)
We randomly pick a time between 10am and 12 am, X is the minutes that have passed since 10am. Say x=67.5, then
\(F(67.5) = P(X \le 67.5)\) = P(we chose a moment between 10am and 11h7.5min am) = 67.5/120 = 0.5625
Some features of cdf’s:
\[ \begin{aligned} &F(x) \rightarrow 0 \text{ as } x \rightarrow - \infty\\ &F(x) \rightarrow 1 \text{ as } x \rightarrow \infty\\ \end{aligned} \]
We roll a a fair die until the the first “6”, X is the number of rolls needed. Let’s find the cdf F.
note \(X \in \{1,2,3,...\}\)
let \(A_i\) be the event “a six on the \(i^{th}\) roll”, i=1,2,3, …. Then
\[ \begin{aligned} &P(X=k) =P(A_1^c\cap ..\cap A_{k-1}^c\cap A_k) = \\ &P(A_1^c)\times ..\times P(A_{k-1}^c)P(A_k) = = \\ &\frac56\times..\times\frac56\frac16 =(\frac56)^{k-1}\frac16 \\ &P(X\le k)=\sum_{i=1}^k P(X=i) = \\ &\sum_{i=1}^k(\frac56)^{i-1}\frac16 = \\ &\frac16\sum_{j=0}^{k-1}(\frac56)^{j} = \\ &\frac16\frac{1-(5/6)^{k-1+1}}{1-5/6} =1-(5/6)^{k} \end{aligned} \]
so for \(k \le x < k+1\) we have \(F(x)=1-(5/6)^k\)
The probability density function of a discrete r.v. X is defined by \(f(x) = P(X=x)\)
Note:
\(f(x)=P(X=x)=P(X\le x)-P(X \le x-1) = F(x)-F(x-1)\)
the pdf of X in the example above is given by
\(f(x) = 1/6*(5/6)^{x-1}\) if \(x \in \{1,2,..\}\), 0 otherwise.
Note that it follows from the definition and the axioms that for any density we have
\[ \begin{aligned} &f(x) \ge 0 \\ &\sum_x f(x) = 1\\ \end{aligned} \]
f is the of a continuous random variable with cdf F if
\(F(x)=\int_{-\infty}^x f(t) dt\)
Again it follows from the definition and the axioms that for any density f we have
\[ \begin{aligned} &f(x) \ge 0 \\ &\int_{-\infty}^{\infty} f(x) dx = 1\\ \end{aligned} \]
Show that \(f(x)= \lambda \exp(- \lambda x)\) if x>0, 0 otherwise defines a pdf, where \(\lambda >0\).
clearly \(f(x) \ge 0\) for all x.
\[ \begin{aligned} & \int_{-\infty}^{\infty} f(x) dx = \\ & \int_{0}^{\infty} \lambda \exp(-\lambda t) dt = \\ & -\exp(-\lambda t)|_{0}^{\infty} = 0-(-1) = 1\\ \end{aligned} \]
This r.v. X is called an exponential r.v. with rate \(\lambda\).
A random vector is a multi-dimensional random variable.
we roll a fair die twice. Let X be the sum of the rolls and let Y be the absolute difference between the two roles. Then (X,Y) is a 2-dimensional random vector. The joint of (X,Y) is given by:
| 0 | 1 | 2 | 3 | 4 | 5 | |
|---|---|---|---|---|---|---|
| 2 | 1 | 0 | 0 | 0 | 0 | 0 |
| 3 | 0 | 2 | 0 | 0 | 0 | 0 |
| 4 | 1 | 0 | 2 | 0 | 0 | 0 |
| 5 | 0 | 2 | 0 | 2 | 0 | 0 |
| 6 | 1 | 0 | 2 | 0 | 2 | 0 |
| 7 | 0 | 2 | 0 | 2 | 0 | 2 |
| 8 | 1 | 0 | 2 | 0 | 2 | 0 |
| 9 | 0 | 2 | 0 | 2 | 0 | 0 |
| 10 | 1 | 0 | 2 | 0 | 0 | 0 |
| 11 | 0 | 2 | 0 | 0 | 0 | 0 |
| 12 | 1 | 0 | 0 | 0 | 0 | 0 |
where every number is divided by 36.
All definitions are straightforward extensions of the one-dimensional case.
for a discrete random vector we have the \(f(x,y) = P(X=x,Y=y)\).
Say above
f(4,0) =
P(X=4, Y=0) =
P({(2,2)}) = 1/36
or
f(7,1) =
P(X=7,Y=1) =
P({(3,4),(4,3)}) = 1/18
Say \(f(x,y)=cxy, 0\le x<y \le 1\) is a pdf. Find c.
\[ \begin{aligned} &\int_{-\infty}^\infty \int_{-\infty}^\infty f(x,y) dxdy = \\ &\int_0^1 \int_0^y cxy dx dy = \\ &\int_0^1 cy \left[x^2/2\vert_0^y\right] dy = \\ &\int_0^1 cy \left[y^2/2\right] dy = \\ &\int_0^1 cy^3/2 dy = \\ &cy^4/8\vert_0^1 = c/8 =1 \end{aligned} \]
so c=8.
Say (X,Y) is a discrete (continuous) r.v. with joint density f. Then the marginal density \(f_X\) is given by
\[ \begin{aligned} &f_X(x) =\sum_y f(x,y)\text{ if Y is discete} \\ &f_X(x) =\int_{-\infty}^\infty f(x,y)dy\text{ if Y is continuous} \\ \end{aligned} \]
For the discrete example above we find
\(f_X(2) = f(2,0) + f(2,1) + .. + f(2,5) = 1/36\)
or
\(f_Y(3) = 6/36\)
Say \(f(x,y)=8xy, 0 \le x<y \le 1\), find \(f_Y(y)\)
\[ \begin{aligned} &f_X(x) = \int_{-\infty}^\infty f(x,y) dy = \\ &\int_x^1 8xy dy = \\ &4xy^2\vert_x^1 = 4x-4x^3;0<x<1 \end{aligned} \]
Note that \(f_X\) is s proper density.
let (X,Y) be a discrete r.v. with joint f(x,y) and marginals \(f_X\) and \(f_Y\). For any x such that \(f_X(x)>0\) the conditional \(f_{Y|X=x}(y|x)\) is defined by
\[f_{Y|X=x}(y|x)=\frac{f(x,y)}{f_Y(y)} \]
For the example above find \(f_{X|Y=y}(x|y)\)
\[ \begin{aligned} &f_{X|Y=y}(x|y) = \\ &\frac{f(x,y)}{f_Y(y)} = \\ &\frac{8xy}{4y^3} = \frac{2x}{y^2}\\ \end{aligned} \] for \(0 \le x\le y\).
Here y is a fixed number!
Note that a conditional density requires a specification for a value of the random variable on which we condition, something like \(f_{X|Y=y}\). An expression like \(f_{X|Y}\) is not defined!