Random Vectors

A random vector is a multi-dimensional random variable.

Example 1 : we roll a fair die twice. Let X be the sum of the rolls and let Y be the absolute difference between the two roles. Then (X,Y) is a 2-dimensional random vector. The joint pmf of (X,Y) is given by:
X\Y 0 1 2 3 4 5
2 1 0 0 0 0 0
3 0 2 0 0 0 0
4 1 0 2 0 0 0
5 0 2 0 2 0 0
6 1 0 2 0 2 0
7 0 2 0 2 0 2
8 1 0 2 0 2 0
9 0 2 0 2 0 0
10 1 0 2 0 0 0
11 0 2 0 0 0 0
12 1 0 0 0 0 0

where every number is divided by 36.


all definitions are straightforward extensions of the one-dimensional case.

Example for a discrete random vector we have the pmf f(x,y) = P(X=x,Y=y)
Say f(4,0) = P(X=4, Y=0) = P({(2,2)}) = 1/36 or f(7,1) = P(X=7,Y=1) = P({(3,4),(4,3)}) = 1/18

Example Say f(x,y)=cxy is a pmf with x{1,2,3} and y{0,2}. Find c.

1=∑xy f(x,y) = f(1,0)+f(1,2)+f(2,0)+f(2,2)+f(3,0)+f(3,2) =

c(1·0+1·2+2·0+2·2+3·0+3·2) = 12c

so c=1/12

Example Say f(x,y)=cxy, 0≤x,y≤1 is a pdf. Find c.

so c=4.

Example Say f(x,y)=cxy, 0≤x<y≤1 is a pdf. Find c.

so c=8.

Example Say (X,Y) is a discrete rv with joint pmf f(x,y)=cpx, x,y{0,1,..}, y≤x, and 0<p<1. Find c

Example Say (X,Y,Z) is a continuous rv with f(x,y,z) = c(x+y)z if 0<x,y,z<1 and 0 otherwise. Find c

so c=2

Example Let's extend the idea of a uniform random variable to two dimensions. To begin, let's start with the unit square [0,1]2. Again, the idea of uniform is taken to mean that the probability of a point (X,Y) being in some area is proportional to the size of the area. Therefore if A is some area in [0,1]2 we have

P((X,Y)∈A) = area(A)

say 0<x,y<1, then

F(x,y) = P(X<x,Y<y) = area([0,x]*[0,y]) = xy

f(x,y) = d2/dxdy F(x,y) =d/dx[d/dy(xy)] = d/dx[x] = 1

Now say (X,Y) is uniform on {(x,y}: 0<x<yα<1} for some α>0. Find the joint pdf of (X,Y)

First we need the total area:

so f(x,y)=α+1 if 0<x<yα<1

Marginal Distributions

Definition

Say (X,Y) is a discrete (continuous) r.v. with joint pmf (pdf) f. Then the marginal pmf (pdf) fX is given by

Example Say X is the sum and Y is the absolute difference of two dice. If we add the row and column totals to the table above we get
X\Y 0 1 2 3 4 5 Total
2 1 0 0 0 0 0 1
3 0 2 0 0 0 0 2
4 1 0 2 0 0 0 3
5 0 2 0 2 0 0 4
6 1 0 2 0 2 0 5
7 0 2 0 2 0 2 6
8 1 0 2 0 2 0 5
9 0 2 0 2 0 0 4
10 1 0 2 0 0 0 3
11 0 2 0 0 0 0 2
12 1 0 0 0 0 0 1
Total 6 10 8 6 4 2 36

and these are the marginals. For example we find fX(2) = 1/36 or fY(3) = 6/36

Example Say (X,Y) is a rv with joint pmf f(x,y)=xy/12 with x{1,2,3} and y{0,2} Now

fX(3) = f(3,0) + f(3,2) = 3·0·1/12+3·2·1/12 = 6/12 = 1/2

fY(0) = f(1,0) + f(2,0) + f(3,0) = 0

Example Say (X,Y) is a rv with joint pdf f(x,y)=8xy, 0≤x<y≤1. Find fY(y)


Note that fY(y) is s proper pdf: fY(y)≥0 for all y and

Example Say (X,Y,Z) is a continuous rv with f(x,y,z) = 2(x+y)z if 0<x,y,z<1 and 0 otherwise.

Conditional R.V.'s

Definition

let (X,Y) be a r.v. with joint pmf (pdf) f(x,y) and marginal fY. For any y such that fY(y)>0 the conditional pmf (pdf) of X|Y=y is defined by


Note that a conditional pmf (pdf) requires a specification for a value of the random variable on which we condition, something like fX|Y=y. An expression like fX|Y is not defined!

 

Note that is is exactly the same as the definition for conditional probabilities of events. For example if (X,Y) is a discrete rv, then

Example Say X is the sum and Y is the absolute difference of two dice. Find fX|Y=5(7|5) and fY|X=3(7|3)
X\Y 0 1 2 3 4 5 Total
2 1 0 0 0 0 0 1
3 0 2 0 0 0 0 2
4 1 0 2 0 0 0 3
5 0 2 0 2 0 0 4
6 1 0 2 0 2 0 5
7 0 2 0 2 0 2 6
8 1 0 2 0 2 0 5
9 0 2 0 2 0 0 4
10 1 0 2 0 0 0 3
11 0 2 0 0 0 0 2
12 1 0 0 0 0 0 1
Total 6 10 8 6 4 2 36


Example f(x,y)=8xy, 0≤x<y≤1. Find fX|Y=y(x|y)

fX|Y=y(x|y) = f(x,y)/fY(y) = 8xy/4y3 = 2x/y2, 0≤x≤y.


Here y is a fixed number!

Again, note that a conditional pdf is a proper pdf:

Example Say (X,Y,Z) is a continuous rv with f(x,y,z) = 2(x+y)z if 0<x,y,z<1 and 0 otherwise.

Example say f(x,y)=α+1 if 0<x<yα<1 Find the marginals and the conditional pdf's. Verify that they are proper pdf's

Independence

Definition

Two r.v. X and Y are said to be independent iff

fX,Y(x,y)=fX(x)fY(y) for all x,y

Notation: we will use the notation X Y if X and Y are independent

Example Say X is the sum and Y is the absolute difference of two dice. Previously we found

fX,Y(7,1) = 1/18

but

fX(7)fY(1) = 1/6·10/36=5/108

so X and Y are not independent

Theorem

say f(x,y) is the joint pdf (pmf) of a random vector (X,Y). Then X and Y are independent if there exist functions g and h such that

f(x,y)=g(x)h(y)

proof: the only difference between the definition and the theorem is that g and h need not be proper densities. But

first of all we can assume that g and h are non-negative, otherwise just take |g| and |h|. Moreover

where 0<c<∞, so g/c = fX , and similarly h/d=fY

Example say f(x,y)=exp(-x-y), x,y>0. Then

f(x,y) = exp(-x-y) = exp(-x)exp(-y) = g(x)h(y)

so X and Y are independent.


Mostly the concept of independence is used in reverse: we assume X and Y are independent (based on good reason!) and then make use of the formula:


Example Say we use the computer to generate 10 independent exponential r.v's with rate λ. What is the probability density function of this random vector?

We have fXi(xi)=λexp(-λxi) for i=1,2,..,10 so

f(X1,..,X10)(x1, .., x10) = λexp(-λx1) *..* λexp(-λx10) = λ10exp(-λ(x1+..+x10))


Example Say (X,Y) is a discrete random vector with
x\y 1 2
1 1/10 1/10
2 1/10 1/2
3 1/10 1/10


Find the conditional pmf of X|Y=y

fX|Y=y(x|y)=f(x,y)/fY(y)

fY(y) = ∑xf(x,y)

fY(1) = f(1,1)+f(2,1)+f(3,1)=3/10
fY(2) = f(1,2)+f(2,2)+f(3,2)=7/10

so

fX|Y=1(1|1)=f(1,1)/fY(1) = (1/10)/(3/10) = 1/3
fX|Y=1(2|1)=f(2,1)/fY(1) = (1/10)/(3/10) = 1/3
fX|Y=1(3|1)=f(3,1)/fY(1) = (1/10)/(3/10) = 1/3
so

fX|Y=2(1|2)=f(1,2)/fY(2) = (1/10)/(7/10) = 1/7
fX|Y=2(2|2)=f(2,2)/fY(2) = (1/2)/(7/10) = 5/7
fX|Y=2(3|2)=f(3,2)/fY(2) = (1/10)/(7/10) = 1/7
so

Example Let the continuous random vector (X,Y) have joint density f(x,y)=e-y, 0<x<y<∞

Show that f is indeed a proper density

Find fY|X=x(y|x)

fY|X=x(y|x) = f(x,y)/fX(x)

so

Show that fY|X=x(y|x) is also a proper density
<

Example We have a "device" which generates a random number Y according to an exponential distribution with rate λ. We don't know exactly what λ is, but we do know that λ=x with probability 0.5x where x=1,2,3,... Find the pdf of Y. Verify that your answer is a proper pdf.

We have a discrete r.v X with pmf

fX(x)=0.5x, x=1,2,..

and a conditional rv Y with pmf

fY|X=x(y|x)=xexp(-xy), y>0

We want fY(y). It turns out that if we are dealing with a continuous rv. it is often better to first find the cdf FY(y)=P(Y≤y). Now first we have


a little bit of care: the geometric series ∑qk only converges if |q|<1. Here y>0, so ey>1 so 1/2ey<0.5<1, we are save.

Now


This type of model is called a hierarchical model, with one rv defined conditional on another. This way of describing a model is very useful in real live.

Law of Total Probability

We have previously seen the law of total probability for events. There are corresponding versions for random variables:

Discrete-Discrete
Say X and Y are discrete rv's with pmf's fX and fY, resp. Let B={X=x} and Ay={Y=y}. Then {Ay, all yS} form a partition and we have

fX(x) = P(X=x) = P(B) = ∑yP(B|Ay)P(Ay) = ∑yfX|Y=y(x|y)fY(y)

Discrete-Continuous
Say X is a discrete rv with pmf fX and Y is a continuous rv with density fY
Here we need to be careful: for a discrete rv fX(x)=P(X=x) makes sense, but for a continuous one we have

P(Y=y) = limh→0P(y≤Y≤y+h) = limh→0yy+h fY(t)dt = limh→0(FY(y+h)-FY(y)) = 0 for all y

first we condition on the discrete rv.: Now the event B={Y=y} does not work because P(B)=0 for all y. Let's instead consider the event B={Y≤y}:

For conditioning on the continuous rv we need to define a new discrete rv Y' with

Y'= if ih≤Y<(i+1)h

Then

because this is a Riemann sum, so it converges to the corresponding integral.

Continuous-Continuous Actually, same as above, the same proof works for this case as well!

Example back to the example above with the "device". Now we have the following solution:

Example again the example above with the "device", but now the rate X has a uniform distribution on [0,1], that is fX(x)=1 if 0<x<1. Then: