Do as much as you can analytically, and for the rest use R.
A rv X has a discrete uniform distribution on 1,..,N if P(X=k)=1/N for \(1\le k\le N\). Here N is some fixed positive integer. We write \(X\sim U\{1,..,N\}\).
Say \(X\sim U\{1,..,N\}\). Find E[X] and Var[X].
Say \(X_1,..,X_k\sim U\{1,..,N\}\) and independent. Say \(M_k=\max\{X_1,..,X_k\}\). Find \(E[M_k]\).
Data from a discrete uniform can be generated with the R sample command. Use this to verify your answers for N=10 and k=2.
\[ \begin{aligned} &E[X] =\sum_{i=1}^N i/N = (N(N+1)/2)/N = (N+1)/2 \\ &E[X^2] =\sum_{i=1}^N i^2/N = (N(N+1)(2N+1)/6)/N = (N+1)(2N+1)/6 \\ &Var[X]=E[X^2]-E{X}^2 = (N+1)(2N+1)/6 - [(N+1)/2]^2 = \\ &(N+1)(N-1)/12 \end{aligned} \]
N=10
x=sample(1:N, size=1e6, replace = TRUE)
round(c(mean(x), (N+1)/2), 2)## [1] 5.5 5.5round(c(var(x), (N+1)*(N-1)/12), 2)## [1] 8.25 8.25\[ \begin{aligned} &P(M_k\le m) =P(X_1\le m, .., X_k\le m) =\\ &P(X_1\le m)\times ..\times P(X_k\le m) = \text{ (by independence)}\\ &P(X_1\le m)^k = \left( \frac{m}{N}\right)^k \\ &P(M_k = m) = P(M_k \le m) - P(M_k \le m-1) = \\ & \left( \frac{m}{N}\right)^k - \left( \frac{m-1}{N}\right)^{k} \\ &E[M_k] =\sum_{m=1}^N m\left( \frac{m}{N}\right)^k - \left( \frac{m-1}{N}\right)^{k} =\\ &\frac{1}{N^k} \sum_{m=1}^N m\left( m^k - (m-1)^k \right) =\\ &\frac{1}{N^k} \left(\sum_{m=1}^N m^{k+1} - \sum_{m=1}^Nm(m-1)^k \right) \end{aligned} \]
N=10
x=matrix(sample(1:N, size=2*1e6, replace = TRUE), ncol=2)
M=apply(x, 1, max)
round(c(mean(M), 1/N^2*(sum((1:N)^3)-sum((1:N)*(1:N-1)^2))), 2)## [1] 7.15 7.15Say \(X\sim Exp(1)\) and \(Y|X=x \sim Pois(x)\). Find Var[Y].
\[ \begin{aligned} &f(x,y) = f_X(x)f_{Y|X=x}(y|x) = e^{-x}\frac{x^y}{y!}e^{-x} \\ &f_Y(y) =\int_0^\infty \frac{x^y}{y!}e^{-2x} dx =\\ &\int_0^\infty \frac1{\Gamma(y+1)}x^{(y+1)-1}e^{-2x} dx =\\ &\frac1{2^y}\int_0^\infty \frac{2^y}{\Gamma(y+1)}x^{(y+1)-1}e^{-2x} dx = \frac1{2^y} \end{aligned} \]
so \(Y\sim Geom(1/2)\) and so Var[Y]=(1-1/2)/(1/2)^2= 2
or much easier:
\[ \begin{aligned} &Var[Y] = Var[E\{Y|X\} ]+E[Var\{Y|X\} ] \\ &E\{Y|X=x\} = x; Var\{Y|X=x\}=x\\ &Var[Y] = Var[X]+E[X] =1+1=2\\ \end{aligned} \]
Say \(X\sim N(0, 1)\).
\[\psi(t)=E[e^{tX}]\]
\[ \begin{aligned} &\psi(t)=E[e^{tX}] =\int_{-\infty}^\infty e^{tx}\frac1{\sqrt{2\pi}} e^{-x^2/2}dx = \\ &\int_{-\infty}^\infty \frac1{\sqrt{2\pi}} \exp \{tx-x^2/2\}dx = \\ &\int_{-\infty}^\infty \frac1{\sqrt{2\pi}} \exp \{-(x^2-2tx+t^2)/2+t^2/2\}dx = \\ &e^{t^2/2}\int_{-\infty}^\infty \frac1{\sqrt{2\pi}} \exp \{-(x-t)^2/2\}dx = e^{t^2/2} \end{aligned} \] because the integral is over a density (N(t,1)) and so is 1.
\[ \begin{aligned} &P(X>a) = P(e^{tX}>e^{ta})\le \frac{E[e^{tX}]}{e^{ta}} = \\ &\frac{e^{t^2/2}}{e^{ta}} =e^{t^2/2-ta} \end{aligned} \]
Notice that in the inequality in b t only appears on the right side, so this inequality holds for all t>0, including th t that minimizes the right side:
\[P(X>a)\le \min\{e^{t^2/2-ta};t>0\}\]
but \(e^{t^2/2-ta}\) is minimized where \(t^2/2-ta\) is minimized. This is a parabola that opens upward, so it has its minimum at the vertex, which is at -(-a)/(2*1/2) = a, so
\[P(X>a)\le \min\{e^{t^2/2-ta};t>0\} = e^{a^2/2-a*a} = e^{-a^2/2}\]
(This is known as the Chernoff bound and it works for all random variables that have a moment generating function).