Stochastic Processes – Problem 19: Prove that the density function of is where ,

Let $X_{1}, X_{2}, \ldots, X_{n}$ be independent continuous random variables with common density function $f$. Denote by $X_{(i)}$ the $i$-th smallest among $X_{1}, X_{2}, \ldots, X_{n}$.

(a) Prove that the density function of $X_{(i)}$ is $$f_{X_{(i)}}(x)=\frac{n!}{(i-1)!(n-i)!}(F(x))^{i-1}(\overline{F}(x))^{n-i}f(x)$$ where $F(x)=\int_{-\infty}^{x}f(y)\,dy$, $\overline{F}(x)=1-F(x)$.

(b) Show that $P(X_{(i)}\leqslant x)=\sum_{k=i}^{n}\binom{n}{k}[F(x)]^{k}[\overline{F}(x)]^{n-k}$.

(c) Using the preceding two parts, prove the probability identity: $$\sum_{k=i}^{n}\binom{n}{k}y^{k}(1-y)^{n-k}=\int_{0}^{y}\frac{n!}{(i-1)!(n-i)!}x^{i-1}(1-x)^{n-i}\,dx.$$

(d) Let $T_{i}$ denote the arrival time of the $i$-th event in a Poisson process $N(t),t\geqslant0$. Find $E[T_{i}|N(t)=n]$ (consider the cases $i\leqslant n$ and $i>n$ separately).

(e) Compute the conditional density function of $T_{1},T_{2},\dots,T_{n-1}$ given $T_{n}=t$.

(a) Step 1. Consider the event $x < X_{(i)} \le x+dx$. Partition the $n$ i.i.d. variables into three groups: $i-1$ in $(-\infty, x]$, 1 in $(x, x+dx]$, $n-i$ in $(x+dx, \infty)$.

Step 2. By independence and the multinomial formula: $P(A) = \frac{n!}{(i-1)!\,1!\,(n-i)!} [F(x)]^{i-1} [f(x)dx + o(dx)] [1-F(x+dx)]^{n-i}$.

Step 3. Dividing by $dx$ and letting $dx \to 0^+$: $f_{X_{(i)}}(x) = \frac{n!}{(i-1)!(n-i)!} [F(x)]^{i-1} [1-F(x)]^{n-i} f(x)$.

(b) $\{X_{(i)} \le x\}$ = "at least $i$ observations $\le x$". Let $Y = \#\{j: X_j \le x\} \sim \text{Binomial}(n, F(x))$. Then $P(X_{(i)} \le x) = P(Y \ge i) = \sum_{k=i}^{n} \binom{n}{k} [F(x)]^k [1-F(x)]^{n-k}$.

(c) Step 1. From (a): $F_{X_{(i)}}(x) = \int_{-\infty}^{x} \frac{n!}{(i-1)!(n-i)!} [F(t)]^{i-1} [1-F(t)]^{n-i} f(t)\,dt$.

Step 2. Substituting $u = F(t)$: $F_{X_{(i)}}(x) = \int_0^{F(x)} \frac{n!}{(i-1)!(n-i)!} u^{i-1}(1-u)^{n-i}\,du$.

Step 3. Setting $y = F(x)$ and equating with (b) yields the identity.

(d) Step 1. When $i \le n$: Given $N(t)=n$, $T_i \stackrel{d}{=} t \cdot U_{(i)}$ where $U_{(i)}$ is the $i$-th order statistic of $n$ i.i.d. Uniform$(0,1)$. Using the Beta function: $E[U_{(i)}] = i/(n+1)$, so $E[T_i|N(t)=n] = it/(n+1)$.

Step 2. When $i > n$: By the memoryless property, $T_i - T_n = \sum_{j=n+1}^{i} E_j$ with $E_j \sim \text{Exp}(\lambda)$ i.i.d. Thus $E[T_i|N(t)=n] = t + (i-n)/\lambda$.

(e) Step 1. Given $T_n = t$, $(T_1,\dots,T_{n-1})$ are distributed as the order statistics of $n-1$ i.i.d. Uniform$(0,t)$ variables.

Step 2. Their joint density is $\frac{(n-1)!}{t^{n-1}}$ on $0 < t_1 < \cdots < t_{n-1} < t$.

(a) QED. (b) $\sum_{k=i}^{n}\binom{n}{k}[F(x)]^k[\overline{F}(x)]^{n-k}$. (c) QED. (d) $\frac{it}{n+1}$ for $i \le n$; $t + \frac{i-n}{\lambda}$ for $i > n$. (e) $\frac{(n-1)!}{t^{n-1}}$, for $0 < t_1 < \cdots < t_{n-1} < t$.