Stochastic Processes Solution – Problem 14: Prove that the density function of is where ,

Question

Let $X_{1}, X_{2}, \ldots, X_{n}$ be independent continuous random variables with common density function $f$ . Denote by $X_{(i)}$ the $i$ -th smallest among $X_{1}, X_{2}, \ldots, X_{n}$ .

(a) Prove that the density function of $X_{(i)}$ is $f_{X_{(i)}}(x)=\frac{n!}{(i-1)!(n-i)!}(F(x))^{i-1}(\overline{F}(x))^{n-i}f(x)$ where $F(x)=\int_{-\infty}^{x}f(y)\,dy$ , $\overline{F}(x)=1-F(x)$ .

(b) Show that $P(X_{(i)}\leqslant x)=\sum_{k=i}^{n}\binom{n}{k}[F(x)]^{k}[\overline{F}(x)]^{n-k}$ .

(c) Using the preceding two parts, prove the following probability identity: $\sum_{k=i}^{n}\binom{n}{k}y^{k}(1-y)^{n-k}=\int_{0}^{y}\frac{n!}{(i-1)!(n-i)!}x^{i-1}(1-x)^{n-i}\,dx.$

(d) Let $T_{i}$ denote the arrival time of the $i$ -th event in a Poisson process $N(t),t\geqslant0$ . Find $E[T_{i}|N(t)=n]$ (consider the cases $i\leqslant n$ and $i>n$ separately).

(e) Compute the conditional density function of $T_{1},T_{2},\dots,T_{n-1}$ given $T_{n}=t$ .

Step-by-step solution

(a) Step 1. Consider the event $x < X_{(i)} \le x+dx$ . Among the $n$ i.i.d. random variables $X_1,\dots,X_n$ , partition them into three groups: $i-1$ falling in $(-\infty, x]$ , exactly 1 falling in $(x, x+dx]$ , and $n-i$ falling in $(x+dx, \infty)$ .

Step 2. Compute the probability of this event. By independence and the multinomial formula: $P(A) = \frac{n!}{(i-1)!\,1!\,(n-i)!} [F(x)]^{i-1} [P(x < X \le x+dx)]^{1} [1-F(x+dx)]^{n-i}$ , where $P(x < X \le x+dx) = f(x)dx + o(dx)$ .

Step 3. Take the limit to obtain the density. Divide both sides by $dx$ and let $dx \to 0^+$ . By the definition of the derivative: $f_{X_{(i)}}(x) = \frac{n!}{(i-1)!(n-i)!} [F(x)]^{i-1} [1-F(x)]^{n-i} f(x)$ .

(b) $\{X_{(i)} \le x\} = \{\text{at least } i \text{ observations} \le x\}$ . Let $Y = \#\{j: X_j \le x\}$ . Then $Y \sim \text{Binomial}(n, F(x))$ , so $P(X_{(i)} \le x) = P(Y \ge i) = \sum_{k=i}^{n} \binom{n}{k} [F(x)]^{k} [1-F(x)]^{n-k}$ .

(c) Step 1. From (a), the CDF is $F_{X_{(i)}}(x) = \int_{-\infty}^{x} \frac{n!}{(i-1)!(n-i)!} [F(t)]^{i-1} [1-F(t)]^{n-i} f(t)\,dt$ .

Step 2. Substituting $u = F(t)$ , $du = f(t)\,dt$ : $F_{X_{(i)}}(x) = \int_{0}^{F(x)} \frac{n!}{(i-1)!(n-i)!} u^{i-1} (1-u)^{n-i}\,du$ .

Step 3. Setting $y = F(x)$ and equating with (b) yields the identity.

(d) Step 1. When $i \le n$ : Given $N(t)=n$ , $T_i$ has the same distribution as $t \cdot U_{(i)}$ , where $U_{(i)}$ is the $i$ -th order statistic of $n$ i.i.d. $\text{Uniform}(0,1)$ variables. Using the Beta function: $E[U_{(i)}] = \frac{i}{n+1}$ , so $E[T_i | N(t)=n] = \frac{it}{n+1}$ .

Step 2. When $i > n$ : By the memoryless property, $T_i - T_n = \sum_{j=n+1}^{i} E_j$ where $E_j \stackrel{\text{i.i.d.}}{\sim} \text{Exp}(\lambda)$ . Thus $E[T_i | N(t)=n] = t + (i-n)/\lambda$ .

(e) Step 1. Given $T_n = t$ , the first $n-1$ arrival times $(T_1,\dots,T_{n-1})$ are distributed as the order statistics of $n-1$ i.i.d. $\text{Uniform}(0,t)$ variables.

Step 2. The joint density of these order statistics is $\frac{(n-1)!}{t^{n-1}}$ on $0<t_1<\cdots<t_{n-1}<t$ .

Final answer

(a) QED. (b) $\sum_{k=i}^{n}\binom{n}{k}[F(x)]^{k}[\overline{F}(x)]^{n-k}$ . (c) QED. (d) $\frac{it}{n+1}$ for $i \le n$ ; $t + \frac{i-n}{\lambda}$ for $i > n$ . (e) $\frac{(n-1)!}{t^{n-1}}$ , for $0 < t_1 < t_2 < \cdots < t_{n-1} < t$ .

Marking scheme

The following is the rubric for this probability problem.

1. Checkpoints (max 7 pts total)

Notes:

Points must be awarded according to the logical chains below.
Merely copying formulas or listing known conditions earns no credit.
For parts with multiple solution paths (e.g., (a) and (e)), score whichever path yields the highest marks; do not combine across paths.
(a) Deriving the order statistic density (1 pt)
Core derivation: Use the infinitesimal method (considering $P(x < X_{(i)} \le x+dx)$ via the multinomial probability) or first derive the CDF and differentiate. Must show intermediate steps; writing only the final formula earns 0. [1 pt]
(b) Expressing the CDF as a binomial sum (1 pt)
Probability reformulation: Identify that $\{X_{(i)} \le x\}$ is equivalent to "at least $i$ observations are $\le x$ " (or similar counting argument), and correctly write the binomial sum. [1 pt]
(c) Proving the integral identity (1 pt)
Connecting the two expressions: Use $F_{X_{(i)}}(x) = \int_{-\infty}^x f_{X_{(i)}}(t)dt$ , apply the substitution $u=F(x)$ , and equate the integral form from (a) with the series form from (b). [1 pt]
(d) Conditional expectation for the Poisson process (2 pts)
Case $i \le n$ [additive]: Identify that given $N(t)=n$ , the arrival times $T_1,\dots,T_n$ are distributed as the order statistics of $n$ i.i.d. $\text{Uniform}(0,t)$ variables, and obtain $E[T_i|N(t)=n] = \frac{i}{n+1}t$ . [1 pt]
Case $i > n$ [additive]: Use the memoryless/renewal property of the Poisson process at time $t$ to show $E[T_i|N(t)=n] = t + \frac{i-n}{\lambda}$ . [1 pt]
(e) Conditional density function (2 pts)
Method identification [1 pt]: (A) State that given $T_n=t$ , $(T_1,\dots,T_{n-1})$ are distributed as the order statistics of $n-1$ i.i.d. $\text{Uniform}(0,t)$ variables; or (B) write the joint density divided by the marginal density of $T_n$ .
Final result [1 pt]: Write the correct density $\frac{(n-1)!}{t^{n-1}}$ with the domain $0 < t_1 < \dots < t_{n-1} < t$ stated or implied.

2. Zero-credit items

In (a), merely copying the formula for $f_{X_{(i)}}$ with no derivation.
In (c), only verifying a special case (e.g., $n=1, i=1$ ) without a general proof.
In (d), giving only the unconditional expectation (e.g., $i/\lambda$ ), completely ignoring the condition $N(t)=n$ .
In (d), incorrectly assuming $T_i$ is an $n$ -equipartition point on $(0,t)$ , yielding $\frac{i}{n}t$ .

3. Deductions

Logical error (Part d): In the $i > n$ case, computing $\frac{n}{n+1}t + \frac{i-n}{\lambda}$ (incorrectly mixing the conditional expectation of $T_n$ with unconditional increments): that sub-part earns 0.
Missing domain (Part e): Writing $\frac{(n-1)!}{t^{n-1}}$ without specifying $0 < t_1 < \dots < t_{n-1} < t$ and without mentioning order statistics: -1 pt.
Conceptual confusion: In (d) or (e), confusing $N(t)=n$ (count condition) with $T_n=t$ (time condition), leading to an incorrect distributional model: that part earns 0.

Total: ______ / 7

Stochastic Processes – Problem 14: Prove that the density function of is where ,