It is (relatively) easy to see, using the theorems we proved about \(aX+b\sim\mathcal N(a\mu+b, a^2\sigma^2)\) earlier that if \(X\sim\mathcal N(\mu,\sigma^2)\), then \[ \frac{X-\mu}{\sigma}\sim\mathcal N(0,1) \]

It's basis is the following:

Definition If \(Z\) is a standard normal variable, and \(W\) is an independent \(\chi^2(n)\) variable, then the \(T\) distribution on \(n\) degrees of freedom is the distribution of \(Z/\sqrt{W/n}\).

\(X\sim\mathcal N(\mu,\sigma^2)\)

We showed in the last lecture that \(W=(n-1)S^2/\sigma^2\sim\chi^2(n-1)\), and we know that \(Z=(\overline X-\mu)/(\sigma/\sqrt{n})\sim\mathcal N(0,1)\). \[ \frac{Z}{\sqrt{W/(n-1)}} = \frac {(X-\mu)/\color{blue}{(\sigma/\sqrt{n})}} {\sqrt{\frac{(n-1)S^2}{\sigma^2}\Big/\color{red}{(n-1)}}} = \frac{X-\mu} {\color{blue}{\sigma/\sqrt{n}}}\cdot \frac{\color{red}{\sqrt{n-1}}}{\sqrt{(n-1)S^2/\sigma^2}} = \\ \frac{X-\mu}{\sigma/\sqrt{n}} \cdot \frac{\sqrt{n-1}}{\sqrt{n-1}\cdot\sqrt{S^2}/\sqrt{\sigma^2}} = \frac {(X-\mu)\color{green}{\sqrt{n-1}}} {\color{orange}{\sigma}\cdot \color{green}{\sqrt{n-1}}\cdot S/ (\sqrt{n}\cdot\color{orange}{\sigma})} \]

The actual \(T\) distribution can be found by, given \(Z\) and \(W\), compute the density functions \(p(t|w)\) and \(p(w)\), and then integrating \(\int p(t|w)p(w)dw\) to find

\[ p_{T(n)}(t) = \left[\frac {\Gamma[(n+1)/2]} {\sqrt{\pi n}\Gamma(n/2)} \right]\cdot \left( 1+\frac{t^2}{n} \right)^{-(n+1)/2} \]

Returning to the confidence intervals for \(\mu\) that we developed earlier:

\[ \frac{\overline X-\mu}{S/\sqrt{n}} \sim T(n-1) \]

so with \(t_{\alpha}=\text{CDF}^{-1}_{T(n-1)}(1-\alpha)\), we get the confidence interval \[ \overline X-t_{\alpha/2}S/\sqrt{n} <\mu< \overline X+t_{\alpha/2}S/\sqrt{n} \]

A new gunpowder was tested by measuring muzzle velocities (ft/s) on eight shots:

\(n=8\), so we use the distribution \(T(7)\).

Suppose \(X_1,\dots,X_n\sim\mathcal N(\mu_X,\sigma^2)\) and \(Y_1,\dots,Y_m\sim\mathcal N(\mu_Y,\sigma^2)\) come from populations with the same variance.

Then, first, \[ Z = \frac{(\overline X-\overline Y)-(\mu_X-\mu_Y)}{\sqrt{\frac{\sigma_X^2}{n}+\frac{\sigma_Y^2}{m}}} \sim\mathcal N(0,1) \\ Z = \frac{(\overline X-\overline Y)-(\mu_X-\mu_Y)}{\sigma\sqrt{\frac{1}{n}+\frac{1}{m}}} \sim\mathcal N(0,1) \]

\(X_1,\dots,X_n\sim\mathcal N(\mu_X,\sigma^2)\) and \(Y_1,\dots,Y_m\sim\mathcal N(\mu_Y,\sigma^2)\).

An unbiased estimator of \(\sigma^2\) is the pooled estimator: \[ S_p^2 = \frac {\sum (X_i-\overline X)^2 + \sum(Y_j-\overline Y)^2} {n+m-2} \\ W = \frac{(n+m-2)S_p^2}{\sigma^2} = \frac{\sum(X_i-\overline X)^2}{\sigma^2} + \frac{\sum(Y_j-\overline Y)^2}{\sigma^2} \] \(W\) is a sum of two independent \(\chi^2\)-distributed random variables. Since \(\chi^2\) is the distribution of a sum of standard normals, the degrees of freedom of a sum of \(\chi^2\) variables is the sum of degrees of freedom. \(W\sim\chi^2(n+m-2)\)

\(X_1,\dots,X_n\sim\mathcal N(\mu_X,\sigma^2)\) and \(Y_1,\dots,Y_m\sim\mathcal N(\mu_Y,\sigma^2)\). \(S_p^2=\left(\sum (X_i-\overline X)^2 + \sum(Y_j-\overline Y)^2\right)/(n+m-2)\) and \(W=(n+m-2)S_p^2/\sigma^2\sim\chi^2(n+m-2)\).

\[ \frac{Z}{\sqrt{W/\nu}} = \left[ \frac{(\overline X-\overline Y)-(\mu_X-\mu_Y)} {\color{green}{\sigma}\sqrt{1/n+1/m}} \right]{\Huge/} \sqrt{\frac {\color{orange}{(n+m-2)}S_p^2} {\color{green}{\sigma^2}\color{orange}{(n+m-2)}}} = \\ =\frac{(\overline X-\overline Y)-(\mu_X-\mu_Y)} {S_p\sqrt{1/n+1/m}} \sim T(n+m-2) \]

\(X_1,\dots,X_n\sim\mathcal N(\mu_X,\sigma^2)\) and \(Y_1,\dots,Y_m\sim\mathcal N(\mu_Y,\sigma^2)\). \(S_p^2=\left(\sum (X_i-\overline X)^2 + \sum(Y_j-\overline Y)^2\right)/(n+m-2)\).

\[ T = \frac{(\overline X-\overline Y)-(\mu_X-\mu_Y)} {S_p\sqrt{1/n+1/m}} \sim T(n+m-2) \] Write \(\Delta\overline X=\overline X-\overline Y\) and \(\Delta\mu=\mu_X-\mu_Y\). Write \(t_\alpha=\text{CDF}_{T(n+m-2)}^{-1}(1-\alpha)\). \[ \Delta\overline X-t_{\alpha/2}S_p\sqrt{\frac1n+\frac1m} <\Delta\mu< \Delta\overline X+t_{\alpha/2}S_p\sqrt{\frac1n+\frac1m} \]

Consider a family of non-randomized hypothesis tests at level \(\alpha\) of \(H_0:\theta=\theta_0\) vs \(H_1:\theta\neq\theta_0\).

The acceptance region \(A(\theta_0)\) given a specific null hypothesis contains \(X\) with probability \(1-\alpha\) under the null hypothesis.

Write \(S(x) = \{\theta: x\in A(\theta)\}\). Then \(\theta\in S(X)\) if and only if \(X\in A(\theta)\), so \[ \mathbb P(\theta\in S(X)) = \mathbb P(X\in A(\theta)) \geq 1-\alpha \]

So \(S(X)\) is a \(1-\alpha\) confidence region for \(\theta\).

Given a confidence region \(S(X)\), define a test function \(\phi(X)=1-\mathbb 1_{S(X)}(\theta_0)\).

This is a test for \(H_0:\theta=\theta_0\) vs \(H_1:\theta\neq\theta_0\). We can calculate its level: \[ \mathbb E_{\theta_0}\phi = \mathbb P(\theta_0\not\in S(X)) \leq \alpha \]

Suppose \(S(X)\) is a confidence region such that \(\phi(X)=1-\mathbb 1_{S(X)}(\theta_0)\) is uniformly most powerful for \(H_0:\theta=\theta_0\) vs \(H_1:\theta>\theta_0\) with level \(\alpha\).

Let \(S^*(X)\) be a competing confidence region, and \(\phi^*(X)=1-\mathbb 1_{S^*(X)}(\theta_0)\) its dual test.

Because \(\phi\) is uniformly most powerful, for \(\theta>\theta_0\), the power bound is \[ \mathbb E_\theta\phi\geq \mathbb E_\theta\phi^* \] Here \[ \mathbb E_\theta\phi = \mathbb P(\theta_0\not\in S(X)|\theta) \qquad \mathbb E_\theta\phi^* = \mathbb P(\theta_0\not\in S^*(X)|\theta) \qquad \]

Suppose \(S(X)\) is a confidence region such that \(\phi(X)=1-\mathbb 1_{S(X)}(\theta_0)\) is uniformly most powerful for \(H_0:\theta=\theta_0\) vs \(H_1:\theta>\theta_0\) with level \(\alpha\).

\[ \mathbb P(\theta_0\not\in S^*(X)|\theta) \leq \mathbb P(\theta_0\not\in S(X)|\theta) \]

Among all \(1-\alpha\) confidence intervals, if \(\theta\) is the true value, \(S(X)\) has the lowest probability of containing any incorrect value \(\theta_0<\theta\).

Claim If \(S(X)\) has the lowest probability of containing an incorrect value for \(\theta\), \(S(X)\) is the shortest \(1-\alpha\) confidence interval for \(\theta\). Let \(\lambda\) be the Lebesgue measure on \(\mathbb R\).

\[ \mathbb E[\lambda(S(X))|\theta] = \mathbb E\left[ \int_{-\infty}^\theta \mathbb 1_{S(X)}(\theta_0)d\theta_0 \middle|\theta\right] = \\ \int \int_{-\infty}^\theta \mathbb 1_{S(X)}(\theta_0)d\theta_0 d\mathbb P_\theta = \int_{-\infty}^\theta \mathbb P(\theta_0\in S(X))d\theta_0 \]

The same derivation holds for \(S^*\). So if the probability is lower, so is the expected Lebesgue measure.

\(X_1,\dots,X_n\sim\mathcal N(0,\sigma^2)\).

The uniformly most powerful test of \(H_0:\sigma=\sigma_0\) vs. \(H_1:\sigma>\sigma_0\) is produced by comparing a complete sufficient statistic to a threshold. \(S^2\) is complete sufficient for \(\sigma^2\).

Hence a uniformly most powerful statistic takes the shape \[ \phi(S^2) = \begin{cases} 1 & S^2 > c \\ 0 & S^2 < c \end{cases} \qquad \mathbb P(S^2>c|\sigma_0) = \alpha \]

\(X_1,\dots,X_n\sim\mathcal N(0,\sigma^2)\).

Since we know \((n-1)S^2/\sigma_0^2\sim\chi^2(n-1)\) if the null hypothesis holds, we may pick \(\chi^2_\alpha=\text{CDF}_{\chi^2(n-1)}^{-1}(1-\alpha)\), so that under the null hypothesis with probability \(\alpha\) \[ \frac{(n-1)S^2}{\sigma_0^2} > \chi^2_\alpha \qquad S^2 > \frac{\chi^2_\alpha\sigma_0^2}{n-1} \] as our rejection region.

\(X_1,\dots,X_n\sim\mathcal N(0,\sigma^2)\) and our uniformly most powerful test has acceptance region for \(S^2\) given by \(A(\sigma_0) = (-\infty,\chi^2_\alpha\sigma_0^2/(n-1))\).

By duality, the confidence region is \[ S(X) = \{ \sigma : S^2\in A(\sigma) \} = \left\{\sigma : S^2 < \frac{\chi^2_\alpha\sigma^2}{n-1} \right\} = \\ \left(\sqrt{\frac{(n-1)S^2}{\chi^2_\alpha}}, \infty\right) \]

\(Y_1, Y_2\sim\text{Uniform}(0,\theta)\), \(X=Y_1+Y_2\). We observe \(X\).

Claim The densities for \(X\) have monotonic likelihood ratios.

In general, adding random variables produces a convolution of densities: \[ p_{X}(x) = \int_{-\infty}^\infty p_{Y_1}(y)p_{Y_2}(x-y) dy \] Now \(p_{Y_1}(y) = p_{Y_2}(y) = 1/\theta\) if \(0<y<\theta\). So \(p_{Y_1}(y)p_{Y_2}(x-y) = 1/\theta^2\) if \(0 < y < x < 2\theta\).

The convolution reduces, for \(0<x<2\theta\) to \[ =\int_0^x\theta^{-2}dy = x/\theta^2 \]

\(Y_1, Y_2\sim\text{Uniform}(0,\theta)\), \(X=Y_1+Y_2\). We observe \(X\).

Claim The densities for \(X\) have monotonic likelihood ratios.

If now \(\theta_1 < \theta_2\), then the likelihood ratio is when defined \[ \frac{\mathcal L(\theta_2|x)}{\mathcal L(\theta_1|x)} = \frac{x/\theta_2^2}{x/\theta_1^2} \]

1. \(x<0\) or \(2\theta_2 < x\): the ratio is undefined: 0/0.
2. \(0 < x < 2\theta_1\): the ratio is constant \(\theta_1^2/\theta_2^2\)
3. \(2\theta_1 < x < 2\theta_2\): the ratio is \(\infty\) (because \(\mathcal L(\theta_1|x)=0\))

\(Y_1, Y_2\sim\text{Uniform}(0,\theta)\), \(X=Y_1+Y_2\). We observe \(X\), which is a statistic with monotonic likelihood ratios.

The Uniformly Most Powerful test will reject \(H_0:\theta=\theta_0\) in favor of \(H_1:\theta>\theta_0\) if \(X>c\) for some \(c\).

To pick \(c\), assume \(H_0\). If \(c>2\theta_0\), the probability is \(0\), so we can ignore that case and calculate \[ \mathbb P(X>c | \theta_0) = \int_c^{\infty}d\mathbb P = \int_c^{2\theta_0}x/\theta_0^2dx = (2\theta_0-c)/\theta_0^2 \\ \alpha = 2/\theta_0 - c/\theta_0^2 \\ c = (2/\theta_0-\alpha)\theta_0^2 = 2\theta_0-\alpha\theta_0^2 \]

\(Y_1, Y_2\sim\text{Uniform}(0,\theta)\), \(X=Y_1+Y_2\). We observe \(X\), which is a statistic with monotonic likelihood ratios.

The uniformly most powerful test rejects when \(X > 2\theta_0-\alpha\theta_0^2\).

Its confidence set is \[ S(X) = \{ \theta : X < 2\theta-\alpha\theta^2 \} = \left(-\infty, \frac{\sqrt{1-x\alpha}+1}{\sqrt{\alpha}}\right) \]

\(X_i\sim\mathcal N(t_i\theta, 1)\) with known \(t_i\), unknown \(\theta\).

Task Find a uniformly most powerful test and its confidence region.

\(X_i\sim\mathcal N(t_i\theta, 1)\) with known \(t_i\), unknown \(\theta\).

Then \(X_i/t_i\sim\mathcal N(\theta, 1/t_i^2)\). So \[ \hat X = \frac1n\sum\frac{X_i}{t_i} \sim \mathcal N\left(\theta, \sum\frac{1}{n^2t_i^2}\right) \] Because the normal distribution is an exponential family, \(\hat X\) has a monotonic likelihood ratio and a UMP takes the shape of rejecting \(H_0:X=\theta_0\) in favor of \(H_1:X>\theta_0\) when \(\hat X>c\).

\(X_i\sim\mathcal N(t_i\theta, 1)\) with known \(t_i\), unknown \(\theta\). \(\hat X=\sum X_i/nt_i\) is a complete sufficient statistic with monotonic likelihood ratio.

Pick \(z_{\alpha}=\text{CDF}^{-1}_{\mathcal N(0,1)}(\alpha)\).

\[ \frac{\hat X-\theta}{\sqrt{\sum 1/n^2t_i^2}}\sim\mathcal N(0,1) \\ \text{accept when} \\ z_{\alpha/2} < \frac{\hat X-\theta_0}{\sqrt{\sum1/n^2t_i^2}} < z_{1-\alpha/2} \]

\(X_i\sim\mathcal N(t_i\theta, 1)\) with known \(t_i\), unknown \(\theta\). \(\hat X=\sum X_i/nt_i\).

The dual confidence interval is \[ S(X) = \left\{ \theta : z_{\alpha/2} < \frac{\hat X-\theta_0}{\sqrt{\sum1/n^2t_i^2}} < z_{1-\alpha/2} \right\} = \\ \left\{ \theta : z_{\alpha/2}\sqrt{\sum 1/n^2t_i^2} < \hat X-\theta_0 < z_{1-\alpha/2}\sqrt{\sum 1/n^2t_i^2} \right\} = \\ \left(\hat X+z_{\alpha/2}\sqrt{\sum 1/n^2t_i^2}\quad,\quad \hat X+z_{1-\alpha/2}\sqrt{\sum 1/n^2t_i^2} \right) \]