Demo: Detection Theory Examples
Switch between two common textbook calculations: a signed mean-shift statistic and an energy statistic for a variance change. The point is to notice that different alternatives call for different sufficient statistics.
Mathematical setup
Mean-shift example:
\[H_0:X_i\sim N(0,1), \qquad H_1:X_i\sim N(\mu_1,1), \qquad T=\bar X.\]For the threshold rule $T>\gamma$,
\[P_{\mathrm{FA}}=1-\Phi(\gamma\sqrt n), \qquad P_D=1-\Phi((\gamma-\mu_1)\sqrt n).\]Variance-change example:
\[H_0:X_i\sim N(0,1), \qquad H_1:X_i\sim N(0,\sigma_1^2), \qquad T=\sum_{i=1}^n X_i^2.\]Under $H_0$, $T\sim\chi_n^2$; under $H_1$, $T/\sigma_1^2\sim\chi_n^2$, or equivalently $T\sim\sigma_1^2\chi_n^2$. For the rule $T>\gamma$,
\[P_{\mathrm{FA}}=\Pr(\chi_n^2>\gamma), \qquad P_D=\Pr\left(\chi_n^2>\frac{\gamma}{\sigma_1^2}\right).\]The displayed variance-change probabilities use these chi-squared survival functions.
What to try
- In mean-shift mode, raise $\mu_1$ and see detection improve without changing the false-alarm calculation for a fixed threshold.
- In variance-change mode, raise $\sigma_1$. The energy distribution under $H_1$ shifts right, so large-energy tests become more powerful.
- Compare threshold scales across the two modes. A reasonable threshold for $\bar X$ is not a reasonable threshold for $\sum_i X_i^2$.
For the variance-change example, the energy statistic uses the scaled chi-squared distribution: under $H_0$, $\sum_i X_i^2\sim\chi_n^2$, and under the larger-variance alternative, $\sum_i X_i^2/\sigma_1^2\sim\chi_n^2$.
Try it in Python
This cell covers both widget modes: the signed mean-shift statistic and the energy statistic for a variance change.
import numpy as np
import matplotlib.pyplot as plt
from scipy import stats
mode = "mean" # "mean" or "variance"
n = 12
gamma = 1.8
mu1 = 0.8
sigma1 = 1.8
if mode == "mean":
se = 1 / np.sqrt(n)
p_fa = stats.norm.sf(gamma, loc=0, scale=se)
p_d = stats.norm.sf(gamma, loc=mu1, scale=se)
grid = np.linspace(min(-1, gamma - 1), max(mu1 + 1, gamma + 1), 400)
plt.plot(grid, stats.norm.pdf(grid, 0, se), label="H0")
plt.plot(grid, stats.norm.pdf(grid, mu1, se), label="H1")
plt.axvline(gamma, color="black", linestyle="--", label="threshold")
plt.xlabel("sample mean")
else:
p_fa = stats.chi2.sf(gamma, df=n)
p_d = stats.chi2.sf(gamma / sigma1**2, df=n)
grid = np.linspace(0.001, max(35, sigma1**2 * n * 3), 500)
plt.plot(grid, stats.chi2.pdf(grid, df=n), label="H0")
plt.plot(grid, stats.chi2.pdf(grid / sigma1**2, df=n) / sigma1**2, label="H1")
plt.axvline(gamma, color="black", linestyle="--", label="threshold")
plt.xlabel("energy statistic")
plt.ylabel("density")
plt.legend()
plt.show()
print(f"mode = {mode}")
print(f"P_FA = {p_fa:.4f}")
print(f"P_D = {p_d:.4f}")