Demo: Point Estimation

For samples from $\mathrm{Uniform}(0,\theta)$, compare a moment estimator with a likelihood estimator that is driven by the support constraint.

Mathematical setup

Let $X_1,\ldots,X_n\sim\mathrm{Uniform}(0,\theta)$ iid. Since $E[X_i]=\theta/2$, the method-of-moments estimator is

The likelihood is

so the maximum likelihood estimator is $\hat\theta_{\mathrm{MLE}}=X_{(n)}$. It is biased downward with $E[X_{(n)}]=n\theta/(n+1)$, while the MOM estimator is unbiased.

What to try

• Use small $n$ to see the MLE’s downward bias clearly. It often has lower spread but tends to sit below the true endpoint.
• Increase $n$ and compare RMSE. Both estimators improve, but they do so for different reasons: averaging versus the maximum moving toward the boundary.
• Change $\theta$ after fixing $n$. The relative behavior is scale-stable, while the absolute RMSE scales with the endpoint.

The MLE uses the support constraint: values of $\theta$ below the sample maximum have likelihood zero.

Try it in Python

This simulation compares the method-of-moments estimator and the support-constrained MLE across many samples.

import numpy as np
import matplotlib.pyplot as plt

theta = 2.0
n = 12
reps = 1000
rng = np.random.default_rng(531)

x = rng.uniform(0, theta, size=(reps, n))
theta_mom = 2 * x.mean(axis=1)
theta_mle = x.max(axis=1)

def summarize(name, estimates):
    bias = estimates.mean() - theta
    rmse = np.sqrt(np.mean((estimates - theta)**2))
    print(f"{name:4s} mean={estimates.mean():.3f}, bias={bias:.3f}, RMSE={rmse:.3f}")

summarize("MOM", theta_mom)
summarize("MLE", theta_mle)
print(f"Theory E[MLE] = {n * theta / (n + 1):.3f}")

plt.hist(theta_mom, bins=35, alpha=0.45, density=True, label="MOM: 2 xbar")
plt.hist(theta_mle, bins=35, alpha=0.45, density=True, label="MLE: max")
plt.axvline(theta, color="black", linestyle="--", label="true theta")
plt.xlabel("estimate")
plt.ylabel("density")
plt.legend()
plt.show()

Back to topic notes