📚 4B Theory of Unbiased Estimators | 4B 无偏估计量理论
In statistical inference, we rarely know the true value of a population parameter. Instead, we use sample data to compute an estimate—a single number that serves as our best guess. But how do we know whether our method of estimation is reliable? The theory of unbiased estimators provides a foundational criterion: on average, an estimator should equal the true parameter. This chapter develops the definition, properties, and proofs behind unbiasedness, with special attention to the sample mean and variance. We also explore related concepts such as mean squared error, efficiency, and the Gauss–Markov theorem.
在统计推断中，我们很少知道总体参数的真实值。我们使用样本数据计算出一个估计值——一个作为最佳猜测的数字。但我们如何知道估计方法是否可靠？无偏估计量理论提供了一个基本准则：平均来说，估计量的期望值应当等于真实参数。本章将阐述无偏性的定义、性质和证明，重点关注样本均值和方差。我们还将探讨均方误差、有效性以及高斯–马尔可夫定理等相关概念。
1. From Parameter to Estimator | 从参数到估计量
mean μ or variance σ², is a fixed but unknown constant. We collect a random sample X₁, X₂, … stimator is a function of the random sample that provides a numerical approximation of the parameter. For example, the sample mean X̄ e sample is observed, the numerical value obtained is called an estimate. In this chapter we treat estimators as random variables because their values vary with the sample.
总体参数，如均值 μ 或方差 σ²，是一个固定但未知的常数。我们从总体中抽取一个容量为 n 的随机样本 X₁, X₂, …, Xₙ。估计量是随机样本的函数，用以提供参数的数值近似。例如，样本均值 X̄ = (Σ Xᵢ)/n 是 μ 的一个估计量。一旦样本被观测到，所得数值称为估计值。本章中我们将估计量视为随机变量，因为它们的值随样本而变化。
We denote an estimator of a parameter θ by θ̂ (theta-hat). The challenge is to choose a function that performs well, not just for one sample, but across all possible samples. The property of unbiasedness addresses the long-run average behaviour of θ̂.
我们用 θ̂ 表示参数 θ 的估计量。挑战在于选择一个不仅在某个样本中，而是在所有可能样本中表现良好的函数。无偏性描述了 θ̂ 的长期平均行为。
2. Defining Unbiasedness | 无偏性的定义
An estimator θ̂ is said to be an unbiased estimator of θ if the expected value of θ̂ equals θ for all possible values of θ.
E(θ̂) = θ
This means that if we were to repeatedly take independent samples of size n, calculate θ̂ each time, and average these estimates, the long-run average would be exactly the true parameter θ. Unbiasedness does not guarantee that a single estimate is correct, but it ensures that the estimation procedure has no systematic tendency to overestimate or underestimate.
如果对于所有可能的 θ 值，估计量 θ̂ 的期望值都等于 θ，则称 θ̂ 为 θ 的无偏估计量。这意味着，如果我们反复抽取容量为 n 的独立样本，每次计算 θ̂ 并取平均，长期平均值将恰好等于真实参数 θ。无偏性并不保证某一次估计是正确的，但它确保估计过程没有系统性的高估或低估倾向。
If E(θ̂) ≠ θ, the estimator is said to be biased. The bias is defined as Bias(θ̂) = E(θ̂ s, a small amount of bias may be acceptable if other properties, such as lower variance, are gained.
如果 E(θ̂) ≠ θ，则该估计量称为有偏的。偏差定义为 Bias(θ̂) = E(θ̂) − θ。正偏差意味着估计量倾向于高估；负偏差则倾向于低估。在许多实际情形中，如果能够获得其他性质（如更小的方差），微小的偏差可能是可以接受的。
3. Unbiased Estimator of the Population Mean | 总体均值的无偏估计量
The sample mean X̄ tor of the population mean μ. We prove its unbiasedness using the linearity of expectation.
样本均值 X̄ = (1/n) Σ Xᵢ 是总体均值 μ 最自然的估计量。我们利用期望的线性性质证明其无偏性。
Assume X₁, X₂, …, Xₙ are independent and identically distributed (i.i.d.) with E(Xᵢ) = μ. Then:
E(X̄) = E[(1/n) Σ Xᵢ] = (1/n) Σ E(Xᵢ) = (1/n) · nμ = μ
Thus, X̄ is an unbiased estimator of μ. This result holds regardless of the population distribution, as long as the mean exists. It is one of the most important facts in statistics because it justifies the use of the sample average as a fair estimate of the centre of the population.
因此，X̄ 是 μ 的无偏估计量。这一结果不依赖于总体分布，只要均值存在即可。这是统计学中最重要的结论之一，因为它证明了使用样本平均数作为总体中心的无偏估计是合理的。
Because E(X̄ sample mean is also unbiased in small samples. This property forms the basis for many inferential procedures, including confidence intervals and hypothesis tests about μ.
由于对于任何样本容量 n 都有 E(X̄) = μ 严格成立，样本均值在小样本下也是无偏的。这一性质为许多推断过程奠定了基础，包括关于 μ 的置信区间和假设检验。
4. Biased and Unbiased Variance Estimators | 方差的有偏与无偏估计量
μ)²]. Two intuitive sample-based estimators arise: one dividing the sum of squared deviations by n, and the other by n−1.
总体方差为 σ² = E[(X − μ)²]。基于样本有两种直观的估计量：一种用 n 除以离差平方和，另一种用 n−1。
Define the "population variance estimator" with denominator n:
σ̂² = (1/n) Σ (Xᵢ − X̄)²
And the sample variance with denominator n−1:
S² = (1/(n−1)) Σ (Xᵢ − X̄)²
通过计算期望可以发现，σ̂² 是有偏的，而 S² 是无偏的。这种差异源于用 X̄ 替代 μ 所损失的一个自由度。
5. Proving S² is Unbiased | 证明样本方差 S² 的无偏性
To prove E(S²) = σ², we expand the sum of squares. Let X̄ be the sample mean. The key identity is:
Σ (Xᵢ − X̄)² = Σ (Xᵢ − μ)² − n(X̄ − μ)²
Taking expectations and using E[(X̄ − μ)²] = Var(X̄) = σ²/n, and E[(Xᵢ − μ)²] = σ², we find:
E[Σ (Xᵢ − X̄)²] = nσ² − n(σ²/n) = (n−1)σ²
1/(n−1)] · (n−1)σ² = σ², confirming that S² is an unbiased estimator of σ². In contrast, σ̂ stimating σ² systematically. The factor (n−1)/n is the bias, which shrinks as n increases.
因此，E(S²) = [1/(n−1)] · (n−1)σ² = σ²，证实了 S² 是 σ² 的无偏估计量。相反，σ̂² 的期望为 ((n−1)/n)σ²，系统性地低估了 σ²。因子 (n−1)/n 即为偏差，随着 n 增大而减小。
This proof is a classic result in any IB Statistics option. It clarifies why statistical software and textbooks routinely use n−1 in the denominator. The loss of one degree of freedom occurs because we have used the data to estimate μ via X̄ before measuring variability.
该证明是 IB 统计模块中的经典结果。它解释了为什么统计软件和教科书通常使用 n−1 作分母。损失一个自由度的原因是我们在度量变异性之前已经用数据估计了 μ（通过 X̄）。
6. Unbiasedness and Sample Size | 无偏性与样本容量
Unbiasedness is a property of the estimator' asedness alone does not guarantee that an estimator is close to the true parameter in any one sample. The variance of the estimator also matters.
无偏性是估计量公式的性质，而非某个具体样本的性质。它对于任何样本容量 n 都成立（对于方差要求 n ≥ 2）。然而，单凭无偏性并不能保证在某一次样本中估计值接近真实参数。估计量的方差同样重要。
For X̄, Var(X̄) = σ²/n. So even though X̄ mples give unbiased estimators with higher probability of being near the population value.
对于 X̄，有 Var(X̄) = σ²/n。因此，尽管 X̄ 始终无偏，其精度随着 n 的增加而提高。同样地，S² 是无偏的，但其方差随着数据增多而减小。因此，更大的样本使得无偏估计量有更高的概率接近总体值。
In many exam questions, you are asked to check whether a newly proposed estimator is unbiased by computing its expectation. If the result equals the parameter, it is unbiased; otherwise, you calculate the bias.
在许多考试题中，会要求通过计算期望来判断一个新提出的估计量是否无偏。若结果等于参数则为无偏；否则需计算偏差。
7. Asymptotic Unbiasedness | 渐近无偏性
es infinity. Such an estimator is called asymptotically unbiased. The variance estimator σ̂ enominator n is asymptotically unbiased because its expectation ((n−1)/n)σ² → σ² as n → ∞.
有时一个估计量在有限样本下有偏，但其偏差随着样本容量趋于无穷而趋于零。这类估计量称为渐近无偏的。使用分母 n 的方差估计量 σ̂² 是渐近无偏的，因为当 n → ∞ 时，其期望 ((n−1)/n)σ² → σ²。
Asymptotic unbiasedness is a weaker condition than unbiasedness, but it is often invoked for maximum likelihood estimators and other large-sample methods. For example, the maximum likelihood estimator of σ² in a normal model uses denominator n, producing a biased estimator that is consistent and asymptotically unbiased.
渐近无偏性是一个比无偏性更弱的条件，但它常被用于极大似然估计量和其他大样本方法。例如，正态模型下 σ² 的极大似然估计量使用分母 n，产生的估计量是有偏但一致的，且渐近无偏。
8. Mean Squared Error: Beyond Unbiasedness | 均方误差：超越无偏性
Unbiasedness is not the only criterion for a good estimator. The mean squared error (MSE) combines both bias and variance:
MSE(θ̂) = E[(θ̂ − θ)²] = Var(θ̂) + [Bias(θ̂)]²
This decomposition shows that it is possible for a biased estimator to have a smaller MSE than an unbiased one if the reduction in variance outweighs the squared bias. In some contexts, a tiny bias is tolerated if it leads to much more precise estimates.
无偏性并不是好估计量的唯一准则。均方误差 (MSE) 综合了偏差与方差。这一分解表明，如果方差的减小超过偏差的平方，有偏估计量可能比无偏估计量具有更小的 MSE。在某些情形下，如果微小的偏差能带来更精确的估计，是会被接受的。
For example, in regularised regression or shrinkage estimation, we deliberately introduce bias to reduce variance and improve overall predictive accuracy. The sample variance S² is unbiased, but its MSE may be compared with that of a biased variant to see which performs better under a particular distribution.
例如，在正则化回归或收缩估计中，我们有意引入偏差以减少方差，从而改善整体预测准确性。样本方差 S² 是无偏的，但其 MSE 可与有偏变体进行比较，以观察在特定分布下哪个表现更好。
9. Efficiency and Minimum Variance Unbiased Estimators | 有效性与最小方差无偏估计量
Among all unbiased estimators of a parameter, the one with the smallest variance is called the minimum variance unbiased estimator (MVUE). The relative efficiency of two unbiased estimators θ̂₁ and θ̂₂ is defined as Var(θ̂₂)/Var(θ̂₁), provided Var(θ̂₁) < Var(θ̂₂). An estimator achieving the Cramér–Rao lower bound is fully efficient.
在一个参数的所有无偏估计量中，方差最小的那个称为最小方差无偏估计量 (MVUE)。两个无偏估计量 θ̂₁ 和 θ̂₂ 的相对有效性定义为 Var(θ̂₂)/Var(θ̂₁)（假设 Var(θ̂₁) < Var(θ̂₂)）。达到克拉美–拉奥下界的估计量是全有效的。
For a normal population with known variance, the sample mean X̄ is the MVUE of μ. No other unbiased estimator can have a smaller variance when the data are normal. Unbiasedness together with minimum variance makes X̄ the "best" in the class of unbiased estimators.
对于方差已知的正态总体，样本均值 X̄ 是 μ 的 MVUE。在数据正态时，任何其他无偏估计量都不可能具有更小的方差。无偏性加最小方差使得 X̄ 成为无偏估计类中的“最佳”。
10. Best Linear Unbiased Estimator (BLUE) | 最佳线性无偏估计量 (BLUE)
In more complex models such as linear regression, we often restrict attention to estimators that are linear combinations of the observations. The Gauss–Markov theorem states that under the classical regression assumptions (linearity, zero mean errors, homoscedasticity, uncorrelated errors), the ordinary least squares (OLS) estimators are the best linear unbiased estimators (BLUE).
在更复杂的模型（如线性回归）中，我们常常将注意力限制在观测值的线性组合型估计量上。高斯–马尔可夫定理指出，在经典回归假设（线性、误差均值为零、同方差性、误差不相关）下，普通最小二乘 (OLS) 估计量是最佳线性无偏估计量 (BLUE)。
"Best" here means minimum variance among all linear unbiased estimators. This property underpins much of classical econometrics and applied statistics. It shows how unbiasedness extends to multi-parameter settings and remains a cornerstone of estimation theory.
这里的“最佳”是指所有线性无偏估计量中方差最小。这一性质奠定了经典计量经济学和应用统计学的基石。它展示了无偏性如何拓展至多参数情形，并始终是估计理论的核心。
11. Unbiasedness and Maximum Likelihood | 无偏性与极大似然
Maximum likelihood estimators (MLEs) are often asymptotically unbiased but may be biased in small samples. For a normal distribution N(μ, σ²), the MLE of μ is X̄, which is unbiased. However, the MLE of σ² is (1/n) Σ (Xᵢ − X̄)², which has a negative bias. The MLE for the Poisson parameter λ is the sample mean, which is unbiased. So unbiasedness of MLEs depends on the parameter and distribution.
极大似然估计量 (MLE) 通常是渐近无偏的，但在小样本下可能是有偏的。对于正态分布 N(μ, σ²)，μ 的 MLE 为 X̄，是无偏的。然而，σ² 的 MLE 为 (1/n) Σ (Xᵢ − X̄)²，具有负偏差。泊松参数 λ 的 MLE 为样本均值，是无偏的。因此 MLE 的无偏性取决于参数和分布。
When an MLE is biased for finite n, we can sometimes multiply by a correction factor to obtain an unbiased version. This is exactly how we moved from σ̂² to S². Such adjustments are common when exact unbiasedness is required.
当 MLE 在有限 n 下有偏时，我们有时可以乘以校正因子以获得无偏版本。这正是我们从 σ̂² 转向 S² 的做法。当需要严格无偏时，此类调整很常见。
12. Summary and Practical Tips | 总结与实用提示
Unbiasedness is a repeated-sample property: E(θ̂) = θ. The sample mean X̄ ownward bias that disappears as n grows. Always check the expectation algebraically when testing for unbiasedness.
无偏性是一种重复样本性质：E(θ̂) = θ。样本均值 X̄ 始终是 μ 的无偏估计量。分母为 n−1 的样本方差是 σ² 的无偏估计量。使用 n 做分母会产生向下的偏差，该偏差随着 n 增大而消失。在检验无偏性时，始终应通过代数计算期望。
When constructing an estimator, consider not only bias but also variance and MSE. The concept of MVUE and BLUE formalises the search for precision within the class of unbiased estimators. Exam questions will often ask you to derive expectations, compute bias, and compare estimators. Mastering the definitions and the proof for sample variance is essential.
在构建估计量时，不仅要考虑偏差，还要考虑方差和 MSE。MVUE 和 BLUE 的概念正式化了在无偏估计类中寻求精度的过程。考试题常要求推导期望、计算偏差并比较估计量。掌握定义以及样本方差的证明至关重要。
Published by TutorHao | Mathematics Revision Series | aleveler.com
更多咨询请联系16621398022（同微信）
或截图后用「扫一扫 → 相册」选择本图