Least Squares Method: Principles and Applications | 最小二乘法原理与应用

📚 Least Squares Method: Principles and Applications | 最小二乘法原理与应用

In IB Mathematics, we often collect data and search for a meaningful relationship between two variables. Least squares is the standard technique for fitting a regression line that best represents the trend in a set of data points.

在IB数学中，我们经常收集数据并寻找两个变量之间的有意义关系。最小二乘法是拟合一条最能代表一组数据点趋势的回归直线的标准技术。

Unlike an interpolation polynomial that passes through every point, a regression line smooths out random error and reveals the underlying pattern. This makes least squares essential in statistics, science, and economics.

与精确穿过每个点的插值多项式不同，回归直线会平滑随机误差并揭示潜在的规律。这使得最小二乘法在统计学、科学和经济学中至关重要。

---

1. Introduction to Curve Fitting | 曲线拟合简介

Given a set of points (xᵢ, yᵢ), the goal is to find a straight line ŷ ummarises the overall pattern. The unknown parameters are the intercept a and the slope b.

给定一组点(xᵢ, yᵢ)，目标是找到一条直线ŷ = a + b x来概括整体趋势。未知参数是截距a和斜率b。

We cannot expect the line to pass through all points exactly, because real data contain measurement error and natural variability. Instead, we search for the line that stays as close as possible to all points in a certain mathematical sense.

我们不能期望直线精确穿过所有点，因为真实数据包含测量误差和自然变异性。相反，我们寻找在某种数学意义下尽可能接近所有点的直线。

---

2. The Core Idea of Least Squares | 最小二乘法的核心思想

The least squares principle begins with residuals. For each data point (xᵢ, yᵢ), the residual eᵢ is the vertical difference between the observed value and the predicted value from the model.

最小二乘原理从残差开始。对于每个数据点(xᵢ, yᵢ)，残差eᵢ是观测值与模型预测值之间的纵向差。

eᵢ = yᵢ - ŷᵢ = yᵢ - (a + b xᵢ)

We square each residual before summing them, so positive and negative errors do not cancel out. The objective is to minimise the sum of squared residuals.

我们将每个残差平方后再求和，这样正负误差不会相互抵消。目标是使残差平方和最小。

S = Σ eᵢ² = Σ (yᵢ - a - b xᵢ)²

Squaring also places a heavier penalty on large deviations, which is often desirable when fitting a model to data.

平方还对大的偏差施加更重的惩罚，这通常在将模型拟合到数据时是可取的。

---

3. Deriving the Least Squares Estimators | 推导最小二乘估计量

To minimise S, we use calculus. We take the partial derivatives of S with respect to a and b and set them equal to zero.

为了使S最小，我们使用微积分。我们求S关于a和b的偏导数并令其为零。

∂S/∂a = -2 Σ (yᵢ - a - b xᵢ) = 0

∂S/∂b = -2 Σ xᵢ (yᵢ - a - b xᵢ) = 0

After simplification, we obtain the normal equations:

化简后，我们得到正规方程：

Σ yᵢ = n a + b Σ xᵢ
Σ xᵢ yᵢ = a Σ xᵢ + b Σ xᵢ²

Solving these two linear equations gives the slope and intercept:

解这两个线性方程即可得到斜率和截距：

b = Sₓᵧ / Sₓₓ, a = ȳ - b x̄

where Sₓᵧ = Σ(xᵢ - x̄)(yᵢ - ȳ) and Sₓₓ = Σ(xᵢ - x̄)².

其中Sₓᵧ = Σ(xᵢ - x̄)(yᵢ - ȳ)，Sₓₓ = Σ(xᵢ - x̄)²。

---

4. The Normal Equations in Matrix Form | 矩阵形式的正规方程

itten as y = Xβ + ε, where X is the design matrix and β contains the unknown coefficients.

对于学习线性代数的学生，同样的结果可以用矩阵更简洁地表达。模型写作y = Xβ + ε，其中X是设计矩阵，β包含未知系数。

β̂ = (XᵀX)⁻¹Xᵀy

This matrix formula is important for multiple regression and provides a clear theoretical foundation. In IB, however, it is sufficient to use the scalar formulas or a calculator.

这个矩阵公式对于多元回归很重要，并提供了清晰的理论基础。但在IB课程中，使用标量公式或计算器就足够了。

---

5. Geometric Interpretation | 几何解释

The least squares solution has a beautiful geometric meaning. The vector of observed values y is projected onto the column space of the design matrix X.

最小二乘解具有优美的几何意义。观测值向量y被投影到设计矩阵X的列空间上。

The residual vector y - Xβ̂ is perpendicular to every column of X. This orthogonality condition is exactly what produces the normal equations.

残差向量y - Xβ̂与X的每一列都垂直。这个正交条件恰恰产生了正规方程。

Thus, the least squares line gives the closest possible prediction in the sense of Euclidean distance, within the space of all linear models.

因此，在所有线性模型构成的空间中，最小二乘直线在欧几里得距离意义下给出了最接近的预测。

---

6. Assumptions for Least Squares Regression | 最小二乘回归的假设条件

For the regression line to be meaningful and reliable, certain conditions must hold.

为了使回归直线有意义且可靠，必须满足某些条件。

• Linearity: the true relationship between x and y is approximately linear.

  线性：x与y之间的真实关系近似线性。

• Independence: the residuals are not correlated with one another.

  独立性：残差之间互不相关。

• Homoscedasticity: the variance of residuals remains constant across all values of x.

  同方差性：残差的方差在x的所有取值范围内保持不变。

• Normality: for statistical tests and confidence intervals, residuals are normally distributed.

  正态性：对于统计检验和置信区间，残差服从正态分布。

Violating these assumptions can lead to misleading estimates or invalid conclusions, so residual plots are always worth checking.

违反这些假设可能导致误导性的估计或无效的结论，因此残差图总是值得检查。

---

7. Coefficient of Determination R² | 决定系数 R²

Once the regression line is found, we want to measure how well it fits the data. The coefficient of determination R² provides this measure.

找到回归直线后，我们要衡量它对数据的拟合程度。决定系数R²提供了这一度量。

Define the total sum of squares SS_tot = Σ(yᵢ - ȳ)² and the residual sum of squares SS_res = Σ(yᵢ - ŷᵢ)².

定义总平方和SS_tot = Σ(yᵢ - ȳ)²，残差平方和SS_res = Σ(yᵢ - ŷᵢ)²。

R² = 1 - SS_res / SS_tot

R² takes values between 0 and 1. A value close to 1 means that most of the variation in y is explained by x, while a value close to 0 means the linear relation is weak.

R²取值在0到1之间。接近1的值意味着y的大部分变异性可由x解释，而接近0的值意味着线性关系较弱。

---

广告 / Sponsored

8. Correlation vs. Regression | 相关性与回归的区别

Correlation and regression are often confused, but they answer different questions. The correlation coefficient r measures the strength and direction of a linear relationship between two variables.

相关与回归经常被混淆，但它们回答的是不同的问题。相关系数r衡量两个变量之间线性关系的强度和方向。

Regression, on the other hand, uses one variable to predict the other. The regression line is expressed in the original units of y and x, whereas r is dimensionless.

另一方面，回归使用一个变量来预测另一个变量。回归直线以y和x的原始单位表示，而r是无量纲的。

Furthermore, regression is not symmetric: the line for predicting y from x differs from the line for predicting x from y. Correlation is symmetric because r does not depend on which variable is called x.

此外，回归不是对称的：用x预测y的直线与用y预测x的直线不同。相关是对称的，因为r不取决于哪个变量被称为x。

---

9. Applications in the IB Syllabus | 在IB课程中的应用

In both IB Mathematics AA and AI, least squares regression appears in the statistics section. Students are expected to use a graphic display calculator to find the regression line and the correlation coefficient.

在IB数学AA和AI中，最小二乘回归都出现在统计部分。学生应能使用图形计算器求回归直线和相关系数。

Typical applications include:

典型应用包括：

• Using bivariate data to predict an unknown y value within the range of the data.

  使用双变量数据在数据范围内预测未知的y值。

• Interpreting the slope and intercept in a real-world context, such as cost or growth.

  在成本或增长等现实情境中解释斜率和截距。

• Comparing the fit of different models using R² or r².

  使用R²或r²比较不同模型的拟合效果。

• Analysing residuals to decide whether a linear model is appropriate.

  分析残差以判断线性模型是否合适。

---

10. Worked Example | 例题详解

er the data points below. We will calculate the least squares regression line y = a + b x.

考虑以下数据点。我们将计算最小二乘回归直线y = a + b x。

x	1	2	3	4	5
y	2	3	5	7	8

Compute the necessary sums: n = 5, Σx = 15, Σy = 25, Σx² = 55, Σxy = 91.

计算所需的总和：n = 5，Σx = 15，Σy = 25，Σx² = 55，Σxy = 91。

Thus x̄ = 3 and ȳ = 5. Then:

因此x̄ = 3，ȳ = 5。于是：

Sₓₓ = 55 - 5 × 3² = 10
Sₓᵧ = 91 - 5 × 3 × 5 = 16

Hence b = 16 / 10 = 1.6, and a = 5 - 1.6 × 3 = 0.2. The regression line is:

因此b = 16 / 10 = 1.6，a = 5 - 1.6 × 3 = 0.2。回归直线为：

y = 0.2 + 1.6 x

This equation can now be used to estimate y for x values within the observed range.

该方程现在可用于在观测范围内根据x估计y值。

---

11. Common Pitfalls and Tips | 常见误区与备考提示

Students often lose marks in regression questions because of avoidable mistakes. Here are some common pitfalls and useful suggestions.

学生在回归题目中常因可避免的错误丢分。以下是一些常见误区和有用建议。

• Do not extrapolate far beyond the data range, because the linear trend may not continue.

  不要外推到数据范围之外很远，因为线性趋势可能不会继续。

• Do not confuse correlation with causation. A strong r value does not prove that x causes y.

  不要混淆相关与因果。较强的r值并不能证明x导致y。

• Check for outliers before trusting the regression line, as a single extreme point can pull the line strongly.

  在信任回归直线之前要检查异常值，因为单个极端点可能会强烈拉动直线。

• Always include the units and interpret the slope and intercept in context in exam answers.

  在考试答案中始终包含单位，并在情境中解释斜率和截距。

• Use the calculator's built-in linear regression function, but also understand the formula for calculation questions.

  使用计算器内置的线性回归功能，但也要理解公式以应对计算题。

By mastering these principles, you will be able to apply least squares confidently in the IB examination and in real-world data analysis.Published by TutorHao | IB Mathematics Revision Series | aleveler.com

更多咨询请联系16621398022（同微信）