📚 Statistics: From Data Collection to Evidence-Based Decisions | 统计学：从数据收集到循证决策
Statistics is the science of collecting, organising, analysing and interpreting data. It gives us a disciplined way to turn raw numbers into conclusions, predictions and tested hypotheses, and it is used across business, economics, the social sciences, medicine and many other fields.
统计学是探索数据收集、整理、分析与解释的科学。它为我们提供了一条严谨的路径，把原始数字转化为结论、预测与经过检验的假设，其应用遍及商业、经济、社会科学、医学等诸多领域。
This article follows the whole statistical workflow: how data are gathered, how they are described, how models are built, how conclusions are inferred, and where the limits of the subject lie.
本文将沿着统计工作的完整流程展开：数据如何获取、如何描述、如何建模、如何推断结论，以及这门学科的边界在哪里。
1. What Statistics Is | 统计学的定义与范畴
Statistics is often divided into two great branches. Descriptive statistics summarises what is already in a dataset - averages, spreads, charts and tables. Inferential statistics goes further and uses a sample to draw conclusions about a much larger population that was never fully measured.
统计学通常分为两大分支。描述统计负责概括数据集中已有的信息，如均值、离散程度、图表与交叉表；推断统计则更进一步，利用样本对从未被完整测量的更大总体做出结论。
The subject sits at the meeting point of mathematics and real-world reasoning. A statistical result is never just a number: it carries assumptions about how the data were produced and a statement about how uncertain the conclusion is.
这门学科处在数学与现实推理的交汇处。一个统计结果从来不只是一个数字：它背后隐含着关于数据如何产生的假设，也包含对结论不确定程度的说明。
A typical statistical investigation follows a repeatable cycle:
一次典型的统计研究遵循一个可重复的循环：
- Define the research question and the population of interest. | 明确研究问题与目标总体。
- Design the study and choose a sampling method. | 设计研究方案并选择抽样方法。
- Collect the data and check its quality. | 采集数据并检查数据质量。
- Explore, summarise and visualise the data. | 探索、概括并可视化数据。
- Build a model and carry out inference. | 建立模型并进行统计推断。
- Communicate the findings, including their uncertainty. | 汇报结论，并说明其不确定性。
2. Data Collection and Organisation | 数据的采集与整理
The foundation of statistics is data collection. No amount of clever modelling can rescue a dataset that was gathered badly, so the reliability of every later conclusion depends on how the data were obtained in the first place.
统计学的根基在于数据的采集。再精巧的建模也无法挽救一份采集得很糟的数据，因此后续每一个结论的可靠性，都取决于数据最初是如何获取的。
Common collection methods include field surveys, questionnaires, designed experiments, and the use of existing records such as government statistics, company databases or medical registries.
常见的数据收集手段有实地调研、问卷发放、实验设计，以及利用现有资料，例如政府统计、企业数据库或医疗登记档案。
Data can be classified in ways that determine which tools are appropriate:
数据可以按类型划分，而类型决定了适宜采用哪些工具：
- Categorical (nominal): unordered labels such as blood type or product category. | 定类数据：无顺序的标签，如血型或产品类别。
- Ordinal: ordered categories such as satisfaction ratings from 1 to 5. | 定序数据：有顺序的类别，如 1 至 5 分的满意度评分。
- Discrete numerical: counts such as the number of customers per day. | 离散数值数据：计数，如每日顾客人数。
- Continuous numerical: measurements such as height, temperature or reaction time. | 连续数值数据：测量值，如身高、温度或反应时间。
Once collected, data must be organised: coded, entered, checked for duplicates, grouped into frequency tables, and cross-tabulated so that patterns become visible. Cleaning is unglamorous but decisive - missing values, impossible values and inconsistent units must all be resolved before analysis begins.
数据到手后必须加以整理：编码、录入、查重，编成频数分布表，并做交叉列表，使规律变得可见。数据清洗虽不显眼却至关重要——缺失值、不可能出现的数值以及不一致的单位，都必须在分析开始前处理妥当。
3. Sampling Methods and Sample Size | 抽样方法与样本量
Because measuring an entire population is usually impossible, statistics works with samples. Choosing the sampling method and the sample size is therefore one of the most consequential decisions in any study.
由于对总体做完整测量通常不可行，统计学借助样本来工作。因此，抽样方法与样本量的选择，是任何研究中最具决定性的一步。
Probability sampling gives every member of the population a known, non-zero chance of selection, which is what makes formal inference possible:
概率抽样使总体中每个个体都有已知且非零的被选中机会，这正是进行正式统计推断的前提：
- Simple random sampling: every individual has an equal chance. | 简单随机抽样：每个个体机会均等。
- Systematic sampling: select every k-th item from a list. | 系统抽样：从名单中每隔 k 个抽取一个。
- Stratified sampling: divide the population into groups and sample within each. | 分层抽样：把总体分组，再在各组内抽样。
- Cluster sampling: randomly select whole groups, then study all members within them. | 整群抽样：随机抽取整个群体，再研究其中的全部成员。
Non-probability methods such as convenience, quota, purposive and snowball sampling are quicker and cheaper, but they cannot support the same strength of generalisation.
便利抽样、配额抽样、目的抽样与滚雪球抽样等非概率方法更快、更省钱，但无法支撑同等强度的推广结论。
For a simple random sample estimating a mean, a common planning formula is:
对于估计均值的简单随机抽样，常用的样本量规划公式为：
n = (z² × σ²) ÷ E²
Here z is the critical value from the normal distribution (1.96 for 95% confidence), σ is the assumed population standard deviation, and E is the acceptable margin of error. Larger samples, more variable populations and smaller tolerated error all push n upwards.
其中 z 为正态分布的临界值（95% 置信水平下为 1.96），σ 为假定的总体标准差，E 为可接受的误差范围。样本量更大、总体变异更强、允许误差更小，都会推高 n 的取值。
4. Data Quality and Sources of Error | 数据质量与常见误差
Statistical error comes in two broad kinds. Sampling error arises simply because we observe a sample rather than the whole population, and it shrinks as the sample grows. Non-sampling error covers everything else: non-response, measurement error, recording mistakes and biased question wording.
统计误差大致可分两类。抽样误差仅仅源于我们观察的是样本而非总体，它会随样本量增大而缩小。非抽样误差则涵盖其余一切问题：无应答、测量误差、记录错误以及有偏的问卷措辞。
Practical threats to data quality include:
实际中威胁数据质量的因素包括：
- Non-response bias: those who refuse to answer often differ systematically from those who do. | 无应答偏差：拒绝回答者往往与回答者在系统上存在差异。
- Leading or ambiguous questions that push respondents towards an answer. | 诱导性或含糊的提问，会把受访者推向某个答案。
- Social desirability bias, where people under-report behaviour they consider shameful. | 社会赞许偏差：人们会少报自己认为不光彩的行为。
- Instrument and observer error, such as a badly calibrated scale. | 仪器与观察者误差，例如未校准的秤。
- Missing data, which can silently distort any summary if it is not missing at random.
- 缺失数据：若缺失并非随机发生，它会在不知不觉中扭曲任何汇总结果。
Two concepts help here. Reliability means the measurement produces consistent results on repeated occasions; validity means it actually measures the construct it claims to measure. A scale can be reliable but invalid - consistently measuring the wrong thing.
有两个概念在此格外有用。信度指测量在重复场合下能产生一致的结果；效度指它确实测量了声称要测量的构念。一把尺子可以很可信却无效——它始终如一地测错了对象。
5. Descriptive Statistics | 描述统计：集中趋势与离散程度
Once data are clean, descriptive statistics reveal their shape. Measures of central tendency describe the typical value; measures of dispersion describe how spread out the values are; measures of shape describe symmetry and the weight of the tails.
数据清洗完成后，描述统计可揭示其形态。集中趋势指标描述典型取值，离散程度指标描述取值的分散情况，形态指标则描述对称性与尾部厚度。
The sample mean is the balance point of the data:
样本均值是数据的平衡点：
x̄ = (Σ xᵢ) ÷ n
The sample standard deviation summarises the typical distance from the mean:
样本标准差概括了各观测值到均值的典型距离：
s = √[ Σ (xᵢ − x̄)² ÷ (n − 1) ]
| Measure | 指标 | What it tells you | 说明 | Best used when | 适用情形 |
| Mean 均值 | Arithmetic average, uses every value | 算术平均，用到每个数值 | Symmetric data, no extreme outliers | 数据对称、无极端离群值 |
| Median 中位数 | Middle value when ordered | 排序后居中的数值 | Skewed data such as incomes | 收入等偏态数据 |
| Mode 众数 | Most frequent value | 出现次数最多的取值 | Categorical data | 分类数据 |
| IQR 四分位距 | Spread of the middle 50% | 中间 50% 的跨度 | Robust spread with outliers | 存在离群值时的稳健离散度 |
| CV 变异系数 | s ÷ x̄, relative variability | s ÷ x̄，相对变异程度 | Comparing units of different scale | 比较不同量纲的数据 |
A z-score standardises an observation, telling you how many standard deviations it lies from the mean:
z 分数对观测值做标准化，说明它距均值多少个标准差：
z = (x − μ) ÷ σ
Skewness measures asymmetry - a long right tail means a few very large values pull the mean above the median. Kurtosis measures how heavy the tails are, which matters because heavy tails make extreme events far more likely than a normal curve would suggest.
偏度衡量不对称性——右尾很长意味着少数极大值把均值拉到了中位数之上。峰度衡量尾部的厚重程度，这很重要，因为厚尾会让极端事件的发生概率远高于正态分布的预期。
6. Correlation and Association | 相关与关联
Correlation quantifies how strongly two variables move together. The Pearson correlation coefficient r ranges from −1 through 0 to +1, where the sign gives the direction and the magnitude gives the strength of a linear relationship.
相关衡量两个变量共同变动的强度。皮尔逊相关系数 r 的取值范围从 −1 经 0 到 +1，符号表示方向，绝对值表示线性关系的强弱。
r = Σ (xᵢ − x̄)(yᵢ − ȳ) ÷ √[ Σ (xᵢ − x̄)² × Σ (yᵢ − ȳ)² ]
Spearman rank correlation is used when the relationship is monotonic but not linear, or when the data are ordinal. For two categorical variables, the chi-square test of independence is the standard tool.
当关系单调但非线性，或数据为定序数据时，可使用斯皮尔曼等级相关。若两个变量均为分类变量，卡方独立性检验是标准工具。
The most important caution in all of statistics is that correlation does not imply causation. Two variables may move together because one causes the other, because causation runs in the opposite direction, or because a third confounding variable drives both.
整个统计学中最重要的告诫是：相关不等于因果。两个变量同步变动，可能是因为一者导致另一者，可能是因为因果方向相反，也可能是因为某个第三方的混杂变量同时驱动了二者。
Anscombe's quartet makes this vivid: four datasets with almost identical means, variances and correlations look completely different when plotted. Always draw the scatterplot before trusting a single number.
安斯库姆四组数据生动地说明了这一点：四组数据的均值、方差与相关系数几乎相同，但画成散点图后形态截然不同。在相信一个数字之前，永远先画出散点图。
7. Probability and Inferential Statistics | 概率与推断统计
Inference rests on probability. A random variable takes values with certain probabilities, and its distribution describes the whole pattern of those probabilities. The normal, binomial, Poisson and t distributions are the workhorses of introductory statistics.
推断建立在概率之上。随机变量以一定概率取到各个值，其分布描述了这些概率的整体模式。正态分布、二项分布、泊松分布与 t 分布是入门统计的骨干工具。
The central limit theorem is the bridge from sample to population: whatever the shape of the population distribution, the sampling distribution of the mean approaches normality as the sample size grows, with mean μ and standard error σ ÷ √n.
中心极限定理是从样本通往总体的桥梁：无论总体分布呈何形状，随着样本量增大，均值的抽样分布都会趋近正态，其均值为 μ，标准误为 σ ÷ √n。
Estimation comes in two forms. A point estimate gives a single best guess, such as x̄. An interval estimate gives a range with a stated confidence level:
估计有两种形式。点估计给出单一最优猜测，如 x̄；区间估计给出带有明确置信水平的范围：
CI = x̄ ± z × (σ ÷ √n)
Hypothesis testing follows a fixed logic. State a null hypothesis H₀ (usually no effect) and an alternative H₁, choose a significance level α, compute a test statistic, and find the p-value - the probability of observing data at least as extreme as these, assuming H₀ is true. If p < α, the result is described as statistically significant.
假设检验遵循固定逻辑：提出原假设 H₀（通常为无效应）与备择假设 H₁，选定显著性水平 α，计算检验统计量，并求出 p 值——即在 H₀ 成立的假定下，观察到至少与本数据同样极端结果的概率。若 p < α，则称结果具有统计显著性。
- Type I error (α): rejecting a true null hypothesis - a false alarm. | 第一类错误（α）：拒绝了真实的原假设，即虚警。
- Type II error (β): failing to reject a false null hypothesis - a missed effect. | 第二类错误（β）：未能拒绝错误的原假设，即漏检。
- Power (1 − β): the probability of detecting an effect that genuinely exists. | 检验功效（1 − β）：真实存在的效应被检测出来的概率。
Statistical significance is not practical importance. With a very large sample, a trivially small difference can produce a tiny p-value, so effect sizes and confidence intervals should always be reported alongside it.
统计显著性并不等于实际重要性。当样本极大时，微不足道的差异也可能产生极小的 p 值，因此报告中必须同时给出效应量与置信区间。
8. Regression Analysis | 回归分析
Regression examines how one or more explanatory variables relate to an outcome. Simple linear regression fits a straight line by ordinary least squares, choosing the line that minimises the sum of squared residuals.
回归分析研究一个或多个解释变量与结果变量之间的关系。简单线性回归用普通最小二乘法拟合一条直线，即选择使残差平方和最小的那条线。
y = β₀ + β₁x + ε
Here β₀ is the intercept, β₁ is the slope - the expected change in y for a one-unit increase in x - and ε is the random error term. Multiple regression extends this to several predictors, so that the effect of each is estimated while holding the others constant.
其中 β₀ 为截距，β₁ 为斜率，即 x 每增加一个单位时 y 的期望变化量，ε 为随机误差项。多元回归把它扩展到多个预测变量，从而在控制其他变量不变的前提下估计每一个变量的效应。
Ordinary least squares relies on assumptions that should always be checked:
普通最小二乘法依赖若干必须逐一检查的假设：
- Linearity: the relationship between predictors and outcome is linear in the parameters. | 线性：预测变量与结果之间在参数上呈线性关系。
- Independence: observations are not correlated with one another. | 独立性：观测之间互不相关。
- Homoscedasticity: the variance of residuals is constant across fitted values. | 同方差性：残差方差在拟合值上保持恒定。
- Normality of residuals, which matters mainly for small samples. | 残差正态性，主要在小样本时重要。
- No severe multicollinearity among predictors. | 预测变量之间不存在严重的多重共线性。
Goodness of fit is usually summarised by R², the proportion of variance in y explained by the model, and by the adjusted R², which penalises the addition of useless predictors. When the outcome is binary, logistic regression models the log-odds instead of the raw value.
拟合优度通常用 R² 概括，即模型解释的 y 的方差比例，并用校正 R² 对加入无用预测变量施加惩罚。当结果变量为二分类时，逻辑回归建模的是对数几率而非原始取值。
Regression coefficients are associations, not automatic proof of cause. Credible causal claims require a sound design: randomisation, natural experiments, instrumental variables, or difference-in-differences.
回归系数表达的是关联，而非自动成立的因果证据。可信的因果论断需要严谨的设计：随机化、自然实验、工具变量或双重差分法。
9. ANOVA, Factor Analysis and Time Series | 方差分析、因子分析与时间序列
When the aim is to compare the means of three or more groups, analysis of variance (ANOVA) is the standard approach. It splits total variation into variation between groups and variation within groups, then compares them with an F statistic.
当目标是比较三个及以上分组的均值时，方差分析（ANOVA）是标准方法。它把总变异拆分为组间变异与组内变异，再用 F 统计量加以比较。
F = MSbetween ÷ MSwithin
A large F suggests the group means differ by more than chance alone would produce. One-way ANOVA handles a single factor; two-way ANOVA handles two factors and, importantly, their interaction. If the overall test is significant, post-hoc comparisons such as Tukey's test identify which pairs actually differ.
F 值较大说明组间均值差异超出了随机波动的范围。单因素方差分析处理一个因素；双因素方差分析处理两个因素，并且重要的是能处理它们的交互作用。若总体检验显著，可用图基检验等事后比较找出真正存在差异的具体组别。
Factor analysis tackles a different problem: reducing many correlated variables to a smaller number of underlying dimensions. Exploratory factor analysis searches for structure, while confirmatory factor analysis tests a hypothesised structure. Loadings show how strongly each variable relates to each factor.
因子分析处理的是另一类问题：把众多相关的变量压缩为数量更少的潜在维度。探索性因子分析用于寻找结构，验证性因子分析则用于检验预设的结构。因子载荷显示每个变量与每个因子的关联强度。
Time series analysis deals with data ordered in time, which is usually autocorrelated - today's value depends partly on yesterday's. A series is typically decomposed into trend, seasonal, cyclical and irregular components.
时间序列分析处理按时间排序的数据，这类数据通常具有自相关——今天的取值部分取决于昨天。一个序列通常可分解为趋势、季节、循环与不规则四类成分。
- Moving averages smooth short-term noise to reveal the trend. | 移动平均可平滑短期噪声，显现趋势。
- Exponential smoothing gives more weight to recent observations. | 指数平滑赋予近期观测更大权重。
- ARIMA models combine autoregression with moving averages and differencing. | ARIMA 模型把自回归、移动平均与差分结合起来。
- Seasonal decomposition separates repeating annual or weekly patterns. | 季节分解可分离出重复出现的年度或周度模式。
Forecasts always widen as the horizon lengthens, and a model fitted to a past regime may fail badly after a structural break such as a policy change or a pandemic.
预测区间总是随预测期延长而变宽，而拟合于过去状态的模型，在政策变动或疫情等结构性断点之后可能严重失效。
