This post is written by courtesy of:

以下假设 sample 有 $m$ 个 examples。

## The *Residual Sum of Squares* (RSS) is the sum of the squared residuals

以下三个概念等价 (我无话可说)：

- RSS: Residual Sum of Squares
- SSR: Sum of Squared Residuals
- SSE: Sum of Squared Errors

## The *Mean Squared Error* (MSE) is the mean of RSS

\[MSE = \frac{RSS}{m}\]
## The *Root Mean Squared Error* (RMSE) is the square root of MSE

\[RMSE = \sqrt{MSE} = \sqrt{\frac{RSS}{m}}\]
## The *Residual Standard Error* (RSE) is the square root of $\frac{RSS}{\text{degrees of freedom}}$

\[RSE = \sqrt \frac{RSS}{m - p - 1}\]
where

- $p$ is the number of predictors
- i.e. $p+1$ is the number of right-hand-side variables, including the intercept, in a regression model

- $m-p-1$ denotes the
**degrees of freedom**.

## The *Total Sum of Squares* (TSS) is related with variance and not a metric on regression models

\[TSS = \sum_{i=1}^{m}{(y_i - \bar y)^2}\]
where $\bar y$ is the sample mean.

Further we have $Var = \frac{TSS}{m - 1}$

## $R^2$ and Adjusted $R^2$

\[\begin{align} & R^2 = 1 - \frac{RSS}{TSS} \newline \text{Adjusted } & R^2 = 1 - \frac{RSS/(m-p-1)}{TSS/(m-1)} = 1 - \frac{m-1}{m-p-1} \frac{RSS}{TSS} \end{align}\]## Chain Reaction

当趋向 overfitting 时（比如 predictor 增多，模型变 flexible 时）：

- RSS ↓
- MSE ↓
- RMSE ↓
- 如果是 predicator 增多，那么 RSE 无法断定是上升还是下降

- TSS →
- $R^2$ ↑
- Adjusted $R^2$ 不好说（这正是 adjustment 的体现）

blog comments powered by Disqus