CV_learning_notes(6)

Fitting

Assumption：仅考虑真实值与预测值之间的误差, 计算的是 $\hat{y}$ 与 $y$ 之间的距离
Formula: $E = argmin_{a,b}\sum_{i = 1}^N \|y_i - ax_i -b\|^2 = \|Y - XB \|^2$ $E = a r g m i n_{a, b} \sum_{i = 1}^{N} ∥ y_{i} - a x_{i} - b ∥^{2} = ∥ Y - X B ∥^{2}$
- Solution: $B = (X^TX)^{-1}X^TY$
- 但是如果line是vertical，最小二乘法将会失效

Assumption: 同时考虑因变量和自变量中的误差，使用直线表示法，可处理vertical case，计算的是 $(x_i,y_i)$ 到直线 $ax+by =d$ 的垂直距离
Formula: $E = argmin_{a,b,d}\sum_{i = 1}^N \|ax_i + by_i -d\|^2$ $E = a r g m i n_{a, b, d} \sum_{i = 1}^{N} ∥ a x_{i} + b y_{i} - d ∥^{2}$
- Solution:
  - 由 $\frac{\partial E}{\partial d} = 0$ 可得 $d = a\overline{x} + b\overline{y}$
  - 将 $d$ 代入可得化简形式
  $\begin{aligned} E &= \sum_{i = 1}^N (a(x_i - \overline{x}) + b(y_i - \overline{y}))^2 \\ &= \left\| \begin{bmatrix} x_1 - \overline{x} & y_1 - \overline{y} \\ \vdots & \vdots \\ x_n - \overline{x} & y_n - \overline{y} \end{bmatrix} \begin{bmatrix} a \\ b \end{bmatrix} \right\|^2 \\ &= (UN)^T(UN) \end{aligned}$
  - 最终对 $\frac{\partial E}{\partial N} = 0 = (U^TU)N$ 求解可最终解得 $N$

-> 为了减少离散点/outliers对最终拟合效果的影响，因此引入了以下两种方法

robust

Overview：
- Choose a small subset of points uniformly at random
- Fit a model to that subset
- Find all remaining points that are “close” to the model and reject the rest as outliers
- Do this many times and choose the best model
需要对参数进行选择才能达到最优的效果:
- 初始的采样点个数
- 距离阈值
- 迭代的次数