# SVM 支持向量机

### 代价函数

$$\cos t({h_\theta }(x),y) = \left{ {\begin{array}{c} { - \log ({h_\theta }(x))} \ { - \log (1 - {h_\theta }(x))} \end{array} \begin{array}{c} {y = 1} \ {y = 0} \end{array} } \right.$$ ，

y=0 时同样，用 $$\cos {t_0}(z)$$ 代替

$$J(\theta ) = C\sum\limits_{i = 1}^m {[{y^{(i)}}\cos {t_1}({\theta ^T}{x^{(i)}}) + (1 - {y^{(i)}})\cos {t_0}({\theta ^T}{x^{(i)}})} ] + \frac{1}{2}\sum\limits_{j = 1}^{\text{n}} {\theta _j^2}$$

$$J(\theta ) = - \frac{1}{m}\sum\limits_{i = 1}^m {[{y^{(i)}}\log ({h_\theta }({x^{(i)}}) + (1 - } {y^{(i)}})\log (1 - {h_\theta }({x^{(i)}})] + \frac{\lambda }{{2m}}\sum\limits_{j = 1}^n {\theta _j^2}$$，可以认为这里的 $$C = \frac{m}{\lambda }$$ ，只是表达形式问题，这里C的值越大，SVM的决策边界的margin也越大，下面会说明

### 2、Large Margin

$$u = \left[ {\begin{array}{c} {{u_1}} \ {{u_2}} \end{array} } \right]$$ ， $$v = \left[ {\begin{array}{c} {{v_1}} \ {{v_2}} \end{array} } \right]$$

$$||u||$$ 表示u欧几里得范数（欧式范数）， $$||u||{\text{ = }}\sqrt {{\text{u}}_1^2 + u_2^2}$$

向量V向量u上的投影的长度记为p，则：向量内积：
$${{\text{u}}^T}v = p||u|| = {u_1}{v_1} + {u_2}{v_2}$$

$$J(\theta ) = C0 + \frac{1}{2}\sum\limits_{j = 1}^{\text{n}} {\theta j^2} = \frac{1}{2}\sum\limits{j = 1}^{\text{n}} {\theta _j^2} = \frac{1}{2}(\theta _1^2 + \theta _2^2) = \frac{1}{2}{\sqrt {\theta _1^2 + \theta _2^2} ^2}$$ ，我们最后的目的就是求使代价最小的θ

### 3、SVM Kernel（核函数）

$${f_1} = similarity(x,{l^{(1)}}) = {e^{ - \frac{{||x - {l^{(1)}}|{|^2}}}{{2{\sigma ^2}}}}}$$
$${f_2} = similarity(x,{l^{(2)}}) = {e^{ - \frac{{||x - {l^{(2)}}|{|^2}}}{{2{\sigma ^2}}}}}$$ . . .

$$J(\theta ) = C\sum\limits_{i = 1}^m {[{y^{(i)}}\cos {t_1}({\theta ^T}{f^{(i)}}) + (1 - {y^{(i)}})\cos {t_0}({\theta ^T}{f^{(i)}})} ] + \frac{1}{2}\sum\limits_{j = 1}^{\text{n}} {\theta _j^2}$$

### 4、使用 scikit-learn 中的 SVM 模型代码

  '''data1——线性分类'''
X = data1['X']
y = data1['y']
y = np.ravel(y)
plot_data(X,y)

model = svm.SVC(C=1.0,kernel='linear').fit(X,y) # 指定核函数为线性核函数


  '''data2——非线性分类'''
X = data2['X']
y = data2['y']
y = np.ravel(y)
plt = plot_data(X,y)
plt.show()

model = svm.SVC(gamma=100).fit(X,y)   # gamma为核函数的系数，值越大拟合的越好