Github SSH Key避免Hexo部署输入密码

高斯分布表达式

一元高斯分布

$$\mathcal{N}\left(x | \mu, \sigma^{2}\right)=\frac{1}{\left(2 \pi \sigma^{2}\right)^{\frac{1}{2}}} \exp \left\{-\frac{1}{2 \sigma^{2}}(x-\mu)^{2}\right\}$$

多元高斯分布

$$\mathcal{N}(\boldsymbol{x} | \boldsymbol{\mu}, \mathbf{\Sigma})=\frac{1}{(2 \pi)^{\frac{D}{2}}} \frac{1}{|\boldsymbol{\Sigma}|^{\frac{1}{2}}} \exp \left\{-\frac{1}{2}(\boldsymbol{x}-\boldsymbol{\mu})^{T} \boldsymbol{\Sigma}^{-1}(\boldsymbol{x}-\boldsymbol{\mu})\right\}$$

性质

高斯分布依赖于下面二次型，即马氏距离

$$\Delta^{2}=(\boldsymbol{x}-\boldsymbol{\mu})^{T} \boldsymbol{\Sigma}^{-1}(\boldsymbol{x}-\boldsymbol{\mu})$$

协方差矩阵$\sum $可以取对称矩阵，任何非对称项都会从指数中消失（也可以从协方差矩阵定义看）可分解成对角矩阵

\begin{array}{l}{\Delta^{2}=[(\vec{X}-\vec{\mu})]^{\top}\left(Q \Lambda Q^{\top}\right)^{-1}[(\vec{X}-\vec{\mu})]} \\ {=\left[Q^{\top}(\vec{X}-\vec{\mu})\right]^{\top} \Lambda^{-1}\left[Q^{\top}(\vec{X}-\vec{\mu})\right]} \\ {=\left[Q^{\top}(\vec{X}-\vec{\mu})\right]^{\top}\left(\Lambda^{-\frac{1}{2}}\right)^{\top} \Lambda^{-\frac{1}{2}}\left[Q^{\top}(\vec{X}-\vec{\mu})\right]} \\ {=\left[\Lambda^{-\frac{1}{2}} Q^{\top}(\vec{X}-\vec{\mu})\right]^{\top}\left[\begin{array}{c}{\Lambda^{-\frac{1}{2}} Q^{\top}(\vec{X}-\vec{\mu})}\end{array}\right]} \\ {=\left[\left(Q \Lambda^{-\frac{1}{2}}\right)^{\top}(\vec{X}-\vec{\mu})\right]^{\top}\left[\left(Q \Lambda^{-\frac{1}{2}}\right)^{\top}(\vec{X}-\vec{\mu})\right]}\end{array}

含义：高斯分布中心在u,平移u之后高斯分布中心到原点，高斯分布椭圆轴拉伸变换成正圆，称一个旋转矩阵，将之前椭圆轴方向旋转到全局坐标系(Oxyz)，特别注意：变换矩阵Q

椭圆中心u,轴方向：变换矩阵Q的列向量

条件高斯分布

多元⾼斯分布的⼀个重要性质是，如果两组变量是联合⾼斯分布，那么以⼀组变量为条件，
另⼀组变量同样是⾼斯分布。类似地，任何⼀个变量的边缘分布也是⾼斯分布

协方差矩阵&精度矩阵&分块矩阵求逆

$$\Sigma=\left(\begin{array}{cc}{\Sigma_{a a}} & {\Sigma_{a b}} \\ {\Sigma_{b a}} & {\Sigma_{b b}}\end{array}\right), \quad A=\left(\begin{array}{cc}{A_{a}} & {A_{b}} \\ {A_{b a}} & {A_{b b}}\end{array}\right), A=\Sigma^{-}$$

分块矩阵求逆

$$\left(\begin{array}{cc}{\boldsymbol{A}} & {\boldsymbol{B}} \\ {\boldsymbol{C}} & {\boldsymbol{D}}\end{array}\right)^{-1}=\left(\begin{array}{cc}{\boldsymbol{M}} & {-\boldsymbol{M B D}^{-1}} \\ {-\boldsymbol{D}^{-1} \boldsymbol{C M}} & {\boldsymbol{D}^{-1}+\boldsymbol{D}^{-1} \boldsymbol{C M B D}^{-1}}\end{array}\right)$$

其中 $M=\left(A-B D^{-1} C\right)^{-}$

完全平方项

通过对比二次型找到均值何方差的表达式

分解式如下

$$\begin{aligned}-\frac{1}{2}(& \boldsymbol{x}-\boldsymbol{\mu}^{T} \boldsymbol{\Sigma}^{-1}(\boldsymbol{x}-\boldsymbol{\mu})=\\ &-\frac{1}{2}\left(\boldsymbol{x}_{a}-\boldsymbol{\mu}_{a}\right)^{T} \boldsymbol{\Lambda}_{a a}\left(\boldsymbol{x}_{a}-\boldsymbol{\mu}_{a}\right)-\frac{1}{2}\left(\boldsymbol{x}_{a}-\boldsymbol{\mu}_{a}\right)^{T} \boldsymbol{\Lambda}_{a b}\left(\boldsymbol{x}_{b}-\boldsymbol{\mu}_{b}\right) \\ &-\frac{1}{2}\left(\boldsymbol{x}_{b}-\boldsymbol{\mu}_{b}\right)^{T} \boldsymbol{\Lambda}_{b a}\left(\boldsymbol{x}_{a}-\boldsymbol{\mu}_{a}\right)-\frac{1}{2}\left(\boldsymbol{x}_{b}-\boldsymbol{\mu}_{b}\right)^{T} \boldsymbol{\Lambda}_{b b}\left(\boldsymbol{x}_{b}-\boldsymbol{\mu}_{b}\right) \end{aligned}$$

二次型

$$-\frac{1}{2}(\boldsymbol{x}-\boldsymbol{\mu})^{T} \boldsymbol{\Sigma}^{-1}(\boldsymbol{x}-\boldsymbol{\mu})=-\frac{1}{2} \boldsymbol{x}^{T} \boldsymbol{\Sigma}^{-1} \boldsymbol{x}+\boldsymbol{x}^{T} \boldsymbol{\Sigma}^{-1} \boldsymbol{\mu}+\text {const}_{-} \text {value}$$

条件概率均值和协方差

$$\begin{array}{l}{\mu_{a b}=\mu_{a}+\Sigma_{a b} \Sigma_{b b}^{-1}\left(x_{b}-\mu_{b}\right)=\mu_{a}-H_{a b}^{-1} A_{b b}\left(x_{b}-\mu_{b}\right)} \\ {\Sigma_{a b}=\Sigma_{a a}-\Sigma_{a b} \Sigma_{b b}^{-1} \Sigma_{b a}=\Lambda_{b b}^{-1}}\end{array}$$

边缘高斯分布

$$p\left(\boldsymbol{x}_{a}\right)=\int p\left(\boldsymbol{x}_{a}, \boldsymbol{x}_{b}\right) \mathrm{d} \boldsymbol{x}_{b}$$

分解式对变量Xb积分

关于Xb的项

$$-\frac{1}{2} \boldsymbol{x}_{b}^{T} \boldsymbol{\Lambda}_{b b} \boldsymbol{x}_{b}+\boldsymbol{x}_{b}^{T} \boldsymbol{m}=-\frac{1}{2}\left(\boldsymbol{x}_{b}-\boldsymbol{\Lambda}_{b b}^{-1} \boldsymbol{m}\right)^{T} \boldsymbol{\Lambda}_{b b}\left(\boldsymbol{x}_{b}-\boldsymbol{\Lambda}_{b b}^{-1} \boldsymbol{m}\right)+\frac{1}{2} \boldsymbol{m}^{T} \boldsymbol{\Lambda}_{b b}^{-1} \boldsymbol{m}$$

其中 $\boldsymbol{m}=\boldsymbol{\Lambda}_{b b} \boldsymbol{\mu}_{b}-\boldsymbol{\Lambda}_{b a}\left(\boldsymbol{x}_{a}-\boldsymbol{\mu}_{a}\right)$
容易求得变量的积分 现在看剩余部分变量Xa和常量

$$\begin{array}{l}{\frac{1}{2}\left[\boldsymbol{A}_{b b} \boldsymbol{\mu}_{b}-\boldsymbol{A}_{b a}\left(\boldsymbol{x}_{a}-\boldsymbol{\mu}_{a}\right)\right]^{T} \boldsymbol{A}_{b b}^{-1}\left[\boldsymbol{A}_{b b} \boldsymbol{\mu}_{b}-\boldsymbol{A}_{b a}\left(\boldsymbol{x}_{a}-\boldsymbol{\mu}_{a}\right)\right]-\frac{1}{2} \boldsymbol{x}_{a}^{T} \boldsymbol{\Lambda}_{a a} \boldsymbol{x}_{a}+\boldsymbol{x}_{a}^{T}\left(\boldsymbol{A}_{a a} \boldsymbol{\mu}_{a}+\boldsymbol{A}_{b b} \boldsymbol{\mu}_{b}\right)+\text {const}_{-} \text {value}} \\ {=-\frac{1}{2} \boldsymbol{x}_{a}^{T}\left(\boldsymbol{A}_{a a}-\boldsymbol{A}_{a b} \boldsymbol{A}_{b b}^{-1} \boldsymbol{h}_{b a}\right) \boldsymbol{x}_{a}+\boldsymbol{x}_{a}^{T}\left(\boldsymbol{A}_{a b} \boldsymbol{A}_{b b}^{-1} \boldsymbol{A}_{b a}\right) \boldsymbol{\mu}_{a}+\text {const}_{-} \text {value }}\end{array}$$

通过对比得

$$\begin{aligned} \mathbb{E}\left[\boldsymbol{x}_{a}\right] &=\boldsymbol{\mu}_{a} \\ \operatorname{cov}\left[\boldsymbol{x}_{a}\right] &=\boldsymbol{\Sigma}_{a a} \end{aligned}$$

总结

对联合高斯分布

$$\begin{array}{l}{\mathcal{N}(\boldsymbol{x} | \boldsymbol{\mu}, \Sigma)} \\ {x=\left(\begin{array}{l}{x_{a}} \\ {x_{b}}\end{array}\right), u=\left(\begin{array}{l}{\boldsymbol{\mu}_{a}} \\ {\boldsymbol{\mu}_{b}}\end{array}\right)} \\ {\boldsymbol{\Sigma}=\left(\begin{array}{cc}{\boldsymbol{\Sigma}_{a a}} & {\boldsymbol{\Sigma}_{a b}} \\ {\boldsymbol{\Sigma}_{b a}} & {\boldsymbol{\Sigma}_{b b}}\end{array}\right), \quad \boldsymbol{A}=\left(\begin{array}{cc}{\boldsymbol{A}_{a a}} & {\boldsymbol{A}_{b b}} \\ {\boldsymbol{A}_{b a}} & {\boldsymbol{A}_{b b}}\end{array}\right), \boldsymbol{A}=\boldsymbol{\Sigma}^{-}}\end{array}$$

条件概率

$$\begin{aligned} p\left(\boldsymbol{x}_{a} | \boldsymbol{x}_{b}\right) &=\mathcal{N}\left(\boldsymbol{x}_{a} | \boldsymbol{\mu}_{a | b}, \mathbf{\Lambda}_{a a}^{-1}\right) \\ \boldsymbol{\mu}_{a | b} &=\boldsymbol{\mu}_{a}-\boldsymbol{\Lambda}_{a a}^{-1} \mathbf{\Lambda}_{a b}\left(\boldsymbol{x}_{b}-\boldsymbol{\mu}_{b}\right) \end{aligned}$$

边缘概率

$$p\left(\boldsymbol{x}_{a}\right)=\mathcal{N}\left(\boldsymbol{x}_{a} | \boldsymbol{\mu}_{a}, \boldsymbol{\Sigma}_{a a}\right)$$

引用

1. Pattern Recognition and Machine Learning
2. 多元高斯分布完全解析

---

转载请注明来源，欢迎对文章中的引用来源进行考证，欢迎指出任何有错误或不够清晰的表达。可以邮件至 yangbenbo@whu.edu.cn