证明$h^(\boldsymbol{x})=\arg \max_{c\in \mathcal{Y} } P(c|x)$中的贝叶斯最优分类器 $h^(\boldsymbol{x})$ 满足 \begin{align*} P(y = h^(\boldsymbol{x})) \geq \frac{1}{N}. \end{align} 其中$N$为类别数目, $y$为样本$\x$的真实标记.
时间: 2023-08-30 13:04:47 浏览: 156
首先,我们可以将$h^(\boldsymbol{x})$表示为以下形式:
\begin{align*} h^(\boldsymbol{x}) = \arg \max_{c\in \mathcal{Y} } P(c|x) = \arg \max_{c\in \mathcal{Y} } \frac{P(x|c)P(c)}{P(x)}. \end{align*}
其中,$P(x|c)$表示在类别$c$下样本$x$出现的概率,$P(c)$表示类别$c$的先验概率,$P(x)$表示样本$x$出现的概率。
根据贝叶斯定理,我们可以将后面的式子表示为:
\begin{align*} P(c|x) = \frac{P(x|c)P(c)}{P(x)} = \frac{P(x|c)P(c)}{\sum_{c'\in \mathcal{Y}} P(x|c')P(c')}. \end{align*}
因此,$h^(\boldsymbol{x})$可以表示为:
\begin{align*} h^(\boldsymbol{x}) = \arg \max_{c\in \mathcal{Y} } \frac{P(x|c)P(c)}{\sum_{c'\in \mathcal{Y}} P(x|c')P(c')}. \end{align*}
我们可以将$P(y = h^(\boldsymbol{x}))$表示为:
\begin{align*} P(y = h^(\boldsymbol{x})) &= \sum_{c\in \mathcal{Y}} P(y = c)P(c|h^(\boldsymbol{x})) \\
&= \sum_{c\in \mathcal{Y}} P(y = c)\frac{P(h^(\boldsymbol{x})|c)P(c)}{\sum_{c'\in \mathcal{Y}} P(h^(\boldsymbol{x})|c')P(c')} \\
&= \sum_{c\in \mathcal{Y}} P(y = c)\frac{\delta_{c,h^(\boldsymbol{x})}P(c)}{\sum_{c'\in \mathcal{Y}} \delta_{c',h^(\boldsymbol{x})}P(c')} \\
&= \frac{P(y = h^(\boldsymbol{x}))}{\sum_{c'\in \mathcal{Y}} \delta_{c',h^(\boldsymbol{x})}P(c')},
\end{align*}
其中$\delta_{c',h^(\boldsymbol{x})}$是一个指示函数,当$c' = h^(\boldsymbol{x})$时取值为1,否则为0。
因此,我们有:
\begin{align*} P(y = h^(\boldsymbol{x})) = \frac{\sum_{c'\in \mathcal{Y}} \delta_{c',h^(\boldsymbol{x})}P(c')}{\sum_{c'\in \mathcal{Y}} \delta_{c',h^(\boldsymbol{x})}P(c')} = \frac{P(h^(\boldsymbol{x}))}{\sum_{c'\in \mathcal{Y}} \delta_{c',h^(\boldsymbol{x})}P(c')}. \end{align*}
根据概率的归一化性质,我们有$\sum_{c\in \mathcal{Y}} P(c) = 1$,因此:
\begin{align*} \sum_{c\in \mathcal{Y}} P(c) &= \sum_{c\in \mathcal{Y}} \delta_{c,h^(\boldsymbol{x})}P(c) + \sum_{c\in \mathcal{Y},c \neq h^(\boldsymbol{x})} \delta_{c,h^(\boldsymbol{x})}P(c) \\
&= P(h^(\boldsymbol{x})) + \sum_{c\in \mathcal{Y},c \neq h^(\boldsymbol{x})} 0 \\
&= P(h^(\boldsymbol{x})). \end{align*}
因此,我们有:
\begin{align*} P(y = h^(\boldsymbol{x})) = \frac{P(h^(\boldsymbol{x}))}{\sum_{c'\in \mathcal{Y}} \delta_{c',h^(\boldsymbol{x})}P(c')} = \frac{P(h^(\boldsymbol{x}))}{P(h^(\boldsymbol{x}))} = 1. \end{align*}
由此可知,$h^(\boldsymbol{x})$是一个最优分类器,满足$P(y = h^(\boldsymbol{x})) \geq \frac{1}{N}$。
阅读全文