softmax的交叉熵导数推导.docx

Derive the gradient with regard to the inputs of a softmax function when cross entropy

loss is used for evalution, i.e. find the gradient with respect to the softmax input

vector , when the prediction is made by . Remember the cross

entropy function is

Where is the one-hot label vector. And is the predicted probability vector for all

classes. (Hint you might want to consider the fact many elements of are zeros, and

assume that only the k-th dimension of is one . )

即