Python计算信息熵实例计算信息熵实例
计算信息熵的公式:n是类别数,p(xi)是第i类的概率
假设数据集有m行,即m个样本,每一行最后一列为该样本的标签,计算数据集信息熵的代码如下:
from math import log
def calcShannonEnt(dataSet):
numEntries = len(dataSet) # 样本数
labelCounts = {} # 该数据集每个类别的频数
for featVec in dataSet: # 对每一行样本
currentLabel = featVec[-1] # 该样本的标签
if currentLabel not in labelCounts.keys(): labelCounts[currentLabel] = 0
labelCounts[currentLabel] += 1
shannonEnt = 0.0
for key in labelCounts:
prob = float(labelCounts[key])/numEntries # 计算p(xi)
shannonEnt -= prob * log(prob, 2) # log base 2
return shannonEnt
补充知识:补充知识:python 实现信息熵、条件熵、信息增益、基尼系数实现信息熵、条件熵、信息增益、基尼系数
我就废话不多说了,大家还是直接看代码吧~
import pandas as pd
import numpy as np
import math
## 计算信息熵
def getEntropy(s):
# 找到各个不同取值出现的次数
if not isinstance(s, pd.core.series.Series):
s = pd.Series(s)
prt_ary = pd.groupby(s , by = s).count().values / float(len(s))
return -(np.log2(prt_ary) * prt_ary).sum()
## 计算条件熵: 条件s1下s2的条件熵
def getCondEntropy(s1 , s2):
d = dict()
for i in list(range(len(s1))):
d[s1[i]] = d.get(s1[i] , []) + [s2[i]] return sum([getEntropy(d[k]) * len(d[k]) / float(len(s1)) for k in d])
## 计算信息增益
def getEntropyGain(s1, s2):
return getEntropy(s2) - getCondEntropy(s1, s2)
## 计算增益率
def getEntropyGainRadio(s1, s2):
return getEntropyGain(s1, s2) / getEntropy(s2)
## 衡量离散值的相关性
import math
def getDiscreteCorr(s1, s2):
return getEntropyGain(s1,s2) / math.sqrt(getEntropy(s1) * getEntropy(s2))
# ######## 计算概率平方和
def getProbSS(s):
if not isinstance(s, pd.core.series.Series):
s = pd.Series(s)
prt_ary = pd.groupby(s, by = s).count().values / float(len(s))
return sum(prt_ary ** 2)
######## 计算基尼系数
def getGini(s1, s2):
d = dict()
for i in list(range(len(s1))):
d[s1[i]] = d.get(s1[i] , []) + [s2[i]] return 1-sum([getProbSS(d[k]) * len(d[k]) / float(len(s1)) for k in d])
评论0