bbc分类 python代码

以下是使用Python对BBC新闻进行分类的示例代码：

1. 安装必要的库

!pip install pandas nltk sklearn

2. 导入库

import pandas as pd
import nltk
from nltk.corpus import stopwords
from nltk.stem import WordNetLemmatizer
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.model_selection import train_test_split
from sklearn.naive_bayes import MultinomialNB
from sklearn.metrics import accuracy_score

3. 加载数据

data = pd.read_csv('bbc_news.csv')

4. 创建停用词列表

stop_words = stopwords.words('english')

5. 创建词形还原器

lemmatizer = WordNetLemmatizer()

6. 对文本进行预处理

def preprocess_text(text):
    # 将文本转换为小写
    text = text.lower()
    # 去除标点符号
    text = ''.join([char for char in text if char.isalpha() or char == ' '])
    # 分词
    tokens = nltk.word_tokenize(text)
    # 去除停用词
    tokens = [token for token in tokens if token not in stop_words]
    # 词形还原
    tokens = [lemmatizer.lemmatize(token) for token in tokens]
    # 将词列表转换为字符串
    preprocessed_text = ' '.join(tokens)
    return preprocessed_text

data['text'] = data['text'].apply(preprocess_text)

7. 将文本转换为TF-IDF向量

tfidf_vectorizer = TfidfVectorizer()
tfidf = tfidf_vectorizer.fit_transform(data['text'])

8. 将数据集分成训练集和测试集

X_train, X_test, y_train, y_test = train_test_split(tfidf, data['category'], test_size=0.2, random_state=42)

9. 训练朴素贝叶斯分类器

naive_bayes = MultinomialNB()
naive_bayes.fit(X_train, y_train)

10. 预测测试集

y_pred = naive_bayes.predict(X_test)

11. 计算准确率

accuracy = accuracy_score(y_test, y_pred)
print('Accuracy:', accuracy)

以上就是对BBC新闻进行分类的Python代码。