用python写一个百度百科的爬虫代码
时间: 2023-03-13 11:55:01 浏览: 237
基于Python的百度百科爬虫+源代码+文档说明
我可以为你提供一个Python爬取百度百科的代码示例:import requests
from bs4 import BeautifulSoup
url = 'https://baike.baidu.com/item/Python'
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36'
}
response = requests.get(url, headers=headers)
soup = BeautifulSoup(response.text, 'lxml')
content = soup.find('div', class_='lemma-summary') # 获取百科内容
print(content.text.replace('\n', '').strip())
阅读全文