python爬虫百度
时间: 2023-12-02 12:03:03 浏览: 78
以下是使用Python爬虫爬取百度搜索结果的示例代码:
```python
import requests
from bs4 import BeautifulSoup
# 搜索关键词
keyword = 'Python'
# 构造请求头
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3'}
# 构造请求URL
url = 'https://www.baidu.com/s?wd=' + keyword
# 发送请求
response = requests.get(url, headers=headers)
# 解析HTML
soup = BeautifulSoup(response.text, 'html.parser')
# 获取搜索结果
results = soup.find_all('div', class_='result')
# 输出搜索结果
for result in results:
title = result.h3.get_text()
link = result.h3.a['href']
abstract = result.find(class_='c-abstract').get_text()
print(title)
print(link)
print(abstract)
print('------------------------')
```
该代码使用requests库发送HTTP请求,使用BeautifulSoup库解析HTML,然后从搜索结果中提取标题、链接和摘要,并输出到控制台。
阅读全文