爬虫python百度代码
时间: 2023-12-19 14:32:29 浏览: 131
爬虫python代码
以下是爬取百度搜索内容的Python代码示例:
```python
import requests
from bs4 import BeautifulSoup
def baidu_search(keyword):
url = 'https://www.baidu.com/s'
params = {'wd': keyword}
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3'}
response = requests.get(url, params=params, headers=headers)
soup = BeautifulSoup(response.text, 'html.parser')
search_results = soup.find_all('h3', {'class': 't'})
for result in search_results:
print(result.get_text())
baidu_search('Python爬虫')
```
这段代码实现了使用Python爬虫爬取百度搜索内容的功能。首先,通过requests库发送HTTP请求,然后使用BeautifulSoup库解析网页内容,最后找到搜索结果并打印出来。
阅读全文