python爬虫爬取网页数据
时间: 2023-07-29 08:10:55 浏览: 54
Python有许多优秀的库可以用来爬取网页数据,比如Requests、BeautifulSoup、Scrapy等。下面是一个使用Requests和BeautifulSoup爬取网页数据的例子:
```python
import requests
from bs4 import BeautifulSoup
url = 'https://www.example.com'
response = requests.get(url)
if response.status_code == 200:
soup = BeautifulSoup(response.content, 'html.parser')
# 获取标题
title = soup.title.string
print(title)
# 获取所有链接
links = soup.find_all('a')
for link in links:
print(link.get('href'))
else:
print('请求失败')
```
这个例子中,我们首先使用Requests库发送了一个GET请求,获取了网页的HTML内容。然后使用BeautifulSoup库解析HTML内容,获取了网页的标题和所有链接。
相关推荐
![pdf](https://img-home.csdnimg.cn/images/20210720083512.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![pdf](https://img-home.csdnimg.cn/images/20210720083512.png)
![pdf](https://img-home.csdnimg.cn/images/20210720083512.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)