用Python写一个爬虫爬取58同城租房信息
时间: 2024-05-18 14:05:57 浏览: 99
python爬虫租房信息在地图上显示的方法
好的,我们可以使用 Python 中的 requests 库和 BeautifulSoup 库来实现爬取 58 同城租房信息的功能。具体步骤如下:
1. 导入所需库:
```python
import requests
from bs4 import BeautifulSoup
```
2. 构造 URL 并发送请求:
```python
url = 'https://hz.58.com/chuzu/pn1/'
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36 Edge/16.16299'
}
response = requests.get(url, headers=headers)
```
3. 解析 HTML 并提取数据:
```python
soup = BeautifulSoup(response.text, 'html.parser')
house_list = soup.select('.list > li')
for house in house_list:
title = house.select('.des > h2 > a')[0].text.strip()
price = house.select('.money > b')[0].text.strip()
house_type = house.select('.room > p')[0].text.strip()
location = house.select('.add > a')[0].text.strip()
print(title, price, house_type, location)
```
完整代码如下:
```python
import requests
from bs4 import BeautifulSoup
url = 'https://hz.58.com/chuzu/pn1/'
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36 Edge/16.16299'
}
response = requests.get(url, headers=headers)
soup = BeautifulSoup(response.text, 'html.parser')
house_list = soup.select('.list > li')
for house in house_list:
title = house.select('.des > h2 > a')[0].text.strip()
price = house.select('.money > b')[0].text.strip()
house_type = house.select('.room > p')[0].text.strip()
location = house.select('.add > a')[0].text.strip()
print(title, price, house_type, location)
```
这样就完成了爬取 58 同城租房信息的功能。需要注意的是,爬虫涉及到的数据抓取和使用需要遵守相关法律法规,如有违反后果自负。
阅读全文