登录千里马招标网站并爬取信息的代码
时间: 2024-06-09 15:07:56 浏览: 192
以下是使用Python爬取千里马招标网站信息的示例代码:
```python
import requests
from bs4 import BeautifulSoup
url = "http://www.qianlima.com/zb/search/index.html"
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Firefox/91.0",
"Referer": "http://www.qianlima.com/zb/search/index.html",
"Content-Type": "application/x-www-form-urlencoded; charset=UTF-8",
"X-Requested-With": "XMLHttpRequest"
}
# 构造请求参数
data = {
"pg": 1,
"pSize": 50,
"orderType": 0,
"zbType": "",
"keywords": "数据分析",
"isShowAll": 0,
"sdate": "",
"edate": "",
"sdate1": "",
"edate1": "",
"sdate2": "",
"edate2": "",
"sdate3": "",
"edate3": "",
"sdate4": "",
"edate4": "",
"sdate5": "",
"edate5": "",
"sdate6": "",
"edate6": "",
"sdate7": "",
"edate7": "",
"sdate8": "",
"edate8": "",
"sdate9": "",
"edate9": "",
"sdate10": "",
"edate10": "",
"sdate11": "",
"edate11": "",
"sdate12": "",
"edate12": "",
"sdate13": "",
"edate13": "",
"sdate14": "",
"edate14": "",
"sdate15": "",
"edate15": "",
"sdate16": "",
"edate16": "",
"sdate17": "",
"edate17": "",
"sdate18": "",
"edate18": "",
"sdate19": "",
"edate19": "",
"sdate20": "",
"edate20": "",
"sdate21": "",
"edate21": "",
"sdate22": "",
"edate22": "",
"sdate23": "",
"edate23": "",
"sdate24": "",
"edate24": "",
"sdate25": "",
"edate25": "",
"sdate26": "",
"edate26": "",
"sdate27": "",
"edate27": "",
"sdate28": "",
"edate28": "",
"sdate29": "",
"edate29": "",
"sdate30": "",
"edate30": "",
"sdate31": "",
"edate31": ""
}
# 发送post请求获取数据
response = requests.post(url, headers=headers, data=data)
soup = BeautifulSoup(response.text, 'html.parser')
items = soup.select(".zb-search-list .zb-search-item")
# 解析数据
for item in items:
title = item.select_one(".zb-title a").get_text().strip()
link = item.select_one(".zb-title a")['href']
time = item.select_one(".zb-time").get_text().strip()
print(title, link, time)
```
代码中,我们首先使用requests库构造POST请求,然后使用BeautifulSoup库解析响应数据,并提取出需要的信息进行展示。需要注意的是,本示例中的请求参数是根据网站的POST请求进行构造的,如果网站有更新,需要根据实际情况适当调整。
阅读全文