qunar_freetrip.csv下载
时间: 2023-06-06 13:02:55 浏览: 44
qunar_freetrip.csv是去哪儿网站上发布的免费旅游套餐数据,可以通过下载该文件来获取相关信息。该文件包含了各个国内旅游景点的旅游套餐,包括旅游线路、酒店住宿、门票价格等详细信息。用户可以直接通过该文件查看并选择想要的旅游套餐,以便更快速地规划自己的旅行计划。
下载qunar_freetrip.csv文件可以使用离线导入功能进行操作。用户需要先将文件下载到本地空间,然后在去哪儿网站登录账号后,选择对应的导入功能,将文件上传到系统中即可。除此之外,用户还可以直接将文件的数据复制粘贴到网站上的相应界面中进行导入。
总之,qunar_freetrip.csv文件是一个非常有用的旅行助手,下载后可以更加方便地查询和选择适合自己的旅游套餐。用户只需要根据自己的需求进行选择,然后便可享受到畅游各地的愉悦体验。
相关问题
import requests import random import parsel visited_urls = set() # 保存已经访问过的链接 page = 1 while True: # 循环爬取 url = f'https://travel.qunar.com/travelbook/list.htm?page={page}&order=hot_heat' html_data = requests.get(url).text selector = parsel.Selector(html_data) lis = selector.css('.list_item h2 a::attr(href)').getall() # 遍历当前页面中的所有链接 for li in lis: detail_url = 'https://travel.qunar.com' + li if detail_url in visited_urls: # 如果链接已经访问过,则跳过 continue visited_urls.add(detail_url) # 将链接加入集合中 detail_html = requests.get(detail_url).text time.sleep(random.randint(3, 5)) detail_selector = parsel.Selector(detail_html) title = detail_selector.css('.b_crumb_cont *:nth-child(3)::text').get() comment = detail_selector.css('.title.white::text').get() view_count = detail_selector.css('.view_count::text').get() date = detail_selector.css('#js_mainleft > div.b_foreword > ul > li.f_item.when > p > span.data::text').get() days = detail_selector.css('#js_mainleft > div.b_foreword > ul > li.f_item.howlong > p > span.data::text').get() money = detail_selector.css('#js_mainleft > div.b_foreword > ul > li.f_item.howmuch > p > span.data::text').get() character = detail_selector.css('#js_mainleft > div.b_foreword > ul > li.f_item.who > p > span.data::text').get() play_list = detail_selector.css('#js_mainleft > div.b_foreword > ul > li.f_item.how > p > span.data > span::text').getall() for play in play_list: print(play) print(title, comment, view_count, date, days, money, character, play_list, detail_url) time.sleep(random.randint(3, 5)) # 判断是否需要继续爬取 if len(lis) == 0 or page >= 20: break page += 1这段代码怎么将爬取出来的所有数据保存到csv文件
你可以使用 Python 自带的 csv 模块将数据保存到 CSV 文件中。在你的代码中,你可以在循环中将每个页面的数据添加到一个列表中,然后在退出循环后将整个列表保存到 CSV 文件中。以下是修改后的代码:
```
import requests
import random
import parsel
import csv
visited_urls = set() # 保存已经访问过的链接
page = 1
data_list = [] # 保存所有数据
while True: # 循环爬取
url = f'https://travel.qunar.com/travelbook/list.htm?page={page}&order=hot_heat'
html_data = requests.get(url).text
selector = parsel.Selector(html_data)
lis = selector.css('.list_item h2 a::attr(href)').getall()
# 遍历当前页面中的所有链接
for li in lis:
detail_url = 'https://travel.qunar.com' + li
if detail_url in visited_urls: # 如果链接已经访问过,则跳过
continue
visited_urls.add(detail_url) # 将链接加入集合中
detail_html = requests.get(detail_url).text
time.sleep(random.randint(3, 5))
detail_selector = parsel.Selector(detail_html)
title = detail_selector.css('.b_crumb_cont *:nth-child(3)::text').get()
comment = detail_selector.css('.title.white::text').get()
view_count = detail_selector.css('.view_count::text').get()
date = detail_selector.css('#js_mainleft > div.b_foreword > ul > li.f_item.when > p > span.data::text').get()
days = detail_selector.css('#js_mainleft > div.b_foreword > ul > li.f_item.howlong > p > span.data::text').get()
money = detail_selector.css('#js_mainleft > div.b_foreword > ul > li.f_item.howmuch > p > span.data::text').get()
character = detail_selector.css('#js_mainleft > div.b_foreword > ul > li.f_item.who > p > span.data::text').get()
play_list = detail_selector.css('#js_mainleft > div.b_foreword > ul > li.f_item.how > p > span.data > span::text').getall()
# 将当前页面的数据添加到列表中
data_list.append([title, comment, view_count, date, days, money, character, play_list, detail_url])
for play in play_list:
print(play)
print(title, comment, view_count, date, days, money, character, play_list, detail_url)
time.sleep(random.randint(3, 5))
# 判断是否需要继续爬取
if len(lis) == 0 or page >= 20:
break
page += 1
# 将所有数据保存到 CSV 文件中
with open('data.csv', 'w', newline='', encoding='utf-8-sig') as f:
writer = csv.writer(f)
writer.writerow(['标题', '评论', '浏览数', '日期', '天数', '花费', '人物', '玩法', '链接'])
for data in data_list:
writer.writerow(data)
```
该代码将数据保存到名为 `data.csv` 的文件中,并在文件中包括标题、评论、浏览数、日期、天数、花费、人物、玩法和链接等信息。
selector = parsel.Selector(html_data) lis = selector.css('.list_item h2 a::attr(href)').getall() details = [] for li in lis: detail_url = 'https://travel.qunar.com' + li detail_html = requests.get(detail_url).text details.append(detail_html) # 输出每个网站的 html 数据 for detail_html in details: print(detail_html) detail_html = requests.get(detail_url).text #解析数据 detail_selector = parsel.Selector(detail_html) title = detail_selector.css('.b_crumb_cont *:nth-child(3)::text').get() comment = detail_selector.css('.title white::text').get() view_count = detail_selector.css('.view_count::text').get() date = detail_selector.css('#js_mainleft > div.b_foreword > ul > li.f_item.when > p > span.data::text').get() days = detail_selector.css('#js_mainleft > div.b_foreword > ul > li.f_item.howlong > p > span.data::text').get() money = detail_selector.css('#js_mainleft > div.b_foreword > ul > li.f_item.howmuch > p > span.data::text').get() character = detail_selector.css('#js_mainleft > div.b_foreword > ul > li.f_item.who > p > span.data::text').get() play_list = detail_selector.css('#js_mainleft > div.b_foreword > ul > li.f_item.how > p > span.data > span::text').getall() for play in play_list: print(play) print(title, comment, view_count, date, days, money, character, play_list, detail_url) time.sleep(random.randint(3,5))这段代码只能输出最后一个网站的信息 该怎么改
这段代码只能输出最后一个网站的信息是因为在循环中每次请求新的网站信息时都会覆盖前面的信息,需要将每次请求的信息存储到一个列表中,最后再一起处理。
可以尝试如下修改:
```python
selector = parsel.Selector(html_data)
lis = selector.css('.list_item h2 a::attr(href)').getall()
details = []
for li in lis:
detail_url = 'https://travel.qunar.com' + li
detail_html = requests.get(detail_url).text
details.append(detail_html)
time.sleep(random.randint(3, 5)) # 每次请求后等待一段时间,避免请求频繁被封
for detail_html in details:
# 解析数据
detail_selector = parsel.Selector(detail_html)
title = detail_selector.css('.b_crumb_cont *:nth-child(3)::text').get()
comment = detail_selector.css('.title.white::text').get()
view_count = detail_selector.css('.view_count::text').get()
date = detail_selector.css('#js_mainleft > div.b_foreword > ul > li.f_item.when > p > span.data::text').get()
days = detail_selector.css('#js_mainleft > div.b_foreword > ul > li.f_item.howlong > p > span.data::text').get()
money = detail_selector.css('#js_mainleft > div.b_foreword > ul > li.f_item.howmuch > p > span.data::text').get()
character = detail_selector.css('#js_mainleft > div.b_foreword > ul > li.f_item.who > p > span.data::text').get()
play_list = detail_selector.css('#js_mainleft > div.b_foreword > ul > li.f_item.how > p > span.data > span::text').getall()
for play in play_list:
print(play)
print(title, comment, view_count, date, days, money, character, play_list, detail_url)
```
这样每次请求得到的网站信息都会被存储到 `details` 列表中,最后再遍历处理每个网站的信息。同时,为了避免请求频繁被封,可以在每次请求后等待一段时间。