https://www.mafengwo.cn/poi/16790.html爬取这个网站的点评信息

时间: 2023-08-07 09:11:00 浏览: 63
好的,您可以使用以下代码来爬取指定网站的点评信息: ```python import time import openpyxl from selenium import webdriver from bs4 import BeautifulSoup url = "https://www.mafengwo.cn/poi/16790.html" edge = webdriver.Edge() edge.get(url) comments_data = [] # 等待页面加载完成 time.sleep(5) # 点击展开更多评论按钮 more_btn = edge.find_element_by_css_selector(".rev-total a") edge.execute_script("arguments[0].click();", more_btn) time.sleep(2) # 循环获取点评信息 while True: # 获取页面源码 html = edge.page_source soup = BeautifulSoup(html, "html.parser") # 获取每条点评的信息 reviews = soup.select(".rev-item") for review in reviews: # 获取点评内容 content = review.select_one(".rev-txt").get_text().strip() print("点评内容:", content) # 获取点评时间 date = review.select_one(".time").get_text().strip() print("点评时间:", date) # 获取点评用户 user = review.select_one(".name").get_text().strip() print("点评用户:", user) # 获取点评评分 score = review.select_one(".score span").get_text().strip() print("点评评分:", score) print("-----------------------------") comments_data.append([content, date, user, score]) # 翻页操作 next_btn = edge.find_element_by_css_selector(".pg-next") if next_btn.get_attribute("class") == "pg-next": edge.execute_script("arguments[0].click();", next_btn) time.sleep(2) else: break # 将爬取的点评信息存入comments.xlsx文件中 wb = openpyxl.Workbook() ws = wb.active ws.append(["点评内容", "点评时间", "点评用户", "点评评分"]) for data in comments_data: ws.append(data) wb.save("comments.xlsx") wb.close() # 关闭浏览器 edge.quit() ``` 执行代码后,将在同一目录下生成一个名为comments.xlsx的Excel文件,其中包含爬取到的点评信息。 请注意,您需要先安装selenium、beautifulsoup和openpyxl库,并且根据您的浏览器类型下载对应的WebDriver并配置到环境变量中。 希望对您有所帮助!如果您有任何其他问题,请随时提问。

相关推荐

Traceback (most recent call last): File "C:\Users\niuxi\pythonProject\main.py", line 15, in <module> more_btn = edge.find_element("css selector", ".rev-total a") File "D:\元气壁纸缓存\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 740, in find_element return self.execute(Command.FIND_ELEMENT, {"using": by, "value": value})["value"] File "D:\元气壁纸缓存\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 346, in execute self.error_handler.check_response(response) File "D:\元气壁纸缓存\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 245, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":".rev-total a"} (Session info: MicrosoftEdge=114.0.1823.79); For documentation on this error, please visit: https://www.selenium.dev/documentation/webdriver/troubleshooting/errors#no-such-element-exception Stacktrace: Backtrace: GetHandleVerifier [0x00007FF6E502AEC2+64226] Microsoft::Applications::Events::EventProperty::~EventProperty [0x00007FF6E4FBC082+765362] (No symbol) [0x00007FF6E4D7CC4C] (No symbol) [0x00007FF6E4DC0BDF] (No symbol) [0x00007FF6E4DC0D6A] (No symbol) [0x00007FF6E4DFAC17] (No symbol) [0x00007FF6E4DDF03F] (No symbol) [0x00007FF6E4DB4BB1] (No symbol) [0x00007FF6E4DF7FC1] (No symbol) [0x00007FF6E4DDEDD3] (No symbol) [0x00007FF6E4DB3BEC] (No symbol) [0x00007FF6E4DB2DD6] (No symbol) [0x00007FF6E4DB4364] Microsoft::Applications::Events::ILogManager::DispatchEventBroadcast [0x00007FF6E52087C9+1319033] (No symbol) [0x00007FF6E4E2D2A8] Microsoft::Applications::Events::EventProperty::~EventProperty [0x00007FF6E4F095E1+33553] Microsoft::Applications::Events::EventProperty::~EventProperty [0x00007FF6E4F01A1F+1871] Microsoft::Applications::Events::ILogManager::DispatchEventBroadcast [0x00007FF6E52073E3+1313939] Microsoft::Applications::Events::ILogConfiguration::operator* [0x00007FF6E4FC46B8+20232] Microsoft::Applications::Events::ILogConfiguration::operator* [0x00007FF6E4FC0CD4+5412] Microsoft::Applications::Events::ILogConfiguration::operator* [0x00007FF6E4FC0DCC+5660] Microsoft::Applications::Events::EventProperty::~EventProperty [0x00007FF6E4FB4A81+735153] BaseThreadInitThunk [0x00007FFEC68526AD+29] RtlUserThreadStart [0x00007FFEC802AA68+40]

Traceback (most recent call last): File "C:\Users\niuxi\pythonProject\main.py", line 18, in <module> more_btn = WebDriverWait(edge, 10).until(EC.presence_of_element_located((By.CSS_SELECTOR, ".rev-more a"))) File "D:\元气壁纸缓存\lib\site-packages\selenium\webdriver\support\wait.py", line 95, in until raise TimeoutException(message, screen, stacktrace) selenium.common.exceptions.TimeoutException: Message: Stacktrace: Backtrace: GetHandleVerifier [0x00007FF6E502AEC2+64226] Microsoft::Applications::Events::EventProperty::~EventProperty [0x00007FF6E4FBC082+765362] (No symbol) [0x00007FF6E4D7CC4C] (No symbol) [0x00007FF6E4DC0BDF] (No symbol) [0x00007FF6E4DC0D6A] (No symbol) [0x00007FF6E4DFAC17] (No symbol) [0x00007FF6E4DDF03F] (No symbol) [0x00007FF6E4DB4BB1] (No symbol) [0x00007FF6E4DF7FC1] (No symbol) [0x00007FF6E4DDEDD3] (No symbol) [0x00007FF6E4DB3BEC] (No symbol) [0x00007FF6E4DB2DD6] (No symbol) [0x00007FF6E4DB4364] Microsoft::Applications::Events::ILogManager::DispatchEventBroadcast [0x00007FF6E52087C9+1319033] (No symbol) [0x00007FF6E4E2D2A8] Microsoft::Applications::Events::EventProperty::~EventProperty [0x00007FF6E4F095E1+33553] Microsoft::Applications::Events::EventProperty::~EventProperty [0x00007FF6E4F01A1F+1871] Microsoft::Applications::Events::ILogManager::DispatchEventBroadcast [0x00007FF6E52073E3+1313939] Microsoft::Applications::Events::ILogConfiguration::operator* [0x00007FF6E4FC46B8+20232] Microsoft::Applications::Events::ILogConfiguration::operator* [0x00007FF6E4FC0CD4+5412] Microsoft::Applications::Events::ILogConfiguration::operator* [0x00007FF6E4FC0DCC+5660] Microsoft::Applications::Events::EventProperty::~EventProperty [0x00007FF6E4FB4A81+735153] BaseThreadInitThunk [0x00007FFEC68526AD+29] RtlUserThreadStart [0x00007FFEC802AA68+40]

最新推荐

recommend-type

华为OD机试D卷 - 用连续自然数之和来表达整数 - 免费看解析和代码.html

私信博主免费获取真题解析以及代码
recommend-type

Screenshot_2024-05-10-20-21-01-857_com.chaoxing.mobile.jpg

Screenshot_2024-05-10-20-21-01-857_com.chaoxing.mobile.jpg
recommend-type

数字图像处理|Matlab-频域增强实验-彩色图像的频域滤波.zip

数字图像处理|Matlab-频域增强实验-彩色图像的频域滤波.zip
recommend-type

2024-2030中国定向转向膜市场现状研究分析与发展前景预测报告.docx

2024-2030中国定向转向膜市场现状研究分析与发展前景预测报告
recommend-type

开源工时填报管理系统安装包

开源工时填报管理系统安装包
recommend-type

zigbee-cluster-library-specification

最新的zigbee-cluster-library-specification说明文档。
recommend-type

管理建模和仿真的文件

管理Boualem Benatallah引用此版本:布阿利姆·贝纳塔拉。管理建模和仿真。约瑟夫-傅立叶大学-格勒诺布尔第一大学,1996年。法语。NNT:电话:00345357HAL ID:电话:00345357https://theses.hal.science/tel-003453572008年12月9日提交HAL是一个多学科的开放存取档案馆,用于存放和传播科学研究论文,无论它们是否被公开。论文可以来自法国或国外的教学和研究机构,也可以来自公共或私人研究中心。L’archive ouverte pluridisciplinaire
recommend-type

实现实时数据湖架构:Kafka与Hive集成

![实现实时数据湖架构:Kafka与Hive集成](https://img-blog.csdnimg.cn/img_convert/10eb2e6972b3b6086286fc64c0b3ee41.jpeg) # 1. 实时数据湖架构概述** 实时数据湖是一种现代数据管理架构,它允许企业以低延迟的方式收集、存储和处理大量数据。与传统数据仓库不同,实时数据湖不依赖于预先定义的模式,而是采用灵活的架构,可以处理各种数据类型和格式。这种架构为企业提供了以下优势: - **实时洞察:**实时数据湖允许企业访问最新的数据,从而做出更明智的决策。 - **数据民主化:**实时数据湖使各种利益相关者都可
recommend-type

spring添加xml配置文件

1. 创建一个新的Spring配置文件,例如"applicationContext.xml"。 2. 在文件头部添加XML命名空间和schema定义,如下所示: ``` <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.springframework.org/schema/beans
recommend-type

JSBSim Reference Manual

JSBSim参考手册,其中包含JSBSim简介,JSBSim配置文件xml的编写语法,编程手册以及一些应用实例等。其中有部分内容还没有写完,估计有生之年很难看到完整版了,但是内容还是很有参考价值的。