python爬虫获取https://544756.yichafen.com/public/queryresult.html里的<tr></tr>标签之间内容
时间: 2023-12-18 07:03:31 浏览: 106
可以使用requests和beautifulsoup库来爬取和解析HTML页面,具体代码如下:
```python
import requests
from bs4 import BeautifulSoup
# 发送请求获取页面内容
url = 'https://544756.yichafen.com/public/queryresult.html'
response = requests.get(url)
html = response.text
# 解析HTML页面
soup = BeautifulSoup(html, 'html.parser')
trs = soup.find_all('tr')
# 遍历<tr>标签获取其中的文本内容
for tr in trs:
print(tr.text.strip())
```
这段代码会输出页面中所有`<tr>`标签里的文本内容,你可以根据需要进一步处理。
相关问题
python爬虫获取https://544756.yichafen.com/public/queryresult.html里的<td></td>标签之间内容
你可以使用 Python 的 requests 和 BeautifulSoup 库来实现爬取和解析 HTML 页面。以下是一个示例代码,可以获取页面中所有的 `<td>` 标签之间的内容:
```python
import requests
from bs4 import BeautifulSoup
url = 'https://544756.yichafen.com/public/queryresult.html'
response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')
td_tags = soup.find_all('td')
for td in td_tags:
print(td.text.strip())
```
这段代码会向指定的 URL 发送 GET 请求,获取响应的 HTML 页面内容。然后,使用 BeautifulSoup 将 HTML 页面解析成一个 BeautifulSoup 对象。最后,使用 `find_all()` 方法查找页面中所有的 `<td>` 标签,并逐个打印标签之间的文本内容。注意,这里的 `strip()` 方法用于去除文本中的空格和换行符。
根据接口“”@FeignClient(name = "UserUgsApi", url = "${newbim.login-domain}") public interface IApplicationServicePlatformClient {@PostMapping("/ums/v1/feign/omp/project/search") ResultWrapper<QueryAllProject> listUserProject(@RequestHeader("BSP_TOKEN") String bspToken, @RequestHeader("BSP_USER_ID") String bspUserId, @RequestHeader("BSP_USER_ENV_ID") String bspUserEnvId, @RequestHeader("BSP_USER_TENANT") String bspUserTenant, @RequestBody ProjectCondition queryProjectDto);和代码“@Component @Slf4j public class ApplicationServicePlatformClientProxy { @Autowired private IApplicationServicePlatformClient applicationServicePlatformClient; public QueryAllProject listUserProject(String appId, String companyId, String designStatusList, String pageSize, ProjectCondition userId) { ResultWrapper<QueryAllProject> result = null; return result.getData(); } }”,修改下面代码:@Override public List<ProjectCountVO> projectCount(String beginTime, String endTime, Integer forceType, String projectId) { List<TaskTask> taskTaskList = this.listStatisticsTask(beginTime, endTime, forceType, projectId); if(CollectionUtil.isEmpty(taskTaskList)){ return Collections.emptyList(); } List<ProjectCountVO> result = new ArrayList<>(); Map<String, List<TaskTask>> projectTaskMap = taskTaskList.stream().collect(Collectors.groupingBy(TaskTask::getProjectId)); for (Map.Entry<String, List<TaskTask>> projectTaskEntry : projectTaskMap.entrySet()) { String currentProjectId = projectTaskEntry.getKey(); ProjectCountVO projectCountVO = new ProjectCountVO(); projectCountVO.setId(currentProjectId); // 查询项目名称 QueryAllProject queryAllProject = applicationServicePlatformClient.listUserProject(); projectCountVO.setName(projectName); List<TaskTask> taskList = projectTaskEntry.getValue(); projectCountVO.setTaskCount(taskList.size()); Integer problemCount = CollectionUtil.isEmpty(taskList) ? 0 : taskList.stream().collect(Collectors.summingInt(task-> Optional.ofNullable(task.getProblemNum()).orElse(0))); projectCountVO.setProblemCount(problemCount); result.add(projectCountVO); } return result; }
需要根据接口中的方法签名,将实际调用的参数传递给FeignClient。修改后的代码如下所示:
```
@Override
public List<ProjectCountVO> projectCount(String beginTime, String endTime, Integer forceType, String projectId) {
List<TaskTask> taskTaskList = this.listStatisticsTask(beginTime, endTime, forceType, projectId);
if (CollectionUtil.isEmpty(taskTaskList)) {
return Collections.emptyList();
}
List<ProjectCountVO> result = new ArrayList<>();
Map<String, List<TaskTask>> projectTaskMap = taskTaskList.stream().collect(Collectors.groupingBy(TaskTask::getProjectId));
for (Map.Entry<String, List<TaskTask>> projectTaskEntry : projectTaskMap.entrySet()) {
String currentProjectId = projectTaskEntry.getKey();
ProjectCountVO projectCountVO = new ProjectCountVO();
projectCountVO.setId(currentProjectId);
// 查询项目名称
ResultWrapper<QueryAllProject> queryResult = applicationServicePlatformClient.listUserProject(
"BSP_TOKEN_VALUE", "BSP_USER_ID_VALUE", "BSP_USER_ENV_ID_VALUE", "BSP_USER_TENANT_VALUE",
new ProjectCondition(currentProjectId)
);
if (queryResult.isSuccess()) {
QueryAllProject queryAllProject = queryResult.getData();
projectCountVO.setName(queryAllProject.getProjectName());
}
List<TaskTask> taskList = projectTaskEntry.getValue();
projectCountVO.setTaskCount(taskList.size());
Integer problemCount = CollectionUtil.isEmpty(taskList) ? 0 : taskList.stream()
.collect(Collectors.summingInt(task -> Optional.ofNullable(task.getProblemNum()).orElse(0)));
projectCountVO.setProblemCount(problemCount);
result.add(projectCountVO);
}
return result;
}
```
其中,需要将FeignClient中的参数`bspToken`、`bspUserId`、`bspUserEnvId`、`bspUserTenant`替换为具体的值。同时,需要将`ProjectCondition`对象传递给`listUserProject()`方法。在查询成功后,可以从返回结果中获取项目名称。
阅读全文