怎么用beautiful soup提取新闻标题和链接,有循环的
时间: 2023-05-26 08:06:22 浏览: 63
假设我们需要从一个新闻网站上提取所有新闻的标题和链接,可以按照以下步骤:
1.使用requests库获取网页内容
```python
import requests
url = 'https://example.com/news'
response = requests.get(url)
content = response.content
```
2.使用BeautifulSoup解析网页内容
```python
from bs4 import BeautifulSoup
soup = BeautifulSoup(content, 'html.parser')
```
3.选取包含新闻标题和链接的标签,使用find_all方法查找多个标签
```python
news_list = soup.find_all('div', {'class': 'news'})
```
4.使用循环遍历新闻列表,提取标题和链接
```python
for news in news_list:
title = news.find('h2').text
link = news.find('a')['href']
print(title, link)
```
完整代码如下:
```python
import requests
from bs4 import BeautifulSoup
url = 'https://example.com/news'
response = requests.get(url)
content = response.content
soup = BeautifulSoup(content, 'html.parser')
news_list = soup.find_all('div', {'class': 'news'})
for news in news_list:
title = news.find('h2').text
link = news.find('a')['href']
print(title, link)
```