用Python做一个外汇网的爬虫代码
时间: 2023-09-04 17:12:13 浏览: 49
好的,下面是一个完整的外汇网的爬虫代码,使用Python3和Requests库:
```python
import requests
from bs4 import BeautifulSoup
import csv
# 目标页面
url = 'https://www.forex.com/en-us/education/education-themes/'
# 发送请求
response = requests.get(url)
# 解析HTML
soup = BeautifulSoup(response.text, 'html.parser')
# 找到主题列表
topics = soup.find_all('div', {'class': 'theme-list__item'})
# 遍历每个主题,提取标题、URL和描述信息
results = []
for topic in topics:
title = topic.find('a', {'class': 'theme-list__link'}).text.strip()
url = topic.find('a', {'class': 'theme-list__link'})['href']
description = topic.find('div', {'class': 'theme-list__description'}).text.strip()
result = {'title': title, 'url': url, 'description': description}
results.append(result)
# 保存结果到CSV文件
with open('forex.csv', 'w', newline='') as csvfile:
fieldnames = ['title', 'url', 'description']
writer = csv.DictWriter(csvfile, fieldnames=fieldnames)
writer.writeheader()
for result in results:
writer.writerow(result)
```
这个爬虫将爬取https://www.forex.com/en-us/education/education-themes/页面,并提取每个主题的标题、URL和描述信息,并将结果保存到名为“forex.csv”的CSV文件中。
你可以根据自己的需求进行修改和扩展。