import requests from bs4 import BeautifulSoup from pprint import pprint for page in range(5): url = "https://movie.douban.com/subject/30391186/comments?start={}&limit=20&sort=new_score&status=P".format(page*20) headers = { "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3" } response = requests.get(url, headers=headers) soup = BeautifulSoup(response.text, "html.parser") comments = [] for comment in soup.select(".comment-item"): content = comment.select_one(".comment-content").get_text().strip() author = comment.select_one(".comment-info a").get_text().strip() likes = comment.select_one(".votes").get_text().strip() time = comment.select_one(".comment-time").get_text().strip() location = comment.select_one(".comment-location").contents[-1].strip() comments.append({ "content": content, "author": author, "likes": likes, "time": time, "location": location }) print("第{}页的数据:".format(page+1)) pprint(comments),我要将数据存入数据库
时间: 2024-04-03 08:35:20 浏览: 76
import reimport requestsfrom bs4 import BeautifulSoupimport t
好的,您需要先安装一个 Python 的数据库操作库,比如 `pymysql` 或者 `sqlite3`。然后可以按照以下步骤将数据存入数据库:
1. 导入数据库操作库,比如 `pymysql`:
```python
import pymysql
```
2. 连接数据库:
```python
conn = pymysql.connect(host='localhost', port=3306, user='root', password='password', db='database_name', charset='utf8mb4')
```
其中,`host` 是数据库的主机地址,`port` 是数据库的端口号,`user` 和 `password` 分别是数据库的用户名和密码,`db` 是要连接的数据库名,`charset` 是字符集。
3. 创建一个游标对象:
```python
cur = conn.cursor()
```
4. 创建一个表:
```python
cur.execute('''CREATE TABLE IF NOT EXISTS comments
(id INT PRIMARY KEY AUTO_INCREMENT,
content TEXT,
author VARCHAR(255),
likes INT,
time VARCHAR(255),
location VARCHAR(255))''')
```
其中,`comments` 是表名,`id` 是自增的主键,`content`、`author`、`likes`、`time`、`location` 分别是要存储的字段。
5. 插入数据:
```python
for comment in comments:
cur.execute('''INSERT INTO comments (content, author, likes, time, location)
VALUES (%s, %s, %s, %s, %s)''',
(comment['content'], comment['author'], comment['likes'], comment['time'], comment['location']))
conn.commit()
```
6. 关闭游标和数据库连接:
```python
cur.close()
conn.close()
```
以上就是将数据存入数据库的步骤,您可以根据自己的需要修改表名、字段名等信息。
阅读全文