用playwright的page.cookies()方法获取页面cookies,然后将其传递给requests的cookies参数,写一段代码
时间: 2024-06-10 14:08:17 浏览: 271
来获取页面数据。
以下是一个使用 Playwright 和 requests 库的示例代码,实现了从页面获取 cookie 并将其传递给 requests 获取页面数据的功能:
```python
import requests
from playwright.sync_api import Playwright, sync_playwright
# 先使用 Playwright 获取页面 cookie
def get_cookies_from_page(page: Playwright):
cookies = page.cookies()
return {c['name']: c['value'] for c in cookies}
# 设置 requests 的 cookie 参数
def set_cookies_for_requests(requests: requests.sessions.Session, cookies: dict):
for name, value in cookies.items():
requests.cookies.set(name, value)
# 读取页面数据
def fetch_data():
with sync_playwright() as playwright:
browser = playwright.chromium.launch()
page = browser.new_page()
page.goto('https://www.example.com')
cookies = get_cookies_from_page(page) # 获取页面 cookie
set_cookies_for_requests(requests.Session(), cookies) # 将 cookie 传递给 requests
# 然后使用 requests 获取页面数据
response = requests.get('https://www.example.com')
print(response.text)
browser.close()
fetch_data()
```
该代码使用 Playwright 获取页面 cookie,然后将其传递给 requests,使用 requests 获取页面数据。注意,在函数 `get_cookies_from_page` 中,我们将 Playwright 获取的 cookie 数据转换为字典格式,方便传递给 requests 的 `cookies` 参数。
阅读全文