selector.xpath(xpath_lan).extract() 和 selector.xpath(xpath_lan).getall()的区别

selector.xpath(xpath_lan).extract() 返回一个包含所有匹配到的元素的列表，每个元素都是一个字符串，包含了该元素的 HTML 代码。

selector.xpath(xpath_lan).getall() 返回一个包含所有匹配到的元素的列表，每个元素都是一个 Selector 对象，可以继续使用 Selector 的方法进行解析。

相关问题

parsel.Selector

Parsel is a Python library used for extracting data from HTML and XML documents. It provides a powerful and flexible API for navigating and manipulating these structured documents. The `Selector` class in Parsel allows you to select elements from the document using CSS or XPath selectors.

With the `Selector` class, you can create a selector object by passing in the document string or response object. Then, you can use various methods to extract data based on your needs. For example, you can use the `css()` method to select elements using CSS selectors or the `xpath()` method to select elements using XPath expressions.

Here's an example of using `Selector` to select elements from an HTML document:

from parsel import Selector

# Create a selector object
selector = Selector(text=html_text)

# Select elements using CSS selectors
titles = selector.css('h1.title::text').getall()

# Select elements using XPath expressions
links = selector.xpath('//a/@href').getall()

# Do something with the extracted data
for title in titles:
    print(title)

for link in links:
    print(link)

In this example, `html_text` is the HTML document as a string. We create a `Selector` object using this HTML string, and then we use `css()` and `xpath()` methods to select elements based on CSS selectors and XPath expressions, respectively. Finally, we can process and use the extracted data as needed.

I hope this answers your question! Let me know if you have any more doubts.

用python的requests和xpath和正则表达式爬取豆瓣电影top250详情页的代码

以下是用Python的requests和xpath和正则表达式爬取豆瓣电影top250详情页的代码：

import requests
from lxml import etree
import re

def get_movie_details(url):
    # 发送请求获取页面内容
    response = requests.get(url)
    html = response.content.decode('utf-8')
    # 解析页面内容
    selector = etree.HTML(html)
    # 获取电影名称
    movie_name = selector.xpath('//h1/span/text()')[0]
    # 获取导演和主演信息
    director_and_cast = selector.xpath('//div[@id="info"]/span[1]/span[@class="attrs"]/a/text()')
    director = director_and_cast[0]
    cast = director_and_cast[1:]
    # 获取上映年份、地区、类型
    year_country_type = selector.xpath('//div[@id="info"]/text()')
    year = re.findall('\d{4}', year_country_type[1])[0]
    country = year_country_type[2].strip().split('/')[0]
    type = year_country_type[2].strip().split('/')[-1]
    # 获取评分和评价人数
    rating = selector.xpath('//strong[@class="ll rating_num"]/text()')[0]
    rating_num = selector.xpath('//div[@class="rating_sum"]/a/span/text()')[0]
    # 获取电影简介
    summary = selector.xpath('//div[@class="indent"]/span[@class="all hidden"]/text()')[0]
    # 构造电影信息字典
    movie_info = {
        '名称': movie_name,
        '导演': director,
        '主演': cast,
        '年份': year,
        '地区': country,
        '类型': type,
        '评分': rating,
        '评价人数': rating_num,
        '简介': summary.strip(),
    }
    return movie_info

if __name__ == '__main__':
    urls = ['https://movie.douban.com/top250?start={}'.format(i) for i in range(0, 250, 25)]
    for url in urls:
        response = requests.get(url)
        html = response.content.decode('utf-8')
        selector = etree.HTML(html)
        # 获取电影详情页链接
        movie_links = selector.xpath('//div[@class="hd"]/a/@href')
        for link in movie_links:
            movie_info = get_movie_details(link)
            print(movie_info)

这段代码会爬取豆瓣电影top250列表页面中每部电影的详情页，并从详情页中提取电影名称、导演和主演、上映年份、地区、类型、评分、评价人数、电影简介等信息，并打印出来。