Web Scraping with Python-英文版

Web Scraping with Python by Ryan Mitchell Part I. Building Scrapers 1. Your First Web Scraper. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Connecting An Introduction to BeautifulSoup Installing BeautifulSoup Running BeautifulSoup Connecting Reliably 3 6 6 8 9 2. Advanced HTML Parsing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 You Don’t Always Need a Hammer Another Serving of BeautifulSoup find() and findAll() with BeautifulSoup Other BeautifulSoup Objects Navigating Trees Regular Expressions Regular Expressions and BeautifulSoup Accessing Attributes Lambda Expressions Beyond BeautifulSoup 13 14 16 18 18 22 27 28 28 29 3. Starting to Crawl. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 Traversing a Single Domain Crawling an Entire Site Collecting Data Across an Entire Site Crawling Across the Internet Crawling with Scrapy 31 35 38 40 45