httpclient爬虫实例

这里给出一个使用HttpClient库实现的简单爬虫实例，可以用来爬取指定网站的页面内容：

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import org.apache.http.HttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;

public class HttpClientSpider {

    public static void main(String[] args) throws IOException {
        CloseableHttpClient httpClient = HttpClients.createDefault();
        HttpGet httpGet = new HttpGet("https://www.baidu.com");
        HttpResponse response = httpClient.execute(httpGet);
        BufferedReader reader = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
        String line;
        while ((line = reader.readLine()) != null) {
            System.out.println(line);
        }
    }
}

以上代码通过创建一个默认的HttpClient实例，使用HttpGet方法获取指定网站的内容，并将返回的内容输出到控制台。需要注意的是，使用完HttpClient实例后需要手动关闭。