java爬取页面的案例怎么带Cookie
时间: 2024-06-10 07:04:41 浏览: 7
您好,关于您的问题,可以使用 Java 的 HttpURLConnection 类来进行网页爬取,并在请求中设置 Cookie。
具体代码如下:
```java
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
import java.util.List;
import java.util.Map;
public class JavaSpiderDemo {
public static void main(String[] args) {
String url = "https://www.example.com";
String cookie = "cookie1=value1; cookie2=value2; cookie3=value3";//设置Cookie
try {
URL obj = new URL(url);
HttpURLConnection httpConn = (HttpURLConnection) obj.openConnection();
httpConn.setRequestMethod("GET");
httpConn.setRequestProperty("Cookie", cookie);//设置请求中的Cookie
int responseCode = httpConn.getResponseCode();
if (responseCode == HttpURLConnection.HTTP_OK) {
BufferedReader in = new BufferedReader(new InputStreamReader(httpConn.getInputStream()));
String inputLine;
StringBuilder response = new StringBuilder();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
System.out.println("Response:" + response.toString());
//获取响应头
Map<String, List<String>> headerFields = httpConn.getHeaderFields();
for (String key : headerFields.keySet()) {
System.out.println(key + ":" + headerFields.get(key));
}
} else {
System.out.println("GET request not worked");
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
```
如上代码所示,只需在请求的 Header 中设置 Cookie 即可带上 Cookie 进行网页爬取。同时,我们还可以在代码中添加响应头的获取,获取网页返回的响应头信息。
希望以上回答能够解决您的问题。如有任何疑问,欢迎继续追问!