编写java爬取若依系统页面数据并写入数据库的代码
时间: 2023-09-10 07:05:17 浏览: 119
### 回答1:
可以参考以下代码:import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.URL;
import java.net.URLConnection;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.ResultSet;public class JavaCrawler {
public static void main(String[] args) throws Exception {
String url = "http://www.ruoyi.vip/";
URL urlObject = new URL(url);
URLConnection uc = urlObject.openConnection();
BufferedReader in = new BufferedReader(new InputStreamReader(uc.getInputStream()));
String inputLine = null;
StringBuffer sb = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
sb.append(inputLine);
}
in.close();
//数据库连接
Class.forName("com.mysql.jdbc.Driver");
String url2 = "jdbc:mysql://localhost:3306/ruoyi";
String username = "root";
String password = "123456";
Connection conn = DriverManager.getConnection(url2, username, password);
String sql = "insert into ruoyi_pages (url, content) values (?, ?)";
PreparedStatement pstmt = conn.prepareStatement(sql);
pstmt.setString(1, url);
pstmt.setString(2, sb.toString());
int result = pstmt.executeUpdate();
System.out.println("写入" + result + "条数据");
pstmt.close();
conn.close();
}
}
### 回答2:
编写Java代码来爬取若依系统页面数据并写入数据库需要用到一些库和技术。下面是一个简单的示例代码:
```java
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.SQLException;
public class WebCrawler {
public static void main(String[] args) {
String url = "jdbc:mysql://localhost:3306/database_name";
String username = "username";
String password = "password";
try {
Connection connection = DriverManager.getConnection(url, username, password);
// 指定要爬取的页面URL
String pageUrl = "http://example.com";
Document document = Jsoup.connect(pageUrl).get();
// 使用CSS选择器定位到要爬取的数据
Elements elements = document.select("h3.title");
for (Element element : elements) {
String title = element.text(); // 获取标题
// 将数据写入数据库
String sql = "INSERT INTO table_name (title) VALUES (?)";
PreparedStatement statement = connection.prepareStatement(sql);
statement.setString(1, title);
statement.executeUpdate();
}
connection.close();
System.out.println("数据写入成功!");
} catch (SQLException e) {
e.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}
}
}
```
上述代码中,我们使用Jsoup库来处理HTML页面,通过连接到指定的页面URL并使用CSS选择器来定位要爬取的数据。然后,我们使用JDBC来连接到数据库,将数据写入到数据库中的特定表中。
请根据实际情况修改代码中的数据库连接信息、页面URL、选择器和插入语句,以适应你要爬取的若依系统页面数据并写入数据库。
### 回答3:
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.SQLException;
public class Spider {
public static void main(String[] args) throws IOException, SQLException {
String url = "http://example.com"; // 替换为你要爬取的若依系统页面的URL
String htmlContent = getHtml(url);
parseAndSaveData(htmlContent);
}
public static String getHtml(String urlString) throws IOException {
StringBuilder content = new StringBuilder();
URL url = new URL(urlString);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
BufferedReader reader = new BufferedReader(new InputStreamReader(connection.getInputStream()));
String line;
while ((line = reader.readLine()) != null) {
content.append(line).append("\n");
}
reader.close();
return content.toString();
}
public static void parseAndSaveData(String htmlContent) throws SQLException {
// 解析htmlContent获取你想要的数据,示例仅做演示
String data = htmlContent.substring(htmlContent.indexOf("<title>") + 7, htmlContent.indexOf("</title>"));
Connection connection = DriverManager.getConnection("jdbc:mysql://localhost:3306/database", "username", "password");
String sql = "INSERT INTO table_name (column_name) VALUES (?)";
PreparedStatement statement = connection.prepareStatement(sql);
statement.setString(1, data);
statement.executeUpdate();
statement.close();
connection.close();
}
}
上述代码首先通过getHtml方法获取指定URL的页面内容,然后调用parseAndSaveData方法来解析页面的HTML内容并获取需要的数据,再将数据写入数据库中,示例代码仅为演示,具体解析和写入数据库的操作需要根据你要爬取的若依系统页面的结构和数据类型来进行修改和适配。
阅读全文