尘","ServerID":"2090","Time":"2023-06-06 14:00:11","Type":"guildlog","eventtime":"2023-06-06 14:00:11"},使用logstash filter 解析输出并自定义message字段解析成json格式,并同时按照解析field根据Type和ServerID组合自定义表Type_ServerID,然后output输出到jdbc中更具定义的表名写入到对应的多个表中
时间: 2023-07-09 20:46:57 浏览: 35
好的,你的问题是如何使用logstash filter对日志进行解析并输出到jdbc中?
首先,在logstash配置文件中添加file input插件以读取日志文件,然后使用grok filter插件对日志进行解析,将解析后的字段存入message字段,并使用mutate filter插件将message字段转换为json格式。
接下来,使用if条件语句判断Type和ServerID字段的值,然后使用jdbc output插件将解析后的日志写入到对应的表中。
具体的配置文件如下:
```
input {
file {
path => "/path/to/logfile"
start_position => "beginning"
}
}
filter {
grok {
match => { "message" => '{"PlayerID":"%{WORD:PlayerID}","GuildName":"%{WORD:GuildName}","GuildID":"%{WORD:GuildID}","LogType":"%{WORD:LogType}","Content":"%{GREEDYDATA:Content}","ServerID":"%{WORD:ServerID}","Time":"%{TIMESTAMP_ISO8601:Time}","Type":"%{WORD:Type}","eventtime":"%{TIMESTAMP_ISO8601:EventTime}"}' }
}
mutate {
convert => { "Content" => "string" }
add_field => { "message" => "{\"PlayerID\":\"%{PlayerID}\",\"GuildName\":\"%{GuildName}\",\"GuildID\":\"%{GuildID}\",\"LogType\":\"%{LogType}\",\"Content\":\"%{Content}\",\"ServerID\":\"%{ServerID}\",\"Time\":\"%{Time}\",\"Type\":\"%{Type}\",\"EventTime\":\"%{EventTime}\"}" }
}
}
output {
if [Type] == "guildlog" {
jdbc {
connection_string => "jdbc:mysql://localhost:3306/mydb"
username => "myuser"
password => "mypassword"
statement => "INSERT INTO Type_%{ServerID} (PlayerID,GuildName,GuildID,LogType,Content,ServerID,Time,Type,EventTime) VALUES ('%{PlayerID}','%{GuildName}','%{GuildID}','%{LogType}','%{Content}','%{ServerID}','%{Time}','%{Type}','%{EventTime}')"
}
}
}
```
这样,你就可以使用logstash filter对日志进行解析并输出到jdbc中了。