搭建ELK6.2.4环境(cent7.4)，收集和分析Nginx日志，使用Filebeat，ES单点，Redis缓存。

需积分: 5 105 浏览量更新于2023-11-22 收藏 4.88MB DOCX 举报

1）准备环境：在192.168.171.128上安装elasticsearch； 2）在192.168.171.129上：安装filebeat和nginx，并使用filebeat收集nginx日志，将日志数据写入Redis； 3）在192.168.171.130上：安装nginx和kibana，配置nginx使得kibana能够使用用户名和密码认证登录； 4）在192.168.171.131上：安装Redis和logstash2，使用logstash2将日志数据从Redis读出，并写入elasticsearch； 5）使用elk6.2.4版本，对收集到的nginx日志数据进行分析和绘图，实现单点elasticsearch和redis缓存。

2)在 192.168.171.129 上:安装 logstash6.2.4+准备准备的 nginx 日志文件:(版本要和 es 版本相同，是日志搜集服务，搜集日志后可写入 es)

a)安装 filebeat(需要 jdk 环境)

[root@bogon ~]# mkdir /usr/local/java

[root@bogon ~]# rz

上传 jdk1.8 包

[root@bogon ~]# ls

jdk-8u144-linux-x64.tar.gz

[root@bogon ~]# tar -zxf jdk-8u144-linux-x64.tar.gz

[root@bogon ~]# ls

jdk1.8.0_144 jdk-8u144-linux-x64.tar.gz

[root@bogon ~]# mv jdk1.8.0_144/ /usr/local/java/

[root@bogon ~]# ls /usr/local/java/

jdk1.8.0_144

[root@bogon ~]# vim /etc/profile

#最后面添加：

JAVA_HOME=/usr/local/java/jdk1.8.0_144

CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

PATH=$JAVA_HOME/bin:$PATH

export JAVA_HOME CLASSPATH PATH

[root@bogon ~]# source /etc/profile

[root@bogon ~]# java -version

java version "1.8.0_144"

Java(TM) SE Runtime Environment (build 1.8.0_144-b01)

Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)

安装 filebeat

[root@bogon ~]# rz

上传 filebeat 包

[root@bogon ~]# ls filebeat-6.2.2-x86_64.rpm

filebeat-6.2.2-x86_64.rpm

[root@bogon ~]# yum -y install filebeat-6.2.2-x86_64.rpm

[root@bogon ~]# cp /etc/filebeat/filebeat.yml /etc/filebeat/filebeat.yml.bak

[root@bogon ~]# vim /etc/filebeat/filebeat.yml #修改 filebeat 配置文件

###################### Filebeat Configuration Example #########################

# This file is an example configuration file highlighting only the most common

# options. The filebeat.reference.yml file from the same directory contains all the

# supported options with more comments. You can use it as a reference.

# You can find the full configuration reference here:

# https://www.elastic.co/guide/en/beats/filebeat/index.html

# For more available modules and options, please see the filebeat.reference.yml sample

# configuration file.

#=========================== Filebeat prospectors =============================

filebeat.prospectors:

# Each - is a prospector. Most options can be set at the prospector level, so

# you can use different prospectors for various configurations.

# Below are the prospector specific configurations.

# nginx 的 access 日志:

- type: log

enabled: true

paths:

- /usr/local/nginx/logs/access.log

fields:

log_source: nginx-access-log

output.redis:

hosts: ["192.168.171.131"]

port: 6379

password: "123"

key: "filebeat2"

db: 6

datatype: list

#注意:上面的 filebeat2 要和 logstash 中从 redis 里对应的 key 要要一致

#db: 6 随便从 redis 里选择了一个序号的数据库,也可 0,1,等等

# Exclude lines. A list of regular expressions to match. It drops the lines that are

# matching any regular expression from the list.

#exclude_lines: ['^DBG']

# Include lines. A list of regular expressions to match. It exports the lines that are

# matching any regular expression from the list.

#include_lines: ['^ERR', '^WARN']

# Exclude files. A list of regular expressions to match. Filebeat drops the files that

# are matching any regular expression from the list. By default, no files are dropped.

#exclude_files: ['.gz$']

# Optional additional fields. These fields can be freely picked

# to add additional information to the crawled log files for filtering

#fields:

# level: debug

# review: 1

### Multiline options

# Mutiline can be used for log messages spanning multiple lines. This is common

# for Java Stack Traces or C-Line Continuation

# The regexp Pattern that has to be matched. The example pattern matches all lines starting with [

#multiline.pattern: ^\[

# Defines if the pattern set under pattern should be negated or not. Default is false.

#multiline.negate: false

# Match can be set to "after" or "before". It is used to define if lines should be append to a pattern

# that was (not) matched before or after or as long as a pattern is not matched based on negate.

# Note: After is the equivalent to previous and before is the equivalent to to next in Logstash

#multiline.match: after

#============================= Filebeat modules ===============================

filebeat.config.modules:

# Glob pattern for configuration loading

path: ${path.config}/modules.d/*.yml

# Set to true to enable config reloading

reload.enabled: false

剩余37页未读，继续阅读

运维实战课程

粉丝: 1578
资源: 410

搭建ELK6.2.4环境(cent7.4)，收集和分析Nginx日志，使用Filebeat，ES单点，Redis缓存。

ELK日志分析系统-filebeat

filebeat收集nginx日志的配置文件

Filebeat 采集 Nginx 日志的方法

filebeat-6.2.4-linux-arm.tar.gz

ELK 6.2.4环境搭建及日志收集实战

ELK 6.2.4环境搭建及日志收集配置详情 (20~40字范围内)

elk(都是6.2.4重点-版本2-收集nginx日志-无filebeat

filebeat.tar.gz

ELK日志分析系统软件包及关键组件介绍

Docker容器治理：资源限制、容器监控与日志管理

最新资源