部署ELK

1.单点部署es

1.下载ES软件
https://www.elastic.co/cn/downloads/past-releases#elasticsearch

2.创建ES的工作目录
[root@elk105 ~]# mkdir /zhuyang/{data,logs,app} -p

3.解压软件包
[root@elk105 app]# tar xf elasticsearch-7.3.1-linux-x86_64.tar.gz

4.创建运行elasticsearch服务用户并修改权限
[root@elk105 ~]# useradd es
[root@elk105 app]# chown es.es /zhuyang/app/elasticsearch-7.3.1 -R

5.修改文件描述符最大及进程数量打开上限及
[root@elk105 config]# vim /etc/security/limits.d/es.conf
*          soft    nofile     65535
*          hard    nofile     131070
*          hard    nproc     8192

6.修改虚拟内存映射上限
临时修改内核参数(重启OS后配置丢失):
[root@elk105 ~]# sysctl -w vm.max_map_count=262144
永久生效内核参数(重启OS后配置不丢失)  强烈推荐
[root@elk105 ~]# vim /etc/sysctl.d/es.conf 
vm.max_map_count=262144
加载文件，使得生效
[root@elk105 ~]# sysctl -f /etc/sysctl.d/es.conf
查看内核参数:
[root@elk105 ~]# sysctl -q vm.max_map_count
vm.max_map_count = 262144

7.修改配置文件
[root@elk105 ~]# vim/zhuyang/app/elasticsearch-7.3.1/config/elasticsearch.yml
...
network.host: 0.0.0.0
...
discovery.seed_hosts: ["10.0.0.105"]
cluster.initial_master_nodes: ["10.0.0.105"]

8.启动es
前台启动:
[root@elk105]# su - es -c "/zhuyang/app/elasticsearch-7.3.1/bin/elasticsearch"
后台启动:
[root@elk105]# su - es -c "/zhuyang/app/elasticsearch-7.3.1/bin/elasticsearch -d"

9.检查服务是否正常访问
[root@elk105]# curl 10.0.0.105:9200
{
  "name" : "elk105",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "aMFqNy-VRi-13t_ZkC0cAg",
  "version" : {
    "number" : "7.3.1",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "4749ba6",
    "build_date" : "2019-08-19T20:19:25.651794Z",
    "build_snapshot" : false,
    "lucene_version" : "8.1.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

10.编写ES服务的启动脚本
[root@elk105]# vim /usr/lib/systemd/system/es7.service
[Unit]
Description=www.zyshaby.com
After=network.target

[Service]
User=es
Group=es
Type=simple
ExecStart=/zhuyang/app/elasticsearch-7.3.1/bin/elasticsearch
LimitNOFILE=131070
LimitNPROC=65535

[Install]
WantedBy=multi-user.target

11.使用systemctl管理es
[root@elk105 ~]# systemctl daemon-reload    #加载文件
[root@elk105 ~]# systemctl enable --now es7 #开机自启es
[root@elk105 ~]# systemctl status es7       #查看es状态

12.使用systemctl管理es前提得先把之前启动的es进程关掉
例如:
[root@elk105 0]# ps -ef|grep elastic
es         1879      1  2 00:14 ?        00:01:10 /zhuyang/app/elasticsearch-7.3.1/jdk/bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.io.tmpdir=/tmp/elasticsearch-1126705440804840429 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=data -XX:ErrorFile=logs/hs_err_pid%p.log -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m -Djava.locale.providers=COMPAT -Dio.netty.allocator.type=unpooled -XX:MaxDirectMemorySize=536870912 -Des.path.home=/zhuyang/app/elasticsearch-7.3.1 -Des.path.conf=/zhuyang/app/elasticsearch-7.3.1/config -Des.distribution.flavor=default -Des.distribution.type=tar -Des.bundled_jdk=true -cp /zhuyang/app/elasticsearch-7.3.1/lib/* org.elasticsearch.bootstrap.Elasticsearch -d
es         1894   1879  0 00:14 ?        00:00:00 /zhuyang/app/elasticsearch-7.3.1/modules/x-pack-ml/platform/linux-x86_64/bin/controller
root      17234   1460  0 00:55 pts/0    00:00:00 grep --color=auto elastic
[root@elk105 0]# kill -9 1879
[root@elk105 ~]# ps -ef|grep elastic
root      17237   1460  0 00:55 pts/0    00:00:00 grep --color=auto elastic
然后再设置为开机自启
[root@elk105 ~]# systemctl daemon-reload 
[root@elk105 ~]# systemctl enable --now es7

2.部署es常见的报错

1.java.lang.RuntimeException: can not run elasticsearch as root
    错误原因：
        运行ES服务不能以root身份运行。
        
    解决方案:
        创建普通用户运行即可。
        
2.could not find java in bundled JDK at /oldboyedu/softwares/elasticsearch-7.17.5/jdk/bin/java
    错误原因:
        找不到jkd路径，但实际上存在，可能是由于运行ES服务相关文件权限问题。
        
    解决方案:
        将ES相关的二进制文件修改为运行用户的身份即可。
        
3.ERROR: [3] bootstrap checks failed. You must address the points described in the following [3] lines before starting Elasticsearch.
bootstrap check failure [1] of [3]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535]
bootstrap check failure [2] of [3]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
bootstrap check failure [3] of [3]: the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured

    错误原因:
        最大文件描述上限，虚拟机内存大小设置及服务发现未配置。
        
    解决方案:
        按照提示解决即可，具体解决方案可参考单点部署。

3.jvm调优

1.修改ES堆内存大小
[root@elk105 ~]# vim /zhuyang/app/elasticsearch-7.3.1/config/jvm.options
...
-Xms256m
-Xmx256m
...

2.重启es服务
[root@elk105 ~]# systemctl restart es7

3.验证堆内存是否修改成功
[root@elk105 ~]# ps -ef | grep java | grep -i xms
es        17296      1 17 00:55 ?        00:00:59 /zhuyang/app/elasticsearch-7.3.1/jdk/bin/java -Xms256m -Xmx256m 

4.安装Oracle JDK

1.Oracle JDK下载地址:
https://www.oracle.com/java/technologies/downloads/#java8-linux

2.解压软件包
[root@elk105 app]# tar xf jdk-8u191-linux-x64.tar.gz

3.配置JDK环境变量
[root@elk105 app]# cat /etc/profile.d/jdk.sh 
export JAVA_HOME=/zhuyang/app/jdk1.8.0_191
export PATH=$PATH:$JAVA_HOME/bin
[root@elk105 app]# source  /etc/profile.d/jdk.sh
[root@elk105 app]# java -version
java version "1.8.0_191"
Java(TM) SE Runtime Environment (build 1.8.0_191-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.191-b12, mixed mode)

4.修改ES启动脚本传递JAVA_HOME变量
[root@elk105 app]# cat /usr/lib/systemd/system/es7.service
[Unit]
Description=www.zyshaby.com
After=network.target

[Service]
User=es
Group=es
Type=simple
ExecStart=/zhuyang/app/elasticsearch-7.3.1/bin/elasticsearch
LimitNOFILE=131070
LimitNPROC=65535
Environment=JAVA_HOME=/zhuyang/app/jdk1.8.0_191

[Install]
WantedBy=multi-user.target
5.重启es
[root@elk105 app]# systemctl daemon-reload
[root@elk105 app]# systemctl restart es7
[root@elk105 app]# systemctl status es7

6.验证命令
[root@elk105 app]# jps     # 仅显示java程序的名称和PID
2164 Jps
2061 Elasticsearch
[root@elk105 app]# jps -l  # 查看java的包名 
[root@elk105 app]# jps -v  # 查看java程序的详细信息。

5.日常查看补充

1.查看服务状态
[root@elk105 ~]# systemctl status es7 -l

2.通过系统日志查看
[root@elk105 ~]# journalctl -u es7 

3.查看启动日志
[root@elk105 ~]# tail -100f /zhuyang/app/elasticsearch-7.3.1/logs/elasticsearch.log 

6.集群部署es

1.创建数据目录和工作目录
[root@elk105 ~]# install -d /zhuyang/{data,logs}/es7 -o es -g es  #递归创建目录并修改目录属主属组权限

2.修改es配置文件
[root@elk105 ~]# vim /zhuyang/app/elasticsearch-7.3.1/config/elasticsearch.yml
# 指定集群名称
cluster.name: my-es7
# 指定数据目录
path.data: /zhuyang/data/es7
# 指定日志目录
path.logs: /zhuyang/logs/es7
# 对外暴露的IP地址
network.host: 0.0.0.0
# 指定集群外部提供的http端口,默认端口
http.port: 9200
# 指定集群内部数据传输端口，默认端口
transport.tcp.port: 9300
# 指定现有集群的主机地址
discovery.seed_hosts: ["10.0.0.105","10.0.0.106","10.0.0.107"]
# 指定集群启动时，初始参与master选举的节点
cluster.initial_master_nodes: ["10.0.0.105","10.0.0.106","10.0.0.107"]

3.拷贝数据目录
[root@elk105 ~]# scp -r /zhuyang/ 10.0.0.106:/
[root@elk105 ~]# scp -r /zhuyang/ 10.0.0.107:/
[root@elk105 ~]# scp /etc/profile.d/jdk.sh 10.0.0.106:/etc/profile.d/
[root@elk105 ~]# scp /etc/profile.d/jdk.sh 10.0.0.107:/etc/profile.d/
[root@elk105 ~]# scp /etc/sysctl.d/es.conf 10.0.0.106:/etc/sysctl.d/
[root@elk105 ~]# scp /etc/sysctl.d/es.conf 10.0.0.107:/etc/sysctl.d/
[root@elk105 ~]# scp /usr/lib/systemd/system/es7.service 10.0.0.106:/usr/lib/systemd/system/
[root@elk105 ~]# scp /usr/lib/systemd/system/es7.service 10.0.0.107:/usr/lib/systemd/system/

4.其他节点创建用户，修改权限，内核参数调优
[root@elk106 ~]# useradd -u 1000  es
[root@elk106 ~]# chown -R es.es /zhuyang/app/elasticsearch-7.3.1
[root@elk106 ~]# install -d /zhuyang/{data,logs}/es7 -o es -g es 
[root@elk106 ~]# sysctl -f /etc/sysctl.d/es.conf

[root@elk107 ~]# useradd -u 1000  es
[root@elk107 ~]# chown -R es.es /zhuyang/app/elasticsearch-7.3.1
[root@elk107 ~]# install -d /zhuyang/{data,logs}/es7 -o es -g es 
[root@elk107 ~]# sysctl -f /etc/sysctl.d/es.conf

5.清空单点的数据
[root@elk105 ~]# systemctl stop es7
[root@elk105 ~]# pkill java
[root@elk105 ~]# rm -rf /zhuyang/data/es7/* /zhuyang/logs/es7/* /tmp/*

6.所有节点同时启动服务
[root@elk105 ~]# systemctl enable --now es7
[root@elk106 ~]# systemctl enable --now es7
[root@elk107 ~]# systemctl enable --now es7

7.验证服务是否正常
[root@elk105 ~]# systemctl status es7
[root@elk106 ~]# systemctl status es7
[root@elk107 ~]# systemctl status es7

[root@elk105 ~]# ss -ntl | grep 9200
[root@elk106 ~]# ss -ntl | grep 9200
[root@elk107 ~]# ss -ntl | grep 9200

[root@elk105 ~]# curl 10.0.0.105:9200
{
  "name" : "elk105",
  "cluster_name" : "my-es7",
  "cluster_uuid" : "JyST2NtrQH6VNQl0ZHpq3g",
  "version" : {
    "number" : "7.3.1",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "4749ba6",
    "build_date" : "2019-08-19T20:19:25.651794Z",
    "build_snapshot" : false,
    "lucene_version" : "8.1.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

[root@elk105 ~]# curl 10.0.0.105:9200/_cat/nodes
10.0.0.105 34 96 0 0.11 0.16 0.11 dim - elk105
10.0.0.106 42 96 0 0.23 0.29 0.17 dim * elk106
10.0.0.107 42 96 0 0.07 0.16 0.11 dim - elk107

ES集群处于不健康状态
[root@elk105 ~]# curl 10.0.0.105:9200
{
  "name" : "elk105",
  "cluster_name" : "my-es7",
  "cluster_uuid" : "_na_",
  "version" : {
    "number" : "7.17.5",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "8d61b4f7ddf931f219e3745f295ed2bbc50c8e84",
    "build_date" : "2022-06-23T21:57:28.736740635Z",
    "build_snapshot" : false,
    "lucene_version" : "8.11.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}
[root@elk105 ~]# curl 10.0.0.105:9200/_cat/nodes
{"error":{"root_cause":[{"type":"master_not_discovered_exception","reason":null}],"type":"master_not_discovered_exception","reason":null},"status":503}

解决方案:
    重复集群部署笔记的5-7步骤即可

7.配置集群免密登录及同步脚本

1.修改主机列表
[root@elk105 ~]# vim /etc/hosts
10.0.0.105 elk105
10.0.0.106 elk106
10.0.0.107 elk107

2.elk105节点上生成密钥对
[root@elk105 ~]# ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa -q

3.elk101配置所有集群节点的免密登录
[root@elk105 ~]# for ((host_id=105;host_id<=107;host_id++));do ssh-copy-id elk${host_id} ;done

4.链接测试
[root@elk105 ~]# ssh elk106
Last login: Tue Jun 18 21:07:12 2024 from 10.0.0.1
[root@elk106 ~]# exit
logout
Connection to elk106 closed.
[root@elk105 ~]# ssh elk107
Last login: Tue Jun 18 21:07:16 2024 from 10.0.0.1
[root@elk107 ~]# 

5.所有节点安装rsync数据同步工具
[root@elk105 ~]# yum -y install rsync
[root@elk106 ~]# yum -y install rsync
[root@elk107 ~]# yum -y install rsync

6.编写同步脚本
[root@elk105 ~]# vim /usr/local/sbin/data_rsync.sh
#!/bin/bash
# Auther: Zhu Yang

if  [ $# -ne 1 ];then
   echo "Usage: $0 /path/to/file(绝对路径)"
   exit
fi 

# 判断文件是否存在
if [ ! -e $1 ];then
    echo "[ $1 ] dir or file not find!"
    exit
fi

# 获取父路径
fullpath=`dirname $1`

# 获取子路径
basename=`basename $1`

# 进入到父路径
cd $fullpath

for ((host_id=106;host_id<=107;host_id++))
  do
    # 使得终端输出变为绿色
    tput setaf 2
    echo ===== rsyncing elk${host_id}: $basename =====
    # 使得终端恢复原来的颜色
    tput setaf 7
    # 将数据同步到其他两个节点
    rsync -apz $basename  `whoami`@elk${host_id}:$fullpath
    if [ $? -eq 0 ];then
      echo "命令执行成功!"
    fi
done

[root@elk105 ~]# chmod +x /usr/local/sbin/data_rsync.sh  #给脚本授权

8.配置ES集群加密

1.生成证书文件
[root@elk105 ~]# cd /zhuyang/app/elasticsearch-7.3.1/
[root@elk105 elasticsearch-7.3.1]# ./bin/elasticsearch-certutil cert -out config/elasticsearch-certificates.p12 -pass ""

2.为证书文件修改属主和属组
[root@elk105 elasticsearch-7.3.1]# chown es:es config/elasticsearch-certificates.p12 

3.同步证书文件到其他节点
[root@elk105 elasticsearch-7.3.1]#  data_rsync.sh /zhuyang/app/elasticsearch-7.3.1/config/elasticsearch-certificates.p12 
===== rsyncing elk106: elasticsearch-certificates.p12 =====
命令执行成功!
===== rsyncing elk107: elasticsearch-certificates.p12 =====
命令执行成功!

4.修改ES集群的配置文件
[root@elk105 elasticsearch-7.3.1]# vim config/elasticsearch.yml 
...
# 在最后一行添加以下内容
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /zhuyang/app/elasticsearch-7.3.1/config/elasticsearch-certificates.p12
xpack.security.transport.ssl.truststore.path: /zhuyang/app/elasticsearch-7.3.1/config/elasticsearch-certificates.p12   #这里是指定证书的位置，可以用绝对路劲和当前路劲都可以

5.同步ES配置文件到其他节点
[root@elk105 elasticsearch-7.3.1]# data_rsync.sh /zhuyang/app/elasticsearch-7.3.1/config/elasticsearch.yml

6.所有节点重启ES集群
[root@elk105 ~]# systemctl restart es7
[root@elk106 ~]# systemctl restart es7
[root@elk107 ~]# systemctl restart es7

7.更改es集群密码
[root@elk105 bin]# ./elasticsearch-setup-passwords interactive 

8.验证es集群
[root@elk105 bin]# curl -u elastic:Zy200211@ 10.0.0.105:9200/_cat/nodes
10.0.0.105 51 63 2 1.21 0.37 0.17 dim * elk105  
10.0.0.107 43 62 1 0.49 0.19 0.13 dim - elk107
10.0.0.106 32 62 1 1.10 0.31 0.14 dim - elk106

#登录es-head访问

部署kibana

1.下载kibana软件

https://www.elastic.co/cn/downloads/past-releases/kibana-7-3-1

2.解压软件包

[root@elk105 app]# tar xf kibana-7.3.1-linux-x86_64.tar.gz

3.修改配置文件

[root@elk105 app]# vim kibana-7.3.1-linux-x86_64/config/kibana.yml 
...
server.host: "0.0.0.0"
...
elasticsearch.hosts: ["http://10.0.0.105:9200","http://10.0.0.106:9200","http://10.0.0.107:9200"]
...
elasticsearch.username: "elastic"
elasticsearch.password: "Zy200211@"
...
i18n.locale: "zh-CN"

4.启动kibana

[root@elk105 kibana-7.3.1-linux-x86_64]#  ./bin/kibana                         #前台启动
[root@elk105 kibana-7.3.1-linux-x86_64]# nohup ./bin/kibana > /zhuyang/logs/kibana.log 2>&1 &  #后台启动

5.配置systemctl管理kibana

1.创建kibana用户
[root@elk105 app]# useradd kibana

2.授权
[root@elk105 app]# chown -R kibana:kibana /zhuyang/app/kibana-7.3.1-linux-x86_64

3.配置kibana.service文件
[root@elk105 app]# cat /usr/lib/systemd/system/kibana.service 
[Unit]
Description=Kibana
Documentation=https://www.elastic.co/guide/en/kibana/current/index.html
Wants=network-online.target
After=network-online.target

[Service]
Type=simple
User=kibana
Group=kibana
ExecStart=/zhuyang/app/kibana-7.3.1-linux-x86_64/bin/kibana
# StandardOutput=/zhuyang/logs/kibana.log
# StandardError=/zhuyang/logs/kibana.log
Restart=on-failure

[Install]
WantedBy=multi-user.target

4.使用systemctl启动kibana
[root@elk105 ~]# systemctl daemon-reload 
[root@elk105 ~]# systemctl start kibana

5.验证是否启动成功
[root@elk105 ~]# ss -ntl|grep 5601
[root@elk105 ~]# ps -ef|grep kibana
登录web访问UI: http://10.0.0.105:5601/

部署logstash

1.下载logstash软件

https://artifacts.elastic.co/downloads/logstash/

2.解压软件包

[root@elk105 app]# tar xf logstash-7.3.1.tar.gz 

3.修改配置文件

[root@elk105 config]# vim logstash.conf
# Sample Logstash configuration for creating a simple
# Beats -> Logstash -> Elasticsearch pipeline.

input{
        tcp {
                mode => "server"
                host => "0.0.0.0"
                port => 4569
                codec => "json"
        }
}
output{
 elasticsearch {
    hosts => ["10.0.0.105:9200","10.0.0.106:9200","10.0.0.107:9200"]
    index => "logstash-rollover"
    user => "elastic"
    password => "Zy200211@"
  }
}

4.启动logstash

[root@elk105 bin]# ./logstash -f ../config     #前台启动
[root@elk105 ~]# nohup /zhuyang/app/logstash-7.3.1/bin/logstash -f /zhuyang/app/logstash-7.3.1/config/logstash.conf > /zhuyang/logs/logstash/logstash.log 2>&1 &   #后台启动

5.验证logstash

[root@elk105 bin]# ps -ef|grep logstash
[root@elk105 bin]# ss -ntl |grep 4569