526互联

linux-kibana

发布时间 2023-06-25 21:35:32作者: wh459086748

kibana

一、部署

1.下载软件包

[root@elk103.com ~]# ll
-rw-r--r--  1 root root 268453142 Apr  4 18:03 kibana-7.17.5-x86_64.rpm

2.安装kibana

[root@elk103.com ~]# rpm -ivh kibana-7.17.5-x86_64.rpm

3.修改kibana配置文件

vim /etc/kibana/kibana.yml 
...
# 指定kibana监听的主机地址，默认是localhost。
server.host: "0.0.0.0"
# 指定kibana连接ES集群地址
elasticsearch.hosts: ["http://10.0.0.101:9200","http://10.0.0.102:9200","http://10.0.0.103:9200"]
# 指定kibana的界面为中文简体，默认是英文。
i18n.locale: "zh-CN"

4.启动kibana

[root@elk103.com ~]# systemctl enable --now kibana

5.访问kibana界面

http://10.0.0.103:5601/

image-20230404234732619

二、基于nginx的反向代理控制访问kibana

(1)部署nginx服务
略，参考之前的笔记即可。

	(2)编写nginx的配置文件
cat > /etc/nginx/conf.d/kibana.conf <<'EOF'
server {
  listen 80;

  server_name kibana.com;

  location / {
     proxy_pass http://10.0.0.103:5601$request_uri;
     auth_basic "kibana web!";
     auth_basic_user_file conf/htpasswd;
  }
}
EOF

	(3)创建账号文件
mkdir -pv /etc/nginx/conf
yum -y install httpd-tools
htpasswd -c -b /etc/nginx/conf/htpasswd admin admin

	(4)启动nginx服务
nginx -t
systemctl reload nginx

	(5)访问nginx验证kibana访问
需要输入用户名和密码才能进入

三、配置ES集群TSL认证

1. elk101节点生成证书文件

[root@elk101.com ~]# cd /es/softwares/es7/elasticsearch-7.17.5/
[root@elk101.com /es/softwares/es7/elasticsearch-7.17.5]# ./bin/elasticsearch-certutil cert -out config/elastic-certificates.p12 -pass ""  --days 3650
[root@elk101.com /es/softwares/es7/elasticsearch-7.17.5/config]# ll
-rw------- 1 root root  3443 Apr 10 19:52 elastic-certificates.p12

2. elk101节点为证书文件修改属主和属组

[root@elk101.com /es/softwares/es7/elasticsearch-7.17.5]# chown es:es config/elastic-certificates.p12
[root@elk101.com /es/softwares/es7/elasticsearch-7.17.5]# ll config/
-rw------- 1 es es  3443 Apr 10 19:52 elastic-certificates.p12

3. elk101节点同步证书文件到其他节点

[root@elk101.com /es/softwares/es7/elasticsearch-7.17.5]# data_rsync.sh `pwd`/config/elastic-certificates.p12 
===== rsyncing elk102.com: elastic-certificates.p12 =====
命令执行成功!
===== rsyncing elk103.com: elastic-certificates.p12 =====
命令执行成功!

4. elk101节点修改ES集群的配置文件

注：如果master角色和data角色分离，则会有问题

[root@elk101.com /es/softwares/es7/elasticsearch-7.17.5]# cat /es/softwares/es7/elasticsearch-7.17.5/config/elasticsearch.yml
node.data: true
#修改所有节点为主节点
node.master: true  

#添加下边的配置开启tsl
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12

5. elk101节点同步ES配置文件到其他节点

[root@elk101.com /es/softwares/es7/elasticsearch-7.17.5]# data_rsync.sh `pwd`/config/elasticsearch.yml 
===== rsyncing elk102.com: elasticsearch.yml =====
命令执行成功!
===== rsyncing elk103.com: elasticsearch.yml =====
命令执行成功!

6. 所有节点重启ES集群

systemctl restart es7

7. 任意主机生成随机密码

#生成密码
[root@elk101.com /es/softwares/es7/elasticsearch-7.17.5]# ./bin/elasticsearch-setup-passwords auto
warning: usage of JAVA_HOME is deprecated, use ES_JAVA_HOME
Future versions of Elasticsearch will require Java 11; your Java version from [/es/softwares/jdk1.8.0_291/jre] does not meet this requirement. Consider switching to a distribution of Elasticsearch with a bundled JDK. If you are already using a distribution with a bundled JDK, ensure the JAVA_HOME environment variable is not set.
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,kibana_system,logstash_system,beats_system,remote_monitoring_user.
The passwords will be randomly generated and printed to the console.
Please confirm that you would like to continue [y/N]y


Changed password for user apm_system
PASSWORD apm_system = ijqdMffW2ZkrwZYRQaCY

Changed password for user kibana_system
PASSWORD kibana_system = lNs85X26g0eC9A6VqwRB

Changed password for user kibana
PASSWORD kibana = lNs85X26g0eC9A6VqwRB

Changed password for user logstash_system
PASSWORD logstash_system = 1TyRJW0zHiFri0dXNcEb

Changed password for user beats_system
PASSWORD beats_system = B7S5O7GPQaz6VpUXCpmV

Changed password for user remote_monitoring_user
PASSWORD remote_monitoring_user = TtbxPHsVjIkfLUvpZNFL

Changed password for user elastic
PASSWORD elastic = NviM8RNB6FnqgpsbS6Rz

8.postman访问

GET 10.0.0.101:9200/_cat/indices

image-20230410200352244

image-20230410200422230

四、配置kibana连接ES集群

1. 修改kibana的配置文件

[root@elk103.com ~]# egrep -vn '^#|^$' /etc/kibana/kibana.yml
5:server.host: "0.0.0.0"
7:elasticsearch.hosts: ["http://10.0.0.101:9200","http://10.0.0.102:9200","http://10.0.0.103:9200"]
9:i18n.locale: "zh-CN"
#取消注释
51:elasticsearch.username: "kibana_system"
#写入生成的随机密码
52:elasticsearch.password: "lNs85X26g0eC9A6VqwRB"

2. 重启kibana

[root@elk103.com ~]# systemctl restart kibana

3. 使用elastic用户登录并修改密码

http://10.0.0.103:5601/

image-20230410200912007

image-20230410200937032

image-20230410201011276

五、配置filebeat连接ES集群

[root@elk103.com /es/softwares/filebeat-7.17.5-linux-x86_64/config]# cat log-to-es_tls.yaml
filebeat.inputs:
- type: log
  paths:
    - /tmp/test-filebeat/test.log 

output.elasticsearch:
  hosts: ["http://10.0.0.101:9200","http://10.0.0.102:9200","http://10.0.0.103:9200"] 
  #添加用户名和密码
  username: "elastic"
  password: "123456"
  index: "linux-es-test"

setup.ilm.enabled: false
setup.template.name: "linux-es"
setup.template.pattern: "linux-es-*"
setup.template.overwrite: true
setup.template.settings:
  index.number_of_shards: 3
  index.number_of_replicas: 0
[root@elk103.com ~]# cat /tmp/test-filebeat/test.log
111
222
333

#启动filebeat
[root@elk103.com /es/softwares/filebeat-7.17.5-linux-x86_64]# filebeat -e -c config/log-to-es_tls.yaml

#进入kibana查看

image-20230410201455345

六、配置logstash连接ES集群

[root@elk101.com ~/config]# cat file-to-es_tsl.conf
input { 
  file {
     # 指定本地文件的路径
     path => "/tmp/linux-es-file"
     # 指定读取文件的起始位置，但前提是该文件之前未读取过或者未在".sincedb"文件中记录。
     start_position => "beginning"
  }
} 


output { 
  # stdout {} 
  
  elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "linux-es-logstash-file"
    user => "elastic"
    password => "123456"
  }
}

[root@elk101.com ~/config]# echo 'aaa' > /tmp/linux-es-file

#启动logstash
[root@elk101.com ~]# logstash -rf config/file-to-es_tsl.conf

##进入kibana查看

image-20230410202815349

补充：

查看logstash采集文件的偏移量路径。
ls -la /es/softwares/logstash-7.17.5/data/plugins/inputs/file/

如果没有传过去，就把这个目录下边的内容删掉，重新传

七、自定义角色使用logstash组件写入数据到ES集群

1. 新建角色

image-20230410203147355

2. 新建用户

image-20230410203249737

3. 配置测试

[root@elk101.com ~/config]# cat file-to-es_tsl.conf
input { 
  file {
     # 指定本地文件的路径
     path => "/tmp/linux-es-file"
     # 指定读取文件的起始位置，但前提是该文件之前未读取过或者未在".sincedb"文件中记录。
     start_position => "beginning"
  }
} 


output { 
  # stdout {} 
  
  elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "linux-es-logstash-file"
    user => "logstash"
    password => "123456"
  }
}

[root@elk101.com ~]# logstash -rf config/file-to-es_tsl.conf 

[root@elk101.com /es/softwares/es7/elasticsearch-7.17.5]# echo 'ccc' >> /tmp/linux-es-file

image-20230410203600904

补充

要遵循权限最小化原则，能不给就不给

八、基于RBAC配置权限

1.配置角色

image-20230410203911118

image-20230410203958142

image-20230410204022608

image-20230410204035742

2.创建用户

image-20230410204115537

3.测试登录

image-20230410204155786

九、配置curl连接ES集群

[root@elk101.com ~]# curl -u elastic:${密码} 10.0.0.101:9200/_cat/nodes
10.0.0.102 70 46 11 0.07 0.26 0.29 cdfhilmrstw * elk102.com
10.0.0.103 70 62  5 0.53 0.70 0.60 cdfhilmrstw - elk103.com
10.0.0.101 67 49  1 0.10 0.41 0.49 cdfhilmrstw - elk101.com