解决k8s集群中opensearch服务CrashLoopBackOff的问题

发布时间 2023-09-18 16:19:06作者: 大师兄啊哈

1. 前言

我使用的k8s版本是1.23.6,前面安装了Kubesphere可视化组件,然后开始的时候没有启用日志模块,而是在安装Kubesphere后启用日志系统(参考在安装后启用日志系统),安装一切正常,但是安装后验证,就出现opensearch-cluster-master-x和opensearch-cluster-data-x这些pod都是CrashLoopBackOff的状态,看日志,显示报错如下:

 ERROR: [1] bootstrap checks failed
 [1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
 ERROR: OpenSearch did not exit normally - check the logs at /usr/share/opensearch/logs/opensearch-cluster.log

目测是虚拟内存的问题。

2. 解决

这些pod都是由有状态服务(statefulset)启动的,分别为:

  • opensearch-cluster-data
  • opensearch-cluster-master

使用Kubesphere或者命令进行编辑:

kubectl edit sts opensearch-cluster-data -n kubesphere-logging-system

找到initContainers,并增加一个新的initc:

        - name: sysctl
          image: 'busybox:latest'
          command:
            - sh
            - '-c'
            - sysctl -w vm.max_map_count=262144
          resources: {}
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          imagePullPolicy: IfNotPresent
          securityContext:
            privileged: true
            runAsUser: 0

将startupProbe的initialDelaySeconds改为30。

同样对opensearch-cluster-master也进行同样的操作,然后重新创建就可以了。

3. 参考

[1] Elasticsearch: Max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

[2] max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

(完)