ElasticSearch之系统关键配置

ElasticSearch之系统关键配置

集群名称

在配置文件$ES_HOME/config/elasticsearch.yml中指定，样例如下：

cluster:
    name: logging-prod

或者

cluster.name: logging-prod

节点的名称

在配置文件$ES_HOME/config/elasticsearch.yml中指定，样例如下：

node:
    name: prod-data-2

或者

node.name: prod-data-2

监听IP地址
在配置文件$ES_HOME/config/elasticsearch.yml中指定，样例如下：

network:
    host: 192.168.1.10

或者

network.host: 192.168.1.10

集群发现的配置

在配置文件$ES_HOME/config/elasticsearch.yml中指定，样例如下：

discovery.seed_hosts:
   - 192.168.1.10:9300
   - 192.168.1.11
   - seeds.mydomain.com
   - [0:0:0:0:0:ffff:c0a8:10c]:9301
cluster.initial_master_nodes:
   - master-node-a
   - master-node-b
   - master-node-c

支持IPv4、IPv6、域名、主机名，允许同时指定端口。

路径
路径相关的关键配置，包括：

• 数据的保存路径
• 日志文件的保存路径

数据的保存路径，在配置文件$ES_HOME/config/elasticsearch.yml中使用配置项path.data指定。
日志文件的保存路径，在配置文件$ES_HOME/config/elasticsearch.yml中使用配置项path.logs指定。

配置样例如下：

path:
  data: /var/data/elasticsearch
  logs: /var/log/elasticsearch

或者

path.data: /var/data/elasticsearch
path.logs: /var/log/elasticsearch

临时目录
默认情况下，ElasticSearch使用/tmp作为临时目录。
可以使用环境变量ES_TMPDIR来指定其它路径，命令样例如下：

export ES_TMPDIR=/path/to/tmp
./bin/elasticsearch

启动elasticsearch时假如遇到了如下报错，需要确认临时目录在挂载时是否指定了noexec选项。

• java.lang.UnsatisfiedLinkerError
• failed to map segment from shared object
• failed to allocate closure

上述报错和JNA和libffi相关，ElasticSearch使用这两个完成一些平台相关的业务逻辑。

假如临时目录所在的分区在挂载时指定了noexec选项，并且出于一些原因，不方便切换新的临时目录。
为使JNA和libffi正常工作，可以指定各自的临时目录，操作命令样例如下：

export LIBFFI_TMPDIR=/path/to/tmp
export ES_JAVA_OPTS="-Djava.io.tmpdir=/path/to/temp/dir"
./bin/elasticsearch

JVM的配置

通过环境变量ES_JAVA_OPTS，可以指定JVM参数，如下是命令样例：

ES_JAVA_OPTS="-Xms2g -Xmx2g" ./bin/elasticsearch

另外一个例子，如下：

ES_JAVA_OPTS="-Xms2g -Xmx2g -Djava.io.tmpdir=/path/to/temp/dir" ./bin/elasticsearch

另外可以通过修改配置文件$ES_HOME/config/jvm.options，增加JVM相关的参数。

当前$ES_HOME/config/jvm.options中默认提供了如下参数。

-XX:+UseG1GC

## JVM temporary directory
-Djava.io.tmpdir=${ES_TMPDIR}

# Leverages accelerated vector hardware instructions; removing this may
# result in less optimal vector performance
20-:--add-modules=jdk.incubator.vector

## heap dumps

# generate a heap dump when an allocation from the Java heap fails; heap dumps
# are created in the working directory of the JVM unless an alternative path is
# specified
-XX:+HeapDumpOnOutOfMemoryError

# exit right after heap dump on out of memory error
-XX:+ExitOnOutOfMemoryError

# specify an alternative path for heap dumps; ensure the directory exists and
# has sufficient space
-XX:HeapDumpPath=data

# specify an alternative path for JVM fatal error logs
-XX:ErrorFile=logs/hs_err_pid%p.log

## GC logging
-Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,level,pid,tags:filecount=32,filesize=64m

增加堆的参数，样例如下：

-Xms4g
-Xmx4g
-XX:MaxDirectMemorySize=2g

修改参数之后，可以通过日志，观察JVM的参数。

[2023-11-22T00:21:47,743][INFO ][o.a.l.i.v.PanamaVectorizationProvider] [jackie-ubuntu] Java vector incubator API enabled; uses preferredBitSize=256
[2023-11-22T00:21:48,670][INFO ][o.e.n.Node               ] [jackie-ubuntu] version[8.11.1], pid[19483], build[tar/6f9ff581fbcde658e6f69d6ce03050f060d1fd0c/2023-11-11T10:05:59.421038163Z], OS[Linux/5.4.0-166-generic/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/21.0.1/21.0.1+12-29]
[2023-11-22T00:21:48,672][INFO ][o.e.n.Node               ] [jackie-ubuntu] JVM home [/home/jackie/software/elasticsearch-8.11.1/jdk], using bundled JDK [true]
[2023-11-22T00:21:48,672][INFO ][o.e.n.Node               ] [jackie-ubuntu] JVM arguments [-Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -Djava.security.manager=allow, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=SPI,COMPAT, --add-opens=java.base/java.io=org.elasticsearch.preallocate, -XX:+UseG1GC, -Djava.io.tmpdir=/tmp/elasticsearch-18105487545703078353, --add-modules=jdk.incubator.vector, -XX:+HeapDumpOnOutOfMemoryError, -XX:+ExitOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,level,pid,tags:filecount=32,filesize=64m, -Xms1926m, -Xmx1926m, -XX:MaxDirectMemorySize=1009778688, -XX:G1HeapRegionSize=4m, -XX:InitiatingHeapOccupancyPercent=30, -XX:G1ReservePercent=15, -Des.distribution.type=tar, --module-path=/home/jackie/software/elasticsearch-8.11.1/lib, --add-modules=jdk.net, --add-modules=ALL-MODULE-PATH, -Djdk.module.main=org.elasticsearch.server]

[2023-11-22T00:22:09,195][INFO ][o.e.e.NodeEnvironment    ] [jackie-ubuntu] heap size [1.8gb], compressed ordinary object pointers [true]

也可以通过API，查看JVM的配置。

curl -X GET "https://localhost:9200/_nodes/jvm?pretty" --cacert $ES_HOME/config/certs/http_ca.crt -u "elastic:ohCxPH=QBE+s5=*lo7F9"

执行结果如下：

{
  "_nodes" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "cluster_name" : "elasticsearch",
  "nodes" : {
    "aKgBu7LgS9a6iPYH8n2JPw" : {
      "name" : "jackie-ubuntu",
      "transport_address" : "127.0.0.1:9300",
      "host" : "127.0.0.1",
      "ip" : "127.0.0.1",
      "version" : "8.11.1",
      "transport_version" : 8512001,
      "index_version" : 8500003,
      "component_versions" : {
        "transform_config_version" : 10000099,
        "ml_config_version" : 11000099
      },
      "build_flavor" : "default",
      "build_type" : "tar",
      "build_hash" : "6f9ff581fbcde658e6f69d6ce03050f060d1fd0c",
      "roles" : [
        "data",
        "data_cold",
        "data_content",
        "data_frozen",
        "data_hot",
        "data_warm",
        "ingest",
        "master",
        "ml",
        "remote_cluster_client",
        "transform"
      ],
      "attributes" : {
        "ml.machine_memory" : "4040327168",
        "ml.allocated_processors" : "4",
        "ml.allocated_processors_double" : "4.0",
        "ml.max_jvm_size" : "2021654528",
        "ml.config_version" : "11.0.0",
        "xpack.installed" : "true",
        "transform.config_version" : "10.0.0"
      },
      "jvm" : {
        "pid" : 19483,
        "version" : "21.0.1",
        "vm_name" : "OpenJDK 64-Bit Server VM",
        "vm_version" : "21.0.1+12-29",
        "vm_vendor" : "Oracle Corporation",
        "using_bundled_jdk" : true,
        "start_time_in_millis" : 1700583699298,
        "mem" : {
          "heap_init_in_bytes" : 2021654528,
          "heap_max_in_bytes" : 2021654528,
          "non_heap_init_in_bytes" : 7667712,
          "non_heap_max_in_bytes" : 0,
          "direct_max_in_bytes" : 0
        },
        "gc_collectors" : [
          "G1 Young Generation",
          "G1 Concurrent GC",
          "G1 Old Generation"
        ],
        "memory_pools" : [
          "CodeHeap 'non-nmethods'",
          "Metaspace",
          "CodeHeap 'profiled nmethods'",
          "Compressed Class Space",
          "G1 Eden Space",
          "G1 Old Gen",
          "G1 Survivor Space",
          "CodeHeap 'non-profiled nmethods'"
        ],
        "using_compressed_ordinary_object_pointers" : "true",
        "input_arguments" : [
          "-Des.networkaddress.cache.ttl=60",
          "-Des.networkaddress.cache.negative.ttl=10",
          "-Djava.security.manager=allow",
          "-XX:+AlwaysPreTouch",
          "-Xss1m",
          "-Djava.awt.headless=true",
          "-Dfile.encoding=UTF-8",
          "-Djna.nosys=true",
          "-XX:-OmitStackTraceInFastThrow",
          "-Dio.netty.noUnsafe=true",
          "-Dio.netty.noKeySetOptimization=true",
          "-Dio.netty.recycler.maxCapacityPerThread=0",
          "-Dlog4j.shutdownHookEnabled=false",
          "-Dlog4j2.disable.jmx=true",
          "-Dlog4j2.formatMsgNoLookups=true",
          "-Djava.locale.providers=SPI,COMPAT",
          "--add-opens=java.base/java.io=org.elasticsearch.preallocate",
          "-XX:+UseG1GC",
          "-Djava.io.tmpdir=/tmp/elasticsearch-18105487545703078353",
          "--add-modules=jdk.incubator.vector",
          "-XX:+HeapDumpOnOutOfMemoryError",
          "-XX:+ExitOnOutOfMemoryError",
          "-XX:HeapDumpPath=data",
          "-XX:ErrorFile=logs/hs_err_pid%p.log",
          "-Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,level,pid,tags:filecount=32,filesize=64m",
          "-Xms1926m",
          "-Xmx1926m",
          "-XX:MaxDirectMemorySize=1009778688",
          "-XX:G1HeapRegionSize=4m",
          "-XX:InitiatingHeapOccupancyPercent=30",
          "-XX:G1ReservePercent=15",
          "-Des.distribution.type=tar",
          "--module-path=/home/jackie/software/elasticsearch-8.11.1/lib",
          "--add-modules=jdk.net",
          "--add-modules=ALL-MODULE-PATH",
          "-Djdk.module.main=org.elasticsearch.server"
        ]
      }
    }
  }
}

相关资料

• Important Elasticsearch configuration
• Advanced configuration
• Important system configuration
• Nodes info API
• Ensure JNA temporary directory permits executables
• Discovery
• Bootstrapping a cluster
• Quorum-based decision making

本栏目推荐文章
• VMware虚拟机安装银河麒麟桌面版V10系统
• OpenHarmony 上跑CV 应用 - Windows上搭建 Linux 桌面系统
• JFinal学生信息管理系统
• 从Bitcask存储模型谈超轻量级KV系统设计与实现
• Linux系统的一些实用操作 [补档-2023-07-30]
• Qt/C++编写视频监控系统83-自定义悬浮条信息
• 软件构造大作业 综合信息管理系统
• 图书管理系统概要设计说明书
• 物流管理系统详细设计说明书
• 成绩管理系统软件体系结构设计说明书

ElasticSearch 关键 系统elasticsearch关键 系统 全文检索elasticsearch completion关键字 elasticsearch logstash kibana系统 品级 套件 关键 系统 关键点 管理系统 关键 系统 能源 系统 广义 关键词 赢得 武器 关键 系统 特性 关键 数据 系统 商机 作用 关键 系统 数据表 电力系统 关键 电力