[JDK/APM] 应用诊断工具之jcmd

发布时间 2023-12-06 20:19:27作者: 千千寰宇

0 序言

  • 工欲善其事,必先利其器。最近遇到一个生产环境的极难排查的问题,已经消耗了近一段时间的较多心力了,故今日稳固和新学习一下JVM相关的诊断工具,以便于磨刀霍霍向"BUG"

1 jcmd 概述

1.1 jcmd 简述

  • jcmd 是在 JDK1.7 以后,新增了一个命令行工具。
    • 即 支持的最低版本: JDK1.7
  • jcmd 是一个多功能的工具,相比 jstat 功能更为全面的工具,可用于获取目标 Java 进程性能统计、JFR、内存使用、垃圾收集、线程堆栈、JVM 运行时间,也可以手动执行 GC导出线程信息导出堆信息等信息。

1.2 版本兼容问题

  • 在 JDK 1.8.0_73中不支持 GC.heap_info 参数

以 JDK 1.8.0_261 示范:

C:\Users\{USER}\Desktop>jcmd 17124 GC.heap_info
17124:
 PSYoungGen      total 139264K, used 12359K [0x000000076c000000, 0x0000000776780000, 0x00000007c0000000)
  eden space 127488K, 0% used [0x000000076c000000,0x000000076c0c9cf8,0x0000000773c80000)
  from space 11776K, 98% used [0x0000000773e00000,0x0000000774948190,0x0000000774980000)
  to   space 22016K, 0% used [0x0000000775200000,0x0000000775200000,0x0000000776780000)
 ParOldGen       total 394752K, used 100223K [0x00000006c4000000, 0x00000006dc180000, 0x000000076c000000)
  object space 394752K, 25% used [0x00000006c4000000,0x00000006ca1dfe90,0x00000006dc180000)
 Metaspace       used 110321K, capacity 115630K, committed 115840K, reserved 1150976K
  class space    used 13288K, capacity 14196K, committed 14208K, reserved 1048576K

2 jcmd 用法

2.1 help

  • jcmd help
C:\Users\{USER}\Desktop>jcmd help
Error parsing arguments: No command specified

Usage: jcmd <pid | main class> <command ...|PerfCounter.print|-f file>
   or: jcmd -l
   or: jcmd -h

  command must be a valid jcmd command for the selected jvm.
  Use the command "help" to see which commands are available.
  If the pid is 0, commands will be sent to all Java processes.
  The main class argument will be used to match (either partially
  or fully) the class used to start Java.
  If no options are given, lists Java processes (same as -p).

  PerfCounter.print display the counters exposed by this process
  -f  read and execute commands from the file
  -l  list JVM processes on the local machine
  -h  this help
  • jcmd {PID} help

命令:jcmd <pid> help
作用:列出当前运行的 java 进程可以执行的操作
说明:不同的 jvm 进程 支持的操作可能是不一样的

# jcmd 17124 help
17124:
The following commands are available:
JFR.stop
JFR.start
JFR.dump
JFR.check
VM.native_memory
VM.check_commercial_features
VM.unlock_commercial_features
ManagementAgent.stop
ManagementAgent.start_local
ManagementAgent.start
VM.classloader_stats
GC.rotate_log
Thread.print
GC.class_stats
GC.class_histogram
GC.heap_dump
GC.finalizer_info
GC.heap_info
GC.run_finalization
GC.run
VM.uptime
VM.dynlibs
VM.flags
VM.system_properties
VM.command_line
VM.version
help

For more information about a specific command use 'help <command>'.

参数描述

  • pid:[常用] 接收诊断命令请求的进程ID。

  • main class :**[不常用] **接收诊断命令请求的进程的main类。匹配进程时,main类名称中包含指定子字符串的任何进程均是匹配的。

    如果多个正在运行的Java进程共享同一个main类,诊断命令请求将会发送到所有的这些进程中。

  • command:[常用] 支持的子命令,示例:GC.class_stats、GC.class_histogram。

    注意: 如果任何参数含有空格,你必须使用英文的单引号或双引号将其包围起来。 此外,你必须使用转义字符来转移参数中的单引号或双引号,以阻止操作系统shell处理这些引用标记。当然,你也可以在参数两侧加上单引号,然后在参数内使用双引号(或者,在参数两侧加上双引号,在参数中使用单引号)。

  • Perfcounter.print:[不常用] 打印目标Java进程上可用的性能计数器。性能计数器的列表可能会随着Java进程的不同而产生变化。

  • -f file:[不常用] 从文件file中读取命令,然后在目标Java进程上调用这些命令。在file中,每个命令必须写在单独的一行。以"#"开头的行会被忽略。当所有行的命令被调用完毕后,或者读取到含有stop关键字的命令,将会终止对file的处理。

  • -l:[常用] 查看所有的进程列表信息。

  • -h:[常用] 查看帮助信息。(同 -help)

2.2 查看具体命令的参数/help

命令:jcmd <pid> help command

作用:列出当前运行的 java 进程可以执行的操作

说明:不同的 jvm 进程 支持的操作可能是不一样的

示例:

# jcmd 20399 help GC.class_stats
20399:
GC.class_stats
Provide statistics about Java class meta data. Requires -XX:+UnlockDiagnosticVMOptions.

Impact: High: Depends on Java heap size and content.

Syntax : GC.class_stats [options] [<columns>]

Arguments:
        columns : [optional] Comma-separated list of all the columns to show. If not specified, the following columns are shown: InstBytes,KlassBytes,CpAll,annotations,MethodCount,Bytecodes,MethodAll,ROAll,RWAll,Total (STRING, no default value)

Options: (options must be specified using the <key> or <key>=<value> syntax)
        -all : [optional] Show all columns (BOOLEAN, false)
        -csv : [optional] Print in CSV (comma-separated values) format for spreadsheets (BOOLEAN, false)
        -help : [optional] Show meaning of all the columns (BOOLEAN, false)

2.3 查看进程/-l

命令: jcmd -l

作用:查看 当前机器上所有的 jvm 进程信息。

同类作用的命令:jcmd、 jcmd -l 、 jps -lm 这三个命令作用相同

示例:

C:\Users\{USER}\Desktop>jps -lm
10036 org.jetbrains.idea.maven.server.RemoteMavenServer36
17124 cn.johnnyzen.bd.dataservice.biz.DataServiceBizApplication -Xlint:unchecked
21556
26260 org/netbeans/Main --branding visualvm --cachedir C:\Users\{USER}\AppData\Local\VisualVM\Cache/2.1.5
15768 org.jetbrains.jps.cmdline.Launcher D:/Program/IDEA/IDEA_COMMUNITY_2023.2/plugins/java/lib/jps-builders.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/plugins/java/lib/jps-builders-6.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/plugins/java/lib/jps-javac-extension.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/lib/util-8.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/lib/util_rt.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/lib/platform-loader.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/lib/annotations.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/lib/protobuf.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/lib/jps-model.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/plugins/java/lib/javac2.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/lib/forms_rt.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/plugins/java/lib/aether-dependency-resolver.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/plugins/java/lib/maven-resolver-connector-basic.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/plugins/java/lib/maven-resolver-transport-file.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/pl
25528 org.jetbrains.idea.maven.server.RemoteMavenServer36
3128 sun.tools.jps.Jps -lm
9948 org.jetbrains.jps.cmdline.Launcher D:/Program/IDEA/IDEA_COMMUNITY_2023.2/plugins/java/lib/jps-builders.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/plugins/java/lib/jps-builders-6.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/plugins/java/lib/jps-javac-extension.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/lib/util-8.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/lib/util_rt.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/lib/platform-loader.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/lib/annotations.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/lib/protobuf.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/lib/jps-model.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/plugins/java/lib/javac2.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/lib/forms_rt.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/plugins/java/lib/aether-dependency-resolver.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/plugins/java/lib/maven-resolver-connector-basic.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/plugins/java/lib/maven-resolver-transport-file.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/pl

2.4 查看内存信息/GC.heap_info

命令:jcmd <pid> GC.heap_info
作用:查看JVM内存信息,虽然名称为heap_info,但是除了堆内存信息,也会有堆外内存之一的Metaspace的信息。相比jstat命令结果会更直观一些。

jcmd 17124 GC.heap_info
17124:
 PSYoungGen      total 133120K, used 92923K [0x000000076c000000, 0x0000000775f80000, 0x00000007c0000000)
  eden space 121344K, 67% used [0x000000076c000000,0x0000000770fb2c18,0x0000000773680000)
  from space 11776K, 96% used [0x0000000773800000,0x000000077430c1a0,0x0000000774380000)
  to   space 20992K, 0% used [0x0000000774b00000,0x0000000774b00000,0x0000000775f80000)
 ParOldGen       total 394752K, used 101212K [0x00000006c4000000, 0x00000006dc180000, 0x000000076c000000)
  object space 394752K, 25% used [0x00000006c4000000,0x00000006ca2d7120,0x00000006dc180000)
 Metaspace       used 110438K, capacity 115822K, committed 115840K, reserved 1150976K
  class space    used 13288K, capacity 14196K, committed 14208K, reserved 1048576K

2.5 查看性能统计/PerfCounter.print

命令:jcmd <pid> PerfCounter.print
作用:查看指定进程的性能统计信息。

示例:

# jcmd 17124 PerfCounter.print
jcmd 17124 PerfCounter.print
17124:
java.ci.totalTime=521327257
java.cls.loadedClasses=20009
java.cls.sharedLoadedClasses=0
java.cls.sharedUnloadedClasses=0
java.cls.unloadedClasses=167
java.property.java.class.path="C:\Users\{USER}\AppData\Local\Temp\classpath852656560.jar;D:\Program\IDEA\IDEA_COMMUNITY_2023.2\lib\idea_rt.jar"
java.property.java.endorsed.dirs="D:\Program\Java\jdk1.8.0_261\jre\lib\endorsed"
java.property.java.ext.dirs="D:\Program\Java\jdk1.8.0_261\jre\lib\ext;C:\Windows\Sun\Java\lib\ext"
java.property.java.home="D:\Program\Java\jdk1.8.0_261\jre"
java.property.java.library.path="D:\Program\Java\jdk1.8.0_261\bin;C:\Windows\Sun\Java\bin;C:\Windows\system32;C:\Windows;c:\Users\{USER}\AppData\Local\Programs\Cursor\resources\app\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\D:\Program\Java\jdk1.8.0_261\bin;D:\Program\Java\jdk1.8.0_261\jre\bin;D:\Program\git\cmd;D:\Program\IDEA\IDEA 2021.3.1\IntelliJ IDEA 2021.3.1\bin;;D:\Program\maven\apache-maven-3.8.1\bin;D:\Program\gradle\gradle-6.8\bin"
java.property.java.version="1.8.0_261"
java.property.java.vm.info="mixed mode"
java.property.java.vm.name="Java HotSpot(TM) 64-Bit Server VM"
java.property.java.vm.specification.name="Java Virtual Machine Specification"
java.property.java.vm.specification.vendor="Oracle Corporation"
java.property.java.vm.specification.version="1.8"
java.property.java.vm.vendor="Oracle Corporation"
java.property.java.vm.version="25.261-b12"
java.rt.vmArgs="-javaagent:D:\Program\IDEA\IDEA_COMMUNITY_2023.2\lib\idea_rt.jar=61512:D:\Program\IDEA\IDEA_COMMUNITY_2023.2\bin -Dfile.encoding=UTF-8"
java.rt.vmFlags=""
java.threads.daemon=92
java.threads.live=97
java.threads.livePeak=124
java.threads.started=9094
sun.ci.compilerThread.0.compiles=1068
sun.ci.compilerThread.0.method=""
sun.ci.compilerThread.0.time=3687
sun.ci.compilerThread.0.type=1
sun.ci.compilerThread.1.compiles=1018
sun.ci.compilerThread.1.method=""
sun.ci.compilerThread.1.time=3163
sun.ci.compilerThread.1.type=1
sun.ci.compilerThread.2.compiles=1020
sun.ci.compilerThread.2.method=""
sun.ci.compilerThread.2.time=3265
sun.ci.compilerThread.2.type=1
sun.ci.compilerThread.3.compiles=16541
sun.ci.compilerThread.3.method=""
sun.ci.compilerThread.3.time=24828
sun.ci.compilerThread.3.type=1
sun.ci.lastFailedMethod="com/google/common/collect/AbstractIterator hasNext"
sun.ci.lastFailedType=1
sun.ci.lastInvalidatedMethod=""
sun.ci.lastInvalidatedType=0
sun.ci.lastMethod="sun/rmi/server/MarshalInputStream done"
sun.ci.lastSize=160
sun.ci.lastType=1
sun.ci.nmethodCodeSize=37567648
sun.ci.nmethodSize=67456872
sun.ci.osrBytes=66982
sun.ci.osrCompiles=98
sun.ci.osrTime=11880286
sun.ci.standardBytes=3590593
sun.ci.standardCompiles=19547
sun.ci.standardTime=509446971
sun.ci.threads=4
sun.ci.totalBailouts=2
sun.ci.totalCompiles=19645
sun.ci.totalInvalidates=0
sun.classloader.findClassTime=12868824700
sun.classloader.findClasses=13619
sun.classloader.parentDelegationTime=366088100
sun.cls.appClassBytes=61762900
sun.cls.appClassLoadCount=16207
sun.cls.appClassLoadTime=121446248
sun.cls.appClassLoadTime.self=84483172
sun.cls.classInitTime=57661686
sun.cls.classInitTime.self=25100737
sun.cls.classLinkedTime=45888862
sun.cls.classLinkedTime.self=6373789
sun.cls.classVerifyTime=39505373
sun.cls.classVerifyTime.self=8706264
sun.cls.defineAppClassTime=33636674
sun.cls.defineAppClassTime.self=1188147
sun.cls.defineAppClasses=14312
sun.cls.initializedClasses=14428
sun.cls.isUnsyncloadClassSet=0
sun.cls.jniDefineClassNoLockCalls=189
sun.cls.jvmDefineClassNoLockCalls=14244
sun.cls.jvmFindLoadedClassNoLockCalls=33235
sun.cls.linkedClasses=17048
sun.cls.loadInstanceClassFailRate=0
sun.cls.loadedBytes=37403400
sun.cls.lookupSysClassTime=1249792
sun.cls.methodBytes=16981693
sun.cls.nonSystemLoaderLockContentionRate=13
sun.cls.parseClassTime=35935806
sun.cls.parseClassTime.self=24288804
sun.cls.sharedClassLoadTime=6571
sun.cls.sharedLoadedBytes=0
sun.cls.sharedUnloadedBytes=0
sun.cls.sysClassBytes=14153775
sun.cls.sysClassLoadTime=4822439
sun.cls.systemLoaderLockContentionRate=0
sun.cls.time=166118829
sun.cls.unloadedBytes=158560
sun.cls.unsafeDefineClassCalls=2138
sun.cls.verifiedClasses=16250
sun.gc.cause="No GC"
sun.gc.collector.0.invocations=72
sun.gc.collector.0.lastEntryTime=110680393836
sun.gc.collector.0.lastExitTime=110680464382
sun.gc.collector.0.name="PSScavenge"
sun.gc.collector.0.time=7267000
sun.gc.collector.1.invocations=4
sun.gc.collector.1.lastEntryTime=3181108754
sun.gc.collector.1.lastExitTime=3182109948
sun.gc.collector.1.name="PSParallelCompact"
sun.gc.collector.1.time=3125050
sun.gc.compressedclassspace.capacity=14548992
sun.gc.compressedclassspace.maxCapacity=1073741824
sun.gc.compressedclassspace.minCapacity=0
sun.gc.compressedclassspace.used=13607704
sun.gc.generation.0.capacity=165675008
sun.gc.generation.0.maxCapacity=1409286144
sun.gc.generation.0.minCapacity=88080384
sun.gc.generation.0.name="new"
sun.gc.generation.0.space.0.capacity=122683392
sun.gc.generation.0.space.0.initCapacity=0
sun.gc.generation.0.space.0.maxCapacity=1408237568
sun.gc.generation.0.space.0.name="eden"
sun.gc.generation.0.space.0.used=85759736
sun.gc.generation.0.space.1.capacity=19922944
sun.gc.generation.0.space.1.initCapacity=0
sun.gc.generation.0.space.1.maxCapacity=469762048
sun.gc.generation.0.space.1.name="s0"
sun.gc.generation.0.space.1.used=16528880
sun.gc.generation.0.space.2.capacity=21495808
sun.gc.generation.0.space.2.initCapacity=0
sun.gc.generation.0.space.2.maxCapacity=469762048
sun.gc.generation.0.space.2.name="s1"
sun.gc.generation.0.space.2.used=0
sun.gc.generation.0.spaces=3
sun.gc.generation.1.capacity=404226048
sun.gc.generation.1.maxCapacity=2818572288
sun.gc.generation.1.minCapacity=176160768
sun.gc.generation.1.name="old"
sun.gc.generation.1.space.0.capacity=404226048
sun.gc.generation.1.space.0.initCapacity=176160768
sun.gc.generation.1.space.0.maxCapacity=2818572288
sun.gc.generation.1.space.0.name="old"
sun.gc.generation.1.space.0.used=103788976
sun.gc.generation.1.spaces=1
sun.gc.lastCause="Allocation Failure"
sun.gc.metaspace.capacity=118620160
sun.gc.metaspace.maxCapacity=1178599424
sun.gc.metaspace.minCapacity=0
sun.gc.metaspace.used=113115576
sun.gc.policy.avgBaseFootprint=268435456
sun.gc.policy.avgMajorIntervalTime=79533
sun.gc.policy.avgMajorPauseTime=77
sun.gc.policy.avgMinorIntervalTime=145939
sun.gc.policy.avgMinorPauseTime=7
sun.gc.policy.avgOldLive=27434336
sun.gc.policy.avgPretenuredPaddedAvg=0
sun.gc.policy.avgPromotedAvg=262202
sun.gc.policy.avgPromotedDev=338687
sun.gc.policy.avgPromotedPaddedAvg=1278264
sun.gc.policy.avgSurvivedAvg=13827576
sun.gc.policy.avgSurvivedDev=2526529
sun.gc.policy.avgSurvivedPaddedAvg=21407164
sun.gc.policy.avgYoungLive=13809281
sun.gc.policy.boundaryMoved=0
sun.gc.policy.changeOldGenForMajPauses=0
sun.gc.policy.changeOldGenForMinPauses=0
sun.gc.policy.changeYoungGenForMajPauses=0
sun.gc.policy.changeYoungGenForMinPauses=0
sun.gc.policy.collectors=2
sun.gc.policy.decideAtFullGc=0
sun.gc.policy.decreaseForFootprint=6
sun.gc.policy.decrementTenuringThresholdForGcCost=0
sun.gc.policy.decrementTenuringThresholdForSurvivorLimit=0
sun.gc.policy.desiredSurvivorSize=21495808
sun.gc.policy.edenSize=122683392
sun.gc.policy.freeSpace=448790528
sun.gc.policy.fullFollowsScavenge=0
sun.gc.policy.gcTimeLimitExceeded=0
sun.gc.policy.generations=3
sun.gc.policy.increaseOldGenForThroughput=0
sun.gc.policy.increaseYoungGenForThroughput=0
sun.gc.policy.incrementTenuringThresholdForGcCost=1
sun.gc.policy.liveAtLastFullGc=63134592
sun.gc.policy.liveSpace=309679072
sun.gc.policy.majorCollectionSlope=0
sun.gc.policy.majorGcCost=1
sun.gc.policy.majorPauseOldSlope=561
sun.gc.policy.majorPauseYoungSlope=678
sun.gc.policy.maxTenuringThreshold=15
sun.gc.policy.minorCollectionSlope=0
sun.gc.policy.minorGcCost=0
sun.gc.policy.minorPauseOldSlope=22
sun.gc.policy.minorPauseTime=6
sun.gc.policy.minorPauseYoungSlope=45
sun.gc.policy.mutatorCost=98
sun.gc.policy.name="ParScav:MSC"
sun.gc.policy.oldCapacity=404226048
sun.gc.policy.oldEdenSize=124256256
sun.gc.policy.oldPromoSize=326107136
sun.gc.policy.promoSize=326107136
sun.gc.policy.promoted=147600
sun.gc.policy.scavengeSkipped=0
sun.gc.policy.survived=16528880
sun.gc.policy.survivorOverflowed=0
sun.gc.policy.tenuringThreshold=15
sun.gc.policy.youngCapacity=145752064
sun.gc.tlab.alloc=14890957
sun.gc.tlab.allocThreads=71
sun.gc.tlab.fastWaste=135
sun.gc.tlab.fills=928
sun.gc.tlab.gcWaste=456634
sun.gc.tlab.maxFastWaste=53
sun.gc.tlab.maxFills=73
sun.gc.tlab.maxGcWaste=104842
sun.gc.tlab.maxSlowAlloc=25
sun.gc.tlab.maxSlowWaste=1864
sun.gc.tlab.slowAlloc=107
sun.gc.tlab.slowWaste=5379
sun.management.JMXConnectorServer.address="service:jmx:rmi://127.0.0.1/stub/rO0ABXN9AAAAAQAkamF2YXgubWFuYWdlbWVudC5yZW1vdGUucm1pLlJNSVNlcnZlcnhyABdqYXZhLmxhbmcucmVmbGVjdC5Qcm94eeEn2iDMEEPLAgABTAABaHQAJUxqYXZhL2xhbmcvcmVmbGVjdC9JbnZvY2F0aW9uSGFtZGxlcjt4cHNyAC1qYXZhLnJtaS5zZXJ2ZXIuUmVtb3RlT2JqZWN0SW52b2NhdGlvbkhhbmRsZXIAAAAAAAAAAgIAAHhyABxqYXZhLnJtaS5zZXJ2ZXIuUmVtb3RlT2JqZWN002G0kQxhMx4DAAB4cHc3AAtVbmljYXN0UmVmMgAADDEwLjMzLjM3LjIyMAAA9E/vzmcbxrNih8TAAMMAAAGMPmJZpIABAHg="
sun.os.hrt.frequency=10000000
sun.os.hrt.ticks=111202048992
sun.perfdata.majorVersion=2
sun.perfdata.minorVersion=0
sun.perfdata.overflow=0
sun.perfdata.size=65536
sun.perfdata.timestamp=84490715009
sun.perfdata.used=19312
sun.property.sun.boot.class.path="D:\Program\Java\jdk1.8.0_261\jre\lib\resources.jar;D:\Program\Java\jdk1.8.0_261\jre\lib\rt.jar;D:\Program\Java\jdk1.8.0_261\jre\lib\sunrsasign.jar;D:\Program\Java\jdk1.8.0_261\jre\lib\jsse.jar;D:\Program\Java\jdk1.8.0_261\jre\lib\jce.jar;D:\Program\Java\jdk1.8.0_261\jre\lib\charsets.jar;D:\Program\Java\jdk1.8.0_261\jre\lib\jfr.jar;D:\Program\Java\jdk1.8.0_261\jre\classes"
sun.property.sun.boot.library.path="D:\Program\Java\jdk1.8.0_261\jre\bin"
sun.rt._sync_ContendedLockAttempts=4985
sun.rt._sync_Deflations=5529
sun.rt._sync_EmptyNotifications=0
sun.rt._sync_FailedSpins=0
sun.rt._sync_FutileWakeups=171
sun.rt._sync_Inflations=5449
sun.rt._sync_MonExtant=2304
sun.rt._sync_MonInCirculation=0
sun.rt._sync_MonScavenged=0
sun.rt._sync_Notifications=16931
sun.rt._sync_Parks=19756
sun.rt._sync_PrivateA=0
sun.rt._sync_PrivateB=0
sun.rt._sync_SlowEnter=0
sun.rt._sync_SlowExit=0
sun.rt._sync_SlowNotify=0
sun.rt._sync_SlowNotifyAll=0
sun.rt._sync_SuccessfulSpins=0
sun.rt.applicationTime=111152257757
sun.rt.createVmBeginTime=1701845232912
sun.rt.createVmEndTime=1701845233497
sun.rt.internalVersion="Java HotSpot(TM) 64-Bit Server VM (25.261-b12) for windows-amd64 JRE (1.8.0_261-b12), built on Jun 18 2020 06:56:32 by "" with unknown MS VC++:1916"
sun.rt.interruptedBeforeIO=0
sun.rt.interruptedDuringIO=0
sun.rt.javaCommand="cn.johnnyzen.bd.dataservice.biz.DataServiceBizApplication -Xlint:unchecked"
sun.rt.jvmCapabilities="1100000000000000000000000000000000000000000000000000000000000000"
sun.rt.jvmVersion=419497228
sun.rt.safepointSyncTime=11526218
sun.rt.safepointTime=39011973
sun.rt.safepoints=9127
sun.rt.threadInterruptSignaled=0
sun.rt.vmInitDoneTime=1701845232994
sun.threads.vmOperationTime=15162630
sun.urlClassLoader.readClassBytesTime=1222495000
sun.zip.zipFile.openTime=409313600
sun.zip.zipFiles=1126

2.6 查看已启动时长/VM.uptime

命令:jcmd <pid> VM.uptime
作用:查看 JVM 的已启动时长。

示例:

# jcmd 17124 VM.uptime
17124:
16096.667 s

2.7 查看系统中【类】的统计信息/GC.class_histogram

命令:jcmd <pid> GC.class_histogram
作用:查看系统中类统计信息。

同类作用的命令:和jmap -histo pid的效果是一样的,可以查看每个类的实例数量和占用空间大小。

示例:

# jcmd 17124 GC.class_histogram > class_histogram.txt
17124:

 num     #instances         #bytes  class name
----------------------------------------------
   1:        128192       14107064  [C
   2:          9404        6427296  [B
   3:         25119        5243544  [Ljava.lang.Object;
   4:        103014        3296448  java.util.concurrent.ConcurrentHashMap$Node
   5:        127042        3049008  java.lang.String
   6:         20991        2324280  java.lang.Class
   7:         23881        2101528  java.lang.reflect.Method
   8:          2106        1381536  com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.internal.shaded.org.jctools.queues.MpscArrayQueue
   9:         11330        1177072  [Ljava.util.HashMap$Node;
  10:         27061        1082440  java.util.LinkedHashMap$Entry
  11:          1984        1035680  [Ljava.util.concurrent.ConcurrentHashMap$Node;
  12:         34136         819264  org.springframework.core.MethodClassKey
  13:         13969         782264  java.util.LinkedHashMap
  14:         10491         752320  [I
  15:         21977         703264  java.util.HashMap$Node
  16:         34733         555728  java.lang.Object
  17:         18802         424088  [Ljava.lang.Class;
  18:          1330         378080  [Lcom.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.Recycler$DefaultHandle;
  19:             8         262272  [Ljava.util.concurrent.ForkJoinTask;
  20:          8084         258688  java.util.Hashtable$Entry
  21:          9897         237528  java.util.ArrayList
  22:          5894         235760  java.lang.ref.SoftReference
  23:          2732         218560  java.lang.reflect.Constructor
  24:          4297         206256  java.util.HashMap
  25:          4880         204632  [Ljava.lang.String;
  26:          5528         176896  java.lang.ref.WeakReference
  27:          3624         173952  org.springframework.core.ResolvableType
  28:          2605         166720  java.util.concurrent.ConcurrentHashMap
  29:            80         136960  [Lcom.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.buffer.PoolSubpage;
  30:          2205         123480  java.lang.invoke.MemberName
  31:          2723         108920  java.util.TreeMap$Entry
  32:          3359         107488  java.util.concurrent.locks.ReentrantLock$NonfairSync
  33:           668         105336  [Ljava.util.Hashtable$Entry;
  34:          2552         102080  java.lang.ref.Finalizer
  35:           508          97536  org.springframework.context.annotation.ConfigurationClassBeanDefinitionReader$ConfigurationClassBeanDefinition
  36:          1425          91200  com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.buffer.PoolSubpage
  37:          1536          86016  java.security.Provider$Service
  38:           852          84192  [Ljava.util.WeakHashMap$Entry;
  39:          1163          83736  java.lang.reflect.Field
  40:          2059          82360  java.lang.invoke.MethodType
  41:          3370          80880  sun.reflect.generics.tree.SimpleClassTypeSignature
  42:          1443          80808  java.lang.Package
  43:          1439          80584  org.springframework.core.annotation.TypeMappedAnnotation
  44:          3330          79920  sun.reflect.annotation.AnnotationInvocationHandler
  45:          2469          79008  java.lang.ref.ReferenceQueue
  46:          3213          77112  java.util.LinkedList$Node
  47:           428          75328  org.springframework.beans.factory.support.RootBeanDefinition
  48:          2266          72512  java.util.LinkedList
  49:          3016          72384  java.security.Provider$ServiceKey
  50:          1239          69384  org.codehaus.groovy.reflection.GeneratedMetaMethod$Proxy
  51:            26          66112  [Ljava.nio.channels.SelectionKey;
  52:          2065          66080  java.lang.invoke.MethodType$ConcurrentWeakInternSet$WeakEntry
  53:           510          64048  [Z
  54:          3370          63376  [Lsun.reflect.generics.tree.TypeArgument;
  55:          1944          62208  com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.buffer.PoolThreadCache$SubPageMemoryRegionCache
  56:          1546          61840  java.util.WeakHashMap$Entry
  57:           942          60288  java.net.URL
  58:           256          59392  com.clickhouse.client.ClickHouseConfig
  59:          1162          55776  org.springframework.util.ConcurrentReferenceHashMap$SoftEntryReference
  60:          2316          55584  java.util.concurrent.ConcurrentLinkedDeque$Node
  61:           825          52800  org.codehaus.groovy.reflection.CachedMethod
  62:          3299          52784  java.util.LinkedHashSet
  63:          3256          52096  sun.reflect.generics.tree.ClassTypeSignature
  64:            48          51072  [Ljava.nio.ByteBuffer;
  65:           512          49152  sun.util.calendar.Gregorian$Date
  66:          2030          48720  java.util.concurrent.atomic.AtomicLong
  67:           867          48552  java.beans.MethodDescriptor
  68:          2034          48160  [Ljava.lang.reflect.Type;
  69:           843          47208  org.codehaus.groovy.runtime.metaclass.MetaMethodIndex$Entry
  70:          1463          46816  java.lang.invoke.DirectMethodHandle
  71:           321          46104  [Ljava.lang.ClassValue$Entry;
  72:          1915          45960  java.lang.Class$AnnotationData
  73:          1329          42528  sun.security.util.ObjectIdentifier
  74:           976          41224  [Lorg.springframework.util.ConcurrentReferenceHashMap$Reference;
  75:          1671          40104  java.util.concurrent.CopyOnWriteArrayList
  76:          2471          39536  java.lang.ref.ReferenceQueue$Lock
  77:           976          39040  org.springframework.util.ConcurrentReferenceHashMap$Segment
  78:          1656          37712  [Lorg.springframework.core.ResolvableType;
  79:           771          37008  org.codehaus.groovy.util.ReferenceType$SoftRef
  80:          1110          35520  com.mysql.cj.conf.BooleanProperty
  81:          2182          34912  java.util.concurrent.atomic.AtomicInteger
  82:           484          34848  sun.reflect.DelegatingClassLoader
  83:          2172          34752  java.lang.Integer
  84:          1047          33504  com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.Recycler$WeakOrderQueue$Link
  85:          2086          33376  java.util.concurrent.locks.ReentrantLock
  86:           817          32680  java.math.BigInteger
  87:          1355          32520  java.beans.MethodRef
  88:          1016          32512  java.util.Vector
  89:           409          31104  [J
  90:          1274          30576  org.springframework.core.convert.converter.GenericConverter$ConvertiblePair
  91:           314          30144  java.util.jar.JarFile$JarFileEntry
  92:           622          29856  java.util.concurrent.ConcurrentHashMap$TreeNode
  93:          1242          29808  org.springframework.beans.factory.annotation.InjectionMetadata
  94:          1235          29640  org.springframework.beans.BeanMetadataAttribute

....

8029:             1             16  sun.security.util.ByteArrayLexOrder
8030:             1             16  sun.security.util.ByteArrayTagOrder
8031:             1             16  sun.text.normalizer.NormalizerBase$Mode
8032:             1             16  sun.text.normalizer.NormalizerBase$NFCMode
8033:             1             16  sun.text.normalizer.NormalizerBase$NFDMode
8034:             1             16  sun.text.normalizer.NormalizerBase$NFKCMode
8035:             1             16  sun.text.normalizer.NormalizerBase$NFKDMode
8036:             1             16  sun.text.normalizer.NormalizerImpl
8037:             1             16  sun.text.normalizer.NormalizerImpl$AuxTrieImpl
8038:             1             16  sun.text.normalizer.NormalizerImpl$FCDTrieImpl
8039:             1             16  sun.text.normalizer.NormalizerImpl$NormTrieImpl
8040:             1             16  sun.util.calendar.Gregorian
8041:             1             16  sun.util.locale.provider.AuxLocaleProviderAdapter$NullProvider
8042:             1             16  sun.util.locale.provider.CalendarDataUtility$CalendarWeekParameterGetter
8043:             1             16  sun.util.locale.provider.SPILocaleProviderAdapter
8044:             1             16  sun.util.locale.provider.TimeZoneNameUtility$TimeZoneNameGetter
8045:             1             16  sun.util.resources.LocaleData
8046:             1             16  sun.util.resources.LocaleData$LocaleDataResourceBundleControl
Total        927801       56802976

2.8 查看线程堆栈信息/Thread.print

命令:jcmd <pid> Thread.print
作用:查看线程堆栈信息。

同类作用的命令:和jstack -l作用一样

jcmd 17124 Thread.print
17124:
2023-12-06 19:21:08
Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.261-b12 mixed mode):

"nacos-grpc-client-executor-3264" #13332 daemon prio=5 os_prio=0 tid=0x000001ebb9644800 nid=0x3574 waiting on condition [0x000000ea2abff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c542eae0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"nacos-grpc-client-executor-3267" #13331 daemon prio=5 os_prio=0 tid=0x000001ebb9649000 nid=0x60dc waiting on condition [0x000000ea2b0ff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c4bd2c40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"nacos-grpc-client-executor-3241" #13330 daemon prio=5 os_prio=0 tid=0x000001ebb9642800 nid=0x43e8 waiting on condition [0x000000ea2a2ff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5d40660> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"nacos-grpc-client-executor-3231" #13329 daemon prio=5 os_prio=0 tid=0x000001ebb9648800 nid=0x49b8 waiting on condition [0x000000ea289ff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5d82830> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"nacos-grpc-client-executor-3263" #13328 daemon prio=5 os_prio=0 tid=0x000001ebb9641800 nid=0x6068 waiting on condition [0x000000ea2b4ff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c542eae0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"nacos-grpc-client-executor-3266" #13327 daemon prio=5 os_prio=0 tid=0x000001ebb9641000 nid=0x1f90 waiting on condition [0x000000ea2b3ff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c4bd2c40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"nacos-grpc-client-executor-3240" #13326 daemon prio=5 os_prio=0 tid=0x000001ebb9647800 nid=0x2c8 waiting on condition [0x000000ea2b1ff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5d40660> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"nacos-grpc-client-executor-3230" #13325 daemon prio=5 os_prio=0 tid=0x000001ebb9646000 nid=0x2d9c waiting on condition [0x000000ea28aff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5d82830> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"RMI TCP Connection(26)-10.33.37.220" #13303 daemon prio=5 os_prio=0 tid=0x000001ebb9647000 nid=0x4c38 runnable [0x000000ea28bfd000]
   java.lang.Thread.State: RUNNABLE
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
        at java.net.SocketInputStream.read(SocketInputStream.java:171)
        at java.net.SocketInputStream.read(SocketInputStream.java:141)
        at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
        - locked <0x000000076fba8a70> (a java.io.BufferedInputStream)
        at java.io.FilterInputStream.read(FilterInputStream.java:83)
        at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:555)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:834)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:688)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$$Lambda$1306/1016102937.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:687)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"RMI TCP Connection(idle)" #12760 daemon prio=5 os_prio=0 tid=0x000001ebb964a800 nid=0x33e4 waiting on condition [0x000000ea2cbfe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c7181628> (a java.util.concurrent.SynchronousQueue$TransferStack)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"JMX server connection timeout 6922" #6922 daemon prio=5 os_prio=0 tid=0x000001ebb610e800 nid=0x5b68 in Object.wait() [0x000000ea2deff000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at com.sun.jmx.remote.internal.ServerCommunicatorAdmin$Timeout.run(ServerCommunicatorAdmin.java:168)
        - locked <0x00000006c7181c38> (a [I)
        at java.lang.Thread.run(Thread.java:748)

"RMI Scheduler(0)" #6921 daemon prio=5 os_prio=0 tid=0x000001ebc2c18800 nid=0x2854 waiting on condition [0x000000ea2ddff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c71856f0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"RMI TCP Accept-0" #6919 daemon prio=5 os_prio=0 tid=0x000001ebc2c1f000 nid=0x6810 runnable [0x000000ea2aefe000]
   java.lang.Thread.State: RUNNABLE
        at java.net.DualStackPlainSocketImpl.accept0(Native Method)
        at java.net.DualStackPlainSocketImpl.socketAccept(DualStackPlainSocketImpl.java:127)
        at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:535)
        at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:189)
        - locked <0x00000006c7181820> (a java.net.SocksSocketImpl)
        at java.net.ServerSocket.implAccept(ServerSocket.java:545)
        at java.net.ServerSocket.accept(ServerSocket.java:513)
        at sun.management.jmxremote.LocalRMIServerSocketFactory$1.accept(LocalRMIServerSocketFactory.java:52)
        at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:405)
        at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:377)
        at java.lang.Thread.run(Thread.java:748)

"grpc-nio-worker-ELG-1-16" #4585 daemon prio=5 os_prio=0 tid=0x000001ebba96f800 nid=0x4df0 runnable [0x000000ea2feff000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c4c3aee0> (a com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x00000006c4c3b008> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c4c3aef8> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:806)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

"grpc-nio-worker-ELG-1-15" #4584 daemon prio=5 os_prio=0 tid=0x000001ebba96f000 nid=0x167c runnable [0x000000ea2fdfe000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c4c20bb0> (a com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x00000006c4c20cd8> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c4c20bc8> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:806)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

"grpc-nio-worker-ELG-1-14" #4580 daemon prio=5 os_prio=0 tid=0x000001ebba96e000 nid=0x4200 runnable [0x000000ea2f9ff000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c4c129d0> (a com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x00000006c4c1efa8> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c4c1ee98> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:806)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

"grpc-nio-worker-ELG-1-13" #4579 daemon prio=5 os_prio=0 tid=0x000001ebba966800 nid=0x6814 runnable [0x000000ea2f8fe000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c4c51e98> (a com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x00000006c4c51fc0> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c4c51eb0> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:806)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

"grpc-nio-worker-ELG-1-12" #4578 daemon prio=5 os_prio=0 tid=0x000001ebba966000 nid=0x40d4 runnable [0x000000ea2f7ff000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c4c7a490> (a com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x00000006c4c7a5b8> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c4c7a4a8> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:806)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

"grpc-nio-worker-ELG-1-11" #4577 daemon prio=5 os_prio=0 tid=0x000001ebba835800 nid=0x60e4 runnable [0x000000ea2f6fe000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c4c9c0b0> (a com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x00000006c4c9c1d8> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c4c9c0c8> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:806)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

"grpc-nio-worker-ELG-1-10" #4575 daemon prio=5 os_prio=0 tid=0x000001ebba834000 nid=0x1f1c runnable [0x000000ea2edff000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c4cb5110> (a com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x00000006c4cb5238> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c4cb5128> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:806)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

"grpc-nio-worker-ELG-1-9" #4574 daemon prio=5 os_prio=0 tid=0x000001ebba83a000 nid=0x523c runnable [0x000000ea2ecfe000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c4c9cdb8> (a com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x00000006c4cb3518> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c4cb3408> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:806)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.naming.updater.3" #247 daemon prio=5 os_prio=0 tid=0x000001ebc2c20000 nid=0x5c60 waiting on condition [0x000000ea2beff000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c57310f8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.naming.updater.2" #198 daemon prio=5 os_prio=0 tid=0x000001ebc2c15800 nid=0xadc waiting on condition [0x000000ea2ccfe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c57310f8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #168 daemon prio=5 os_prio=0 tid=0x000001ebba839000 nid=0x6a20 waiting on condition [0x000000ea2cdff000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5d24e38> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #166 daemon prio=5 os_prio=0 tid=0x000001ebba838800 nid=0x68d4 waiting on condition [0x000000ea2bcfe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5ca7628> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #162 daemon prio=5 os_prio=0 tid=0x000001ebba83a800 nid=0x672c waiting on condition [0x000000ea2e9fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5d24e38> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #160 daemon prio=5 os_prio=0 tid=0x000001ebba82d000 nid=0x5598 waiting on condition [0x000000ea2dfff000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5ca7628> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #156 daemon prio=5 os_prio=0 tid=0x000001ebba833000 nid=0x587c waiting on condition [0x000000ea2ceff000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5d24e38> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #153 daemon prio=5 os_prio=0 tid=0x000001ebba830000 nid=0xc7c waiting on condition [0x000000ea2affe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5ca7628> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #149 daemon prio=5 os_prio=0 tid=0x000001ebba837800 nid=0x2ac4 waiting on condition [0x000000ea2c2ff000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5d24e38> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #147 daemon prio=5 os_prio=0 tid=0x000001ebba83b800 nid=0x6718 waiting on condition [0x000000ea2b2fe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5ca7628> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #145 daemon prio=5 os_prio=0 tid=0x000001ebba82f800 nid=0x386c waiting on condition [0x000000ea30bff000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5d24e38> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #144 daemon prio=5 os_prio=0 tid=0x000001ebba82e800 nid=0x3990 waiting on condition [0x000000ea30aff000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5ca7628> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"DestroyJavaVM" #142 prio=5 os_prio=0 tid=0x000001ebba834800 nid=0x2e10 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"grpc-nio-worker-ELG-1-8" #120 daemon prio=5 os_prio=0 tid=0x000001ebc2c17800 nid=0x54bc runnable [0x000000ea2f4fe000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c4cca7b8> (a com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x00000006c4cca8e0> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c4cca7d0> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:806)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

"grpc-nio-worker-ELG-1-7" #119 daemon prio=5 os_prio=0 tid=0x000001ebc2c1c000 nid=0x67f0 runnable [0x000000ea2f3fe000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c4ce8150> (a com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x00000006c4ce8278> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c4ce8168> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:806)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.remote.worker" #118 daemon prio=5 os_prio=0 tid=0x000001ebc2c14800 nid=0x6bec waiting on condition [0x000000ea2f2fe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5ef0608> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:418)
        at com.alibaba.nacos.common.remote.client.RpcClient$3.run(RpcClient.java:317)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.remote.worker" #117 daemon prio=5 os_prio=0 tid=0x000001ebc2c1b800 nid=0x2010 waiting on condition [0x000000ea2f1fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5e6b490> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at com.alibaba.nacos.common.remote.client.RpcClient$2.run(RpcClient.java:295)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #116 daemon prio=5 os_prio=0 tid=0x000001ebc2c1a800 nid=0x738 waiting on condition [0x000000ea2f0fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5d24e38> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #115 daemon prio=5 os_prio=0 tid=0x000001ebc2c14000 nid=0x6cf8 waiting on condition [0x000000ea2efff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5d24e38> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #114 daemon prio=5 os_prio=0 tid=0x000001ebc2c13000 nid=0x36a8 waiting on condition [0x000000ea2eefe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5e4b6a8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:418)
        at com.alibaba.nacos.client.config.impl.ClientWorker$ConfigRpcTransportClient$4.run(ClientWorker.java:744)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #109 daemon prio=5 os_prio=0 tid=0x000001ebb4a0c000 nid=0x656c waiting on condition [0x000000ea2e8ff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c464ea40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"grpc-nio-worker-ELG-1-6" #108 daemon prio=5 os_prio=0 tid=0x000001ebb4a0a000 nid=0x7258 runnable [0x000000ea2e7fe000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c4d09a48> (a com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x00000006c4d09b70> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c4d09a60> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:806)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

"grpc-nio-worker-ELG-1-5" #107 daemon prio=5 os_prio=0 tid=0x000001ebb4a08800 nid=0x56fc runnable [0x000000ea2e6ff000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c4d202f8> (a com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x00000006c4d20420> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c4d20310> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:806)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.remote.worker" #106 daemon prio=5 os_prio=0 tid=0x000001ebb4a09000 nid=0x4898 waiting on condition [0x000000ea2e5fe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5d5eae8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:418)
        at com.alibaba.nacos.common.remote.client.RpcClient$3.run(RpcClient.java:317)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.remote.worker" #105 daemon prio=5 os_prio=0 tid=0x000001ebb4a0b800 nid=0x5610 waiting on condition [0x000000ea2e4ff000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5ca8d68> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at com.alibaba.nacos.common.remote.client.RpcClient$2.run(RpcClient.java:295)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #104 daemon prio=5 os_prio=0 tid=0x000001ebb4a07800 nid=0x54e8 waiting on condition [0x000000ea2e3fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5ca7628> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #103 daemon prio=5 os_prio=0 tid=0x000001ebb4a0a800 nid=0x1818 waiting on condition [0x000000ea2e2fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5ca7628> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #102 daemon prio=5 os_prio=0 tid=0x000001ebb4a07000 nid=0x588c waiting on condition [0x000000ea2e1ff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5ca86b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:418)
        at com.alibaba.nacos.client.config.impl.ClientWorker$ConfigRpcTransportClient$4.run(ClientWorker.java:744)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.naming.updater.1" #101 daemon prio=5 os_prio=0 tid=0x000001ebb49ff800 nid=0x5d5c waiting on condition [0x000000ea2e0fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c57310f8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"http-nio-9527-Acceptor" #97 daemon prio=5 os_prio=0 tid=0x000001ebb49fd000 nid=0x4ce0 runnable [0x000000ea2dcfe000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:424)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:252)
        - locked <0x00000006c628af98> (a java.lang.Object)
        at org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:574)
        at org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:80)
        at org.apache.tomcat.util.net.Acceptor.run(Acceptor.java:106)
        at java.lang.Thread.run(Thread.java:748)

"http-nio-9527-ClientPoller" #96 daemon prio=5 os_prio=0 tid=0x000001ebb4a04000 nid=0xc40 runnable [0x000000ea2dbfe000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c62dc740> (a sun.nio.ch.Util$3)
        - locked <0x00000006c62dc730> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c62dc6b0> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:816)
        at java.lang.Thread.run(Thread.java:748)

"http-nio-9527-exec-10" #95 daemon prio=5 os_prio=0 tid=0x000001ebb4a03000 nid=0x3a10 waiting on condition [0x000000ea2dafe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c62be420> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:108)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

"http-nio-9527-exec-9" #94 daemon prio=5 os_prio=0 tid=0x000001ebb49fe800 nid=0x62d8 waiting on condition [0x000000ea2d9fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c62be420> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:108)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

"http-nio-9527-exec-8" #93 daemon prio=5 os_prio=0 tid=0x000001ebb4a01800 nid=0x37ec waiting on condition [0x000000ea2d8fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c62be420> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:108)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

"http-nio-9527-exec-7" #92 daemon prio=5 os_prio=0 tid=0x000001ebb4a02800 nid=0x5328 waiting on condition [0x000000ea2d7fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c62be420> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:108)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

"http-nio-9527-exec-6" #91 daemon prio=5 os_prio=0 tid=0x000001ebb4a01000 nid=0x36a0 waiting on condition [0x000000ea2d6ff000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c62be420> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:108)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

"http-nio-9527-exec-5" #90 daemon prio=5 os_prio=0 tid=0x000001ebb4a00000 nid=0x3684 waiting on condition [0x000000ea2d5fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c62be420> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:108)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

"http-nio-9527-exec-4" #89 daemon prio=5 os_prio=0 tid=0x000001ebba972000 nid=0x5f10 waiting on condition [0x000000ea2d4fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c62be420> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:108)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

"http-nio-9527-exec-3" #88 daemon prio=5 os_prio=0 tid=0x000001ebb953b800 nid=0x590 waiting on condition [0x000000ea2d3fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c62be420> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:108)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

"http-nio-9527-exec-2" #87 daemon prio=5 os_prio=0 tid=0x000001ebbf778800 nid=0x5ca0 waiting on condition [0x000000ea2d2fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c62be420> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:108)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

"http-nio-9527-exec-1" #86 daemon prio=5 os_prio=0 tid=0x000001ebbf777000 nid=0x52d0 waiting on condition [0x000000ea2d1fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c62be420> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:108)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

"http-nio-9527-BlockPoller" #85 daemon prio=5 os_prio=0 tid=0x000001ebbf777800 nid=0x3ef8 runnable [0x000000ea2d0ff000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c63718e8> (a sun.nio.ch.Util$3)
        - locked <0x00000006c63718d8> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c6371778> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at org.apache.tomcat.util.net.NioBlockingSelector$BlockPoller.run(NioBlockingSelector.java:331)

"Nacos-Watch-Task-Scheduler-1" #84 prio=5 os_prio=0 tid=0x000001ebbf776000 nid=0x4620 waiting on condition [0x000000ea2cfff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5d7ac40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.naming.updater.0" #81 daemon prio=5 os_prio=0 tid=0x000001ebbf76d000 nid=0x69ec waiting on condition [0x000000ea2c0fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c57310f8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"grpc-nio-worker-ELG-1-4" #77 daemon prio=5 os_prio=0 tid=0x000001ebbf772800 nid=0x7070 runnable [0x000000ea2cafe000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c4d21000> (a com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x00000006c4d308b0> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c4d307a0> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:806)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

"grpc-nio-worker-ELG-1-3" #76 daemon prio=5 os_prio=0 tid=0x000001ebbf771000 nid=0x4c50 runnable [0x000000ea2c9fe000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c4d324a8> (a com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x00000006c4d325d0> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c4d324c0> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:806)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.remote.worker" #75 daemon prio=5 os_prio=0 tid=0x000001ebbf771800 nid=0x1f60 waiting on condition [0x000000ea2c8ff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5410fe8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:418)
        at com.alibaba.nacos.common.remote.client.RpcClient$3.run(RpcClient.java:317)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.remote.worker" #74 daemon prio=5 os_prio=0 tid=0x000001ebbf769800 nid=0xb6c waiting on condition [0x000000ea2c7fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c54109a8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at com.alibaba.nacos.common.remote.client.RpcClient$2.run(RpcClient.java:295)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.naming.grpc.redo.0" #73 daemon prio=5 os_prio=0 tid=0x000001ebbf76b000 nid=0x5028 waiting on condition [0x000000ea2c6fe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5541040> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.naming.push.receiver" #72 daemon prio=5 os_prio=0 tid=0x000001ebbf76a000 nid=0x210c runnable [0x000000ea2c5fe000]
   java.lang.Thread.State: RUNNABLE
        at java.net.DualStackPlainDatagramSocketImpl.socketReceiveOrPeekData(Native Method)
        at java.net.DualStackPlainDatagramSocketImpl.receive0(DualStackPlainDatagramSocketImpl.java:124)
        - locked <0x00000006c565eb48> (a java.net.DualStackPlainDatagramSocketImpl)
        at java.net.AbstractPlainDatagramSocketImpl.receive(AbstractPlainDatagramSocketImpl.java:149)
        - locked <0x00000006c565eb48> (a java.net.DualStackPlainDatagramSocketImpl)
        at java.net.DatagramSocket.receive(DatagramSocket.java:812)
        - locked <0x00000006c588d000> (a java.net.DatagramPacket)
        - locked <0x00000006c565eb08> (a java.net.DatagramSocket)
        at com.alibaba.nacos.client.naming.core.PushReceiver.run(PushReceiver.java:102)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.naming.security" #71 daemon prio=5 os_prio=0 tid=0x000001ebbf76e000 nid=0x6c14 waiting on condition [0x000000ea2c4fe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c575e630> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.naming.failover" #66 daemon prio=5 os_prio=0 tid=0x000001ebba968000 nid=0x4da0 waiting on condition [0x000000ea2c1ff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5547b68> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"nacos.publisher-com.alibaba.nacos.client.naming.event.InstancesChangeEvent" #65 daemon prio=5 os_prio=0 tid=0x000001ebb9539000 nid=0x4804 waiting on condition [0x000000ea2b7ff000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c59332d8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:403)
        at com.alibaba.nacos.common.notify.DefaultPublisher.openEventHandler(DefaultPublisher.java:117)
        at com.alibaba.nacos.common.notify.DefaultPublisher.run(DefaultPublisher.java:95)

"spring.cloud.inetutils" #64 daemon prio=5 os_prio=0 tid=0x000001ebb9538800 nid=0x32ec waiting on condition [0x000000ea2b6fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5731440> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #63 daemon prio=5 os_prio=0 tid=0x000001ebb953d000 nid=0x3220 waiting on condition [0x000000ea2b5fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c464ea40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #55 daemon prio=5 os_prio=0 tid=0x000001ebba96d800 nid=0x2d14 waiting on condition [0x000000ea2adfe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c464ea40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"HikariPool-1 housekeeper" #53 daemon prio=5 os_prio=0 tid=0x000001ebba971000 nid=0x3d24 waiting on condition [0x000000ea2bfff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5541a58> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #51 daemon prio=5 os_prio=0 tid=0x000001ebba970800 nid=0x6728 waiting on condition [0x000000ea2bdfe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c464ea40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"mysql-cj-abandoned-connection-cleanup" #49 daemon prio=5 os_prio=0 tid=0x000001ebba96c800 nid=0xee4 in Object.wait() [0x000000ea2bbff000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144)
        - locked <0x00000006c5856018> (a java.lang.ref.ReferenceQueue$Lock)
        at com.mysql.cj.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:91)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"container-0" #47 prio=5 os_prio=0 tid=0x000001ebba96c000 nid=0x24f4 waiting on condition [0x000000ea2bafe000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
        at java.lang.Thread.sleep(Native Method)
        at org.apache.catalina.core.StandardServer.await(StandardServer.java:570)
        at org.springframework.boot.web.embedded.tomcat.TomcatWebServer$1.run(TomcatWebServer.java:197)

"Catalina-utility-2" #46 prio=1 os_prio=-2 tid=0x000001ebba964800 nid=0x6fcc waiting on condition [0x000000ea2b9ff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c560ba68> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

"Catalina-utility-1" #45 prio=1 os_prio=-2 tid=0x000001ebba965000 nid=0x575c waiting on condition [0x000000ea2a3ff000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c560ba68> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #44 daemon prio=5 os_prio=0 tid=0x000001ebba96b000 nid=0xcb8 waiting on condition [0x000000ea2b8fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c464ea40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"grpc-nio-worker-ELG-1-2" #30 daemon prio=5 os_prio=0 tid=0x000001ebb953b000 nid=0x3ab8 runnable [0x000000ea2acfe000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c4c8e848> (a com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x00000006c4c8e860> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c4c8e7c8> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:806)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

"grpc-nio-worker-ELG-1-1" #28 daemon prio=5 os_prio=0 tid=0x000001ebb8a02000 nid=0xb80 runnable [0x000000ea2aafe000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c4cbd928> (a com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x00000006c4cbd980> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c4cbd858> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:806)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.remote.worker" #27 daemon prio=5 os_prio=0 tid=0x000001ebb6eb3000 nid=0x2a88 waiting on condition [0x000000ea2a9fe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c4ce3b70> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:418)
        at com.alibaba.nacos.common.remote.client.RpcClient$3.run(RpcClient.java:317)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.remote.worker" #26 daemon prio=5 os_prio=0 tid=0x000001ebb5a0d800 nid=0x58dc waiting on condition [0x000000ea2a8fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c4ce34e0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at com.alibaba.nacos.common.remote.client.RpcClient$2.run(RpcClient.java:295)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"nacos.publisher-com.alibaba.nacos.common.notify.SlowEvent" #24 daemon prio=5 os_prio=0 tid=0x000001ebba958800 nid=0x44fc waiting on condition [0x000000ea2a7ff000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c4d04cf0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:403)
        at com.alibaba.nacos.common.notify.DefaultPublisher.openEventHandler(DefaultPublisher.java:117)
        at com.alibaba.nacos.common.notify.DefaultPublisher.run(DefaultPublisher.java:95)

"com.alibaba.nacos.client.Worker" #22 daemon prio=5 os_prio=0 tid=0x000001ebbaa06800 nid=0x46ec waiting on condition [0x000000ea2a6ff000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c464ea40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #21 daemon prio=5 os_prio=0 tid=0x000001ebb7246000 nid=0x5d74 waiting on condition [0x000000ea2a5fe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c464eed0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:418)
        at com.alibaba.nacos.client.config.impl.ClientWorker$ConfigRpcTransportClient$4.run(ClientWorker.java:744)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #20 daemon prio=5 os_prio=0 tid=0x000001ebb7245800 nid=0x4c58 waiting on condition [0x000000ea2a4fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c464ea40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.identify.watcher.0" #17 daemon prio=5 os_prio=0 tid=0x000001ebba827800 nid=0x2694 waiting on condition [0x000000ea2a1ff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c464e068> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"Service Thread" #11 daemon prio=9 os_prio=0 tid=0x000001ebb586a000 nid=0x6748 runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C1 CompilerThread3" #10 daemon prio=9 os_prio=2 tid=0x000001ebb57cb800 nid=0x5f04 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread2" #9 daemon prio=9 os_prio=2 tid=0x000001ebb57ca000 nid=0x5dfc waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread1" #8 daemon prio=9 os_prio=2 tid=0x000001ebb57c4800 nid=0x27ec waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" #7 daemon prio=9 os_prio=2 tid=0x000001ebb57c3800 nid=0x4fa4 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Monitor Ctrl-Break" #6 daemon prio=5 os_prio=0 tid=0x000001ebb57c0000 nid=0x44e4 runnable [0x000000ea29aff000]
   java.lang.Thread.State: RUNNABLE
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
        at java.net.SocketInputStream.read(SocketInputStream.java:171)
        at java.net.SocketInputStream.read(SocketInputStream.java:141)
        at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
        at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
        at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
        - locked <0x00000006c402f1e0> (a java.io.InputStreamReader)
        at java.io.InputStreamReader.read(InputStreamReader.java:184)
        at java.io.BufferedReader.fill(BufferedReader.java:161)
        at java.io.BufferedReader.readLine(BufferedReader.java:324)
        - locked <0x00000006c402f1e0> (a java.io.InputStreamReader)
        at java.io.BufferedReader.readLine(BufferedReader.java:389)
        at com.intellij.rt.execution.application.AppMainV2$1.run(AppMainV2.java:53)

"Attach Listener" #5 daemon prio=5 os_prio=2 tid=0x000001ebb2bda800 nid=0x1c3c waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" #4 daemon prio=9 os_prio=2 tid=0x000001ebb2bc4000 nid=0x1b64 runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Finalizer" #3 daemon prio=8 os_prio=1 tid=0x000001ebb2bad000 nid=0x71ac in Object.wait() [0x000000ea297ff000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144)
        - locked <0x00000006c403f3c8> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165)
        at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:216)

"Reference Handler" #2 daemon prio=10 os_prio=2 tid=0x000001ebb2ba5800 nid=0x1264 in Object.wait() [0x000000ea28dff000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:502)
        at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
        - locked <0x00000006c40308f0> (a java.lang.ref.Reference$Lock)
        at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)

"VM Thread" os_prio=2 tid=0x000001ebb2b78800 nid=0x718 runnable

"GC task thread#0 (ParallelGC)" os_prio=0 tid=0x000001eb97e2b000 nid=0x2f74 runnable

"GC task thread#1 (ParallelGC)" os_prio=0 tid=0x000001eb97e2e000 nid=0x6660 runnable

"GC task thread#2 (ParallelGC)" os_prio=0 tid=0x000001eb97e31800 nid=0x780 runnable

"GC task thread#3 (ParallelGC)" os_prio=0 tid=0x000001eb97e34000 nid=0x2fb4 runnable

"GC task thread#4 (ParallelGC)" os_prio=0 tid=0x000001eb97e35000 nid=0x4c24 runnable

"GC task thread#5 (ParallelGC)" os_prio=0 tid=0x000001eb97e3d000 nid=0x67c8 runnable

"GC task thread#6 (ParallelGC)" os_prio=0 tid=0x000001eb97e3b800 nid=0x3218 runnable

"GC task thread#7 (ParallelGC)" os_prio=0 tid=0x000001eb97e39000 nid=0xf78 runnable

"VM Periodic Task Thread" os_prio=2 tid=0x000001ebb5938000 nid=0x21e8 waiting on condition

JNI global references: 2220

2.9 导出内存Dump文件/GC.heap_dump

命令:jcmd <pid> GC.heap_dump FILE_NAME
作用:查看 JVM 的Heap Dump。导出的 dump 文件,可以使用MAT 或者 Visual VM 等工具进行分析(如果只指定文件名,默认会生成在启动 JVM 的目录里)

同类作用的命令:和 jmap -dump:format=b,file=heapdump.phrof pid 作用一样

示例:

# jcmd 17124 GC.heap_dump ./dump.hprof 
17124: Heap dump file created

2.10 查看JVM系统属性/VM.system_properties

命令:jcmd <pid> VM.system_properties
作用:查看 JVM 的属性信息。

同类作用的命令:和jinfo -sysprops pid作用一样。jinfo pid会输出 VM.system_properties 和 VM.flags,命令短,更好记住,所以推荐直接使用jinfo pid

示例:

# jcmd 17124 VM.system_properties
17124:
#Sun Nov 14 12:39:36 CST 2021
java.runtime.name=OpenJDK Runtime Environment
java.protocol.handler.pkgs=org.springframework.boot.loader
sun.boot.library.path=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.282.b08-1.el7_9.x86_64/jre/lib/amd64
java.vm.version=25.282-b08
java.vm.vendor=Red Hat, Inc.
java.vendor.url=https\://www.redhat.com/
.......省略.......
sun.io.unicode.encoding=UnicodeLittle
sun.cpu.endian=little
sun.cpu.isalist=

2.11 查看JVM启动参数/VM.flags

命令:jcmd <pid> VM.flags
作用:查看 JVM 的启动参数。

同类作用的命令:和jinfo -flags {pid}作用一样。jinfo pid会输出 VM.system_properties 和 VM.flags。

示例:

# jcmd 17124 VM.flags
17124:
-XX:CICompilerCount=4 -XX:InitialHeapSize=264241152 -XX:MaxHeapSize=4227858432 -XX:MaxNewSize=1409286144 -XX:MinHeapDeltaBytes=524288 -XX:NewSize=88080384 -XX:OldSize=176160768 -XX:+UnlockCommercialFeatures -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseFastUnorderedTimeStamps -XX:-UseLargePagesIndividualAllocation -XX:+UseParallelGC

2.12 查看JVM启动命令行/VM.command_line

命令:jcmd <pid> VM.command_line
作用:查看 JVM 的启动命令行。

示例:

# jcmd 17124 VM.command_line
17124:
VM Arguments:
jvm_args: -Xmx512m 
java_command: /etc/java-app/springtest-0.0.1-SNAPSHOT.jar
java_class_path (initial): /etc/java-app/springtest-0.0.1-SNAPSHOT.jar
Launcher Type: SUN_STANDARD

2.13 执行finalization操作/GC.run_finalization

命令:jcmd <pid> GC.run_finalization
作用: 对 JVM 执行 java.lang.System.runFinalization()。即 :执行一次 finalization 操作
调用已经失去引用的对象的finalize方法,但JVM可以选择执行或者不执行。

示例:

# jcmd 17124 GC.run_finalization
17124:
Command executed successfully

2.14 执行垃圾收集/GC.run

命令:jcmd <pid> GC.run
作用:对 JVM 执行 java.lang.System.gc()。同 GC.run_finalization 告诉垃圾收集器打算进行垃圾收集,但是JVM可以选择执行或者不执行。

$ jcmd 17124 GC.run
17124:
Command executed successfully

2.15 查看JVM进程版本信息/VM.version

命令:jcmd <pid> VM.version
作用:查看目标jvm进程的版本信息。

示例:

# jcmd 6339 VM.version
6339:
OpenJDK 64-Bit Server VM version 25.282-b08
JDK 8.0_282

2.16 VM.native_memory

命令:jcmd <pid> VM.native_memory
作用:查看目标jvm进程的Native Memory Tracking (NMT)信息,用于追踪JVM的内部内存使用。
注意:

  • 打开NMT会带来5%-10%的性能损耗。
  • -XX:NativeMemoryTracking=xx参数必须放在 -jar 前面
    • 示例:java -XX:NativeMemoryTracking=detail -jar -Xmn512m user.jar
  • 使用 -XX:NativeMemoryTracking=summary 可以用于开启NMT,其中该值默认为off。
    • 可以设置summary、detail来开启;开启的话,大概会增加5%-10%的性能消耗;
    • 使用-XX:+UnlockDiagnosticVMOptions -XX:+PrintNMTStatistics。 可以在jvm shutdown的时候输出整体的native memory统计;
    • 其他的可以使用jcmd pid VM.native_memory相关命令进行查看、diff、shutdown等
  • 整个memory主要包含了Java Heap、Class、Thread、Code、GC、Compiler、Internal、Other、Symbol、Native Memory Tracking、Arena Chunk这几部分;
    • 其中 reserved 表示应用可用的内存大小,committed 表示应用正在使用的内存大小
jcmd <pid> VM.native_memory [summary | detail | baseline | summary.diff | detail.diff | shutdown] [scale= KB | MB | GB]
 
# summary: 分类内存使用情况.
# detail: 详细内存使用情况,除了summary信息之外还包含了虚拟内存使用情况。
# baseline: 创建内存使用快照,方便和后面做对比
# summary.diff: 和上一次baseline的summary对比
# detail.diff: 和上一次baseline的detail对比
# shutdown: 关闭NMT

示例:

# jcmd 17124 VM.native_memory
17124:
Native memory tracking is not enabled
  • 查看是否已开启NMT
# jinfo -flag NativeMemoryTracking 17124
-XX:NativeMemoryTracking=off

# jinfo -flag UnlockDiagnosticVMOptions 17124

# jinfo -flag PrintNMTStatistics 17124
  • 实时设置开启NMT
# jinfo -flag NativeMemoryTracking=summary/detail 17124 
	[X] 不支持运行时修改 Exception in thread "main" com.sun.tools.attach.AttachOperationFailedException: flag 'NativeMemoryTracking' cannot be changed
	
# jinfo -flag +UnlockDiagnosticVMOptions 17124
	[X] 不支持运行时修改
	
# jinfo -flag +PrintNMTStatistics 17124
	[X] 不支持运行时修改
  • 查看 NMT JVM 内存统计结果
# jcmd 24568 VM.native_memory
24568:

Native Memory Tracking:

Total: reserved=6091612KB, committed=1355164KB
-                 Java Heap (reserved=4128768KB, committed=661504KB)
                            (mmap: reserved=4128768KB, committed=661504KB)

-                     Class (reserved=1141748KB, committed=104052KB)
                            (classes #16686)
                            (malloc=11252KB #23205)
                            (mmap: reserved=1130496KB, committed=92800KB)

-                    Thread (reserved=101832KB, committed=101832KB)
                            (thread #100)
                            (stack: reserved=101376KB, committed=101376KB)
                            (malloc=307KB #506)
                            (arena=149KB #195)

-                      Code (reserved=255298KB, committed=35598KB)
                            (malloc=5698KB #8972)
                            (mmap: reserved=249600KB, committed=29900KB)

-                        GC (reserved=161302KB, committed=149514KB)
                            (malloc=10390KB #325)
                            (mmap: reserved=150912KB, committed=139124KB)

-                  Compiler (reserved=173KB, committed=173KB)
                            (malloc=42KB #802)
                            (arena=131KB #7)

-                  Internal (reserved=274448KB, committed=274448KB)
                            (malloc=274384KB #22420)
                            (mmap: reserved=64KB, committed=64KB)

-                    Symbol (reserved=23692KB, committed=23692KB)
                            (malloc=20327KB #195671)
                            (arena=3364KB #1)

-    Native Memory Tracking (reserved=3961KB, committed=3961KB)
                            (malloc=17KB #196)
                            (tracking overhead=3944KB)

-               Arena Chunk (reserved=391KB, committed=391KB)
                            (malloc=391KB)


# jcmd 24568 VM.native_memory summary scale=MB
24568:

Native Memory Tracking:

Total: reserved=5950MB, committed=1325MB
-                 Java Heap (reserved=4032MB, committed=646MB)
                            (mmap: reserved=4032MB, committed=646MB)

-                     Class (reserved=1115MB, committed=102MB)
                            (classes #16686)
                            (malloc=11MB #23336)
                            (mmap: reserved=1104MB, committed=91MB)

-                    Thread (reserved=100MB, committed=100MB)
                            (thread #101)
                            (stack: reserved=100MB, committed=100MB)

-                      Code (reserved=249MB, committed=35MB)
                            (malloc=6MB #9278)
                            (mmap: reserved=244MB, committed=29MB)

-                        GC (reserved=158MB, committed=146MB)
                            (malloc=10MB #325)
                            (mmap: reserved=147MB, committed=136MB)

-                  Internal (reserved=268MB, committed=268MB)
                            (malloc=268MB #22425)

-                    Symbol (reserved=23MB, committed=23MB)
                            (malloc=20MB #195671)
                            (arena=3MB #1)

-    Native Memory Tracking (reserved=4MB, committed=4MB)
                            (tracking overhead=4MB)

X 参考文献