Oracle OEM中EMGC_OMS1指标警报The heap usage使用率过高

OEM报警EMGC_OMS1目标堆内存占用率过高。

我的环境OEM版本是13.3。

默认监控目标EMGC_OMS1的最大内存使用是1740M，当前的最大堆内存值对于实际的 OMS 负载来说太低了。

由于已经定期生成这些警报和通知，所以增加 OMS 进程使用的堆内存。

OMS 的堆内存设置使用以下属性定义：

OMS_HEAP_MAX　　　　--此次需要更改的值
OMS_HEAP_MIN
OMS_PERMGEN_MIN
OMS_PERMGEN_MAX

可以使用以下命令检查当前设置：

[oracle@oem13c ~]$ export ORACLE_HOME=$OMS_HOME
[oracle@oem13c ~]$ export PATH=$OMS_HOME/bin:$PATH

[oracle@oem13c ~]$ emctl get property -name OMS_HEAP_MAX -sysman_pwd oracle123
Oracle Enterprise Manager Cloud Control 13c Release 3  
Copyright (c) 1996, 2018 Oracle Corporation.  All rights reserved.
Value for property OMS_HEAP_MAX for oms local_oms is 1740M
 
[oracle@oem13c ~]$ emctl get property -name OMS_HEAP_MIN -sysman_pwd oracle123
Oracle Enterprise Manager Cloud Control 13c Release 3  
Copyright (c) 1996, 2018 Oracle Corporation.  All rights reserved.
Value for property OMS_HEAP_MIN for oms local_oms is 256M

[oracle@oem13c ~]$ emctl get property -name OMS_PERMGEN_MIN -sysman_pwd oracle123
Oracle Enterprise Manager Cloud Control 13c Release 3  
Copyright (c) 1996, 2018 Oracle Corporation.  All rights reserved.
Value for property OMS_PERMGEN_MIN for oms local_oms is 128M

[oracle@oem13c ~]$ emctl get property -name OMS_PERMGEN_MAX -sysman_pwd oracle123
Oracle Enterprise Manager Cloud Control 13c Release 3  
Copyright (c) 1996, 2018 Oracle Corporation.  All rights reserved.
Value for property OMS_PERMGEN_MAX for oms local_oms is 768M

也可ps查看：

[oracle@oem13c ~]$ ps -ef | grep java | grep EMGC_OMS1
oracle    10218  10158 17 16:18 ?        00:09:15 /u01/app/oem13c/middleware/oracle_common/jdk/bin/java -server -Xms256M -Xmx1740M -XX:PermSize=128M -XX:MaxPermSize=768M -XX:CompileThreshold=8000 -XX:-DoEscapeAnalysis -XX:+UseCodeCacheFlushing -XX:ReservedCodeCacheSize=100M -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSClassUnloadingEnabled -Dweblogic.Name=EMGC_OMS1 ........

至于OMS_HEAP_MAX设置多大，根据OEM13.3官档先确定监控架构规模，我这边环境规模是Small

Size	Agent Count	Target Count	Concurrent User Sessions
Eval	< 10	< 100	<3
Small	< 100	< 1000	<10
Medium	>= 100, < 1000	>= 1000, < 10,000	>= 10, < 25
Large	>= 1000	>= 10,000	>= 25, <= 50*
Extra Large	>= 5000	>=50,000	>=50, <=100

OMS_HEAP_MAX对应规模应该设置的建议值官档也有，如下：

OMS_HEAP_MAX (-Xmx)

As targets are added after the initial installation/setup of Enterprise Manager, increasing the HEAP size is recommended to avoid any unforeseen Out Of Memory Error of Tenured/Old Gen.

32 bit –

Small/Medium/Large: 1524M

64 bit -

Small: 1740M

Medium: 4096M

Large: 8192M

For IBM JVM, irrespective of the app size, there are no limits on the heap size.

Same as mentioned in the Default section.

These are post installation defaults, thus the recommended setup.

All these parameters should be changed, once users experience a lower throughput over a period of time, due to consistently high memory usage. The person (preferably sysadmin) manipulating the parameters must be aware of the limits/warnings.

表格建议值Small是1740M，当前已经是1740M的情况下警报了，因此调整值Medium对应的4096M。

修改：

[oracle@oem13c ~]$ emctl set property -name OMS_HEAP_MAX -value 4096M -sysman_pwd oracle123
Oracle Enterprise Manager Cloud Control 13c Release 3  
Copyright (c) 1996, 2018 Oracle Corporation.  All rights reserved.
Property OMS_HEAP_MAX for oms oem13c:4889_Management_Service has been set to value 4096M
OMS restart is required to reflect the new property value

重启OMS：

[oracle@oem13c ~]$ emctl stop oms -all
Oracle Enterprise Manager Cloud Control 13c Release 3  
Copyright (c) 1996, 2018 Oracle Corporation.  All rights reserved.
Stopping Oracle Management Server...
WebTier Successfully Stopped
Oracle Management Server Successfully Stopped
Oracle Management Server is Down
JVMD Engine is Down
Stopping BI Publisher Server...
BI Publisher Server Successfully Stopped
AdminServer Successfully Stopped
BI Publisher Server is Down

[oracle@oem13c ~]$ emctl start oms
Oracle Enterprise Manager Cloud Control 13c Release 3  
Copyright (c) 1996, 2018 Oracle Corporation.  All rights reserved.
Starting Oracle Management Server...
WebTier Successfully Started
Oracle Management Server Successfully Started
Oracle Management Server is Up
JVMD Engine is Up
Starting BI Publisher Server ...
BI Publisher Server Successfully Started
BI Publisher Server is Up

[oracle@oem13c ~]$ emctl get property -name OMS_HEAP_MAX -sysman_pwd oracle123
Oracle Enterprise Manager Cloud Control 13c Release 3  
Copyright (c) 1996, 2018 Oracle Corporation.  All rights reserved.
Value for property OMS_HEAP_MAX for oms local_oms is 4096M

新值已经生效了。

ps角度看：

[oracle@oem13c ~]$ ps -ef | grep java | grep EMGC_OMS1
oracle    47372  47312 94 17:34 ?        00:06:27 /u01/app/oem13c/middleware/oracle_common/jdk/bin/java -server -Xms256M -Xmx4096M -XX:PermSize=128M -XX:MaxPermSize=768M -XX:CompileThreshold=8000 -XX:-DoEscapeAnalysis -XX:+UseCodeCacheFlushing -XX:ReservedCodeCacheSize=100M -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSClassUnloadingEnabled -Dweblogic.Name=EMGC_OMS1......