Introduction
Document Status
The Diagnostics Agent troubleshooting guide used to be available as a PDF document. It is now replaced by this wiki page. In case the content of this page does not address your problem, please refer to the former PDF document, being still available as DiagnosticsAgentTroubleshooting.pdf attachement to SAP Note 1833501.
Diagnostics Agent Installer
For more general information on installation strategy, installer versions, etc., please refer to Diagnostics Agents.
Troubleshooting Ressource Consumption and Booting Issues of the Diagnostics Agent
Check OS Settings on Linux Hosts (ulimit and umask)
After the installation of the Diagnostics Agent on UNIX platforms, remind to double check the following OS environment settings for the Diagnostics Agent OS user (often daaadm):
ulimit
- Log on with the agent OS user, e.g.
su – daaadm - Open an sh shell with the command
sh - Execute the command
ulimit -a - Compare the values with the recommendedations in the Installation & Setup Guide for the most recent installer (see SAP Note 1833501).
- If you change the limits, remember to stop the Diagnostics Agent (sapstop), kill the Diagnostics Agent processes (sapstartsrv) and start the Agent again (sapstart). In this way the Diagnostics Agent will take the new settings into account.
If the user’s limit parameters for the OS Diagnostics Agent user are not correct, high CPU consumption situations or out-of-memory errors can result.
umask
Please refer to SAP note 1163751.
Trigger Thread Dumps for Diagnostics Agents
It is important to do thread dumps of the Diagnostics Agent process, to understand the problem context in case of a deadlock, not responding Diagnostic Agent or a high CPU time consumption situation.
Trigger a Thread Dump on Windows
The Diagnostics Agent is an SAP system and available in SAP MMC.
In SAP MMC, open the node “AS Java Process Table” of the Diagnostics Agent identified by the SID and instance number. Select the desired Agent Node and right click on it to display the context menu, then click the item “Dump Stack Trace”.
In SAP MMC, open the node “AS Java Process Table” of the Diagnostics Agent identified by the SID and instance number. Select the desired Agent Node and right click on it to display the context menu, then click the item “Dump Stack Trace”.
Trigger a Thread Dump on Unix
For Diagnostics Agents 7.1x, 7.2x, 7.3x or installed with SWPM 1.0 SP 3 and higher, log on in a UNIX shell with the OS user of the Diagnostics Agent, navigate to the folder /usr/sap/
sapcontrol.exe -useradm -nr -function J2EEControlProcess smdagent DumpStackTrace
example:
./sapcontrol -user daaadm abc123 -nr 98 -function J2EEControlProcess smdagent DumpStackTrace
Alternatively a thread dump can be triggered via "kill -3 ", with being the process ID of the Diagnostics Agent's Java process.
Where Is the Thread Dump Stored?
The thread dump is written into the file /usr/sap/
Trigger Heap Dumps for Diagnostics Agents
Verbose Class Loading
To enable verbose logging of the class loading of Diagnostics Agent add -verbose:class-verbose:class to the property smdagent.javaParameters in /usr/sap/<>/SMDA<>/SMDAgent/smdagent.properties . Restart the Diagnostics Agent and find the verbose class loading information at usr/sap/<>/SMDA<> /work/jvm_smdagent.out.
Common Issues Related to Ressource Consumption and Booting of the Diagnostics Agent
The Diagnostics Agent Log Contains Out-of-Memory Errors caused by abapReadSyslog
Symptom:
The log contains an exception like this:
Cause:
The syslog size is at least nearly as big as the Java maximum heap size.
Resolution:
- To resolve the issue increase the Java maximum heap size (see below) to be sufficiently bigger than the syslog size.
- Alternatively, limit the size of the syslog or delete it.
The Diagnostics Agent Log Contains other Out-of-Memory Errors
Symptom:
java.lang.OutOfMemoryError: Java heap space and java.lang.OutOfMemoryError: GC overhead limit exceeded can be found in the log file.
Cause:
By default, the SMD Agent's Java heap maximum size is set to 256 MB. In the event of a large data collection, a heap size of 512 MB or more is required.
Resolution:
To resolve the issue, do the following for the Diagnostic Agent on host :
1. On your managing host, browse to the following directory:
Windows :\usr\sap\DAA\SMDA98\SMDAgent
Unix /usr/sap/DAA/SMDA98/SMDAgent
Note: In the above example DAA is the name of the SID and the instance number is 98. This may differ on your system.
2. Open the file smdagent.properties using a text editor
3. Locate the property smdagent.javaParameters then locate the parameter -Xmx
4. Change the value of -Xmx to -Xmx512m then save the changes. For example:
smdagent.javaParameters=-DP4ClassLoad=P4Connection -Xmx512m -Xms256m -XX:MaxPermSize=128m
5. Restart the managing SMD Agent.
The Diagnostics Agent Does Not Start
For issues related to the start-up of the Diagnostics Agent, please refer to this Wiki page.
Extracting Configuration From Secure Store File Failed
If the SMDSystem.*.log contains errors similar to the entries listed below, the secure store of the Diagnostics Agent might be broken due to incompatible updates of the Diagnostics Agent or the JVM:
To reset the secure store, proceed as follows:
- Stop the Diagnostics Agent
- Delete /usr/sap/
/SMDA /SMDAgent/configuration/secstore.properties. - Delete /usr/sap/
/SMDA /SMDAgent/configuration/security/.CertificatesKeyStore - Start the Diagnostics Agent.
- In the Agent Administration → Non-Authenticated Agents select the respective Diagnostics Agent and click on Trust Agent.
In case the issue persists after the execution of the steps above, it is possible to follow the steps in the following article to correct this scenario.
- 2447919 - Diagnostic Agent does connect to Solution Manager with the error: 'record with key "$internal/mode" is missing.'
The Diagnostics Agent is Blocking HANA Client Ports
When the Diagnostics Agent is started, all connections from the Solution Manager to the SAP HANA fail (e.g. Telnet on port 22 or the HANA port that is used by the Solution Manager server).
Solution:
A possible cause might be that there are enough ports, but they cannot be reused fast enough. This can be configured using the below mentioned parameters. At SAP Note 2382421 especially check the sections "net.ipv4.ip_local_port_range" and "SAP Host Agent Configuration". The latter points to SAP Note 401162, which should also be taken into account. Furthermore, check the parameters "net.ipv4.tcp_tw_reuse" and "net.ipv4.tcp_tw_recycle" as described at SAP Note 2382421.
SAP Solution Manager Java Stack Scalability Settings
Please refer to SAP Note 2417827
No comments:
Post a Comment