How to Monitor IBM MQ with AppDynamics using LAMAXU

Defining the AppDynamics Agent

To integrate LAMAXU with AppDynamics, simply include the AppDynamics -javaagent option to the LAMAXU startup.sh script.

Please contact your AppDynamics administrators as they need to generate, and provide you with, a new client configuration bundle for every LAMAXU agent deployed.

The ./conf/controller-info.xml located in the AppDynamics agent’s directory contains information about the server LAMAXU will push JMX updates to.

Example; This example uses the following values to identify the LAMAXU agent.

./conf/controller-info.xml

Adding the AppDynamics Agent to the LAMAXU Start-up

Add the option in red below to the ${LAMAXU_HOME}/bin/startup.sh|startup.bat files.
nohup "$JAVA_HOME/bin/java" -javaagent:/app/AppServerAgent-4.0.6.0/ver4.0.6.0/javaagent.jar -DLOG_PATH=${LAMAXU_HOME}/logs -cp ${LAMAXU_CLASSPATH} com.qm.lamaxu.Main -config ${LAMAXU_HOME}/config/config.xml >nohup.out 2>&1 &

Testing AppDynamics works with LAMAXU

Once the LAMAXU agent has been started, open AppDynamics and navigate to the LAMAXU MQ Application on the menu.

Once in the LAMAXU MQ application, navigate to App Servers and select the node.

Testing AppDwithLamaxu

Navigate to ‘More/JMX’ and click on the ‘Refresh Domains’ button to view the monitored MQ queue manager MBeans, as show in screen shots below.

Testing AppDwithLamaxu1

Testing AppDwithLamaxu2

How to Monitor IBM MQ with Solarwinds and Queuemetrix Lamaxu

Queuemetrix Lamaxu extends the monitoring capability of solarwinds SAM by enabling the monitoring of IBM MQ message flows and metrics.

LAMAXU interfaces natively with solarwinds ® using standard Java JMX MBeans, making IBM MQ appear just like a Java Application Server.

SolarWindsSummaryPage

SolarWindsComponentStats

SETTING UP THE LAMAXU JMX GATEWAY

The JMX Gateway can be accessed using an RFC2609 compliant service url:

service:jmx:rmi://<TARGET_MACHINE>:<JMX_RMI_SERVER_PORT>/jndi/rmi://<TARGET_MACHINE>:<RMI_REGISTRY_PORT>/jmxrmi

Given the above and the following parameters:

TARGET_MACHINE = auprod1
JMX_RMI_SERVER_PORT = 3098
RMI_REGISTRY_PORT = 3099

We can derive the following functional URL:

service:jmx:rmi://auprod1:3098/jndi/rmi://auprod1:3099/jmxrmi

For convenience, this URL is also written to the LAMAXU log file at start-up:

2015-11-23 18:07:37,039 [main] JmxGateway remote url is: service:jmx:rmi://auprod1:3098/jndi/rmi://auprod1:3099/jmxrmi

You can now use the URL above to connect to the LAMAXU JMX gateway process.

JconsoleUrl

Locate and execute the JDK utility jconsole. On startup you will be prompted for a URL, username and password. You should be able to connect to LAMAXU successfully,  assuming authentication and SSL details have been defined correctly.

1. As illustrated in the screenshot below, the MQ queue manager information is displayed under the MBeans tab.
2. A successful connection via jconsole means you can now proceed to connecting Solarwinds using the same parameters.

SSL secured communications

A LAMAXU process secured by SSL can be accessed by specifying an appropriately configured JKS trust store.

The command below can be specified on the jconsole command line:

jconsole -J-Djavax.net.ssl.trustStore=/app/LAMAXU/keystore.jks -J-Djavax.net.ssl.trustStorePassword=changeit

Jconsole

QUEUE MANAGER STATUS

SolarWindsJMXmbeanSelect1

QUEUE STATUS

SolarWindsJMXmbeanQStatus

CHANNEL STATUS

SolarWindsJMXmbeanChStatus1

PUB/SUB STATUS

SolarWindsJMXmbeanPSStatus

How to Monitor IBM MQ with SPLUNK and LAMAXU

What is SPLUNK

Splunk (the product) captures, indexes and correlates real-time data in a searchable repository from which it can generate graphs, reports, alerts, dashboards and visualizations.
SPLUNK Web Site

Sample SPLUNK Dashboard

SPLUNK_dashboard

Using SPLUNK

Once you have added the LAMAXU logs data to SPLUNK the MQ metrics can be queried using SPLUNK search syntax.

UsingSplunk

Graphing the Current Queue Depth

source=”statusData.log” sourcetype=”_json_lamaxuStatus” “identifier.dataSource”=DEMO “attributes.CURDEPTH”=”*” “attributes.QUEUE”!=”AMQ.*” “attributes.QUEUE”!=”SYSTEM.*”| timechart avg(“attributes.CURDEPTH”) by attributes.QUEUE

GraphCurrent QueueDepth

Graphing Message Volume Statistics

source=”statData.log” sourcetype=_json_lamaxuEvents “identifier.dataType”=STATISTICS  identifier.dataSource”=DEMO ” attributes.QUEUE”=”QM_TEST_REPLY1*” | eval pfields=split(‘attributes.MQIAMO_PUTS’, ” “)  | eval PUTSNP=mvindex(pfields,0)| eval PUTSP=mvindex(pfields,1) |eval TPUTS=PUTSNP+PUTSP| eval gfields=split(‘attributes.MQIAMO_GETS’, ” “) | eval GETSNP=mvindex(gfields,0)| eval GETSP=mvindex(gfields,1) |eval TGETS=GETSNP+GETSP | timechart avg(attributes.MQIAMO_Q_MAX_DEPTH)  as “Hwm” avg(TPUTS) as “Puts” avg(TGETS) as Gets

GraphMsgVolumeStats

Graphing Message Time on Queue

source=”statusData.log” “attributes.QTIME”=”*” “attributes.QUEUE”!=”SYSTEM.*” “attributes.QUEUE”!=”AMQ.*” sourcetype=”_json_lamaxuStatus” “identifier.dataSource”=DEMO| timechart avg(“attributes.QTIME”) by attributes.QUEUE

GraphMsgTimeOnQueue

LAMAXU/SPLUNK Source Types

The following source type need to be added to your SPLUNK props.conf file in order to correctly recognise the time-stamp fields in the log file. Alternatively you could manually create the source types using the red highlight values as a guide to defining the time-stamps.

Source type for LAMAXU Status logs

[_json_lamaxuStatus]
INDEXED_EXTRACTIONS = json
KV_MODE = none
NO_BINARY_CHECK = true
TIMESTAMP_FIELDS = createDate
TIME_FORMAT = %d/%m/%Y %OI:%M:%S %p
TZ = Australia/Sydney
category = Structured
description = JavaScript Object Notation format. For more information, visit
http://json.org/
disabled = false
pulldown_type = true

Source type for LAMAXU Event logs

[_json_lamaxuEvents]
INDEXED_EXTRACTIONS = json
KV_MODE = none
NO_BINARY_CHECK = true
TIMESTAMP_FIELDS = attributes.MQCAMO_END_DATE,attributes.MQCAMO_END_TIME
TIME_FORMAT = %Y-%m-%d %H.%M.%S
TZ = Australia/Sydney
category = Structured
description = JavaScript Object Notation format. For more information, visit
http://json.org/
disabled = false
pulldown_type = true

Log File Format for statusData.log

Log file attributes are modeled on the IBM MQ runmqsc attribute names.

QMGR_STATUS
{
    "identifier": {
        "dataSource": "DEMO",
        "dataType": "QMGR_STATUS",
        "sui": "DEMO"
    },
    "createDate": "10/10/2015 9:47:45 AM",
    "attributes": {
        "STARTTI": "13.44.07",
        "INSTDESC": "",
        "CONNS": "24",
        "STANDBY": "NOPERMIT",
        "INSTNAME": "Installation1",
        "CHINIT": "RUNNING",
        "STATUS": "RUNNING",
        "STARTDA": "2015-09-30",
        "CMDSERV": "RUNNING",
        "LDAPCONN": "INACTIVE",
        "QMNAME": "DEMO"
    },
    "category": "STATUS"
}
QUEUE_STATUS
{
    "identifier": {
        "dataSource": "DEMO",
        "dataType": "QUEUE_STATUS",
        "sui": "LOCALQNAME"
    },
    "createDate": "10/10/2015 11:47:42 AM",
    "attributes": {
        "IPPROCS": "2",
        "LGETDATE": "",
        "QTIME": "-1 -1",
        "QUEUE": "LOCALQNAME",
        "MONQ": "OFF",
        "OPPROCS": "2",
        "CURDEPTH": "0",
        "LGETTIME": "",
        "MSGAGE": "-1",
        "LPUTDATE": "",
        "LPUTTIME": "",
        "TYPE": "QSTATUS",
        "UNCOM": "NO"
    },
    "category": "STATUS"
}
SUBSCRIPTION_STATUS
{
    "identifier": {
        "dataSource": "DEMO",
        "dataType": "SUBSCRIPTION_STATUS",
        "sui": "TEST.SUBSCRIBER "
    },
    "createDate": "10/10/2015 9:47:45 AM",
    "attributes": {
        "MCASTREL": "-1 -1",
        "DURABLE": "NO",
        "LMSGTIME": "",
        "RESMDATE": "2015-10-10",
        "QMNAME": "DEMO",
        "CURDEPTH": "4",
        "MSGAGE": "0",
        "TOPICSTR": "TEST.TOPIC",
        "SUB": "TEST.SUBSCRIBER",
        "LMSGDATE": "",
        "SUBID": "414d512044454d4f2020202020202020885a0b56049a9c20",
        "NUMMSGS": "0"
    },
    "category": "STATUS"
} 
PUBSUB_STATUS
{
    "identifier": {
        "dataSource": "DEMO",
        "dataType": "PUBSUB_STATUS",
        "sui": "DEMO"
    },
    "createDate": "10/10/2015 9:47:45 AM",
    "attributes": {
        "TPCOUNT": "6",
        "STATUS": "ACTIVE",
        "TYPE": "LOCAL",
        "SUBCOUNT": "3",
        "QMNAME": "DEMO"
    },
    "category": "STATUS"
}
LISTENER_STATUS
{
    "identifier": {
        "dataSource": "DEMO",
        "dataType": "LISTENER_STATUS",
        "sui": "DEMO.1414"
    },
    "createDate": "10/10/2015 9:47:26 AM",
    "attributes": {
        "STARTTI": "00.31.37",
        "PORT": "1414",
        "PID": "38076556",
        "TRPTYPE": "TCP",
        "BACKLOG": "100",
        "IPADDR": "::ffff:10.30.40.74",
        "STATUS": "RUNNING",
        "LISTENER": "DEMO.1414",
        "STARTDA": "2015-10-10",
        "DESCR": "DEMO default Listener"
    },
    "category": "STATUS"