Perf Counters using JMX MBeans

Performance Counters are part of Windows and are easily available within any .NET app. A bit of kludgy install code and you’re pretty much done. Various monitoring system pick up data, monitor the site and alarm Ops.

JMX MBeans is a Java approach for monitoring and management of your application. MBeans not only allow you to instrument the code but also allow your application to notify listeners when something goes wrong. Moreover, they allow for a back-channel to your application to allow for changing behavior on a fly. While the latter feature if nice, I honestly don’t see anyone changing production environment in this manner. Still, it’s a rather slick solution for, say, evaluating various config settings during development or stress testing.

When tasked with instrumenting my application, start by implementing generic “perf counters”. Check .net documentation and implement similar objects in Java. For example, for Average perf counter specify two APIs: add and get

public void add(int delta)
{
operationscounter.incrementAndGet();
timecounter.addAndGet(delta);
}

public double get()
{
int currentValue = operationscounter.getAndSet(0);
int accumulatedTime = timecounter.getAndSet(0);

return accumulatedTime / (double) currentValue;
}

While both both operationscounter and timecounter are AtomicIntegers, the above APIs are not thread-safe. I purposely left out synchronized keyword to get better performance. We’re not counting $$ after all, so it’s ok if data is slightly off. The second - get - api also resets both counters to 0 (ok, check that denominator is not zero; check for overflow – small details).

Similarly, you can create a rate perf counter:
public void increment()
{
mycounter.incrementAndGet();
}

public double get()
{
long timeNow = System.currentTimeMillis();
final double delta = timeNow – lastPollTime;
lastPollTime = timeNow; //reset time stamp
double result = currentValue/delta * multiplier;
// multiplier=1000 when returning results in seconds
return result;
}

Once perf counters are written, instrument your application and expose data via an MBean. MBean is a regular java class implementing a specially constructed MBean interface: only few rules to follow: your class must have a default constructor; if you class name is MyService, than the MBean interface must be name MyServiceMBean, etc. Amazon has a few JMX-related books; they don’t cover MXBeans which are thoroughly described in this article.

e.g. MBean interface:
public interface MyServiceMBean
{
public int getRequestsCount();
public int getSystemFailuresCount();
public double getAverageProcessingTime();
public double getRequestRate();
}

public class MyService implements MyServiceMBean

{
...
}
As with Windows perf counters installation, there is a bit of initialization code (5 lines below) and then any JMX-aware monitoring system will be able to see MBeans and poll for data. You can also run jconsole during development phase to verify your JMX MBean implementation.

Initialization code:
MyService instance = new MyService();
String jmxlabel = “MyService:type=MoneyMakerService”;
MBeanServer mbs = ManagementFactory.getPlatformMBeanServer();
ObjectName name = new ObjectName(jmxlabel);
mbs.registerMBean(instance, name);

One caveat: you can’t pack various counter data (request count, rate, etc) into an object or return them as array: otherwise (as of today) data won’t be processed correctly by some management systems. MXBeans appeared to be an answer as the data can be indeed wrapped into a class and shows up nicely on the monitoring side… but, at least with Manage Engine, the values are treated as strings so you can’t alert on them. You can easily convert to MXBeans later on if need be – but the perf counters classes you build would not have to change!