The smart approach to distributed performance monitoring with Java

If you’re developing distributed applications, you probably want to measure the performance characteristics of your applications to find out which remote methods — on which objects, on which machines — take longest to run. To accomplish this, you could use a distributed profiling tool, but such products tend to be expensive and difficult to configure. You could also make changes to the client application to time each method call, but this is intrusive. Fortunately, there is a better way — smart stubs. The smart stub approach is cheap to implement, lightweight, and requires no modification of the original client and server code.

Whilst several commercial CORBA object request brokers (ORBs) such as VisiBroker and Orbix provide smart stub (or smart proxy) implementations, this isn’t a feature provided out of the box for RMI and Java IDL. With RMI and Java IDL support for smart stubs, you’d have the ability to intercept remote method calls in a completely noninvasive and fully reversible way. I have so far put this idea to use in security applications (denying access to certain remote objects according to IP address) and in audit trailing (writing a usage profile to a log file), as well as in the context of distributed performance monitoring, as described here.

This article introduces stubs, describes what it means for them to be smart, and tells how they may be used for noninvasive performance monitoring. It concludes by offering a technique for adding smart stub behavior to RMI and Java IDL applications.

What are stubs?

In any distributed environment such as RMI or CORBA, client applications communicate with server-side objects through stubs. Each stub, a client-side representation of an associated server-side object, hides the complex network communications between the client and server, as illustrated in Figure 1.

To be more precise, communication occurs through proxies, where a proxy object is an instance of a stub class. The terms stub and proxy are often used interchangeably.

Stub classes are generated as part of the distributed development process. The stub generator for RMI applications is called rmic, while for Java IDL it is called idltojava.

What are smart stubs?

Smart stubs are direct replacements for the original stubs that perform the same function as the originals with additional behavior. They can be added and removed easily with no code changes within the client application or the server object. The additional behavior may be the timing of method calls, or caching of data so that each client request does not necessarily result in a costly invocation across the network.

The Figure 2 shows an example client-server Bank application that displays a list of accounts and allows updates to balances. The pop-up Bank Client Monitor displays average call times to the remote methods on the Bank server object, along with the percentage of calls that were cached locally.

Caching kicks in when a trigger-happy user repeatedly presses the Refresh button, since we don’t want to make a costly remote invocation every time the button is pressed. As the cached percent increases, the average call time decreases.

At this point, I should emphasize the fact that the call timer, the local cache, and even the Client Monitor window are handled entirely within the smart stub. No changes whatsoever have been made to the original client or server code. Furthermore, the client application has no knowledge of the Client Monitor window that appears automatically when the stub is instantiated, and the server object is not aware that its responses are being timed and cached.

A smart stubs implementation for RMI

Using RMI as an example, a remote server object will have an interface (IBank.java):

public interface IBank extends java.rmi.Remote
{
   public String[] getAllAccounts() throws java.rmi.RemoteException;
   public String getAccount(String accno) throws java.rmi.RemoteException;
   public void setAccount(String accno, int newAmount) throws java.rmi.RemoteException;
}

and an implementation (Bank.java):

public class Bank extends UnicastRemoteObject implements common.IBank
{
   public Bank() throws RemoteException {...}
   public String[] getAllAccounts() throws java.rmi.RemoteException
   { // return all account details }
   public String getAccount(String accno) throws java.rmi.RemoteException
   { // return a single account balance }
   public void setAccount(String accno, int newAmount) throws java.rmi.RemoteException
   { // update an account with a new amount }
}

Running the rmic command on the compiled Bank implementation class (Bank.class) produces a stub called Bank_Stub.class. If you run the rmic utility with the -keep option, you get the original Java source file (Bank_Stub.java) for this stub.

To make the stub smart, all you need to do is edit the Bank_Stub.java file and recompile it. Modifying the original stub is easier said than done, however. First of all, the Java code is difficult to understand and it is not obvious where changes should be made, as the code segment below demonstrates:

public final class Bank_Stub extends java.rmi.server.RemoteStub implements common.IBank, java.rmi.Remote
...
public final java.lang.String getAccount(java.lang.String $param_String_1) throws java.rmi.RemoteException
{
  try {
    if (useNewInvoke) {
      Object $result = ref.invoke(this, $method_getAccount_0, new java.lang.Object[] {$param_String_1}, 6316030027643029302L);
      return ((java.lang.String) $result);
    } else {
      java.rmi.server.RemoteCall call = ref.newCall((java.rmi.server.RemoteObject) this, operations, 0, interfaceHash);
      try {
        java.io.ObjectOutput out = call.getOutputStream();
        out.writeObject($param_String_1);
      } catch (java.io.IOException e) {
        throw new java.rmi.MarshalException("error marshalling arguments", e);
        }
      ref.invoke(call);
...

Secondly, the code begins with a stark warning:

// Stub class generated by rmic, do not edit.
// Contents subject to change without notice.

This is good, but not helpful, advice. I will bend this rule slightly and make two simple edits: I’ll change the name of the class from Bank_Stub to Bank_Smub and remove the final keyword from each method. The actual content of each stub method is (wisely) left intact.

These minor changes allow the original Bank_Stub, now renamed Bank_Smub, to be extended by a new class that I create called Bank_Stub. From the client application’s point of view, the new Bank_Stub class will be indistinguishable from the original stub, and it will use the services provided by the original stub, which is now its renamed superclass. It’s all illustrated in Figure 3.

An extract from the newly created Bank_Stub.java file is shown in the code below, with an indication of where the additional behavior should go:

public class Bank_Stub extends Bank_Smub
{
  public Bank_Stub()
  {
    super();
    // POP-UP WINDOW HERE
  }
  public Bank_Stub(java.rmi.server.RemoteRef ref)
  {
    super(ref);
    // POP-UP WINDOW HERE
  }
  public java.lang.String getAccount( java.lang.String accno) throws java.rmi.RemoteException
  {
    // START TIMER HERE
    result=super.getAccount(accno);
    // STOP TIMER HERE
    // SEND RESULTS TO POP-UP WINDOW
    return result;
  }
}

The pop-up Client Monitor window trick is achieved by modifying the constructor so that the timer results window appears when the stub is instantiated.

Automating the solution

Creating individual smart stubs by hand is hard work. You must rename each original stub and remove its final keywords. Also, for each stub you must create a replacement stub that delegates all method calls to its superclass and performs the required additional behavior.

You should also keep in mind that the procedure for removing the additional behavior is to rerun the rmic stub generator to overwrite your stub classes with the default RMI implementations; it thus frustratingly easy to lose the results of your hard work.

For all but the simplest one-time-only applications, I would recommend using Java Reflection to implement a replacement for, or an add-on to, the rmic command that would generate smart stubs with the required behavior for any given Java class. You would run the new srmic command to apply smart stubs, and to remove them you would run the original rmic command.

For links to an example smart stub generator for RMI and Java IDL (upon which this article is based) and two of the popular CORBA ORBs that support smart stubs, refer to the Resources section.

More ideas

The smart stubs approach is as applicable to Java IDL as it is to RMI, except that the names are changed. The RMI stub generator produces a stub file called Bank_Stub.java, whereas that Java IDL stub generator produces a stub file called _BankStub.java.

The potential for smart stubs is not limited to the performance monitoring and caching examples that I have described here. Other ideas include load-balanced stubs that choose between server objects, and nine-to-five stubs that stop working when the bank is closed. I hope to collect more ideas, so email me with any good ones.

Source: www.infoworld.com