Application Performance Monitoring Part 2: Client-Based.

This article is the second in a two-part series. The first part appeared in the February issue of CTR.

Client-based monitoring is probably the biggest part of the performance monitoring and reporting applications market, and can be broken down into two sub-categories: client agents and client simulations. Performance monitoring via client agents may involve the pre-installation of an agent in the desktop machine or may use a web applet or script that loads itself via the browser. The former is more likely to be of interest to an enterprise monitoring the performance of internal applications (whether to local or remote users) or Application Service Providers (ASPs), both of whom have some degree of control over the configuration of the client. The latter is likely to be of more use to web sites that can't control their users' configurations.

Examples of these applications include Candle's ETEWatch, Lucent's VitalSuite, and NetIQ's Pegasus, which use client agents; and Candle's eBA, the Hewlett Packard Web Transaction Observer, and the Landmark Systems WebWatcher, which use applets or scripts.

As might be expected, client-based monitoring is the most accurate way of gathering information about the actual user experience. It can reveal important metrics such as the response time for a page download in a browser or the response time for a specific interaction in an application. It delivers highly granular data and is not very invasive.

However, it does require installation, one way or the other, of an agent on the client. This can be labor intensive and can impact the reliability of the client. Web applets and scripts in particular are at the mercy of the browser version in the client and may break if this is changed. Client agents of either sort must capture a very large volume of data to develop useful metrics, and these must be uploaded to the monitoring application. And, even though the information is extremely accurate on a per-user basis, to be truly useful for judging how well business goals are being met, the data from many users must be average. In short, these applications are very data-intensive.

Client Simulations

Client simulation applications can also be divided into two subclasses: applications that are owned and installed by the customer, which might be called capture/playback tools, and Internet service offerings. The former includes Agilent's Firehunter, NetScout's Application Service Level Manager, and Tivoli's Application Performance Management. The latter includes Keynote Systems' Perspective and Mercury Interactive's Topaz and Active Watch.

Capture/playback tools use applications at the client that can record actual user interactions and then replay them against the server to measure response times. By deploying these agents at multiple locations, a manager can obtain continuous measurements without depending on actual user interactions. The advantage is that a capture/playback tool delivers information about real transactions at whatever interval is chosen, and this information can be carefully tailored to match real business needs.

However, since they are real transactions, they load up the network and host the same way, and the information they deliver must be balanced against this impact. In addition, the playback scenarios must be updated every time an application is added or changed.

Internet Service Offerings

Seeing how labor-intensive many forms of application performance monitoring are, it's no surprise that, according to the Gartner Group, some 30% of this market is now handled by service providers. This is most commonly done via some form of client simulation, as in the case of Keynote Systems, which uses a wide network of clients throughout the Internet to monitor the performance of web sites over a variety of metrics.

The independent auditing of site performance such a service provides is one of its greatest strengths: it yields a consistent, constantly available measure of performance that can help benchmark a company's systems against its competitors. And offloading the management burden of performance monitoring leaves managers more time for more critical business tasks. These service providers constantly upgrade their capabilities and can offer surprisingly granular insight into application performance--for a price.

However, these services, unless highly customized (and therefore expensive) tend to offer a one-size-fits all approach that is most useful for comparison against similar sites but does not fit a business profile as closely as the other approaches outlined above. At the low end these services will not deliver the highly granular information that many businesses need. In addition, it is easy for web sites to optimize their sites for the client, making the metrics less useful because they're less representative of real user experiences. Related to this problem is the placement of clients: in many cases they are in ISP Points of Presence (POPs), connected via 10 Mbps Ethernet. While still useful for comparison, this is not exactly representative of the general user experience, even at businesses with broadband Internet connections.