Voice application and infrastructure testing: a "win-win" proposition. (Product and Services Selection Guide).

Self-service applications based on IVR, CTI and voice recognition technologies have become essential features of the successful enterprise contact center -- and no wonder. A recent GartnerGroup study indicated that the average cost of a single agent-based transaction ranges from $5.50 to $7.00, while the same transaction using a self-service IVR application runs a mere 45 cents. The fiscal logic is inescapable: over 55 percent of respondents to a just-released survey of contact center professionals said that the implementation of additional self-service alternatives is a high priority for the coming year. (1)

The problem, of course, is that self-service applications only pay off if customers use them. Performance problems, such as slow databases or host responses, incorrect call routings and failed transactions will cause customers to "zero out" to an agent -- at first in frustration and eventually by reflex -- taking a heavy toll in agent productivity, line charges and customer satisfaction, Self-service applications are only as strong (and profitable) as their weakest links and given the growing complexity and sophistication of today's IVR- and CTI-based systems, there are a lot of links to worry about. For far too many companies, the expected operating savings from these self-service applications fail to materialize as many customers opt out to agents due to performance problems. Almost everyone has a personal horror story about being stuck in an endless IVR loop, routed to the wrong person or repeatedly asked for the same information. None of this means that self service IVR and CTI applications (and ultimatel y, fully Web-enabled contact center operations) aren't the way of the future. It does mean, however, that when investing in and operating these new technologies, contact center managers need to be sure they are getting the most from these valuable workhorse systems.

How do contact centers ensure that they're getting what they pay for in self-service applications? The answer is both straightforward and cost effective: comprehensive, automated testing of IVR/CTI applications. This preventive measure is the key to making certain that self-service applications live up to their full potential in contact center environments. Automated testing in the design and predeployment stages of application development ensures that contact center managers pinpoint and fix scalability issues and performance bottlenecks before an application goes "live" (and contact centers end up testing their customers' patience and loyalty as they try to fix problems in production).

There are now product and service-based solutions on the market that enable contact center operation managers, application developers, QA engineers and technical staff to pinpoint and resolve performance problems before they undermine customer confidence and drive up operating costs. These automated testing solutions reveal hitherto expensive and hard-to-find scalability problems by emulating thousands of callers in real-world traffic patterns that accurately assess the performance of an application and its underlying voice infrastructure -- before and after the application is deployed.

Quality By The Numbers

Automated, predeployment load and performance testing helps contact centers fulfill multiple operating objectives. First and foremost, it takes a comprehensive approach to performance from the customer's perspective rather than providing a narrow view of whether each subsystem is working, it looks at an entire transaction from the PSTN all the way out to an agent. By measuring performance at each step of the test script under real-world conditions, automated testing finds application errors and performance bottlenecks by tracking the efficiency and interoperability of speech recognition servers, IVRs, database servers, middleware components, WAN/LAN network connections, switches, routers and ACD/load balancers.

Second, automated predeployment testing provides clear and detailed performance reports that help contact centers significantly improve the performance of their systems to ensure that vendors and service providers live up to their claims and that customers receive optimal quality of experience (QoE).

Pre-deployment testing is a fast, cost-effective way to boost performance and quality, ensuring that customers are happier about using self-service applications and that hidden operating costs are eliminated before an application "goes live."

Third, good testing delivers a firm foundation of archived data (and reusable test scripts) that provide a basis for corrective retesting, performance benchmarking and ongoing monitoring of self-service applications.

In short, predeployment testing turns out to be the most cost-effective way to ensure that voice applications and infrastructure will scale to meet performance and business objectives, thereby enabling contract centers to realize the full potential of their investment in self-service applications.

Four Steps To Effective Testing

The most effective and comprehensive testing processes generally follow a four-step protocol. The first step is to set the test objectives and design the testing methodology. The crucial requirements for this step are to understand the business requirements of the system (e.g., to handle up to 10,000 calls per hour) and the metrics to be used in measuring system performance. Detailed performance metrics should be developed for each of the components (e.g., 99 percent of all database response times are below three seconds) and these component metrics should add up to the overall performance metrics and an acceptable QoE for your customers.

The next step is to establish a project timeline and assemble an effective test team that includes, at a minimum, senior testers, database administrators, telephony architects and the application developers themselves. In defining both the timetable and the team's work assignments, managers should build adequate retest time into the project plan, since newly developed or updated IVR and CTI systems invariably manifest more problems than expected.

Once a project plan is in place and performance metrics are agreed upon, the test plan should be mapped against the system architecture to ensure that all mission-critical transactions and components are stressed during the test. To ensure that the system will continue to scale as the business grows, each high-volume or high-value call path should be exercised at a level of intensity based on realistic projections of future capacity needs.

The fourth step in the process is to execute the test to identify errors and performance bottlenecks. A typical test will consist of many iterative testing sessions in which the test is run, problems are found and the test is stopped so the problems can be fixed before testing resumes. Prior agreement on performance metrics and methodology will enable the test team to quickly work through the issues and fine-tune performance.

Once every component of the system is stressed and fine-tuned and follow-up tests indicate that users will enjoy adequate QoE, all performance issues and improvements should be documented to facilitate problem-solving in future tests or in a production environment.

Common Obstacles To Effective Testing

Unfortunately, a host of internal obstacles can impede -- and sometimes prevent -- effective testing of new or existing applications and infrastructure. Many companies lack the tools, budget or staff to conduct testing or often are under such right deadlines that the testing process is curtailed (or even eliminated entirely). Other companies may have the tools, but lack technicians who know how to use them. Even those companies that possess both the tools and technical talent often lack the expertise needed to design tests that can effectively stress the system and find performance issues.

Perhaps the most daunting obstacle is the sheer organizational complexity of today's contact center environment. Different vendors supply different parts of the system -- parts that are often cobbled together by a third party. Key components (IVR, database, switch, etc.) may be geographically dispersed and under the control of different parts of the organization, or even outsourced. Potential problems can occur in any component -- the switch, IVR, CTI, back-end databases, etc. -- or with the network connecting the components.

Of course, simply recognizing that these obstacles exist -- and addressing them in a comprehensive plan -- can help a contact center minimize their impact on system quality and performance. In addition, test managers should make every effort to ensure not only that everyone agrees in advance on objective performance standards, but also that test data are distributed as widely as possible, so all interested parties are working toward the same goals.

Managed Testing Services Can Verify Scalability Quickly And Affordably

One of the easiest, quickest and most affordable ways to overcome these internal hurdles is to enlist a qualified third-party to test new or existing applications and infrastructure. Because call volumes can be generated over the PSTN, the setup process is relatively fast and simple. Often, companies testing internally check only for average volumes on an average day, which often causes testers to miss many common but expensive problems. Another common pitfall is to test only for those circumstances in which callers do everything correctly rather than conduct" negative testing" -- or testing miscues by callers. Third-parry testing experts typically design many different scenarios to find problems that may only occur under a certain set of traffic conditions, after a certain amount of run time, at particular times of day or under different types of load.

Testing companies can also provide a suite of tools that allow contact centers to access and analyze performance data in real time -- helping everyone, regardless of geographic location, get on the same page and examine performance of all aspects of the application and infrastructure (switch, database, etc.). Effective testing can speed the application development process and reassure companies that their application will scale. Testing also provides the necessary performance data to verify that vendors and service providers are living up to service agreements or ROI promises. Testing companies can also document performance improvements and store these test scripts and results online, providing easier access to interested parties throughout the entire development and production life.

Getting What You Pay For

Whether performed by outside experts or by a properly trained and equipped internal team, automated testing of IVR, CTI and voice recognition applications is an indispensable step in ensuring that these important contact center technologies live up to their full potential. Without a sensible testing plan, self-service applications can actually exacerbate the very customer service and operating cost problems they are supposed to solve. With the rich and derailed data provided by automated, end-to-end transaction, testing, contact center managers can enhance, their customers QoE, free up agent resources and rake charge of their toll and hardware costs. For contact centers and callers alike, automated resting is a "win-win" proposition.

Charlie Grzybinski is senior voice consultant at Empirix (www.empirix.com), a test and monitoring company for voice, Web and network applications. He has over 13 years' experience consulting to Fortune 1000 customers regarding test strategy and execution for Web, voice and network applications and infrastructure.

(1.) Benchmark Portal, April 2002.

For information and subscriptions, visit www.TMCnet.com or call 203-652-6800.