Managing successful speech projects: ten mistakes to avoid along the way.

Speech technologies offer call center managers important opportunities to improve customer satisfaction and reduce costs. According to industry analysts at Forrester, good speech-enabled phone self-service systems offer interaction advantages that even well-designed touch-tone systems can't match. Speech projects, however, introduce complexities--even risks--that must be carefully managed by the team developing and deploying the system.

[ILLUSTRATION OMITTED]

What are some of the common mistakes, and how can project managers and other team members work to avoid such challenges?

As with any project, speech recognition installations require a comprehensive methodology to eliminate risk and simplify the design, development and management of the project. One example of this process is the Nuance Global Engagement Methodology (GEM), a process that ensures optimal results at each phase of the lifecycle--from discovery of caller, business and system requirements through application design, development and optimization. For speech projects, a heavier focus is put into the discovery and optimization phases, as there are unique elements that drive subtleties in design from a caller's perspective, requiring additional attention while gathering requirements and rolling out the solution.

[ILLUSTRATION OMITTED]

The Discovery Phase

At the discovery phase, speech teams define the enterprise voice strategy and create the solution blueprint. In this phase, the organization identifies business, caller and system requirements and defines the relationship among the three.

In identifying business requirements, organizations must clarify specific business rules, list organizational and application goals and specify the metrics that will define application success.

End user, or caller, requirements demand a detailed examination of the range of callers: what do they know, what do they want to accomplish, and what expectations do they bring to the call?

System requirements, of course, specify what the system must provide in order to accomplish the business and caller goals most effectively.

Mistake #1: Insufficient education. Insufficient education may be the first mistake team leaders make. A voice-enabled application requires many complex tasks and several important skill sets. When incorporating a speech-enabled system into a call center environment, organizations are most effective when team members understand the theory and best practices underlying each task. Even if your team relies on experienced industry professionals to do much of the work, managers and team members will need to be conversant in the various tasks and technologies in order to contribute to the overall process and make informed decisions.

Project managers must understand the details of each project task and the work each task requires to estimate effort appropriately during the scoping stage. Knowing what it takes to develop a speech application helps managers see dependencies and determine the critical path in the project plan. Understanding the tasks allows a manager to bring together the right members for the project team. Understanding the skills required for each task allows managers to match tasks to skill sets when assigning project tasks and to plan for additional education where necessary.

Many people on a speech team have backgrounds in touch-tone IVR systems or Web interface design and development; they may assume that a speech application is a simple extension of what they already know. Their prior knowledge is certainly valuable, but conversational applications require knowledge of human factors, engineering, linguistics, design and programming. Successful project managers will define and initiate an education strategy for themselves and their team members as early in the discovery phase as possible.

Mistake #2: Failing to have the right people and the right discussion at the kickoff meeting. Another big mistake is sometimes made as early as the kickoff meeting. A kickoff meeting is notable, of course, as it represents the handoff of a project from sales to services and as it initiates the process of formal requirement gathering. If critical staff members aren't in attendance or if meeting leaders assume that everyone understands speech-related tasks, the meeting will not meet its full potential.

Project managers should ensure that the following representatives attend and actively engage in the kickoff meeting:

* Stakeholders;

* Decision makers;

* Voice user interface designer(s);

* Marketing representatives;

* Call center managers; and

* Systems administrators.

Discussion at the meeting should cover a series of topics, including project expectations and methodology, metrics and logging, and testing and tuning. Managers will need to explain what they need and when they need it.

Mistake #3: Creating design elements before the requirements are clear. Rushing design decisions without addressing a comprehensive series of design questions increases risk. A good design is caller-centered. It recognizes and addresses caller needs and expectations. It incorporates strategies to optimize recognition accuracy. Without clear requirements defined in advance, a system design will yield confusing or misleading prompts, as well as an inconsistent prompt style and error handling strategy. It won't contain the functionality specified in the requirements. The net result is that users may experience difficulty using the system.

The Design Phase

In the design phase, organizations transform critical requirements into specification documents that guide implementation, testing and tuning. The speech team selects the voice and personality traits that identify the solution and help define the client's relationship with their callers. You may even consider a custom test-to-speech voice to deliver dynamic information. Scripting a call flow that nonetheless adapts to various caller behaviors requires careful use of social psychology, linguistics and human factors and also a solid understanding of the limits and potential of speech technology.

Mistake #4: Creating prompts that are inconsistent with voice user interface (VUI) design principles. Successful speech systems adhere to the core principles of VUI design. The design includes a desirable persona speaking carefully worded prompts that guide the caller yet adapt to the caller's input. Good designers ensure efficiency and clarity by avoiding unnecessary words or complex sentence structures. They encourage barge-in so that callers can proceed when they are ready.

Good designers also aid accuracy by thinking ahead to grammar development as they craft prompts. They identify "natural language" parts of speech that a caller may offer in addition to the necessary information, such as, "Let's see, I'd like to return on October 27, please." They avoid similar-sounding responses in the same state; for example, "review" or "preview." They anticipate synonyms or relative phrases that callers are likely to speak: "January 17" may also be "Wednesday" or "tomorrow."

Good designers aid graceful error recovery through careful call flow design, as follows:

* Systems acknowledge that something went wrong: "My mistake," and that something went wrong again: "My mistake again."

* Systems respond when a caller makes a one-step correction, "No, the 18th."

* Systems are designed to offer context-specific help prompts or DTMF fallback to help callers continue through difficult collection contexts.

* Designers ensure low cognitive load by avoiding short lists that sound like yes/no questions, by avoiding long lists that are hard to track, and by avoiding confusing constructions.

Mistake #5: Failing to coordinate prompts and grammars. In specifying the grammar for each state, a designer should aim for broad coverage coordinated with prompts. Grammars must be revisited whenever the prompt wording is modified. The risk here is that a failure to understand the relationship between prompts and grammars may result in insufficient allocation of data or time resources. Grammar decisions must be tested on a sufficient number of potential callers. The test group must be truly representative of the actual caller demographics.

That said, good designers avoid total grammar coverage, since complexity can affect speed and performance. A better option is start with a basic grammar coverage and expand it based on real user data. To successfully coordinate prompts and grammars, choose the most appropriate grammar (or grammars) for each state and accommodate all reasonable caller input. And be sure to sufficiently test grammars and call flow.

The Realization Phase

To realize an application, organizations develop, test and deploy the solution. This phase involves grammar and application development, voice recording, database and telephony integration, functional testing, usability studies and staged deployment.

Mistake #6: Failing to coach the voice talent or tune the text-to-speech engine. Amateur voice talents and voice talents who have worked only on touch-tone systems may not be suitable for speech applications. Someone on the team should be prepared to coach the voice talent to ensure that the recordings effectively create the designed persona, that each prompt is spoken clearly, and that the progression from one prompt to the next produces a consistent and smooth effect. If an organization selects text-to-speech applications, individual recordings should be analyzed for quality, and time should be allocated to tune the output, if necessary.

Mistake #7: Shortening testing cycles. Given the complexity and cost of speech systems, some organizations are tempted to make up for time or budget slippages by shortening the testing phase or performing different tests in parallel (rather than sequentially). This strategy is a mistake that results in an incomplete, buggy or untested system.

Pre-deployment testing is critical at several milestones:

* Wizard of Oz (WoZ) tests are useful to validate the design concept and/or research callers. In WoZ tests, a human acts in place of an application; callers call what they think is an automated system, but the human is actually recognizing the input and directing which prompt is played next. Thus, these tests can be run very early in the design phase and the knowledge gained can save time and money later in the project lifecycle.

* Lab-based usability studies--onsite or via telephone--are critical tests for the system's effectiveness with callers. These tests require the application and grammars to be developed and the prompts to be recorded, though they are not real calls and no actual transactions occur. Any modifications defined post-usability must be implemented before regression and QA testing begins.

* Dialog traversal testing, regression testing, logging validation, and backend queries are critical for QA, as is load testing.

Testing should be addressed at the kickoff meeting, and the test plan should be prepared early in the process with frequent schedule reviews. Project managers ensure the system meets the original user requirements, validate early sizing and load estimates, and test in a production environment.

The Optimization Phase

At the optimization phase, organizations should monitor ongoing solution performance, which includes application tuning, grammar updates and business performance assessments. Ongoing optimization is critical to ensure a superior customer experience and peak usability.

Mistake #8: Failing to recognize the importance of pilot phase tuning. By identifying and resolving issues prior to full deployment, organizations can minimize the risks of caller dissatisfaction, costly rework and service disruption. But if organizations fail to develop a detailed plan with success criteria and a clearly defined end, the pilot phase can be plagued by insufficient data or documentation, data logging issues, or time and resource constraints.

Tuning improves an application's performance since it requires a careful analysis of field data, resulting in recommendations for improvement in one or more elements of the application. The best speech systems leverage the pilot phase as an opportunity to analyze the application's prompts and grammars, to consider the effects of recognition parameters, to consider overall system resources, to identify and correct pronunciation problems, and to re-evaluate dialog flow.

As it can be difficult to anticipate and exhaustively test for every potential situation and condition before a speech system goes live, you can expect to make changes. Your organization should have processes in place to identify and rapidly resolve the production issues identified during tuning. This process will minimize your callers' exposure to negative elements in the application.

Mistake #9: Poorly documenting elements among team members. Overseeing the communication and interplay among team members through every stage of the project lifecycle is critical for the overall success of any speech project. The VUI design specification document should always document what's in the application, so it should be updated regularly as changes are made to call flow, prompt wording, grammar elements, user-defined logging, database queries and so on.

Mistake #10: Underestimating the complexity of building a speech application. Speech can be complex. Launching a successful conversational speech application requires more than tools and software. Exceptional project management, strategic planning, application development and testing know-how, along with specialized expertise in areas like human-computer dialog, speech and signal processing, and speech acoustic modeling are essential to building effective speech solutions.

However, a proven model for managing successful speech deployments is to shadow an experienced project team and learn to emulate their proven deployment methodology. That way, your team will soon develop the skills they need to balance the art and science of deploying speech solutions.

Deanne Harper is manager of Speech University for Nuance Communications (www.nuance.com).

By Deanne Harper

Nuance Communications