Is your research any good? (a) yes (b) no (c) don't know: how to ensure that your data is accurate, actionable, timely and above reproach.

Marketing research is a powerful management tool. On the strategic level, it can be used both for risk management and decision making. It can, for example, help a financial institution judge the bank's positioning in the marketplace and determine if it should target a new customer segment. On a tactical level it can help a bank decide whether it needs to:

* Introduce a new ad campaign.

* Change a delivery channel.

* Take steps to ensure that it is meeting customer needs.

Too often research is used to validate a decision that has already been made, rather than used as a tool to guide the decision. This approach diminishes the importance of research, encouraging management to cut its funding--further reducing the chances that whatever research is done will be performed properly. Among other things, skimpy resources lead to a compressed timetable for completing the project which, in turn, results in inappropriate data collection methods and, many times, superficial data. Inadequate data further undercuts the perceived value of research.

For marketing research to play an important role, management must have confidence that the information is accurate, actionable, timely and above reproach. Here's how to ensure that your marketing research project meets all of these criteria.

Determine the purpose

A marketing research project starts with a need for information that will either guide a decision or provide the data necessary for determining whether certain actions will be taken. Hence, it is critical to determine clearly the purpose of the research and how the information will be collected, reported and analyzed.

Many times research projects are launched without a well-defined purpose. This shortcoming leads to the use of inappropriate methods for collecting and reporting information. The information collection method is often decided on the basis of cost and not on the ability of the method to generate accurate and credible information to satisfy the purpose. This situation is why, when the collected information fails to meet the intended need, management starts to question the usefulness of research as a decision-making tool.

Set the objective: Defining and setting the objective is the first step in conducting successful marketing research.

One of the most common objectives is to determine whether the bank is meeting customer needs and, if not, where it is falling short.

Many times bank retail personnel develop an objective that is too broad and confuses the objective with the data collected. An example is management launching research to measure customer satisfaction. The objective does not set guidelines for how the information will be used, thus making it difficult to determine the most appropriate data collection method. This can lead the researcher to select a data collection method and number of interviews that do not meet management's underlying objectives.

For example, using the above objective, a marketing research company executed a telephone interview among checking account customers. In order to control expense, the researchers kept the interview short, at less than 10 minutes, and included questions covering overall satisfaction with the bank and agree/disagree statements.

The researchers completed 500 interviews in total. While the project was sufficient to measure overall customer satisfaction across the customer base, management did not know how it could effectively use the data. Management's original unstated goal was to provide feedback to the branches concerning information directly related to the branch. The research was not conducted in a manner that provided a sufficient number of interviews to provide feedback at the branch level. In addition, management wanted to identify and source the problems encountered by its customers.

Had the research plan specified how management wanted to use the information, the researchers would have evaluated the merit of a mail survey and included questions asking customers to describe the problems they experienced. In addition, the researchers would have completed a sufficient number of interviews by branch so that management could have reported to branch managers on specific branch-level performance.

A meeting between the market researcher or market research department and the intended user of the information is usually a big help in setting a well-defined and clear objective. During this meeting the parties should define the issues and questions that need to be answered and determine when the information is needed. It is particularly important to understand how the information will help management guide its decision and how the information is intended to reduce uncertainty associated with the decision.

Getting it right: Let's say management is considering how it can increase its share of wallet and fee revenue. As such, it is considering the options of training and licensing platform staff to sell insurance and investment products versus hiring dedicated personnel to sell the products. At a meeting with the marketing research department, management outlines its objective and provides the information it requires from customers and noncustomers and employees before making its decision. Based on this information, the research department develops a plan and timetable for providing management with the requested information.

Example: A financial institution needs help in deciding whether or not to change the checking account package it offers consumers or small businesses. Marketing research can help the institution decide whether or not the new packages are preferred by customers and noncustomers over the current products.

To help determine which of two checking account packages to introduce, researchers decide to measure consumer acceptance of the checking account packages. A package will be considered better or preferred if consumers prefer package "A" to package "B" by 10 percent or more at the 95 percent confidence level. (A 95-percent confidence level means that if the research was replicated 100 times, the results would be the same 95 percent of the time.)

Determine the Methodology

Selecting the method to collect the data. Selecting the right method for data collection is vitally important to meeting the research objective. Without a proper method for gathering the data, the findings produced by the research project will be flawed and inappropriate decisions can be derived from the data. The researcher should review the objectives of management and the information that needs to be collected as well as any critical decision criteria that will be used in determining the most appropriate methodology.

There are many methods of collecting data. The most popular are listed below. All have their benefits and deficiencies.

* Mail surveys. The surveys are cost effective, provide a framework for thoughtful input, and provide customers with a sense of comfort associated with knowing the survey is from the bank and that the he or she can provide the information in private. The mail, however, typically elicits a lower cooperation rate than other methods, 10 percent to 25 percent. A key point to consider is whether the consumers who respond to the mail survey are likely to have the same opinions and attitudes as those that do not respond. If the answer is yes, then the data gathered from the respondents returning questionnaires should be representative of the population. Another key point to consider is timing. Mail surveys typically take longer owing to the time it takes to mail questionnaires, mail reminders to complete and return questionnaires, and then receive the completed questionnaires.

* Telephone surveys. Telephone surveys still offer the most representative samples and provide the researcher the ability to not only quantify information such as consumer awareness, usage and attitudes, but also probe into the reason for a consumer's beliefs or actions. The ability to probe and clarify consumer responses during the telephone interviewer is a key strength of the procedure. Surveys can be conducted through random-digit dialing of telephone owning households in a defined area or via customer lists with telephone numbers. Cooperation rates have declined recently in step with consumer concern for privacy and issues related to telemarketing. Using a customer list and providing the bank's name at the beginning of the survey and keeping the interview short helps ensure cooperation.

* Mystery shopping. Unlike an interview with a customer or consumer where the interviewer records information obtained from a respondent, the shopper or interviewer is the respondent since he or she poses as a customer or consumer and records the information provided by the employee. Unlike customer surveys, mystery shopping is free from customer bias and, if conducted properly, provides information as to what really happens when branch and telephone representatives meet or speak with customers and non-customers. Mystery shopping also provides one of the few ways to measure the sales process and determine why prospects reject the bank. Even if the bank had an accurate prospect list, the survey would be limited to the prospects ability to remember what happened and is biased in the sense that prospects would have an agenda of getting the best deal possible from the bank.

* Web surveys. The use of the Internet or the Web to conduct research continues to grow. Many panels of consumers exist to conduct Internet-based research. The internet panels are particularly helpful for general population surveys and surveys targeted at hard-to-reach consumers, e.g. doctors or other busy professionals. They fall short when the institution requires information on branch-based service since most institutions do not have accurate and sufficient numbers of customer e-mail addresses.

* Personal or exit surveys. The surveys are either conducted in a branch, when a customer leaves a branch, or in shopping mall setting. The branch-based interviews are particularly helpful for targeting customers of a branch and gathering information about the branch experience when it is fresh in the customers' mind. Shopping mall surveys are helpful for intercepting consumers to participate in a survey requiring some type of visual stimulus. The branch and shopping mall surveys, however, only represent the consumers visiting the branch or mall that day and, hence, the data user needs to assume that consumers who visited the branch or shopping mall that day will provide the same information as those that did not.

* Focus groups. This type of research can be conducted in different markets with eight to 10 consumers per focus group. They help management stay close to its market, understand a topic, confirm its understanding, or point out issues not understood. They are particularly helpful for understanding consumers and customers' views and decision making and are often a prelude to a quantitative market research project. The information, however, is qualitative in nature since it is based on a discussion with small panels of consumers. The opinions voiced during the discussion can often be colored by one or two dominant participants and should not be viewed as representing the bank's customer base. Therefore decisions should not be based solely on the information produced by focus groups.

Determining the right number of interviews

The number of interviews completed depends on the purpose of the research and the decisions that will be made using the information. To ensure that the information is actionable, the researcher must complete a certain minimum number of interviews to provide management with a level of confidence for making decisions. Before launching the research, the researcher should set action standards for the research and the information it provides.

Many times the number of interviews conducted is based on cost. Statistical reliability and setting action standards to make decisions take a back seat to cost and the time the information is needed. Sometimes this is done because management is using research only to validate a decision it has already made. For example, while asking 50 consumers to compare and evaluate the usability of a new bill pay service to the current service may be more cost effective and faster than conducting the survey with 200 consumers, the results may be entirely different. The sampling error surrounding interviews with 50 consumers leaves much more room for error than interviewing 200 consumers (14 percent sampling error versus 7 percent at the 95 percent confidence level). In addition, interviewing 50 consumers leaves little room for examining differences by subgroup, e.g. high-income versus lower-income consumers. The inability of the research to identify differences in preferences between subgroups may result in management alienating one of the customer segments when it makes a decision to change bill pay service.

Developing the questionnaire and the analytic approach

Ensuring the questionnaire is easy to understand and administer and free from bias is vitally important to collecting action-oriented marketing information. Marketing research surveys are often too long and contain questions with little relevance to the research objective.

In order to get the best "bang for the buck," a bank may notify different departments about the survey and include questions from departments that have little to do with the research purpose. This lengthens the interview and ultimately the level of cooperation and the accuracy of the answers provided by the respondents.

Key points to remember are to keep the questionnaire short and easy to understand and administer. It is important to ask the right questions in an order free from bias. The answers customers provide in the beginning of a survey influence the answers they provide later in the survey. The order in which questions are asked can bias the results and provide misleading information.

Asking a customer how satisfied he or she is with the institution at the end of survey may result in a completely different response than asking overall satisfaction at the beginning. The satisfaction measure taken at the end of a survey will be biased as the customer will have been introduced to a variety of topics which may or may not be important to the customer. Asking customer satisfaction at the beginning of the questionnaire limits bias as the customer will think only of the topics important to him or her when answering the question.

Determining the reporting and analytical approach

Selecting the right analytic and reporting approach can make or break a research project. The approach for reporting and analyzing the data must meet the objectives of the research. It must describe the key decision criteria and point out patterns in the data important to the key decision makers. In a research program designed to motivate branch service relative to goals, it should quantify customer satisfaction and report on the level of service quickly relative to goal. It should tell the staff" what they do well and the areas where they can improve.

Many times, bank personnel will use the information for inappropriate purposes. For example: Using a program that was designed to motivate branch service to measure overall customer satisfaction and to detect and correct problems not related to branch- level service. Core customer satisfaction with the bank can be very different than satisfaction with branch-based service. Inappropriate uses of this sort, again, can lead management to question the results and the action ability of the data collected.

By following the above steps, researchers can help ensure that their marketing research is done properly--and that management will view the results as an action-oriented decision-making tool that is vital to the success of a financial institution.

Paul C. Lubin, is senior vice president of Informa Research Services, which has its headquarters in Calabasas, Calif. He manages the Barry Leeds and Market Trends divisions of the company. Telephone: [phone]. E-mail: [e-mail]