Choosing and using a statistical consultant.

Over the course of many years providing statistical assistance, I've observed that 90 percent or more of analytic needs can be managed with relatively simple techniques. "Simple is better" is a good guideline for analyzing and reporting data. That does, of course, leave that nagging 10 percent. At some point, the statistical needs of your district or project may exceed local capabilities. You will need a statistical consultant.

The most common frustration I have heard voiced by statistical consultants occurs when they are called late in the process and must try to make sense of data that were not gathered with a plan. It may be the most common mistake in the use of a statistical data consultant.

Bringing in a statistical data consultant at the end of a project when all the data are in and asking her or him, in essence, "Can you make sense of this?" may result in dyspepsia on the part of the consultant and disappointment for you. In some instances consultants are forced to tell project managers that the data really do not answer the questions that were asked.

I was asked recently to analyze data for a multi-district project. The parties wanted to measure achievement gains based on a minor intervention. Consideration of outcomes at the beginning of the project rather than the end might have avoided a lack of results.

Thus the simple answer to the question "When do I call in a consultant?" is as soon as it is recognized that one is needed.

Choosing a Consultant

If internal analytic services are unavailable, you may need to go outside your school district to find a consultant. In many cases, contacting faculty at a nearby college or university will yield potential consultants. You are more likely to be successful by looking to social scientists who are facile with applied data analysis.

Selecting someone with the right skills for your needs may take a bit of effort. The individual's training and credentials certainly are a first level of evidence of the consultant's competence. Just as important is to find someone with whom you can work comfortably.

If your school district intends to apply for federal funding through one of the government's many funding initiatives, you will be required to build an evaluation component into the project. That plan must include details about how you intend to manage data collection, analysis and reporting. You should consider a statistical consultant in the planning as you construct the funding application, especially as the complexity of the likely analyses increases.

The central element to successful use of a statistical consultant is that the process from the beginning uses two-way communication. The process must be collaborative. Therefore you should assess whether (1) the consultant listens to you and (2) the consultant can translate the abstract world of statistics into useful and understandable information for you and your district.

Be sure your consultant is willing and able to communicate results to the multiple audiences that must be informed. A complex analysis will be appropriate and important for the funding agency. However, it will be of little value to you for a presentation that is scheduled in one week before the board of education. Is your consultant comfortable communicating to both audiences?

Statistical knowledge in the absence of the ability to translate that knowledge into an understandable message is useless.

Clear Expectations

Your consultant must be able to understand your setting and translate the outcomes of more complex analyses into meaningful English. Do not be afraid to ask the consultant to slow down and speak in terms that you understand. All disciplines have their own jargon. As professionals we fall into the use of jargon forgetting that others outside our discipline may not share the same concepts. If I start talking about problems of "multicolinearity" and you quietly nod your head, I'll probably continue babbling and lose you in the process.

This assurance of understanding goes both ways. I find it often takes a bit of effort to ensure I understand just what the school's or district's analytic needs are. These are frequently complex matters and our respective jargons may not match. We may use the same words to refer to different concepts.

I once worked with an urban school district to create a set of quality indicators for schools. Principals immediately voiced concerns that their school would be unfairly jeopardized because of high student turnover. Together we created a set of categories to define the varied forms of student mobility. We then were able to show that high student mobility contributed to lowered achievement. The quality indicators were interpreted relative to student mobility.

The data also triggered policy discussions about how best to respond to problems brought on by high student mobility.

Be sure you make clear that you expect your consultant to leave a paper trail describing the analyses that she or he conducts. If the consultant has wrestled with missing data, for example, how did he or she deal with the problem? To the degree that the consultant creates specific indexes or measures, it is critical that he or she leaves clear documentation of how those indexes or measures were derived.

Desireable Qualities

By and large, the need for a statistical consultant arises in the few cases in which a higher level of statistical competence or research and evaluation design is needed. If you know that the services of a statistical consultant are to be needed, engage the consultant as early as feasible. In looking for someone to serve that role, you certainly want to work with a person who possesses knowledge of data analytic techniques. In seeking a consultant, choose one:

* who is responsive to your needs and who listens;

* who can communicate with multiple audiences;

* with whom you can work comfortably; and

* who fits well with what you want to accomplish.

Gary Ingersoll is a professor of counseling and educational psychology at Indiana University, 4024 Education Building, Bloomington, IN 47405. E-mail: [e-mail]