A global agenda: the world gets serious about communication measurement.

IABC members played an important role in a recent global survey on communication measurement conducted by Benchpoint, with the help of Donald K. Wright, professor of communication at the University of South Alabama, USA.

PR measurement is a hot topic. Most conference programmes now have a section on measurement, where the latest high-tech, real-time media evaluation tools are rolled out and people debate the best methods.

The Measurement Summit, held annually in New Hampshire, is an international meeting devoted entirely to measurement. To put the ongoing debate into some kind of global context for the 2004 meeting, Benchpoint Ltd. decided to try and find out who is measuring what, which tools are working and which are not.

More than 1,000 communicators in 25 countries responded to an online survey organised by Benchpoint. IABC responded enthusiastically to the call, as did the LI.S. IPR and many national associations in the U.K., Germany, Italy, Poland, Eastern Europe, South Africa and Australia. The influential Arthur W. Page Society and the International Public Relations Association (IPRA) also supported the exercise.

EXECUTIVES PUSH FOR MEASUREMENT

The good news is that nearly 70 percent of the world's professional communicators working in PR say they measure the effectiveness of what they do, and 61 percent go so far as to say that measurement is an integral part of the PR process. There were no significant differences among regions or job functions. Everyone agreed.

A cynical response might be, "Well, they would say that, wouldn't they?" But as we dug deeper into the results, an interesting picture emerged.

The movement toward measurement is being led from the top. Board directors and CEOs are slightly more likely to say, "measurement is part of the PR process" and "we will be doing more measurement in the future" than other employees. In fact, 27 percent of all respondents say they will be doing more measurement in the future, and a massive 77 percent of those who say they do not currently measure, plan to do so in the future.

That still leaves about 22 percent who do not plan to measure anything, and we'll examine their reasons in just a bit.

So what are the measurers measuring?

To get a clearer picture, the survey was split between external communication and internal communication.

EXTERNAL COMMUNICATION

Professional communicators currently measure outputs rather than outcomes, and the majority rate the effectiveness of the tools they use as "uncertain" or only "somewhat effective," which suggests that there is a lot of scope, both for tool refinement and the education of measurers.

* The survey asked, "What criteria do you use to judge if you have been successful?" Respondents answered that achieving the desired outcome, hitting target media and having an effect on awareness and image is more important than being on time/budget, although the scores were close.

* There was a much greater divergence when it came to the actual tools used for measurement. Media evaluation, internal reviews and benchmarking are the most-used tools, although one could argue that these are measuring outputs rather than outcomes. Focus groups and opinion surveys, which actually measure outcomes, got lower scores than AVEs (advertising value equivalents).

* Participants were asked which tools they regarded as most effective. Most tools scored around 44 percent for "uncertain" or "somewhat effective," which suggests that as far as the profession is concerned, the jury is out.

* Opinion surveying is viewed as the most effective tool, with higher scores for "somewhat" and "very" effective.

* Dashboards, league table rankings and AVEs rank as the least effective tools.

* Board level and senior practitioners are more likely to prefer dashboards and AVEs than their junior colleagues.

* Consultants and chief communication officers tended to rate the effectiveness of measurement tools slightly higher than their colleagues, including consultancy board directors.

INTERNAL COMMUNICATION

Internal communicators said that they measure more outcomes than outputs, presumably because it is easier to deploy the tools, but 23 percent still rely on instinct to judge their success.

* Employee feedback is the most-used criterion for success, followed by "desired outcome."

* Nearly 23 percent of respondents use "instinct" to judge whether they have been successful.

* Only 18 percent regard meeting the budget as a criterion for success.

* Employee surveys are the most used and effective tools. Focus groups, benchmarking and dashboards score significantly lower than other tools for internal communication.

* All of the internal measurement tools have higher, "very effective," scores than the external tools.

* North American communicators consistently rate the tools' effectiveness higher than their European counterparts.

* Thirty-one percent of internal communicators correlate the results of their measurements with HR data, and 24 percent correlate with marketing.

WHY DON'T PEOPLE MEASURE?

There are no surprises here. It's all about time and money, according to survey respondents.

* Cost is the most significant barrier with more than 47 percent of non-measurers saying it's a "major" barrier, and 30 percent saying it's a "very major" barrier.

* Time required for research is the next biggest barrier.

* Expertise and the questionable value of the results are about equal, at around 58 percent.

* "Uncertainty of what we might find" is not seen as a major barrier. However, reverse this, and correlate it with "question the value of the results," and one might see this answer as, "What's the point? We know what we're going to find anyway."

* There are no significant differences among organisation types.

* Europe is more concerned about cost than North America. North America is more concerned about expertise than Europe. There are no other significant regional differences.

However, there's an interesting twist here. Non-measurers hold the simultaneous belief that "measurement is an integral part of the PR process" and "cost is a major barrier to measurement." The gap between those who say "yes" to measurement and say that cost is not a major barrier is a massive 40 percent, with chief communication officers and board directors more likely to hold these views than other employees. Interestingly, CEOs are less negative on this point, which suggests they may be open to persuasion on the budget issue.

THE MEASUREMENT DEBATE: WHAT'S THE POINT?

All survey participants feel positive about the benefits of measurement and believe it is possible to calculate ROI on communication, but with some reservations. Although 65 percent of respondents think it is possible, only 13 percent strongly hold this view.

* But the popularity of this belief increases steadily as people consider the benefits--bigger budgets and more resources.

* Among respondents, 88 percent say they would be interested in the development of an ROI tool.

* There is little agreement with the propositions "you can't measure the intangible" and "professionals don't need measurement."

* The proposition "measurement is a good idea, but costly and impractical" finds support with 45 percent of communicators. Clients (not a statistically valid sample) tend to feel slightly more strongly than communicators, except on the practicality issue. So when it comes to a measurement budget, you might be pushing a (slightly) open door.

* This view is reinforced if you consider that more than 80 percent of all respondents believe that measurement is the key to demonstrating the value of communication (and, therefore, budgets and resources), and that measurement helps inform better strategies and policies.

* Seventy percent say they will be doing more measurement in the future.

* Seventy-eight percent believe that communication represents good value for money. Clients (not a valid statistical sample) also agree, but not as enthusiastically.

MEASUREMENT CONSCIOUS

This survey provides a snapshot of what communication leaders around the world think about measurement. (More than 800 participants asked to see survey results, and that list of people and companies makes for impressive reading.)

Communicators are very measurement conscious and would like to (indeed, intend to) do more. But while there is a broad consensus on the most effective tools, the fact that existing tools do not attract high scores for "very effective," and that non-measurers have serious concerns about cost, indicates that the providers of measurement tools have to do a lot of work to demonstrate both the relevance and the value of their measurements.

Cynics will muse that it's easy to support measurement in a survey, but when it comes to the measurement budget, it's altogether different.

Nevertheless, the survey reveals that measurement is a fundamental part of PR and communication--not so much because you can see how well you've done (which is nice), but so you can use measurement to inform and define what you should be doing, and how you can do it better.

Aligning communication objectives with the business objectives of the organisation and implementing programmes that truly make a difference--using relevant and effective channels, and then measuring the results--is a powerful formula for success.

Richard Gaunt is the CEO of Benchpoint Ltd. and principal of Gaunt Corporate Affairs & Business Communication in London. He has worked as a professional communicator for more than 35 years, and can be reached at [e-mail].