Printer Friendly

Evaluating trade promotions.

As budgets tighten and costs continue to mount, trade promotion organizations and enterprises will be under increasing pressure to justify their expenditures. And good managers will be looking harder at the question of how to spend their resources most effectively. This should lead to greater use of evaluation procedures.

This article offers some guidelines for designing evaluation exercises or -- preferably -- ongoing evaluation programmes. It should also be useful to organizations and enterprises for reviewing the scope and effectiveness of their existing evaluation practices. Although this article concentrates on the evaluation of participations in trade fairs, and touches on evaluating some other forms of promotional communications, the basic approaches described here can be applied to evaluating other trade promotion activities.

Why evaluate?

The opening paragraph of this article mentioned two reasons for conducting evaluation exercises: to justify expenditures and to help decide how to spend most effectively. There are, in fact, a variety of reasons to evaluate, or benefits to be gained from evaluations. It is important to be clear on one's reasons before undertaking such an exercise. The specific purposes of an evaluation exercise should help to shape its design, to decide which elements of an activity and its results will be evaluated. Some possible reasons for conducting evaluation exercises:

To justify the existence of your organization or section or post; to win increased or at least continued support. This objective is rarely stated, and sometimes may even be subconscious. At any rate, it can be counted on to lead to an evaluation with positive findings.

To be seen to be evaluating. Since evaluation is widely accepted as a "good thing," a sign of modern, conscientious management and astute stewardship of public funds, it does an organization no harm to be seen to be evaluating, perhaps to have a full-fledged evaluation unit, even if the evaluation findings tend to be somewhat vague and not widely publicized.

Evaluations done for the above reasons might be considered as public relations exercises, or, less charitably, as political or bureaucratic ploys. But there are other, more solid, reasons for conducting evaluations. These are based on the concept of using evaluation as a management and policymaking tool. They include the following:

To help select the most effective types of activities for achieving particular tactical objectives. A record of evaluations conducted over a period of time can provide guidelines for selecting one or a mix of alternative activities, such as trade fairs, solo exhibitions or missions, to most economically achieve specific objectives such as introducing new exporters to potential agents in a foreign market.

To select the most effective and appropriate specific event or medium for achieving tactical objectives (a specific trade fair, a specific trade journal and so on).

To improve your organization and methods by identifying its strengths, weaknesses and problems. How accurately are you budgeting and controlling costs? Why do your trade fair displays sometimes arrive late? Do you attract as many visitors to your stands as other countries do to theirs?

To make optimal use of your resources for achieving your strategic objectives. An evaluation programme can help to achieve this by delivering the above benefits, and also by providing guidelines for resource allocation -- a picture of comparative cost-benefits of activities and programmes in terms of industries or types of enterprises assisted, target markets and so on. Do you get a greater increase in export value per dollar spent by helping the handicraft sector or the manufacturing sector ... small companies or large ones ... new-to-market companies or experienced exporters? In which markets have your promotional efforts been most effective?

To sum up, the evaluation process can help management answer two vital questions: Has your money been well spent? How can you spend it more effectively?

An on-going process:

An intensive evaluation exercise can be costly and time consuming, so it is not feasible to evaluate all activities with equal intensity on an on-going basis. Intensive, thorough evaluations of specific types of activities or major single events will have to be done on a periodic, possibly infrequent, basis, if at all.

However, for many types of activity, enough data can be collected regularly without great cost to provide useful indicators for planning work programmes and improving performance.

Even if formal evaluations are carried out only occasionally, maintaining continuous, systematic collection of data is highly desirable. In many cases the findings of an evaluation exercise have very limited significance in themselves; they must be interpreted by comparison with the results of other events over a period of time. So procedures for collecting comparable data over a period of time are necessary. Such data will also be necessary for evaluating whole programmes (as opposed to individual projects or activities).

To achieve this systematic collection of comparable data, the process must be an integral part of required, standard operational procedures. This will often be seen by staff as a bother, and they should be helped to understand the value of these procedures. Apart from being necessary for the evaluation process, the systematic collection of data on costs and results can help to instill a result and efficiency-oriented mentality in an organization.

Objectives and results

Establishing objectives is -- or should be -- the starting point for planning trade promotion activities. And the stated objectives provide a basis for evaluating the results of an activity; the results are measured against the objectives, as well as against the costs.

It is obvious that the ultimate objective of an export promotion activity should be an increase in exports. Unfortunately for evaluators, it is rarely possible to measure the ultimate impact on exports of a single activity such as a participation in a trade fair, let alone an entire trade promotion programme. Too many other factors enter into play, including economic conditions in the market, events and conditions at home, and other sales promotion activities that may be carried out by the trade promotion organization or by individual enterprises.

Common sense tells you that if you have been carrying out an active promotional programme in a market for several years, and there has been an increase of your exports to that market during this period (beyond the historic growth trend), your activities probably contributed to it. And if there was no increase in exports, you can fairly surmise that your promotional efforts have not been effective. But your evaluation requirements usually demand more precise and solid answers.

Generally the most feasible way to deal with this problem is to frame your objectives and base your evaluations in terms of causing "desirable events," which are often called "immediate objectives." These are events that must happen if the ultimate objective is to be realized, or that at least can be expected to contribute to its realization.

Examples of such desirable events are visits to the exhibition stand by "qualified visitors," requests for catalogues, sales to new customers and so on. ("Qualified visitors" are the targets for the exhibitor, people who can help achieve the exhibitor's objectives. The qualifications should be defined when objectives are set.) The box on page 16 contains more examples of desirable events, expressed as indicators of results.

Quantify objectives:

Objectives should be expressed in quantitative terms, to the extent that reliable data on results can actually be obtained in these terms. This helps to measure cost-effectiveness and to compare results of different activities. Some examples of such objectives are:

"A minimum of $50,000 of confirmed orders during the event, as a result of contacts made at the fair."

"Sales to at least 20 first-time buyers."

"Contacts with a minimum of 100 importers, wholesalers or retailers."

"At least 80% of the identified target audience will have visited the exhibition."

"All members of the target audience will have been exposed to our advertising at least once."

"At least six new-to-the-market companies will sign or have started negotiating representation agreements."

"Information on trade prices of at least ten competing suppliers will be obtained."

"All of the exhibitors will have received either positive confirmation of their product's acceptability or information on the need for specific product improvements."

"Advertisers will receive an average of at least five trade inquiries per issue of the magazine."

"A minimum of 80% of subscribers to the export bulletin will renew their subscriptions annually."

"At least 50% of the people on the mailing list will request additional information."

Check the objectives:

Evaluators should not take the objectives for granted. The setting of objectives is an important and common area of weakness, so the process of evaluation should include an examination of the objectives themselves. Were the objectives of the event or activity, even if they were met, worth the cost? Were they realistic? Were they appropriate for the type of event? (Appropriate objectives for a trade exhibition, for example, would usually not be the same as for a store promotion.) Were they sufficiently precise? Were they consistent with longer term strategic objectives?

Over a period of time the results of evaluations should provide a basis for formulating increasingly realistic and useful objectives. You should develop benchmarks. For example, how many trade inquiries it would be reasonable to expect in response to mailing a catalogue; how many visitors to your stand at a given fair should be your minimum target, and how much in immediate sales; what your attendance should be for a solo exhibition.

It would be helpful to have generally established benchmarks for export promotion activities, based on the experience of many countries -- for example, the average cost per qualified visitor to a trade fair stand. Some data of this nature is available from trade fair organizers, exhibition industry and publishers' associations, and similar sources, but it is of limited value to developing countries, because it is generally based on experience of companies in or from industrialized countries.

Relating results to costs:

When you are evaluating results of promotional activities, you should relate them to the cost of achieving them -- their cost-effectiveness. This is the common denominator management needs to help choose its promotional options. The exhibitions manager may be happy to report that $50,000 in firm orders were written during a fair, or that ten qualified importers visited the stand. Senior management, or the finance office, may be less happy to learn that it cost $60,000 to achieve these sales, or $6,000 to meet each of the ten importers; they may become downright unhappy with these results if they compare them with, say, the results of a trade mission to the same market that cost $40,000 and resulted in $100,000 in immediate sales and contacts with 20 qualified importers (at a cost of $2,000 per importer).

Full cost data should be collected as part of the evaluation process. This should include indirect costs, such as staff time required for preparations. It should also include the costs incurred by the participating companies.

Many trade promotion organizations in developing countries participate in trade fairs with financial assistance from overseas sources, and may tend to overlook the cost of this assistance. This is justified (from the evaluation point of view) if the assistance was tied to participating in a particular fair and could not otherwise be used. But if the benefiting trade promotion organization has any discretion in how it uses the available financial resources, it should count it into the cost of the activity.

Evaluating the operation

Apart from evaluating the results of an exhibition or other specific activity, or the longer term results of a whole programme, you should be evaluating how well they are planned, organized and executed. This obviously has a direct bearing on cost-effectiveness. Evaluation in this area is more concerned with qualitative assessments than with quantitative data.

Apart from analyzing results, a thorough evaluation of trade fair participation would cover the following:

1. The decision to exhibit: Was it well considered? What about...

* The suitability of the market(s) for your exporters.

* The suitability of the fair for meeting your objectives.

* The decision-making process itself. Was the decision based on adequate market information and supply data?

2. Selection of exhibitors: Is their profile consistent with your promotional strategy (in such terms as type of business, experience in exporting, experience in the particular market, readiness to export, size, ownership and so on)?

3. Selection of products: Were they suitable in terms of present and/or expected demand, price, quality, supply, importance to your economy?

4. The objectives. Were they...

* Clearly stated?

* Consistent with your position in the market, and your promotional strategy?

* Realistic?

5. Follow-up actions by exhibitors: Exhibitors must follow up on contacts and sales made at the fair if their participation is to have a lasting impact. So for evaluation purposes it is useful to establish what follow-up actions have been taken. Have exporters corresponded with persons met at the fair? How many of them have or plan to visit the market again within one year or one and a half years after the fair? Have they had visits from contacts made at the fair?

6. Effectiveness and efficiency of the organization and arrangements: This should be examined with a view to identifying weak spots and improving performance in the future. Aspects that might be considered include:

* Market research conducted before the exhibition.

* The visitor promotion campaign and its response.

* Logistics, housekeeping, adequacy of the stand staff.

* Briefing of exhibitors.

* Readiness of exhibitors.

* The stand or pavilion: adequacy of size, location, design, decoration.

* Budgeting and cost control efficiency.

Getting the information

Carrying out a useful evaluation exercise requires a great deal of information. Some of this is background information that in a properly run organization would be recorded and filed as a matter of course. And some (such as market and trade information) can be collected from documentary research sources. But a great deal of the information that is needed must be collected specifically for evaluation purposes. This requires planning and preparations before the event, and conscientious pursuit during and after the event. Deciding to evaluate the results of an event after it takes place, without such preparation, is not likely to result in a very meaningful evaluation.

Here is a list of the information needed for a full-fledged evaluation of trade fair participations, and some suggested sources:

1. Background information: The original project proposal, market research reports, reports of previous events, press reports, trade statistics and so on.

2. Profiles of exhibitors: The exporters register (if it exists), supplemented by information provided by the "Application to Participate" form.

3. Information from exhibitors: Collect as much information as possible from exhibitors before, during and after exhibitions by means of personal interviews, conducted with the aid of questionnaires. During the exhibition, daily meetings should be held to provide fresh feedback.

The reliability of information obtained from exhibitors will vary greatly, depending in large part on the relationship that has been established by the trade promotion organization (TPO) and its personnel with the exhibitors. Information from exhibitors should be cross-checked as much as possible, by comparing their daily and summary reports, their reports with the stand manager's observations and with information from visitors, and so forth.

Standardized forms should be designed to elicit information that is clear, quantified to the extent possible and comparable. The following forms are suggested:

* Application to participate. Ideally, trade promotion advisers should assist in filling this out, formulating objectives and so on. Possibly supplementing an exporter's profile in the exporters register, information in this form should cover the exhibitor's past and current activity in the target market and other overseas markets, representation already established, past fair experience and the like.

* Contact forms or summary of contacts. Exhibitors, as well as stand staff, should be encouraged (if not obliged) to fill out a contact form for every contact with a "qualified visitor," and to turn in a copy of each at the end of each day. If exhibitors are unwilling to reveal the identity of their contacts, they should be provided with forms that can summarize the nature of each contact.

* End-of-show questionnaire. This summarizes what happened. Generally exhibitors are asked to complete this before they depart for home.

* Follow-up report. The report should be written six or nine months after a show, and possibly a follow-up one year later.

The provision of information about results can be obligatory, written into the participation agreement, but this does not guarantee that full information will be provided by exhibitors or that it will be accurate.

4. Central visitor registration: This is easy to arrange at a solo exhibition, and sometimes is feasible at trade fairs. Having a prominent general information booth on a national stand can facilitate this. Using handout gifts or a sampling counter also helps round up visitors. The traffic flow can be designed into the stand to channel visitors past a registration point, but this should not interfere with the stand's other functions.

5. Observation by the stand manager, other TPO staff: This should include formal, systematic observation if information from exhibitors is likely to be incomplete or subject to question. For example, the staff can count the number of visitors on each company booth at specific times during each day, using a prepared check form. This data can later be compared with contacts reported by exhibitors to help estimate their accuracy.

6. Interviews by stand manager, staff: Interviews with a sampling of visitors should be conducted at least informally to ascertain their identity and importance, their interests, reactions to the exhibits, business conducted, how they learned about the presence of the national stand (effect of publicity, stand design) and so on.

Exhibition auditing firms can be hired. They use on-site observations, interviews and/or mail surveys to evaluate stand effectiveness in terms of such indicators as visitor number and profiles, reactions to the stand, recall of the stand and advertising messages, time spent on the stand and so forth. However, such services are used by governments only rarely because of their cost.

7. Follow-up interviews or mail surveys of visitors some time after the show. Ideally this would be carried out by trade representatives based in the host country and elsewhere in the exhibition catchment area.

8. Response cards enclosed in invitation mailings can help gauge interest.

9. Report of the fair organizer. Attendance data, audience profile and business reported by all exhibitors can be useful for comparing overall performance trends with that of your own stand.

10. Trade press reports of show attendance and results.

11. Analysis of trade statistics: Generally, such data is useful only for assessing the long-term impact of exhibition programmes over a period of time, rather than the results of individual shows, and there are many problems. Trade data that could reflect the impact of an exhibition is not likely to be available before at least 12 months after a show, and generally even later. Trade statistics are often not precise enough to match with the specific products exhibited. In some countries computerized customs data makes it possible to track performance of specific exporters who exhibited, but in many other cases specific exporters are not identifiable unless they are known to be the only ones for a particular product category. And there is no necessary link between the exhibition and the exports.

Evaluating advertising

The same basic approach can be applied to various forms of overseas advertising including publication of your own brochures, magazines and newsletters; advertising in various media; and direct mail advertising.

In a sense, evaluation of advertising is simpler than evaluating trade fair participations (certainly group participations) because the operational side is less complicated. But the results of advertising are often very hard to track. A lot depends on the type of response elicited. Evaluating the ultimate cost-effectiveness of advertising can become dauntingly complicated and costly, and impractical for a trade promotion agency or a small firm to attempt. But enough data can be collected to give at least some meaningful indications as to whether or not advertising has been worthwhile. If substantial amounts of money are going to be spent on advertising, an attempt at assessing its value should certainly be made.

The need to establish clear objectives is as important for advertising and publicity as for exhibitions. They are necessary not only to provide a basis for evaluation, but to help ensure that the activities are worthwhile in the first place, whether or not they are to be formally evaluated.

Criteria used in evaluating advertising in its various forms are: reach, response, commercial results, cost and qualitative assessments.

Reach:

This refers to the number of members of the target audience who see your advertising. This should be measured both in absolute numbers as well as a percentage of the total target audience.

A critical element is obviously defining and then identifying the target audience: the people who make decisions and carry out actions (such as buying or specifying), which result in your achieving your promotional objectives. It may also include people who influence this first group of people.

The extent of the total target audience can be determined or estimated through market research (desk as well as field research). A valuable source of information is the circulation data available from the media (particularly specialized publications) that are targeted at the same audience you are aiming at. If you take advertising in a particular publication, its circulation and readership data will give at least some indication of your advertising's reach.

For assessing the reach of the publications you mail out yourselves, you should analyze your mailing list in comparison with the target audience profile and available media data. But the mailing list is valid as an indicator of reach only if it is composed primarily of paying subscribers or if it is "controlled," i.e. if people must identify themselves and periodically request that they be kept on the mailing list. Otherwise it is likely that many of the people you mail the publication to do not actually read it.

Analysis of responses is useful as one indicator of reach, if enough data is available.

A postal survey of addresses can be used to identify the job functions of people in a company who actually read the magazine or newsletter.

For nonperiodic publications, such as product brochures that are not mailed to a regular subscription list, obtaining reliable data on reach is more difficult. The mailing list must be examined as regards its sources, the method of compilation, how it is updated and its extent. But this does not provide reliable data on how many people actually see the publication.

Response:

This refers to a defined, specific action taken by the reader (or listener, or viewer) as a result of seeing advertising or receiving the publication or other mailing. Measurable responses include writing for more information, entering a contest, accepting an invitation to visit a trade fair, renewing a subscription to your magazine and so on.

The advertisement or publication or direct mail shot must be designed to elicit these specific, measurable responses. Not all advertising is. With their limited resources, TPOs should be very cautious in producing material that is not designed to provoke a measurable response.

Reader reply cards can be used to help stimulate and measure response to publications and direct mail advertising. For advertisements in the press, reply coupons and keyed addresses can be used ("Reply to Dept. A...").

A major problem for a TPO in measuring the response to its publications is obtaining data on inquiries sent directly to companies. This can be avoided to some extent by giving the TPO's address, or the address of its overseas offices, as the reply address, but at the cost of a higher workload for the TPO and reduced efficiency. A better alternative is a periodic survey of companies advertised in the publications.

One of the best tools for confirming that response to your publication is satisfactory is to charge for advertising in it. If companies are willing to keep paying, it is fair to assume that they are receiving enough responses to justify the cost.

Results:

These are the ultimate results achieved by the advertising. Because you are mainly concerned with promoting exports or investments, the results (reflecting your primary objectives) should be of a commercial nature: orders placed, sales achieved, new foreign investments realized. These results are usually more difficult to measure than responses, so they are not as useful as evaluation tools, certainly when evaluation is carried out as a continuous process.

For a trade promotion agency, one way to assess the results of its advertising is to survey advertisers or other companies mentioned in TPO publications or advertisements, by mail or by interviews.

A survey of readers can also be used to give at least an indication of new business initiated by means of your own publication, as well as an idea of advertising reach, but this is not a practical technique for measuring the results of advertising in the press or other media.

Cost:

The total cost of advertising or producing a publication obviously must be taken into account when evaluating the activity. This should include not only direct costs, such as paying for space, graphics, printing and mailing, but also the indirect cost of staff time. For translating costs into measures of cost-effectiveness, two criteria are often used:

* Cost per target reader (or 1,000 target readers) reached.

* Cost per inquiry. This is generally more meaningful, if sufficient data can be obtained.

Qualitative appraisal:

Publications and advertising should also be appraised qualitatively. Do they give a desirable impression of your country as a supplier or investment opportunity? How professional do they look beside advertising of competitors? It is possible to assess such qualitative aspects in a way that produces measurable, comparable data, but this is generally prohibitively expensive for a TPO (except perhaps when there is an opportunity to participate in a multiclient research project). Nevertheless, it is worthwhile to sit down and try to compare the quality of your advertising material with that of international advertisers.

Bruce Bendow is ITC's adviser on trade promotion communications.

Indicators of results

A variety of indicators can be used to evaluate the results of exhibiting. They may be directly related not only to the volume of business concluded or initiated, but also to the visitors or the exhibitors.

Which of these indicators you use should be determined by their relevance to the type of products and the nature of the trade, the exhibitors' commercial objectives, the nature of the trade fair and the type of reliable data that is available.

Results can be measured in terms of:

* Costs.

* Results of other exhibitions.

* Results from previous participations in the same fair.

* Targets (stated objectives).

* Results of the fair as a whole (attendance and sales trends).

Indicators related to business done or initiated:

* Increase in exports of product types exhibited, within X years after the fair.

* Increase in market share for types of products exhibited, within X years after the fair.

* Value of confirmed orders written at the fair as a result of contacts initiated at the fair.

* Value of orders directly attributable to the fair received by exhibitors within nine or twelve months after the fair.

* Number of exhibitors receiving repeat orders within two years after the fair.

* Number of buyers purchasing from the country for the first time.

* Number of buyers purchasing from an exhibitor for the first time.

* Number of agreements signed with sales representatives during the fair or within 12 months after.

* Number of requests for price quotations or more information.

* Number of catalogues distributed on request.

Indicators related to visitors:

* Number of "qualified" or "useful" visitors.

* Number of contacts with prospective sales representatives.

Indicators related to exhibitors:

* Number of exhibitors making their first sales in the market.

* Number of exhibitors making sales to new customers.

* Number of exhibitors making their first export sales.
COPYRIGHT 1992 International Trade Centre UNCTAD/GATT
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 1992 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Bendow, Bruce
Publication:International Trade Forum
Date:Jul 1, 1992
Words:4636
Previous Article:Silk goods: selling to the Japanese market.
Next Article:The basic functions of national trade promotion organizations.
Topics:

Terms of use | Copyright © 2016 Farlex, Inc. | Feedback | For webmasters