What can one TV exposure do?
That a single commercial exposure could be effective remained an alternative theory - occasionally discussed yet largely ignored. Of course, a single-exposure effect was recognized as central to most copy-testing systems, but many advertising practitioners regarded this as a fundamental weakness of copy testing and a major reason to avoid it.
In 1994, the debate was rekindled. Powerful single-exposure effects were identified and estimated by John Philip Jones using single-source data (Jones, 1994; published in JAR, Jones, 1995). While many accepted Professor Jones's work, some questioned his analysis on methodological grounds.
The data presented here were developed by a private advertiser to assist in the evaluation of its advertising copy. The larger implications for advertising theory and research practice proved to be an important by-product. Since the findings resulted from tightly controlled experiments, they avoid the methodological questions raised about Professor Jones's analyses.
This work confirms the extraordinary potential of a single exposure of a television commercial.
The Data Base
In April 1976, General Mills conducted the first of a series of TRI-NET real world experiments to measure the absolute effect of a single additional exposure of a TV commercial. TRI-NET is a member of the class of real-world experiments - experiments in which respondents cannot know they are subjects in an advertising effectiveness study, either at the time of the commercial exposure or at the time of the research measurement.
Since 1958, General Mills had used a laboratory copy pretesting procedure known as SST to measure the relative effects of alternative commercials prior to final production. SST is based on "forced" exposure in which respondents know they are participants in an advertising test but cannot identify the test sponsor.
TRI-NET was intended to supplement rather than replace SST. While expensive, TRI-NET addresses many of the questions raised about SST. TRI-NET evaluates finished commercials, exposed in a completely natural "unforced" environment, with no hint of cognitive bias. Absolute scores also guard against the possibility of selecting the best alternative from a set of mediocre commercials.
Over the next nine years, 60 commercials were tested in TRI-NET procedures - 3 were tested twice. These commercials covered the full range of General Mills food products including cereals, dessert mixes, baking mixes, casseroles, snacks, and yogurt. One fashion commercial was also tested. Most of these commercials were relatively new but almost all advertised established brands.
These 63 TRI-NET test scores constitute a unique set of data - the only publicly available data base which shows the actual effect on brand choice of a single additional TV commercial exposure. These data provide, for the first time, direct evidence from controlled experiments that a single additional TV commercial exposure can produce measurable effects, that television advertising does not require multiple exposures to be effective.
On behalf of myself and all advertising researchers. I wish to express my gratitude to General Mills for allowing publication of the data.
The first of the TRI-NET experiments had two objectives:
1. To develop additional evidence(1) of the predictive validity of the SST Laboratory method by testing three pairs of commercials in both methods.
2. To determine the operational feasibility of the TRI-NET procedure.
Table 1 Original TRI-NET Design Market ABC CBS NBC 1 B1 B2 - - H1 H2 T1 - T2 2 - B1 B2 H2 - H1 T2 T1 - 3 B2 - B1 H1 H2 - - T2 T1
Design. The fundamental building block of TRI-NET's experimental design is the use of a specific time slot (usually during early or late evening news) on all three major network channels in each of a series of three TV channel and markets. Test commercials are aired in a balanced design across channels and markets. On the following day, interviews are conducted to measure brand choice and program attendance.
In the initial TRI-NET project, three pairs of 30-second commercials for noncompetitive products (B1, B2; H1, H2; and T1, T2) were tested (see Table 1). The commercials were rotated across channels and markets to balance audience composition by channel and to eliminate any differential commercial/program interactions.
Table 2 Original TRI-NET Markets Northwest South North Central West Buffalo Atlanta Indianapolis Denver Pittsburgh Louisville Cleveland Seattle Boston Nashville St. Louis Albuquerque
This basic design was replicated across four regions to control for the differential regional effects previously observed in Laboratory copy testing (see Table 2). Pairings and positions were rotated in the regional replications.
These rotations ensure the random equivalence of the audience for the different commercials. Within sampling error, the audience for each commercial was the same before the test commercial exposure - the same in product usage, demographics, and in exposure to competitive commercials. Any postexposure differences must be attributed to the test exposure itself.
Execution. Proper airing of the commercials in the test design demanded extremely detailed control procedures.
* Two weeks before the test, letters were sent to the 36 station managers emphasizing the need for strict compliance with the scheduled air time for the test commercials.
* The necessity of compliance was reiterated in the cover letter sent with the commercial prints.
* Two days before the test, the station managers were telephoned by General Mills to ensure that the commercial prints had been received and the airing properly scheduled.
* On the afternoon of the test, local Burke representatives called each station to reconfirm the scheduled airing. (The Burke representatives had been supplied with a duplicate set of the prints.)
* The 15 minutes before and after the test were audio recorded on each channel to check for competitive commercials.
* Actual airing of the test commercials was independently monitored in each market by two Burke representatives using different TV monitors.
Data collection started the next morning among a randomly selected sample of telephone households. Over 7,600 interviews were completed that day.
The questionnaire was designed to gather brand choice data prior to any reference to television. The interview began with questions measuring brand attitude - salience, awareness and purchase, and coupon selection for the test categories.(2) The criterion measure was share of coupon selection (see Table 3).
Table 4 Sample TRI-NET Findings Test Control Share of coupons 3.0% 2.3% N (Product class users) 580 1126 Absolute effect + 0.7% Probability effect positive .87
Subsequent questions identified program viewers so that respondents could be classified into those who could and those who could not have seen the different commercials. Questions on program viewing the previous night included recall of specific news stories adjacent to the commercials.
Findings were reported as share of coupons chosen, weighted for product-class usage, for the test brand in the fifteen-minute program audience which could have seen the test commercial (test) compared to the share of coupons chosen by the program audience which could not have seen the test commercial (control)(3) (see Table 4).
* Probability statements were computed for the confidence level that the test effect was positive. Probability statements of more than .50 indicate that the point estimate was positive; probabilities of less than .50 indicate a negative point estimate.
* In contrast, SST is a laboratory copy test which uses a forced exposure of a competitive environment with a pre/post measurement (see Figure 1). The criterion measure is share of-coupon selection based on a question very similar to that used in TRI-NET. Interviews are conducted in 6 to 12 geographically dispersed shopping centers.
* SST findings were reported as change in share-of-coupon choice (see Table 5). Primary emphasis was placed on the relative change produced by the alternative commercials tested. Probability levels were computed for the superiority of the winning commercial.
Table 5 Sample SST Findings X1 X2 Share of coupons-Pre 26.3% 26.3% Share of coupons-Post 28.3% 24.6% Relative effectiveness +1.9% -1.7% Difference +3.6% N (Product class users) 196 197 Probability: X1 [greater than] X2 .86
Findings. TRI-NET and SST findings were essentially identical for the three pairs of commercials [ILLUSTRATION FOR FIGURE 2 OMITTED]. The same commercials "won" in each procedure, and the rank order of the absolute magnitude of the "wins" was the same.
The complex TRI-NET procedure was successfully executed(4). The correct commercials were aired as scheduled; the interviews were completed; and the data accepted. The stringent controls exercised by General Mills and the competence of Burke Marketing, which fielded the study, proved essential.
Subsequent TRI-NET projects were conducted for operational purposes only - to post-test finished commercials providing absolute measures of real-world effectiveness. Typically two, three, or four commercials for noncompetitive products were tested in each project.
This difference in study objectives enabled design changes - changes which simplified the execution of the TRI-NET without damaging its character as a real-world controlled experiment (see Table 6). Typically a single commercial minute was purchased on one channel of a three TV channel market, rotating the exposure across channels in other markets. The design was replicated in four regions.
Interviewing efficiency was also improved by starting the interview with questions concerning exposure to various sources of news "yesterday." Those respondents who reported no exposure to news through TV were not interviewed. This change in screening had no apparent effect on the data.
Overall, the most striking discovery is the magnitude of the effect of a single exposure of some of the commercials (see Table 7). For several commercials, gains of 25 percent, 50 percent, or more in share of choice were observed.
Table 6 Modified TRI-NET Design Market ABC CBS NBC 1 GMI - - 2 - GMI - 3 - - GMI Table 7 Largest TRI-NET Gains Commercial Test Control Effect Probability # 1 5.1% 1.6% +3.5% .99 2 21.0% 13.6% +7.4% .99 3 23.1% 16.9% +6.2% .98 4 9.8% 7.4% +2.4% .98 5 48.6% 39.6% +9.0% .98 6 3.7% 2.6% +1.1% .97 7 3.6% 1.6% +2.0% .96 8 11.8% 7.0% +4.8% .96
For example, a single exposure of Commercial #1 tripled the test brand's share of coupons. Among the audience which could have seen this commercial the test brand share of choice was 5.1 percent compared to 1.6 percent in the randomly equivalent audience which could not have seen the commercial. (Incidentally, within a few weeks after this commercial first aired, the test brand's prior five-year sales decline was reversed.)
Significant negative effects were also found for some commercials (see Table 8). Share-of-choice losses of 5 percent to 60 percent were observed with very small probabilities that the effect could have been positive.
Of course, some commercials had little, if any, effect (see Table 9).
Probability statements for the commercials were highly skewed positively [ILLUSTRATION FOR FIGURE 3 OMITTED]. Twelve of the sixty-three commercials were positive at .90 or greater probability, and only five commercials were negative with less than a .10 chance of positive effect.
The positive skew becomes even more apparent when the same data are viewed as a percentage distribution [ILLUSTRATION FOR FIGURE 4 OMITTED]. About 19 percent of the commercials were strongly positive with probabilities of .90 or greater; 54 percent were positive at probabilities of .60 or greater. Only 8 percent were strongly negative with the probabilities of positive effect at less than. 10.
A special analysis, based on over half the commercials, showed a direct relationship between TRI-NET real-world post-test findings and SST Laboratory pretesting [ILLUSTRATION FOR FIGURE 5 OMITTED]. Of the commercials which had been pretested, 59 percent produced positive effects at the .60 probability level or more, and only 6 percent had negative effects at the .40 probability level or less. Of the commercials which had not been pretested, only 33 percent were positive while 45 percent were negative!
Table 8 Largest TRI-NET Losses Commercial Test Control Effect Probability #56 6.3% 8.7% -2.4% .20 57 6.1% 8.9% -2.8% .17 58 42.3% 45.2% -2.9% .15 59 1.5% 2.8% -1.3% .06 60 .9% 2.1% -1.2% .06 61 21.7% 30.1% -8.4% .02 62 3.4% 9.4% -6.0% .02 63 20.7% 22.7% -2.0% .01 Table 9 Little TRI-NET Effect Commercial Test Control Effect Probability #39 9.9% 9.5% +.4% .54 40 27.0% 26.7% +.3% .54 41 23.8% 23.7% +.1% .52 42 19.3% 19.3% - .50 43 1.3% 1.3% - .50 44 23.0% 23.0% - .50 45 70.8% 71.5% -.7% .43 46 34.4% 35.1% -.7% .43
This analysis again demonstrates that TRI-NET and SST measure the same phenomenon and the findings add to the credibility of SST.
More important, these findings suggest that pretesting is essential to eliminate negative-effect commercials. Judgment alone is insufficient.
There is no evidence of the product-class effect which is always found in copy testing based on ad-related criteria - recall, recognition, and communication [ILLUSTRATION FOR FIGURE 6 OMITTED]. Very positive and very negative effects were found in each product category as shown by high- and low-probability statements for commercials in each.
Three commercials were retested with similar results in two of the three cases (see Table 10). In the third, positive original findings were reversed when the commercial was retested for suspected "wear out" 14 months later.
Brand "Y" has several sub-brands and the similarity in test/retest results for these subbrands was especially striking (see Table 11).
The findings demonstrate three basic conclusions about the way TV commercials work.
* A single additional exposure of a TV commercial can change brand attitude for an established brand.
The answer to this fundamental question is definitive. A viewer's brand attitude can be changed by a single exposure; multiple exposures are not necessary. A frequency of one can be effective.
* The range of possible effects of a single additional exposure is huge - from very positive to very negative.
The enormous range of effects is shocking - particularly since effects were measured, on the following day, for established brands, among the 15-minute program audience who could have, but did not necessarily, see the commercial.
The negative effects are also shocking. Some TV commercials literally hurt the advertised brand by driving down its share of choice.
Incidentally, other advertisers may not find a similar range of effects in their creative stream. When General Mills started its copy-testing program, only one or sometimes two commercials were available for testing. As acceptance of copy testing grew, the number and variety of commercials tested increased dramatically.
* The range of possible effects is not related to the product category for this range of food products.
This finding contrasts sharply with the widely held belief that effective advertising is easier in some categories, harder in others. This belief derives from recall data where this pattern is always seen.
Implications for Advertising Theory
For years, advertising theoreticians have tried to rationalize two seemingly contradictory facts about TV advertising. On the one hand, TV advertising obviously can have huge effects. We all know of advertising campaigns which built brands - advertising which even built companies. On the other hand, the effect of a single additional exposure on sales or brand attitude is seldom observed.
The central logical issue is what to assume about the effect of a single additional exposure. Most theoreticians have assumed that one-time effects must be small. Some have suggested the minimum number of exposures (e.g., three) necessary to change a viewer's brand attitude.
Many have theorized that advertising operates through a "Hierarchy of Effects" - a series of necessary but not sufficient steps through which the advertising moves the viewer step-by-step (e.g., Attention, Communication, Integration, Motivation, Action).
Table 10 Test-Retest Cases Case Date Test Control Effect Probability 81 5/83 46.6% 43.0% +3.6% .85 82 2/85 49.1% 45.6% +3.5% .89 Y1 2/84 25.8% 27.5% -1.7% .36 Y2 4/84 32.3% 33.3% -1.0% .43 X1 11/83 11.8% 7.0% +4.8% .96 X2 1/85 6.1% 8.9% -2.8% .17 Table 11 Y Test-Retest Case Product Effect 1 Effect 2 Probability 1 Probability 2 Y-a -.5% -.2% .43 .43 Y-b -3.5% -4.3% .11 .10 Y-c -.1% +.4% .48 .58 Y-d +2.4% +3.1% .79 .86 Y -1.7% -1.0% .36 .43
"Hierarchy of Effects" models are the foundation for the entire stream of copy research on recall, recognition, and communication. Findings from this huge research stream have been consistent. Almost all commercials have a measurable effect on some stage of the hierarchy. Brand attitudes are at a late stage and are difficult to change even by campaigns. Commercial effects must be evaluated relative to the product-class norm.
A three-dimensional view of the advertising response function suggested by these theoreticians (Theory 1) is shown in Figure 7. Notice the familiar "S" shape in the weight dimension resulting from the need for multiple exposures. Creative effects are significant but there are no negative effects. The response surface is complex in the weight dimension but relatively simple in the content dimension.
There has long been an alternative minority theory - a theory which assumes that a single additional exposure can have a significant effect. Several reasons have been suggested for not observing this effect. First, a particular commercial may have no effect to observe. Second, a commercial effect is difficult to observe since it probably erodes rapidly, assuming that competitors also advertise. Third, the effect may be difficult to observe because the actual audience for any single commercial exposure is so small. (This suggests that multiple exposures actually may be necessary for effective TV advertising in order to build the TV audience rather than to affect an individual viewer). Finally, few researchers have actually looked for a one-time effect.
Some advertising research has been conducted on the basis of this alternate theory; Leo Bogart's classic newspaper project, "What One Little Ad Can Do," Eric Marder's(5) Saturday Evening Post magazine service; Eric Marder's validation study for his TEC Audit; a few unpublished television studies; and many unpublished print split runs.
Findings from these studies are consistent with the TRI-NET findings but inconsistent with findings from research based on Theory 1. Unfortunately, almost everything practitioners think they know about advertising is based on the mainstream (Theory 1) research.
The advertising response function suggested by TRI-NET and the other alternative theory (Theory 2) research findings are shown in Figure 8. This surface shows that some advertising copy produces greater positive effects as weight increases; some advertising copy produces no effect, no matter how much is spent; and some advertising copy produces greater negative effects as weight increases.
Notice that there is no "S" curve, the incremental effect decreases as weight increases. The content dimension is more complex than the weight dimension.
Implications for Advertising Research
These findings demand that we spend more time and money on research to evaluate copy. They also support the use of copy testing based on "forced," single exposure using an attitude criterion.
The findings also suggest that weight testing must be repositioned. Since optimal weight depends on copy rather than brand, copy research must always precede weight testing. The response function is too complex for direct experimentation until the creative dimension has been resolved.
The limitations of marketing-mix models are more apparent. Results must be interpreted with precision because the model parameters are based on past advertising effects. These effects are copy specific, not brand specific, and cannot be extrapolated to different copy. Of course, this complexity then affects the parameters of the other marketing variables.
Implications for Advertising Practice
The extraordinary range of effects observed in these data demonstrate that the creative process deserves far greater time, effort, and funding. Many more copy alternatives should be prepared and tested.
These TRI-NET findings are strikingly consistent with the Ph.D. dissertation of Dr. Irwin Gross. In his dissertation, Gross examined the cost and payout of the creative process. Based on expert opinion about the importance of copy, he suggested it might be profitable to spend 25 percent to 35 percent of the entire advertising budget developing and evaluating content - five to eight times more than current practice.
Gross concluded by pointing out the inconsistency of commonly held beliefs about advertising and actual advertising practice. For example, one can believe advertising content is extremely important and current spending practice is irrational; or, one can believe advertising content is not very important and current spending practice is rational; however, it is totally inconsistent to believe advertising content is important while spending only 4 percent or 5 percent of the advertising budget on content.
Given the potential effect of a single exposure, the media objective should no longer focus on an arbitrary minimum frequency. Reach should become dominant.
Since commercial effects probably erode quickly, commercials should be aired in the closest possible proximity to customer buying and using occasions for maximum effect.
A Final Thought
All of us face serious problems today. The marketing function has lost influence in the corporation; agency billing growth has slowed; marketing research departments have been downsized, ignored, and even eliminated. Meanwhile private labels challenge national brands; volume slows; profit margins erode for clients and agencies; and only promotion budgets grow.
These are not independent phenomena. I believe the fates of clients and advertising agencies, marketing and marketing research, and national brands and advertising are inextricably intertwined.
Our problems will not be solved with the same old tactics we have used in the past by marketing spending even more discretionary dollars on promotion, by advertising agencies preaching the unmeasured, long-term effects of advertising by researchers arguing recall versus persuasion. As the old saying goes, "If we keep doing what we're doing, we'll keep getting what we're getting."
These TRI-NET data contain the seeds of an entirely different strategy. They dramatize the extraordinary immediate power of great advertising. They illustrate the weakness of judgment alone in identifying this advertising. They demonstrate that sound research can identify the advertising that works.
We can adopt the strategy the data suggest by simply acting on our often-stated belief in advertising. We can allocate far more of our resources to the creative process. We can use these resources to create many more alternative commercials, validate our research methods and use the research to evaluate the alternatives.
If we adopt this simple strategy, I believe that we will rebuild our national brands, our functions, and our institutions.
1 In four previous comparisons, SST and Ad Lab/AdTel sales experiments had yielded congruent results.
2 In the first TRI-NET feasibility study, criterion questions were also asked for toothpaste - a dummy category not involved in the test. Differences between test and control shares for toothpaste brands were well within random expectations.
3 Shares of choice in control groups were reassuringly close to actual market shares.
4 The entire General Mills TRI-NET, SST, Ad Lab/AdTel advertising research program was directed with great distinction by Dr. Grove P. Laybourn.
Dr. Laybourn also served as editor of this paper before submission to the JAR.
5 While Eric Marder did not participate in the design or operation of the TRI-NET program, his insights from the development of a wide variety of real-world advertising measurement systems stimulated the thinking on which TRI-NET was based and helped create the climate in which TRI-NET was possible.
RELATED ARTICLE: Table 3
TRI-NET Criterion Measure
"As a token of their appreciation for your participation in this survey, the Bureau of Family Opinion would like to send you five 10[cents] off coupons for cold breakfast cereals."
"These coupons can be used at any food store and they are good for one year . . ."
"You could choose all five coupons for the same cold breakfast cereal or you could choose one coupon for each of five different cold breakfast cereals or you could choose . . ."
"Please tell me the names of the cold breakfast cereals for which you would like to receive your five 10[cents] off coupons, and the number of coupons you'd like for each?"
Bogart, Leo, B. Stuart Tolley, and Frank Orenstein. "What One Little Ad Can Do." Journal of Advertising Research 10, 4 (1970): 3-13.
Gibson, Lawrence D. "If I Don't Want to Loan You the Plow." In Proceedings of the 14th Annual ARF Conference. New York: Advertising Research Foundation, 1968.
Gross, Irwin. "The Creative Aspects of Advertising." Sloan Management Review 14, 1 (1972): 83-109.
-----. "An Analytical Approach to the Creative Aspects of Advertising Operations." Unpublished Ph.D Thesis. Cleveland, Case Institute of Technology, 1967.
Jones, John Philip. In Transcript Proceedings of Effective Frequency Research Day. "When Ads Work: New Proof of How Advertising Triggers Sales." New York: Advertising Research Foundation, 1994.
-----. "Single Source Research Begins to Fulfill Its Promise." Journal of Advertising Research 35, 3 (1995): 9-16.
Krugman, Herbert E. "Why Three Exposures May Be Enough." Journal of Advertising Research 12, 6 (1972): 11-15.
Marder, Eric. Choice Research. Forthcoming.
McDonald, Colin. "Effective Frequency: The Relationship between frequency and Advertising Effectiveness." In Transcript Proceedings of the ARF Effective Frequency Research Day. New York: Advertising Research Foundation, 1994.
Naples, Michael. Effective Frequency: The Relationship Between Frequency and Advertising Effectiveness. Association of National Advertisers, 1979.
LAWRENCE D. GIBSON serves major consumer and industrial marketers as a senior associate of Eric Marder Associates, Inc. He is also a private marketing consultant.
He was the marketing research director of General Mills from 1965 to 1985 and built one of the nation's largest, most influential, and most respected research departments. Prior to that he was a vice president and director of Audits & Surveys and before that was commercial research director with CPC. He also held positions at Dun & Bradstreet, American Molasses Company, and Donahue & Coe Advertising.
He is a member of the faculty of the AMA's School of Marketing Research and a frequent speaker at professional meetings. He has lectured at many schools such as The Wharton School of Business, the Tuck School, and Stanford University, among others. His speeches and articles have been published in a wide variety of journals.
He received his M.B.A. and his Bsc. sc. from Ohio State University.
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||effectiveness of television advertising|
|Author:||Gibson, Lawrence D.|
|Publication:||Journal of Advertising Research|
|Date:||Mar 1, 1996|
|Previous Article:||Paradise tossed.|
|Next Article:||Using judgment profiles to compare advertising agencies' and clients' campaign values.|