Editorial Environment and Advertising Effectiveness

Valentine Appel

Advertising researchers have long been conducting tests of the hypothesis that certain publications by virtue of their unique editorial environments will either enhance or detract from the way in which advertising appearing in the publication will be perceived by the readers. For example, a business advertisement in Business Week will be better perceived than will the same advertisement in a news weekly, or an advertisement in The National Enquirer will be less favorably perceived than the same ad in a less sensational publication such as McCall's or Reader's Digest. The theory and the difficulties of conducting research in this area were well described by Weilbacher as early as 1960.

Review of the Literature

A search of the literature has identified eight studies published in the United States in which the influence of editorial environment upon the perception of advertising has been directly investigated. The use of the word directly is meant to exclude the many studies where respondents were asked to rate the advertising appearing in particular publications without actually using the advertising as a stimulus. None of these eight studies, notwithstanding the conclusions they may have reached, give strong support to the editorial environment thesis.

The earliest identified published attempt on the editorial environment thesis was one by Fortune magazine done in 1959. The study employed a sample of business executives taken from the Fortune subscription list who also claimed to be regular readers not only of Fortune but also of both Business Week and U.S. News & World Report. Using an appropriate rotational procedure, all ads were exposed to all respondents either outside the magazine or in Fortune, Business Week, or U.S. News. The study concluded that all ads were rated higher when shown in one of the magazines than when shown alone, more so for Fortune than for the other two magazines, Business Week or U.S. News. Unfortunately, because all respondent names were taken from a Fortune subscription list, there is the likelihood that the respondents were more strongly dedicated to Fortune than to the other two magazines and may therefore have been biased in Fortune's favor.

A study sponsored by McCall's magazine in 1962 tipped 12 identical ads into advance copies of McCall's, Life, and Look. Using a test-control group design, test issues were placed in homes of these magazines' readers with instructions to read them the next day. The following day the interviewer returned and asked a series of four attitude and awareness questions regarding each of the test brands without making any mention of the advertising. Test- versus control-group increments were then determined for each ad separately when the ad had been tipped into McCall's and when it had been tipped into Life or Look. The mean net difference was then calculated across all 12 ads, comparing the test-control group differences in brand perception when the ad was tipped into McCall's and when it was tipped into Life and Look.

Although no statistically significant mean net difference (p < .05) was reported either for 'familiarity with brand' or 'acceptability of claims,' significant mean net differences of 3.1 percentage and 2.7 percentage points were reported for the top two box ratings of 'brand quality' and 'interest in buying the brand,' respectively. However, since the between-ad variance and degrees of freedom employed in the statistical tests were not published, it is not possible 24 years later to assess what was actually done.

A study by Charles Winick in 1962 presented four non-competing ads to a sample of women, the ads having been identified as taken from four different magazines. Half the time ad A was identified as having been taken from magazine X and the other half of the time from magazine Y. When respondents were asked to rank the four ads according to believability and overall appeal, the ad ranked higher when it was identified as coming from magazine X than when it was identified as coming from magazine Y.

The difficulty with this study was the contrivance of separating the ads from the magazines, 'conspicuously' printing - 'This advertisement is from such-and-such magazine' - and then having the respondents rank the advertising for non-competing products. A more serious difficulty of the ranking procedure was that it was subject to bias depending upon the extent of audience duplication with the other three magazines with which the ads were identified.

A fourth study also published in 1962 was sponsored by Life using a tip-in procedure where respondents were exposed to three noncompetitive ads-one in Life, one in Look, and one in a portfolio containing other ads. A total of twelve different ads were included in the study, and, for each presentation of these ads, the respondent was asked to select the one which best met six different criteria, for example, 'Which ad did the best job of persuading you to buy the product?'

Three analyses were performed: (1) based on a national probability sample of adults, (2) based on regular (8 or more out of 10 issues) readers of Life and/or Look, and (3) based on readers (1 or more out of 10 issues) of both magazines. All three analyses indicated that the ads performed better in Life than they did in Look.

Unfortunately, this study suffers from much the same problem as the Winick study published in the same year. In addition to the contrivance of comparing three noncompetitive ads presented in three different media contexts, the questioning procedure was inherently biased in favor of Life, which at the time had a larger audience of more frequent readers. This inequity necessarily resulted in a larger proportion of Look readers also reading Life than the reverse. For this reason, any paired comparison between the two magazines necessarily gave an unfair advantage to Life.

Following this flurry of activity in the late 1950s and 1960s, there appears to have been no published attempts at this kind of research until 1973 when Black Enterprise published a study of upscale black and white consumer reactions to identical ads running in  a then current copy of Newsweek and Black Enterprise.  Unfortunately, since the sample was selected without regard to readership of these magazines and no analysis by readership is provided, none of the findings are particularly pertinent.

William Blair, in 1966, and Timothy Joyce, in 1983 - both at American Marketing Association Attitude Research Conferences - also reviewed the literature and concluded that the research conducted to date had fallen far short of demonstrating the existence of the phenomenon despite the common acceptance of the premise.

More Recent Research

Subsequent to the Joyce presentation in 1981, three other studies have been published: one by Newsweek (1986), one by People (1986) and one by The National Enquirer (1982).  All three studied reader reactions immediately after having been exposed to test ads tipped into these and competing publications.  All three studies failed to find material differences in editorial environment of the publications studied.

Despite the apparent failures to date, the industry continues to explore the validity of the editorial environment hypothesis: Paul Chook, in a 1985 article, reported the results of an industry poll taken by the Advertising Research Foundation (ARF) in which 26 magazine research areas were rated in terms of need.

Overwhelmingly, the impact of editorial environment on advertising performance received the highest overall score. On the basis of this study, the ARF has created a committee specifically charged with the assignment of pursuing further research in this area.

The purpose of the present article is to report the results of some additional analyses which have been performed upon the National Enquirer data since publication and which, hopefully, will throw more light upon the issue of editorial environment and its influence upon advertising effectiveness.

The National Enquirer Study 

Sample design

The study was based on a sample of 1027 adult female readers of one or more of the following publications: National Enquirer, McCall's, People and Reader's Digest. A reader was defined as someone claiming to read at least two issues out of four.

The interviewing was conducted in the summer of 1982 door-to-door in five markets: Chicago, Detroit, Los Angeles, New York, and Philadelphia. A local directory in each market was used as the sampling frame, and a total of 20 starting points were selected at random for each. Within each cluster the interviewer was instructed to identify up to seven National Enquirer readers and five non-National Enquirer readers who were readers of one or more of the other three.

This procedure was adopted to produce an estimated 500 National Enquirer readers who later were down-weighted to compensate for their over-representation in the total sample, as indicated in Table 1.

Table 1: Sample Composition

 

Unweighted sample

Weighted sample

 

Enquirer readers

Nonreaders

Total

Enquirer readers

Nonreaders

Total

Total sample

506

521

1027

187

838

1025

McCall's readers

108

194

302

40

311

351

People readers

182

247

429

67

398

465

Reader's Digest readers

134

291

425

50

467

517

Magazine Preparation

Each interviewer carried a kit containing four sets of the current issue of the four publications. Into each of these test issues three of twelve ads were tipped-in so that each ad appeared in each publication an equal number of times. Three of the twelve ads were mail-order ads which had run previously in the Enquirer, and the others were for brands which had run in one or more of the other three publications included in the study.

The twelve ads were organized into four groups of three as shown in Table 2. Each of the twelve ads was then duplicated and tipped into each of the four test publications using the appropriate page size and paper stock in order to be indistinguishable from run of press copy. The tipping-in was accomplished according to the latin square design, shown in Table 3, where the letters P, M, R, and E stand for People, McCall's, Reader's Digest, and National Enquirer, respectively.

Table 2: Grouping of the Twelve Ads

Group

Brand

Product Category

4C/BW

I

Sensacort

anti-itch aerosol

4C

Playtex

tampons

4C

Dexatrim

appetite suppressant

4C

II

Personal Products*

hair remover

BW

American Motors

corporate

BW

K mart

underwear

4C

III

Norman Rockwell*

silver

BW

Ban

deodorant

4C

Wella Kolestral

hair conditioner

BW

IV

Valray*

wrinkle remover

BW

Brite

floor wax

4C

Bayer

aspirin

4C

*Mail order

Table 3: The Latin Square Design

 

Magazine Set

Ad group

1

2

3

4

I

P

M

R

E

II

E

P

M

R

III

R

E

P

M

IV

M

R

E

P

Each interviewer carried a kit containing all four groups of ads and magazine sets, showing every fourth respondent the same set. In this way, each ad was presented in each publication approximately an equal number of times.

The interview

Once an eligible respondent was identified, the respondent was shown the test ads appearing in each of the four test publications with both the magazine set and the order of presentation having been pre-designated. The respondents were led to believe that the test ads had actually appeared in the publications in which they were shown, and as each ad was exposed, were asked to rate the quality of the product advertised and the believability of the advertising using a five-point scale.

Following this, ten National Enquirer articles from the June 1, 1982 issue were shown, and the respondent was asked to rate the believability of each, again using a five-point scale.

Belief in National Enquirer Edit

The range of believability ratings of the articles went from a mean 1.70 for a story about a giant UFO to a high of 2.97 for a story about a college student who fell eight stories and lived. Without exception, the National Enquirer reader was more likely to believe each article than was the nonreader, the grand means across all 10 articles being 3.17 and 2.36, respectively. The 10 ratings given by each respondent were then summed to produce an overall believability score, and the distribution of these scores was divided into thirds to produce high, medium, and low levels of National Enquirer editorial believability. The result of this grouping is shown in Table 4, indicating that the readers of the Enquirer were much more likely to believe the editorial content than were the nonreaders.

Table 4: Readers and Nonreaders Belief in Edit

 

 

National Enquirer

Editorial believability

Score range

Readers %

Nonreaders %

High

30-50

62

27

Medium

21-29

29

34

Low

10-20

9

39

Further examination of these data, shown in Table 5, indicates clearly that the extent to which the National Enquirer edit is believed varies not only by readership but also by level of education. Those most likely to believe the articles were the readers who had never been to college. Two thirds of this group were classified as having high believability Those least likely to believe the edit were the college-educated nonreaders, less than a quarter (21%) of whom were classified as having high believability.

Table 5: Editorial Believability by readership and education

 

National Enquirer readers

Nonreaders

Editorial believability

No college

Some college

No college

Some college

Unweighted base

(390)

(116)

(334)

(185)

High 

67%

49%

30%

21%

Medium

27

36

33

36

Low

6

15

37

43

Editorial believability and ad effectiveness

When the product quality and advertising believability ratings were compared for these believability groups, we found that the higher the rating of editorial believability, the higher the rating of the advertising. This relationship applies both to the total sample and to the readers and nonreaders as well. These data are shown in Table 6 expressed as grand means across all 12 ads.

Table 6: National Enquirer ad ratings by belief in edit (mean ratings)

 

 

Enquirer believability

Quality of product

Total sample

2.80

2.45

2.79

3.17

National Enquirer readers

.3.07

2.19

2.83

3.28

Nonreaders

2.74

2.45

2.78

3.10

Advertising believability

Total sample

2.62

2.22

2.59

3.04

National Enquirer readers

2.97

1.92

2.77

3.19

Nonreaders

2.54

2.22

2.55

2.93

The remainder of this article presents only the believability ratings since the response pattern for the product quality ratings was virtually identical.

When the grand means of the believability ratings for all ads were compared as a function of the publication in which the ads appeared, there was no difference among the readers of these publications when they were rating the ads appearing in each one (see Table 7). Among the nonreaders, however, while parity was observed among McCall's, People, and Reader's Digest, the mean believability rating was significantly lower when the ads appeared in the National Enquirer than when they appeared ii any of the other three magazines. *

Table 7: Means of twelve as believability ratings*

 

Advertising exposed in

Test Magazine

National Enquirer

McCall's

People

Reader's Digest

Readers

2.97

3.01

2.94

2.92

Nonreaders

2.54*

2.92

2.97

2.93

Mean difference

+.43

+.09

-.03

-.01

Standard error

.04

.03

.06

.03

t - ratio vs. zero

10.75

3.00**

<1

<1

t - ratio vs. National Enquirer

NA

6.80

6.57

8.80

* The t-ratio comparing the Enquirer to each of the other magazines among nonreaders is 8.84 or greater.  With a d.f. a t-ratio of 5.84 is significant at the .01 level.

** Not statistically significant at the .05 level using 3 d.f.

This latter observation was uniformly true for all 12 ads with one minor exception: the Bayer ad was rated slightly higher (not statistically significant) when it appeared in the National Enquirer (3.15) than when it appeared in Reader's Digest (3.07). In all other instances, the rating was lower when the ad appeared in the National Enquirer than when it appeared in any of the other three publications.

The question naturally arises bout whether this parity in ad ratings in the National Enquirer (vis-a-vis the other three magazines among their readers), in the lace of the lower ratings in the Enquirer among the nonreaders, is attributable to reaction to the editorial environment or to differences in audience composition. Perhaps readers of the National Enquirer are simply more responsive to advertising than are nonreaders, and this greater responsiveness is sufficient to have offset whatever negative effects the National Enquirer editorial environment may have caused.

In order to resolve this question, the ad ratings of all four publications were examined separately among the readers and nonreaders of the National Enquirer. It was reasoned that if the National Enquirer reader/nonreader differences that were observed when the ads were exposed in the National Enquirer would also appear when the ads were exposed in the other three magazines, the observed differences would then be likely attributable to audience composition rather than editorial environment. A lack of difference would support the reverse argument.

The result of this analysis is shown in Table 8 which indicates that the differences in ad rating between readers and nonreaders of the National Enquirer were much larger when the ads appeared in the National Enquirer than when they appeared elsewhere. When the ads appeared in the other three magazines, the differences, although relatively small, were statistically significant.

table 8: mean believability ratings among National Enquirer readers and nonreaders

 

Advertising exposed in

National Enquirer

National Enquirer

McCall's

People

Reader's Digest

Readers

2.97

3.06

3.06

3.08

Nonreaders

2.54

2.93

2.93

2.88

Mean difference

+.43

+.13

+.13

+.20

Standard error

.04

.04

.04

.06

t-ratio vs. zero

10.75

3.25*

3.25*

3.33*

t-ratio vs. National Enquirer

NA

5.17**

5.00**

4.14**

* Significantly greater than zero (p<.05) using 3 d.f.

** Significantly smaller than the National Enquirer (p<.05) using 6 d.f.

On the basis of these findings, we could conclude that the differences observed in the way that readers and nonreaders have reacted to advertising in the National Enquirer are attributable to two factors. The first factor is a difference in audience composition as evidenced by the fact that the Enquirer readers rated the advertising more favorably than did the nonreaders regardless of where it appeared. The more important factor, however, is the National Enquirer's editorial environment, as evidenced by the fact that the difference in rating produced by the readers and nonreaders was much larger when the ads appeared in the National Enquirer than when they appeared elsewhere.

Discussion

The concept of editorial environment as a mediator of advertising effectiveness is much more complex than would appear at first blush. Numerous investigators have attempted to demonstrate the validity of the concept but none have done so to the point where it is generally accepted.

The present study may explain why belief in the editorial environment concept continues in the absence of firm research support. The editorial environment of the National Enquirer clearly depressed the effectiveness of the ads appearing in it. It did so, however, only among the nonreaders who, by definition, have little or no opportunity to be exposed to the advertising in the real world.

Media planners as well as researchers interested in understanding the effects of editorial environment upon advertising effectiveness should be ever mindful that such effects are likely to be quite different for readers and nonreaders of particular publications. Since media planners cannot possibly be readers of the hundreds of publications they are obliged to consider, they should bear in mind that their nonreader perspective may be quite different from the perspective of the reader. Similarly, researchers who wish to study the problem must be careful to separate out the effects of differences in audience composition from differences in editorial environment. 

References

Audits & Surveys. 'How Professionals/Managers Read Business and Newsweekly Magazines.' Study sponsored by Newsweek, 1986.

Blair, William S. 'Attitude Research and the Qualitative Value of Magazines.' Paper delivered to the First American Marketing Association Attitude Research Conference, 1966.

Chook, Paul H. 'A Continuing Study of Magazine Environment, Frequency, and Advertising Performance.' Journal of Advertising Research 25, 4 (1985): 23-33.

Earl S. Graves Marketing Research Inc. 'A Study of Advertising Responsiveness Among Upscale Blacks and Upscale Whites.' Study sponsored by Black Enterprise, 1973.

Joyce, Timothy. 'Can Attitude Research Make A Contribution To Media Research.' Paper delivered to the American Marketing Association Attitude Research Conference. Hot Springs, VA, 1981.

Lieberman Research, Inc. 'The Impact of Editorial Environment on Brand Acceptance.' Study sponsored by People, 1986.

Nowland and Company. 'The Effect of Media Context on Advertising.' Study sponsored by Life, 1962.

Alfred Politz Research. 'A Measurement of Advertising Effectiveness: The Influence of Audience Selectivity and Editorial Environment.' Study sponsored by McCall's, 1962.

Social Research, Inc. 'Advertising Impact and Business Magazines.' Study sponsored by Fortune, 1959.

Simmons Market Research Bureau. 'Reader Reaction to Identical Ads in McCall's, National Enquirer, People, and Reader's Digest.' Study sponsored by National Enquirer, 1982.

Weilbacher, W. M. 'The Qualitative Values of Advertising Media.' Journal of Advertising Research 1, 2 (1960): 12-17.

Winick, Charles. 'Three Measures of Advertising Values of Media Context.' Journal of Advertising Research 2, 2 (1962): 23-33.

The author wishes to acknowledge the support of Ralph Gallagher, advertising director for the National Enquirer, without whose encouragement this article would not have been written.

*The statistical significance of the reported differences between publications and readers versus nonreaders were tested mindful of the fact that all respondents rated all ads, and that, for any given respondent, a group of three ads was rated in a given publication. In order, therefore, to satisfy the requirement of statistical independence of observations as is required in a t-test, the error estimates were calculated on the basis of the four group means using either three or six degrees of freedom as appropriate.

NOTES & EXHIBITS


Valentine Appel

Valentine Appel