Does Media Concentration Lead to Biased Coverage? Evidence from Movie Reviews

1
Does Media Concentration Lead to Biased Coverage?
Evidence from Movie Reviews

• Stefano DellaVigna, UC Berkeley and NBER
• Alec Kennedy, San Francisco Fed
• Moscow Media Conference
• October 29, 2011
• Preliminary, comments most welcome

2
Introduction

• Dec. 13, 2007 News Corp. acquires from Bancroft
  family Dow Jones Company ? Wall Street Journal
• Unlike Bancroft family, Murdoch's holdings
  include
• Cable channels (i.e., Fox Sports and Fox News)
• Satellite television (Sky group)
• Movie distributor (20th Century Fox)
• Media conglomerate implies conflict of interest ?
  Coverage of such businesses (or their
  competitors) in the Wall Street Journal may be
  biased.

3
Introduction

• Wall Street Journal situation hardly unique
• Media conglomerates
• Comcast owns NBC
• Hearst Corporation owns ESPN and print outlets
• Time Warner owns AOL and Time magazine
• ? Pervasive conflict of interest
• But should media consolidation lead to
  distortion?
• Cost Loss of reputation if bias is revealed
• Benefit Can persuade audience (if audience naïve
  about bias)
• Generally, persuasion rates of 5-10 from the
  media (DellaVigna-Gentzkow, ARE 2010)

4
Introduction

• This paper Focus on two conglomerates
• News Corp.
• Time-Warner
• Measure how media outlets in these groups review
  movies distributed by an affiliate
• 20th Century Fox
• Warner Bros.
• Identification of bias transparent
  (diff-in-diff)
• Compare review of 20th Century Fox movie by WSJ
  and NYT
• Further control group review of Paramount movie
  by WSJ and NYT

5
Introduction

• Why movie reviews?
• Frequent (500,000 reviews in data set)
• Easily quantifiable (coded on 0-100 scale by
  metacritic.com)
• Large industry (60bn annual revenue)
• Some evidence that movie reviews influence movie
  attendance (Reinstein and Snyder (2005)? benefits
  to the distributor from increased ticket sales

6
Data

• First data set of reviews from metacritic.com,
  scored 0-100

7
Data

• Second data set of reviews from
  rottentomatoes.com, scored on a 0-1 freshness
  scale

8
Data

• Merged data set contains 548,764 reviews from 336
  media from 1985 to July 2011

9
Data

• Studios which distribute movies in the sample
  Major, Independent, and Other

10
Average Bias News Corp.
11
Average Bias Time Warner
12
Average Bias Statistical Test

• Is the bias for News Corp. statistically
  significant?
• Is it robust to introducing controls?
• OLS regression
• captures the effect of conflict of
  interest in Fox
• captures the effect of conflict of
  interest in Time Warner
• Sample of 473,727 reviews (qualitative reviews in
  RT have no 0-100 score)
• Standard errors clustered at the movie level

13
Average Bias Statistical Test
14
Average Bias Statistical Test
15
Average Bias Statistical Test

• In favorite specification, conflict of interest
  increases rating for News Corp. by 2.6 points out
  of 100
• Estimate of bias increases with extra controls ?
  Unobservables unlikely to bias coefficient upward
  (Altonji, Elder, and Taber, 2005)
• Magnitude of bias
• Equivalent to 1 extra star every ten reviews
• Small but still economically significant impact
• Reinstein and Snyder (2005) estimate 25 higher
  revenue for two thumbs up by Roger Ebert
• Are all media conglomerates the same?
• No evidence of bias for Time Warner, can reject
  bias of 0.9 points

16
Average Bias Robustness

• Evidence of bias using RT 0-1 freshness indicator

17
Calculation of benefits

• Back-of-the-envelope calculation of estimated
  benefits from distortion of movie review
• Case of New York Post (NYP)
• Suppose that NYP gives one extra star per Fox
  movie
• 500,000 average readers
• Persuades an extra 1 of readers to watch movie
• Ticket sales increase by 5,000, or
  85,00040,000
• Studio receives about half of increased sales,
  plus another half from higher rights from DVD
  licensing ? 40,000
• About 400 20th Century Fox movies since 1985
• ? Potential benefits to NewsCorp. from one extra
  star per Fox review by NYP over 25 years
  16,ooo,ooo

18
Average Bias Explanations

• Three main explanations for the results
• (E) Explicit editorial policy conveyed to
  journalists
• (J) Bias by a journalist ultimately due to the
  conflict of interest, but lacking editorial
  pressure
• (T) Correlation in taste between the media
  reviewer (or the media audience) and the
  affiliated studio
• To separate explanations, we present evidence
  on
• Clustering of bias within a conglomerate (E and
  T, maybe J)
• Editorial policies (E only)
• Selective bias by type of movie (E or J, not T)
• Omission of reviews (E or J)

19
Clustering of Bias By Media

• News Corp. media Statistical evidence for NYP,
  2-3 point bias for all 6 media
• No evidence of bias for any of the Time Warner
  media

20
Clustering of Bias By Journalist

• Most media have a small number of journalists
  reviewing movies
• Chicago Sun-Times, News of the World, Wall Street
  Journal and CNN.com have essentially only one

21
Clustering of Bias By Journalist

• News Corp Statistical evidence of bias for 3 out
  of 4 NYP journalists, and one of TV Guide
  journalists
• Time Warner No evidence of positive bias
• Significant Clustering Pattern suggestive of
  editorial bias, but could also be correlated
  tastes, or similar journalists

22
Editorial Policies

• Two tests of editorial policies
• Personnel Policy Change of reviewers at change
  of ownership No evidence
• Bias in Assignment Assign affiliated movies to
  more generous reviewers
• Estimate average reviewer generosity in score
• Reviewers differ significantly in their
  generosity
• Are movies by affiliate studios more likely to be
  assigned to more generous reviewer?
• No evidence assignment quasi-random
• Bias at Newscorp. is unlikely of editorial origin
• Could be correlation in journalist bias or in
  tastes

23
Editorial Policies
24
Selective Bias

• If bias is due to conflict of interest, should be
  optimal
• Bias when marginal return (persuasion) is highest
• Maximize impact on revenue
• Proposition 2. Small or no bias for very low
  quality movies
• If bias is due to correlated tastes, no such
  prediction
• Assumption for this test
• Movies with negative reviews by others are
  unlikely to benefit from a lone positive review
• Movies with other positive reviews more likely to
  benefit from more positive review

25
Selective Bias News Corp.
26
Selective Bias News Corp.
27
Selective Bias New York Post
28
Selective Bias Time Warner
29
Selective Bias Statistical Evidence

• Evidence of selective bias for NYPost and
  qualitatively for WSJ Bias due to conflict of
  interest
• No evidence for other media

30
Bias by Omission

• If bias is due to conflict of interest, and
  audience not fully rational, bias by omission
• Review high-quality affiliated movies
• Omit review of low-quality affiliated movies
• Do not expect this pattern if correlated tastes
• Rare setting to separate bias by omission and by
  commission
• Outlets differ substantially in probability of
  review ? Use matching procedure
• For each media, find ten matching media in terms
  of average probability of review of a movie
• Plot local polynomial regression of dummy for
  review on average review score

31
Bias by Omission New York Post
32
Bias by Omission Time
33
Bias by Omission Entertainment Weekly
34
Bias by Omission Statistical Test

• Newscorp. No systematic evidence of omission
  bias
• Time Warner Evidence for 2 outlets
• Bias by omission and by commission substitutes,
  not complement

35
Bias in Movie Aggregator

• So far focus on most obvious conflict, for
  reviewers
• Conflict of interest hardly stops there
  Rottentomatoes.com, also independent when
  launched in 1998, was acquired by IGN
  Entertainment in June 2004, and IGN was purchased
  by News Corp. in September 2005. IGN, and hence
  RottenTomatoes, was then sold in January of 2010
  by Newscorp.
• Conflict of interest for RT more positive
  reviews of the 20th Century Fox movies in
  2006-2009
• Control for rating of same review in MC

36
Bias in Movie Aggregator

• Remarkably, no evidence of bias, even for
  qualitative reviews, where bias is easier to hide
• Can reject small bias

37
Bias in Movie Aggregator

• Local polynomial regression of fresh indicator
  on average movie score no evidence of bias

38
Bias in Movie Aggregator

• Event study comparing residual freshness (after
  controlling for score) for FOX and non-FOX movies

39
Summary of Results

• We documented the extent of bias due to conflict
  of interest for two media conglomerates
• Average bias
• 2.6 points bias out of 100 for Newscorp. outlets
• No bias for Time Warner outlets, can reject even
  small bias
• Bias is clustered within a media conglomerate
• No evidence of editorial policy to assign movies
• Selective Bias Evidence for New York Post, not
  for other media
• Omission Bias Evidence for two Time Warner
  outlets
• Interpretation
• Best fits with bias due to conflict of interest
  for journalists, with clustering of such bias

40
Conclusion

• Overall, remarkably little evidence of distortion
  from conflict of interest
• No distortion in review for Time Warner
• No distortion for Rottentomatoes
• No distortion in editorial assignment
• However, bias still does occur
• Small, but significant, bias for Newscorp.
  Outlets
• Some omission bias for Time Warner outlets
• Suggests that transparency and emphasis on
  reputation (for example because of competition)
  critical to keep media honest
• Paper allowed us to decompose potential media
  bias in novel ways
• Relates to Conflict of interest in other
  settings
• Significantly less distortion than for analysts
• Less distortion than for advertising

41
(No Transcript)
42
Average Bias
43
Average Bias
44
Selective Bias II

• Different types of movies can have different
  returns to positive reviews
• Snyder and Reinstein Larger effect of movie
  reviews for independent movies
• However Independent movies also have smaller
  revenue, so bias may be less worthwhile
• Examine the effect of conflict of interest
  separately

45
Selective Bias Indy movies

• Significant bias for News Corp. only for
  mainstream movies