The Hockey Stick Saga: Some Lessons for the Research Community

1
The Hockey Stick Saga Some Lessons for the
Research Community

• Prof. Ross McKitrick
• Associate Professor of Economics
• University of Guelph
• Guelph Ontario, Canada
• Keynote Address
• Western Research Forum
• University of Western Ontario
• April 18, 2005

2
The hockey stick graph

• Mann, Bradley and Hughes (MBH98 and MBH99)

3
The hockey stick graph

• Became an icon of global warming research
• Consensus view, authoritative conclusions
• Yet
• Results never replicated
• Methodology was not adopted
• Data not released until recently
• Computer code never disclosed

4
The hockey stick graph

• MM Audit
• Errors in data and methodology
• Fallout
• Lessons

5
The importance of the hockey stick graph

• Used in IPCC Report (2001)
• Summary for Policymakers
• Technical Summary
• Chapter 2, Assessment Report Figs 2.20 and 2.21
• Synthesis Report (twice)
• Basis for claim that temperatures in the latter
  half of the 20th century were unprecedented

6
The importance of the hockey stick graph
7
The importance of the hockey stick graph
8
The importance of the hockey stick graph

• This gives a fairly clear signal that this
  isn't just a future issue, it's happening now,
  Mr. Hengeveld said. Among the strongest evidence
  is the fact that the past century has likely been
  the warmest in the Northern Hemisphere in the
  past millennium, he said. Not only that, the
  1990s ranked as the warmest decade of the
  millennium, and 1998 was the warmest year of the
  millennium in the Northern Hemisphere, which is
  where most of their data have been acquired.
• Henry Hengeveld,
• Canadas Chief Climate Science Advisor,
• Globe and Mail January 22, 2001 (emph. added)

9
Background Medieval Warm Period

• IPCC 1995

10
Background MWP

• Huang et al. 1997 (GRL)

11
Background MBH98

• Multiproxy technique relying mainly on tree rings
• NH coverage
• Math involved several steps
• Principal Component Analysis
• Calibration using least-squares fitting
• Projection into the past
• 1998 Nature
• 1999 GRL
• 2001 IPCC Report shown as consensus view

12
MM Team Up

• Stephen McIntyre
• 55 year-old businessman in Toronto
• No academic experience
• Background in auditing and mineral analysis
• Had obtained Manns data in Spring 2003
• Began discovering errors and couldnt get
  straight answers about them
• Sent me pile of notes in late Summer 2003

13
Data Structure

• 112 proxy series
• Of these
• 71 are individual site records
• 31 are weighted averages of larger underlying
  groups
• The weighted averages are called Principal
  Components

14
MM03 (Energy Env. Nov. 03)

• We found
• Truncation of sources
• Arbitrary fills of missing data
• Obsolete data
• Duplication of series
• Geographical mislabeling
• Nonstandard/incorrect PC calculations

15
MM03 (Energy Env. Nov. 03)
16
MM03 (Energy Env. Nov. 03)

• Mann then released a new data archive
• Claimed we had not replicated his method exactly
• We requested his source code (Nov 2003)
• Request refused, remains undisclosed to date

17
New data archive

• Most series matched file we had analyzed already
• We found
• Numerous discrepancies with Nature data listing
• Notified Nature in January 2004
• Led to Corrigendum of July 2004
• PC algorithm (Fortran code)

18
PC computational glitch

• Standard method
• subtract mean, divide by standard deviation
• Yields series with mean0, variance1
• PC algorithm then looks for dominant patterns

19
PC computational glitch

• Manns method
• subtract 1902-1980 mean (rather than series
  mean), divide by standard error

20
PC computational glitch

• Manns method
• subtract 1902-1980 mean (rather than series
  mean), divide by standard error

21
PC computational glitch

• Result mean of series which trend up in 20th
  century gets boosted
• PC algorithm picks weights that increase with
  size of shift in mean

22
PC computational glitch

• Example

23
PC computational glitch

• Implications Program efficiently mines for
  hockey sticks even where none exist
• Statistical significance claims in MBH99 were
  spurious

24
PC computational glitch

• Out of about 400 series, only about 20 have
  upward trend in 20th century
• All but 1 are bristlecone pines from western USA
• Growth spurt widely acknowledged not to be
  temperature proxy
• Yet these series get all the weight in the final
  results
• Other one Gaspé cedar tree

25
PC computational glitch
26
The 2nd Period Score

• The 20th century climate is no longer unique
  compared to the 15th century
• The hockey stick model has no statistical
  significance for projecting past climates
• The results all depend on including a small group
  of flawed bristlecone pines from western USAall
  the rest of the data set are over-ridden by these

27
3rd PeriodResponses by Mann

• The hockey stick can be partly recovered by
• (a) Using 5 PCs in North America rather than 2
• (b) Skipping the PC steps and using proxies
  directly

28
3rd PeriodFlaws in responses

• (a) Using 5 PCs
• PC Analysis ranks patterns by importance. The
  hockey stick pattern only appears in 4th PC of
  North American network no longer dominant
  signal
• Still depends on bristlecone pines
• Result disappears if CO2 adjustment properly
  applied
• Model is still statistically insignificant either
  way
• (b) Dropping PC step
• Original purpose to deal with geographical
  imbalance
• Without it, 80 of 95 proxies are from NA
  bristlecones still dominate data set

29
3rd Period Other Reactions

• The findings hit me like a bombshell, and I
  suspect it is having the same effect on many
  others. Suddenly the hockey stick, the
  poster-child of the global warming community,
  turns out to be an artifact of poor mathematics.
  
• Professor Richard Muller, University of
  California at Berkeley
• It is strange that the climate reconstruction of
  Mann passed both peer review rounds of the IPCC
  without anyone ever really having checked it. I
  think this issue will be on the agenda of the
  next IPCC meeting in Peking this May.
• Dr. Rob van Dorland, an IPCC Lead Author and
  climate scientist at the Dutch National
  Meteorological Agency

30
3rd Period Other Reactions

• He Climatologist Ulrich Cubasch discussed with
  his coworkers - and many of his professional
  colleagues - the objections, and sought to work
  them throughBit by bit, it became clear also to
  his colleagues the two Canadians were right.
  Between 1400 and 1600, the temperature shift was
  considerably higher than, for example, in the
  previous century. With that, the core conclusion,
  and that also of the IPCC 2001 Report, was
  completely undermined.
• Das Erste, Feb 16, 2005
• The IPCC review process is fatally flawed... The
  scientific basis for the Kyoto protocol is
  grossly inadequate.
• Dr Hendrik Tenekes, Ret. Director, Royal
  Meteorological Inst., Netherlands, Feb 22, 2005

31
Next Steps

• www.climateaudit.org
• Some further publications
• Lessons for scientific community

32
Lesson 1 Beware the laughter of the gods

• In the realm of seekers after truth there is no
  human authority. Whoever attempts to play the
  magistrate there founders on the laughter of the
  gods
• -Albert Einstein
• Our academic titles and credentials count for
  nothing with Mother Nature

33
Lesson 2Everyone makes mistakes

• If we knew what we were doing, it wouldnt be
  research
• No one will fault you for mistakes if you
• Are transparent about your work,
• Admit your errors
• Fix them promptly

34
Lesson 3 The Internet has changed the world

• People expect access to data and methods
• Non-academics expect to be involved in
  high-profile cases
• Expertise more diffused
• Threat? Benefit? Well see!

35
Lesson 4Math (stats) matters

• Spend time mastering as much as you can
• Common language of science

36
Lesson 5Interdisciplinary Research

• Arises from mastery of discipline
• Opposite Anti-disciplinary research
• Many key fields are ID
• To get into them, master a discipline

37
END of presentation see climateaudit.org for
more