Epidemiology Workshop II Oak Ridge Reservation Health Effects Subcommittee December 3, 2001

1
Epidemiology Workshop IIOak Ridge Reservation
Health Effects SubcommitteeDecember 3, 2001
Evaluating an Epidemiologic Study Lucy Peipins,
PhD Sherri Berger, MSPH Division of Health
Studies
2
How Science Works
Every epidemiologic study and every analysis is
built on a body of knowledge and is done to
expand that body of knowledge
Is there a relationship between a particular
exposure and a disease?

• Epi Study

• Animal studies

• Case Reports

• Epidemiologic study

• Toxicologic Models

• Epidemiologic study

3
Evaluating an Epidemiologic Study

• What is the purpose of scientific publication?
• To present a new finding. Results are shared and
  debated with other scientists and the public
• To demonstrate validity of study results by
  presenting the study methods, design, analysis
  and interpretation.
• This provides the opportunity for others to judge
  the quality of the study

4
Evaluating an Epidemiologic Study

• Like all scientists, epidemiologists have
  criteria to judge the scientific quality and
  validity of a study
• Has the author presented a good case for his or
  her hypothesis?

5
Evaluating an Epidemiologic Study

• Each journal judges the scientific content and
  validity of an article by a peer review process
• Reviewers typically do not know the identity of
  the author
• Reviewers make recommendations on whether the
  article should be published

6
Evaluating an Epidemiologic Study

• To illustrate these criteria, we will use the
  article Cancer Mortality near Oak Ridge,
  Tennessee by J. Mangano published in the
  International Journal of Health Services
• The criteria we outline can be applied to any
  study.

7
Evaluating an Epidemiologic Study

• The first thing we need to find out is
• Why was this study done?
• or, another way of saying this is
• What hypothesis is the author testing?

8
Evaluating an Epidemiologic Study

• The main purpose was to examine change in cancer
  mortality in 94 counties within 100 miles
  surrounding Oak Ridge, Tennessee, in 1950-1952
  and in 1987-1989, and compare this with mortality
  rate change in the U.S. and the Southeast
• Exposure of interest is radiation from Oak Ridge
  weapons production which began in the early 1940s

9
Evaluating an Epidemiologic Study

• Specific Hypotheses
• Increase in all cancer mortality near Oak Ridge
  should exceed the national and regional areas
• Within Oak Ridge area, increase in cancer
  mortality rate should be greatest in rural areas
• Increase in cancer mortality should be greatest
  near the weapons plant
• Cancer mortality should be greatest in
  mountainous areas
• Cancer mortality increases should be greater
  downwind of weapons plant

10
Evaluating an Epidemiologic Study

• What type of study was this?
• Descriptive study. This type of study describes
  health outcomes alone. It answers the who,
  what, when and where of a disease in a group of
  people.

11
Evaluating an Epidemiologic Study

• Analytic study. This type of study answers why
  a population or group of people get a particular
  disease.

12
Evaluating an Epidemiologic Study

• This is primarily descriptive. Exposure is based
  on county of residence
• Hypotheses based on where-- counties nearest
  to weapons plant, counties downwind, mountainous
  counties, rural counties
• It is also analytic because the comparison
  groups--Southeast or U.S.

13
Evaluating an Epidemiologic Study

• Keeping in mind that epidemiology is the study of
  groups of people or populations, who was the
  study group?

14
Evaluating an Epidemiologic Study

• White population living less than 100 miles (in
  94 counties) from Oak Ridge between 1950-1952 and
  1987-1989
• Comparison population was the white population of
  the Southeast and the U.S. in those time periods

15
Evaluating an Epidemiologic Study

• What was the exposure of interest?

16
Evaluating an Epidemiologic Study

• Several types of exposure were mentioned
• radiation, or chronic, low-level ingested
  radiation,
• emissions from nuclear plant into both air and
  water, and radiation from consumption of milk and
  vegetables were also mentioned

17
Evaluating an Epidemiologic Study

• How was exposure defined and measured?

18
Evaluating an Epidemiologic Study

• Exposure was not measured.
• Exposure was defined as residence in a county
  near weapons plant, residence in mountainous
  counties, residence in downwind counties and
  residence in rural counties.

19
Evaluating an Epidemiologic Study

• Lets review the hierarchy of exposure assessment
  from our first workshop and see how to
  characterize Manganos exposure classification

20
Measuring Exposure

• We can measure exposures or their surrogates
  along the entire path from emissions to body
  burden
• Types of Data Approximation to Actual
  Exposure
• __________________________________________________
  _______________________
• 1) Residence in a defined geographical
  Poorest x
• area (e.g. county) of a site
• 2) Residence in a geographical area in
  x
• proximity to to a site where exposure is
  assumed x
• 3) Distance or duration of residence
• 4) Distance and duration of residence
• 5) Quantified surrogate of exposure (e.g.
  estimate
• of drinking-water use
• 6) Quantified area measurements in vicinity of
  the
• residence
• 7) Quantified personal measurement Best
• Baker D, et al. Environmental Epidemiology A
  Textbook on Study Methods and Public Health
  Applications, 1999

21
Evaluating an Epidemiologic Study

• What are some limitations or criticisms you might
  have about how exposure was measured?

22
Evaluating an Epidemiologic Study

• Residence in county is a poor measure of
  exposure.
• All persons living in the 94 counties are
  assigned the same exposure
• Persons who lived in the county at the time of
  death may not have been long-term residents or
  persons who were long-term residents may have
  retired, moved and died in another county or
  state
• All persons in the rest of the southeast or U.S.
  are assigned no exposure

23
Evaluating an Epidemiologic Study

• Proximity to site is poorly defined in this
  study.
• Why is Roane county not considered to be near the
  site?
• Why are the comparison 12 counties within 40
  miles of the site and not further away?
• Different pathways of exposure (water and air
  pathways) may exert a stronger influence than
  vicinity alone

24
Evaluating an Epidemiologic Study

• Mountainous vs. non-mountainous counties
• Rainfall in mountain areas is higher that valleys
  in general, but there is a great deal of
  variation between mountainous counties and
  between mountains and valleys in each county.
  Category of mountainous is too general.

25
Evaluating an Epidemiologic Study

• No rationale (or references) for the assumption
  of urban/rural differences in cancer mortality.
• Why select Georgia, NC, SC and Tennessee and not
  include Kentucky and Virginia?
• No references on why rural population might be a
  lower risk of cancer than urban populations.
• Rural areas are not free of risks -- pesticides
  and herbicides, smoking, and others

26
Evaluating an Epidemiologic Study

• Downwind counties
• We have more information from dose
  reconstructions that contradicts some of the
  hypothesized wind patterns

27
Evaluating an Epidemiologic Study

• What was the outcome of interest?
• Deaths from cancer among whites in 94 counties
  between 1950-1952 and 1987-1989 compared with
  cancer deaths in U.S. and Southeast.

28
Evaluating an Epidemiologic Study

• How was the health outcome measured in this
  study?
• Cancer mortality data by counties were obtained
  from the National Cancer Institute
• Based upon death certificates

29
Evaluating an Epidemiologic Study

• What are some limitations or criticisms you might
  have about how the outcome was chosen and
  measured?

30
Evaluating an Epidemiologic Study

• All cancers together rather than specific cancers
  were considered.
• Remember, cancer is not one disease but over 100
  different diseases arising from 40 sites in the
  body
• There are many causes of these cancers-not all
  can be linked to environmental causes.

31
Evaluating an Epidemiologic Study

• There is considerable inaccuracy in death
  certificate data (25 error)
• Accuracy may vary by region-depending on who
  codes the certificates
• Accuracy may vary by time. Improvements in
  diagnosis or coding may have led to an increase
  in cancer rates over time

32
Evaluating an Epidemiologic Study

• Selection bias - could there have been bias
  (distortion or error) in how study subjects got
  selected into the study?
• Mangano used already existing data so subjects
  were not selected.

33
Evaluating an Epidemiologic Study

• Information bias - could there have been bias
  (distortion or error) in how information was
  collected from the study group and the comparison
  group or groups?
• Accuracy of cause of death certification may vary
  by region of the country

34
Evaluating an Epidemiologic Study

• Could the results have been due to confounding?
• Confounding is the possibility that the
  relationship we see between exposure and disease
  could have been due, totally or in part, to other
  factors or differences between the study group
  and comparison group.

35
Evaluating an Epidemiologic Study

• Potential Confounders
• Smoking, diet, physical activity, and
  occupational exposures might have differed
  between the study counties and the rest of the
  Southeast or U.S.
• Author assumes little or no migration. Yet we
  know there was a large influx of people into the
  Oak Ridge area in the 1940s and a subsequent
  declined in population. Younger people have a
  lower risk of cancer.

36
Evaluating an Epidemiologic Study

• Life expectancy may differ by region over time.
  As life expectancy increased in the southeast,
  deaths from cancer might increase since it is
  primarily a disease of older people
• Other sources of environmental contamination may
  have varied by region
• Potential increase in pollution from utility and
  other plants in the area over time

37
Evaluating an Epidemiologic Study

• Did the author measure any confounders? No.
• No information on confounders is found on death
  certificates

38
Evaluating an Epidemiologic Study

• Describe the methods used to measure the
  relationship between exposure and outcome.
• Age-adjusted mortality rates (deaths/population)
  for selected counties were calculated for
  1950-1952 and 1987-1989 and compared to Southeast
  counties and U.S.

39
Evaluating an Epidemiologic Study

• Statistical Analysis
• Very little information on statistical methods
  including why 1950 standard was used
• Why choose only 1950-52 and 1987-89? Why not
  rates by decade or calculate average annual
  increases?
• Why not calculate rates for different age groups
• Why not calculate rates for different cancers

40
Evaluating an Epidemiologic Study

• What are the major results of the study. Author
  states that
• The cancer mortality rate increase in the Oak
  Ridge area (94 counties) exceeded the national
  and regional increases (30.5 vs. 5.1)
• Rural areas experienced the greatest increase

41
Evaluating an Epidemiologic Study

• Counties downwind and in those in N.W. wind
  direction had greatest increase
• Anderson county had greater increase than
  selected counties lt 40 miles from plant
• Most (but not all) mountainous areas had greatest
  increase

42
Evaluating an Epidemiologic Study

• Results were positive for all hypotheses,
  however,
• Has the author justified his conclusions?

43
Evaluating an Epidemiologic Study

• How strong was the association between exposure
  and outcome?
• Large differences were seen between study
  counties and the U.S.
• Not so strong for Southeast

44
Evaluating an Epidemiologic Study

• Has this relationship been seen by other
  researchers conducting other types of studies in
  different populations?
• Several studies have looked at residence near
  nuclear facilities. Worker studies have also
  looked at radiation and cancer.

45
Evaluating an Epidemiologic Study

• Is there a biologically plausible explanation
  between exposure and disease? Yes.
• Ionizing radiation is known to cause cancer.
  However, radiation levels could not be measured
  --county of residence was the surrogate for
  ionizing radiation

46
Evaluating an Epidemiologic Study

• Does the risk of disease increase with increasing
  exposure (dose-response).
• Not demonstrated. No doses calculated, therefore
  no dose-response

47
Evaluating an Epidemiologic Study

• Has exposure occurred before disease?
• Unknown for an unknown number of people due to
  moving into and out of study counties

48
Evaluating an Epidemiologic Study

• Authors conclusion
• Analysis of change in cancer mortality suggests a
  link between radiation contamination and
  increased risk to populations living near nuclear
  sites

49
Evaluating an Epidemiologic Study

• Has the author convinced us of his conclusions?
• Did this study advance our knowledge about the
  relationship between exposure and disease?

50
Evaluating an Epidemiologic Study

• Limitations of study
• No exposure measure Limitations in use of
  residence in county surrogate for radiation
  exposure
• Limitations to use of death certificate data and
  mortality rather than incidence of cancer
• Lack of control for potential confounders
• Focus on all cancers rather than those most
  likely to be related to exposure

51
Evaluating an Epidemiologic Study

• Because of these limitations, this study does
  not effectively demonstrate a relationship
  between cancer mortality rates and radiation
  exposure

52
Evaluating an Epidemiologic Study

• Epidemiology is about measurementA study is only
  as good as its measures of exposure and
  outcomeboth are essential as are minimizing bias
  and confounding