Classification and Bias of Clinical Research - PowerPoint PPT Presentation

1 / 45
About This Presentation

Classification and Bias of Clinical Research


'If a research study is so methodologically ... Do Storks Bring Babies? Population of Oldenburg, Germany, 1930-1936 ... Storks (1000s) Humans (1000s) References ... – PowerPoint PPT presentation

Number of Views:2146
Avg rating:3.0/5.0
Slides: 46
Provided by: loc684


Transcript and Presenter's Notes

Title: Classification and Bias of Clinical Research

Classification and Bias of Clinical Research
  • Rick Chappell, Ph.D.
  • Professor,
  • Department of Biostatistics and Medical
  • University of Wisconsin Medical School

Good Ethics is Good Science
  • If a research study is so methodologically
    flawed that little or no reliable information
    will result, it is unethical to put subjects at
    risk or even to inconvenience them through
    participation in such a study. Clearly, if it
    is not good science, it is not ethical.
  • - U.S. Dept. of Health and Human Services,
    Policy for Protection of Human Subjects (45 CFR
    46, 1/1/92 ed.)

Types of Studies Classified by Temporal Point
of View
  • I. Instantaneous Studies - Surveys
  • II. Longitudinal Studies
  • A. Retrospective Studies
  • Historical Observational Cohort
  • Case - Control
  • B. Prospective Studies
  • Prospective Observational Cohort
  • Clinical Trial
  • C. Hybrid Designs

A Schematic for Temporal Classification
Observational Cohort
Observational Cohort
Clinical Trial
Case - Control
Instantaneous Survey
I. InstantaneousPopulation-Based Studies
  • Synonyms
  • Survey
  • Population-Correlation Study
  • Ecological Study
  • Two or more populations are instantaneously
    compared through the prevalences of both exposure
    and disease.
  • As summarized units get smaller (country ? region
    ? neighborhood ? individual), a survey
    approaches a historical observational cohort

Population-Based Studies
  • Advantages
  • Instantaneous.
  • Easy access to a large and varied population.
  • Good for hypothesis generation.
  • Disadvantages
  • Intervention is usually not feasible.
  • Very little information on causality IARC
    standards require individual-based evidence.

(No Transcript)
(No Transcript)
II. LongitudinalIndividual-Based Studies
  • A longitudinal study observes exposures and
    events for individuals over a period of time.
  • There are two types, depending on whether one is
    looking forwards (prospective) or backwards
    (retrospective) from the present.

Longitudinal StudiesA. Retrospective
  • Historical Observational Cohort
  • Synonyms - survey, retrospective cohort study.
  • Examines outcomes among patients with past
  • E.g., track down 1950s asbestos miners
    determine current status.
  • Case - Control (Breslow and Day, 1980)
  • Synonyms - case referent, retrospective study.
  • Examines past exposures among a group of patients
    with current outcomes.
  • E.g., interview mesothelioma patients determine
    past exposures.

Historical Observational Cohort Studies
  • Advantages
  • Quick results - no wait.
  • Easy to get large samples by mining databases.
  • Yields wide range of sequelae.
  • Useful for investigating rare treatments or
  • Disadvantages
  • No opportunity to customize data collection.
  • No possibility for blinding.
  • Many possible biases
  • Confounding
  • Selection
  • Information

Case - Control Studies
  • Disadvantages
  • Gives narrow picture of risks due to treatment or
  • Biases
  • Confounding
  • Selection
  • Recall
  • Yields only estimates of relative, not absolute
  • Advantages
  • Cheap, quick - record searching can be automated.
  • Useful for pilot studies.
  • Useful for investigating rare disorders.

Hypothetical Historical Cohort Study
  • Exposed Group
  • 100 Patients
  • 10 Events
  • Rate .1
  • Odds Ratio ??2
  • Control Group
  • 100 Patients
  • 5 Events
  • Rate .05

Hypothetical Case-Control Study
  • Event Group
  • 100 Patients
  • 10 Exposures
  • Event Rate per Exposure
  • (Not 100/200).
  • Non-Event (Control) Group
  • 100 Patients
  • 5 Exposures
  • Odds Ratio ??2

Longitudinal StudiesB. Prospective
  • General Advantages
  • Can collect detailed exposure, treatment,
    disease, and demographic information.
  • Blinding is possible.
  • Recall and information bias may be eliminated.
  • Useful for investigating rare treatments or
  • Classification depends on the presence of

Prospective Studies
  • Prospective Observational Cohort
  • Synonyms - prospective trial, clinical trial.
  • No intervention.
  • Randomized Controlled Clinical Trial
  • Synonyms - prospective interventional cohort
    study, experiment, prospective trial, clinical
  • Experimenters directly intervene in patient
    treatment, usually on a randomized basis with

Prospective Observational Cohort Study
  • Additional
  • Advantage
  • Passive observation no need to dictate
  • Disadvantages
  • May take a long time to accrue cases and wait for
  • Potential confounding bias due to lack of
    randomization and suitable controls.

Clinical Trials
  • Additional Advantages
  • The most definitive tool for evaluation of the
    applicability of clinical research - 1979 NIH
  • Biases may be eliminated.
  • Good design may make analysis simple.
  • Disadvantages
  • As above, may take a long time.
  • Must be ethically and laboriously conducted.
  • Requires treatment on basis (in part) of
    scientific rather than medical factors. Patients
    may make some sacrifice (Meier, 1982).

Phases of a Clinical Trial
  • Biochemical and pharmacological research.
  • Animal Studies (Gart, 1986 Schneiderman, 1967).
  • Phase I (Storer, 1989) - estimate toxicity rates
    using few ( 10 - 40) healthy or sick subjects.
  • Phase II (Thall Simon, 1995) - determines
    whether a therapy has potential using a few very
    sick patients.

Phases of a Clinical Trial (cont.)
  • Phase III - large randomized controlled, possibly
    blinded, experiments
  • Phase IV - a controlled trial of an approved
    treatment with long-term followup of safety and

Longitudinal StudiesC. Hybrid Designs
  • Prospective Treatment, Historical Controls
  • Currently treated series of patients is compared
    with a previous series.
  • See Gehan Freireich (1974), Gehan (1984).
  • Advantages
  • Doesnt assign treatments.
  • No need to recruit controls.

Longitudinal StudiesC. Hybrid Designs (cont.)
  • Prospective Treatment, Historical Controls
  • Disadvantages
  • Same as in Historical Observational Cohort
    except that characteristics of treated patients
    (only) can be collected.
  • Selection bias likely because of time lag between

Hybrid Designs
  • Prospective Treatment with Both Prospective and
    Historical Controls
  • Uses both types of controls to maximize
    efficiency and minimize bias
  • See Pocock (1976a and 1976b).

Bias in Clinical Studies
  • Definition Bias is a systematic error in
    estimation which is not reduced by increasing the
    study sample size (as opposed to random
  • See Sacket (1979) and other articles in the same
    issue Rose (1982) and Lachin (1988).
  • Classification is based on whether bias occurs at
    the time of patient Selection or at the time of
    Information collection or at the time of
  • They are all variants of Confounding, in which a
    third variable is related to both treatment and

I. Selection Bias
  • Prevalence - Incidence Bias
  • Prevalence (observed occurrence) of a trait ??
    Incidence (rate of onset).
  • Cause gap between exposure, selection of
  • Not a problem with irreversible events such as
    mortality, if detectable.
  • E.g., hypertension may disappear with onset of CV
    disease and can be overlooked as a risk factor.
  • See Neyman, 1955.
  • (Any retrospective study, especially

Selection Bias
  • Admission Rate Bias
  • Patients may differ from noninstitutionalized
    subjects in size or direction of effects.
  • E.g., systemic weakness vs. arthritis
  • Negative relation among inpatients
  • Positive relation among outpatients.
  • See Berkson, 1946.
  • (Any nonrandomized study with a mix of patient
    sources, especially case-control.)

Selection Bias
  • Nonrespondant (Volunteer) Bias
  • Nonparticipation may be related to the subject of
  • E.g., smokers ignore surveys more often than do
    non-smokers (Seltzer, 1974).
  • For general methods to analyze data with
    nonignorable nonresponse see Little and Rubin
    (1987) and Rubin (1987).
  • (Case-control, though drop-outs can effect any
    study not analyzed intent to treat.)

Example Where to add armor to fighter planes?
  • In World War II, the U.S. Air Force conducted an
    investigation into where armor could most
    effectively be added to fighter planes.
  • Researchers examined returning aircraft, mapped
    the locations of bullet holes, and recommended
    that the most commonly pierced areas be
  • Their recommendation neglected the most vital
    part of the aircraft, which was intact in all
    returning aircraft the area surrounding the
    pilots head!

II. Information Bias
  • Detection Signal (Diagnostic Suspicion) Bias
  • In unblinded studies, an exposure may be
    considered a risk factor for an endpoint, and
    such patients preferentially observed.
  • In blinded studies, an exposure may make an
    endpoint more detectable.
  • E.g., estrogen causes bleeding from uterine
    cancer to be more easily detectable.
  • (Any unblinded study except case-control also
    clinical trials with sensitive endpoints.)

Reports of Original Studies JAVMA 191, 12/1/87
High-rise syndrome in cats Wayne O. Whitney,
DVM Cheryl J. Mehlhaff, DVM
Selection and/or detection bias
Information Bias
  • Exposure Suspicion Bias
  • An outcome may cause the investigator to look for
    a particular exposure.
  • The temporal reverse of detection signal bias.
  • E.g., arthritis and knuckle-cracking.
  • (Case-control studies.)

Information Bias
  • Recall (family information) Bias
  • Similar to exposure suspicion bias, but errors
    originate with the subject or his/her family.
  • E.g., in a study of prescription use among women
    with fetal malformation, 28 reported
    unverifiable exposure vs. 20 of the controls
    (Klemetti Saxen, 1967).
  • (Case-control studies.)

III. Publication (Reporting) Bias
  • Even a perfect study leads to bias if
    dissemination depends on the direction of its
  • Causes
  • Commercial reasons
  • Researchers personal motivations
  • Editorial Policy !
  • Vickers, et al. (1998) show that the problem is
    widespread in some countries, 100 of
    publications show treatment effects.

Publication (Reporting) Bias
  • A version of the multiple comparisons problem
    (Miller, 1985), or testing to a foregone
  • E.g., ORG-2766 protected nerves from cytotoxic
    injury in 55 women with ovarian cancer - NEJM
    lead article (van der Hoop, et al., 1990) a
    subsequent negative study of 133 women - ASCO
    Proceedings abstract (Neijt, et al., 1994).
  • (All Studies.)

(No Transcript)
A type of reporting bias Multiple Comparisons
(Data Dredging)
  • A p-value is interpreted as the probability of
    attaining a result as extreme that observed given
    that the result is false (under the null
    hypothesis) it can be viewed as the false
    positive rate under the null hypothesis.
  • This assumes that only a single test is
    conducted. If many tests are performed, it is
    possible to sample to a foregone conclusion and
    produce a falsely low p-value.
  • For example, if twenty-five independent tests are
    conducted, the probability of at least one
    p-value being less than .01 is .22.
  • Often only the significant result is reported,
    and the 24 others ignored.

(No Transcript)
(No Transcript)
(No Transcript)
(No Transcript)
IV. Confounding (General)
  • Caused by any situation in which
  • A third variable exists which isnt known or at
    least isnt accounted for
  • It is associated with the cause
  • and
  • It is also associated with the effect.
  • Then
  • The supposed cause-effect relation will be
    confounded by the third variable.
  • (Any nonrandomized study)

Do Storks Bring Babies?
Population of Oldenburg, Germany,
1930-1936 (Ornithologische Monatsberichte 44,
Jahrgang, 1936, Berlin)
Humans (1000s)
Storks (1000s)
  • Berkson, J. Limitations of the application of
    fourfold table analysis to hospiital data (1946).
    Biometrics Bulletin 2, 47-53.
  • Breslow, N.E. and Day, N.E. (1980). Statistical
    Methods in Cancer Research 1 The Analysis of
    Case-Control Ctudies. Oxford Oxford University
  • Dorr, Robert T. (1997). Personal communication.
  • Gart, J.J. et al. (1986). Statistical Methods
    in Cancer Research 3 The Design and Analysis of
    Long-Term Animal Experiments. Oxford Oxford
    University Press.
  • Gehan, Edmund A. The evaluation of therapies
    Historical control studies, with discussion
    (1984). Statistics in Medicine 3, 315-324.
  • Gehan, Edmund A. and Freireich, Emil (1974). The
    New England Journal of Medicine, 198-203.
  • IARC. Monographs on the Evaluation of
    Carcinogenic Risk of Chemicals to Humans. Lyon
  • Klemetti, A. and Saxen, L. Prospective vs.
    retrospective approach in the search for
    environmental causes of malformations. American
    Journal of Public Health 57, 2071-2075.
  • Lachin, J. Statistical properties of
    randomization in clinical trials (1988).
    Controlled Clinical Trials 9, 289-311.
  • Little, R.J.A. and Rubin, D.B. (1987).
    Statistical analysis with Missing Data. New
    York Wiley.
  • Neyman, J. Statistics - servant of all sciences
    (1955). Science 122, 401.
  • Meier, Paul. Current research in statistical
    methodology for clinical trials (1982).
    Proceedings of Current Topics in Biostatistics
    and Epidemiology A Memorial Symposium in Honor
    of Jerome Cornfield. Pages 141- 150.

  • Miller, R. Publication bias (1985). Entry in
    The Encyclopedia of Statistical Sciences, Volume
    5. S. Kotz and N.L. Johnson, eds., pp. 679-689.
    New York Wiley.
  • National Institutes of Health, Division of
    Research Grants, Research Analysis and Evaluation
    Branch, Bethesda, MD (1979). NIH inventory of
    clinical trials fiscal year 1979, Volume I.
  • Neijt, et al. (1994). Proceedings of the
    American Society for Clinical Oncology.
  • Pocock, S.J. The combination of randomized and
    historical controls in clinical trials (1976a).
    Journal of Chronic Diseases 29, 175-188.
  • Pocock, S.J. Randomized versus historical
    controls A compromise solution (1976b).
    Proceedings of the International Biometric
    Conference 9/1, 245-260.
  • Rose, G. Bias (1982). British Journal of
    Clinical Pharmacology 13, 157-162.
  • Rubin, D.B. (1987). Multiple Imputation for
    Nonresponse in Surveys. New York Wiley.
  • Sacket, D.L. Bias in analytic research (1979).
    Journal of Chronic Diseases 32, 51-63.
  • Schneiderman, M.A. Mouse to man statistical
    problems in bringing a drug to clinical trial
    (1967). Proceedings of the 5th Berkeley
    Symposium in Mathematical Statistics and
    Probability, Volume IV. L.M. LeCam and J.
    Neyman, eds. Berkeley.
  • Seltzer, C.C. et al. Mail response by smoking
    status (1974). American Journal of Epidemiology
    100, 453-477.
  • Storer, B.E. Design and analysis of phase I
    clinical trials (1989). Biometrics 46, 33-38.
  • Thall, Peter F. and Simon, Richard M. Recent
    developments in the design of phase II clinical
    trials (1995). In Recent Advances in Clinical
    Trial Design and Analysis. Peter Thall, ed. ,
    pp. 49-72. New York Kluwer.
  • Unger, D.L. Does knuckle cracking lead to
    arthritis of the fingers? letter. Arthritis
    Rheumatism 41. 949-50, 1998.
  • van der Hoop, et al. (1990). New England Journal
    of Medicine 322, 89-84.
  • Vickers, et al. (1998). Controlled Clinical
    Trials 19, 159-166.
Write a Comment
User Comments (0)