LEVELS OF EVIDENCE FROM DIABETES REGISTRIES Registry-based Epidemiology?

1
LEVELS OF EVIDENCE FROM DIABETES REGISTRIES
Registry-based Epidemiology?

• John M. Lachin
• Professor of Biostatistics, Epidemiology and
  Statistics
• The Biostatistics Center
• The George Washington University

2
EuBIRO-D vs. USA

• Ciao Fabrizio e Massimo
• No regional or national healthcare program
• No national or regional registries
• HMO network
• Translating Research into Action for Diabetes
• Comparative Effectiveness Research
• Agency for Healthcare Quality and Research
  patient satisfaction, quality of life
• National Institutes of Health Clinical outcomes
• GRADE study

3
Science and Uncertainty

• Jacob Bronowsky
• All information is imperfect. We have to treat it
  with humility... Errors are inextricably bound up
  with the nature of human knowledge
• The degree of uncertainty is controlled through
  the application of the scientific method,
• and is quantified through statistics.

4
Statistical Test of an Hypothesis

• Null Hypothesis (H0)
• The hypothesis to be disproven
• The hypothesis of no difference.
• Alternative Hypothesis (H1)
• The hypothesis to be proven
• The hypothesis that a difference exists.
• Two types of errors
• Type I False positive, probability ?
• Type II False negative, probability ?
• Power 1 - ?

5
Factors that Affect ? and Power

• Selection and Observational/Experimental Bias
• Poor study design or execution
• Missing data
• Reproducibility (precision) of assessments

6
Missing DataThe Fundamental Issue - BIAS

• Numerators and denominators may be biased
• Estimates of population parameters, differences
  between treatments or exposure groups may be
  biased.
• Statistical analyses, pvalues and confidence
  limits may be biased.
• p 0.05 may mean a false positive error rate (?)
  much greater than 0.05
• N800, 20 missing in treated/exposed, true ?
  0.40.

7
Cant Statistics Handle This?

• Not definitively.
• The magnitude of the bias can not be estimated,
  no correction possible.
• Analyses can be conducted under certain
  assumptions.
• But there is no way to prove that the assumptions
  apply.
• Best way to deal with missing data is to prevent
  it.

8
Sample Size Adjustments

• Can adjust sample size to allow for
  losses-to-follow-up and missing data, e.g.
  increase N by 10 if expect 10 losses
• BUT, this adjusts only for the loss of
  information,
• NOT for any bias introduced by missing data.

9
Precision or Reliability of Measures

• Reliability coefficient ? proportion of total
  variation between subjects due to variation in
  the true values.
• 1 - ? proportion of variation due to random
  errors of collection, processing and measurement.

10
Impact of Reliability
Power decreases as ? decreases.
Power
Reliability (?)
11
Impact of Reliability

• If N is the sample size needed for a precise
  measure then N/? is needed for an imprecise
  measure.

? 1.0 0.9 0.8 0.7 0.6 0.5
1/? 1.0 1.11 1.25 1.43 1.67 2.0
12
Impact of Reliability

• Maximum possible correlation between Y and X is a
  function of the respective reliabilities Max(R2)
  ?x ?y

?x ?y Max(R2)
1.0 0.9 0.90
0.9 0.9 0.81
0.9 0.7 0.63
0.9 0.5 0.45
0.7 0.7 0.49
0.7 0.5 0.35
13
Impact of Misclassifications

• m fraction of treatment or exposure
  misclassifications, or fraction of outcomes
  misclassified
• N/(1-2m)2 is needed

m 0 0.1 0.8 0.7 0.6 0.5
1/(1-2m)2 1.0 1.56 2.78 6.25 25.0 8
14
Randomized Clinical Trial

• Randomization
• Subjects assigned to each treatment independently
  of patient characteristics
• No selection bias. Treatment groups expected to
  be similar for all variables measured and
  unmeasured.
• No confounding of the experimental treatment with
  other uncontrolled factors
• May infer a cause effect relationship between
  treatment and the outcome, provided the trial is
  of good quality.

15
Randomized Clinical Trial

• Precisely defined population
• Precisely defined exposure (the treatments)
• Precisely defined outcome measure
• Results clearly interpretable

16
Observational Study

• Many types, e.g. case-control study
• Prospective cohort study
• No randomized controls
• Maybe a precisely defined population
• Maybe a precisely defined exposure (the
  treatments)
• Maybe a precisely defined outcome measure

17
Observational Study

• Many potential biases
• Selection bias composition of groups
• Confounding with other factors
• Statistical adjustments substituted for
  randomization

18
Observational Study

• Necessary in settings where a randomized study is
  impossible
• Smoking and lung cancer
• Generally describe an association between the
  exposure factor and an outcome that may not
  represent a causal relationship.
• Difficult to establish causality, though possible
  with replication of a highly specific
  association.

19
Observational Evidence

• The essential issues with observational evidence
  is the degree to which an observed relationship
  can or can not be explained by
• other variables,
• other mechanisms, or
• biases
• even after statistical adjustment

20
Confounding

• When the study factor (groups) are associated
  with another (confounding) factor that is a
  direct cause of the outcome.
• Coffee consumption and cancer.
• Coffee consumption confounded with smoking.
• Higher fraction of smokers among coffee drinkers.

21
Statistical Adjustment for Confounding

• Regression or stratification model including the
  study factor and the possible confounding
  factor(s)
• Assumes that the operating confounding factors
  have been identified and measured.
• Assumes that the regression model specifications
  are correct.

22
Statistical Adjustment for Confounding

• Estimates the association of the factor with the
  outcome IF the confounding factor were equally
  distributed among the groups.
• Difference in cancer risk between coffee drinkers
  and non-drinkers IF the fraction of smokers was
  the same among drinkers and non-drinkers.
• Coffee drinking and smoking are alterable. Thus,
  the results would have a population
  interpretation.

23
Statistical Adjustments

• NOT all covariate imbalances introduce bias, in
  which case adjustment itself introduces bias.
• Gender inherently confounded with body weight
• Gender adjusted for body weight estimates the
  gender difference if males and females had the
  same weight distribution.

24
Statistical Adjustments

• Adjustment for weight provides a biased estimate
  of the overall malefemale difference in risk in
  the population
• But weight-adjusted estimate describes the
  additional malefemale difference in risk, if
  any, that is associated with gender differences
  other than weight
• Of mechanistic interest.

25
Omitted Covariates

• Observational study can only adjust for what has
  been measured.
• Adjustment for observed factors can not eliminate
  bias due to imbalances in unmeasured covariates.

26
Inappropriate Covariates

• Analysis should follow the prospective history of
  covariates
• Statistically invalid to define a covariate over
  a period of exposure that goes beyond the
  observation of an event.
• Example, mean HbA1c over 5 years as a predictor
  of outcomes observed during the 5 years.
• Rather, use the mean HbA1c up to the time of each
  successive event.

27
Confounding by Indication

• In some cases, however, exposure to a factor
  (e.g. drug) may be confounded with the
  indications leading to the exposure.
• Example statins indicated in the presence of
  hyperlipidemia.
• Recent data suggests that statin use may also
  increase risk of T2D in IFG/IGT.
• But is the increased risk due to the statin use
  or the prior history of hyperlipidemia?

28
Confounding by Indication

• In other cases an adjusting factor (e.g. dose)
  may likewise be confounded with an indication.
• Example Hemkens et al. analysis of the
  association of insulin glargine vs. human insulin
  with cancer in a German claims database.
• 14 decrease in age, gender adjusted risk.
• But substantial dose imbalance.
• 14 increase in risk when also adjusted for dose.

29
Reasons for Dose Imbalance

• Confounding by indication, or allocation bias.
• High or low glargine (or human insulin) dose may
  be determined by unmeasured patient factors that
  are differentially distributed within groups.
• e.g. high glargine dose only administered to
  severely ill patients.
• Impossible to statistically adjust for such
  confounding
• Adjusted analysis results are biased.

30
Registries

• Many types
• 100 population captured, e.g. public health care
  system
• Non-random subsample, e.g. insurance provider or
  hospital based
• In latter case, registry population may not
  represent the full population of interest
• Inherently prospective
• But no standardized follow-up schedule

31
Registries

• Relies on data capture in conjunction with the
  administration of medical care
• No specific exposure of interest when
  established, in epidemiological sense
• No specific outcome measure of interest.
• Rather medical status and treatment recorded
  (possible exposures) and other major morbidities
  and mortality recorded (possible outcomes).

32
Registries

• Epidemiologic analyses may be attempted.
• But, difficult to precisely define exposure to a
  factor
• When is a subject
• First at risk of being exposed (e.g. when is a
  drug introduced to the market?)
• Actually first exposed (e.g. starts drug)
• Removed from exposure (e.g. off drug)
• Confounding by indication often an issue

33
Registries

• Coding, classification of events may not be
  standardized
• Often no adjudication
• May be difficult to determine whether or exactly
  when an outcome event occurred, e.g.
  macroalbuminuria is interval-censored
• May be difficult to determine when subject no
  longer at risk (right censored)
• Incidence may be difficult to assess reliably.

34
Registries - Uses

• Prevalence
• Distribution of patient status or conditions in
  the population
• Cross-sectional associations
• If representative but not proportionally,
  weighted analyses can provide estimates in the
  broader population.
• Disadvantaged populations (poverty, uninsured)
  may not be represented

35
Registries - Epidemiology

• Exposure to a factor and outcomes
• Open to many biases.
• Statistical adjustments may be inadequate.
• But, a registry can be the foundation for
  first-rate epidemiologic studies.

36
Registries - Epidemiology

• Nested case-control studies
• Sub-sample of possible cases that is carefully
  adjudicated
• Sub-sample of possible controls (matched by
  follow-up time) also verified.
• Exposure (risk) and confounding factors also
  verified.

37
Registries - Epidemiology

• Prospective cohort studies
• Identify eligible subjects -- representative of
  the registry (general) population
• Formally enroll subjects (consent) with a
  systematic follow-up schedule
• Careful characterization of exposure (risk) and
  confounding factors
• Specific outcome reporting (assessments) with
  adjudication.

38
Registries - Epidemiology

• Embedded cohort study
• Identify eligible subjects
• Enroll subjects (consent)
• Establish a schedule of assessments to be
  conducted as part of routine care
• Send notices to patients when visits due
• Capture exposure (risk) and confounding factors
• Identify possible outcomes through medical
  reports, with subsequent adjudication.

39
Registries - Epidemiology

• A hybrid
• Establish an embedded cohort study.
• Also implement a formal prospective study in a
  sub-sample.
• The latter can serve as a quality check on the
  former.

40
Registries - Epidemiology

• LARGE Sample Size
• N needed to detect a rare outcome (e.g. fulminant
  hepatotoxicity, or angioedema)
• If risk is 1 in 10,000, need N 29,956 to be 95
  confident that at least one case will be
  observed.
• If wished to have 85 power to detect a 50
  increased risk, at least 75 events required.
• N 836,000 followed for 1 year!!

41
Conclusions

• Registry can provide superior descriptions of
  quality of care and distribution of factors in
  broad population of interest.
• Not as rigorous as a formal prospective
  epidemiologic study, but can form the basis for
  such studies.
• Affords opportunities for large sample sizes
  needed to detect rare outcomes.