Interrupted Time Series: What, Why and How - PowerPoint PPT Presentation

About This Presentation
Title:

Interrupted Time Series: What, Why and How

Description:

Interrupted Time Series: What, Why and How An Example From Suicide Research Karen Smith Design Considerations Post-hoc analysis Very difficult to identify suitable ... – PowerPoint PPT presentation

Number of Views:137
Avg rating:3.0/5.0
Slides: 41
Provided by: KarenS176
Category:

less

Transcript and Presenter's Notes

Title: Interrupted Time Series: What, Why and How


1
Interrupted Time SeriesWhat, Why and How
An Example From Suicide Research
  • Karen Smith

2
Acknowledgement
  • Motivated by consultancy work with the Centre for
    Suicide Research, University of Oxford
  • All analyses and graphs produced by Helen Bergen,
    Centre for Suicide Research

3
  • Motivating example
  • What is Interrupted Time Series?
  • Why use it?
  • Design issues
  • Analysis issues
  • Guidelines on use

4
Motivating Example
  • Between 1997 and 1999 the analgesic co-proxamol
    was the single drug used most frequently for
    suicide by self-poisoning in England and Wales,
    with 766 over the 3 year period
  • There is a relatively narrow margin between
    therapeutic and potentially lethal levels
  • Death occurs largely because of the toxic effects
    of dextropropoxyphene on respiration and cardiac
    conduction
  • MHRA conducted a review of the efficacy/safety
    profile
  • Committee on Safety of Medicines advised
    withdrawal from use in the UK, the final date
    being 31 December 2007
  • Patients who find it difficult to move to an
    alternative drug can still be prescribed
    co-proxamol

5
The Problem
  • How to evaluate the impact of the announcement to
    withdraw co-proxamol on
  • Prescribing of analgesics
  • Mortality involving co-proxamol
  • Mortality involving other analgesics
    (substitution of method is of concern)

6
Available Data
  • Quarterly data on prescriptions of co-proxamol,
    cocodamol, codeine, codydramol, dihydrocodeine,
    NSAIDs, paracetamol and tramadol (from
    Prescription Statistics department of the
    Information Centre for Health and Social Care,
    England, and Prescribing Service Unit, Health
    Solutions Wales)
  • Quarterly data on drug poisoning deaths
    (suicides, open verdicts and accidental
    poisonings) involving co-proxamol alone,
    cocodamol, codeine, codydramol, dihydrocodeine,
    NSAIDs, paracetamol and tramadol, based on death
    registrations in England and Wales (from ONS)
    single drug, with and without alcohol
  • Quarterly data for overall drug poisoning deaths
    and for all deaths receiving suicide and
    undetermined verdicts

7
Simple Analysis
  • Compare the proportion of deaths involving
    co-proxamol prior to the legislation with
    proportion following legislation
  • Compare total number of poisoning deaths before
    and after legislation
  • Time series plots of prescriptions and deaths
  • Co-proxamol withdrawal has reduced suicide from
    drugs in Scotland, E. A. Sandilands D. N.
    Bateman, British Journal of Clinical
    Pharmacology, 2008.

8
Whats Wrong With This?
  • Ignores any trends, both before and after change
    in legislation (or intervention in a more general
    setting)
  • Ignores any possible cyclical effects
  • Doesnt pick up on any discontinuity
  • Variances around the means before and after the
    intervention may be different
  • Effects may drift back toward the
    pre-intervention level and/or slope over time if
    the effect wears off
  • Effects may be immediate or delayed
  • Doesnt take account of any possible
    autocorrelation

9
A Solution Interrupted Time Series
  • A special kind of time series in which we know
    the specific point in the series at which an
    intervention occurred
  • Causal hypothesis is that observations after
    treatment will have a different level or slope
    from those before intervention the interruption
  • Strong quasi-experimental alternative to
    randomised design if this is not feasible

10
Ramsay et al, 2003
11
The Model
  • Use segmented regression analysis (Wagner et al,
    2002)
  • Yt ß0 ß1 x timet ß2 x interventiont ß3 x
    time_after_interventiont et
  • Yt is the outcome
  • time indicates the number of quarters from the
    start of the series
  • intervention is a dummy variable taking the
    values 0 in the pre-intervention segment and 1 in
    the post-intervention segment
  • time_after_intervention is 0 in the
    pre-intervention segment and counts the quarters
    in the post-intervention segment at time t
  • ß0 estimates the base level of the outcome at the
    beginning of the series
  • ß1 estimates the base trend, i.e. the change in
    outcome per quarter in the pre-intervention
    segment
  • ß2 estimates the change in level in the
    post-intervention segment
  • ß3 estimates the change in trend in the
    post-intervention segment
  • et estimates the error

12
Threats to Validity
  • Forces other than the intervention under
    investigation influenced the dependent variable
  • Could add a no-treatment time series from a
    control group
  • Use qualitative or quantitative means to examine
    plausible effect-causing events
  • Instrumentation how was data collected/recorded
  • Selection did the composition of the
    experimental group change at the time of
    intervention?
  • Poorly specified intervention point diffusion
  • Choice of outcome usually have only routinely
    collected data
  • Power, violated test assumptions, unreliability
    of measurements, reactivity etc.

13
Design Considerations
  • Add a non-equivalent no-treatment control group
  • Add non-equivalent dependent variables
  • Intervention should not affect but would respond
    in the same way as primary variable to validity
    threat
  • Remove intervention at a known time
  • Add multiple replications
  • Add switching replications

14
Problems
  • Interventions implemented slowly and diffuse
  • Effects may occur with unpredictable time delays
  • Many data series much shorter than the 100
    observations recommended for analysis
  • Difficult to locate or retrieve data
  • Time intervals between each data point in archive
    may be longer than needed
  • Missing data
  • Undocumented definitional shifts

15
Applied to the Co-Proxamol Data
  • 28 quarters in the pre-intervention period and 12
    in post-intervention
  • Examined a number of common analgesics
  • Prescriptions
  • Deaths
  • Examined overall suicides
  • Some evidence of autocorrelation in the data,
    hence Cochrane-Orcutt autoregression used (Durbin
    Watson statistic of final models close to 2)

16
Prescriptions for analgesics dispensed in
England and Wales, 1998-2007
excluding liquids, suppositories, granules,
powders and effervescent preparations
17
Mortality in England and Wales from analgesic
poisoning (suicide and open verdicts), 1998-2007,
for persons aged 10 years and over (substances
taken alone, /- alcohol)

18
Prescriptions
Pre-intervention Pre-intervention Pre-intervention Pre-intervention Post-intervention Post-intervention Post-intervention Post-intervention
Base level, ß0 (SE) p Base trend, ß1 (SE) p Change in level, ß2 (SE) p Change in trend, ß3 (SE) p
Co-proxamol 3050.1 (139.9) lt0.001 -45.9 (7.7) lt0.001 -554.8 (74.9) lt0.001 -46.8 (16.8) 0.01
Cocodamol 1349 (12.4) lt0.001 34.1 (0.8) lt0.001 300.5 (53.6) lt0.001 30.7 (6.4) lt0.001
Codeine 204.8 (4.2) 0.007 9.6 (0.2) lt0.001 20.5 (11.6) 0.089 3.5 (1.2) 0.007
Codrydamol 1055.2 (6.1) lt0.001 -1.1 (0.3) 0.004 148.7 (35.8) lt0.001 -4.2 (4.1) 0.316
Dihydrocodeine 686.4 (31.1) lt0.001 -1.5 (1.4) 0.291 -29.7 (2.4) lt0.001 -0.8 (2.4) 0.731
NSAIDs 4652.4 (47.2) lt0.001 28.4 (3) lt0.001 -622.9 (70.2) lt0.001 -66.2 (8.5) lt0.001
Paracetamol 1493.4 (56.1) lt0.001 42.1 (3.0) lt0.001 232 (66.6) 0.001 23 (8.2) 0.01
Tramadol 47.2 (51.4) 0.365 31.4 (2.7) lt0.001 -41.9 (6.8) lt0.001 16.3 (5.2) 0.004
19
Prescriptions
Pre-intervention Pre-intervention Pre-intervention Pre-intervention Post-intervention Post-intervention Post-intervention Post-intervention
Base level, ß0 (SE) p Base trend, ß1 (SE) p Change in level, ß2 (SE) p Change in trend, ß3 (SE) p
Co-proxamol 3050.1 (139.9) lt0.001 -45.9 (7.7) lt0.001 -554.8 (74.9) lt0.001 -46.8 (16.8) 0.01
Cocodamol 1349 (12.4) lt0.001 34.1 (0.8) lt0.001 300.5 (53.6) lt0.001 30.7 (6.4) lt0.001
Codeine 204.8 (4.2) 0.007 9.6 (0.2) lt0.001 20.5 (11.6) 0.089 3.5 (1.2) 0.007
Codrydamol 1055.2 (6.1) lt0.001 -1.1 (0.3) 0.004 148.7 (35.8) lt0.001 -4.2 (4.1) 0.316
Dihydrocodeine 686.4 (31.1) lt0.001 -1.5 (1.4) 0.291 -29.7 (2.4) lt0.001 -0.8 (2.4) 0.731
NSAIDs 4652.4 (47.2) lt0.001 28.4 (3) lt0.001 -622.9 (70.2) lt0.001 -66.2 (8.5) lt0.001
Paracetamol 1493.4 (56.1) lt0.001 42.1 (3.0) lt0.001 232 (66.6) 0.001 23 (8.2) 0.01
Tramadol 47.2 (51.4) 0.365 31.4 (2.7) lt0.001 -41.9 (6.8) lt0.001 16.3 (5.2) 0.004
20
Prescriptions
Pre-intervention Pre-intervention Pre-intervention Pre-intervention Post-intervention Post-intervention Post-intervention Post-intervention
Base level, ß0 (SE) p Base trend, ß1 (SE) p Change in level, ß2 (SE) p Change in trend, ß3 (SE) p
Co-proxamol 3050.1 (139.9) lt0.001 -45.9 (7.7) lt0.001 -554.8 (74.9) lt0.001 -46.8 (16.8) 0.01
Cocodamol 1349 (12.4) lt0.001 34.1 (0.8) lt0.001 300.5 (53.6) lt0.001 30.7 (6.4) lt0.001
Codeine 204.8 (4.2) 0.007 9.6 (0.2) lt0.001 20.5 (11.6) 0.089 3.5 (1.2) 0.007
Codrydamol 1055.2 (6.1) lt0.001 -1.1 (0.3) 0.004 148.7 (35.8) lt0.001 -4.2 (4.1) 0.316
Dihydrocodeine 686.4 (31.1) lt0.001 -1.5 (1.4) 0.291 -29.7 (2.4) lt0.001 -0.8 (2.4) 0.731
NSAIDs 4652.4 (47.2) lt0.001 28.4 (3) lt0.001 -622.9 (70.2) lt0.001 -66.2 (8.5) lt0.001
Paracetamol 1493.4 (56.1) lt0.001 42.1 (3.0) lt0.001 232 (66.6) 0.001 23 (8.2) 0.01
Tramadol 47.2 (51.4) 0.365 31.4 (2.7) lt0.001 -41.9 (6.8) lt0.001 16.3 (5.2) 0.004
21
Suicides
Pre-intervention Pre-intervention Pre-intervention Pre-intervention Post-intervention Post-intervention Post-intervention Post-intervention
Base level, ß0 (SE) p Base trend, ß1 (SE) p Change in level, ß2 (SE) p Change in trend, ß3 (SE) p
Co-proxamol 81.0 (4.5) lt0.001 -1.194 (0.3) lt0.001 -28.3 (4.9) lt0.001 0.6 (0.6) 0.355
Other analgesics 51.3 (2.8) lt0.001 -0.3 (0.2) 0.095 6.4 (6.0) 0.297 -0.3 (0.6) 0.724
All drugs except co-proxamol and other analgesics 221.2 (7.3) lt0.001 -0.5 (0.5) 0.299 21.6 (12.8) 0.100 -5.4 (1.4) lt0.001
All drugs 353.7 (10.2) lt0.001 -2.0 (0.7) 0.008 0.004 (18.1) 1.000 -4.9 (1.7) 0.007
All causes 1319.0 (22.5) lt0.001 -4.8 (1.4) 0.002 12.8 (34.8) 0.716 -5.4 (4.1) 0.192
22
Estimating Absolute Effect
  • The model may be used to estimate the absolute
    effect of the intervention. This is the
    difference between the estimated outcome at a
    certain time after the intervention and the
    outcome at that time if the intervention not
    taken place.
  • For example, to estimate the effect of the
    intervention at the midpoint of the
    post-intervention period (when time 34.5 and
    time_after_intervention 6.5), we have
  • Y34.5 ß0 ß1 x 34.5 without
    intervention
  • Y34.5 ß0 ß1 x 34.5 ß2 ß3 x 6.5 with
    intervention
  • Thus, the absolute effect of the intervention is
  • ß2 ß3 x 6.5
  • Standard errors calculated using method of Zhang
    et al
  • s22 6.52 x s32 2 x 6.5 x s23
  • Non-significant terms included due to correlation
    between slope and level terms

23
Results - Prescriptions
Estimates of absolute effect during 2005 to 2007 Estimates of absolute effect during 2005 to 2007 Estimates of absolute effect during 2005 to 2007 Estimates of absolute effect during 2005 to 2007
Mean quarterly estimated number pre announcement Mean quarterly number post announcement Mean quarterly change (95 CI)
Prescriptions (x1000)
Co-proxamol 1465.1 605.7 -859 (-1065 to -653)
Cocodamol 2524.7 3024.6 500 (459 to 540)
Codeine 534.6 578 43 (31 to 55)
Codrydamol 1018.2 1140.0 122 (99 to 145)
Dihydrocodeine 634.6 600.0 -35 (-68 to -2)
NSAIDs 5633.8 4581.0 -1053 (-1186 to -920)
Paracetamol 2947.0 3330.0 382 (268 to 497)
Tramadol 1130.1 1193.9 64 (-5 to 133)
24
Results - Deaths
Estimates of absolute effect during 2005 to 2007 Estimates of absolute effect during 2005 to 2007 Estimates of absolute effect during 2005 to 2007 Estimates of absolute effect during 2005 to 2007
Mean quarterly estimated number pre announcement Mean quarterly number post announcement Mean quarterly change (95 CI)
Suicides, Open
Co-proxamol 39 15 -24 (-37 to -12)
Other analgesics 39 44 5 (-5 to 15)
All drugs except co-proxamol and other analgesics 204 191 -13 (-34 to 8)
All drugs 283 252 -31 (-66 to 3)
All causes 1152 1130 -22 (-89 to 45)
25
Co-Proxamol Prescriptions
  • Prescription data for England and Wales showed a
    steep reduction in prescribing of co-proxamol in
    the first two quarters of 2005, with further
    reductions thereafter.
  • Regression analyses indicated a significant
    decrease in both level and slope in prescribing
    of co-proxamol - the number of prescriptions
    decreased by an average of 859 (95 confidence
    interval (CI) 653 to 1065) thousand per quarter
    in the post-intervention period.
  • This equates to an overall decrease of
    approximately 59 in the three year
    post-intervention period, 2005 to 2007.

26
Other Analgesic Prescriptions
  • There were also significant decreases in
    prescribing of NSAIDS of an average of 1053 (95
    CI 920 to 1186) thousand per quarter, equating
    to an approximate 19 decrease overall for 2005
    to 2007 and for dihydrocodeine of an average of
    35 (95 CI 2 to 68) thousand per quarter, or
    approximately 6 overall for 2005 to 2007.
  • Prescriptions for the other analgesics increased
    significantly in the post-intervention period,
    apart from tramadol. Based on mean quarterly
    estimates this equated to percentage increases
    over the 2005 to 2007 period of approximately 20
    for cocodamol, 13 for paracetamol, 12 for
    codydramol, and 8 for codeine.

27
Co-Proxamol Deaths
  • Marked reduction in suicide and open verdicts
    involving co-proxamol in the first quarter of
    2005, which persisted until the end of 2007.
  • Prior to 2005 deaths due to co-proxamol alone
    were 19.5 (95 CI 16.9 to 22.2) of all drug
    poisoning suicides, whereas between 2005 and 2007
    they constituted just 6.4 (95 CI 5.2 to 7.5).
  • Regression analyses indicated a significant
    decrease in both level and slope for deaths
    involving co-proxamol which received a suicide or
    open verdict - decreased by on average 24 (95 CI
    12 to 37) per quarter in the post-intervention
    period.
  • This equates to an estimated overall decrease of
    295 (95 CI 251 to 338) deaths, approximately
    62, in the three year post-intervention period
    2005 to 2007.
  • When accidental poisoning deaths involving
    co-proxamol were included, there was a mean
    quarterly decrease of 29 (95 CI 17 to 42)
    deaths, equating to an overall decrease of 349
    (95 CI 306 to 392) deaths, approximately 61,
    in the three year post-intervention period 2005
    to 2007.

28
Other Deaths
  • There were no statistically significant changes
    in level or slope in the post-intervention period
    for deaths involving other analgesics (cocodamol,
    codeine, codydramol, dihydrocodeine, NSAIDs,
    paracetamol and tramadol) which received a
    suicide or open verdict (both including and
    excluding accidental deaths).
  • There was a substantial though not statistically
    significant reduction during the
    post-intervention period in deaths (suicide and
    open verdicts) involving all drugs (including
    co-proxamol and other analgesics), with the mean
    quarterly change between 2005 and 2007 being -31
    (95 CI -66 to 3) deaths.
  • The overall suicide rate (including open
    verdicts) during this period also decreased,
    though to a lesser extent, and the mean quarterly
    change of -22 (95 CI -89 to 45) deaths was not
    statistically significant.

29
Substitution of Method of Suicide
  • Possible substitution of method must be
    considered in estimating the effect of changing
    availability of a specific method of suicide.
  • Research evidence on failed suicide attempts
    suggests that its unusual for completely
    different method to be used.
  • Withdrawal of co-proxamol was associated with
    changes in prescribing of other analgesics.
  • Significant increases in prescribing of
    co-codamol, paracetamol, and codydramol occurred
    during 2005-2007.
  • Analyses of suicide and open verdict deaths
    involving other analgesics combined indicated
    little evidence of substitution.

30
NSAIDs
  • An abrupt reduction in prescribing of NSAIDs
    occurred shortly before the announcement of the
    withdrawal of co-proxamol due to concerns about
    Cox 2 inhibitors.
  • However, NSAIDs are rarely a direct acute cause
    of death, especially by suicide.

31
Interpretation
  • Following the announcement of the withdrawal of
    co-proxamol in January 2005 there was an
    immediate large reduction in prescriptions. This
    was associated with a 62 reduction in suicide
    deaths (including open verdicts), or an estimated
    295 fewer deaths.
  • Inclusion of accidental deaths, some of which
    were likely to have been suicides increased the
    estimated reduction in number of deaths to
    approximately 349 over 3 years.
  • Overall suicide and open verdict deaths decreased
    in England and Wales during 2005 to 2007. Thus
    underlying downward trends in suicide cannot
    explain the full extent of the decrease in
    co-proxamol related deaths following the MHRA
    announcement to withdraw co-proxamol.

32
Limitations
  • Interrupted time series autoregression controls
    for baseline level and trend when estimating
    expected changes in the number of prescriptions
    (or deaths) due to the intervention.
  • The estimates of the overall effect on
    prescriptions and mortality involved
    extrapolation, which is inevitably associated
    with uncertainty.
  • The regression method assumes linear trends over
    time, and the co-proxamol prescribing data, in
    particular, had a poor fit, resulting in large
    standard errors in the post-intervention period.
  • Estimates of the standard errors for absolute
    mean quarterly changes in number of prescriptions
    or deaths were determined exactly, including the
    covariance of level and slope terms.
  • Estimates of percentage changes over the three
    year post-estimation period are point estimates
    and were not determined with standard error
    calculations.

33
Threats to Validity
  • Co-proxamol prescription only drug, often given
    to elderly patients suffering arthritis
  • Most poisoning suicides are in younger people, so
    co-proxamol use considered to be opportunistic
  • Recording of suicide may be coroner dependent
  • Open verdict may be given when there is lack of
    suicide note
  • Prescriptions numbers were reducing as GPs tried
    to move patients to alternative analgesics prior
    to withdrawal so some diffusion of intervention

34
Design Considerations
  • Post-hoc analysis
  • Very difficult to identify suitable
    non-equivalent no treatment control group
  • Inclusion of overall suicide rates goes some way
    towards examination of validity threat

35
Problems
  • Diffusion of intervention but in this case prior
    to identified intervention point as demonstrated
    by decrease in prescriptions prior to
    announcement
  • Data series rather shorter than the ideal of 100
    observations, but minimum of 12 before and after
    intervention point considered not unreasonable
  • Quarterly figures give reasonable time intervals
  • Theres no concrete evidence of a definitional
    shift in suicides in this time interval although
    this cannot be ruled out
  • Open verdicts were included as a sensitivity
    check as one way of addressing potential missing
    data
  • Almost impossible to consider impact of missing
    data on agent used in self-poisoning

36
Guidelines on Use
  • Ramsay et al, 2003
  • Quality criteria
  • Intervention occurred independently of other
    changes over time
  • Intervention was unlikely to affect data
    collection
  • The primary outcome was assessed blindly or was
    measured objectively
  • The primary outcome was reliable or was measured
    objectively
  • The composition of the data set at each time
    point covered at least 80 of the total number of
    participants in the study
  • The shape of the intervention effect was
    prespecified
  • A rationale for the number and spacing of data
    points was described
  • The study was analyzed appropriately using time
    series techniques

37
Findings of the Systematic Reviews
  • Mass media review of 20 studies, Guideline
    dissemination and implementation review of 38
    studies
  • Most studies had short time series
  • Standard errors increased
  • Reduced power
  • Type I error increased
  • Failure to detect autocorrelation or secular
    trends
  • Over 65 analysed inappropriately
  • Of the 37 re-analysed, 8 had significant
    pre-intervention trends
  • Most were underpowered
  • Rule of thumb with 10 pre- and 10
    post-intervention time points the study would
    have at least 80 power to detect a change in
    level of five standard deviations of the pre-data
    if the autocorrelation gt0.4
  • Long pre-intervention phase increases power to
    detect secular trends

38
References
  • Shadish, Cook and Campbell, 2002, Experimental
    and quasi-experimental designs for generalised
    causal inference, Houghton Mifflin.
  • Ramsay CR, Matowe L, Grilli R, Grimshaw JM,
    Thomas RE. Interrupted time series designs in
    health technology assessment Lessons from two
    systematic reviews of behavior change strategies.
    Int.J.Technol.Assess.Health Care 200319613-23
  • Wagner AK, Soumerai SB, Zhang F, Ross-Degnan D.
    Segmented regression analysis of interrupted time
    series studies in medication use research.
    J.Clin.Pharm.Ther. 200227299-309
  • Zhang, F, Wagner, A, Soumerai, S. B., and
    Ross-Degnan, D. Estimating confidence intervals
    around relative changes in outcomes in segmented
    regression analyses of time series data. 15th
    Annual NESUG (NorthEast SAS Users Group Inc)
    Conference Last update 2002. http//www.nesug.inf
    o/Proceedings/nesug02/st/st005.pdf. Accessed 22
    October 2008.

39
Examples of Use
  • Matowe, L, Ramsay, C. R., Grimshaw, J. M.,
    Gilbert F. J., MacLeod, M.-J. and Needham, G.
    Effects of mailed dissemination of the Royal
    College of Radiologists Guidelines on general
    practitioner referrals for radiography a time
    series analysis. Clinical Radiology 2002, 57,
    575-578
  • Neustrom, M. W. and Norton, W. M. The impact of
    drunk driving legislation in Louisiana. Journal
    of Safety Research, 1993, 24, 107-121
  • Ansari, F, Gray, K, Nathwani, D, Phillips, G,
    Ogston, S, Ramsay, C and Davey, P. Outcomes of an
    intervention to improve hospital antibiotic
    prescribing interrupted time series with
    segmented regression analysis. Journal of
    Antimicrobial Chemotherapy, 2003, 52, 842-848
  • Morgan, O. W., Griffiths, C and Majeed, A.
    Interrupted time-series analysis of regulations
    to reduce paracetamol (acetaminophen) poisoning.
    PLoS Medicine, 2007, 4, 0654-0659

40
  • K. Hawton, H. Bergen, S. Simkin, A. Brock, C.
    Griffiths, E. Romeri, K. L. Smith, N. Kapur, D.
    Gunnell (2009). Effect of withdrawal of
    co-proxamol on prescribing and deaths from drug
    poisoning in England and Wales time series
    analysis. BMJ, 338b2270
Write a Comment
User Comments (0)
About PowerShow.com