Short and long term prognosis of disability in Multiple Sclerosis

1

• Short and long term prognosis of disability in
  Multiple Sclerosis
• Some Tools, Models and Validation

A. Neuhaus, M. Daumer
2
Outline

• Background about MS
• Online Analytical Processing Tool Risk Profile
• Segmented Regression and Correction for Error
• Validation Strategy examples

3
Background
Multiple Sclerosis (MS)

• common neurological degenerative disease
• 2.5 million people affected worldwide
• drugs have shown efficacy on short-term outcomes
• agents are by no means cure many patients
  have disease activity
• long term determination of efficacy is necessary

4
Background
Multiple Sclerosis (MS)
Cause
CNS
Disease courses
Multiple Sclerosis
Disability
MRI
Relapse
5
Background
Multiple Sclerosis (MS)
Cause

• Specific cause is unknown
• female male 2 1
• more common in
• Caucasians
• autoimmune process
• environmental factors
• genetic predisposition

CNS
Disease courses
MRI
Disability
MRI
Relapse
http//medstat.med.utah.edu
6
Background
Multiple Sclerosis (MS)
Cause unknown
CNS
Disease courses
Disability
MRI
Relapse
http//www.msdecisions.org.uk
7
Background
Multiple Sclerosis (MS)
Cause unknown
CNS
Disease courses
Number of enhancing lesions
Lesion Volume
Disability
MRI
Relapse
8
Background
Multiple Sclerosis (MS)
Cause unknown
CNS

• Sudden failures in functional systems
• Recovery after a few days or weeks
• vision problems
• problems with walking
• tremor
• difficulties with speech
• fatigue
• bladder and bowel problems

Disease courses
Disability
MRI
Relapse
9
Background
Multiple Sclerosis (MS)
Cause unknown
CNS
Disease courses
Disability
MRI
Relapse
sudden failures
10
Background
Multiple Sclerosis (MS)
Cause unknown
CNS
Disease courses
Disability
MRI
EDSS
Relapse
sudden failures
11
Outline

• Background
• OnLine Analytical Processing Tool
• Segmented Regression and Correction for Error
• Improvement of Outcome Measures
• Validation Strategies

12
OLAP
Aim

• Make the database of the SLCMSR (20.000
  patients, 45 data sets)available to health care
  professionals via the internet
• Identification of database subgroups based on
  clinical parameters
• Statistical analyses of subgroups
• Illustration of future disease course of
  subgroups

13
OLAP
Tool

• OnLine Analytical Processing Tool (OLAP-Tool)
• accessible via the internet
• no need for data transfer
• no need for local software installation
• server based on Java and R
• Individual Risk Profile (IRP)
• 1059 MS patients from placebo arms
• of controlled clinical trials
• definition of patient profile
• display the course of database patients
• with same characteristics

14
OLAP
Hurdles

• Patient profile can be defined by combining

Age at MS onset
Disease Duration
Number of Relapses
EDSS
Course
?
1.059 patients
15
OLAP
Hurdles

• if a few or no matching patients are found
• weight characteristics according to their
  importance

determine weights by means of
number of relapses in the first year
increase of disability
Linear Regression
Poisson Regression
16
OLAP
Selection of most similar patients
17
OLAP
Outcomes
18
OLAP
Outcomes
19
OLAP
Outcomes
20
OLAP
Next steps

• Evaluate performance of expert opinion vs.
  tool/model (Validation)
• Include patient history treatment data
• Develop and validate models for predicting
  treatment (non-)responders
• OLAP tool for evidence based decision support
  when to switch treatment(Disease Management)
• Prospective evaluation in a clinical trial if
  promising

Similar to path taken for CTG monitoring
21
Outline

• Background
• OnLine Analytical Processing Tool
• Segmented Regression and Correction for Error
• Improvement of Outcome Measures
• Validation Strategies

22
Models
Problem

• What are the factors affecting the start of the
  progressive phase?
• What are the factors predicting subsequent
  disability best?

Joint work with J. Noseworthy, Mayo clinic,
Rochester, USA, L. Kappos, Basel, CH, T.
Augustin H. Küchenhoff, LMU, Munich, Germany
23
Models
Restrictions to data

• Patients in the first phase of the disease
  (RRMS)
• disability level lt 6.5
• inclusion in a controlled clinical trial
• at least 4 observations in longitudinal data
• complete data in covariates

355 RRMS patients from placebo arms of 16
clinical trials
24
Models
Data
Mean S.D. Range Female to male
ratio 2.7 Age of onset (years) 28.3 7.0 13
48 Disease duration before entry (years) 7.0
6.1 0.7 34.8 Observation period
(months) 26.6 12.7 2.8 59.3 EDSS score at
entry 2.7 1.4 0 6 Relapse rate 2 years
prior study 1.5 0.7 0 4
25
Models
Methods

• Two Step Analysis

EDSS
Segmented Regression Model
Time to progressive phase
Survival Analysis (with error correction)
Predictive factors
26
Models
Methods / Segmented Regression Model

• piecewise linear regression
• model describes disease
• process D

D?ß?(t) ? ß (t-?)
0 ? ? ? 10, ß gt 0, ? gt 0 and (t-?) max(0, t-?)

• dispersion of estimates

27
Models
Methods / Survival Analysis

• Correct determination of time of change is
  impossible
• ?, estimated time to progression, is overlaid
  by an error e
• magnitude of the error will be considered in
  the survival model

• Assumptions
• true, but unknown, event times t follow a
  Weibull distribution
• relation between ? and e ? t e, t ? e
• ? log ? log t log e
• log ? xß ?(? ?), ? ?-1 log e
• exp -? ?(?,?)

• Survival function follows a Burr distribution

S(?) 1 exp(-x'?)??-1 ?-2 ?2- ?2?-2
?2 var (log ?) and ? ?2 ?-2
28
Models
Methods / Survival Analysis

• Regression parameter are specified using
• maximum likelihood estimation

• The log-likelihood is given by

29
Models
Methods / Survival Analysis
Weibull regression
Error adjusted regression
Parameter Std.Error p-Value Parameter Std.Error
p-Value Intercept 6.86 0.25 lt0.001 7.59 0.22 lt0
.001 Relapse rate 0.20 0.11 0.08 EDSS
-0.10 0.06 0.07 -0.15 0.07 0.02 Log
(scale) -0.08 0.07 0.22 0.02 0.09 0.84 Scale
0.92 0.98
The higher the EDSS level, the shorter the time
to progression. Higher relapse rate longer
time to progression? Importance of relapse rate
instable.
30
Models
Methods / Survival Analysis
31
Outline

• Background
• OnLine Analytical Processing Tool
• Segmented Regression and Correction for Error
• Improvement of Outcome Measures
• Validation Strategies

32
Validation
Need for validation Model selection

• Over-fitting of data
• Scenario
• Many models checked for describing data set
• Model with best fit is used for further analyses
• Model fit is tested using standard statistical
  methodology
• Result
• Danger of over-fitting since model selection
• and model validation is based on same dataset
• Danger enhanced if method applied to small
  subgroups

33
Validation
Need for validation Data driven hypotheses

• Theory
• Neither the model nor the hypothesis to be
  tested should be data driven
• Practice
• Data are visualized before models are fit and,
  frequently, before
• hypotheses are formulated
• Effect
• Promising hypotheses are being tested
• Actual level of tests far exceed nominal levels,
  leading to a large
• number of false positive results

34
Validation
Our strategy Splitting of data set
Open part
Closed part
Learning or training sample
Confirmation or validation sample
Development of tools Statistical analyses
Confirmation of findings - Final result
Free investigation of data set
Significant results
35
Validation
SLCMSR Database

• Inclusion/exclusion criteria
• Plausibility check
• Harmonization/homogenization
• Pooling
• Split into
• training sample (40) and validation sample
  (50)
• ? Analysis / modeling

10
Mixing sample
40
50
Training sample
Validation sample
36
Validation
Validation Procedure

• Validation concept validation results of
  open
• part of SLCMSR database are sent to
  Validation Committee
• Validation Committee approves proposed
  validation concept
• or alternately suggests specific
  modifications for consideration
• by the authors of the project
• Data trustee executes analysis agreed upon by
  Validation Committee
• and authors, programming code is provided by
  project team
• Validation Committee and authors agree upon
  formulation
• of results summary

37
Validation
Examples

• Relapses and subsequent worsening of disability
  in RRMS
• Occurrence of relapses in the first 3 months on
  study appeared
• to be the best predictor for a shorter
  subsequent time to sustained
• increase of the EDSS.
• Signif. even after naïve Bonferroni adjustment
  for multiple testing.
• BUT Unable to validate this on an independent
  (validation) part
• of the SLCMSR dataset relationship between
  relapses and subsequent disability either
  non-existent or very weak
• Correlating T2 lesion burden on MRI with the
  clinical manifestations
• of multiple sclerosis (Li, Held et al,
  submitted to Neurology)
• Question How does one validate a plateauing
  relationship?
• Visualization, with CI for Spearmans
  correlation coefficient and
• significant improvement in model fit
  with non-linear approach
• Validation was successful there is a plateau,
  lesion load doesnt increase with disability, no
  good surrogate marker

38
Validation
Examples

• How to predict on-study relapse rate? (Held et
  al, Neurology, in press)
• ? validation was successful pre-study relapse
  rate is the most important predictor for future
  relapse rate. MRI information doesnt add much.

39
Validation
Invited Session for IBC 2006, Montreal

• Session organizers M. Daumer, U. Held (SLCMSR)
• Discussant John Petkau (Prof. of Statistics,
  UBC, Vancouver)
• Speakers
• Trevor Hastie (Prof. of Statistics, Stanford
  University)
• Validation in Genomics
• Ulrike Held (SLCMSR)
• Validation Procedure of the SLCMSR
• Methodological and Practical Aspects
• Martin Schumacher (Prof. of Biometry, Freiburg
  University, GER)
• Assessment and Validation of Risk
• Prediction Models

40
Validation
41
Literature
Barkhof F, Held U, Simon JH, Daumer M, Fazekas F,
Filippi M, Frank JA, Kappos L, Li D, Menzler S,
Miller DH, Petkau J, Wolinsky J. Predicting
gadolinium-enhancement status in MS patients
eligible for randomized clinical trials.
Neurology in press Compston A, Ebers G, Lassmann
H, McDonald I, Matthews B, Wekerle H. Mc Alpines
Multiple Sclerosis 3rd Edition, Churchill
Livingstone, 1998. Freedman MS, Patry DG,
Grand'Maison F, Myles ML, Paty DW, Selchen DH.
Treatment optimization in multiple sclerosis, Can
J Neurol Sci 33 (2)157-68, 2004. Held U,
Heigenhauser L, Shang C, Kappos L, Polman C.
Predictors of relapse rate in MS clinical trials.
Neurology in press Küchenhoff H. An exact
algorithm for estimating breakpoints in segmented
generalized linear models, Computational
Statistics 12, 235 247, 1997. Kurtzke JF.
Rating neurologic impairment in multiple
sclerosis An expanded disability status scale
(EDSS), Neurology 33(11)1444-52, Nov.
1983. Pittock SJ, Mayr WT, McClelland RL,
Jorgensen NW, Weigand SD, Noseworthy JH,
Weinshenker BG, and Rodriguez M. Change in
MS-related disability in a population-based
cohort A 10-year follow-up study. Neurology 62
51-59, 2004. Hellriegel B, Daumer M, Neiß A.
Analysing the course of multiple sclerosis with
segmented regression models, Tech. rep.,
Ludwig-Maximilians-University Munich, SFB
Discussion Paper, 2003. Skinner CJ, Humphreys K.
Weibull Regression for Lifetimes Measured with
Error, Lifetime Data Analysis 5, 23-37,
1999. Neuhaus A. Modelling Time to Progression
in Multiple Sclerosis, Diploma Thesis,
Ludwig-Maximilians-University Munich,
http//www.slcmsr.org, 2004 Schach S, Daumer M,
Neiß A. Maintaining high quality of statistical
evaluations based on the SLCMSR data base -
Validation Policy, http//www.slcmsr.org. Ioannid
is PDA. Why most publishes research findings are
false, PLoS Med 2(8) e124, 2005. Ioannidis PDA.
Microarrays and molecular research noise
discovery?, Lancet 365 454-55, 2005.
42
Outline

• Background
• OnLine Analytical Processing Tool
• Segmented Regression and Correction for Error
• Improvement of Outcome Measures
• Validation Strategies

43
Outcome Measures

• Time to progression

• Time to sustained worsening/progression
• widely used outcome measure in
• Phase III clinical trials
• outcome depends on confirmation period
• effective study duration is shortened since
• last visit(s) can only be used as confirmation

44
Outcome Measures

• Definition of sustained worsening divides cohort
  in 3 groups

• current procedure

• What about ?

• consideration of confirmation period
• consideration of visit schedule

45
Outcome Measures

• random matching of
• non-confirmed worsening
• to one of the other groups

Estimation based on standard definition Estimatio
n without non-confirmed patients
room for improvement
46
Models
Methods / Segmented Regression Model
n 158
n 129
n 68
within observation period 1.2 y /- 1.0 y 0.01 y
4.6 y
Prior to first observation
after last observation 1.9 y /- 1.1 y 0.2 y
4.6 y
Estimated start of progressive phase
censoring times