Analysis of Repeated Measures Will G Hopkins, Auckland University of Technology, Auckland, NZ

1
Analysis of Repeated MeasuresWill G Hopkins,
Auckland University of Technology, Auckland, NZ
A tutorial lecture presented at the 2003 annual
meetingof the American College of Sports Medicine

• This presentation applies to continuous or
  ordinal numeric dependent variables, including
  data from most Likert scales.

• It does not apply to nominal dependent variables
  or variables representing counts or frequencies.

• Make sure you view this presentation as a full
  slide show, to get the benefit of the build-up of
  information on each slide.

2
OVERVIEW
Basics

• What change has occurred in response to a
  treatment/intervention?
• Analysis by ANOVA, within-subject modeling, mixed
  modeling.
• Fixed and random effects individual responses
  and asphericity.

Accounting for Individual Responses

• What is the effect of subject characteristics on
  the change?

Analyzing for Patterns of Responses

• What is the treatment's effect on trends in
  repeated sets of trials?

Analyzing for Mechanisms

• How much of the change was due to a change in
  whatever?

3
Basics What change has occurred in response to a
treatment or intervention?
4
Basics Interventions

• A repeated measure is a variable measured two or
  more times, usually before, during and/or after
  an intervention or treatment.

• Y
• Dependent variable
• Repeated measure

• Analysis by ANOVA, t statistics and
  within-subject modeling, and mixed modeling.

5
Basics Analysis by ANOVA

• Data are in the form ofone row per subject

• Select columns to definea within-subjects factor.

• If there is no control group, use a 1-way
  repeated-measures ANOVA
• The 1 way is Trial
• "(How) does Trial affect Y?"
• With a control group, use a 2-way
  repeated-measures ANOVA.

• The 2 ways are Group and Trial.
• You investigate the interaction Group?Trial
• "(How) does Trial affect Y differently in the
  different groups?"

6
Basics Analysis by t Statistics and
Within-Subject Modeling

• If there is no control group, use a paired t
  statistic to investigate changes between
  interesting measurements.

• With a control group, calculate change scores and
  use the unpaired t statistic to investigate the
  difference in the changes.

• Use un/paired t statistics for other interesting
  combinations of repeated measurements. I call it
  within-subject modeling.
• Example time course of an effect

7
Basics More Within-Subject Modeling
Ann

• To quantify a time course
• fit lines or curves to each subject's points
• predict interesting things for each subject
• analyze with un/paired t statistic.
• Method 1. Fit lines Y a b.T
• At Time 0 and 3, Y a and a3b.
• Change in Y b per week.
• Method 2. Fit quadratics Y a b.T c.T2
• At Time 0 and 3, Y a and a3b9c.
• Change in Y 3b9c over 3 weeks.
• Maximum occurs at Time -b/(2a).
• Method 3. Fit exponentials Y a b.eT/c
• Needs non-linear curve fitting to estimatetime
  constant c.

Bev
Lyn
Y
May
0
1
2
3
Time (wk)
8
Basics Analysis by Mixed Modeling

• Data are in the form of one row per subject per
  trial

• Analysis is via maximizing likelihood of
  observed valuesrather than ANOVA's approachof
  minimizing error variance.
• You investigate fixed effects
• Trial, if there's only one group.
• Group?Trial, if there's morethan one group.
• You also specify and estimate random effects.
• "Mixed" fixed random.
• Some mixed models are also known as hierarchical
  models.

9
Basics Fixed Effects

• Fixed effects are differences or changes in the
  dependent variable that you attribute to a
  predictor (independent) variable.
• They are usually the focus of our research.
• Their value is the same (fixed) for everyone in a
  group.
• They have magnitudes represented by differences
  or changesin means.
• Example of difference in means
• girls' performance 48
• boys' performance 56
• so effect of sex (maleness) on performance 56
  48 8.
• Example of change in a mean
• girls' performance in pretest 48
• girls' performance after a steroid 56
• so effect of the steroid on girls' performance
  56 48 8.

10
Basics Random Effects

• Random effects have values that vary randomly
  within and/or between individuals.
• They provide confidence limits or p values for
  the fixed effects.
• They provide other valuable information usually
  overlooked.
• They are mostly hidden in ANOVA, are accessible
  in t tests, and are up front in mixed modeling.
• They are the key to understanding repeated
  measures.
• They have magnitudes represented by standard
  deviations (SD).
• Examples of between-subject SD or random effects
• Variation in ability SD of girls' performance
  (Y) 9.2
• Individual responses SD of effect of a steroid
  on Y 5.0,
• so you can say the effect of the steroid is 8.0
  5.0 (mean SD).
• Example of a within-subject SD or random effect
• Error of measurement SD of any girl's Y in
  repeated tests 2.0

11
Basics The "Hats" Metaphor for Random Effects

• When you measure something, it's like adding
  together numbers drawn from several hats.
• Each hat holds a zillion pieces of paper, each
  with a number.
• The numbers are normally distributed with mean
  0, SD ??
• Example measure a girl's performance several
  times.

Suppose true mean performance of all girls 48.3

• The random effects in SAS are Girl and Girl?Trial
  ( the residuals).

12
Basics Hats plus a Fixed Effect

• Example give steroid with a fixed effect of 8.0
  between Trials 1 and 2, and measure several
  girls.

• Subject hat not shown.

• The stats program uses them to estimate the fixed
  and random effects.

13
Basics A Hat for Individual Responses

• Example different responses to the steroid.

• To estimate the SD for individual responses, you
  need a control group (see later) or an extra
  trial for the treatment group.

14
Basics Individual Responses and Asphericity

• It's important to quantify individual responses,
  but
• More importantly, they are the most frequent
  reason for the asphericity type of non-uniform
  error in repeated measures.
• You must somehow eliminate non-uniformity of
  error to get trustworthy confidence limits or p
  values.
• Here's the deal on asphericity.
• Conventional ANOVA is based on the assumption
  that there is only one random-effects hat, error
  of measurement.
• We can use ANOVA for repeated measures by turning
  the subjects random effect into a subjects fixed
  effect.
• But it doesn't work properly when there is
  asphericity that is,more than one source of
  error, such as individual responses.
• There are four approaches to the asphericity
  problem.

15
Basics Dealing with Asphericity in Repeated
Measures

• Four approaches
• MANOVA (multivariate ANOVA)
• (Univariate) ANOVA with adjustment for
  asphericity
• Within-subject modeling with the
  unequal-variances t statistic
• Mixed modeling
• I base my assessment of these approaches mainly
  on my experience with the Statistical Analysis
  System (SAS).
• Other stats programs may produce different output.

16
Basics MANOVA/adjusted ANOVA for Asphericity
(NOT!)

• Both these approaches involve different
  assumptions about the relationship between the
  repeated measurements.
• They produce an overall p value for each fixed
  effect.
• Incredibly, the p value is too small if sample
  size and individual responses differ between
  groups.
• Adjusted ANOVA (Greenhouse-Geisser or
  Huynh-Feldt) is worse than MANOVA.
• Subjects with any missing value are first
  deleted.
• So there is needless loss of power, if the
  missing value is for a minor repeated measurement
  (e.g., post2).
• In the old-fashioned approach, you are allowed to
  "test for where the difference is" only if the
  overall plt0.05.
• So there is further loss of power, because you
  could fail to detect an effect on the overall p
  or the subsequent test.

17
Basics More on MANOVA/adjusted ANOVA

• The overall p value is OK when the extra random
  effects are the same in both groups, even when
  sample sizes differ.
• Example two repeated-measures factors for
  example, several measurements on one day repeated
  at monthly intervals.
• The program then does p values for the requested
  contrasts (differences in the changes e.g., post
  pre for exptal control).
• These comparisons are simply equal-variance t
  tests.
• So the p values are too small if sample size and
  individual responses differ between groups.
• There is no adjustment other than Bonferroni for
  inflation of Type I error for contrasts involving
  repeated measures.
• Good! But researchers still dial up Tukey or
  other adjustments and think that the resulting p
  values are adjusted. They're not.
• In summary avoid MANOVA and adjusted ANOVA.

18
Basics Unequal-Variances t Statistic Deals with
Asphericity

• Example controlled trial of effect of the
  steroid on performance.

Variance of postpre change scores

• Big differences in variances.
• So use unequal-variances t statistic to analyze
  changes.
• Bonus estimate of individual responses as an SD
  ?(SDChgExpt2 SDChgCont2)

19
Basics Summary of t Statistic for Repeated
Measures

• Advantages
• It works!
• It's robust to gross departures from
  non-normality, provided sample size is
  reasonable.
• 10 in each group is forgiving, 20 is very
  forgiving.
• Missing values are not a problem.
• Because you analyze separately the changes of
  interest.
• Students can do most analyses with Excel
  spreadsheets.
• Include my spreadsheet for confidence limits and
  clinical/practical/mechanistic probabilities.
• You can include covariates by moving to simple
  ANOVAs or ANCOVAs of the change scores.
• Example how does age modify the effect of the
  steroid on performance? (See later.) But

20
Basics More on t Statistic for Repeated Measures

• Disadvantages
• ANOVAs or ANCOVAs of the change scores aren't
  strictly applicable, if variances of the change
  scores differ markedly.
• You can't easily get confidence limits for the SD
  representing individual responses.
• That is, I don't have a formula or spreadsheet
  yet.
• There's always bootstrapping, but it's hard work.
• The disdain of editors and peer reviewers, most
  of whom think state of the art is
  repeated-measures ANOVA with post-hoc tests
  controlled for inflation of Type I error.
• In conclusion, I recommend within-subject
  modeling using unequal-variances t statistic for
  analysis of straightforward data.
• Otherwise use mixed modeling

21
Basics Mixed Modeling for Asphericity

• You take account of potential sources of
  asphericity by including them as random effects.
• Advantages
• It works!
• Impresses editors and peer reviewers.
• Confidence limits for everything.
• Complex fixed-effects models are relatively easy
• individual responses, patterns of responses,
  mechanisms
• Disadvantages
• Not available in all stats programs.
• Takes time and effort to understand and use.
• The documentation is usually impenetrable.
• Sample size for robustness to non-normality not
  yet known.

22
Accounting forIndividual Responses What is the
effect of subject characteristics on the change?
23
Individual Responses and Subject Characteristics

• Subjects differ in their response to a treatment

• due to subject characteristics interacting
  with the treatment.
• It's important to measure and analyze their
  effect on the treatment.
• Using value of Trialpre as a characteristic needs
  special approach to avoid artifactual regression
  to the mean. See newstats.org.
• Use mixed modeling, ANOVA, or within-subject
  modeling.

24
Individual Responses by Mixed Modeling

• You include subject characteristics as covariates
  in the fixed-effects model.
• The SD representing individual responses will
  diminish and represent individual responses not
  accounted for by the covariate.
• The precision of the estimates of the fixed
  effects usually improves, because you are
  accounting for otherwise random error.
• Covariates can be nominal (e.g., sex) or numeric
  (e.g., age).
• Example how does sex affect the outcome?
• First, you can avoid covariates by analyzing the
  sexes separately.
• Effect on females 8.8 units effect on males
  4.7 units.
• Effect on females males 8.8 4.7 4.1
  units.
• You can generate confidence limits for the 4.1
  "manually", by combining confidence limits of the
  effect for each sex.
• Include individual responses for each sex 8.8
  5.2 4.7 2.5.

25
Individual Responses More Mixed Modeling

• The full fixed-effects model is Y ? Group?Trial
  Sex?Group?Trial.
• The term Sex?Group?Trial yields the female-male
  difference of 4.1 units (90 confidence limits
  1.5 to 6.7, say).
• The overall effect of the treatment (from
  Group?Trial) is for an average of equal numbers
  of females and males.
• Try including random effects for individual
  responses in males and females.
• Example how does age affect the outcome?
• Either convert age into age groups and analyze
  like sex.
• Or if the effect of age is linear, use it as a
  numeric covariate.
• Age?Group?Trial provides the outcome as effect
  per year 1.3 units.y-1 (90 confidence limits
  -0.2 to 2.8).
• Note that the overall effect of the treatment is
  for subjects with the average age.

26
Individual Responses by Repeated-Measures ANOVA

• It is possible in principle to include a subject
  characteristic as a covariate in a
  repeated-measures ANOVA.
• But SPSS (Version 10) provides only the p value
  for the interaction. Incredibly, it does not
  provide magnitudes of the effect.
• If a covariate accounts for some or all of the
  individual responses, the problem of asphericity
  will diminish or disappear.
• I don't know whether it's possible to extract the
  SD representing individual responses from a
  repeated-measures ANOVA, with or without a
  covariate.

27
Individual Responses by Within-Subject Modeling

• Calculate the most interesting change scores or
  other within-subject parameters

• If no control group, analyze effect of subject
  characteristics on change score with unpaired t,
  regression, or 1-way ANOVA.
• With a control group, analyze with ?2-way ANOVA.
• As before, a characteristic that accounts
  partially for individual responses will reduce
  the problem of asphericity.

28
Analyzing for Patterns of Responses What is the
effect of a treatment on trends within repeated
sets of trials?
29
Patterns of Responses Bouts within Trials

• Typical example several bouts for each of
  several trials.

• We want to estimate the overall increase in Y in
  the exptal group in the mid and post trials, and
• the greater decline in Y in the exptal group
  within the mid and post trials (representing, for
  example, increased fatigue).
• Use mixed modeling, ANOVA, or within-subject
  modeling.

30
Patterns of Responses by Mixed Modeling and ANOVA

• With mixed modeling, Bout is simply another
  (within-subject) fixed effect you add to the
  model.
• The model is Y ? Trial Bout Trial?Bout.
• Bout can be nominal or numeric.
• If numeric, Bout specifies the slope of a line,
  and Trial?Bout specifies a different slope for
  each level of Trial.
• Add Bout?Bout(?Trial) to the model for
  quadratic(s).
• Elegant and easy, when you know how.
• With ANOVA, you have to specify Bout as a nominal
  effect and try to take into account
  within-subject errors using adjustments for
  asphericity.
• Specifying a quadratic or higher-order polynomial
  Bout effect is possible but difficult (for me,
  anyway).
• Within-subject modeling is much easier

31
Patterns of Responses by Within-Subject Modeling

• The trick is to convert the multiple Bout
  measurements into a single value for each
  subject, then analyze those values.

• In the example, derive the Bout mean and slope
  (or any other parameters) within each trial for
  each subject.

• Derive the change in meanand the change in
  slopebetween pre and post(or any other Trials)
  for each subject.

• For the changes in the mean, do an unpaired t
  test between the exptal and control groups. Ditto
  for the changes in the slope.
• Simple, robust, highly recommended!

32
Analyzing for Mechanisms How much of the change
was due to a change in whatever?
33
Analyzing for Mechanisms

• Mechanism variable something in the causal path
  between the treatment and the dependent variable.
• Necessary but not sufficient that it "tracks" the
  dependent.

• Important for PhD projects or to publish in
  high-impact journals.
• It can put limits on a placebo effect, if it's
  not placebo affected.
• Can't use ANOVA can use graphs and mixed
  modeling.

34
Mechanisms Why not ANOVA?

• For ANOVA, data have to be one row per subject

• You can't use ANOVA, because it doesn't allow you
  to match up trials for the dependent and
  covariate.

35
Mechanisms Analysis Using Graphs

• Choose the most interesting change scoresfor the
  dependent and covariate

• Then plot the change scores

36
Mechanisms More Analysis Using Graphs

• Three possible outcomes with a real mechanism
  variable

• The covariate is an excellent candidate for a
  mechanism variable.

37
Mechanisms More Analysis Using Graphs

• Three possible outcomes with a real mechanism
  variable

2. Apparently poor tracking of individual
responses
but it could be due to noise in either variable.

• The covariate could still be a mechanism variable.

38
Mechanisms More Analysis Using Graphs

• Three possible outcomes with a real mechanism
  variable

• The covariate is a good candidate for a mechanism
  variable.

39
Mechanisms Graphical Analysis how NOT to

• Relationship between change scores is often
  misinterpreted.

?

• "The correlation between change scores for X and
  Y is trivial.
• Therefore X is not the mechanism."

?
?

• "Overall, changes in X track changes in Y well,
  but
• Noise may have obscured tracking of any
  individual responses.
• Therefore X could be a mechanism variable."

?
40
Mechanisms Quantitative Analysis by Mixed
Modeling - 1

• Need to quantify the role of the mechanism
  variable, with confidence limits.
• I have devised a method using mixed modeling.
• Data format isone row per trial

41
Mechanisms More Quantitative Analysis by Mixed
Modeling

• Run the usual fixed-effects model to get the
  effect of the treatment.
• Example 4.6 units (90 likely limits, 2.1 to 7.1
  units).
• Then include a putative mechanism variable in the
  model.
• The model is then effectively a multiple linear
  regression, so
• You get the effect of the treatment with the
  mechanism variable held constant
• which means the same as the effect of the
  treatment not explained by the putative mechanism
  variable.
• Example it drops to 2.5 units (90 likely
  limits, -1.0 to 7.0 units).
• So the mechanism accounts for 4.6 - 2.5 2.1
  units.
• If the experiment was not blind, the real effect
  is gt2.1 units
• and the placebo effect is lt2.5 units...
• provided the mechanism variable itself is not
  placebo affectible!

42
Summary
Basics

• Use the unequal-variance t statistic and
  within-subject modeling for straightforward
  models.
• Repeated-measures ANOVA may not cope with
  non-uniform error.
• Mixed modeling is best for fixed and random
  effects.

Accounting for Individual Responses

• Use within-subject modeling or mixed modeling.

Analyzing for Patterns of Responses

• Use within-subject modeling or mixed modeling.

Analyzing for Mechanisms

• Interpret graphs of change scores properly.
• Use mixed modeling to get estimates of the
  contribution of a mechanism variable.

43
This presentation was downloaded from
A New View of Statistics
newstats.org
SUMMARIZING DATA
GENERALIZING TO A POPULATION
Simple Effect Statistics
Precision of Measurement
Confidence Limits
Statistical Models
Dimension Reduction
Sample-Size Estimation