Experimental design

1
Experimental design

• Dag M Eide
• Nasjonalt Folkehelseinstitutt

2
Present some common themes

• Level of measurement
• Define experimental unit
• Briefly some experimental designs
• Discuss in detail Crabbe al
• The design used
• Briefly sample size estimation

3
Level of measurement

• Categorical or nominal data
• lowest level of measurement
• least information
• Stats Khi-squares
• Logistic regr.
• Probit, logit

4
Level of measurement 2

• Ordinal data
• Categorical, but ordered
• More information
• May be sorted
• Difficult statistics
• MAY be treated as interval data
• Regression, ANOVA

5
Level of measurement

• Continuous or interval data
• Highest level of measurement
• ALWAYS possible to introduce an intermediate
  value
• May be sorted
• Allows parametric statistics
• Sample size est.
• ANOVA, Regression etc

6
Interval data

• Maximum information
• allows parametric statistics
• transformation of data
• parametric statistics is..
• E.g. based on normally distributed error term
• Pearsons correlations
• linear regression
• ANOVA, MANOVA

7
Result table may look like
8
Part II Defining the experimental unit

• You want to test two vaccines

9
Trial 1

• What i N i this case?

10
Trial 2

• N in this case?
• BUT you can not use this set-up always..

11
How to analyze an experiment with
non-independence between observations?

• SAS Proc mixed, specify subject and covariance
  structure
• Proc mixed datamouse
• Class cage drug
• Model bloodpressure drug
• Random intercept / subject cage structurevc
• Run
• S-plus use grouping

12
PART III Experimental designs

• How can you use your animals most effectively
• Parallel (randomized) design
• Crossover designs
• Factorial design
• Sequential design
• Other designs

13
Parallel (randomized) design

• one treatment for each animal

Groups
Time (days)
14
Notes on parallel design

• Large sample size
• groups may not be comparable
• selected base population
• Can be improved
• stratification (see also factorial d.)
• Repeated measurements (they are not independent!)

15
Crossover design

• All animals go through all treatments

Time
16
Notes on crossover design

• saves N
• saves experimental units
• wash out must be real
• Time consuming
• Equal group size mandatory
• Easily biased
• Sequence of groups
• Latin square w/variations
• multiple cross over

17
Multiple cross over

• Few animals, many experimental units
• Animals their own controls
• Similar comments as std cross over

Group 1
Group 2
18
Sequential design

• When it is not ethical to run a complete
  factorial or parallel design
• Ex Cytostatica evaluation
• Decision process
• Define stop citeria
• Start testing matched pairs
• Calculate the differences within pair
• Continue till borders are crossed
• Stop the experiment, make decision

19
Factorial design

• Do you want to test several effects on the same
  material simultaneously?
• Do you have blocking of data, like
• Several animal strains/species?
• Several test sites?
• Different cages?
• Performing the study at different time?
• try FACTORIAL design

20
Factorial set-up

• Highly effective design
• Ex WE wish to test
• Two vaccines
• two adjuvants
• In several mouse strains
• Easy set-up (in eg. JMP)
• Allows the resource equation method for sample
  size estimation

21
Crabbe al Factorial design

• 8 genotypes
• 3 universities
• 2 sexes
• 2 sources of animals
• 8 X 3 X 2 X 2 table

Download the original paper Go to the related
website
22
Table cells in the design
23
How to estimate sample size

• And when.

24
Determination of sample size

• Power Analysis method
• Power??
• ..depends on what?
• Resource Equation method

25
Parametric tests require 1

• Continuous dependent variables - what you measure
  in your animals
• Interval data

26
Parametric tests require 2

• Normally distributed error term
• Recorded data need not be normally distrib.

Identical data, but changed set point half way
through the experiment.
27
You may transform data to yield normality

• Logarithms, Square root on left-skewed data
• Arcsine on -data

28
Transformation successful!
Blood enzyme levels
Square root transformation -gt
29
Power analysis depends on 6 variables

• Simplest decision first
• 1 or 2-sided test
• Significance level (p-value)
• The chosen power
• The standard deviation
• The effect of treatment
• The group (sample) size - your interest.

30
When do you need plt0.05?

• Use p0.05 most of the time, EXCEPT when you work
  with
• Single locus effects (X-s)
• Many dependent variables (Y-s)
• Multiple comparisons (What?)
• AND you run a lot of tests on the same material

31
Multiple comparisons

• Crabbe al (1999)
• 8 mouse strains
• What is the difference between B6 and A?
• B6 and C?
• B6 and 129?
• Etc..

32
The standard deviation

• Def The spread of the data above and below the
  mean
• How do we guesstimate?
• Pilot study
• Others studies
• SD from papers
• RMSE from anova

33
The Power of the experiment

• Statistical power means
• Probability of not making a type II error
• That is the Probability that your experiment
  will detect a difference THAT REALLY EXISTS.
• 2 B6 and 2 A/J will not detect the difference
• Any Exception?
• Some times your data are so clear that statistics
  is not needed

34
The effect of treatment

• The difference between the means (average) for
  the groups you want to test may be large or small
• Easy to determine?
• one of the most difficult tasks in power
  analysis

35
Power analysis - what if..table
36
The Resource Equation Method

• E N T B
• E error degrees of freedom
• N Total degrees of freedom
• T Treatment degrees of freedom
• B Block degrees of freedom
• Aim for 10ltElt20
• Solve for N NT B E
• (Mead 1988)

37
Conclusion RE-method

• (Crabbe al paper)
• 64 OK
• 96 number of cells
• What is correct?
• Maybe a power analysis on the whole lot? (like
  the authors did)