Experimental Design How to organise a successful research project Including notes on sample size and - PowerPoint PPT Presentation

1 / 20
About This Presentation
Title:

Experimental Design How to organise a successful research project Including notes on sample size and

Description:

Experimental Design. How to organise a successful research project ... and Statistics for Life Scientists, Bios Instant Notes, 2005 [mostly maths] ... – PowerPoint PPT presentation

Number of Views:99
Avg rating:3.0/5.0
Slides: 21
Provided by: bruce51
Category:

less

Transcript and Presenter's Notes

Title: Experimental Design How to organise a successful research project Including notes on sample size and


1
Experimental DesignHow to organise a successful
research projectIncluding notes on sample size
and reliability
  • Research Methods Lecture
  • Bruce Lynn
  • Director, MSc School of human Health and
    Performance
  • March 09

2
Experimental DesignHow to organise a successful
research project
  • The initial idea
  • Reading the literature
  • Planning
  • Doing
  • Writing

3
Generating the initial idea
  • Something you always wanted to investigate
  • Look at past projects and shamelessly borrow
    ideas
  • Read around a subject of interest looking for
    loose ends
  • Talk to one of the labs associated with the MSc
    Programmes

4
Reading the literature
  • Essential
  • Make sure project has originality
  • Find out about the methods used in the field of
    interest

5
PlanningThe key to a successful research project
  • Start early
  • Decide what measures to use
  • Who are the test population?
  • What is being compared to what?
  • Statistical issues

6
What measure(s) to use?
  • Questionnaires.
  • Find suitable validated instruments.
  • Try developing your own?
  • Physiological measures
  • Explore availability of equipment
  • Investigate suitable analytical software
  • Example
  • Investigating heart rate variability.
  • Find suitable heart rate monitoring equipment,
    e.g. Polar
  • Find software package to analyse beat by beat
    data
  • Decide which HRV parameter(s) are appropriate

7
Who are the test population?
  • Biological investigations, e.g. animal studies,
    or studies of tissue properties, or cadaver
    anatomy studies
  • Healthy volunteer studies
  • Patient studies
  • All are regulated re ethics. In particular
    patient studies need Central (NHS) ethical
    clearance (NRES)
  • Recruitment for human studies
  • Posters
  • Letters or e-mails
  • Will subjects be paid if so who funds this?
  • Subject information sheets
  • Will enough subjects be available within the time
    period available for the project?

8
What is being compared with what?Experimental
designs (1)
  • Example
  • A double blind random controlled trial (RCT)
  • Prospective subjects are checked out in advance
    of intervention
  • Controlled. 2 (or more) groups run in parallel.
    One receiving an ineffective treatment that will
    appear equivalent to the real treatment, thus
    controlling for any placebo effect. This is
    often the hardest part of any design.
  • Randomisation. Must not select subjects. More
    info in a minute.
  • Subject blinds. Can be impossible. E.g. testing
    health education literature.
  • Observer blinds. Nearly always possible. Need an
    assistant to organise this. May not be necessary
    if outcome measure is very objective, e.g. blood
    count.
  • Dropout rates. Must be monitored. Differential
    dropout from control or treatment group can
    introduce bias.

9
What is being compared with what?Experimental
designs
  • Other designs
  • Crossover trials. Control group subsequently
    receives treatment and vice versa. Each subject
    own control good feature, but carry-over of
    effects is a problem.
  • Non-experimental designs surveys
  • Longitudinal. Following same subjects for period
    of time.
  • Cross-sectional. Looking at sample of
    population at one time point.
  • Retrospective. Looking back at health records
    of patients.

10
More on randomisation
Some random numbers from a published table 03 47
43 73 86 36 96 47 36 61 97 74 24 67 62 42 81
14 57 20 16 76 62 27 66 56 50 26 71 07 12 56 85
99 26 96 96 68 27 31 55 59 56 35 64 38 54 82
46 22
If you were using odd for treatment and even for
control, then note long runs of up to 8 all the
same. When n is small, these occasional long
runs can upset the balance of the study
  • So pure random sequences can cause problems.
    Need to remove long runs. Also need all numbers
    to appear within any given sequence the same
    number of times for this use random
    permutations.
  • Often useful to ensure same number of treatment
    (T) and control (non-treatment, NT) within blocks
    of subjects. So every 10, make sure you have
    equal T/NT even in a design for 30. Means if you
    have to stop short, still have roughly equal T
    and NT
  • Often need to ensure balance by randomising
    separately for male/female, old/young.
    severe/less severe etc as appropriate. Make sure
    you have extra random allocations in sub-groups
    if not sure of numbers, e.g. for M/F

11
Statistical issues how many subjects
  • Key statistical issue in the planning phase is
    whether enough subjects can be studied to get a
    clear result.
  • Involves the question of statistical power.
  • Need to know 4 things
  • ? The criterion for rejecting the null
    hypothesis, normally 5
  • The criterion for missing a possible result,
    normally 10 or 20 (NB 100-?, is the Power
    of the study)
  • The standard deviation of the outcome measure. Or
    if using counted data or another non-continuous
    measure, then the appropriate error measure
  • d The extent of change to be detected.
  • For a comparison of 2 groups with same n,
    formulae for required n is
  • n 16?2/d2

12
Statistical issues how many subjects
  • Example (From Statistics at Square One,
    http//www.bmj.com/collections/statsbk/13.dtl)
  • BP Trial, comparing 2 groups. Want to find 5mm Hg
    change if it occurs standard deviation in
    population is 10mm Hg.
  • n 161010/55 64
  • If we decide only a 10mm Hg change is of
    significance to us, then
  • n 161010/1010 16
  • So sample size needed depends very much on
    criterion we set.
  • Obviously, the smaller the standard deviation in
    the measure/population, the smaller n will be.
    How do we find out s.d. in advance of the study!!
    Can find literature values. Or do a small pilot
    study.
  • Also, if we set a tougher criterion for rejecting
    the null hypothesis, e.g. 1, then n rises.
  • A more detailed discussion of this issue, and the
    formulae for other types of design and measure,
    is in Kirkwood, Essentials of Medical Statistics,
    Chapt 26

13
Sample size for standardised difference at power
80 or 90
  • This is for comparing an average result against a
    fixed criterion or for paired data where SD is
    SD(diff) for comparing 2 groups need to double n
    in each group and use SD(error) i.e. RMS(within)
    from ANOVA or SD(diff)/sqrt(2).
  • Difference you wish to detect is given in
    standardised form, i.e. difference/standard
    deviation. e.g Diff to detect in actual units
    3, SD of repeated measurements in population
    being studied 5, then "standardised" diff to
    detect is 3/5 0.6

Note the very large effect of the difference you
choose to set as your criterion. Note also that
this is the ratio of the actual difference and
the error SD. So if you can reduce errors (by
better technique etc), can often greatly reduce
number needed for study.
14
Sources of error
  • Systematic leading to bias
  • Random
  • Bias is the bigger problem. Can have apparently
    clear result that is wrong. Bias may be caused
    by inadequate randomisation, or not ensuring good
    match between T and NT samples, or inadequate
    technique (e.g. not maintaining calibration of
    equipment during study)
  • Random error will not lead to false positives
    but will of course lead to more false negatives.
    Important to spend time optimising methods. Also
    to consider using only a relatively homogeneous
    population, e.g. restrict to young males.

15
Reliability statistics
  • Need to know how reliable a measurement is.
  • Is it repeatable, e.g on test-retest.
  • Also if comparing groups, how much variation
    within the group?
  • Comparison with other methods the calibration
    problem.

16
Reliability statistics
  • Repeatability
  • 1. Is it repeatable, e.g on test-retest.
  • 2. Comparing groups, how much variation within
    the group?
  • Need this information to compare different
    methods
  • Importantly need it for sample size estimation
  • Can just take a lot of measurements and calculate
    SD
  • Often have pairs of measurements. The SD(diffs),
    the SD of the differences between the values in
    each pair, will be SD21/2
  • See Reliability.doc and the example on
    Reliability.xls, sheetTest-retest on WebCT or
    at http//www.archway.ac.uk/Activities/Departments
    /SHHP/current/Res_Methods/resmet1.htm

17
Reliability statistics
  • Comparison of new method with a standard method
  • Sometimes referred to as validity, but this term
    also covers wider aspects of any test.
  • Common approach is to measure same subjects with
    both methods
  • Then can do SD(diffs) as for test-retest
    comparisons
  • Often to methods are plotted as an XY plot, with
    correlation coefficient, r used to assess
    reliability
  • This is bad practice, r depends on range of X
    values as well as errors. So more variable
    methods tested on a wide range of subjects can
    look better than a less variable methods tested
    over a more limited range.
  • See Reliability.doc and the example on
    Reliability.xls, sheet Comparison on WebCT or
    at http//www.archway.ac.uk/Activities/Departments
    /SHHP/current/Res_Methods/resmet1.htm

R 0.85 SD(diffs) 5.5ml/min.kg
18
Doing the studyThe fun bit!
  • Usually need to run initial trials to learn
    methods, establish reliability etc
  • Then plan the main data collection. Try to keep
    things standardised
  • If unexpected observations occur, then DO FOLLOW
    THEM UP
  • Back up data. Keep back ups in a separate place
    from the main data storage. Hard disc crashes do
    occur!!

19
Sorting out and analysing the data
  • Excel is useful.
  • Be sure to graph data before you analyse.
  • Visualising the data usually indicates the real
    trends your eyes are good at statistics!
  • Look at average trends normalise data where
    appropriate.
  • For most regression, ANOVA or non-parametric
    tests you will need a statistical package. SPSS
    is on the cluster machines but is rather user
    unfriendly, so expect to spend some time finding
    your way around.

20
Resources
  • Statistics at square one, last chapter. Good
    introduction to statistical issues.
    http//www.bmj.com/collections/statsbk/
  • Kirkwood, B.R. Essentials of Medical Statistics,
    Chapts 21-26
  • Bland, M. An Introduction to Medical Statistics,
    Chapt 15, Reliability measures Chapt 18, smaple
    size
  • Field, A. Discovering Statistics using SPSS
    Detailed on regression good explanation of SPSS
    computer package
  • Mackenzie, A. Mathematics and Statistics for
    Life Scientists, Bios Instant Notes, 2005 mostly
    maths
  • Random permutations
  • http//calculators.stat.ucla.edu/perm.php
  • http//www.webcalculator.co.uk/statistics/rpermut
    e3.htm
  • NRES (National Research Ethics Service)
  • http//www.nres.npsa.nhs.uk/
Write a Comment
User Comments (0)
About PowerShow.com