Read by Tuesday February 28th and bring copies to class

1
Read by Tuesday February 28th and bring copies
to class

• Class workshop Critical appraisal of 2 outcome
  measure articles
• Owens E et al. Paraspinal skin temperature
  patterns an interexaminer and intraexaminer
  reliability study
• Humphreys et al. An investigation into the
  validity of cervical spine motion palpation using
  subjects with congenital block vertebrae as a
  'gold standard'

2
Properties of Useful Clinical Outcome Measures

• Used in clinical practice as well as research

3
Outcome Measures

• The tools used to track the progress of a
  patient's health
• The more measurable or quantifiable the OM is the
  better it is
• ROM is a good example
• A patient stating they are feeling a little
  better today is a bad example
• OMs are collectively used to assess outcomes

4
Outcomes Assessment (AO)

• The collection and recording of information
  relative to healthcare processes
• Measurements of quality, satisfaction, and
  effectiveness are essential for making healthcare
  decisions
• Also needed to develop health policy
• A means of documenting medical necessity
  (primarily for third party payers)

5
OA Supports Continuing Care
6
Properties of Useful Outcomes

• Utility
• The outcome measure must be useful
• Reliability
• It must be dependable
• Validity
• It must be able to measure what it is supposed to
  measure
• Relatively easy to use
• Cost effective

7
Properties of Outcomes Cont.

• Sensitivity
• An OMs ability to identify patients with a
  specific condition
• Specificity
• An OMs ability to identify those who do not have
  the condition
• Responsiveness
• Ability to measure differences over time

8
Useful Measures of Outcome

• Questionnaires concerning
• General health status
• Pain
• Functional status
• Patient satisfaction
• Physiological outcomes
• i.e., ROM, orthopedic tests, blood tests
• Utilization measures
• Cost measures

9
OA Tools

• Classified as being subjective or objective
• Subjective
• Relies on information derived from patients
• Some think they are unreliable, but research has
  shown the opposite
• Objective
• Tests carried out on patients
• ROM, strength, proprioception, etc.

10
Subjective OA Tools

• General Health
• Pain Perception
• Condition-specific
• Psychometric
• Disability Prediction
• Patient Satisfaction

11
General Health OA Tools

• These tools can be used for all patients,
  regardless of their ailment
• Since they are not condition specific
• Because they deal with quality of life issues can
  be used for patients with HAs, back pain,
  digestive problems, etc.
• Will usually parallel the patients level of
  disability

12
General Health OA Tools Cont.

• A weakness of general health OA tools
• Other co-existing health conditions may cloud
  the main problem
• Should be combined with a specific outcome
  measure such as a pain scale

13
Pain Perception OA Tools

• Pain is always subjective
• Its measurement is primarily dependant upon the
  patient
• Also how the practitioner draws out responses
  during the history and exam
• Scales are often used where the patient chooses a
  level of pain that best matches theirs
• i.e., mild, moderate, severe
• or scaled 0-10

14
Pain Perception Cont.

• Pain scales
• Visual analog scales
• Patient marks their pain level on a line that
  represents a scale ranging from no pain to
  terrible pain
• Numerical pain scales
• A range of numbers (usually from 0-10) with 0
  being no pain and 10 being worst imaginable
• Patient circles a number that is closest to their
  pain

15
Pain Perception Cont.

• McGill Pain Questionnaire
• Consists of 20 pain-related items
• Short form is 15 items
• Pain Diagrams
• Patient draws a representation of their pain on
  an outline of a body
• Key to help them characterize the pain
• i.e., 000 ache, XXX stabbing, /// burning

16
Condition-specific OA Tools

• Spinal and non-spinal subcategories
• Spinal condition-specific instruments
• Numerous types are available
• Also includes those that deal with headaches,
  dizziness, and others
• Non-spinal
• Extremities, knees, shoulders, etc.
• Asthma, arthritis, depression, etc.

17
Psychometric OA Tools

• Include various instruments designed to evaluate
  
• Affect
• Depression/Psychological status
• Malingering and symptom magnification
• Expression of a subjectively experienced emotion
  like joy or anger

18
Disability Prediction OA Tools

• Questionnaires designed to help predict patients
  who are likely to become problem cases caused
  by
• Chronic pain
• Work dissatisfaction
• Fear avoidance

19
Patient Satisfaction OA Tools

• Assess patients perception regarding
• Acceptance of care
• Technical competence of the doctor
• Setting where care was provided
• Effectiveness of the doctor

20
Questionnaires

• Subjective outcomes information is gathered from
  the patient
• Self-administered
• Scored by either the doctor or staff
• Some are computerized
• Typically very valid and reliable
• Often more so than objective tests like x-rays,
  examination findings, etc.

21
Questionnaires Cont.

• The same outcome measures should be used
  throughout the course of treatment with each
  patient
• Provide valid evidence to third parties in
  support of the care being rendered
• Questionnaires are often called survey
  instruments
• Especially in epidemiology

22
Physiological Outcomes

• Physical performance tests
• Strength and endurance
• Psychosocial
• How the patient reacts to pain stimulus
• ROM and flexibility
• Including inclinometer evaluation, SLR, etc.
• Proprioception
• One-leg standing test
• Functional activity measures
• e.g., Timed up and go test

23
NOIR Applies to Outcomes Measures

• Nominal
• Palpating for the presence or absence of
  subluxation
• Ordinal
• Grading muscle strength on a 0-5 scale
• Interval
• Temperature
• Ratio
• Range of motion

24
Outcome Meanings

• Health Care Customer - Meaning of Outcomes
• Payers-purchasers Cost containment
• Regulators Compliance
• Administrators Efficiency - low
  utilization
• Clinical Researchers Support for a premise
• Outcomes Experts Patients benefit
• Health Care Providers Clinical-health status
• (Hansen DT, Mior S, Mootz RD in Yeomans SG
  The Clinical Application of Outcomes Assessment,
  Stamford Connecticut, Appleton Lange, 2000)

25
Reliability Experimental Designs
26
Why Bother With Test Validity Reliability?

• Whenever a test or other measuring device is used
  as part of the data collection process, the
  validity and reliability of that test is
  important
• It would be wrong to use a test that does not
  truly measure what its purported to measure
• Use of a thermometer to test BP (extreme)
• Use of AK to diagnose a brain tumor

27
Test Validity Reliability? Cont.

• Test results are relied upon to show patient
  progress (or lack of)
• If the measures are erroneous, the diagnosis and
  treatment plan may also be erroneous
• Measurement tools used in practice must be both
  valid and reliable

28
So What!

• Some tests/devices marketed to chiropractors are
  not valid/reliable
• You should question new or non-standard tests and
  equipment before wasting money
• Both yours and the patients
• May have insurance reimbursement problems
• Possible legal problems

29
Test Reliability

• The degree of consistency of a test when it is
  repeated
• Consistency is a function of random error
• All tests have some random error (variability)
• Reliable tests have less

30
Test-retest Reliability

• Measures stability over time in repeated
  applications of a test
• Test-retest reliability checks how similar
  results are when the research is repeated under
  similar circumstances
• In studies looking at test-retest reliability
• All other variables that might affect the test
  results must be controlled
• The condition being studied must remain constant
  from test to test

31
Internal Consistency

• Applicable to clinical questionnaires
• e.g., Pain, disability, and general health
• Tests the degree of correlation between
• Individual questions and the total score
• Individual questions and other questions on the
  instrument
• Determines the degree that questions
• Measure the same construct
• And are consistent with each other

32
Test Reliability

• The consistency of a test
• Both between repeated tests carried out by the
  same evaluator
• Intraexaminer reliability
• And between different evaluators
• Interexaminer reliability
• Determined by having two or more observers test
  the same patients and then correlate their
  observations

33
A Reliable Test

• Is reproducible when performed by different
  examiners
• (Interexaminer Reliability)
• And when repeatedly performed by the same
  examiner
• (Intraexaminer Reliability)

34
Test Validity

• Refers to the degree which a test or other
  measuring device is truly measuring what it is
  intended to measure
• For a test to be considered valid, it must be
  able identify the condition it is being used to
  test for
• Some tests are more valid than others
• e.g, Immune response markers for infection

35
Face Validity

• When evaluating the validity of an outcome
  measure -
• Does it seem to have merit as a useful test?
• Does it have reasonable face value?
• Examples
• Does it make sense to use measurement of cranial
  girth as a predictor for LB injury?
• Initial ROM and pain level as a predictor of
  recovery from LB injury?

36
Content Validity

• A tests ability to include or represent all of
  the content of a particular construct
• As an example
• Can low back pain be adequately assessed by
  asking a question about neck pain?
• No, because the question lacks content validity
• How about by a question about smoking in a health
  survey?

37
Content Validity cont.

• Another example
• A test designed to detect radicular neurological
  involvement that only deals with muscle strength
  would have poor content validity
• Deep tendon reflexes and superficial sensations
  are also major constructs that should be included

38
Concurrent Validity

• A measurements ability to vary directly with a
  measure of the same construct
• Test validity is determined by comparison with an
  already valid test
• An exam test to detect meniscus tears is compared
  with the findings of arthroscopic surgery (Gold
  standard)
• Concurrent validity is established if tests yield
  similar results

39
Concurrent Validity Cont.

• Gold Standard
• The most specific and sensitive test to diagnose
  a disease
• The term used to describe a method or procedure
  that is widely recognized as the best available
• Can be used to compare with other tests

40
Convergent Validity

• The ability of a measurement to correlate well
  with another measure that is related, but
  different
• Range of motion measured with an inclinometer
  compared with a goniometer
• Comparison of one version of a health survey
  instrument with another
• Measures of constructs that theoretically should
  be related to each other are, in fact, observed
  to be related to each other

41
Convergent Validity Cont.

• To establish convergent validity, one must show
  that measures that should be related are in
  reality related
• Example survey questions attempting to measure
  the construct of disability
• I am unable to lift more than X pounds
• I am unable to walk more than X blocks
• Are they related regarding the construct?
• Do they both depict disability?

42
Discriminant Validity

• The ability of a test to discriminate findings
  into categories like normal vs. abnormal or
  positive vs. negative
• Discriminant validity is important for screening
  tests
• Measures of constructs that theoretically should
  not be related to each other are, in fact,
  observed to not be related to each other

43
Discriminant Validity Cont.

• To establish discriminant validity, one must show
  that measures that should not be related are in
  fact not related
• The relationship between measures from different
  constructs should be very low
• For example
• A measure of visual acuity and a measure of knee
  function should be independent of one another

44
Predictive Validity

• A tests capacity to be a valid screening tool
  for some future health problem
• Are positive findings detected on a postural exam
  valid predictors for future back pain
• Is elevated cholesterol a valid predictor for
  developing diabetes

45
Predictive Validity cont.

• Predictive validity is established by measuring
  the trait now and waiting to see if the event
  occurs as predicted

46
Responsiveness

• For a test to be responsive
• It must be able to detect clinically important
  changes
• It must be sensitive to real change
• A responsive test will accurately measure the
  amount of change in a condition when change
  actually occurs

47
Percent Change

• A way of evaluating responsiveness
• Typically expressed as a percentage
• Initial (baseline) score is compared to the
  post-treatment score
• Example ROM 30 degrees at baseline and 60
  degrees post-treatment
• Change score 30/60 .5 50

48
t-test

• Studies involving groups may be evaluated for
  change using a t-test (the equivalent groups
  variety)
• Initial mean scores are compared with post
• ANOVA when more than two evaluations are involved