Everything you Wanted to Know about the CPI: Now you can ask - PowerPoint PPT Presentation

Loading...

PPT – Everything you Wanted to Know about the CPI: Now you can ask PowerPoint presentation | free to view - id: 73df8-OTFjM



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Everything you Wanted to Know about the CPI: Now you can ask

Description:

– PowerPoint PPT presentation

Number of Views:107
Avg rating:3.0/5.0
Slides: 78
Provided by: aptaedu
Category:
Tags: cpi | ask | everything | know | now | wanted

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Everything you Wanted to Know about the CPI: Now you can ask


1
Everything you Wanted to Know about the CPI Now
you can ask
2
History of the Development of the CPI
  • Prior to the late 70s and early 80s each
    Physical Therapist and Physical Therapist
    Assistant Program had its own Clinical
    Performance/Evaluation Form.
  • Some Clinical Education Sites developed their OWN
    evaluation instrument.
  • 80s 90s Clinical Education Consortiums began
    to develop and some regional forms were
    developed.

3
  • November, 1993, APTA appointed a 10-member Task
    Force to develop consistent clinical education
    evaluation instruments to measure student
    performance outcomes in PT and PTA clinical
    education.

4
Task Force Members
  • Physical Therapy members Sherry Clark,MS, Susie
    Duesinger, PhD, PT, Barbara Gresham, MS, PT,
    Pamela Gramet, PhD, PT, Bella J. May EdD, PT,
    FAPTA, Kathryn Roach, PhD, PT, Babette Sanders,
    MS, PT
  • Non PT Members Rebecca Lewthwaite, PhD, Paul
    Hagler, PhD
  • Michael J. Strube, PhD (Consultant)

5
Design Characteristics
  • Must measure a performance of multiple skills and
    behaviors of the PT and PTA on multiple levels
    (Novice to Entry-Level)
  • Content and protocol of the instrument must be
    responsive to the clinical AND academic
    communities.
  • Psychometrically sound (valid and reliable)

6
Process
  • Task Force met in 1994.
  • Review of numerous instruments both within and
    outside the profession (nursing, OT, Speech,
    medicine) and international instruments (Canadian
    and Australian).
  • 50 Person Sounding Board was formed to respond to
    the initial draft of the Task Force.

7
  • The first draft was written based on A
    Normative Model for Physical Therapist Education,
    the Guide to PT Practice, Part One, the
    Evaluative Criteria for PT and PTA Education
    Programs and reference to the commonly used
    evaluation tools for clinical performance at that
    time.

8
Format
  • Designed to use Professional Judgment in a
    complex clinical environment
  • Performance Criteria were broad with specific
    Sample Behaviors given to guide the evaluation of
    the criteria. The Sample Behaviors were just
    that, a sample, and not all inclusive.

9
Format
  • Performance Dimensions were to be used to
    describe performance of behaviors of each
    criteria.
  • Quality
  • Supervision/Guidance
  • Consistency of Performance
  • Complexity of tasks/environment
  • Efficiency of Performance

10
VAS
  • The VAS was chosen as the means of recording the
    level of observed behavior for each item in the
    CPI.
  • Horizontal line of 100 mm represents a continuum
    of performance from Novice to Entry-Level (Draft
    1 had Expert as end point).

11
VAS
  • Continuous scales can recognize degree of change
    better than categorical scales. (Clinicians
    often used the /- in other scales)
  • No numbers on the scale made it more likely that
    meaning would not be attached to the numbers.
    Use of the VAS scale encourages rating based on
    the end anchors and not numbers.

12
VAS
  • Entry-level is defined in each clinical setting
    and the rater uses professional judgment to
    determine the degree to which a student meets
    that criteria.

13
Instructions for use
  • Specific instructions were written for the use of
    the form and that was included in the first
    draft.
  • Task Force believed that a therapist should be
    able to READ the instructions and be able to use
    the form.

14
Sounding Board
  • The sounding board of 50 people were sent the
    first draft including instructions for use and
    changes were made in the instrument based on this
    feedback for the development of Draft 2.

15
Draft Two
  • Pilot study of Draft 2 between October 1995 and
    June of 1996.
  • Concurrently, regional forums were being
    conducted by the task force and structured
    feedback was solicited.
  • Concurrently, copies of Draft 2 were sent out to
    academic programs and clinical educators with a
    feedback form included.

16
Draft 3
  • Data from Pilot Study
  • Feedback from forums
  • Feedback from multiple communities of interest
  • Consultation of Psychometrician
  • Development of Draft 3, the Field Study version.

17
Draft 4
  • Current version of the CPI
  • Based on results of Field Study of Draft 3
  • Feedback from communities of interest
  • Consultation of Psychometrician
  • Changes in terminology of current APTA documents

18
Dissemination
  • The Physical Therapist Student version of the CPI
    was approved by the APTA BOD in November 1997.
  • The Physical Therapist Assistant Student version
    of the CPI was approved by the APTA BOD in March
    1998.
  • Currently 95 of PT and PTA education programs
    have purchased the right to use the CPI

19
Thanks for the Memory Reference
  • The Development and Testing of APTA Clinical
    Performance Instruments. Anticipated Publication
    in April 2002, Physical Therapy.
  • Roach K, Gandy J, Deusinger SS, Clark S, Gramet
    P, Gresham B, Hagler P, Lewthwaite R, May BJ,
    Rainey Y, Sanders B, Strube MJ.

20
CLINICAL PERFORMANCE INSTRUMENT Training and
Educational Needs An ACCE Perspective
21
OVERVIEW
  • Entry-Level Construct
  • Visual Analog Scale
  • Questions About Training

22
ENTRY-LEVEL CONSTRUCT
All 24 items on CPI are components of entry-level
physical therapist practice.
23
ENTRY-LEVEL CONSTRUCT
Example 17. Provides consultation to
individuals, businesses, schools, government
agencies or other organizations
With Distinction
Not Observed
Entry Level
Novice
SAMPLE BEHAVIORS c) Uses knowledge and expertise
to help others solve PT-related problems d)
Provides consultation such as ergonomic
evaluations, school system assessments
Comments N/A Only experienced
therapists do consultation in this clinic --not
part of student experience.
24
VISUAL ANALOG SCALE (VAS)
  • Measures degree to which a phenomenon is present
    by defining two ends of a continuum and asking
    rater to make a judgment.
  • Clinically, used to measure a subjective
    experience
  • Important to operationally define both anchors so
    rater has sense of the whole
  • Criteria referenced vs. norm referenced
    evaluation of performance

25
VISUAL ANALOG SCALE (VAS)
  • Confusion about entry-level anchor what does it
    really look like in practice?
  • Keeping definition of entry-level current for
    practice setting
  • Recognition of the full continuum dont forget
    the left anchor
  • Variation in right anchor based on level of
    student experience

26
STRATEGIES FOR TRAINING
Refer to definitions of anchors in instructions
and encourage a reflective process
  • Entry-level performance
  • consistently and efficiently provides quality
    care
  • with simple or complex patients
  • in a variety of clinical environments.
  • Usually needs no guidance or supervision except
    when addressing new or complex situations.

27
STRATEGIES FOR TRAINING
Facilitate collaborative process in which
clinical faculty define items for their practice
setting especially the frequently not observed
items
  • Sample Behaviors - may need to customize
  • Performance dimensions

Manage CI and student expectations related to
scores progression across full continuum
28
Impact of Training
What impact is training having on reliability and
validity of CPI? Initial field testing performed
with CIs reading the written instructions
Are CIs getting mixed messages? CCCE asked how
my program wants them to use the CPI
Future research?
29
CLINICAL PERFORMANCE INSTRUMENT Training and
Educational Needs A CCCE Perspective
30
OVERVIEW
  • Entry-level anchor on VAS and describing
    entry-level practice
  • Skill acquisition and implications for
    establishing expectations
  • Clinical Instructor training needs

31
Entry-level Anchor
  • Dynamic Quality of Entry Level Anchor
  • Evolving nature of clinical practice
  • Cultural competence
  • Advocacy
  • Evidence based decision making
  • Changes in entry level curriculum and varying
    academic expectations for outcomes of clinical
    education

32
Entry-level Anchor
Question What is a realistic expectation for
entry level practice??
Entry level practice is what constitutes safe and
effective practice for your setting
In a given setting, how can we increase the
consistency with which CIs assess clinical
situations and draw conclusions?
33
Entry-level Practice
Describing Entry-level in Behavioral Terms Engage
leadership and clinicians who will be working
with students and are familiar with recent
curricular changes in process
Do sample behaviors describe practice in your
setting accurately and completely?
Need to identify examples of how the item might
be seen in clinical practice
34
EXAMPLE
10. Screens patients using procedures to
determine effectiveness of and need for PT
services
Entry Level
Novice
SAMPLE BEHAVIORS b) Selects appropriate
screening procedures
Inpatient medical record, consultation with
other professionals Outpatient self administered
patient questionnaire, patient interview
35
Skill Acquisition
  • Skill Acquisition
  • Clinical Knowledge integrate theory / experience
  • Developmental process
  • Relies on rules
  • Relies on experience
  • Focuses on tasks
  • Focuses on whole person
  • Situations made up of equally relevant parts
  • Prioritize among parts of a given situations

36
CI Rating Issues
Consistent use of entry-level as the measurement
standard
Weighting of sample behaviors
Concerns about the point on the VAS that
constitutes passing for academic institution
37
CPI Training Issues
Clinical Instructor identified training needs
  • Instructions require time and discussion to fully
    digest
  • Need for clarity around behaviors that describe
    entry level practice in setting
  • Use of performance dimensions
  • Dynamic nature of entry level

38
CLINICAL PERFORMANCE INSTRUMENT Implementation by
Academic Programs An ACCE Perspective
39
  • Four major areas of focus
  • Clinical faculty training and use.
  • Interpretation and grading of the CPI.
  • CPI use in program and curriculum evaluation.
  • CPI as a tool to facilitate student development
    and self-evaluation.

40
CPI Interpretation and Grading
  • Many variations exist between academic programs.
  • Performance expectations for various clinical
    levels
  • CPI interpretation
  • Weighting of criteria
  • Grading processes.

41
CPI Interpretation and Grading
  • The Novice to Entry level VAS moves the
    evaluation of student performance beyond
    considering primarily the amount of supervision
    needed.
  • Requires a change in the academic mindset.
  • Performance standards must be reestablished.
  • Expected rate of progression to entry level must
    be considered.

42
CPI Interpretation and Grading
  • The specifics of performance issues are not
    always immediately obvious.
  • Performance Criteria such as Treatment/Interventi
    on (PT version) or Patient Interventions (PTA
    version), cover many component skills and
    behaviors.
  • Sample behaviors not all-inclusive
  • Performance dimensions not broken out

43
CPI Interpretation and Grading
  • CI written comments
  • should indicate the specific performance issues
    identified.
  • should be consistent with the VAS.
  • Wide variation seen in the quality and quantity
    of CI written comments.

44
CPI Interpretation and Grading
  • Tendency for CI inflation of student
    performance continues.
  • Interpreting the VAS line and comments may
    necessitate a call to the CI or CCCE.

45
CPI Interpretation and Grading
  • Should special significance be given to red
    flag items 1-5?
  • Should other performance criteria be weighted
    or considered as equal?
  • What to do with criterion marked as not
    observed?
  • How to structure grading matrices?
  • Much can be gained by academic programs comparing
    approaches.

46
Program and Curriculum Evaluation
  • Academic programs need data regarding student
    performance on clinicals to evaluate and modify
    curricula.

47
Program and Curriculum Evaluation
  • Data gathered from VAS scores may show linkage to
    the broader categories of the Performance
    Criteria.
  • However, the CPI does not provide specific
    information on the performance dimensions or the
    knowledge, skills and behaviors involved.

48
Program and Curriculum Evaluation
  • Academic faculty must look for other ways to
    capture the more specific information needed for
    thorough curriculum evaluation.
  • CI comments must also be recorded and evaluated.
  • Triangulation of data continues to be important.

49
Facilitating Student Development - The CPI as A
Roadmap
  • Students need a tool to assist them with
    self-assessment, goal setting, and self-directed
    learning.
  • Student memories of verbal feedback and
    discussions with CI/CCCE can be short and
    unreliable.

Novice
Entry level
50
Facilitating Student Development
  • Entry level performance is a multi-dimensional
    construct and may not be readily understood by
    students.
  • Students must be trained in the use of the CPI in
    order to make use of it as a tool for ongoing
    development and self-assessment.

51
CLINICAL PERFORMANCE INSTRUMENT A Student
Perspective
52
Students Questions about the CPI
Is it fair? Is it useful? Is the feedback
meaningful?
YES ! YES ! YES !
53
Other Student Perspectives
CPI comments and discussion with CI more useful
than marks on the VAS.
  • At times students identify need to discuss
    changes in practice with their CIs,
    without overstepping bounds --
  • CPI can facilitate this discussion

Students not focused on entry-level instead,
want to be best therapist they can be
54
Where do we go from here?
  • What are the questions we most need to answer?
  • What are some research ideas?

55
Studies
  • Study published in the Journal of PT Education
    using the CPI to compare the outcomes of student
    performance in multiple short-term vs. single
    long-term experience.

56
2002 CSM
  • Relationship between GPA clinical performance
    of PT students using the CPI
  • Hewson,Ryglewicz, Michaels, Neemuchwala, and
    Liebeck
  • Analysis of data from the CPI for students in
    beginning, middle, and final clinical experiences
  • Knab Portney

57
2002 CSM
  • An evaluation of the reliability validity of
    the CPI
  • Adams, Fish, Hughes, Roberts, Viricel Geher
  • Essential PT clinical experiences
  • Ingram, Montgomery, Reese Stone

58
Purpose
  • The primary purpose of this study was to identify
    the essential physical therapy clinical
    experiences, as defined by the Academic
    Coordinators of Clinical Education (ACCE), that
    all students must experience and/or attain entry
    level performance prior to graduation.

59
Methods
  • Subjects
  • 185 ACCEs of U.S. CAPTE accredited programs

60
Survey Instrument
  • Systems
  • Cardiopulmonary
  • Integumentary
  • Musculoskeletal
  • Neuromuscular
  • Life Span
  • Infants
  • Toddlers
  • Children
  • Adolescents
  • Adults
  • Older Persons

61
Survey Instrument
  • Settings
  • Hospitals
  • Outpatient Clinics
  • Rehabilitation Facilities
  • Skilled Nursing Extended Care
  • Home Health
  • Schools Playgrounds
  • Hospices
  • Corporate Health Centers
  • Occupational Environments
  • Athletic Facilities
  • Fitness Centers

62
Survey Instrument
  • Roles
  • Primary Care
  • Secondary/Tertiary Care
  • Fitness Wellness
  • Consultation
  • Education
  • Critical Inquiry
  • Administration

63
Survey Instrument
  • Demographic Information
  • Type of program
  • Size of program
  • Funding of program
  • State of program
  • Direct access
  • Number of years of experience as an ACCE/DCE

64
Scale
  • 5 absolutely essential
  • 4 mostly essential
  • 3 essential
  • 2 mostly not essential
  • 1 absolutely not essential

65
Results
  • 107 ACCEs responded (57)
  • 63 had direct access
  • 43 did not have direct access
  • 53 had gt5 years experience
  • 54 had lt5 years experience

66
ACCE Opinions of Systems Students Should
Experience and Attain Entry-Level (median
responses)
67
Ranked Order of Mostly to Absolutely Essential
For Systems
  • Experience
  • Musculoskeletal
  • Neurological
  • Cardiopulmonary
  • Integumentary
  • Entry Level
  • Musculoskeletal
  • Neurological
  • Cardiopulmonary
  • Integumentary

68
ACCE Opinions of Settings Students Should
Experience (median responses)
69
ACCE Opinions of Settings Students Should Attain
Entry-Level (median responses)
70
Mostly Not to Absolutely Not Essential For
Settings
  • Experience
  • Hospice
  • Corporate Health
  • Entry Level
  • Hospice
  • Corporate Health

71
ACCE Opinions of Life Span Students Should
Experience and Attain Entry-Level (median
responses)
72
Life Span Categories
  • Having experience and gaining entry level with
    adults and older persons were rated more
    essential than with children

73
ACCE Opinions of Roles Students Should Experience
and Attain Entry-Level (median levels)
74
Mostly Essential to Absolutely Essential For Roles
  • Experience
  • Secondary
  • Critical Inquiry
  • Entry Level
  • Secondary
  • Critical Inquiry

75
Analysis
  • Mann Whitney U Utilized to determine difference
    among demographic groups
  • Years of experience as ACCE
  • The following significant differences were found
  • ACCE with 5 years or less experience rated the
    following items higher than the ratings of ACCEs
    with greater than 5 years experience
  • Entry-level in hospital setting (.04)
  • Experience in outpatient setting (.04)
  • Experience with older adults (.049)
  • Experience and entry-level in secondary/tertiary
    care (.03 and .01 respectively).
  • T-test comparing experience with entry-level
  • No significant differences found

76
Recommendations for Future Study
  • Survey ACCEs regarding actual required
    experiences
  • Identify a method for CIs to report to ACCEs the
    assessment of student performance in systems,
    settings, and life spans. Roles are addressed
    currently in the CPI.
  • Survey clinicians regarding their opinions

77
What are your recommendations?
  • What are the themes that have been expressed
    today?
  • What suggestions would you make?
About PowerShow.com