An Introduction to Understanding the OT and OTA Fieldwork Performance Evaluations (FWPEs)

1
An Introduction to Understanding the OT and OTA
Fieldwork Performance Evaluations (FWPEs)

• Karen Atler, MS, OTR
• Colorado State University, Fort Collins, CO
• Roberta Wimmer, OTR/L
• Pacific University, Forest Grove, OR

? 2003 The American Occupational Therapy
Association, Inc. www.AOTA.org
2
Task Force Members

• Carole Dennis, PhD, OTR
• Ithaca College, New York
• Carole Hays, MA, OTR
• Springfield Hospital Center, Maryland
• Becky Robler, MEd, OTR
• Pueblo Community College, Colorado
• Karen Atler, MS, OTR, Co-Chairperson
• Colorado State University
• Roberta Wimmer, OTR, Co-Chairperson
• Pacific University, Oregon

3
Objectives

• Describe the
• Entry-level practice competencies for OT and OTA
  students
• Purpose, format, content, and scoring of the
  companion evaluation forms
• New concepts and terminology used in the FWPE
  from the OT Practice Framework
• Begin to score items on the FWPE

4
Task Forces Charge

• Revise/develop evaluation tools to measure
  assistant and professional Level II fieldwork
  student performance.
• Expectations
• Conduct review of literature across disciplines
• Synthesize feedback on current AOTA FWE/OT forms
• Incorporate 1997 NBCOT Practice Analysis results
• Address identified desired characteristics

5
Desired Characteristics

• Companion documents for assistant and
  professional level that
• Measure entry-level competence
• Focus on occupation-based practice
• Reflect current and future practice
• Can be used in a variety of settings
• Provide feedback to students
• Can be easily used in a timely manner

6
NBCOT Practice Analysis 1997

• What OTs OTAs Do
• Determining needs/priorities for interventions
• Identifying/designing interventions
• Implementing interventions
• Reporting/evaluating intervention effectiveness
• Providing OT services for populations
• Managing delivery of OT services
• Advancing effectiveness ofthe OT profession

• What OTs OTAs Need To Know
• Human development and performance
• Principles/strategies in the identification/evalua
  tion of strengths and needs
• Principles/strategies in intervention/treatment
  planning
• Principles/strategies inintervention
• Nature of occupation and occupational performance
  
• Service management
• Responsibilities as a professional

7
Standards of Practice for Occupational Therapy

• Identifies minimum standards
• Identifies key performance areas for the OT and
  OTA
• Professional standing and responsibility
• Referral
• Screening
• Evaluation
• Intervention plan
• Intervention
• Transition services
• Discontinuation

8
ACOTE Minimum Standards and Outcomes

• OT
• Be a generalist
• Achieve entry-level competence
• Articulate, apply, and justify occupation
  interventions
• Supervise and collaborate with the OTA
• Keep current with best practice
• Uphold the ethics, values, and attitudes of the
  profession
• Be an effective consumer of research and
  knowledge

• OTA
• Be a generalist
• Achieve entry-level competence
• Work under the supervision of and in cooperation
  with the OT
• Articulate, apply, and justify interventions
  related to occupation
• Keep current with best practice
• Uphold the ethics, values, and attitudes of the
  profession

9
Goal of Level II Fieldwork Education for the OT
and OTA Student

• Develop competent, entry-level generalists
• Include an in-depth experience in delivering
  occupational therapy services
• Be designed to promote reasoning, enable ethical
  practice, and develop professionalism

10
The Process

• Began with OTA evaluation
• Reviewed by experienced panel
• Submitted to COE
• Made revisions
• Completed pilot studies (2 OTA, 1 OT)

11
Design and Analysis of Pilot Studies The Rasch
Measurement Model
less able Student Ability more able
12
Results of Pilot Studies

• Good representation in pilot samples
• Students and educators preferred new form
• Good scale and response validity
• Inaccurate use of scale

13
The FWPEs for OT and OTA Students

• Companion documents
• Terminology
• Content layout
• Purpose
• Design
• Rating scale
• Scoring system

14
FWPEs OT and OTACompanion Documents

• Focus
• The occupational therapy process
• The clinical reasoning process
• Roles and responsibilities of the OT and OTA
• Structure
• Collaborative processstudent and FW educator
• Same layout
• Same rating/scoring system

15
Terminology of the FWPEs

• Reflects
• Standards of Practice for Occupational Therapy
  and ACOTE Education Standards
• Occupational Therapy Practice Framework
• The glossary

16
Content Layout of FWPEs

• Summary Sheet
• Overview/instructions
• Organization of items
• Space for commentsmidterm and final
• Performance Rating Summary Sheet

17
Content of OT and OTA Evaluations

• OTA
• Fundamentals of practice (3)
• Basic tenets (3)
• Evaluation/screening (5)
• Intervention (6)
• Communication (2)
• Professional behavior (6)

• OT
• Fundamentals of practice (3)
• Basic tenets (4)
• Evaluation/screening (10)
• Intervention (9)
• Management of OT services (5)
• Communication (4)
• Professional behavior (7)

18
Primary Purpose of the FWPEs

• Measures entry-level competence
• Designed to differentiate the competent student
  from the incompetent student
• Not designed to differentiate levels above
  entry-level competence

19
Purpose (continued)

• Provides student with accurate assessment of his
  or her competence for entry-level practice over
  time
• Growth occurs over time
• Midterm and final scores reflect this change
• Midterm scores Satisfactoryunsatisfactory
• Final scores Passno pass

20
Purpose (continued)

• Provides feedback to student
• Provides opportunity for student self-assessment

21
Design of the FWPEs

• The doing of the occupational therapy process
  is evaluated, not the individual tasks in
  isolation
• NOT all items are equal in level of difficulty
  (i.e., simple to complex)
• Evaluation is supplemented with development of
  site-specific objectives

22
RASCH Ordering of Items OT
23
Rasch Ordering of Items OTA
24
Rating Scale of FWPEs
25
Scoring System of FWPEs

• Each item must be scored
• Ethics and safety items must be passed
• Each item rating recorded on Performance Rating
  Summary Sheet
• All items summed up at midterm and final
• Score compared to scales provided

26
Midterm and Final Scoresof the FWPEs

• OTA
• Overall Midterm Score
• Satisfactory 54 above
• Unsatisfactory 53 below
• Overall Final Score
• Pass 70 above
• No Pass 69 below

• OT
• Overall Midterm Score
• Satisfactory 90 above
• Unsatisfactory 89 below
• Overall Final Score
• Pass 122 above
• No Pass 121 below

27
Rating Performance Using the FWPEs

• Case Scenarios
• JohnOT Evaluation
• MaryOT Intervention
• SandraOTA Evaluation/Screening
• DavidOTA Intervention

28
Individualizing the FWPEs

• Designed for additional objectives to be written
  to add clarification
• Site-specific objectives
• NOT supervisor-specific
• If an item is very clear and meets the RUMBA
  test, then there is no need to write another
  objective

29
Objectives An Example From FWPE for the OT
Student

• 16. Establishes accurate and appropriate plan
• SchoolProvides behavioral-based, measurable OT
  goals during IEP process
• Acute CareOverall intervention plan is
  achievable within clients length of stay

30
Summary

• Evaluations designed to measure entry-level
  competence, NOT level of performance above
  competency
• OT practice examined as a generalist
• Evaluations reflect the occupational therapy
  process
• Performance develops over time