Assessing Competencies: Context, Challenges, and Tools

1
Assessing Competencies Context, Challenges,
and Tools

• Catherine L. Grus, Ph.D.,
• Associate Executive Director for
• Professional Education and Training
• Education Directorate
• American Psychological Association

2
THANK YOU

• Many thanks to
• Nadya Fouad, University of Wisconsin
• Nadine Kaslow, Emory University
• Benchmarks Workgroup
• Assessment Toolkit Workgroup

3
Overview

• Environmental Scan
• Competency Benchmarks
• Competency Assessment Toolkit

4
a culture of competenceRoberts, Borden,
Christiansen Lopez (2005)

• a shift within professional psychology toward an
  emphasis on the acquisition and maintenance of
  competence as a primary goal

5
(No Transcript)
6
Are all equal?
7
A Shift in Education and Training

• Competency models focus on learning outcomes
• Assessment can be used to guide performance
  improvement
• What is being measured (i.e., the competencies)
  relates to actual behaviors one will use in the
  course of ones professional activities
• As compared to models of education and training
  focused on learning objectives

8
Competence in Professional Psychology

• Competencies discrete knowledge, skills,
  attitudes that are elements of competence
• Competence presumes the integration of multiple
  competencies

9
(No Transcript)
10
Key Competency Initiatives in Professional
Psychology
ADPTC /CCTC Practicum Competencies
BEA Task Force on Assessment of Competence
CoA revises GP
APA Ethics code revised
APPIC Competencies Conf.
Competency Benchmarks Work Group
Competency Assessment Toolkit
1990s 2000 2002 2006 2008
11
School Psychology Competencies

• Historical focus of specialty area (1970s)
• Provider competencies and assessment of the
  outcome of interventions
• Guidelines for Performance Based Assessment and
  Program Accountability and Development (2001) and
  Standards for Training and Field Placement
  Program in School Psychology (2000) (NASP)
• APA Task Force on School Psychology Standards
  (2001)
• Broad and general competencies as defined in
  accreditation standards combined with specialty
  specific competencies in the application of
  psychological knowledge and methods within
  educational settings or with those involved in
  the learning process (Tharinger, Pryzwansky
  Miller, 2008)

12
Competence Initiatives in Health Care Professions

• ACGME Toolbox of Assessment Methods
• Competencies in dentistry, nursing, pharmacy
• Interprofessional Professionalism Work Group
• Cross-disciplinary competencies for health care
  disciplines serving older adults (American
  Geriatrics Society)

13
The Benchmarks A Developmental Model for
Defining Competencies
14
Assessment of Competency Benchmarks Workgroup
(2006)

• Chair Nadya Fouad, Ph.D.
• 32 member workgroup
• Charge to better inform understanding of entry
  level to practice in light of the APA policy on
  Education and Training Leading to Licensure
• Product A document that delineates competency
  benchmarks, or measurable standards of
  performance, that are developmental and
  integrated through the sequence of professional
  education and training

15
Caveats

• Applies most directly to those preparing for the
  practice of health service provision, i.e., those
  who will seek licensure
• Not meant to be prescriptive, a tool for programs
  to implement if they chose and in accordance with
  their model of education and training
• Assessment of competence must be balanced with
  the primary mission of the program education and
  training

16
Rodolfa, E. R., Bent, R. J., Eisman, E.,
Nelson, P. D., Rehm, L., Ritchie, P. (2005). A
cube model for competency development
Implications for psychology educators and
regulators. Professional Psychology Research and
Practice, 36, 347-354.
17
Foundational and Functional Competencies

• Foundational - the knowledge, skills, attitudes,
  and values that serve as the foundation for the
  functions a psychologist is expected to carry out
• Functional - encompass the major functions, or
  actions, that a psychologist is expected to carry
  out, each of which requires reflective
  integration of foundational competencies in
  problem identification and resolution

18
Core Competencies

• Foundational Competencies
• Professionalism
• Reflective practice
• Scientific knowledge and methods
• Relationships
• Individual and cultural diversity
• Ethical and legal standards and policy
• Interdisciplinary systems

• Functional Competencies
• Assessment
• Intervention
• Consultation
• Research and evaluation
• Supervision
• Teaching
• Administration
• Advocacy

19
How are the Benchmarks Organized?

• Core Foundational and Functional Competencies
• Essential Component what are the critical
  elements of/what knowledge/skills/attitudes that
  make up this competency?
• Behavioral Anchor what would it look like if you
  saw it (essential component)?

20
Essential Components
21
Behavioral Anchors
22
Competency Assessment for Toolkit for
Professional Psychology (2008)

• Chair Nadine Kaslow, Ph.D.
• Six members in workgroup
• Charge from APA Board of Educational Affairs
  Develop a Toolkit for professional psychology
• Purpose Promote broader implementation of
  competence assessment and provide information
  about application of assessment methods to the
  assessment of competence
• Coordinated with Benchmarks Work Group

23
Components of Toolkit

• Background and Introduction
• Assessment Method Fact Sheets
• Description
• Use specific to core competencies, formative vs.
  summative, developmental level
• Implementation
• Psychometrics
• Strengths/Challenges
• Future Directions
• Grid of Assessment Methods and Competencies Best
  Used for

24
Assessment Methods in Toolkit

• Annual Rotation Performance Review
• Case Presentation
• Client/Patient Process/Outcome Measure
• Competence Evaluation Rating Form
• Consumer Satisfaction Survey
• OSCE
• Portfolio

• Ratings of live or recorded performance
• Record Review
• Self-Assessment
• Simulation/Role Play
• Standardized Client/Patient
• Structured Oral Exam
• Written Examination
• 360 evaluation

25
Grid of Possible Assessment Methods
26
Portfolio Review Application
From Competency Assessment Toolkit
27
Putting it all Together
Benchmarks
Toolkit
28
(No Transcript)
29
(No Transcript)
30
Next Steps

• Manuscripts submitted to TEPP
• Gather empirical support
• Develop evaluation system(s) for Benchmarks
• Modules for education and training programs
• Resources related to students with competence
  problems
• Presentations to Training Councils

31

• For more information
• http//www.apa.org/ed/graduate/competency.html