Assessment in Psychiatric Rehabilitation for Planning Interventions and Tracking Outcomes - PowerPoint PPT Presentation

Loading...

PPT – Assessment in Psychiatric Rehabilitation for Planning Interventions and Tracking Outcomes PowerPoint presentation | free to view - id: 1693fc-YjQ5Y



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Assessment in Psychiatric Rehabilitation for Planning Interventions and Tracking Outcomes

Description:

Assessment in Psychiatric Rehabilitation for Planning Interventions and Tracking ... Developed by Wallace, Lecomte, Wilde, & Liberman (2001) at UCLA ... – PowerPoint PPT presentation

Number of Views:1124
Avg rating:3.0/5.0
Slides: 67
Provided by: kimmacdon
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Assessment in Psychiatric Rehabilitation for Planning Interventions and Tracking Outcomes


1
Assessment in Psychiatric Rehabilitation for
Planning Interventions and Tracking Outcomes
  • Kim MacDonald-Wilson, Sc.D.
  • Alexis Henry, Sc.D.
  • Sandra Wilkniss, Ph.D.
  • Joanne Nicholson, Ph.D.

2
Agenda
  • Introduction to Assessment
  • Brief Primer on Issues in Measurement
  • Person Level Assessment for Intervention Planning
  • Program Level Assessment for Outcome Tracking
  • Integrating Assessments in PR Programs

3
Introduction to Assessment
  • Kim MacDonald-Wilson

4
Review of Assessment Experiences
  • Questions
  • How do you use assessments?
  • What assessments do you use?
  • What questions do you have about assessment
    (problems, issues)?

5
What is it?
  • Assessment is appraising or estimating the
    attributes or characteristics of a person, a
    group, or programs. The tools of assessment can
    include
  • Checklists
  • Inventories
  • Observational Schedules
  • Needs Assessments
  • Rating Scales
  • Tests
  • In PR, we focus most often on
  • Assessment of skills, resources, supports
  • Measurement of outcomes of our interventions

6
Why Do It?
  • Intervention planning
  • Understanding clients strengths, resources,
    challenges, and needs
  • Understanding the clients values and goals
  • Setting intervention goals
  • Outcome tracking
  • Recording change in key areas
  • Demonstrating impact of the intervention

7
Principles of Assessment in PR
  • Focus on functioning
  • Centered on and driven by person in recovery
  • Environment is important
  • Useful for practitioners and managers
  • Easily administered

8
Issues in Measurement
  • Alexis Henry

9
Assessment Methods
  • Standardized or informal
  • May assess skills, knowledge, attitudes
  • Methods
  • Observational simulated or situational
  • Generally used to assess skills
  • Paper-and-pencil self-ratings or checklists
  • Generally used to assess knowledge and attitudes
  • Interviews
  • Can be used to assess skills, knowledge and
    attitudes
  • Best for developing rapport and setting
    intervention goals

10
General Considerations in Conducting Assessment
  • When
  • Intervention planning
  • At intake (of course), but also periodically
  • Outcome tracking
  • When it occurs (of course)
  • Short, medium or longer term outcomes
  • Where
  • Program/clinic in real-world context
  • Relevance
  • Are assessments relevant to interventions/services
  • Important for both intervention planning and
    outcome tracking
  • Reliability, validity, cultural sensitivity

11
Reliability of Assessment
  • Stability, consistency, dependability
  • Internal consistency
  • how well do items on an instrument measure the
    same concept/construct
  • test-retest reliability
  • stability over time
  • interrater reliability
  • stability/agreement across raters

12
Validity of Assessment
  • Does assessment measure what it purports to
    measure?
  • meaningfulness, usefulness
  • Face validity
  • Does the assessment look valid to the person
    being assessed
  • Content validity
  • does the assessment adequately cover the domain
    being assessed
  • Construct validity
  • does the assessment correspond to the theoretical
    concepts (constructs) underlying the measure
  • e.g. self-esteem scale

13
Validity
  • Construct Validity
  • Convergent and divergent (discriminant) validity
  • Criterion-related validity
  • concurrent - uses an existing criterion
  • predictive - uses a future criterion
  • Responsiveness
  • Is the assessment sensitive enough to detect
    small changes

14
Cultural Sensitivity
  • Considerations
  • Gender
  • Age
  • Race
  • Ethnicity and language
  • Cultural norms
  • Socioeconomic status
  • Context/environment

15
Person Level Assessment for Intervention Planning
  • Kim MacDonald-Wilson

16
Why conduct assessments for intervention planning?
  • Involve the person in rehabilitation and recovery
  • Provide direction
  • Efficiency
  • Mobilize needed resources
  • Organize the interventions

17
Domains for Person Level Assessment
  • Work
  • School
  • Housing/Community Living
  • Social/Recreational/Community Participation
  • Parenting and other Social Roles
  • Quality of Life
  • Empowerment and Recovery

18
What to Measure in Person Level Assessment
  • What is the goal environment or domain?
  • Skills strengths and deficits
  • What skills are required to be successful and
    satisfied?
  • What can the person do or not do?
  • Environmental Resources and Supports
  • What supports are required to be successful and
    satisfied?
  • What does the person have or not have?

19
Assessment of Functioning
  • Assessment of Functioning does not Skills
    assessment
  • Global Functioning Focus on Outcome of
    Functioning
  • Performance in Roles (spouse, worker)
  • Performance in Domains (social, psychological)
  • Status (employment, educational, residential)
  • Tools GAF, BASIS-32, Multnomah Community
    Ability Scale, Life Skills Profile

20
Skill Assessment Examples
  • Skill Functioning Focus on Behaviors
  • Living
  • Test of Grocery Shopping
  • Independent Living Skills Survey ILSS
  • Client Assessment of Skills, Interests, and Goals
    CASIG
  • Work
  • Work Behavior Inventory WBI
  • Vocational Cognitive Rating Scale - VCRS
  • Parenting
  • Parenting Well Strengths and Goals

21
Resource Assessment Examples
  • Social Support
  • MOS Social Support Scale
  • Parenting
  • Family Resources Scale

22
Examples of Assessments for Intervention Planning
  • Observational assessments
  • Test of Grocery Shopping Skills
  • Vocational Cognitive Rating Scale
  • Work Behavior Inventory
  • Self-report measures
  • ParentingWell Strengths and Goals
  • Family Resource Scale
  • MOS Social Support Scale
  • Interviews
  • Independent Living Skills Survey
  • Client Assessment of Strengths, Interests and
    Goals (CASIG)

23
Test of Grocery Shopping Skills
  • Developed to be used in conjunction with a
    grocery shopping skills training curriculum for
    people with serious mental illness
  • Assesses persons ability to locate 10 specific
    common food items in a regular grocery store
  • Can be used as a baseline assessment and a
    follow-up assessment after training 2 versions
  • Developed by Hamera and Brown at University of
    Kansas Medical Center
  • http//www2.kumc.edu/pdrp/interventionandoutcome.h
    tm

24
Test of Grocery Shopping Skills
  • Examines the persons ability to find
  • Correct item
  • Correct size
  • Lowest price
  • Takes speed into account
  • Sample items
  • 15 oz. can red kidney beans
  • 1 lb. bag salad mix
  • 16 oz. frozen corn
  • 8 oz. low fat/nonfat strawberry yogurt
  • 16 oz. spaghetti

25
Vocational Cognitive Rating Scale
  • A 16-item instrument designed to measure
    cognitive skills in the workplace
  • Ratings are accomplished by a clinician who has
    observed the individual in the work setting and
    has interviewed the onsite supervisor
  • Behavioral anchors are provided for each rating
    on each item
  • Developed by Greig, Nicholls, Bryson, Bell
    (2004) at Yale University School of Medicine

26
VCRS Sample Item
  • Stays focused when performing a simple task
  • Consistently Inferior performance. Is
    consistently off task. Requires nearly continuous
    help from job coach/supervisor to stay focused.
  • Occasionally inferior performance. Can remain on
    task for brief intervals, but requires frequent
    monitoring by job coach to stay focused.
  • Adequate performance. Generally is able to stay
    on task and keep focused when doing simple tasks
    with minimal or infrequent assistance.
  • Occasionally superior performance. Stays on task,
    and only rarely requires assistance with
    focusing.
  • Consistently superior performance. Consistently
    completes simple tasks in a focused, organized
    manner.

27
Work Behavior Inventory
  • A 36-item work performance assessment
  • Social Skills
  • Cooperativeness
  • Work Habits
  • Work Quality
  • Personal Presentation
  • Ratings on a 5-point scale are accomplished by a
    clinician who has observed the individual in the
    work setting and has interviewed the onsite
    supervisor
  • Indicates both strengths and deficits, Behavioral
    anchors are provided for each rating on each item

28
WBI Sample Items
  • Each item rated on a 5-point scale from
    1-Consistently inferior performance to
    5-Consistently superior performance
  • Expresses positive feelings appropriately
  • Accepts constructive criticism without becoming
    upset
  • Begins work tasks promptly
  • Corrects own mistakes
  • Personal hygiene is satisfactory
  • General Rating of Work Behavior

29
ParentingWell Strengths and Goals
  • A parenting self-assessment and collaborative
    goal setting tool
  • Developed for use with parents with mental
    illness and their families
  • 26 items covering everyday demands of parenting
  • Developed by Nicholson Henry
  • Available at www.parentingwell.org

30
ParentingWell Strengths and Goals Sample Items
31
Family Resource Scale
  • Assesses family self-sufficiency by determining
    the adequacy of different types of resources in
    households of young children.
  • 30 item self-report tool, rated using a 5-point
    Likert-type scale anchored by "not at all
    adequate" to "almost always adequate.
  • Covers access to food, shelter, financial
    resources, transportation, health care, time to
    be with family, child care, and time for self.
  • Developed by Leet Dunst
  • Published in Dunst, C. J., Trivette, C. M.,
    Deal, A. G. (Eds.) (1994). Supporting and
    strengthening families. Methods, strategies and
    practices. Cambridge, MA Brookline Books.
    http//brooklinebooks.com/

32
Family Resource Scale Sample items
33
MOS Social Support Survey
  • 19 item self-report tool measuring multiple
    dimensions of social support
  • Emotional/informational
  • Tangible
  • Affectionate
  • Positive social interaction
  • Designed for use with individuals with long-term
    conditions, including serious mental illness
  • Developed by Sherbourne Stewart
  • Available from http//www.rand.org/health/surveys
    _tools/mos/mos_socialsupport.html

34
MOS Social Support Survey Sample items
35
Independent Living Skills Survey
  • Designed to assess ADLs and IADLs among people
    with schizophrenia and other serious mental
    illnesses
  • Two versions
  • Self-Report Interview 76 items
  • Personal appearance care of possessions food
    prep and storage health maintenance money
    management transportation leisure and community
    participation job seeking and job maintenance
  • Informant Interview 104 items
  • Same as above also includes eating behaviors
    social relationships
  • Both instruments are available in appendix of
    published article Wallace, C. J., Liberman, R.
    P., Tauber, R., Wallace, J. (2000). The
    Independent Living Skills Survey A comprehensive
    measure of the community functioning of severely
    and persistently mentally ill individuals.
    Schizophrenia Bulletin, 26(3), 631-658.

36
Independent Living Skills Survey-I Sample items
37
Clients Assessment of Strengths, Interests and
Goals (CASIG)
  • A lengthy, structured interview designed for use
    with people with serious mental illness, covering
    multiple domains of functioning
  • Assesses clients goals related to
  • Housing/living financial/vocational
    relationships spiritual/religious health
  • Assesses clients skills/strengths/needs related
    to
  • Lifestyle supports money management health
    management nutrition vocational
    transportation friends leisure personal
    hygiene care of possession medications and side
    effects personal rights cognition symptoms
    risk behaviors
  • Assesses satisfaction with
  • Quality of life quality of treatment
  • Developed by Wallace, Lecomte, Wilde, Liberman
    (2001) at UCLA
  • Available at http//www.ct.gov/dmhas/LIB/dmhas/MR
    O/CASIG.pdf

38
CASIG Sample items
  • Goals
  • Housing/Living Goals
  • One year from now, what would you like your
    living arrangements to be?
  • Financial/Vocational Goals
  • Would you like to improve your money/finances in
    the next year? How might you be able to improve
    them? Do you want to get a job or go to school
    in the coming year?

39
CASIG Sample items
  • Skills/Strengths
  • Health management
  • …at any time in the last 3 months did you
  • Make most of your own appointments with your
    doctor, case manager, etc?
  • Administer your own medication?
  • Food preparation
  • …at any time in the last 3 months did you
  • Plan your meals with a health balance of foods?
  • Use a stove to prepare meals?

40
Individualized assessment and goal setting
  • Goal Attainment Scaling
  • Alexis Henry

41
Goal Attainment Scaling
  • Framework for developing service goals (outcomes)
    that are measurable, relevant, attainable
  • Individualized goals
  • Quantitative index of clients progress
  • Can compare client to self over time
  • Can compare attainment across clients

42
Advantages of GAS
  • Not bound by any theoretical orientation,
    treatment or outcome measure
  • Allows for individualization of goals
  • Allows for prioritizing goals
  • Numeric index of performance over time
  • Easily computed
  • Useful in informal evaluations or in formal
    experimental designs

43
Disadvantages of GAS
  • Reliability issue needs to be considered
  • Outcomes best assessed by blind judge
  • Possibility of floor effect
  • Possibility of misuse if goals are too easy

44
Steps in GAS
  • Identify overall objective or domain
  • Identify specific problem area
  • Identify behavior that indicates improvement
  • Determine data collection method
  • Specify expected outcome
  • Specify most and least favorable outcome
  • Determine current status and timeframe for
    evaluation

45
Goal Attainment Scale
  • -2 most unfavorable outcome
  • -1 less than expected outcome
  • 0 expected outcome
  • 1 greater than expected outcome
  • 2 most favorable outcome
  • Goals can be weighted to reflect priorities

46
GAS Example 25 y.o. male wants to increase work
hours and earnings
  • 2 works 20 hrs/week at 10/hour
  • 1 works 10 hrs/week at 10/hour
  • 0 works 10 hrs/week at minimum wage
  • -1 works less than 10 hours/week
  • -2 no longer working

47
GAS Example 35 y.o. mother wants to have more
fun with kids age 8 and 10
  • 2 family participates in mutually enjoyable
    leisure/recreational activity 1x per week
  • 1 family identifies mutually enjoyable
    leisure/recreational activity participate 2x per
    month
  • 0 family engages in leisure together about
    once per month
  • -1 family leisure/recreational activities
    happen less than once per month
  • -2 no regular family leisure/recreational
    activities

48
Program Level Assessment for Outcome Tracking
  • Sandra Wilkniss, PhD

49
Program Level Assessment for Outcome Tracking
  • Why track outcomes on a program level?
  • Domains of outcome tracking
  • Clinical Status
  • Recovery Status
  • Quality of Life
  • Functional Status
  • Safety and Health
  • Context and Methods for collecting and
    interpreting outcomes
  • Types of Services and related outcome tracking
    issues
  • Sample outcome tracking tools
  • Exercise Selecting assessment tools for outcome
    tracking

50
Why track outcomes on a program level?
  • Access to services
  • Quality of services
  • Cost/benefit

51
Domains of outcome tracking
  • - Demographics Clinical Status
  • Recovery Status/Quality of Life
  • Functional Status
  • Roles/Community Integration
  • Skills
  • Resources
  • Safety and Health

52
Demographics Clinical Status
  • Demographics
  • Diagnostic and Historical information
  • Service use (baseline and change)
  • Physical health related outcomes
  • Cognition
  • Hospitalization
  • Jail and prison contact

53
Recovery/Quality of Life Status
  • Domains
  • Recovery
  • Empowerment
  • Self Efficacy
  • Hope
  • Satisfaction
  • Quality of Life
  • Methods
  • Consumer Self-report tools
  • Provider Evaluation tools
  • Family/significant other input
  • Interviews
  • Quantitative
  • qualitative
  • Proximal markers

54
Functional Status
  • Living situation
  • Housed or homeless
  • Level of independence
  • In community of choice
  • Education
  • Type of institution
  • Number of credit hours attempted/completed
  • Degree or voc/tech certificate
  • Work
  • In a job of consumers preference
  • Competitive
  • Salary and/or benefits
  • How much
  • For how long
  • Advancement
  • Follow along support variables

55
Functional Status (continued)
  • Community Integration
  • Instrumental Roles
  • Skills and Personal Resources
  • Social
  • Vocational
  • Financial
  • Recreational
  • Cognitive
  • Others
  • Environmental resources and supports available
  • Community
  • Family
  • Social capital
  • Others

56
Safety and Health
  • Personal and other safety
  • Physical Health and Wellness

57
Context and Methods for Collecting and
Interpreting Outcomes
  • Context
  • Cultural Sensitivity
  • Philosophy and interpretation
  • Specific measures
  • Cognition
  • Methods
  • Baseline tracking
  • Interval tracking
  • sources

58
Services and Interventions Offered Type,
quantity, and quality
  • Outreach
  • Linkage
  • PSR
  • Individual and group
  • Peer Support
  • Integrated Health care (primary care mental
    health)
  • Pharmacotherapy
  • Adjunct Interventions (e.g., cognitive support)
  • Evidence-Based or emerging best practices
  • GOI/penetration
  • Fidelity

59
Sample Tracking Tools
  • Comprehensive outcomes packages
  • Specific Measures
  • Cultural diversity measures
  • Recovery/Satisfaction Measures
  • Empowerment
  • Cognitive Functioning
  • Skills
  • Environment

60
Exercise Selecting Tools for Outcome Tracking
  • Selection Strategies
  • Determine program emphasis/focus
  • Evaluate needs and preferences of individual
    consumers
  • Evaluate resources and means for data collection,
    description/analysis, interpretation and
    dissemination
  • Review sample tools in each domain of interest
  • evaluate pros and cons
  • Psychometric properties vs. uniqueness
  • Thorough vs. brief and more accessible
  • Language geared toward consumers
  • Issues of cultural diversity
  • Etc.

61
Integrating Assessments in Psychiatric
Rehabilitation Programs Where the rubber meets
the road…
  • Joanne Nicholson
  • So far weve covered…
  • Principles for assessment
  • Issues in measurement
  • Person-level assessment
  • Program-level assessment

62
Integrating assessments in programs requires
consideration of
  • Relevant domains, e.g., work, education, etc.
  • Skills
  • Resources and supports
  • Individuals goals
  • Programs goals
  • Additional outcome tracking needs, e.g., data
    required for program funders?
  • Organizational resources and supports

63
Providers Motivations for Using Evidence-Based
Practices
  • Enhanced professional competence
  • Perceived utility
  • Fit with usual tasks and duties
  • Fit with ones own experience
  • Fit with mission of organization, structure of
    the intervention, organizing conceptual model

(Aarons Palinkas, 2007)
64
Potential Barriers to Use
  • Lack of training, motivation, connection
  • Measures, methods viewed as irrelevant
  • Measures, methods viewed as burdensome
  • Measures, methods not viewed as useful
  • Lack of organizational support?resources
  • Program measures and research measures dont
    mix well

65
Strategies for Overcoming Barriers to Assessment
  • Educate re rationale
  • Engage program staff participants in process
  • Anticipate address logistical challenges
  • Create opportunities for feedback refinement
  • Use data in meaningful, productive ways

66
Learning Activity
  • Break into groups ? Select a domain
  • Discuss assessment issues
  • Intervention planning? Outcomes?
  • Existing measures? Adaptation? Refinement?
  • Logistics Why? Who? How?
  • Mechanisms for feedback to staff, participants?
  • Identify implementation challenges suggest
    strategies for addressing
  • Report back
About PowerShow.com