Assessment in Psychiatric Rehabilitation for Planning Interventions and Tracking Outcomes - PowerPoint PPT Presentation


PPT – Assessment in Psychiatric Rehabilitation for Planning Interventions and Tracking Outcomes PowerPoint presentation | free to view - id: 1693fc-YjQ5Y


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation

Assessment in Psychiatric Rehabilitation for Planning Interventions and Tracking Outcomes


Assessment in Psychiatric Rehabilitation for Planning Interventions and Tracking ... Developed by Wallace, Lecomte, Wilde, & Liberman (2001) at UCLA ... – PowerPoint PPT presentation

Number of Views:1124
Avg rating:3.0/5.0
Slides: 67
Provided by: kimmacdon


Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Assessment in Psychiatric Rehabilitation for Planning Interventions and Tracking Outcomes

Assessment in Psychiatric Rehabilitation for
Planning Interventions and Tracking Outcomes
  • Kim MacDonald-Wilson, Sc.D.
  • Alexis Henry, Sc.D.
  • Sandra Wilkniss, Ph.D.
  • Joanne Nicholson, Ph.D.

  • Introduction to Assessment
  • Brief Primer on Issues in Measurement
  • Person Level Assessment for Intervention Planning
  • Program Level Assessment for Outcome Tracking
  • Integrating Assessments in PR Programs

Introduction to Assessment
  • Kim MacDonald-Wilson

Review of Assessment Experiences
  • Questions
  • How do you use assessments?
  • What assessments do you use?
  • What questions do you have about assessment
    (problems, issues)?

What is it?
  • Assessment is appraising or estimating the
    attributes or characteristics of a person, a
    group, or programs. The tools of assessment can
  • Checklists
  • Inventories
  • Observational Schedules
  • Needs Assessments
  • Rating Scales
  • Tests
  • In PR, we focus most often on
  • Assessment of skills, resources, supports
  • Measurement of outcomes of our interventions

Why Do It?
  • Intervention planning
  • Understanding clients strengths, resources,
    challenges, and needs
  • Understanding the clients values and goals
  • Setting intervention goals
  • Outcome tracking
  • Recording change in key areas
  • Demonstrating impact of the intervention

Principles of Assessment in PR
  • Focus on functioning
  • Centered on and driven by person in recovery
  • Environment is important
  • Useful for practitioners and managers
  • Easily administered

Issues in Measurement
  • Alexis Henry

Assessment Methods
  • Standardized or informal
  • May assess skills, knowledge, attitudes
  • Methods
  • Observational simulated or situational
  • Generally used to assess skills
  • Paper-and-pencil self-ratings or checklists
  • Generally used to assess knowledge and attitudes
  • Interviews
  • Can be used to assess skills, knowledge and
  • Best for developing rapport and setting
    intervention goals

General Considerations in Conducting Assessment
  • When
  • Intervention planning
  • At intake (of course), but also periodically
  • Outcome tracking
  • When it occurs (of course)
  • Short, medium or longer term outcomes
  • Where
  • Program/clinic in real-world context
  • Relevance
  • Are assessments relevant to interventions/services
  • Important for both intervention planning and
    outcome tracking
  • Reliability, validity, cultural sensitivity

Reliability of Assessment
  • Stability, consistency, dependability
  • Internal consistency
  • how well do items on an instrument measure the
    same concept/construct
  • test-retest reliability
  • stability over time
  • interrater reliability
  • stability/agreement across raters

Validity of Assessment
  • Does assessment measure what it purports to
  • meaningfulness, usefulness
  • Face validity
  • Does the assessment look valid to the person
    being assessed
  • Content validity
  • does the assessment adequately cover the domain
    being assessed
  • Construct validity
  • does the assessment correspond to the theoretical
    concepts (constructs) underlying the measure
  • e.g. self-esteem scale

  • Construct Validity
  • Convergent and divergent (discriminant) validity
  • Criterion-related validity
  • concurrent - uses an existing criterion
  • predictive - uses a future criterion
  • Responsiveness
  • Is the assessment sensitive enough to detect
    small changes

Cultural Sensitivity
  • Considerations
  • Gender
  • Age
  • Race
  • Ethnicity and language
  • Cultural norms
  • Socioeconomic status
  • Context/environment

Person Level Assessment for Intervention Planning
  • Kim MacDonald-Wilson

Why conduct assessments for intervention planning?
  • Involve the person in rehabilitation and recovery
  • Provide direction
  • Efficiency
  • Mobilize needed resources
  • Organize the interventions

Domains for Person Level Assessment
  • Work
  • School
  • Housing/Community Living
  • Social/Recreational/Community Participation
  • Parenting and other Social Roles
  • Quality of Life
  • Empowerment and Recovery

What to Measure in Person Level Assessment
  • What is the goal environment or domain?
  • Skills strengths and deficits
  • What skills are required to be successful and
  • What can the person do or not do?
  • Environmental Resources and Supports
  • What supports are required to be successful and
  • What does the person have or not have?

Assessment of Functioning
  • Assessment of Functioning does not Skills
  • Global Functioning Focus on Outcome of
  • Performance in Roles (spouse, worker)
  • Performance in Domains (social, psychological)
  • Status (employment, educational, residential)
  • Tools GAF, BASIS-32, Multnomah Community
    Ability Scale, Life Skills Profile

Skill Assessment Examples
  • Skill Functioning Focus on Behaviors
  • Living
  • Test of Grocery Shopping
  • Independent Living Skills Survey ILSS
  • Client Assessment of Skills, Interests, and Goals
  • Work
  • Work Behavior Inventory WBI
  • Vocational Cognitive Rating Scale - VCRS
  • Parenting
  • Parenting Well Strengths and Goals

Resource Assessment Examples
  • Social Support
  • MOS Social Support Scale
  • Parenting
  • Family Resources Scale

Examples of Assessments for Intervention Planning
  • Observational assessments
  • Test of Grocery Shopping Skills
  • Vocational Cognitive Rating Scale
  • Work Behavior Inventory
  • Self-report measures
  • ParentingWell Strengths and Goals
  • Family Resource Scale
  • MOS Social Support Scale
  • Interviews
  • Independent Living Skills Survey
  • Client Assessment of Strengths, Interests and
    Goals (CASIG)

Test of Grocery Shopping Skills
  • Developed to be used in conjunction with a
    grocery shopping skills training curriculum for
    people with serious mental illness
  • Assesses persons ability to locate 10 specific
    common food items in a regular grocery store
  • Can be used as a baseline assessment and a
    follow-up assessment after training 2 versions
  • Developed by Hamera and Brown at University of
    Kansas Medical Center
  • http//

Test of Grocery Shopping Skills
  • Examines the persons ability to find
  • Correct item
  • Correct size
  • Lowest price
  • Takes speed into account
  • Sample items
  • 15 oz. can red kidney beans
  • 1 lb. bag salad mix
  • 16 oz. frozen corn
  • 8 oz. low fat/nonfat strawberry yogurt
  • 16 oz. spaghetti

Vocational Cognitive Rating Scale
  • A 16-item instrument designed to measure
    cognitive skills in the workplace
  • Ratings are accomplished by a clinician who has
    observed the individual in the work setting and
    has interviewed the onsite supervisor
  • Behavioral anchors are provided for each rating
    on each item
  • Developed by Greig, Nicholls, Bryson, Bell
    (2004) at Yale University School of Medicine

VCRS Sample Item
  • Stays focused when performing a simple task
  • Consistently Inferior performance. Is
    consistently off task. Requires nearly continuous
    help from job coach/supervisor to stay focused.
  • Occasionally inferior performance. Can remain on
    task for brief intervals, but requires frequent
    monitoring by job coach to stay focused.
  • Adequate performance. Generally is able to stay
    on task and keep focused when doing simple tasks
    with minimal or infrequent assistance.
  • Occasionally superior performance. Stays on task,
    and only rarely requires assistance with
  • Consistently superior performance. Consistently
    completes simple tasks in a focused, organized

Work Behavior Inventory
  • A 36-item work performance assessment
  • Social Skills
  • Cooperativeness
  • Work Habits
  • Work Quality
  • Personal Presentation
  • Ratings on a 5-point scale are accomplished by a
    clinician who has observed the individual in the
    work setting and has interviewed the onsite
  • Indicates both strengths and deficits, Behavioral
    anchors are provided for each rating on each item

WBI Sample Items
  • Each item rated on a 5-point scale from
    1-Consistently inferior performance to
    5-Consistently superior performance
  • Expresses positive feelings appropriately
  • Accepts constructive criticism without becoming
  • Begins work tasks promptly
  • Corrects own mistakes
  • Personal hygiene is satisfactory
  • General Rating of Work Behavior

ParentingWell Strengths and Goals
  • A parenting self-assessment and collaborative
    goal setting tool
  • Developed for use with parents with mental
    illness and their families
  • 26 items covering everyday demands of parenting
  • Developed by Nicholson Henry
  • Available at

ParentingWell Strengths and Goals Sample Items
Family Resource Scale
  • Assesses family self-sufficiency by determining
    the adequacy of different types of resources in
    households of young children.
  • 30 item self-report tool, rated using a 5-point
    Likert-type scale anchored by "not at all
    adequate" to "almost always adequate.
  • Covers access to food, shelter, financial
    resources, transportation, health care, time to
    be with family, child care, and time for self.
  • Developed by Leet Dunst
  • Published in Dunst, C. J., Trivette, C. M.,
    Deal, A. G. (Eds.) (1994). Supporting and
    strengthening families. Methods, strategies and
    practices. Cambridge, MA Brookline Books.

Family Resource Scale Sample items
MOS Social Support Survey
  • 19 item self-report tool measuring multiple
    dimensions of social support
  • Emotional/informational
  • Tangible
  • Affectionate
  • Positive social interaction
  • Designed for use with individuals with long-term
    conditions, including serious mental illness
  • Developed by Sherbourne Stewart
  • Available from http//

MOS Social Support Survey Sample items
Independent Living Skills Survey
  • Designed to assess ADLs and IADLs among people
    with schizophrenia and other serious mental
  • Two versions
  • Self-Report Interview 76 items
  • Personal appearance care of possessions food
    prep and storage health maintenance money
    management transportation leisure and community
    participation job seeking and job maintenance
  • Informant Interview 104 items
  • Same as above also includes eating behaviors
    social relationships
  • Both instruments are available in appendix of
    published article Wallace, C. J., Liberman, R.
    P., Tauber, R., Wallace, J. (2000). The
    Independent Living Skills Survey A comprehensive
    measure of the community functioning of severely
    and persistently mentally ill individuals.
    Schizophrenia Bulletin, 26(3), 631-658.

Independent Living Skills Survey-I Sample items
Clients Assessment of Strengths, Interests and
Goals (CASIG)
  • A lengthy, structured interview designed for use
    with people with serious mental illness, covering
    multiple domains of functioning
  • Assesses clients goals related to
  • Housing/living financial/vocational
    relationships spiritual/religious health
  • Assesses clients skills/strengths/needs related
  • Lifestyle supports money management health
    management nutrition vocational
    transportation friends leisure personal
    hygiene care of possession medications and side
    effects personal rights cognition symptoms
    risk behaviors
  • Assesses satisfaction with
  • Quality of life quality of treatment
  • Developed by Wallace, Lecomte, Wilde, Liberman
    (2001) at UCLA
  • Available at http//

CASIG Sample items
  • Goals
  • Housing/Living Goals
  • One year from now, what would you like your
    living arrangements to be?
  • Financial/Vocational Goals
  • Would you like to improve your money/finances in
    the next year? How might you be able to improve
    them? Do you want to get a job or go to school
    in the coming year?

CASIG Sample items
  • Skills/Strengths
  • Health management
  • …at any time in the last 3 months did you
  • Make most of your own appointments with your
    doctor, case manager, etc?
  • Administer your own medication?
  • Food preparation
  • …at any time in the last 3 months did you
  • Plan your meals with a health balance of foods?
  • Use a stove to prepare meals?

Individualized assessment and goal setting
  • Goal Attainment Scaling
  • Alexis Henry

Goal Attainment Scaling
  • Framework for developing service goals (outcomes)
    that are measurable, relevant, attainable
  • Individualized goals
  • Quantitative index of clients progress
  • Can compare client to self over time
  • Can compare attainment across clients

Advantages of GAS
  • Not bound by any theoretical orientation,
    treatment or outcome measure
  • Allows for individualization of goals
  • Allows for prioritizing goals
  • Numeric index of performance over time
  • Easily computed
  • Useful in informal evaluations or in formal
    experimental designs

Disadvantages of GAS
  • Reliability issue needs to be considered
  • Outcomes best assessed by blind judge
  • Possibility of floor effect
  • Possibility of misuse if goals are too easy

Steps in GAS
  • Identify overall objective or domain
  • Identify specific problem area
  • Identify behavior that indicates improvement
  • Determine data collection method
  • Specify expected outcome
  • Specify most and least favorable outcome
  • Determine current status and timeframe for

Goal Attainment Scale
  • -2 most unfavorable outcome
  • -1 less than expected outcome
  • 0 expected outcome
  • 1 greater than expected outcome
  • 2 most favorable outcome
  • Goals can be weighted to reflect priorities

GAS Example 25 y.o. male wants to increase work
hours and earnings
  • 2 works 20 hrs/week at 10/hour
  • 1 works 10 hrs/week at 10/hour
  • 0 works 10 hrs/week at minimum wage
  • -1 works less than 10 hours/week
  • -2 no longer working

GAS Example 35 y.o. mother wants to have more
fun with kids age 8 and 10
  • 2 family participates in mutually enjoyable
    leisure/recreational activity 1x per week
  • 1 family identifies mutually enjoyable
    leisure/recreational activity participate 2x per
  • 0 family engages in leisure together about
    once per month
  • -1 family leisure/recreational activities
    happen less than once per month
  • -2 no regular family leisure/recreational

Program Level Assessment for Outcome Tracking
  • Sandra Wilkniss, PhD

Program Level Assessment for Outcome Tracking
  • Why track outcomes on a program level?
  • Domains of outcome tracking
  • Clinical Status
  • Recovery Status
  • Quality of Life
  • Functional Status
  • Safety and Health
  • Context and Methods for collecting and
    interpreting outcomes
  • Types of Services and related outcome tracking
  • Sample outcome tracking tools
  • Exercise Selecting assessment tools for outcome

Why track outcomes on a program level?
  • Access to services
  • Quality of services
  • Cost/benefit

Domains of outcome tracking
  • - Demographics Clinical Status
  • Recovery Status/Quality of Life
  • Functional Status
  • Roles/Community Integration
  • Skills
  • Resources
  • Safety and Health

Demographics Clinical Status
  • Demographics
  • Diagnostic and Historical information
  • Service use (baseline and change)
  • Physical health related outcomes
  • Cognition
  • Hospitalization
  • Jail and prison contact

Recovery/Quality of Life Status
  • Domains
  • Recovery
  • Empowerment
  • Self Efficacy
  • Hope
  • Satisfaction
  • Quality of Life
  • Methods
  • Consumer Self-report tools
  • Provider Evaluation tools
  • Family/significant other input
  • Interviews
  • Quantitative
  • qualitative
  • Proximal markers

Functional Status
  • Living situation
  • Housed or homeless
  • Level of independence
  • In community of choice
  • Education
  • Type of institution
  • Number of credit hours attempted/completed
  • Degree or voc/tech certificate
  • Work
  • In a job of consumers preference
  • Competitive
  • Salary and/or benefits
  • How much
  • For how long
  • Advancement
  • Follow along support variables

Functional Status (continued)
  • Community Integration
  • Instrumental Roles
  • Skills and Personal Resources
  • Social
  • Vocational
  • Financial
  • Recreational
  • Cognitive
  • Others
  • Environmental resources and supports available
  • Community
  • Family
  • Social capital
  • Others

Safety and Health
  • Personal and other safety
  • Physical Health and Wellness

Context and Methods for Collecting and
Interpreting Outcomes
  • Context
  • Cultural Sensitivity
  • Philosophy and interpretation
  • Specific measures
  • Cognition
  • Methods
  • Baseline tracking
  • Interval tracking
  • sources

Services and Interventions Offered Type,
quantity, and quality
  • Outreach
  • Linkage
  • PSR
  • Individual and group
  • Peer Support
  • Integrated Health care (primary care mental
  • Pharmacotherapy
  • Adjunct Interventions (e.g., cognitive support)
  • Evidence-Based or emerging best practices
  • GOI/penetration
  • Fidelity

Sample Tracking Tools
  • Comprehensive outcomes packages
  • Specific Measures
  • Cultural diversity measures
  • Recovery/Satisfaction Measures
  • Empowerment
  • Cognitive Functioning
  • Skills
  • Environment

Exercise Selecting Tools for Outcome Tracking
  • Selection Strategies
  • Determine program emphasis/focus
  • Evaluate needs and preferences of individual
  • Evaluate resources and means for data collection,
    description/analysis, interpretation and
  • Review sample tools in each domain of interest
  • evaluate pros and cons
  • Psychometric properties vs. uniqueness
  • Thorough vs. brief and more accessible
  • Language geared toward consumers
  • Issues of cultural diversity
  • Etc.

Integrating Assessments in Psychiatric
Rehabilitation Programs Where the rubber meets
the road…
  • Joanne Nicholson
  • So far weve covered…
  • Principles for assessment
  • Issues in measurement
  • Person-level assessment
  • Program-level assessment

Integrating assessments in programs requires
consideration of
  • Relevant domains, e.g., work, education, etc.
  • Skills
  • Resources and supports
  • Individuals goals
  • Programs goals
  • Additional outcome tracking needs, e.g., data
    required for program funders?
  • Organizational resources and supports

Providers Motivations for Using Evidence-Based
  • Enhanced professional competence
  • Perceived utility
  • Fit with usual tasks and duties
  • Fit with ones own experience
  • Fit with mission of organization, structure of
    the intervention, organizing conceptual model

(Aarons Palinkas, 2007)
Potential Barriers to Use
  • Lack of training, motivation, connection
  • Measures, methods viewed as irrelevant
  • Measures, methods viewed as burdensome
  • Measures, methods not viewed as useful
  • Lack of organizational support?resources
  • Program measures and research measures dont
    mix well

Strategies for Overcoming Barriers to Assessment
  • Educate re rationale
  • Engage program staff participants in process
  • Anticipate address logistical challenges
  • Create opportunities for feedback refinement
  • Use data in meaningful, productive ways

Learning Activity
  • Break into groups ? Select a domain
  • Discuss assessment issues
  • Intervention planning? Outcomes?
  • Existing measures? Adaptation? Refinement?
  • Logistics Why? Who? How?
  • Mechanisms for feedback to staff, participants?
  • Identify implementation challenges suggest
    strategies for addressing
  • Report back