From Concept to Implementation: The Challenges Facing Evidence-Based Social Work - PowerPoint PPT Presentation


PPT – From Concept to Implementation: The Challenges Facing Evidence-Based Social Work PowerPoint presentation | free to view - id: 72bdcb-MmU3N


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation

From Concept to Implementation: The Challenges Facing Evidence-Based Social Work


Evidence-Based Social Work Is A Concept Awaiting ... Implementation Monitoring Audit Examples of ... of direct practical importance to ... – PowerPoint PPT presentation

Number of Views:125
Avg rating:3.0/5.0
Slides: 48
Provided by: Edward234


Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: From Concept to Implementation: The Challenges Facing Evidence-Based Social Work

From Concept to Implementation The Challenges
Facing Evidence-Based Social Work
  • Faculty Research Insights A Series Featuring
    CUSSW Faculty Research
  • Edward Mullen Aron Shlonsky
  • September 22, 2004

Evidence-Based Social Work Is A Concept Awaiting
  • Much discussed
  • Required in EPAS
  • Not clearly Defined
  • Not yet implemented in pure form

--- the conscientious, explicit and judicious
use of current best evidence in making decisions
regarding the welfare of service-users and
carers (Sheldon, 2003, p. 1).
Placing the clients benefits first,
evidence-based practitioners adopt a process of
lifelong learning that involves continually
posing specific questions of direct practical
importance to clients, searching objectively and
efficiently for the current best evidence
relative to each question, and taking appropriate
action guided by evidence (Gibbs et al., 2003,
p. 6).
Evidence-based medicine has been defined as ---
the conscientious, explicit and judicious use of
current best evidence in making decisions about
the care of individual patients (Sackett, et
al., 1996, p. 71) and the "integration of best
research evidence with clinical expertise and
patient values" (Sackett, Straus, Richardson,
Rosenberg, Haynes, 2000, p. 1).
Origin of Evidence-Based Social Work
  • Adaptation of evidence-based approaches found in
    medicine and healthcare
  • Evolution from earlier attempts to join
  • Effectiveness research
  • Eclecticism
  • Empiricism the scientist-practitioner model
  • Social intervention research
  • Outcomes measurement
  • Systematic review methodologies meta-analysis
  • Eileen Gambrill early translator (1999)
  • Leonard Gibbs first workbook-text (2003)
  • Aaron Rosen Enola Proctor on guidelines (2003)

The EBP Model
Sackett et al., 1997
Newest EBP Model
Haynes, Devereaux, and Guyatt, 2002
Steps of EBP
  • Become Motivated to Apply EBP
  • Step 1Convert information need (prevention,
    assessment, treatment, risk) into an answerable
  • Step 2Track down current best evidence.
  • Step 3Critically appraise the evidence.
  • Step 4Integrate critical appraisal with practice
    experience, clients strengths, values, and
  • Step 5Evaluate effectiveness and efficiency in
    exercising steps 1-4 and seek ways to improve
    them next time.
  • Step 6Teach others to follow the same process

Gibbs, 2003
The Cycle of EBP
Appropriate for this client?
Actuarial risk assessment
Other valid assessment measures?
Clients preferred course or at least willing to
Effective services
Barriers (e.g., cultural conflict
Adapated from Haynes, Devereaux, and Guyatt,
2002 as presented in Shlonsky and Wagner, in
Hallmarks of EBP
  • Individualized assessment (strengths, needs,
    values, preferences of client)
  • Well-formulated question (database query)
  • Well-executed search of research literature
  • Deciding its applicability to clients
  • Considering the evidence together with the values
    and preferences of the client.

Sackett et al., 1997 Gibbs, 2003
Origins of Evidence-Based Healthcare Public
  • Used to inform health care policy at hospital,
    regional, national levels (Davies Boruch,
    2001 Romanow, 2002 Gray, 2001 Davies, Nutley,
    Smith, 2000)
  • Became component of New Labours modernizing
    government strategy - extended to other sectors
    including social care (Davies, 2004 Walker,
  • Facilitated by
  • Cochrane Campbell Collaborations
  • UK ESRCs evidence-based practice policy
  • UK Social Care Institute for Excellence
  • Sweden Institute for Evidence-Based Social Work
  • Publications e.g., Evidence-based Policy A
    Journal of Research, Debate and Practice

Policy Applications
  • An approach to policy that
  • helps people make well informed decisions about
    policies, programs projects
  • EBPolicy contrasts with opinion-based policy
  • which relies heavily on either the selective use
    of evidence (single studies irrespective of
  • untested views of individuals or groups
  • Philip Davies (2004), Director of Policy
    Evaluation in the Cabinet Office of the UK Prime
    Ministers Strategy Unit

What is evidence-based healthcare?
  • Discipline centered on evidence-based
    decision-making about
  • -Groups of patients -Populations
  • Evidence from wide range of disciplines
  • Use of scientific research findings and logic
  • Healthcare problems
  • Health improvement
  • Decision-making in healthcare

Gray 2001
Evidence-Based Policy Definition
Decision-making process in which policy-makers,
purchasers, managers, or practitioners together
with key stakeholders make decisions about
problem or need assessment, goals objectives,
interventions outcomes
EBPolicy Process (Gray, 2001)
Sources of Evidence in EBP
Adapted from Davies, 2004
EBPolicy Steps
  • Finding and appraising evidence pertaining to the
    decision at hand
  • Assessment or intervention options
  • Intended outcomes
  • Research method
  • Developing organizational capacity
  • Getting the evidence into practice
  • Preparing a policy
  • Making cultural changes
  • Designing systems for implementation
  • Implementation
  • Monitoring
  • Audit

Gray, 2001
Examples of Evidence-based Policy Initiatives
  • ESRC Evidence Network - UK Centre for Evidence
    Based Policy Practice
  • U.K. Cabinet Office
  • University of St. Andrews
  • Romanow Commission Canada
  • U.S. Implementing Evidence-based Practices

Arguments For Evidence-Based Social Work (EBSW)
  • Enhances decision quality
  • Fosters learning of assessment skills
  • Incorporates client values and expectations
  • Fosters evidence search and appraisal skills
  • Makes best use of best evidence
  • Framework for self-directed, life-long learning
  • Identifies gaps in knowledge
  • Common interdisciplinary language

Sackett et al, 2000
Arguments For Evidence-Based Social Work (EBSW)
  • Eschews authoritarian practices and policies
  • Promotes SW ethics through
  • Informed consent
  • Using effective services
  • Wisely allocating scarce resources
  • Involving clients in practice process
  • Individualizing personalizing decisions

Gambrill, 2003
Arguments For Evidence-Based Social Work (EBSW)
  • Helping clients develop critical appraisal skills
  • Involving clients in design and critique of
    practice and policy related research
  • Involving clients as informed participants who
    share in decision making
  • Recognizing clients unique knowledge in terms of
    application concerns
  • Promoting transparency and honesty
  • Encouraging a systemic approach for integrating
    practical, ethical and evidentiary issues
  • Maximizing the flow of knowledge and information
    about knowledge gaps

Gambrill, 2003
Challenges Facing EBSW
  • Evidence-of-effectiveness
  • Authority
  • Conflicting hierarchy
  • Definition of evidence
  • Shortage of evidence
  • Variation by field of practice
  • Inflation of evidence
  • Small transient effects
  • Assessment validity
  • Nomothetic versus ideographic knowledge
  • Efficacy versus effectiveness
  • Diffusion, adoption and implementation
  • Policy system level issues
  • Training sustaining

Evidence-of-effectiveness Challenge
  • A key assumption of EBP is that it results in
    better outcomes than other forms of practice
  • Yet, this assumption is not empirically testable
  • This proof is no more achievable for the new
    paradigm (EBM) than it is for the old, for no
    long-term randomized trials of traditional and
    evidence-based medicine are likely to be carried
    out. (Evidence-Based Medicine Working Group,
    1992, 2424)

Authority Challenge
  • As originally conceived the founders of
    evidence-based medicine envisioned that medical
    practitioners would seek out evidence appraise
    the quality of that evidence, alone or in teams
    of practitioners
  • The new paradigm puts a much lower value on
    authority. The underlying belief is that
    physicians can gain the skills to make
    independent assessment of evidence and thus
    evaluate the credibility of opinions being
    offered by experts (Evidence-Based Medicine
    Working Group, 1992)

Authority Challenge
  • Lack of evidence that EBM can be transmitted into
  • Medical students have not shown an interest in
    becoming evidence-based practitioners only
    evidence users
  • As evidence users rather than independent
    appraisers, practitioners are simply replacing
    one authority with another researchers are the
    new authorities!

Authority Challenge
  • To date it has not been shown that average
    practitioners can gain the skills needed to
    conduct independent assessments of the evidence
  • Practitioners can become evidence-users not
    evidence-based practitioners say Upshur Tracy
  • Evidence-users accept evidence that has been
    vetted by others thus replacing one authority
    for another! But what is the basis for the
    authority of the evidence appraisers?

Conflicting Hierarchy Challenge
  • Hierarchy of Evidence Ranked by Quality
  • N of 1 randomized trials
  • Systematic reviews/Meta-analysis of randomized
  • Randomized Controlled Trials
  • Systematic review of observational studies
    addressing patient-important outcomes
  • Physiological/Laboratory experiments
  • Unsystematic clinical observation

Conflicting Hierarchy Challenge
  • Practice recommendations (e.g., practice
    guidelines) are to be based on the soundness of
    the evidence
  • Many evidence hierarchies have been published
    these hierarchies are not commensurable (e.g.,
    randomized, controlled trials and meta-analysis
    versus well designed epidemiological studies)
  • Fundamentally these hierarchies do not rest on
    evidence but rather on the consensus of experts
    the beliefs of the few!

Definition of Evidence Challenge
  • Because contemporary health care and human
    services are multidisciplinary there are many
    differing perspectives that require consideration
  • Accordingly, the concept of a univocal
    understanding of evidence is not sustainable
  • A restrictive view of evidence is not sustainable

Shortage of Evidence Challenge
  • The question that faces proponents of EBP is
    whether there are enough high-quality studies so
    that evidence-based decisions can be made.
  • Surprisingly for a field that places a high
    premium on research, few studies have examined
  • The conclusion at this point, based on just a few
    studies, is that there are still many decisions
    that are made that are not based on good
    evidence, but the picture is not nearly as bleak
    as opponents to EBP would have us believe.
  • Professionals must remember, though, that when
    they make decisions for which little or no
    evidence exists, that they should exercise
    caution and perhaps be even more vigilant in
    monitoring outcomes.

Shortage of Evidence
  • Varies by field of practice
  • Confounded by inflation of evidence by proponents
    and users
  • Clouded by dominance of small and transient
  • Applies to assessment evidence as well as
    intervention evidence

Shortage of Evidence Varies by Field of Practice
  • Adequacy varies considerably by field of practice
  • Most robust is mental health
  • In other fields of practice much more of a
  • Unevenness of the research base across fields of
    practice looms as a major challenge

Inflation of Evidence Challenge
  • Bias in reported research due to
  • Tendency for investigators or assistants to give
    a leg up to interventions they favour -
    expectation for improvement
  • Dependence on client self-report to measure
    intervention outcomes which may reflect social
    desirability, expectancy, cognitive dissonance

Small Transient Effect Size Challenge
  • The rule is small effect sizes which may not be
    clinically or social important
  • Gains frequently do not persist, especially for
    complex and chronic problems such as addressed by
    social workers

Assessment Challenge
  • Reliable, valid, relevant assessment tools are
    needed to support evidence-based social work
  • Rapid assessment measures have contributed
    greatly (Corcoran Fisher, 2000)
  • Social work practitioners generally do not use
    any standardized assessment procedures to guide
    their practice (Mullen Bacon, 2004)

Shortage of Evidence Challenge
  • Evidence-based social workers must remember that
    when they make decisions for which little or no
    evidence exists, that they should exercise
    caution and perhaps be even more vigilant in
    monitoring outcomes
  • Knowledge gaps point the way to needed research
  • Methodological corrections should be instituted
    to deal with the inflation of evidence issue
  • Where interventions are found to have small with
    transient effects of limited importance this
    should be acknowledged and addressed
  • The underutilization of assessment instruments
    should be corrected by enhance training in the
    use of appropriate instruments dissemination of
    information about available instruments.

Challenge of Applying Results to Individuals
  • Results of RCTs are analyzed by comparing the
    mean score of the experimental group against that
    of the placebo or control group (or some
    comparable summary statistic). This masks the
    fact that there is always individual variability
    around the means, overlap in the distributions
    of scores for the two groups. The result of this
    is that a proportion of people in the
    experimental group actually do worse than some in
    the control group and, conversely, some in the
    comparison group improve more than some people in
    the active treatment group.
  • Practitioners cannot blindly apply a proven
    procedure and assume that a particular individual
    receiving that procedure will benefit

Challenge of Applying Results to Individuals
  • We are at least able to quantify the probability
    with which an individual person will respond to a
    given procedure (Number Needed to Treat)
  • Alternative to using evidence-based interventions
    with their known rate of failure is to use
    unproven procedures, based only on the hope that
    they may work
  • Practitioners can and should view each case or
    situation as an N 1 study and collect data

Efficacy Versus Effectiveness Challenge
  • The efficacy versus effectiveness challenge
    requires a careful consideration of the trade off
    between internal and external validity
  • Claims to evidence-based practice typically must
    be grounded in random, controlled trials which
    usually translate into efficacy studies
  • Such studies do not address how effective such
    interventions would be in real world contexts
  • At the present time there is a paucity of
    evidence of relevance to social work based on
    effectiveness studies conducted in real world
  • it may be more efficient to begin with studies
    based on conditions as realistic as possible so
    as to avoid the misinformation that too
    frequently comes from misinterpretations of
    efficacy studies

Adoption Implementation Challenge
  • Two major challenges
  • (1) adoption effective implementation of
    evidence-based practice in social agencies
  • (2) educating for evidence-based practice

Training, Time, Resources Challenges
  • EBP requires
  • Training in search techniques
  • Training in critical appraisal
  • Computer resources
  • Electronic resources

Killer Bs
  • Low BASE rate
  • Incompatible client or community BELIEFS
  • BAD BARGAIN in terms of scarce resources
  • BARRIERS too high

Adoption Implementation Challenge
  • Motivate core group
  • Acceptance of EBP
  • Facilitate adoption
  • Implementation of effective services
  • Evaluate outcomes

Rogers Shoemaker, 1971
Gira, Kessler and Poertner (2004)
  • Survey of reviews of adoption of research
    evidence medical practice
  • Educational outreach visits and audit and
    feedback showed weak to moderate effects
  • Certain types of continuing education and the use
    of computers as decision aids showed moderate
  • There are no magic bullets (Oxman et al., 1995)
  • The literature from health care suggests that
    disseminating information alone is insufficient.
    Many interventions have been designed to improve
    practitioners adherence to EBP guidelines and
    are differentially effective. To date, no
    intervention has demonstrated powerful effects
    (p. 77-78).

Adoption Implementation Challenge
  • Can social agencies adopt EBP approach?
  • Can social workers become EB practitioners?
  • What are the barriers? Facilitators?
  • What resources are needed?
  • What training is required?
  • What systems need to be established?
  • How will EBP change client experience?
  • What outcomes will be achieved?

Challenges in Teaching Evidence-based Social Work
  • Fear of Loss of Human Context
  • Disconnect between classroom and field
  • Overwhelmed by new skills and knowledge needed

(Bilsker Goldner, 2004)
A Challenge to Critics
  • Knowledge and information revolution
  • Valuable resource
  • Cannot be ignored
  • Must be harnessed

A Measured Optimism
  • Real world constraints -limitation of
    approach -social and political realities
  • Great promise, but need for caution -harnessing
    knowledge and information -management of scarce
    resources -unknowns abound
  • Hold EBP to its own evidentiary standards -the
    jury is still out

Knowledge needs to be managed much more
effectively than it has been in the past
The challenge is to rethink our view of what
knowledge is how best to facilitate its rapid
generation, sharing, and application in a manner
that closes the policy-research-practice gap