From Evidence-based Practice to Practice-based Evidence: Behavior Analysis in Special Education

1
From Evidence-based Practice to Practice-based
Evidence Behavior Analysis in Special Education

• Ronnie Detrich
• Wing Institute

2
What is Evidence-based Practice?

• EBP is a decision-making approach that places
  emphasis on evidence to
• guide decisions about which interventions to use.
• evaluate the effects of an intervention.
• Sackett et al (2000) defined evidence-based
  practice in medicine as the integration of best
  research evidence with clinical expertise, and
  patient values.

3
What is Evidence-based Practice?

• Ultimately, EBP is a consumer protection issue.
• Assumes that evidence-based interventions are
  more likely to be effective than interventions
  that are not evidence-based.
• By validating interventions as evidence-based
  there is the implication that there are standards
  for reviewing interventions.
• Standards should be transparent.

4
The Scope of the Problem

• Kazdin (2000) identified 550 named interventions
  for children and adolescents.
• A very small number of these interventions have
  been empirically evaluated.
• Of those that have been evaluated, the large
  majority of them are behavioral or
  cognitive-behavioral.

5
The Scope of the Problem

• We have a great deal of evidence but much of that
  evidence is not being applied in practice
  settings.
• Evidence-based interventions are less likely to
  be used than interventions for which there is no
  evidence or there is evidence about lack of
  impact (Kazdin, 2000).
• In many instances practitioners are not aware of
  evidence-based interventions (Kratochwill, Albers
  Shernoff, 2004).
• There is very little guidance to parents and
  practitioners about how to decide among
  interventions.

6
Why the Increased Interest in Evidence-based
Interventions?

• No Child Left Behind (NCLB) requires that
  interventions used to improve educational
  performance are based on scientific research.
• In NCLB there are over 100 references to
  scientific research.
• Individuals with Disabilities Education
  Improvement Act IDEIA (2004) requires that
  interventions are scientifically based
  instructional practices.

7
Special Education and Evidence-based Interventions

• Pre-service and professional development for all
  who work with students with disabilities to
  ensure such personnel have the skills and
  knowledge necessary to improve the academic
  achievement and functional performance of
  children with disabilities, including the use of
  scientifically based instructional practices, to
  the maximum extent possible.

8
Special Education and Evidence-based Interventions

• Scientifically based early reading programs,
  positive behavioral interventions and supports,
  and early intervention services to reduce the
  need to label children as disabled in order to
  address the learning and behavioral needs of such
  children.

9
Special Education and Evidence-based Practice

• The Individualized Education Program (IEP) shall
  include a statement of the special education and
  related services and supplementary aids and
  services, based on peer-reviewed research to the
  extent practicable, to be provided to the child,
  or on behalf of the child, and a statement of the
  program modifications or supports for school
  personnel that will be provided for the child.

10
Ethical Conduct and Evidence-based Interventions

• Most national psychological and educational
  organizations have ethical standards requiring
  science-based practices to address problems.
• American Psychological Association Ethical
  Standard 2.04
• Psychologists work is based on the established
  scientific and professional knowledge of the
  discipline.

11
Ethical Conduct and Evidence-based Practice

• National Association of School Psychologists
• Standard III F 4.
• School psychology faculty members and clinical or
  field supervisors uphold recognized standards of
  the profession by providing training related to
  high quality, responsible, and research-based
  school psychology services.

12
Ethical Conduct and Evidence-based Practice

• National Association of School Psychologists
• Standard IV 4.
• School psychologists use assessment techniques,
  counseling and therapy procedures, consultation
  techniques, and other direct and indirect service
  methods that the profession considers to be
  responsible, research-based practice.

13
Ethical Conduct and Evidence-based Practice

• Behavior Analysis Certification Board
• Standard 2.09a
• The behavior analyst always has the
  responsibility to recommend scientifically
  supported, most effective treatment procedures.
  Effective treatment procedures have been
  validated as having both long-term and short-term
  benefits to clients and society.
• Standard 2.09b
• Clients have a right to effective treatment
  (i.e., based on the research literature and
  adapted to the individual client).

14
Controversies in Evidence-based Education

• There is no consensus about what constitutes
  evidence.
• The definition offered in NCLB permits both
  quantitative and qualitative evidence without
  specifying the types of questions that each
  approach best answers.
• In this context, we are most often concerned with
  evidence that establishes a causal relation
  between an intervention and a class of social or
  academic behaviors.
• .

15
Controversies in Evidence-based Education

• There is no agreed upon standard for either the
  quantity or the quality of evidence necessary to
  validate an intervention as being evidence-based.
• Several organizations have established standards
  but there is limited agreement among them.
• It is possible for an intervention to meet one
  standard but not a second.

16
Applied Behavior Analysis and Evidence

• The research methods of single participant design
  are convincing demonstrations of causal
  relations.
• Behavior analysis research has often been
  criticized for limited generalizability because
  of the small number of participants in a research
  study.

17
Applied Behavior Analysis and Evidence

• Generally, the developed standards for validating
  an intervention as evidence-based have relegated
  single participant designs to a lower quality of
  evidence.
• This is problematic because in some areas of
  educational research almost all of the evidence
  is based on these designs.
• Autism
• Developmental Disabilities

18
Applied Behavior Analysis and Evidence

• There are no established standards within applied
  behavior analysis for validating interventions.
• There is no single resource that decision-makers
  can turn to for guidance about the best
  intervention to use for a particular problem.
• May lead decision makers to base decisions on
  criteria other than evidence.

19
Applied Behavior Analysis and Evidence

• Some related fields have published standards
• Horner, et. al. (2005) provides guidelines for
  evaluating evidence established with single
  participant research.
• Flay, et. al. (2005) standards of Society for
  Prevention Research.
• National Standards Project is developing
  standards of evidence for interventions for
  individuals with autism.
• Task Force on Evidence-based Interventions in
  School Psychology.
• What Works Clearinghouse.
• Only WWC has reviewed the literature in a
  specific area to provide guidance about effective
  interventions.

20
Implementation Research to PracticeWhere good
interventions go to die

• Identifying evidence-based interventions is
  necessary but not sufficient to assure they will
  be implemented in practice settings.
• It is also necessary to address complex issues
  associated with implementation (research to
  practice).

21
Four Concerns for Implementation

• Dissemination of evidence-based interventions in
  a manner that is accessible and useful to
  decision-makers.
• Publication in professional journals is not an
  effective dissemination strategy for decision
  makers.
• Evidence about interventions are distributed
  across multiple journals.
• Most practice level decision-makers are not
  adequately trained to evaluate primary source
  data.
• Because of time constraints, reading professional
  journals is a low priority activity for many
  decision-makers.

22
Four Concerns for Implementation

• Selection of evidence-based interventions.
• Often variables unrelated to effectiveness
  influence selection of an intervention.
• Cost
• Expert opinion
• Personal experience
• Effort associated with systems change

23
Four Concerns for Implementation

• Initial Implementation of new intervention
• What is necessary to gain practitioner support?
• Often inconsistent with previous training and
  experience.
• What is required to train practitioners?
• What is necessary to assure integrity of
  implementation?
• Implementation without integrity is
  implementation of an unspecified intervention
  with unknown effects.
• How can new intervention be adapted to meet local
  circumstances?
• If changed too much it becomes a different
  intervention for which there are no data.
• If not modified to meet local circumstances it
  may not be implemented at all.

24
Four Concerns for Implementation

• Sustainability is the ultimate goal for
  successful interventions.
• Definition of sustainable programs
• Maintains over time.
• Maintains across generations of practitioners.
• Supported with existing resources of system.
• If effective interventions are not sustainable
  then the risk is great that alternative,
  ineffective programs will emerge.
• The larger the scale of implementation the more
  complex all of the issues become and
  sustainability is more difficult to achieve.

25
Applied Behavior Analysis and Implementation

• Service delivery in applied behavior analysis is
  a mediated model.
• Behavior Analysts coach parents, teachers, or
  other mediators to implement an intervention
  plan.
• Mediated model requires that behavior analysts
  address many of the issues of implementation for
  each project.

26
Progress MonitoringPractice-based Evidence

• Even if an evidence-based intervention is
  implemented well, it is still necessary to
  systematically evaluate the impact.
• Progress monitoring is practice-based evidence
  about the effects of an evidence-based practice.
• It is difficult to predict which students will
  benefit from intervention prior to
  implementation.
• No intervention will be effective for all
  students.
• Progress monitoring is consistent with ethical
  guidelines.

27
Ethical Standards and Progress Monitoring

• National Association of School Psychologists
• Standard IV C 1b.
• Decision-making related to assessment and
  subsequent interventions is primarily data-based.
• Standard IV 6.
• School psychologists develop interventions that
  are appropriate to the presenting problems and
  are consistent with the data collected. They
  modify or terminate the treatment plan when the
  data indicate the plan is not achieving the
  desired goals.

28
Ethical Standards and Progress Monitoring

• Behavior Analysis Certification Board
• Standard 4.04
• The behavior analyst collects data or asks the
  client, client-surrogate, or designated other to
  collect data needed to assess progress within the
  program.
• Standard 4.05
• The behavior analyst modifies the program on the
  basis of data.

29
Progress Monitoring and Special Education

• Progress monitoring is fundamental to the IEP
  process.
• Progress monitoring is at the heart of Response
  to Intervention (RTI).
• All students are routinely and systematically
  monitored to determine if they are making
  adequate progress against established standards.

30
Progress Monitoring and Behavior Analysis

• Data collection and review is the sine qua non of
  applied behavior analysis.
• It is not applied behavior analysis if
  intervention data are not collected and reviewed
  to determine impact.
• Behavior analysts rely on direct measures of
  behavior and its products to determine progress.
• Words read correctly as a measure of reading
  ability.
• Frequency of specific problem behaviors or
  adaptive behaviors.

31
Progress Monitoring and Behavior Analysis

• Behavior Analysis best represents the
  relationship between evidence-based practices and
  practice-based evidence.
• For many areas in special education there are a
  wide range of behavior analytic evidence-based
  interventions.
• The fundamental emphasis on progress monitoring
  in behavior analysis makes it well suited for
  leading the movement to a comprehensive approach
  to evidence-based practices.