Evidence-based Practice: A Workshop for Training Adult Basic Education, TANF and One Stop Practitioners and Program Administrators

1
Evidence-based Practice A Workshop for Training
Adult Basic Education, TANF and One Stop
Practitioners and Program Administrators

2
Welcome

• Focus of this workshop
• How can you promote evidence-based practice in
  adult education programs?

3
Introductions

• Name
• Program name
• Your role in the program
• One strength of your program

4
Objectives By the end of this workshop, you
should be able to..

• Discuss your own attitudes about research and its
  connection to practice and policy
• Define some basic concepts and terminology about
  research design and methodology
• Describe the connection between evidence-based
  practice and program/classroom improvement
• Describe what empirical evidence looks like and
  how it can be integrated with professional wisdom
  to make decisions about instruction and services
  for adult students
• Implement a plan for improving reading
  instruction in your programs utilizing
  evidence-based practices
• Cite strategies you can use to continuously
  access, understand, judge and use research to
  make decisions about practices in your classrooms
  and programs

5
Agenda

• Todays activities include
• Introductory Activity Welcome, Objectives and
  Agenda
• Attitudes Towards Research Discussion Activity
• Introduction to Research Design
• Understanding Research and Identifying
  Evidence-based Practice
• Using Evidence-based Practice to Improve Your
  Programs
• Next Steps Action Planning
• Accessing, Understanding, Judging and Using
  Research in Your Own Programs
• Evaluation and Closure

6
Attitudes Towards Research

• How do you feel about research and its role in
  improving practice?

7
Evidence-based Practitioners Are

• Questioners
• Consumers
• Producers

8
Your Stance as Evidence-based Practitioners

• Questioners
• Believe that practice should be based on evidence
  and professional wisdom.
• We ask,
• Why should I use this technique or strategy and
  what is the evidence that supports it?
• Is it based on evidence I have about students
  performance, on other practitioners evidence
  (professional wisdom), or on research evidence?

9
Your Stance as Evidence-based Practitioners

• Consumers
• Believe that new evidence is critical to our work
  and proactively seek research evidence.
• Learn enough about research and its findings to
  integrate what has been found to be effective
  with our own knowledge of students, and then
  change our practice accordingly.

10
Your Stance as Evidence-based Practitioners

• Producers
• Believe that we should not only be consumers but
  also researchers in our own classrooms.
• Generate knowledge that can be shared with others
  through
• program and classroom research,
• co-research with university-based researchers or
• documenting how we implemented evidence-based
  practices.

11
Introduction to Research Design
Random Selection
Validity
Analysis
Quantitative Data
Findings Implications
Means Median Mode
Standard Deviation
Reliability
12
Research DesignThe Research Question

• Does practice in trying to flip the ball into the
  cup lead to improvement in that skill?

13
Research DesignLimitations of the Study

• Validity
• Was the number of subjects enough to reach a
  reasonable conclusion?
• Were the control and experimental groups chosen
  fairly?
• Was the sample typical of the population we want
  to know about?
• Was the right measurement chosen to test whether
  skills increased?

14
Research DesignLimitations of the Study

• Reliability
• Were the measurements accurate?
• Were all subjects measured the same way?
• Were data accurately recorded?

15
Research DesignExperimental Research

• So this is experimental, gold standard research
  (except for sample sizesmall n).
• Research question
• Random sampling
• Control group
• Baseline quantitative data (pre-test)
• Intervention
• Outcome quantitative data (post-test)
• Findings
• Analysis
• Rival hypotheses
• Conclusions
• Implications
• Validity and reliability

16
Research DesignResearch Options for Our Study

• True experiments Randomized trials that test a
  specific strategy or approach.
• Experiments answer "What works?" questions.
• Correlational studies Research that looks at the
  relationship between different variables and
  outcomes.
• Correlational studies answer "What influences
  what? questions.
• Case studies Highly detailed descriptions of
  individuals or small groups.
• Case studies answer "What's going on?" questions.

17
Evidence-based PracticeScientifically Based
Research Must

• employ systematic, empirical methods that draw on
  observation or experiment
• involve rigorous data analyses that are adequate
  to test the stated hypotheses and justify the
  general conclusions
• rely on measurements or observational methods
  that provide valid data across evaluators and
  observers, and across multiple measurements and
  observations and
• be accepted by a peer-reviewed journal or
  approved by a panel of independent experts
  through a comparatively rigorous, objective, and
  scientific review.
• What does all this really mean?
• From What is Scientifically based Research? A
  Guide for Teachers, at http//www.nifl.gov/partner
  shipforreading/publications/html/science/stanovich
  .html

18
Evidence-based PracticeRecognizing Effective
Research

• Has the study been published in a peer-reviewed
  journal or approved by a panel of independent
  experts?
• What does this mean?
• How would you know?
• Have the results of the study been replicated by
  other scientists?
• What does this mean?
• How would you know?
• Is there consensus in the research community that
  the study's findings are supported by a critical
  mass of additional studies?
• What does this mean?
• How would you know?

19
Evidenced-based PracticeAnalysis Group Task

• In your small group, you have 10 minutes to read
  the assigned research brief and discuss it with
  members of your group to be sure that you
  understand
• Who the learners were
• What the research question was
• What the researchers found
• What you think these findings mean for teaching
  reading to adult students
• Whether or not you think this study is
  scientifically based research and why
• You should feel comfortable enough with what the
  research says to be able to describe it to
  someone else.

20
Evidence-based PracticeDiscussion Group Task

• Each group member has five minutes to describe
  the information in the article to the other group
  members, including
• Who the learners were
• What the research question was
• What the findings were
• What your analysis group thought the findings
  mean for teaching reading to adults
• Whether or not your analysis group thought the
  study was scientifically based
• Encourage discussion

21
Evidence-based PracticeDefinition

• Evidence-based practice is defined as
• the integration of professional wisdom with the
  best available empirical evidence in making
  decisions about how to deliver instruction.
• Grover Whitehurst,
• Director of Institute of Education Sciences

22
Uses for Research

• Research justifies what we are already doing
  well.
• Research helps us improve something that we are
  already doing.
• Research suggests a solution to a problem we
  have.
• Research uncovers an interesting and intriguing
  practice.

23
Using Evidence-based Practice to Improve Your
ProgramYour Current Program Improvement Process

• In your local program groups, discuss a recent
  program change
• Who initiated the change?
• What problem did the change try to address?
• Who was involved in finding and deciding on a
  solution (change)?
• Who was involved in implementing the change?
• How did you know the change did or did not work?
• Who was involved in evaluating whether the change
  should be permanent?

24
Using Evidence-based Practice to Improve Your
Program A Case Study

• Read the case study and then discuss
• Are there any differences between your programs
  change process and the program process described
  in the case study?
• Are there parts of the case study process that
  might improve your own programs process?
• Are there parts of your programs process that
  might improve the case study process?

25
Action PlanningCreating a Plan for
Evidence-based Reading Instruction

• Stay in your local program groups.
• Identify one of the evidence-based reading
  practices that might contribute to your program
  or solve a problem you have had related to
  reading instruction.
• Look at the handout Action Plan for
  Evidence-based Reading Instruction.
• Draft an action plan to try out the strategy or
  practice in your program. Refer back to your
  notes from the last activity about program
  improvement process.
• TIME 15 minutes

26
Your Programs Own Strategy for Evidence-based
Practice

• Accessing research (finding it)
• In response to needs identified through program
  self assessment or data analysis
• In reading current research
• Understanding what it says and means
• Judging whether it holds promise for improving
  your services
• Using it in your program

27
Your Programs Own Strategy for Evidence-based
Practice

• To make a habit of accessing, understanding,
  judging and using research for program
  improvement, you need a structure and a process
• Who (in or out of your program) is involved?
• How do they continually access, understand, judge
  and use research to make a change?
• What do they need to do it?

28
When You Get Home

• Review the action plan for evidence-based reading
  practice.
• Review your draft strategy for evidence-based
  practice (accessing, understanding, judging and
  integrating research into your program on a
  continual basis).
• Discuss plan and strategy with colleagues.
• Refine/revise plan and strategy, and begin to
  implement!

29
Force Field Analysis Making Our Plans and
Strategies Work

• A Force Field Analysis is a tool for thinking
  about how to implement a strategy or plan.
• Brainstorm the restraining forces those that
  will make it harder to implement your strategy
  for evidence-based practice.
• Brainstorm the positive forces those that will
  make implementation easier.
• Discuss how you might reduce negative forces and
  strengthen the positive forces.

30
Accessing Research Findings

• www.ncsall.net
• National Center for the Study of Adult Learning
  and Literacy

31
(No Transcript)
32
Accessing Research Findings

• www.nifl.gov/lincs/
• The National Institute for Literacys Database

33
(No Transcript)
34
Accessing Research Findings

• www.cal.org/caela/
• The Center for Adult English Literacy Acquisition

35
(No Transcript)
36
Accessing Research Findings

• www.tesol.org/s_tesol/index.asp
• Teachers of English to Speakers of Other Languages

37
(No Transcript)
38
Evaluation

• What was good (useful, helpful) in this workshop?
• What suggestions do you have for how to improve
  the workshop?
• Thank you for participating!