EvidenceBased Practice: Yes, but with Cautions

1
Evidence-Based Practice Yes, but with Cautions
Recommendations

• Steve Graham
• Vanderbilt University

2

• I will use Writing, my area of expertise to
  illustrate the 5 points that I plan to make.

3
POINT 1

• The use of Evidence-Based practices is more
  trustworthy than the use of practices based on
  professional (expert) judgment or teacher
  experience

4
2 Sources for Effective Practice

• Professional Writers many draw on their
  experience to give advice on either how to teach
  writing or facilitate writing development
• This advice ranges from useful to simplistic to
  ridiculous

5

• Dont use words too big for the subject.
• C.S. Lewis
• When you catch an adjective kill it.
• Mark Twain

6

• Have something to say.
• Say it.
• Stop when you have said it.
• Give it an accurate title.

7
Rules for Writing a Paragraph

• Set type for as long as you can hold your breath
  without getting blue in the face
• Then put in a comma
• When you yawn put in a semicolon, and when you
  sneeze, thats time for a paragraph.

8
Sources for Effective Practice

• Experienced Writing Teachers
• This can involve the insights they have acquired
  from teaching or insights acquired from others
  who study them

9
LIMITATIONS

• Difficult to separate wheat from shaft
• Often no direct evidence on effectiveness
  (validity -- testimonials)
• Evidence is often selective
• (generalizability -- possible bias)
• Evidence often based on 1 or a few teachers
• (reliability cant predict its effectiveness)

10
Sources for Effective Practice

• Scientific Studies -- Should be more trustworthy
  than insight and experience
• Collect evidence
• Present findings for all participants
• Replicability
• Strength of Impact

11
Point 2

• Make the best of the available evidence. The
  data-base is currently very thin for some areas.

12

• I illustrate this with writing instruction for
  students with LD This is based on a recent
  chapter by myself, Natalie Olinghouse, and Karen
  Harris

13
FINDINGS 4 or more studies

• 1. Teach students with LD strategies for
  planning, revising, and editing their
  compositions (strong positive impact).
• 2. Use direct instruction to teach grammar skills
  to students with LD (moderate positive impact).

14
FINDINGS 2 to 4 studies

• 3. Have students with LD work cooperatively with
  other struggling writers to plan, draft, revise,
  and edit their compositions (strong positive
  impact).
• 4. Explicitly teach students with LD strategies
  for producing written summaries of reading
  material (strong positive impact).

15
FINDINGS 2 to 4 studies

• 5. Make the process writing approach more
  effective for students with LD by explicitly
  teaching them strategies for carrying out the
  processes of planning, revising, and editing
  (strong positive impact).
• 6. Set clear and specific goals for what students
  with LD are to accomplish in their writing
  (moderate to strong positive impact).

16

• This is IT!!!
• We can only make 6 recommendations on writing
  instruction for students with LD that have some
  scientific backing

17
POINT 3 The What Works Clearinghouse Syndrome

• In other words only certain types of studies
  will be included in determining what is or is not
  evidence-based
• They have adopted a best evidence synthesis
  approach

18

• As a result, they are often referred to as
• THE WHAT DOESNT WORK CLEARINGHOUSE

19
RECOMMENDATION

• We need to take a careful look at our definitions
  of evidence-based practice
• Are they too stringent are they too lax?
• More pointedly, we need to systematically study
  the process and criteria used to identify
  evidence-based practices.

20
POINT 4

• Until our data base is more sufficiently
  developed, we will need to supplement our
  evidence-based recommendations with
  recommendations taken from areas other than
  experimental and single-subject design research

21

• We will need to continue to draw on the practices
  of effective teachers.
• For example, we located 5 qualitative studies
  that looked at the writing instructional
  practices of effective teachers and schools. We
  identified those practices employed by a majority
  of teachers as promising

22
Some Examples

• Dedicate time to writing, including writing
  across the curriculum.
• Involve students in various forms of writing over
  time.
• Have students plan, draft, revise, edit, and
  share their work.
• Keep students engaged and on-task by involving
  them in thoughtful activities (such as planning a
  composition) versus activities that require
  little thoughtfulness (such as completing a
  workbook page that can be finished quickly).

23
Some More Examples

• Model, explain, and provide guided assistance
  when teaching.
• Provide just enough support so that students can
  make progress or carry out writing tasks and
  processes, but encourage students to act in a
  self-regulated fashion, doing as much as they can
  on their own.
• Be enthusiastic about writing and create a
  positive environment, where students are
  constantly encouraged to try hard, believe that
  what they are learning will permit them to write
  well, and attribute success to effort and what
  they learned.
• Set high expectations for students, encouraging
  them to surpass their previous efforts or
  accomplishments.

24
POINT 5

• Be Cautious About What Is Promised, as there is
  no guarantee that evidence-based practices travel
  easily or as expected

25
RTI Is A Case In Point

• Based in part on the premise that the
  implementation of effective practices
  (evidence-based) in the regular classroom will
  reduce the number of students who are
  misidentified as LD (in other words, eliminate
  educational casualties)

26

• While this may be the case, this is a huge burden
  for Evidence-Based practices, as we have some
  schools where the writing of three quarter or
  more of the students is not strong enough to meet
  classroom demands.
• In addition, much can go wrong in the
  implementation of any educational approach

27
Case in Point 2

• The National Reading Panel Recommendations
• MAY BE A CASE OF OVER-SELL THAT WILL COME BACK
  AND LITERALY BITE US IN THE BUTT.

28

• WE MUST BE REASONABLE AND CAUTIOUS IN HOW WE SELL
  EVIDENCE-BASED PRACTICES TO THE EDUCATIONAL
  COMMUNITY