Advances in the Dynamic Indicators of Basic Early Literacy Skills DIBELS

1
Advances in the Dynamic Indicators of Basic Early
Literacy Skills (DIBELS)
NASP Workshop Dallas, TX, March 31, 2004

• Roland H. Good IIIUniversity of Oregon
• Ruth A. Kaminski
• Pacific Institutes for Research

http//dibels.uoregon.edu
2
Advances in DIBELS Overview

• Introduction
• Core Components of Beginning Reading
• Review of DIBELS Measures
• Use of DIBELS within an Outcomes Driven Model
• Assessing individual students and making
  instructional recommendations
• Providing individual consultation to teachers
• Providing systems-wide consultation to schools
  and districts

3
Beginning Reading Core Components

• 1. Phonemic Awareness The ability to hear and
  manipulate sound in words.
• 2. Phonics The ability to associate sounds with
  letters and use these sounds to read words.
• 3. Fluency The effortless, automatic ability
  to read words in isolation (orthographic reading)
  and connected text.
• 4. Vocabulary Development The ability to
  understand (receptive) and use (expressive) words
  to acquire and convey meaning.
• 5. Reading Comprehension The complex cognitive
  process involving the intentional interaction
  between reader and text to extract meaning.

National Reading Panel. (2000). Teaching
children to read An evidence-based assessment of
the scientific research literature on reading and
its implications for reading instruction Reports
of the subgroups. Bethesda, MD National
Institute of Child Health and Human Development.
Available http//www.nationalreadingpanel.org/.
4
Reading FirstFour Kinds/Purposes of Reading
Assessment
An effective, comprehensive, reading program
includes reading assessments to accomplish four
purposes

• Outcome - Assessments that provide a bottom-line
  evaluation of the effectiveness of the reading
  program.
• Screening - Assessments that are administered to
  determine which children are at risk for reading
  difficulty and who will need additional
  intervention.
• Diagnosis - Assessments that help teachers plan
  instruction by providing in-depth information
  about students skills and instructional needs.
• Progress Monitoring - Assessments that determine
  if students are making adequate progress or need
  more intervention to achieve grade level reading
  outcomes.

Source Reading First Initiative Secretarys
Leadership Academy
5
Using an Outcomes Driven Model to Provide
Decision Rules for Progress Monitoring

• Outcomes Driven model Decision making steps
• 1. Identifying Need for Support
• 2. Validating Need for Instructional Support
• 3. Planning and Implementing Instructional
  Support
• 4. Evaluating and Modifying Instructional Support
• 5. Reviewing Outcomes for Individuals and Systems

Good, R. H., Gruba, J., Kaminski, R. A.
(2002). Best Practices in Using Dynamic
Indicators of Basic Early Literacy Skills
(DIBELS) in an Outcomes-Driven Model. In A.
Thomas J. Grimes (Eds.), Best Practices in
School Psychology IV (pp. 679-700). Washington,
DC National Association of School Psychologists.
6
Using the Outcomes Driven Model
3 times per year progress monitoring - Low
RiskFrequent progress monitoring - At Risk
7
Progress Monitoring Model for Beginning Reading
Core Areas
ISF
3 times per year progress monitoring - Low
RiskFrequent progress monitoring - At Risk
Good, R. H., Simmons, D. C., Kame'enui, E. J.
(2001). The importance and decision-making
utility of a continuum of fluency-based
indicators of foundational reading skills for
third-grade high-stakes outcomes. Scientific
Studies of Reading, 5, 257-288.
8
Progress Monitoring

• Repeated, formative assessment to evaluate
  progress toward important goals for the purpose
  of modifying instruction or intervention.
• Frequency of Progress Monitoring
• 3 times per year for students at low risk (All
  Students)
• Benchmark
• 1 per month for students with some risk
• Strategic
• 1 per week for students at risk
• Intensive

9
Research on Progress Monitoring

• Progress monitoring has been extensively
  researched in Special Education
• For example Fuchs, L. S., Fuchs, D. (1986).
  Effects of systematic formative evaluation A
  meta-analysis. Exceptional Children, 53, 199-208.
• With Reading First, progress monitoring is not
  just for special education any more.

10
Effects of Progress Monitoring

• Fuchs and Fuchs (1986) found the average effect
  size associated with progress monitoring was
• 0.70 for monitoring progress
• 0.80 when graphing of progress was added
• 0.90 when decision rules were added
• A student at the 50th percentile would be
  expected to move to the 82nd percentile (i.e., a
  score of 100 would move to a score of 114)
• Perhaps more important, a student at the 6th
  percentile would be expected to move to the
  average range (25th percentile)(i.e., a score of
  76 would move to a score of 90)

11
Progress Monitoring Tools

• Meaningful and important goals, waypoints, or
  benchmarks representing reading health or
  wellness.
• Meaningful and Important
• Public and Measurable
• Ambitious
• Brief, repeatable, formative assessment of
  progress toward benchmark goals that is sensitive
  to intervention.
• Brief and Efficient
• Repeatable - weekly or monthly
• Reliable and Valid indication of risk and growth

12
Secretarys Leadership AcademyAssessment
Committee
Team Leader Edward J. Kameenui, University of
Oregon

• David Francis, University of Houston
• Lynn Fuchs, Vanderbilt University
• Roland Good, University of Oregon
• Rollanda OConnor, University of Pittsburgh
• Deborah Simmons, University of Oregon
• Gerald Tindal, University of Oregon
• Joseph Torgesen, Florida State University

Kameenui, E. J., Francis, D., Fuchs, L. Good, R.
OConnor, R. Simmons, D., Tindal, G., Torgesen,
J. (2002). Secretarys Leadership Academy,
Reading First Initiative, Assessment Committee
Presentation. US Dept. of Education Washington,
DC.
13
idea.uoregon.edu/assessment
14
(No Transcript)
15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
Progress Monitoring Model for Beginning Reading
Core Areas
ISF
3 times per year progress monitoring - Low
RiskFrequent progress monitoring - At Risk
Good, R. H., Simmons, D. C., Kame'enui, E. J.
(2001). The importance and decision-making
utility of a continuum of fluency-based
indicators of foundational reading skills for
third-grade high-stakes outcomes. Scientific
Studies of Reading, 5, 257-288.
19
DIBELS Assess the Big Ideas
20
DIBELS Initial Sound FluencyDynamic Indicators
of Basic Early Literacy Skills 6th
Ed.Directions for Administration and Scoring

• Initial Sound Fluency is intended for most
  children from the last year of preschool through
  the middle of kindergarten. It may be
  appropriate for monitoring the progress of older
  children with very low skills in phonological
  awareness.
• The benchmark goal is 25 to 35 in the middle of
  kindergarten. Below 10 in the middle of
  kindergarten is indicates need for intensive
  instructional support.

BIG IDEA of Early Reading Phonemic Awareness
21
DIBELS Initial Sound Fluency

• This is a mouse, flowers, pillow, letters (point
  to each picture while saying its name).
• Mouse begins with the sound /m/ (point to the
  mouse). Listen /m/, mouse. Which one begins
  with the sounds /fl/?

22
Phoneme Segmentation FluencyDynamic Indicators
of Basic Early Literacy Skills 6th
Ed.Directions for Administration and Scoring

• Phoneme Segmentation Fluency works well for most
  children from winter of kindergarten through
  spring of first grade. It may be appropriate for
  monitoring the progress of older children with
  low phonological awareness skills.
• The benchmark goal is 35 to 45 correct phonemes
  per minute in the spring of kindergarten and fall
  of first grade. Students scoring below 10 in the
  spring of kindergarten and fall of first grade
  may need intensive instructional support to
  achieve benchmark goals.

BIG IDEA of Early Reading Phonemic Awareness
23
DIBELS Phoneme Segmentation Fluency
I am going to say a word. After I say it, you
tell me all the sounds in the word. So, if I
say, sam, you would say /s/ /a/ /m/. Lets try
one. (one second pause). Tell me the sounds in
mop Ok. Here is your first word.
24
DIBELS Nonsense Word FluencyDynamic Indicators
of Basic Early Literacy Skills 6th
Ed.Directions for Administration and Scoring

• Nonsense Word Fluency is intended for most
  children from spring of kindergarten through
  spring of first grade. It may be appropriate for
  monitoring the progress of older children with
  low skills in alphabetic principle.
• The benchmark goal for Nonsense Word Fluency is
  50 correct letter sounds per minute by mid first
  grade. Students scoring below 30 in mid first
  grade may need intensive instructional support to
  achieve first grade reading goals.

BIG IDEA of Early Reading Alphabetic Principle
25
DIBELS Nonsense Word Fluency

• Here are some more make-believe words (point to
  the student probe). Start here (point to the
  first word) and go across the page (point across
  the page). When I say, begin, read the words
  the best you can. Point to each letter and tell
  me the sound or read the whole word. Read the
  words the best you can. Put your finger on the
  first word. Ready, begin.

26
DIBELS Oral Reading FluencyDynamic Indicators
of Basic Early Literacy Skills 6th
Ed.Directions for Administration and Scoring

• DIBELS Oral Reading Fluency is intended for most
  children from mid first grade through third
  grade. The benchmark goals are 40 in spring of
  kindergarten, 90 in spring of second grade, and
  110 in the spring of third grade. Students may
  need intensive instructional support if they
  score below 10 in spring of first grade, 50 in
  spring of second grade, and below 70 in spring of
  third grade.

BIG IDEA of Early Reading Accuracy and Fluency
with Connected Text
27
DIBELS Oral Reading Fluency

• Please read this (point) out loud. If you get
  stuck, I will tell you the word so you can keep
  reading. When I say, stop I may ask you to
  tell me about what you read, so do your best
  reading. Start here (point to the first word of
  the passage). Begin.

28
DIBELS Retell FluencyDynamic Indicators of
Basic Early Literacy Skills 6th Ed.Directions
for Administration and Scoring

• DIBELS Retell Fluency is intended for most
  children from mid first grade through third grade
  who are reading at least 40 words per minute. It
  has been developed to provide a comprehension
  check for the DORF Assessment.

BIG IDEA of Early Reading Comprehension
29
DIBELS Retell Fluency

• Please tell me all about what you just read.
  Try to tell me everything you can. Begin. Start
  your stopwatch after you say begin.

30
Validity of ORF with RTF for Reading Comprehension

• Desirable standards r .60 to .80
• First grade ORF with consistent retell
  correlates with Woodcock Johnson Broad Reading
  Cluster r .81 (average of 2 probes)
• But, ORF with inconsistent retell correlates r
  .42
• Third grade A single probe ORF and RTF
  correlates with Oregon State Assessment --
  Reading and Literature Subtest r .73
• RTF by itself generally correlates in the .20s,
  .30s, .40s and .50s with a variety of measures of
  comprehension.

31
Inconsistent Retell in First Grade

• Students reading more than 40 words correct per
  minute, typical retell is about 50 of ORF score.
  
• Consistent retell is greater than or equal to 25
  of ORF score.
• An inconsistent retell is less than 25 of the
  ORF score.

32
ORF with inconsistent Retell has lower validity
with Reading Outcomes

• ORF with consistent retell r .81
• ORF with inconsistent retell r .42

33
Inconsistent Retell in Third Grade

• Students reading more than 40 words correct per
  minute, typical retell is about 50 of ORF score.
  
• Consistent retell is greater than or equal to 25
  of ORF score.
• An inconsistent retell is less than 25 of the
  ORF score.

34
DIBELS Word Use FluencyDynamic Indicators of
Basic Early Literacy Skills 6th Ed.Directions
for Administration and Scoring
Word Use Fluency
Word Use Fluency (WUF) is intended for most
children from fall of kindergarten through third
grade. A benchmark goal is not provided for WUF
because additional research is needed to
establish its linkage to other big ideas of early
literacy. Tentatively, students in the lowest 20
percent of a school district using local norms
should be considered at risk for poor language
and reading outcomes and those between the 20th
and 40th percentile should be considered at some
risk.
BIG IDEA of Early Reading Vocabulary and Oral
Language
35
DIBELS Word Use Fluency

• Format Examiner orally presents word and asks
  child to tell a sentence using the word.
• Listen to me use this word in a sentence. Jump.
  I like to jump rope. Your turn to use a word in
  a sentence. Pool.

36
Word Use Fluency
Bottom Sit on your bottom (4) Anyone Anyone
can go to my party (6) Dress Dress yourself.
Im not gonna dress you.(7) Middle Careful
thats middle (0) Total 40
Street Dont go in the street (5) Today Today
have a fun day (5) Against Youre against me
(3) Snow I like to play in the snow (7) Bats
Bats are scary (3)
37
WUF Distributions
38
Statewide WUF DistributionsKindergarten
02-03/03-04
39
Statewide WUF DistributionsFirst Grade
02-03/03-04
40
Statewide WUF DistributionsSecond Grade
02-03/03-04
41
Statewide WUF DistributionsThird Grade
02-03/03-04
42
Word Use FluencySummary

• 1-month Alternate forms reliability .59 - .65
• 1-week Alternate forms reliability .65 -.71
  (4-5 probes for r .90)
• Criterion-Related Validity
• PPVT .31 - .55
• TOLD .44 - .55
• EVT .22 - .57
• WJ-LC .36 - .47
• WRMT Reading Comprehension .28 - .41
• Language Sample DWR .44 - .72
• Sensitive to growth over time in K-1 (mean slope
  of 3.12 words per minute per month)
• Easy and practical to administer
• No Benchmark goals established -- Use local norms

43
DIBELS Letter Naming FluencyDynamic Indicators
of Basic Early Literacy Skills 6th
Ed.Directions for Administration and Scoring

• Letter Naming Fluency works well for most
  children from fall of kindergarten through fall
  of first grade.
• Students are considered at risk for difficulty
  achieving early literacy benchmark goals if they
  perform in the lowest 20 of students in their
  district. That is, below the 20th percentile
  using local district norms. Students are
  considered at some risk if they perform between
  the 20th and 40th percentile using local norms.
  Students are considered at low risk if they
  perform above the 40th percentile using local
  norms.

BIG IDEA of Early Reading NONE indicator of
risk
44
LNF Probes

• Each probe is a random sort of 2 lower case and 2
  upper case alphabets.
• Lines help students to keep their place.
• Serial naming and fluency aspects of the task are
  important.

45
Using DIBELS Within an Outcomes Driven Model to
Provide Decision Rules for Progress Monitoring

• Outcomes Driven model Decision making steps
• 1. Identifying Need for Support
• 2. Validating Need for Instructional Support
• 3. Planning and Implementing Instructional
  Support
• 4. Evaluating and Modifying Instructional Support
• 5. Reviewing Outcomes for Individuals and Systems

Good, R. H., Gruba, J., Kaminski, R. A.
(2002). Best Practices in Using Dynamic
Indicators of Basic Early Literacy Skills
(DIBELS) in an Outcomes-Driven Model. In A.
Thomas J. Grimes (Eds.), Best Practices in
School Psychology IV (pp. 679-700). Washington,
DC National Association of School Psychologists.
46
Three Levels of Assessment

• Benchmark Assessment
• Assess all children 3 - 4 times/year (e.g., Fall,
  Winter, Spring)
• How is the program (e.g.,classroom, school,
  curriculum, instruction) doing overall?
• Are there children who may need additional
  support to achieve outcomes?
• Which children may need additional support to
  achieve outcomes?
• Strategic Monitoring
• Assess at risk children more frequently (e.g.,
  monthly)
• Is current program sufficient to keep progress on
  track or are additional supports/intervention
  needed?
• Continuous or Intensive Care Monitoring
• Assess students needing more intensive, effective
  intervention weekly
• Are instructional supports/strategies effective
  or is a change in intervention needed?

47
Benchmark Assessment - First Grade

• Benchmark assessment screening all children to
  identify need for support to achieve goals in
  Core Components of literacy phonemic awareness,
  alphabetic principle, accuracy and fluency with
  connected text, Vocabulary, and Reading
  Comprehension for all children.
• Beginning September, October, or November
• Middle December, January, or February
• End March, April, May, or June

48
1. Identifying Need for Support

• Key Decision for Screening Assessment
• Which children may need additional instructional
  support to attain important reading outcomes?
• Data used to inform the decision
• Compare individual students performance to
  benchmark goals or local normative context to
  evaluate need for additional instructional
  support.
• Benchmark Goals A deficit in a foundation skill
  is a strong indicator that instructional support
  will be needed to attain later benchmark goals.
• Local normative context First, choose a
  percentile cutoff. 20th percentile seems a good
  place to start, but a district could choose 15th
  percentile or 25th percentile or other cutoff
  depending on resources.

49
http//DIBELS.uoregon.edu

50
Beginning of Kindergarten
51
Identify Students who Need Support to Reach NEXT
Benchmark Goal

• In September of Kindergarten, Melissa has a
  deficit on initial sounds. She may need
  additional instructional support to achieve
  kindergarten benchmark goals in Phonemic
  Awareness.
• Tevin is on track with to achieve Phonemic
  Awareness goals with effective core curriculum
  and instruction.

52
Longitudinal Outcomes for DIBELS Benchmark
Assessment

• Odds of achieving subsequent early literacy goals
  for DIBELS Benchmark Assessments at the
  beginning, middle, and end of kindergarten,
  first, second, and third grades (12 screening
  points across K - 3) are available at
• dibels.uoregon.edu/techreports/decision_rule_summa
  ry.pdf
• Students are at risk if the odds are against
  achieving subsequent early literacy goals.
• The purpose of screening is to provide additional
  instructional support -- strategic or intensive
  -- sufficient to thwart the prediction of
  difficulty achieving reading outcomes.

53
Sample Odds of Achieving Early Literacy Goals for
Different Patterns of DIBELS Performance
dibels.uoregon.edu/techreports/decision_rule_summa
ry.pdf
54
Instructional Recommendations for Individual
Patterns of Performance on Middle of Kindergarten
DIBELS Benchmark Assessment
Table Continues
dibels.uoregon.edu/techreports/decision_rule_summa
ry.pdf
55
Instructional Recommendations for Individual
Patterns of Performance on Middle of Kindergarten
DIBELS Benchmark Assessment (continued)
Table Continues
Odds of achieving specific early literacy goal.
For example, 69 of students with Established,
Some Risk, Low Risk pattern in the middle of
kindergarten achieved the end of first grade
DIBELS Oral Reading Fluency goal of 40 or more
words read correct per minute.
dibels.uoregon.edu/techreports/decision_rule_summa
ry.pdf
56
Instructional Recommendations for Individual
Patterns of Performance on Middle of Kindergarten
DIBELS Benchmark Assessment (continued)
Incidence or how often a pattern of performance
occurs. For example, among students with a
Deficit on ISF and Some Risk on LNF, achieving in
the At Risk range on PSF would be a more common
pattern, but achieving in the Some Risk range
would be an unusual pattern.
Table Continues
Extremely rare patterns may indicate a need to
retest. For example, it would be extremely rare
for a student to have Established ISF, Low Risk
on LNF, and At Risk status on PSF. Their PSF
score may not be accurately estimating their
phonemic awareness skill.
dibels.uoregon.edu/techreports/decision_rule_summa
ry.pdf
57
Instructional Recommendations for Individual
Patterns of Performance on Middle of Kindergarten
DIBELS Benchmark Assessment (continued)
Table Continues
Instructional Support Recommendation. For
students with odds in favor of achieving
subsequent literacy goals, benchmark instruction
is recommended. For students with odds against
achieving subsequent literacy goals, intensive
support is recommended. For about 50 50 odds,
strategic support is recommended.
dibels.uoregon.edu/techreports/decision_rule_summa
ry.pdf
58
Decision Utility of DIBELS

• Pattern of performance on DIBELS measures
  determines overall risk status and instructional
  recommendation. In fall of first grade, for
  example,
• LNF gt 37, DIBELS PSF gt 35, DIBELS NWF gt
  24Instructional Recommendation Benchmark - At
  grade level. Effective core curriculum and
  instruction recommended,
• Odds of reading 40 or more words correct per
  minute at the end of first grade 84
• Odds of reading less than 20 words correct per
  minute at the end of first grade 2
• LNF lt 25, DIBELS PSF lt 10, DIBELS NWF lt 13
  Instructional Rec Intensive - Needs substantial
  intervention
• Odds of reading 40 or more words correct per
  minute at the end of first grade 18 (unless
  given intensive intervention)
• Odds of reading less than 20 words correct per
  minute at the end of first grade 48 (unless
  given intensive intervention)
• Value of knowing the instructional recommendation
  and the goal early enough to change the outcome
  Priceless.

59
Sample Cutoffs for Low Risk, Some Risk, At Risk
for Kinder DIBELS Performance
60
Middle of Kindergarten
61
End of Kindergarten
62
Identify Need for Support Using Local Norms
X
X
X
63
2. Validate Need for Support

• Key Decision
• Are we reasonably confident the student needs
  instructional support?
• Rule out easy reasons for poor performanceBad
  day, confused on directions or task, ill, shy, or
  similar.
• More reliable information is needed to validate
  need for support than for screening decisions.
• Data used to inform the decision
• Repeated assessments on different days under
  different conditions
• Compare individual students performance to local
  normative context or expected performance to
  evaluate discrepancy.

64
Identify Need Which children may need
additional support?Teachers list of children in
class and DIBELS scores January of K
Initial Sound Fluency
At risk At risk Some risk Some risk Some risk Low
risk Low risk Low risk

• In January of Kindergarten
• Sandra, Matrix, Brandon, and Danielle have a
  deficit on Initial Sound Fluency. They may need
  additional instructional support to attain
  kindergarten benchmarks.
• Joseph and Tiffany are on track with established
  skills on ISF.
• Halley and Latisha have emerging skills and
  should be monitored strategically

65
Aggregating multiple, brief assessments increases
reliability

• When brief, 1-minute probes are used, it is
  important to consider error as one possible cause
  of poor performance. A pattern of low
  performance across 3 - 4 probes is much more
  reliable.

66
Validating Need for Support

• Verify need for instructional support by
  retesting with alternate forms until we are
  reasonably confident.

Phoneme Segmentation Fluency
Mid-year cutoff at risk
67
3. Planning and Implementing Instructional Support

• Key Decisions for Diagnostic Assessment
• What are the Goals of instruction?
• Where are we? Where do we need to be? By when?
  What course do we need to follow to get there?
• What skills should we teach to get there?
• Focus on the beginning reading core areas
  Phonological Awareness, Alphabetic Principle,
  Accuracy and Fluency with Connected Text
• Specific skills based on error analysis or
  additional diagnostic assessment (e.g., CTOPP).
• What kind of instructional support is needed?
• Intensive Instructional Support
• Strategic Instructional Support
• Benchmark Instruction

68
Exploring Support - Aimline for Brandon

• The aimline connects where we are to where we
  need to be by when, and shows the course to
  follow to get there.

End-year Benchmark Goal
Phoneme Segmentation Fluency
Aimline
End-year cutoff at risk
69
Planning Support - Aimline for Sandra

• The aimline connects where we are to where we
  need to be by when, and shows the course to
  follow to get there.

End-year Benchmark Goal
Phoneme Segmentation Fluency
Aimline
End-year cutoff at risk
70
Instructional Goals for Core Components of
Beginning Reading

• Benchmark Goals to be On Grade Level
• Step 1 Phonological Awareness with 25 - 35 on
  DIBELS Initial Sound Fluency by mid kindergarten
  (and 18 on PSF)
• Step 2 Phonemic Awareness with 35 - 45 on DIBELS
  Phoneme Segmentation Fluency by end of
  kindergarten (and 25 on NWF)
• Step 3 Alphabetic principle 50 - 60 on DIBELS
  Nonsense Word Fluency by mid first grade (and 20
  on DORF)
• Step 4 Fluency with 40 - 50 on DIBELS Oral
  reading fluency by end of first grade.
• Step 5 Fluency with 90 on DIBELS Oral reading
  fluency by end of second grade
• Step 6 Fluency with 110 on DIBELS Oral reading
  fluency by end of third grade

71
Instructional Steps from Kindergarten to
Successful Reading Outcomes
The outcome of each step depends on (a) students
beginning skills, (b) effectiveness of core
curriculum and instruction, and (c) effectiveness
of system of additional instructional support.
72
Planning Support What skills should we teach?

• Focus on the Big Ideas
• Initial Sounds Fluency and Phoneme Segmentation
  Fluency - Phonemic Awareness
• Nonsense Word Fluency - Alphabetic Principle
• Oral Reading Fluency - Accuracy and Fluency with
  Connected Text
• Retell Fluency - Comprehension
• Word Use Fluency - Vocabulary

73
What specific skills to teach?

• For specific skill level use
• Error analysis of DIBELS performance
• Knowledge of child performance in class
• Curriculum-linked assessment, e.g., mastery
  measures
• Use supplementary assessment as needed
• What can the child do/not do?

74
Phonemic Awareness

• Is the child accurate but not fluent? Build
  fluency.
• Does the child have errors? What is the error
  rate?
• Few (5-10), some (10-33), many (33-90), all?
• What is the pattern of errors?

75
ISF Response Patterns

• Produces multiple random errors
• Substitutes name of letter for initial sound
• Repeats word when prompted for sound
• Recognizes but does not produce initial sounds
  consistently
• Recognizes and produces initial sounds
  confidently
• Difficulty with consonant sounds
• Difficulty with vowel sounds
• Pronunciation differences due to dialect or
  second language
• Frequent articulation difficulties, consider
  referral to language specialist
• Difficulty remembering picture names
• Frequent self corrections

76
PSF Response Patterns

• Stage 1 repeats entire word
• Stage 2 produces initial sound or sounds only
• Stage 3 produces onset and rhyme
• Stage 4 produces initial and final sounds
  correctly errors on middle sounds
• Stage 5 produces initial, middle and final
  sounds correctly does not segment blends
• Stage 6 correctly segments all phonemes
  including phonemes in blends
• Produces consonant sounds correctly misses vowel
  sounds
• Omits final sounds
• Pronunciation differences due to dialect or
  second language
• Frequent articulation difficulties, consider
  referral to language specialist
• Frequent phoneme additions
• Frequent phoneme omissions
• Frequent self corrections

77
Sequence of Phonological Awareness Skills - K

• 1. Sound and Word Discrimination
• Tells whether words or sounds are the same or
  different (cat/cat same cat/cardifferent).
• Identifies which word is different (e.g., sun,
  fun, sun).
• Tells the difference between single speech
  sounds (e.g., Which one is different? s, s, k).
• 2. Rhyming
• Identifies whether words rhyme (e.g., cat/mat
  ring/sing).
• Produces a word that rhymes with another (e.g.,
  "A word that rhymes with rose is nose. Tell me
  another word that rhymes with rose.)
• 3. Blending
• Orally blends syllables (mon-key) or
  onset-rimes (m-ilk) into a whole word.
• Orally blends 2-3 separately spoken phonemes
  into one-syllable words (e.g., m-e me u-p up
  f-u-n fun).
• 4. Segmentation
• Claps or counts the words in a 3-5 word
  sentence (e.g., Sue can jump far).
• Claps or counts the syllables in 1-, 2-, and
  3-syllable words.
• Says each syllable in 2- and 3-syllable words
  (di-no-saur).
• Identifies the first sound in a one-syllable
  word (e.g., /m/ in man).
• Segments individual sounds in 2- and 3-phoneme,
  one-syllable words (e.g., run /r/ /u/ /n/ feet
  /f/ /ee/ /t/).

78
Alphabetic Principle

• Nonsense Word Fluency
• Is the child accurate but not fluent? Build
  fluency.
• Does the child have errors? What is the error
  rate?
• Does the child have errors? What is the pattern
  of errors?

79
NWF Response Patterns

• Stage 1 Has isolated letter-sound
  correspondences but lacks a systematic strategy
  for attacking unknown words.
• Stage 2 Produces correct consonant sounds
  incorrect vowel sounds.
• Stage 3 Produces most sounds correctly
  sound-by-sound, but does not recode into complete
  word.
• Stage 4 Produces sounds correctly sound-by-sound
  and then recodes into complete word (e.g., /m/
  /o/ /t/ mot)
• Stage 5 Fluently applies systematic trategy for
  attacking unknown words (i.e., reads mot as
  mot)
• Substitutes real words for nonsense words
• Produces sounds correctly sound-by-sound recodes
  sounds out of order (e.g.,/bis/ sib)
• Consistent error for a specific consonant/vowel
  sound requiring review
• Pronunciation differences due to dialect or
  second language
• Frequent articulation difficulties, consider
  referral to language specialist
• Frequent sound additions
• Frequent sound omissions
• Frequent self corrections

80
Critical Alphabetic Principle Skills

• Letter-Sound Correspondences
• Example
• (Teacher points to letter m on board). "The sound
  of this letter is /mmmmm/. Tell me the sound of
  this letter.
• Sounding Out Words
• Example (Teacher points to the word map on the
  board, touches under each sound as the students
  sound it out, and slashes finger under the word
  as students say it fast.) "Sound it out."
  (/mmmmmmmmaaaaaaap/) "Say it fast." (map)
• Reading Words
• Reading Words in connected text

81
Accuracy and Fluency with Connected Text

• Oral Reading Fluency
• Is the child accurate but not fluent? Build
  fluency.
• Does the child have errors? What is the pattern
  of errors?
• Correctly decodes easy, phonetically correct
  words, misses long and/or irregular words
• Consistently makes errors on words with specific
  blends, digraphs, etc.
• Only reads simple and common words correctly
  consistently (e.g., the and)

82
Vocabulary

• WUF Response Patterns
• Stereotypical response pattern, e.g., I like to
  ____
• Word use is sparse and employs minimum utterances
• Word use is fluent and confident employing
  elaborated sentences
• Response often unrelated to target word
• Student appears shy and reticent to talk
• Student uses similar sounding word, may have
  difficulty hearing target word
• Student frequently asks for the word to be
  repeated, may have difficulty hearing target word

83
What Curriculum and/or program?Good News - Bad
News - Good News

• Good News All but small number of children can
  learn to read.
• Bad News No Magical Curriculum or program that
  is effective for all students.
• Good News The Magic is in the system of
  support that matches each child with the support
  that is effective for her/him.

84
Planning Support What curriculum/program to use?

• Three levels of instructional support
• Benchmark -- Core Curriculum
• Strategic -- Supplemental Curriculum
• Intensive -- Intervention Curriculum

85
Benchmark Instruction - Core Comprehensive
Reading Programs

• Purpose to provide complete instruction in the
  core components of reading
• Examples
• Open Court Reading, SRA/McGraw Hill
• Houghton Mifflin
• Reading Mastery

86
Strategic Support - Supplemental Reading Programs

• Purpose to provide additional instruction in one
  or more areas of reading for students who require
  strategic instructional support to reach
  benchmark goals.
• Examples
• phonemic awareness programs
• Phonemic Awareness in Young Children A Classroom
  Curriculum, Paul H. Brookes Publishing Co., Inc.
• fluency building programs
• Read Naturally, Read Naturally, Inc.
• Read Well, Sopris West
• comprehension strategy programs
• Soar to Success, Houghton Mifflin Co.
• Collaborative Strategic Reading

87
Intensive Support - Intervention Reading Programs

• Purpose to provide additional instruction to
  students who have skill deficits and need
  intensive support to reach benchmark goals.
• Examples
• Corrective Reading, SRA/McGraw-Hill
• Scott Foresman Early Reading Intervention
  (Optimize), Scott Foresman
• Phonological Awareness Training for Reading, AGS
  Publishing

88
Instructional Strategies

• Grouping
• Small group instruction
• Flexible instructional grouping
• Effective Instruction
• Focused and systematic
• Explicit
• Direct explanation
• Modeling
• High student engagement
• Lots of opportunities to respond
• Lots of Guided Practice with immediate feedback
• Scaffolding to support learning
• Integration of skills
• Review

89
4. Evaluating and Modifying Instructional Support

• Key Decision for Progress Monitoring Assessment
• Is the intervention effective in improving the
  childs early literacy skills?
• How much instructional support is needed?
• Enough to get the child on trajectory for
  Benchmark Goal.
• When is increased support needed?
• Monitor childs progress during intervention by
  comparing their performance and progress to past
  performance and their aimline. Three consecutive
  assessments below the aimline indicates a need to
  increase instructional support.

90
Evaluating Support Sandra Is the intervention
working?
Phoneme Segmentation Fluency
Aimline
91
Evaluating Support Brandon Is the intervention
working?

• Whoops! Time to make a change!

Phoneme Segmentation Fluency
Aimline
92
Evaluating Support - BrandonIs Instructional
Support Sufficient Now?
Aimline
Phoneme Segmentation Fluency
93
Where are we?
What is our goal?
What course should we follow?
How are we doing?
94
Progress Monitoring The Teachers MapThe GPS
for Educators
Aimline
95
Dynamic Interventions Build in anAssessment ??
InterventionFeedback Loop

• Good interventions are identified by their
  outcomes - not our philosophy, or beliefs, or the
  quality of their packaging.
• Good interventions are individual an effective
  intervention for one child may not be effective
  for another.
• Integrating assessment and intervention driven by
  outcomes is a key aspect of an effective
  intervention.

96
Step 1 Initial Sound Fluency in First Half of
Kindergarten

• Mid year goal 25 on ISF
• Beginning K
• Low risk gt 8
• At risk lt 4
• Middle K
• Low risk gt 25
• At risk lt 10
• Additional Goal PSF gt 18

97
Step 2 Phoneme Segmentation Fluency in Second
Half of Kinder

• End K goal 35 on PSF
• Middle K
• Low risk gt 18
• At risk lt 7
• End K PSF
• Established PSF gt 35
• Deficit lt 10
• Additional Goal NWF gt 25

98
Step 3 Nonsense Word Fluency in First Half of
First Grade

• Middle first goal 50 on NWF
• Beginning first
• Low risk gt 24
• At risk lt 13
• Mid first NWF
• Established NWF gt 50
• Deficit lt 30
• Additional Goal ORF gt 20

99
Step 4 Oral Reading Fluency in Second Half of
First Grade

• End first goal 40 on ORF
• Middle first ORF
• Low risk gt 20
• At risk lt 8
• End first ORF
• Low risk gt 40
• At risk lt 20
• Additional Goal Retell gt ORF/4

100
Step 5 Oral Reading Fluency in Second Grade

• End second goal 90 on ORF
• Beg second ORF
• Low risk gt 44
• At risk lt 26
• End second ORF
• Low Risk gt 90
• At Risk lt 70
• Additional Goal Retell gt ORF/4

101
Step 6 Oral Reading Fluency in Third Grade

• End third goal 110 on ORF
• Beg third ORF
• Low risk gt 77
• At risk lt 53
• End third ORF
• Low Risk gt 110
• At Risk lt 80
• Additional Goal Retell gt ORF/4

102
Is Progress is Related to Outcomes?

• The logic of the Evaluating and Modifying Support
  step relies on evidence that amount of progress
  toward goals is related to important reading
  outcomes.
• Is slope of progress on NWF in the Fall of first
  grade related to first grade reading outcomes?
  This questions was recently examined by
• Fuchs, L. S., Fuchs, D., and Compton, D. L. (in
  press). Monitoring early reading development in
  first grade Word Identification Fluency versus
  Nonsense Word Fluency. Exceptional Children.
• Fuchs et al. also examined the validity of spring
  slope, whole year slope, and the validity of fall
  level, all of which will not be addressed here.

103
Validity of Slope on NWF in Fall of First Grade
for Oral Reading Fluency Outcomes

• Based on 151 at risk children, Fuchs et al.
  correlated slope of progress in fall of first
  grade with spring of first grade reading
  outcomesNote. WIF is Word
  Identification Fluency

104
Conclusions Validity of NWF Slope

• coefficients for the nonsense word fluency
  measure slopes were disappointingly low, ranging
  from -.04 to .16. Because nonsense word fluency
  is recommended for progress monitoring in the
  fall of first grade within the DIBELS system
  (Good et al., 2001), these findings raise serious
  concern. An increasing pattern of scores through
  the first semester of first grade on DIBELS
  nonsense word fluency appears to bear little
  relationship to students end-of-year reading
  status. (p. 21)
• practitioners can have confidence that increases
  in word identification fluency over time reflect
  improved performance on important end-of-year
  reading outcomes. As our results suggest, the
  same is not true for DIBELS nonsense word
  fluency, and findings are particularly compelling
  because data were collected on the same group of
  children using the same methods. (p. 23)

105
Concerns and Questions

• Before accepting these conclusions, some concerns
  should be addressed.
• 1. The simple correlation between slope and
  reading outcome addresses the wrong question.
• There is no rational or logical reason why slope
  by itself should be related to reading outcomes
  without considering the students initial skills.
• The crucial question is, Given the students
  initial skills, does slope of progress add to the
  variance explained in reading outcomes?
• Nick has NWF slope of 0.70 while Nora has NWF
  slope of 1.50. Who would you expect to have
  higher reading outcomes in the spring?

106
Answer It depends on initial skills.

• Nora has a slope twice that of Nick, but
  substantially lower reading outcome because her
  initial skills are so much lower.

Slope, by itself without considering initial
skills is not enough to predict outcomes.
107
Now Consider Nora and Nell

• Nora and Nell have similar initial skills
  Nells higher slope predicts higher skills in
  middle of first grade and higher reading outcomes.

Spring DORF 37
Spring DORF 27
108
Given Initial Skills, Does Slope Add to
Predictions of Outcomes?

• Students with complete data from 2002-2003 in the
  DIBELS Data System were examined for level of
  risk, slope of progress, and reading outcomes.

109
Fuchs et al. At Risk Sample

• Mean DIBELS NWF score is in the low risk range.
• An estimated 70 of the sample would be above the
  NWF cutoff of 23 for low risk.

110
Utility of Initial NWF Risk Categories

• Initial skills on NWF are a very strong predictor
  of reading outcomes.

0 -12
111
Does Slope Add to the Prediction of Reading
Outcomes After Risk Level and Initial Skills?

• Rules for evaluating effects
• Significance. With N gt 20,000 everything is
  significant.
• Percent of variance explained. More than 10 of
  variance explained is a good indication of a
  strong effect. Greater percent is stronger.
• Educationally meaningful effects. Analysis of
  outcomes to see if the predicted differences
  would be educationally important to teachers,
  students, parents.

112
Does Slope Add to the Prediction of Reading
Outcomes After Risk Level and Initial Skills?

• Sequential model predicting first grade DORF
  reading outcomes from (1) risk category, (2)
  initial NWF skill given risk, and (3) slope given
  risk and initial skill.

Risk category, initial skills, and slope combined
explain 59 of reading outcomes.
113
Variance Explained by Slope for Each Risk Category

• A separate analysis was conducted for each risk
  category.

Rate of progress is especially important for
students who are at risk for low reading
outcomes.
114
Variability in Slope for At Risk Students

• About 68 of students trajectories are between
  the low slope and the high slope.

115
Are Differences in Slope Educationally Meaningful
for At Risk Students?

• Yes. Predicted reading outcomes are substantially
  different.

116
Conclusions Validity of DIBELS NWF Slope

• Initial risk status and initial skills on DIBELS
  Nonsense Word Fluency are very important in
  predicting reading outcomes in first grade,
  explaining 48 of variance in outcomes.
• An increasing pattern of scores through the first
  semester of first grade on DIBELS Nonsense Word
  Fluency appears to be a very important predictor
  of reading outcomes for students who are at risk
  and for each risk category.
• We can be confident that increases in DIBELS
  Nonsense Word Fluency reflect improved
  performance on essential skills that contribute
  to important end-of-year reading outcomes.

117
5. Reviewing Outcomes

• Key Decisions for Outcome/Accountability
  Assessment
• Does the child have the early literacy skills
  predictive of successful reading outcomes?
• Does the school have core curriculum and
  instruction as well as a system of effective
  instructional support so their students achieve
  literacy outcomes?
• Data used to inform the decision
• Evaluate individual students performance with
  respect to benchmark goals that with the odds in
  favor of achieving subsequent literacy goals.
• Compare school/district outcomes to goals and
  outcomes from previous year.
• Evaluate core curriculum and system of additional
  support for each step to identify strengths and
  areas for improvement.

118
Reviewing Outcomes - School Level 1998 99 First
Grade Reading
28 Established Readers 57 Emerging Readers 15
Non-Readers
119
Reviewing Outcomes - School Level 1999 00 First
Grade Reading
57 Established Readers 36 Emerging Readers 6
Non-Readers
120
Heartland Early Literacy Project Across Year
First Grade Oral Reading Fluency Outcomes

• 2001-2002 Beginning Middle 4229 End 4414
• 2000-2001 Beginning Middle 4037 End 4152
• 1999-2000 Beginning Middle 1595 End 1879

121
Reviewing Outcomes Effectiveness of Benchmark
Instruction (Core Curriculum)

• For each step toward literacy outcomes, a school
  with an effective core curriculum and instruction
  supports students who are on track (i.e., low
  risk or benchmark) to achieve the goal.
• For students with the odds in favor of achieving
  literacy goals, it is the job of the core to
  teach the core components so that all students
  (100) achieve the goals.

122
Reviewing Outcomes Effectiveness of Strategic
and Intensive Intervention

• For each step toward literacy outcomes, a school
  with an effective system of effective
  interventions supports students who are not on
  track (i.e., at some risk or at risk of
  difficulty achieving literacy goals) to achieve
  the goal.
• For students with the odds against achieving
  literacy goals unless we provide an effective
  intervention, it is the job of the system of
  additional support to augment the core curriculum
  so that all students (100) achieve the same
  benchmark goals.

123
Instructional Steps from Kindergarten to
Successful Reading Outcomes
The outcome of each step depends on (a) students
beginning skills, (b) effectiveness of core
curriculum and instruction, and (c) effectiveness
of system of additional instructional support.
124
Step by Step, Core and Intervention
Effectiveness of Benchmark (core) for School A
AB
AS
Effectiveness of Strategic support for School A
AI
Effectiveness of Intensive support for School A
125
Evaluating the Effectiveness of the Core
Curriculum and Instruction

• 1. Is the core curriculum and instruction
  getting at least 95 of Benchmark students to the
  next early literacy goal?
• If children are on track, the core should keep
  them on track.
• What would it take to achieve 100?
• 2. Is the core curriculum and instruction as
  effective as other schools in getting Benchmark
  students to the goal?
• If typical schools are not getting 100 of
  Benchmark students to the goal, then
  supplementing the core in this area can improve
  reading outcomes.

126
(No Transcript)
127
Step 1 Beginning K to Middle K
Middle kindergarten outcomes for students with
benchmark, strategic, and intensive instructional
recommendations at the beginning of kindergarten
Intensive Median School
Note. Graph based on all schools participating in
the DIBELS Data System in the 2001 2002
academic year.
Benchmark Median School
AS
AI
AB
A typical (middle) school had 62 of children
with a beginning kindergarten benchmark
recommendation achieve the middle of kindergarten
goal, and 2 of children with intensive support
recommendation.
128
(No Transcript)
129
Step 2 Middle K to End K
End of kindergarten outcomes for students with
benchmark, strategic, and intensive instructional
recommendations in the middle of kindergarten
AB
AS
AI
Benchmark Median School
Intensive Median School
A typical (middle) school had 90 of children
with a middle kindergarten benchmark
recommendation achieve the end of kindergarten
goal, and 26 of children with intensive support
recommendation.
130
(No Transcript)
131
Step 3 Beginning First to Middle First
Middle of first grade outcomes for students with
benchmark, strategic, and intensive instructional
recommendations in the beginning of first grade
AI
Intensive Median School
Benchmark Median School
AB
AS
A typical (middle) school had 68 of children
with a beginning first grade benchmark
recommendation achieve the middle of first grade
goal, and 0 of children with intensive support
recommendation.
132
(No Transcript)
133
Step 4 Middle First to End First
End of first grade outcomes for students with
benchmark, strategic, and intensive instructional
recommendations in the middle of first grade
AB
Benchmark Median School
AI
Intensive Median School
AS
A typical (middle) school had 96 of children
with a middle first grade benchmark
recommendation achieve the end of first grade
goal, and 0 of children with intensive support
recommendation.
134
(No Transcript)
135
Step 5a Beginning Second to Middle Second
Middle of second grade outcomes for students with
benchmark, strategic, and intensive instructional
recommendations at the beginning of second grade
AI
Intensive Median School
Benchmark Median School
AB
AS
A typical (middle) school had 90 of children
with a beginning second grade benchmark
recommendation achieve the middle of second grade
goal, and 0 of children with intensive support
recommendation.
136
(No Transcript)
137
Step 5b Middle Second to End Second
End of second grade outcomes for students with
benchmark, strategic, and intensive instructional
recommendations at the middle of second grade
Intensive Median School
Benchmark Median School
AB
AI
AS
A typical (middle) school had 92 of children
with a middle second grade benchmark
recommendation achieve the end of second grade
goal, and 4 of children with intensive support
recommendation.
138
(No Transcript)
139
Step 6a Beginning Third to Middle Third
Middle of third grade outcomes for students with
benchmark, strategic, and intensive instructional
recommendations at the beginning of third grade
Intensive Median School
AB
Benchmark Median School
AS
AI
A typical (middle) school had 90 of children
with a beginning third grade benchmark
recommendation achieve the middle of third grade
goal, and 0 of children with intensive support
recommendation.
140
(No Transcript)
141
Step 6b Middle Third to End Third
End of third grade outcomes for students with
benchmark, strategic, and intensive instructional
recommendations at the middle of third grade
AI
Intensive Median School
Benchmark Median School
AS
AB
A typical (middle) school had 91 of children
with a middle third grade benchmark
recommendation achieve the end of third grade
goal, and 0 of children with intensive support
recommendation.
142
Step by Step, Core and Intervention
Support Effectiveness is less than a typical
school and less than goal.
Typical Effectiveness is less than goal, but
typical of other schools.
Strength Effectiveness is at goal or greater
than typical of other schools
143
Outcomes Driven ModelRepeat for Each Step
3 time per year progress monitoring - Low
RiskFrequent progress monitoring - At Risk
144
Instructional Steps from Kindergarten to
Successful Reading Outcomes
Step by step to important reading goals and
outcomes. Implicit in this logic is a linkage to
High Stakes Reading Outcomes.
145
Third Grade Oral Reading Fluency to Oregon
Statewide Assessment Test
Exceeds
r .73 53 of Variance
Meets
Does not meet Expectations

• Odds of meets expectation on OSAT given 3rd
  grade TORF of 110 90 of 91 or 99.
• Odds of meets expectation on OSAT given 3rd
  grade TORF below 70 4 of 23 or 17.

146
Linkage of Third-Grade TORF to Illinois State
Assessment Test (ISAT)
r .79 63 of Variance

• Odds of meets standards on ISAT given
  Third-Grade TORF of 110 or above 73 of 74 or
  99.
• Odds of meets standards on ISAT given
  Third-Grade TORF of 70 or below 1 of 8 or 12.

Sibley, D., Biwer, D., Hesch, A. (2001).
Unpublished Data. Arlington Heights, IL
Arlington Heights School District 25.
147
Above 110, the odds are strong the student will
rank proficient on the AK State Benchmark.
Proficient
Alaska State Benchmark in Reading
Below Proficient
Below 70, the odds are low the student will rank
proficient on the AK State Benchmark.
3rd Grade Benchmark in Reading - CBM
Linner, S. (2001, January). Curriculum Based
Assessment in reading used as a predictor for the
Alaska Benchmark Test. Paper presented at the
Alaska Special Education Conference, Anchorage,
AK.
148
Linkage of Oral Reading Fluency to State Reading
Outcome Assessments
Above 110, the odds are 91 the student will rank
adequate on the FL State Assessment.
Below 80, the odds are 19 the student will rank
adequate on the FL State Assessment.
Buck, J., Torgesen, J. (2003). The
relationship between performance on a measure of
oral reading fluency and performance on the
Florida Comprehensive Assessment Test (Technical
Report 1). Tallahassee, FL Florida Center for
Reading Research,.
149
Themes

• Dont loose track of the bottom line. Are we
  getting closer to important and meaningful
  outcomes?
• Monitor Progress on -- and teach -- what is
  important Phonemic Awareness, Alphabetic
  Principle, Accuracy and Fluency with Connected
  Text
• Oral Reading Fluency is an important
  instructional goal and target of progress
  monitoring.
• Use progress monitoring to make decisions that
  change outcomes for children.
• Progress monitoring should be efficient and
  purposeful.
• Start early! Trajectories of reading progress
  are very difficult to change.

Slide 150