Making Consistent Decisions About Accommodations for English Language Learners

1
Making Consistent Decisions About Accommodations
for English Language Learners
Research Summit
Texas Comprehensive Center _at_ SEDL Austin,
Texas March 1617, 2009
2
Language Factors in the Assessment of English
Language Learners
Dr. Jamal Abedi CRESST University of California,
Davis School of Education
3
How Important is Assessment for ELL Students?

• Assessment results affect ELL students in the
  following areas
• Classification
• Instruction
• Accountability (the NCLB issues)
• Promotion
• Graduation

4
Are the Standardized Achievement Tests
Appropriate for ELLs?
The Standards for Educational and Psychological
Testing (American Educational Research
Association, American Psychological Association,
National Council on Measurement in Education,
1999) elaborated on this issue For all test
takers, any test that employs language is, in
part, a measure of their language skills. This is
of particular concern for test takers whose first
language is not the language of the test.
5
Performance/Reliability-Gap Between ELL and
Non-ELL Students
6
Are the Standardized Achievement Tests Reliable
and Valid for These Students?

• The reliability coefficients of the test
  scores for ELL students are substantially
  lower than those for non-ELL students.
• ELL students test outcomes show lower
  criterion-related validity.
• Structural relationships between test
  components and across measurement domains
  are lower for ELL students.

7
Site 2 Stanford 9 Sub-scale Reliabilities
(Alpha), Grade 9
8
Principle of Equity and Fairness

• Therefore, the Principle of Equity and Fairness
  demands assistance to these students.

• Provide assistance in the form of accommodations.

9
Samples of Accommodations Used for ELL Students
That May Not Be Relevant

• Test-taker marks answers in a test booklet
• Copying assistance provided between drafts
• Test-taker indicates answers by pointing or other
  similar method
• Paper is secured to work area with tape/magnet
• Physical assistance is provided

10
Samples of Accommodations Used for ELL Students

• Enlarged answer sheets provided
• Breaks provided
• Tests individually administered
• Tests administered in small groups
• Tests administered in a location with minimal
  distractions

11
Presenting Language-Related Accommodations for
ELLs

• English Dictionary
• English Glossary
• Bilingual Dictionary/Glossary
• Customized Dictionary
• Native Language Testing
• Linguistically Modified Test

12
Clear Language of Instruction and Assessment
Works for ELLs, SWDs, and Everyone

• What is language modification of test items?

13
Language of Assessment

• Clear and concise language is a requirement for
  reliable and valid assessments for ELL students.
• It may also be an important consideration for
  students with learning disabilities.
• Students with learning disabilities may have
  difficulty processing complex language in the
  assessment.
• Reducing the unnecessary linguistic complexity
  of test items can help students with the most
  challenging academic career.

14
Linguistic Modification Concerns

• Familiarity/frequency of non-math vocabulary
• Unfamiliar or infrequent words changed
• A certain reference file gt Macks company

OriginalA certain reference file contains
approximately six billion facts. About how many
millions is that?A. 6,000,000B. 600,000C.
60,000D. 6,000E.
600 Modified Macks company sold six billion
pencils. About how many millions is that? 
A. 6,000,000 B. 600,000 C.
60,000 D. 6,000 E.
600
15
Linguistic Modification Concerns Cont.

• Length of nominals long nominals shortened
• Last years class vice president gt Vice
  president
• The pattern of puppys weight gain gt The pattern
  above
• Question phrases complex question phrases
  changed to simple question words
• At which of the following times gt When
• Which is best approximation of the number gt
  Approximately how many

16
Linguistic Modification cont.

• Voice of verb phrase passive verb forms
  changed to active
• The weights of 3 objects were compared gt Sandra
  compared the weights of 3 rabbits
• If a marble is taken from the bag gt If you take
  a marble from the bag
• The boy was bitten by the
  dog gt The dog bit the boy

• Conditional clauses conditionals either
  replaced with separate sentences or order of
  conditional and main clause changed
• If two batteries in the sample were found to be
  dead gt He found two broken pencils in the sample
• If Lee delivers x newspapers gt Lee
  delivers x newspapers

17
CRESST Studies on the Assessment and
Accommodation of ELL Students

• Impact of Language Factors On Assessment of ELLs
• A Chain of Events
• Fourteen studies on the assessment and three on
  the instruction (OTL) of ELL students

18
Examining Complex Linguistic Features in
Content-Based Test Items
19
Additional Complex Linguistic Features

• More recent research has identified these
  additional features
• Complex verbs
• Subordinate clauses (including relative clauses)
• Complex noun phrases
• Various entities as subjects

20
Studies on Linguistic Modifications

• Results of national studies are not conclusive.
• Most of the CRESST studies found significant gain
  for ELL students on the linguistically modified
  version.
• However, the outcome of national research on the
  impact of linguistic modification is mixed
  (Francis, et al. 2006).
• Factors that lead to such discrepancies include
  variation in methodology in implementing
  linguistic modification approach, sampling and
  power issues, variation in test items and the
  nature of linguistic complexity, etc.

21
Improvements in the Language of Instruction

• Unnecessary linguistic complexity may impact
  students understanding of instructional
  materials
• Opportunity to learn for ELLs and SLDs may be
  affected
• Research show ELLs and SLDs benefiting less from
  instructions with complex linguistic structure

22
Conclusions and Recommendations

• Assessments and instructions for ELLs
• Must be free of construct irrelevant factors
• Must be free of unnecessary linguistic
  complexities
• Must include a sufficient number of ELLs and SWDs
  in its development process (field testing,
  standard setting, etc.)
• Must be free of cultural biases
• Must be sensitive to students linguistics and
  cultural needs

23
Impact of Linguistic Modification Research on
Practice

• Many test publishers have incorporated the
  concept of linguistic modifications in their test
  development process.
• Among these test publishers Educational Testing
  Service (ETS), Harcourt Assessment, CTB/McGraw
  Hill, Measurement Incorporated, etc.
• Many states have adopted this approach into their
  assessments
• The U.S. Department of Education incorporated
  this process into the National Assessments (NAEP)
  and has highly recommended this approach to be
  incorporated into the state NCLB Title I
  assessments.

24
For more information, please contact Jamal Abedi
at CRESST/UC Davis

• [phone]
• or
• jabedi_at_ucdavis.edu