Critical examination of the achievement gap between English Learners and NonELLs

1
Critical examination of the achievement gap
between English Learners and Non-ELLs

• Christy K. Boscardin
• UCLA/CRESST (christyk_at_ucla.edu)
• Zenaida Aguirre-Muñoz
• Texas Tech University/CRESST (z.aguirre_at_ttu.edu)

AERAMontreal, Canada April 15, 2005
2
Overview

• Provide background for study
• Review measurement of opportunity to learn
• Highlight key findings
• Present new directions of this work

3
Assessment Reform Project Background

• Primary Assessment Purpose
• Act as reform agent
• Reporting yearly progress
• ELLs redesignation criteria
• Curriculum Embedded Assessments
• Standards-Based
• Grades 4-11, in English Language Arts and
  Mathematics

4
Study Rationale

• High ELL Participation Requirement
• Examine Relationship Between OTL ELL
  Performance
• Examine Validity of ELL Scores
• Examine OTL variables that impact ELL achievement

5
Performance Assessment Prompt

• In a work of literature, a heroic character is
  often someone with extraordinary courage or
  ability who performs noble deeds or makes
  sacrifices. However, an ordinary person who faces
  extraordinary challenges can also be a heroic
  character.
• Select a heroic character from a literary work
  you have read in class this year. Using specific
  details from the text, explain why you think this
  character is heroic. Some of the things you can
  write about are the characters
• physical and personality traits
• impact on the story
• thoughts and motivations
• actions and relationships with other characters

6
Proficient Criteria
The response demonstrates solid reading
comprehension skills and the ability to analyze a
major literary element (characterization).

• Some of the important character features are
  described clearly (RC 2.0 WA 2.4)
• Some statements about the heroic qualities of a
  character are generally supported or explained
  through references to the text (RC 2.7 WC 1.1)
• Most ideas are logically organized (WA 2.2,
  2.4WS 1.0, 1.2, 1.6)
• Mechanical errors may be present but do not
  impede communication in most of the response. (WC
  1.0)

7
Key Writing Assessment Characteristics

• Model for Standards-Based Instruction
• Literary Analysis
• Curriculum-Embedded Design
• Flexibility of Administration
• Open Scoring Criteria
• Direct Assessment of Standards

8
Opportunity to Learn

• The mere presence of a curriculum does not
  necessarily guarantee opportunity to learn.
• Winfield, 1993

9
Key OTL Dimensions

• Teaching Experience
• Teacher Expertise
• Content Coverage (Literary Analysis Response to
  Literature)
• Instructional Processes
• Assessment Practices
• Instructional Resources

Based on Porter CRESST Studies
10
Previous Findings-Survey Method

• Student level
• Gender
• Course grades
• English proficiency
• Teacher level
• Teacher Expertise
• Content Coverage
• Socio-economic Status
• Differential impact between ELs and nonELs
• Content coverage
• Gender

11
Research Questions

• What are the effects of students language status
  and other background characteristics on students
  OTL and performance assessment score?
• Does the OTL effect vary depending on students
  ELL status?

12
Student Sample

• Students
• 1,038 Six graders
• ELLs (41)
• 51 Hispanic
• Only 23 had parents with higher than HS
• 95 free lunch
• LAPA scores distribution
• 1(20), 2(40), 3(26), 4 (14)

13
Teacher Sample

• 27 Teachers From 7 Schools
• Teaching Experience 26 (gt 3), 44 (4 10 yrs),
  30 (10 yrs)
• 40 - Emergency Teaching Credentials

14
Analysis

• Logistic HLM
• Student level
• SAT-9, Gender, Language Prof. Ethnicity, SES,
  Parent Education
• Teacher level
• Experience, Expertise, Content Coverage (Literary
  Analysis, Writing), Instructional processes,
  Assessment practice, Resources

15
Student-Level Results

• Girls out performed boys
• Higher score on SAT-9 ? higher scores on LAPA
• Hispanic students performed higher than African
  American students (after controlling for language
  proficiency)
• ELLs scored lower than non-ELLs

16
OTL Results (Teacher Level)

• Positive effect of content coverage
• Literary analysis
• Written responses to literature
• Differential impact
• Gap increases with greater content coverage of
  written responses to literature

17
OTL Results (Teacher Level)

• Positive effect of content coverage
• Literary analysis
• Written responses to literature
• Differential impact
• Gap increases with greater content coverage of
  written responses to literature

18
Differential Impact
19
Limitations of the Study

• Classification of ELLs
• Teacher sample size
• Methodologyteacher level results
• Generalizability of one assessment measure

20
Conclusions

• Opportunity learn the content of test has a
  positive effect on student performance.
• Instruction in writing without direct support to
  EL may be insufficient in bridging the gap
  between ELLs and non-ELLs.
• Need further studies examining
• Key OTL variables sensitive to linguistic needs
  of ELLs
• Interaction effects of ELL-directed process
  strategies and a focus on linguistic dimensions
  of academic language

21
The Revision ProcessFrom Graphic Organizer...
22
The Revision Process...to First Draft...
23
The Revision Process...to Final Draft
24
Implications

• Teachers need to be better prepared to meet the
  demands of current reform efforts.
• Writing instruction should focus on patterns of
  academic written discourse to convey their
  text-based interpretations.
• Better models for defining academic registers are
  needed.

25
Framework for Investigating EL-Sensitive OTL
Teacher Experience Expertise
School/Classroom Environment
ELL OTL Indicators
Content Exposure 1. Content Language
Objectives 2. Content Coverage 3. Academic
Language

• Access Development
• 1. Delivery Format
• ELL Process Strategies
• Feedback Assessment

26
Framework for Investigating EL-Sensitive OTL
Content knowledge
Processes
EL-Specific
Domain-AL
Balance
Language Acquisition
Coverage
Breadth/Depth Time/Quality
27
Focus of CRESST OTL Instrument

• Integration of EL-specific opportunities
• Teacher expertise
• Content coverage-explicit instruction on academic
  language
• EL-directed instructional practice
• Assessment practice-feedback to students

28
What is Academic Language?

• Language used in the classroom for the purpose of
  acquiring knowledge
• Specific terminology
• Structure for communicating
• Differs from oral language (spoken, informal
  discourse)
• Register (level of formality)
• Function (language to accomplish specific tasks)
• Requires awareness of
• Expectations (e.g., analyze, compare, summarize,
  etc.)
• Authors/Speakers purpose/point of view

29
Instrument Reliability
30
Academic Language Definition Parameters

• Academic written discourse-response to literature
• Correspond to English (L2) language development
  stage
• Transparent to teachers and students

31
Functional Grammar-Halliday

• Provides a general framework for examining
  language
• Examines discourse patterns associated with the
  context and genre of writing
• Corpus-based research in EL writing development

32
Characteristics of Academic Written Discourse

• Expanded noun phrases
• Variety of processes (verbs)
• Variety of cohesion strategies-use of abstraction
• Impersonal Context
• Implicit point of view

33
The Revision ProcessOriginal Draft
34
The Revision ProcessFinal Draft
35
Impact on EL Achievement

• Deliverable coming soon
• www.cresst.org