Surveys of Enacted Curriculum for English Language Learner Instruction, Standards,

1

Surveys of Enacted Curriculum for English
Language Learner Instruction, Standards, and
Assessments Evaluating Language Demands and
Language Complexity Edynn Sato, WestEd John
Smithson, WCER Jaime Usma, WCER Peter Worth,
WestEd Enhanced Assessment Grant SEC-ELL
Coding Washington, DC May 5-7, 2008
2
Training Goals

• Introduce ELL components of study
• Language demands
• Language complexity
• Training and calibration in SEC-ELL coding
  protocol

3
Agenda

• Introducing the linguistic analysis aspect of the
  study
• Components of linguistic analysis
• Coding procedures
• Coding practice
• Questions

4
Study Goals

• Improve methods of aligning instruction to
  standards and assessments for English language
  learners.
• Analyze relationship of alignment to student
  achievement.

5

• What is the nature of the language that
  students need to meaningfully engage with and
  achieve academic content?

6
Content and Academic Language

• What is the relationship?

Content
Academic Language
Content
Academic Language
Content
Academic Language
7
Interacting Dimensions of Content and Language

• Content/Topics
• Cognitive Demands
• Language DemandsAcademic English Language
  Functions
• Language Complexity

8

• Given the expected student outcome (content,
  learning), how do we expect students to use
  (receive, produce) language to acquire/demonstrate
  understanding of the content?

9
Example
The average weight of 50 prize-winning tomatoes
is 2.36 pounds. What is the combined weight, in
pounds, of these 50 tomatoes? (Multiple-choice
math item)

• Content
• Mean, median, mode
• Operations
• Multiply decimals

• Language
• Description
• Inquiring

10
Example

• Which sentence best sums up the selection?
• Two Presidents gave Jesse Owens important awards
  for his achievements.
• Jesse Owens set a new Olympic record in the long
  jump.
• Talent and hard work helped Jesse Owens become a
  great runner.

• Content
• Main idea(s), key concepts

• Language
• Inquiring
• Description
• Summarization

11
Example

• Content
• Geometric concepts

• Language
• Identify
• Classify
• Compare/Contrast
• Define
• Analyze
• Generalize
• Evaluate
• Variation in
• Length
• Amount of detail
• Level of abstraction
• Vocabulary
• Sentence structure
• Discourse style

12
Components to be analyzed

• Focus
• Mode
• Mode category
• Language demand
• Language complexity
• Density
• Construction
• Overall complexity

13
Example

• Which sentence best sums up the selection?
• Two Presidents gave Jesse Owens important awards
  for his achievements.
• Jesse Owens set a new Olympic record in the long
  jump.
• Talent and hard work helped Jesse Owens become a
  great runner.

14
Focus

• Is the language in this standard/item academic or
  social/functional?

• Which sentence best sums up the selection?
• Two Presidents gave Jesse Owens important awards
  for his achievements.
• Jesse Owens set a new Olympic record in the long
  jump.
• Talent and hard work helped Jesse Owens become a
  great runner.

15
Mode

• Is the language required of the student in this
  standard/item receptive or productive? Or both?

• Which sentence best sums up the selection?
• Two Presidents gave Jesse Owens important awards
  for his achievements.
• Jesse Owens set a new Olympic record in the long
  jump.
• Talent and hard work helped Jesse Owens become a
  great runner.

16
Mode Category

• Specifically, does the standard/item require the
  student to listen or read? Speak or write?

• Which sentence best sums up the selection?
• Two Presidents gave Jesse Owens important awards
  for his achievements.
• Jesse Owens set a new Olympic record in the long
  jump.
• Talent and hard work helped Jesse Owens become a
  great runner.

17
Language Demand

• What function(s) does academic language in each
  mode serve in this standard/item?

• Which sentence best sums up the selection?
• Two Presidents gave Jesse Owens important awards
  for his achievements.
• Jesse Owens set a new Olympic record in the long
  jump.
• Talent and hard work helped Jesse Owens become a
  great runner.

18
Language Demands
?2008
19
Language ComplexityDensity

• Is the density of the minimum amount of academic
  language required to achieve this standard/item
  low or high?

• Which sentence best sums up the selection?
• Two Presidents gave Jesse Owens important awards
  for his achievements.
• Jesse Owens set a new Olympic record in the long
  jump.
• Talent and hard work helped Jesse Owens become a
  great runner.

20
Language ComplexityDensity
21
Language ComplexityConstruction

• Is the construction of the minimum amount of
  academic language required to achieve this
  standard/item simple or complex?

• Which sentence best sums up the selection?
• Two Presidents gave Jesse Owens important awards
  for his achievements.
• Jesse Owens set a new Olympic record in the long
  jump.
• Talent and hard work helped Jesse Owens become a
  great runner.

22
Language ComplexityConstruction
23
Overall complexity

• What is the overall complexity of the academic
  language in the standard/item? (1-4)

• Which sentence best sums up the selection?
• Two Presidents gave Jesse Owens important awards
  for his achievements.
• Jesse Owens set a new Olympic record in the long
  jump.
• Talent and hard work helped Jesse Owens become a
  great runner.

24
Coding Procedures

• Rater Document Date Page of

25
Coding Sample

• Which sentence best sums up the selection?
• Two Presidents gave Jesse Owens important awards
  for his achievements.
• Jesse Owens set a new Olympic record in the long
  jump.
• Talent and hard work helped Jesse Owens become a
  great runner.

26
Decision Rules

• Guidelines that help ensure standardization and
  reliability of the coding process.
• Examples
• Grades 11-12 Written and Oral Language
  Conventions standard 1.2 is understood to address
  only producing one's own writing, not correcting
  other authors' writing.
• In grades 11-12 Reading standard 2.1, the phrase
  public documents was interpreted as including
  editorials, commentaries and opinion pieces on
  topics of public concern.

27
Coding Standards

• Work in groups of 3 or 4
• Code standard for each component
• Not a consensus model
• Ensure intra-rater consistency

28
Coding a Standard

• ELL Standard I (Grades 36)
• Standard ListeningComprehend and interpret the
  spoken language of others.
• Objective Listen to others to clarify, question,
  and extend.
• Expectation Mark or write a response that shows
  comprehension of evidence or conclusions in a
  selection.

29
Expectation Mark or write a response that shows
comprehension of evidence or conclusions in a
selection.
30

• Algebra (Grade 4)
• 4.1 Generate algebraic rules and use all four
  operations to describe patterns, including
  nonnumeric growing or repeating patterns.

31

• Algebra
• 4.1 Generate algebraic rules and use all four
  operations to describe patterns, including
  nonnumeric growing or repeating patterns.

32

• Reading (Grades Nine and Ten)
• 2.1 Analyze the structure and format of
  functional workplace documents, including the
  graphics and headers, and explain how authors use
  the features to achieve their purposes.

33

• Reading (Grades Nine and Ten)
• 2.1 Analyze the structure and format of
  functional workplace documents, including the
  graphics and headers, and explain how authors use
  the features to achieve their purposes.

34

• ELL Standard I (Grades 36)
• Standard Listening comprehend and interpret
  the spoken language of others.
• Objective Listen to others to clarify, question,
  and extend.
• Expectation Create a picture or mark a correct
  response that shows comprehension of evidence or
  conclusions in a selection.

35

• Expectation Create a picture or mark a correct
  response that shows comprehension of evidence or
  conclusions in a selection.

36

• Algebra (Grade 4)
• 2.4 Compare and order decimals and estimate
  fraction and decimal amounts in real-world
  problems.

37

• Algebra
• 2.4 Compare and order decimals and estimate
  fraction and decimal amounts in real-world
  problems.

38

• Geometry (High School)
• 2.0 Students write geometric proofs,
  including proofs by contradiction.

39

• Geometry
• 2.0 Students write geometric proofs,
  including proofs by contradiction.

40

• Reading (Grades Nine and Ten)
• 3.9 Explain how voice, persona, and the
  choice of a narrator affect characterization and
  the tone, plot, and credibility of a text.

41

• Reading (Grades Nine and Ten)
• 3.9 Explain how voice, persona, and the
  choice of a narrator affect characterization and
  the tone, plot, and credibility of a text.

42
Reminders

• Code up to six language demands for each mode
• Consider complexity of the minimum language
  required to achieve the standard
• Use decision rules to record interpretations

43
Questions
44

• If you have questions or would like additional
  information, please contact Edynn Sato, Ph.D.
  esato_at_wested.org