SPINE

1
DFID/ESRC RES-167-25-0263
Measuring student subject learning in whose
language(s)? Guoxing Yu, Pauline Rea-Dickins,
Zuleikha Khamis, Oksana Afitska, Haji Mwevura,
Shumbana Said, Amour Khamis, Abdulla
Mohammed University of Bristol, State
University of Zanzibar Student Performance in
National Examinations the Dynamics of Language
(SPINE, 2007-2010)

• 1. 1 Introduction

• 3. Further questions
• To what extent does the language background of
  test takers affect their performance in
  content-based assessment delivered in English?
• To what extent do test takers English language
  abilities (esp. reading and writing skills)
  affect their performance in such assessments?
• To what extent is test takers performance
  affected by language complexity and linguistic
  demands of test items?
• How might test accommodations (e.g. modifying
  linguistic demands of items) affect performance
  in content-based assessments?

• 6. Impact on teaching and learning
• Three aspects of classroom learning and language
  use
• Code switching and mixing and discourse features
  of teacher talk, e.g. safe talk (Hornberger
  Chick (2001) accommodating students as well as
  teachers low language ability, translation as
  coping strategy (Brock-Utne Holmarsdottir
  2004).
• Classroom pedagogy a limited range of pedagogic
  activities to engage learners in subject and
  language learning, within environments with very
  limited resources available.
• Learner classroom language lack of
  opportunities for participation in class e.g.,
  due to the unfamiliarity with the language of the
  instruction.

On a school wall in Kiswahili

• 5. Impact on individuals in tests
• AERA/APA/NCME 1999 91
• For all test takers, any test that employs
  language is, in part, a measure of their language
  skills. This is of particular concern for test
  takers whose first language is not the language
  of the test.
• 2. Effects of language background/ability on test
  performance
• 3. Then, can TEST ACCOMMODATIONS (e.g.
  modification of linguistic complexity of items,
  use of native language, see Butler and Stevens
  1997, 2001) provide a quick fix as the most
  direct and immediate policy remedy to address the
  complex issues surrounding language in
  examinations?
• 4. Effects of accommodations on performance
• (a) Modifications of linguistic demands of items
• Studies using linguistic modifications for EAL
  learners in content-based assessment have drawn
  on conceptually from findings of psychological
  studies where a changing in the wording or
  structure of a test item has been observed to
  affect students (mainly L1 learners) performance,
  however, in practice, the effects of linguistic
  modifications on test performance on EAL learners
  appear much more complex, interacting with
  students language abilities, type of items and
  subject areas, etc.
• (b) Using L1, or dual-language or side-by-side
  bilingual tests, e.g.
• Duncan et al. (2005) and Abedi et al. (2006) both
  found dual-language maths tests did not affect
  performance, though preferred by test takers
  (Duncan et al. 2005).
• Eisemon et al. (1993) the language of
  assessment, French or Kirundi, profoundly
  influenced the measurement of achievement in most
  subjects tested. And the most able students were
  most affected by being measured through French.

• 7. Social and political impacts
• The opportunity to use examinations as a lever
  for change (Kellaghan Greaney 1992, 2004) in
  monitoring and improving education quality in SSA
  are often missed, misused or even abused, leading
  to a serious waste of scarce educational
  resources (Kellaghan Greaney 2004 13), and
• rather seriously, raises issues of social and
  individual inequality with discrimination
  against minorities, rural populations, girls,
  and students whose first language differs from
  that of the examination (ibid. 7)
• and raises at least three questions in relation
  to educational policies
• Whether the problems of underachievement are
  related to a language of instruction that is
  different from the students home language as
  well as their teachers?
• Whose language(s) should be used as a medium of
  instruction to meet the targets of quality
  Education for All?
• Which language(s) should be used for formal and
  high stakes examinations?

• 4. Examples of test items
• Maths
• Calculate the lateral angles in the figure below.
• Given that a2-b2(ab)(a-b). Evaluate
  (204)2-(196)2
• In a certain hotel of 30 customers, 17 enjoy
  Cocacola, 19 enjoy Fanta and 2 dont enjoy
  either. Fill in the venn diagram below and use it
  to find how many enjoy i) Cocacola only, ii)
  Fanta only and iii) both Cocacola and Fanta?
• 4. The combined ages of Juma and Asha are 10
  years. The difference of Ashas age from twice
  Jumas age is 8 years. Find the ages of each one.
• Chemistry
• Name the process that could be used to obtain the
  named compound in the following mixture a)
  coconut oil from water, b) iodine and sand, c)
  salt from salt solution
• Biology
• Give a brief explanation on what will happen on
  the following A locust not suffocating when its
  head is immersed in water while the remaining
  part of body is outside.

• 1.2 Context
• NCLB Act of 2001 (USA) and Every Child Matters
  green paper (2003, UK)
• World Declaration on Education for All and
  Framework for Action (UNESCO, 1990)
• For students in SSA, English is an exoglossic
  language and their teachers are learners of
  English too and are not necessarily competent
  users of English.
• that poor countries often operate expensive and
  often complex language policies, whereas rich
  countries usually operate simple and relatively
  cheap language policies . (Williams 20061987)

• 2. Focus of Study 1
• What evidence is there that language of
  examination constitutes a determining factor in
  the demonstration of subject knowledge of
  students whose first language differs from the
  language of examination
• What assessment accommodations (including
  linguistic accommodations) are capable of
  addressing such linguistic unfairness in
  examinations?
• What are the social and political impact of
  language of examination in Sub-Saharan African
  contexts?

• 8. Some conclusions on the use of assessment
  accommodations
• There are significant conceptual, methodological
  and political challenges of applying
  accommodations in a reliable, valid and feasible
  manner.
• The inconsistency of research findings may be
  partly due to the complex nature and the
  interactions of accommodations with numerous
  other factors such as subject areas, student
  characteristics and the ways in which
  accommodations are implemented.
• It is crucial to look beyond accommodations to
  analyze the impact of instruction-embedded
  factors and the ways in which learners
  examination performance may be shaped by the
  nature of language use in teaching and learning
  in order to gain better understandings of the
  interrelationships between the language of
  assessment and the language of instruction.

Language of examinations
Impacts on society and educational policies
national and international
Impacts on individuals performance and progress
BORA
Students Test performance
Teaching and learning
Interviews with teacher (video) and learners
(audio) on the use of L1 and L2 in science
learning