Military Language Testing at the National Defence University and the Common European Framework

1
Military Language Testing at the National Defence
University and the Common European Framework

• BILC CONFERENCE 2006. BUDAPEST

2

• General language examination in five languages

• ARMA bilingual military examination in nine
  languages

• NATO STANAG 6001 monolingual language
  examination

3

• Interviews with officers and NCOs

• Job descriptions of soldiers serving in NATO
  positions

• Hungarian officers and NCOs serving at the
  NATO headquarters

4

• Observance of the regulations for the conduct of
  examinations

• Continuous pretesting and renewal of items

• Continuous training and observing of testers

5
(No Transcript)
6
(No Transcript)
7
(No Transcript)
8
(No Transcript)
9
(No Transcript)
10
(No Transcript)
11
(No Transcript)
12
COMMON EUROPEAN FRAMEWORK
13
ESL Exams in Hungary

• University Of Cambridge ESOL
• Trinity College London ESOL GESE
• The European Language Certificates
• Pitman Qualifications
• Österreichisches Sprachdiplom Deutsch ÖSD
• Goethe-Institut vizsgarendszere
• LEXINFO
• LCCIEB EFB angol üzleti szaknyelv
• Euro
• ECL nyelvvizsga
• Bibliai héber
• Diplomáciai szaknyelvi vizsga
• Certifications de l Alliance française de Paris

14
Linking Separate Assessments

• Equating
• Calibrating
• Statistical moderation
• Benchmarking
• Social moderation

15
Linking Process

• Selection of appropriate procedures
• Project planning
• Coordination of the participants
• Application of the procedures
• Recording of results

16
Linking Procedures

• Familiarisation
• Specification
• Standardisation
• Empirical Validation

17
Familiarisation

• Discussion of the Common European Framework
  levels
• Qualitative analysis of the Common European
  Framework levels

18
(No Transcript)
19
The Specification Stage
20
ManualRelating Language Examinations to the
Common European Framework of Reference for
Languages Learning, Teaching, Assessment (CEF)
21
Specification

• Description of the coverage of the examination in
  relation to the Common European Framework

22
Content analysis procedures

• filling in a number of checklists
• using relevant CEF descriptors to relate the exam
  to the categories and levels of the CEF

23
The Aims of the Specification

• To analyse the content in relation to the
  relevant categories of the CEF
• To profile the examination in relation to the
  relevant descriptor scales of the CEF
• To make a claim that an examination is related to
  a particular level of the CEF

24

• Phase 1
• A general description of the examination or test
  tasks
• Phase 2
• A detailed description of the examination or
  test tasks

25
Stage 1General Examination Description Form (A1)
26
Stage 2Forms on Different Aspects of the
Examination

• The test development process (Form A2)
• The marking schemes and scoring rules for
  different subtests (Form A3)
• The grading and standard-setting procedures for
  different subtests (Form A4)
• The reporting of results (Form A5)
• The analysis and review procedures (Form A6)

27
Form 3A Marking
28
The Detailed Description of the Examination
29
Forms for the Communicative Language Activities

• Listening Comprehension
• Reading Comprehension
• Spoken Interaction
• Written Interaction
• Spoken Production
• Written Production
• Integrated Skill Combinations
• Integrated Skills
• Spoken Mediation
• Written Mediation

30
Listening Comprehension
31
After reading the scale for Overall Listening
Comprehension, indicate and justify at which
level(s) of the scale the subtest should be
situated
32
Forms for the Aspects of Communicative Language
Competence

• Reception
• Interaction
• Production
• Mediation

33
Outcome

• A narrative description of the examination
• Completed versions of the relevant forms
• A graphical presentation of the relationship of
  the examination to the CEF levels

34
Thank you for your attention