Language Testing

1
Language Testing

• Topic 3. Criteria of Language Test (1)

2
3.1. Basic concepts and key terms

• Validity
• in testing refers to the degree to which a test
  actually measures what it is supposed to measure.
• Reliability
• refers to the consistency of test scores.
• Authenticity
• is seen as the correspondence between features of
  test tasks and those of real life tasks.

3

• Interactiveness
• is viewed as the extent of involvement of
  students in completing a test task.
• Practicality
• refers to the considerations that a test is
  practical for its economy, ease of
  administration, scoring and ease of
  interpretation.
• Washback / Backwash
• means the impact that a test may exert on
  teaching or on learning. It can be harmful or
  beneficial. (Please refer back to Topic 1.)

4

• Discrimination
• refers to the ability of a test item to separate
  good students from poor ones in a particular
  aspect of the foreign language.
• Face validity
• the extent to which a test appears to measure
  what it claims to measure.
• Content validity
• the extent to which the content of the test
  represents the sampled domain of specification.

5

• Construct validity
• refers to the quality of a test in terms of the
  theoretical model on which the test is based. It
  is also called theoretical validity.
• Concurrent validity
• the extent to which scores on a new test
  correspond to scores of another existing test.
• Predictive validity
• the correlation between scores from a test and
  scores later.
• The true score
• the average score, which best represents the
  students ability.

6
3.2. Six major criteria of language tests

• Validity, Reliability, Authenticity,
  Interactiveness, Practicality, Beneficial
  Backwash
• These six elements are inseparable, and should be
  regarded as a unified whole when we aim to design
  a valid means to measure language proficiency.

7
3.3. Validity of language tests

• 3.3.1. Nature of validity
• 3.3.2. Factors influencing validity
• 3.3.3. Methods of judging validity
• 3.3.4. Learning exercises (Review Quiz 1 and
  Review Quiz 2)

8
Review Quiz 1

• Part I
• Identify the item from each pair which is most
  likely to be an invalid measure of the outcome
  variable in question. (The outcome variable is
  given in parentheses.)
• PAIR 1
• ____ The child will correspond intelligibly with
  an assigned French-speaking pen pal. (Understands
  French)
• ____ The child will correspond intelligibly with
  the French-speaking pen pal assigned to him/her.
  (Appreciates French culture)

9

• PAIR II
• ____ The student will identify examples of the
  principles of physics in the kitchen at home.
  (Understands principles of physics)
• ____ The student will choose to take additional
  courses in the physical sciences. (Appreciates
  physical sciences)

10

• Part II
• Write I next to statements that indicate an
  invalid instrument. Write V next to those which
  indicate a valid instrument. Write N if no
  relevant information regarding validity is
  contained in the statement.
• ___ The questions were so hard I was reduced to
  flipping a coin to guess the answers.
• ___ This test measures mere trivia, not the
  important outcomes of the course.
• ___ To rule out the influence of memorized
  information regarding a problem, only topics
  which were entirely novel to all the students
  were included on the problem-solving test.

11

• ___ The only way he got an A was by having his
  girl friend write the term paper for him.
• ___ The length of the true-false English test was
  increased from 30 to 50 items to minimize the
  chances of getting a high score by guessing.
• ___ The teacher ruled out the likelihood of
  cheating by giving the students at the same table
  a different form of the test.
• ___ Since the personality test had such a hard
  vocabulary level, it probably came closer to
  measuring intelligence than any personality
  factors.

12
Review Quiz 2

• Indicate the type of technical validity required
  for each of the following situations. Choose from
  this list
• Content validity
• Predictive validity
• Concurrent validity
• Construct validity

13

• ___ The test designer has developed an Anxiety
  Measurement Scale and wants to verify that it
  really measures a characteristic which can be
  labeled anxiety.
• ___ The counselor wants to select students into
  his schools college preparatory program based on
  the likelihood that they will succeed in college,
  and he wants to know whether a certain test can
  help him accomplish this selection process.
• ___ The test designer has developed a new,
  ten-minute IQ test and wants to demonstrate that
  it measures about the same thing as the
  Stanford-Binet IQ Test.

14

• ___ The dean wants to make sure all the exams in
  the English composition courses cover all the
  objectives they are supposed to cover.
• ___ The teacher wants to find out if the students
  who fail her final comprehensive exam really are
  the ones who will have trouble with related
  materials the next year.

15
Review Quiz 1

• Part I
• Identify the item from each pair which is most
  likely to be an invalid measure of the outcome
  variable in question. (The outcome variable is
  given in parentheses.)
• PAIR 1 (B)
• ____ The child will correspond intelligibly with
  an assigned French-speaking pen pal. (Understands
  French)
• ____ The child will correspond intelligibly with
  the French-speaking pen pal assigned to him/her.
  (Appreciates French culture)

16

• PAIR 2 (B)
• ____ The student will identify examples of the
  principles of physics in the kitchen at home.
  (Understands principles of physics)
• ____ The student will choose to take additional
  courses in the physical sciences. (Appreciates
  physical sciences)

17

• Part II
• Write I next to statements that indicate an
  invalid instrument. Write V next to those which
  indicate a valid instrument. Write N if no
  relevant information regarding validity is
  contained in the statement.
• _I_ The questions were so hard I was reduced to
  flipping a coin to guess the answers.
• _I_ This test measures mere trivia, not the
  important outcomes of the course.
• _V_ To rule out the influence of memorized
  information regarding a problem, only topics
  which were entirely novel to all the students
  were included on the problem-solving test.

18

• _I_ The only way he got an A was by having his
  girl friend write the term paper for him.
• _V_ The length of the true-false English test was
  increased from 30 to 50 items to minimize the
  chances of getting a high score by guessing.
• _V_ The teacher ruled out the likelihood of
  cheating by giving the students at the same table
  a different form of the test.
• _I_ Since the personality test had such a hard
  vocabulary level, it probably came closer to
  measuring intelligence than any personality
  factors.

19
Review Quiz 2

• Indicate the type of technical validity required
  for each of the following situations. Choose from
  this list
• Content validity
• Predictive validity
• Concurrent validity
• Construct validity

20

• _d_ The test designer has developed an Anxiety
  Measurement Scale and wants to verify that it
  really measures a characteristic which can be
  labeled anxiety.
• _b_ The counselor wants to select students into
  his schools college preparatory program based on
  the likelihood that they will succeed in college,
  and he wants to know whether a certain test can
  help him accomplish this selection process.
• _c_ The test designer has developed a new,
  ten-minute IQ test and wants to demonstrate that
  it measures about the same thing as the
  Stanford-Binet IQ Test.

21

• _a_ The dean wants to make sure all the exams in
  the English composition courses cover all the
  objectives they are supposed to cover.
• _b_ The teacher wants to find out if the students
  who fail her final comprehensive exam really are
  the ones who will have trouble with related
  materials the next year.

22
3.4 Discussion questions

• 1. What are the criteria for good language tests?
• 2. What are the major types of validity in
  language testing?
• 3. What are the major factors influencing
  validity?
• Lets watch "validity"