SLAT6827 Second Language Literacy Semester 2, 2004

1
SLAT6827 Second Language Literacy Semester 2,
2004

• Assessing Writing
• Week 11

2
Aims of the lecture

• Introduce basic approaches to testing and
  assessing writing
• Survey major types of writing tests
• Examine components of good writing as the basis
  for assessment
• Evaluate the role of grammar correction in
  writing feedback
• Compare different procedures for scoring
  writing samples.

3
Validity reliability in writing tests

• Validity reflects the degree to which a test
  measures what it is supposed to measure.
• A test is valid for something. For example
  program placement, measuring achievement, etc.
• Reliability refers to consistency of measurement.
• A test is reliable if it yields consistent
  results across different formats, times, raters,
  etc.

4
Approaches to testing writing

• Direct testing
• assessing a sample of writing
• Indirect testing
• assessing microskills that contribute to
  writing, such as knowledge of grammar rules,
  vocabulary, etc.
• Indirect tests have been considered more reliable
  than direct tests.

5
Major types of testing

• Summative assessment
• measures of achievement
• Formative assessment
• measures used to develop specific skills
• Communicative testing
• measures attempting to replicate real-life
  language tasks

6
Validity in writing tests

• Face validity To the outsider observer, is it a
  valid test of writing?
• Construct validity Does the test reflect the
  underlying psychological reality of area?
• Content validity Does the test assess relevant
  content?
• Criterion validity Does the test correlate with
  other tests that assess writing, perhaps given at
  the same time (concurrent validity)?
• Predictive validity Does the test correlate with
  measures collected at a latter time?

7
Components of writing

• What is good writing?
• Content Organization
• Language use Vocabulary
• Mechanics Conventions
• Judgement (Rhetorical) Style
• Ideas Voice
• Sentence fluency Word choice
• Impact ?????

8
Feedback on writing

• Formative evaluation intended to improve writing
  skill.
• Local versus global errors
• Should grammar be corrected?
• The value of grammar correction is taken for
  granted.

9
The case against grammar correction Truscott
(1996)

• gt Grammar correction does not work.
• There are empirical reasons why it doesnt work
• There are theoretical reasons why it doesnt
  work
• Grammar correction may even be harmful.

10
Grammar correction a definition.

• Grammar correction is the correction of
  grammatical errors for the purpose of improving
  grammatical accuracy.

11
Evidence against grammar correction

• Evidence shows that grammar correction in the L1
  has no effect
• Grammar correction had little or no effect on
  students writing, regardless of learner level,
  what types of errors were corrected, or how the
  corrections were done (see e.g., Leki 1990).
• Does it make a difference in the L2?
• Apparently not (Krashen, 1992 Van Patten,
  1986a,b)

12
Testing the effectiveness of grammar correction.

• Robb, Ross, Shortreed (1986). Japanese EFL
  university
• Students (N134)
• Four feedback conditions
• 1) Explicit correction (n30) Errors and correct
  forms identified
• 2) Correction code (n37) Type and location of
  error identified
• 3) Highlight errors (n37) Errors underlined but
  no explanation
• 4) Errors tallied Number of errors per line
  tallied but not explained.

13
Results

• No differences in writing ability across the four
  conditions.
• Gradual increases in the mean scores occurred
  regardless of the feedback method.
• gtgt The direct methods do not produce results
  commensurate with the amount of effort required
  by the instructor... (p.88)

14
Grammar correction can also have a negative effect

• Negative impact on fluency (Semke, 1984)
• The group that received comments were less
  accurate than those who did not.
• Discourages learners (Hillocks, 1986)
• Takes time (Cohen, 1987)

15
Why doesnt grammar correction appear to work?

• Conclusion
• Grammar correction does not work because the
  learner goes through developmental sequences in
  the acquisition of L2 grammatical knowledge. The
  process is gradual with learning occuring when
  the learner is ready.

16
Comparing two kinds of scoring procedures.

• In-class activity. Handouts provided.

17
Last slide SLAT6827 week 11