Online Peer Reviews

1
Online Peer Reviews

• William J. Wolfe
• Professor of Computer Science
• California State University Channel Islands
• william.wolfe_at_csuci.edu

2
Homework Grading Bottleneck
3
Students/Graders
Teacher/Supervisor

• But
• N x N-1 copies!
• Students cant grade accurately.
• Too much work for the students.
• Cheating?

4
Peer Reviews Why?

• Students learn from each other.
• Students get lots of feedback.
• Students develop skills as evaluators.
• Students learn to appreciate evaluation criteria.
• Students see how they compare to peers.
• Students see class from the teachers
  perspective.
• Students get to know each other.
• Teacher plays role of supervisor
• (A much better use of the teachers
  skills/knowledge).

5
Peer Reviews Why Not?

• Students dont know the subject.
• Students are not skilled evaluators.
• Students can not, or will not, do that much work.
• Students will copy (cheat)!
• Keeping track of the reviews is very difficult.
• Student privacy.

6
Peer Reviews How?

• Student Web Pages
• Students post homework solutions on their own
  web page.
• Course Web Site
• Set up course web site to manage all the peer
  review activity. Keep track of
• Links to student web pages,
• Peer Reviews
• Scores,
• Comments.
• Anonymous reviews.

7
The Course Web Site
8
List of Student Links
9
Student Web Pages
10
Grading Criteria (Rubric)
11
Entering a Peer Review
12
Peer Reviews Received
13
Sample Peer Review
 Looks pretty good
perfunctory \pur-FUNGK-tuh-ree\ --adjective
Done merely to carry out a duty performed
mechanically done in a careless and superficial
manner characterized by indifference
14
Sample Peer Review
You should have requirements that detail the
concepts in section 4.2. Although you had some
very good points (i.e. the database should look
up student's degree requirements view should
list courses, etc...) almost all your
requirements can be more detailed. Go through
section 4.2 (each of the sections) and think of
what the program would need to do to effective
run. Some good examples of what requirements are
necessary are on others' websites, however I'll
give some to you now1.Is there a timeline
requirement?2.Is there a requirement on how
much(or how little) this will cost?3.Is there
security requirements?4.Is there user view
requirements?These(and many other questions) are
what you should answer in your requirements
definition document. Good luck on Assignment 3.
15
Average Peer Review Score
16
Scoring Comparison
17
Number of Reviews
Software Engineering (CSC 4508) 34
students Theory 1 Assignment 1,122
reviews. 15 Assignments 16,830
reviews. Fact 1 Assignment 300 400
reviews. 15 Assignments 5,212 reviews.
18
Software Engineering (CSC4508) Fall 2002
19
Software Engineering (CSC 4508) Fall 2002
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
Software Engineering (CSC 4508) Fall 2002
25
(No Transcript)
26
Summary
Stimulated class activity. Some passionate
participation. The audience effect brought up
all performance levels. Very accurate evaluations
(as a whole). Immediate access to examples of
good and poor work. Addressed late, incomplete,
and sloppy work. Needed access to web servers and
web page skills.
27
Acknowledgements
Carol Holder (Director of Faculty Development
CSUCI) Paul Rivera (Economics, CSUCI) Harley
Baker (Psychology, CSUCI) Bob Bleicher
(Education, CSUCI) Ivona Grzegorzcyk
(Mathematics, CSUCI) Nathaniel Emerson
(Mathematics, CSUCI) David Hibbits (Computer
Science, CSUCI) Todd Gibson (Colorado Institute
of Technology) Michael Cook (Forstmann Leff).
28
References
1. Online Student Peer Reviews, Proceedings of
ACM SIGITE Annual Conference, Salt Lake City
Utah, Oct. 28-30, 2004. 2. Student Peer Reviews
in an Upper-Division Mathematics Class, exchanges
THE ONLINE JOURNAL OF TEACHING AND LEARNING IN
THE CSU, (From the Classroom), September,
2003. 3. Course Web Site http//compsci.csuci.ed
u/wwolfe/ucd/online Password GUEST 4.
William.Wolfe_at_csuci.edu
29
Real Analysis (Math 351) Spring 2003