Building Resources Through Electronic Peer Review: The Expertiza Platform - PowerPoint PPT Presentation

1 / 45
About This Presentation
Title:

Building Resources Through Electronic Peer Review: The Expertiza Platform

Description:

A shorter version of this presentation can be found at http://research.csc.ncsu. ... via an online testing system, such as Webassign, WebCT, LON-CAPA, or Mallard. ... – PowerPoint PPT presentation

Number of Views:34
Avg rating:3.0/5.0
Slides: 46
Provided by: edwardfge
Category:

less

Transcript and Presenter's Notes

Title: Building Resources Through Electronic Peer Review: The Expertiza Platform


1
Building Resources Through Electronic Peer
ReviewThe Expertiza Platform
  • Edward F. Gehringer
  • Dept. of Electrical Computer Engineering
  • Dept. of Computer Science
  • North Carolina State University
  • efg_at_ncsu.edu
  • A shorter version of this presentation can be
    found at http//research.csc.ncsu.edu/efg/expertiz
    a/presentations/Expertiza-short.htm

2
Outline
  • Introduction
  • Rationale
  • The Expertiza platform

3
Introduction
  • Electronic peer review is students reviewing
    other students work over the Web.
  • Building resources through electronic peer review
  • gets students working together to improve others
    learning experiences,
  • helps them learn, by performing tasks that are
    similar to real-world responsibilities,
  • gives them experience in writing their ideas up
    for an audience of their peers,
  • allows each cohort to stand on the shoulders of
    students in earlier classes.

4
Homework, traditionally
  • Heretofore, the homework process has served only
    for students to demonstrate mastery of the
    subject.
  • Every student does the same thingredundant
    effort.
  • Work is graded and thrown away, never benefiting
    anyone but the student who did it.
  • Now the best work can be automatically
    published, to help others learn.

5
Why Electronic peer review?
  • Only electronic peer review can produce these
    benefits.
  • Overhead of doing it manually is too great.
  • Assuring that enough students do each piece of
    work
  • (otherwise, none may be good enough).
  • Grading the work, which is time consuming
    students submit solutions to different problems.
  • Assembling the best submissions into a single
    unit, and getting permission to publish them for
    others.

6
Why Electronic peer review, cont.
  • Having the resources doesnt help unless the
    instructor can determine which are best.
  • The student reviews show the instructor exactly
    where to look for good materials.
  • Manual implementation would consume faculty/staff
    time, instead of saving it.

7
Outline
  • Introduction
  • Rationale
  • The Expertiza platform

8
The Rationale
  • Improve student learning
  • Improve teaching
  • Better utilize resources

9
Improving student learning
  • The Expertiza platform improves student learning
    in these ways
  • Integrates active and cooperative learning
  • Extends active learning to out-of-classroom
    activities and distance education
  • Discourages plagiarism

10
Integrating active and cooperative learning
  • Active learning allows students to take
    responsibility for their own learning.
  • E.g., short in-class exercises with one groups
    reporting to whole class.
  • Active learning keeps students more engaged and
    interested
  • Active learning encourages nontraditional
    students to stick with a program
  • Larger of milestones helps those with poorer
    study habits
  • Double-blind electronic review with Expertiza
    helps overcome biases against women,
    underrepresented minorities.

11
Extending active learning outside the classroom
and to DE
  • Provides a structure for out-of-class activities
    that encourages persistence and helps avoid
    distraction
  • Students give each other feedback on how to
    improve.
  • The larger number of deadlines and milestones
    makes it harder to be a slacker.
  • Diminishes difficulties of finding students with
    compatible schedules
  • Asynchronous out-of-class collaboration is now
    possible.

12
Electronic peer review allows active learning and
DE to mix
  • DE has been a roadblock to the use of active
    learning students viewing lectures remotely can
    work only by themselves.
  • Online discussions can help, but they are no
    panacea.
  • The instructor or TA needs to monitor them,
    interact with students, grade student
    submissions.
  • Time-consuming Needs to be done on a timely
    basis, several times per week.

13
Discouraging plagiarism
  • Multiple deadlines and milestones make it
    impossible to submit a finished product obtained
    from an external source.
  • Student feedback on peers contributions makes it
    easy to catch cheaters.

14
The Rationale
  • Improve student learning
  • Improve teaching
  • Better utilize resources

15
Improving teaching
  • The Expertiza platform improves teaching in these
    ways
  • Helps create weekly mastery quizzes
  • Increases the supply of examples/homework
    problems/test questions
  • Focuses students on explaining/understanding the
    concepts that are hardest to master
  • Helps courses keep up with technology

16
Periodic mastery quizzes
  • Students can be assigned to create
    machine-scorable questions over lecture material.
  • Peer review determines which questions are the
    best.
  • Having a supply of such questions makes it
    possible to give daily/weekly quizzes via an
    online testing system, such as Webassign, WebCT,
    LON-CAPA, or Mallard.

17
Increasing the supply of good examples and
questions
  • Students can be assigned to make up a
    peer-reviewed homework or test question.
  • Increases the supply of worked-out problems
  • Improves homework and/or exams, by adding to the
    supply of good textbook questions
  • (This is of interest to textbook publishers,
    too!)
  • Some questions can be used as examples in class
    or given to students for self-study.

18
Focusing students on mastering the hardest
concepts
  • Students can be assigned to create materials that
    address subjects that are difficult to master.
  • Annotate lecture notes with references to
    background material from the Web.
  • Identify the hardest concept from Lecture k and
    write an example that clearly explains it.
  • Improve the diagram that illustrates x.
  • Add animation to the PowerPoint slides from this
    lecture.
  • All of these, of course, can be peer-reviewed.

19
Helping courses keep up with technology
  • Students can be assigned to research emerging
    areas of technology.
  • E.g., how communication is managed in a new
    parallel computer architecture.
  • They can be asked to create lectures, or slides,
    describing the material, that would fit in with
    the course they are taking.

20
The Rationale
  • Improve student learning
  • Improve teaching
  • Better utilize resources

21
How Expertiza improves resource utilization
  • Some work is peer-graded, so teaching assistants
    can spend more time working with students and
    less time grading.
  • Not only does this help the students, but TAs
    learn skills that will benefit them in the
    workplace.
  • Having inadequate TA support no longer limits the
    amount and kinds (e.g., design problems) of
    homework that can be assigned.
  • Students rely more on their peers for help, less
    on the course staff.

22
How Expertiza improves resource utilization, cont.
  • Makes teaching large classes an advantage!
  • Large classes can produce more and better
    resources.
  • The usual disadvantages of large classes (lack of
    engagement and personal attention) are mitigated
    by
  • active learning,
  • better materials,
  • giving TAs more time to work with students

23
Outline
  • Introduction
  • Rationale
  • The Expertiza platform

24
The Expertiza platform
  • Can accept submissions in almost any format.
  • Web is a familiar interface.
  • Web-creation skills are important to students.
  • Web interface allows use in DE.
  • Facilitates the production of Web resources.

25
Overview
  • The implementation
  • The review cycle
  • How peer grading has been used
  • Student reaction
  • The Expertiza platform

26
The Implementation PG
  • Students develop homework in the form of one or
    more Web pages.
  • Students submit their pages to the system.
  • The system copies them to a new Web address,
    concealing the submitters identity.
  • Reviewers are assigned semiautomatically.
  • Reviewers and authors communicate via a shared
    Web page.
  • Reviewers assign grades, and the system averages
    them.

27
The PG Login Page
28
Choosing an Assignment
29
Selecting Submission to Review
30
Submitting and Reviewing
  • A student logging in has a choice of whether to
    submit or review.
  • If submitting
  • can use a browser to select a file.
  • can submit one file at a time, or
  • a whole network of pages in a single Zip file.
  • If reviewing
  • is presented with a set of pages to review.
  • can click on one and type in comments.

31
Reviewing a Submission
  • Review is based on a rubric.
  • A numeric score is assigned to several questions.
  • Grade is calculated from these scores.
  • Ample opportunity for comments.

32
Overview
  • The implementation
  • The review cycle
  • How peer grading has been used
  • Student reaction
  • The Expertiza platform

33
Submit-Review-Publish Cycle
  • Signup phase. A limited number of students
    allowed to sign up for each choice.
  • Initial feedback phase. Students given 27 days
    to make initial comments.
  • Resubmission phase. 27 days to revise work in
    response to reviewer comments.
  • Grading phase. 37 days to make final comments
    and assign scores.
  • Review of review phase. Students review each
    others reviews.
  • Web-publishing phase. PG creates a Web page with
    the best assignment in each category.

34
Overview
  • The implementation
  • The review cycle
  • How peer review has been used
  • Student reaction
  • The Expertiza platform

35
How Peer Review Has Been Used
  • Researching lecture material.
  • Find links related to each lecture.
  • Annotating on-line lecture notes.
  • Writing research papers.
  • Reviewing papers from the literature.
  • Making up homework problems.
  • Making up machine-scorable questions.
  • Weekly reviews.

36
Annotating a Lecture
  • Students electronically sign up to review a
    particular lecture,
  • then add hyper-links to instructors on-line
    notes.

37
Research Papers
  • Can include hyperlinks to Web documents.
  • Different students can sign up to write on
    different topics.

38
Madeup Problems
39
Choosing Assignment Types
40
Producing Resources
  • A divide and conquer approach to large
    projects.
  • Lecture annotations for an entire semester.
  • Machine-scorable questions for each lecture.
  • Madeup problems for the Computer Architecture
    Course Database (cf. WCAE 1998, WCAE 2000).

41
Overview
  • The implementation
  • The review cycle
  • How peer grading has been used
  • Student reaction
  • The Expertiza platform

42
Student Reaction
  • Students have reacted quite positively to peer
    review.
  • When asked to rate on a scale of 1 to 5, Peer
    review is helpful to the learning process,
  • five classes rated it 3.41 to 4.24, with the
    highest score given by one of the classes that
    did the most peer-reviewed assignments.
  • By a score of 3.9, students said that reviews of
    reviews motivated them to do careful reviews.

43
Overview
  • The implementation
  • The review cycle
  • How peer grading has been used
  • Student reaction
  • The Expertiza platform

44
The Expertiza Platform
  • PG is one of three components of the Expertiza
    platform.
  • Shimmer for signing up for assignments,
    allowing a task to be divided into individual
    parts
  • PG for peer-reviewing work
  • Conoscenza a Web-based database that makes the
    best student work accessible to registered users
    over the Web.

45
The Expertiza Platform (cont.)
  • Using Expertiza, it is possible
  • to organize classes efficiently
  • to produce resources that will
  • enhance the educational experience of others.

PG
Conoscenza
Shimmer
Write a Comment
User Comments (0)
About PowerShow.com