Developing Formative Peer Assessment of Instruction in Libraries

1
Developing Formative Peer Assessment of
Instruction in Libraries

• Susan Kraat kraats_at_newpaltz.edu
• Stephan J. Macaluso macaluss_at_newpaltz.edu
• SUNYLA 2003

http//www.newpaltz.edu/macaluss/peer/sunyla.html
2
Considering Peer Assessment?

• What issues and concerns do you envision?

3
A Short History of Library Assessment

• Student Satisfaction Survey (2000)
• Faculty Satisfaction Survey (2000)
• LibQual (2003)
• What about Library Instruction?

4
...Middle Statesrated library operations and
services very highly. Its commendatory report
especially recognized the librarys user-centered
focus, its strong and effective assessment
program, and its leadership role in information
literacy initiatives on campusSojourner Truth
Library, Annual Report, 2000/01
5
Standards?

• Outside Accrediting Agencies
• SUNY Board of Trustees
• GEAR Group
• ACRL
• Campus Assessment organizations

6
General Education III competency areas

• Systematic Inquiry
• Effective Expression
• Information Literacy
• Ethical Reflection

7
Why peer assessment?

• Instruction librarians face unique circumstances
  with assessment at the course level since they
  most frequently contribute to classes where they
  are not the instructor of record.
• Debra Gilchrist, ACRL Immersion Instructor

8
Peer Evaluation is important...

• Colleagues from the same fieldare especially
  well-qualified to assess the appropriateness of
  teaching and materials Keig and Waggoner,
  1994.
• Most students are not able to evaluate teacher
  effectiveness Malik, 1996

9

• Facultyneed to be in each others classes
  regularlyobservations keep instruction fresh
  and make clear the complexities of the
  teaching-learning phenomenon Weimer, 1990.
• We have identified conditions under which
  facultyadopt new instructional approaches
  exposure to relevant theory, provision for
  practicing new techniquesreceiving feedback,
  and opportunities for being coached Menges,
  1985, 1990

10
What is Formative Assessment?

• Diagnostics and descriptive feedback with which
  to improve instruction
• Emphasis on development, improvement
• Results are used by the individual, as opposed to
  the administrator (Middleton, 2002)
• Growth, change, longevity, individuality
• Evaluation v. Assessment

11
Not just observation...

• Peer review syllabi, assignments, handouts,
  websites, ppts
• videorecordings (often self-evaluation)
• Review of student outcomes e.g., their completed
  assignments

12
Composite Goals of Peer Assessment

• Improve BI/LI instruction program
• Provide feedback from experts
• Multidimensional feedback
• Create collegial atmosphere frequent, positive
  feedback that was formative, not summative. Low
  Stress.

13
Common Themes

• Participation should be voluntary
• Choose your own observer
• Observations occur by invitation only
• Participants determine what gets observed, in
  advance
• Results should be kept confidential
• 2 times per year

14
Common Theme 3 Steps to an Observation

• Step One Pre-Observation Conference
• What will happen in this class?
• What are your goals?
• Will this class be typical of your teaching?
• What would you like me to focus on?
• Consider how youll collect data

15
Step 2 The Observation

• Note
• Student/Teacher attitudes
• Observations of student interest
• Organization
• Clarity
• Closure
• Time on Task
• Strategies

• Arrive Early
• Be unobtrusive

Tacit or Active Observation? See Laguardia
Oka, 2000
16
What Observations Reveal

• Physical-temporal setting
• Classroom Structure Procedures
• Rhetorical dimension

• The Drammaturgical-sociopolitical dimension
• Curricular Context
• Effects Outcomes

(Hart, 1987)
17
Step 3 Post-Observation

• Feedback can be immediate, or after a breather
• Formal and informal feedback
• If video is used watch together
• How do you think it went?
• ...accomplish what you set out to do?
• Non-weighted language

18
Resources for Observers

• Communications Public Speaking faculty
• Liaison training
• See our Handouts for sample questions

See Also Peer coaching, mentoring
19
Grass Roots Beginnings What questions do we
want to answer?

• How do we grow as teachers?
• What might be improved?
• What is beyond our control?
• What instructional tools and resources do we use?
• What might we use more than we do?

20
Observation Form

• Three objectives
• Three things I want feedback on...
• One thing I do not want feedback on...
• Predictor I think the class will (go well)
  (not go well) (be about average)

21
Peer Assessment(Observation) Form

• Objectives realistic, understood by audience
• Pacing
• Interaction, opportunities for participation
• Specifics e.g., handouts, demos, exercises, etc.
• What the instructor did especially well
• Areas for improvement
• Insights gained

22
Self-Assessment Form

• How I prepared for the class
• My strengths
• Areas for improvement
• One insight gained
• Other

23
Observee comments

• Nervous - felt flustered and uncomfortable
• Positive feedback reassuring
• Empty praise not helpful
• Librarians can be your toughest audience
• Specifics more helpful than general comments
  e.g., louder, perhaps a better example
• Did not get all information requested on the
  form did they not understand it?

24
Observer Comments

• Difficult to stay silent
• Are the forms understandable?
• What happens if I have to be the observer for
  everyone?
• What about videotape?
• Few people filled out the self-assmt. forms

25
Common Qualities of Meaningful Assessment
(Janesick, 2001)

• Observation and demonstration
• Some type of coaching
• Awareness of limits
• Genuineness
• Feedback was continuous
• Enjoyable and challenging
• New learning based on prior knowledge
• Can-do attitude

26
Moving Forward

• The point is not to get it right all at once, but
  to continue to review and renew what we do, in
  order to heighten the learning experience for our
  students and ourselves.