Prepared for the College of Education at SDSU

1

• An Analysis of Student Feedback Systems

• Prepared for the College of Education at SDSU
• By Michael J. Montgomery

2
Description of the COE

• Comprises seven academic departments
• Offers a variety of
• Degrees, credentials, and certificates
• MA and MS
• Two doctoral programs in partnership with the
  University of San Diego and Claremont Graduate
  University
• 3rd doctoral program with UCSD to begin in Fall
  2004

3
Why conduct this study?

• Deans Office aware of problems with current
  structure/system for collecting end-of-course
  data
• Interest in making the system sounder and more
  responsive
• Need for theoretical knowledge related to student
  feedback systems
• Provide groundwork for next steps

4
Evaluation Process

• Two-prong approach
• Review of the literature
• Review of COE policy

5
Evaluation Process Literature Review

• Themes addressed
• What constitutes teacher effectiveness
• Student credibility
• Suspicions associated with student ratings
• Student tools for assessing teacher effectiveness
• Instrument quality
• Triangulation of data
• Administration of student ratings

6
Evaluation Process COE Policy

• Reviewed
• COE Policy File
• Retention/tenure/promotion (RTP) procedures
• Role of ratings in RTP process
• Quantitative and qualitative questionnaires
• Administration guidelines
• Interviewed
• COE Associate and Assistant Deans
• COE Department Chairs

7
Findings Related to Themes

• Teacher effectiveness
• Multidimensional enterprise
• Student credibility
• Rational, reliable, and credible
• Suspicions associated w/student ratings
• Majority have been refuted

8
Findings Related to Themes

• Student tools for assessing teacher effectiveness
• Items to be measured must be clear
• Derived from various stakeholders
• Field tested
• Instrument quality
• Psychometrically and statistically sound
• Protected against unwanted variables

9
Findings Related to Themes

• Triangulation of data
• Should not be sole or primary source
• Combine with peer reviews, alumni ratings,
  classroom observations, teaching portfolios, and
  self-evaluations
• Administration of student ratings
• Standardized procedures
• Anonymous
• Absent of the instructor

10
Findings COE Policy Looking at the Current
State of Affairs

• Survey administration
• Includes clear instructions
• Quality of survey items and constructs
• Many items warrant revision
• Measurement scales not consistent with good
  measurement design
• Does not collect demographic data about students

11
Findings COE Policy

• Use of student ratings data
• Most important RTP component
• No specific weight assigned
• May not be weighted appropriately
• Students are unsure if responses are used for
  improvement purposes
• Time gap between student completion and receipt
  of feedback by faculty
• Takes longtime to receive feedback

12
Recommendations

• Several recommendations were made to the COE
• These will be explored by COEs Policy Council