The Role of Assessment in the EdD

1
The Role of Assessment in the EdD The USC
Approach
2
To Whom is the EdD Accountable?

• What are the checks and balances?
• For the PhD degree, one of the major quality
  control mechanisms is the dissemination of work
  for eternal peer review
• For the EdD, if the primary role is not to
  produce theoretical, generalizable knowledge for
  academic outlets, what is the equivalent?

3
The USC Context

• Large - 150 plus students per year
• One EdD - school wide
• Common mission improve urban education
• Common themes that define core (learning,
  accountability, diversity, leadership
• All faculty are EdD faculty
• Thematic dissertations - 1 faculty team of 8-9
  students

4
How We Are Approaching Assessment

• At USC, the mission of the School is to
  strengthen urban education locally, nationally,
  and globally. We define urban education broadly,
  conceptualizing it as something that takes place
  within many contexts including pre-kindergarten
  through high school, in human services, higher
  education, and workplace settings. We seek to
  transform urban education by
• Preparing and developing educational leaders who
  are change agents with a commitment to focusing
  on urban education, developing the competencies
  to address the complex educational and social
  issues facing urban communities.
• Leading the search for innovative, efficacious,
  and just solutions to the challenges in urban
  education by engaging in collaborative action
  research on educational practices and policy.
• Creating mutually beneficial partnerships to
  rethink curriculum, improve educational
  environments, and develop sound policy.

5
External Evaluation

• Our new program is now 10 years old we are
  conducting an external evaluation (West Ed) with
  the following questions
• How have the programs been implemented?
• To what extent have the programs achieved their
  student learning, program quality, and educator
  placement objectives?
• What implementation and student factors are
  predictive of student outcomes in the programs
  and beyond in the students professional
  employment?

6
Approach

• An adapted version of the Kirkpatrick Four Level
  Evaluation model measures are organized by type
  of outcome
• Reactions to the Program,
• Classroom learning,
• Transfer/implementation of learning, and
• Program outcomes.

Kirkpatrick, D. L., Kirkpatrick, J. D. (2006).
Evaluating training programs (3rd ed.). San
Francisco Berrett-Koehler Publishers.
7
A sampling of instruments and collection methods
- Formative

• Reactions (Ensures that students are motivated
  and interested in learning.)
• Student feedback forms to measure emotions,
  motivation, and self-efficacy
• Student end-of-course evaluation
• Faculty end-of-course evaluations
• Administrators and student advisor feedback of
  students reactions
•  
• Learning (Measures knowledge acquired, skills
  learned and improved, and/or attitudes changed)
• Student assignments and grading rubrics
• Student pre-assessment of knowledge and skills at
  the beginning of a course compared to achievement
  of course goals measured by course grading
  rubrics (pre- and post-assessments)

8
Formative Assessment Continued

• Transfer/Implementation/Behavior (Measures
  learning transfer and the extent to which
  behavior changes as a result of learning)
• Student performance in subsequent courses
• Early warning systems
• Student self-report of using new knowledge and
  skills in their professional work
• Faculty perceptions of students applying
  knowledge and skills learned in previous classes
  to learning opportunities in current classes.
• Capstone alignment with program goal, quality
  indicators, and completion rates
• Employer reports about student job performance
  throughout the program

9
A sampling of instruments and collection methods
- Summative

• Program Results (Measures the extent to which the
  program goal has been met or exceeded.)
• Graduation rates
• Placement statistics
• Graduate attainment of career goals
• Graduate perceptions of knowledge, experience,
  and self-efficacy in performing the program goal
• Faculty perceptions of the implementation of the
  program
• Employer perceptions of graduate performance
• Third-party and University evaluations

10
Establishing a Process

• The results of this external evaluation will be
  used by the Dean and the Governance Committee to
  create a long term strategy/structure/process for
  ongoing assessment