Title: Update on the Program Review Process: What Worked and What Didnt
1Update on the Program Review Process What
Worked and What Didnt
- NCATE Clinic
- Margaret Crutchfield
- May 2006
2Principles for the Reform of the Program Review
Process
- Continue to rely on expertise of SPAs
- Reduce costs and burdens for SPAs and
institutions - Streamline process
- Consistency across SPAs
- Closer alignment to NCATE Standard 1
3Continue to rely on expertise of SPAs?
- SPAs develop standards
- Select and co-train reviewers
- Maintain audit teams
- Collaborate to develop consistent policies,
forms, etc
4Have we reduced costs and burdens for SPAs and
institutions?
- SPAs
- Reduced staff time commitments but not yet to the
expected level - Will be reducing financial costs
- Have reduced some of the logistical burden of
dealing with reports, etc - SPAs still provide significant staff and
volunteer time to make this process work - Institutions
- Consistency in the process has been a great
improvement - But making the required changes in assessments
has taken time
5Have we streamlined the process?
- Yes, for institutions
- Submit on-line
- Identical forms etc have reduced confusion
- No, for SPA reviewers and audit teams
- Technology has not been our friend
- The audit review has been added to the process
6Is the process more consistent across SPAs?
- Consistent assessments
- 5 required
- State licensure
- Content
- Planning
- Clinical Practice
- Student Learning
- Consistent program templates
- Some SPA-specific requirements
7- Consistent policies for entire program review
system - Consistent decision criteria
- Still some inconsistency among reviewer
decision-making
8Provide closer alignment to NCATE Standard 1?
- Candidates preparing to work in schools as
teachers or other school personnel know the
content of their fields, demonstrate professional
and pedagogical knowledge, skills, and
dispositions and apply them so that students
learn. Assessments indicate that candidates meet
professional, state, and institutional standards.
9Required Assessments
- Content Knowledge
- State Test
- Another Content Based Assessment
- Professional and Pedagogical Knowledge, Skills,
and Dispositions - Planning
- Clinical Practice
- Candidate Impact on Student Learning
- Student Learning
10Principles for the Reform of the Program Review
Process
11Issues Addressed This Past Year
12What are the numbers?
13Challenges
- Maintaining deadlines
- Having sufficient reviewers
- Maintaining consistent reviewer decision-making
- Approval rates
14Approval Rates
- Pilot rates very low (31 and 22). In response,
- SPAs expanded criteria for nationally recognized
with conditions - NCATE and SPAs provided more resources and
guidance for programs especially concerning
alignment issues - NCATE provided specific guidance on how to use
unit-wide assessments to demonstrate candidate
mastery of SPA standards
15What was the result for initial reports?
16Approval rates for revised reports?
- 62 revised reports were received through Fall 05
- 60 received a positive decision
17Approval Rates by SPA
18Consistency in training reviewers and reviewer
decision-making
- Great variety in content of SPA reviewer training
and in the amount of time SPAs devote to reviewer
training - A larger issue is ongoing training and updating
of reviewers
19What NCATE is doing
- Asking SPAs to help develop criteria for training
- Developing on-line training modules for reviewers
- Improving NCATE Guidelines for Reviewers
including specific examples, models, and
problems - Implementing a newsletter for reviewers to keep
them up-to-date and connected - First issue sent out in 3/06
20What are the SPAs doing?
- Several have developed training materials
- Several have increased resources on their web
site - Many have helped NCATE develop materials for
NCATEs web site
21Ongoing challenges
- Continue to increase uniformity of system where
appropriate - Continue to support and train reviewers to
improve consistency of reviewer decision-making - Continue to support and provide guidance to
institutions
22- Deal with technical issues until new database is
ready - Making substantial progress in developing new
reviewer interface, pilot in Spring 07, implement
fully in Fall 07
23One More Important Lesson
- It is possible for programs to demonstrate that
their candidates have mastered the SPA program
standards using a limited number of comprehensive
assessments.