Margie Crutchfield

1
Program Reviewer Seminar

• Welcome!
• Margie Crutchfield
• Associate Vice President, Program Review
• Please remember to set-up your audio
• Go to tools, then audio, then audio set-up
  wizard
• For those of you who have a microphone, we will
  do a microphone sound check one by one,
  starting at 130
• 330 pm - First audio sound check
• 345 pm Second audio sound check
• 355 pm - Final audio sound check
• 400pm - Session will begin

2
Session Agenda

• Eluminate Product Features
• Reviewer Responsibilities
• Updates on Program Review
• Data Rule
• Linking
• Timeliness
• Approval Rates
• Update on New NCATE Initiatives
• PRS Overview
• Q and A

3
(No Transcript)
4

• I have been assigned a review
• now what?

5
What should I expect from the lead reviewer?

• It is the lead reviewers responsibility to
  contact co-reviewers and set up time line for
  completing work.
• BUT if you dont hear from the lead reviewer
  within a week, you should contact them.
• If you still have problems contacting your team
  members contact your NCATE staff person

6

• In PRS open up your assigned program reviews
• Check to make sure all components of the report
  are present
• If you have concerns or questions about the
  report talk to your lead reviewer and/or your
  NCATE staff person

7

• Prepare at least a draft of your report
• Discuss your report with your team members
• Lead reviewer has information about setting
  conference call
• Be sure and submit your report (otherwise you get
  a lot of annoying e-mails)

8

• Lead reviewers must post the final team
  recognition report.
• This final report will be reviewed by the audit
  team.
• The audit teams report will be posted in PRS
  for you and your team to see

9

• Some items to note

10
Important Reminders

• You must have Version 7 or higher of Adobe
  Acrobat Readerfree download available on PRS
• Mac usersmake sure you review the document on
  the PRS log-in page

11
Making Decisions on Standards

• Each standard Met, Met with Conditions (except
  for NCTM and NASP), or Not Met
• If standard is Met, you are not required to fill
  in Comment section
• If standard is Not Met or MET with Conditions,
  you MUST include a comment

12
Making Final Decision

• Follow your SPAs decision rule
• Does the program have in place a series of 6 to 8
  key assessments that taken as a whole show
  candidate mastery of the SPA standards?
• Do candidates perform appropriately on those
  assessments?

13
Making the Final Decision

• Follow your SPAs decision rule
• But in general

14

• Recognized
• Program isnt perfect, but is well on its way,
  understands performance assessments, alignment of
  standards and assessments, etc.
• Recognized with condition
• Program understands performance assessment and
  alignment, but may have some serious deficiencies
  in some scoring guides and/or some assessments
• May not have sufficient data

15

• Further development required/recognized with
  probation/not recognized
• NCATE staff will determine which of the above
  applied
• Program really misses the mark, little or no
  alignment of the standards and the assessments

16

• If the decision is National Recognition with
  Conditions

17

• THE CONDITIONS BOX
• MUST BE FILLED OUT

18

• Write in the SPECIFIC conditions that need to
  be met by the institution on the next report
  submission.

19

• This report functions as a contract between
  NCATE-SPA and the Institution.
• In order for an Institution to retain Recognition
  after a Recognized with Conditions decision has
  been granted

20

• The team that looks at this when it is
  re-submitted will zero in on this conditions
  section to determine whether they were met or not
• High Stakes!

21
TIMELINE SUMMARYRemember we have two cycles in
the spring
Institution deadline SPA
deadline Review Team Audit
Team Spring 1 2/1 3/1 4/1
6/1 Spring 2 4/15 4/30 6/15
7/15

22
NCATE Deadline

• Final Posting to Institution
• Spring 09-1 cycle July 15
• Spring 09-2 cycle Sept. 1

23
Update on Program Review

• Data Rule For Spring 09
• Two years of data for all assessments (with
  understanding of programs context)
• If a key assessment is in a course that is taught
  once a year then one year of data one semester
  of data
• If program is brand new or going through program
  review for the very first time, one year of data
  is required

24
Linking

• Programs that have identical assessments but have
  some significant difference (degree level, etc)
• Must have a shell in PRS for each program but
  these can be linked (as evidenced by the color
  coding)

25
(No Transcript)
26

• Compilers submit all information in first report
  and submit it.
• When they open second (and subsequent) linked
  reports all information and attachments from the
  first report is copied into the second report.
• The compiler changes any pertinent information on
  the second report and submits it.

27
Reviewers do the same

• Open up the first report, evaluate the report.
• Open the first recognition report template, input
  text and information for all programs.
• Submit the first report.
• Open the second recognition report, change
  information on the cover sheet and submit the
  report.

28
Timeliness

• For the last three semester 95 of all reports
  were returned to the institutions on the posted
  deadline (hooray!!!!)
• And THANK YOU!!!

29
Approval Rates, Initial Submissions
R Nationally Recognized, C Nationally
Recognized with Conditions
30
Approval Rates, Rev/RC reports
Rev Revised Reports, RC Response to
Conditions Reports R Nationally Recognized, C
Nationally Recognized with Conditions
31
Approval Rates, Initial Reports
32
Initial (cont.)
33
Initial (cont.)
34
Approval Rates, Rev. and Cond.
35
Revised and Response to Conditions, cont.
36
Revised and Response to Conditions (cont)
37
What are we working on?

• SASB Task Force on SPA Standards
• Streamlining Initiatives
• Adding Options for Program Review

38
Task Force to look at SPA standards

• SPA standards are very diverse
• In number, organization, specificity
• In coverage of content and pedagogy
• Task Force has developed a set of 4
  guiding/organizing principles
• Next meeting is later in March it is expected
  that a draft will be posted on the web in late
  April for public comment

39
Middle level programs

• Change in policy for review of true middle level
  programs (is now policy)
• Must meet criteria for middle level programs
• Programs submit one report to NMSA
• 80 of candidates must pass state test in content
  areas
• Secondary program must reach national recognition
• If not secondary program, state must review
  content preparation program

40
Programs in secondary content areas that do not
provide content preparation

• This year NCATE will develop option for these
  programs
• During this year, these programs can defer review
  until NCATE develops process and they have time
  to implement
• Programs will not address SPA standards, they
  will address a generic set of standards (Task
  Force principles?)
• Will submit one report
• Will be expected to have assessments
  demonstrating candidate competence

41
Proposed further streamlining and new options
42
Streamline current program report

• Possible options
• Section I Question 1 to be answered if needed
  delete questions 3, 4, and 5 delete faculty
  information other deletions are under
  consideration.
• Section II Flexibility to consolidate some
  assessments.
• Section IV Condense and clarify instructions
• Section V Move to unit review?

43

• Decrease required amount of data
• 1 to 2 years?
• Investigate strategies for reviewing programs
  with extremely low enrollments

44
Potential Options for Program Review

• Current process will still remain an option!!!!!!

45
Options for Initial Review

• Allow institutions to choose all of their own
  assessments (with some constraints)
• No more than 8
• Must submit state test data
• Must demonstrate content, pedagogical content
  knowledge and skills, and impact on student
  learning
• SPAs develop set of pre-approved assessments

46
Options for continuing review

• Reduce requirements for continuing national
  recognition so that only new assessments need to
  be approved and minimal data provided.
• Focus on what is now Section Vself study and
  continuous improvement
• Permit an institution to conduct validity studies
  of its assessments in lieu of other program
  report evidence requirements

47
One more item

• Considering moving program review to mid-cycle (3
  years prior to visit) in conjunction with other
  changes for unit review
• All of the options should be seen in context of
  proposed changes to the unit review.

48
We need your feedback!!!

• Draft of proposal will be posted to the NCATE web
  site in mid-March for public comment
• You will receive an email with information and
  instructions on how to respond
• Please read the information and give us your
  feedback quickly

49
Lets walk through PRS
50
Logging into PRS

• To get started you will need to follow these
  steps
• Step 1. Go to the URL http//aims.ncate.org,
  click on PRS
• Step 2. Enter your ID (e-mail)
• Step 3. Enter your password

51

• When you go to the URL,
• you will see the log-in screen

52
(No Transcript)
53
(No Transcript)
54

• Once in the site
• You will see your name and program assignments

55
(No Transcript)
56
Some reminders

• Access Program Report from the institution
• Click on PDF icon in the column labeled Program
  Review/Current
• Access SPA Program Recognition Report template
• PRS (template is retrieved by clicking on the
  pen icon under lead reviewer for the assignment
  and is entered entirely on-line (content is
  exactly the same)
• Submit the completed SPA Program Recognition
  Report
• PRS--- continue to click next until you get
  to the submission page, then simply click submit

57
Writing the Report Some Reminders
58

• THE CONDITIONS BOX
• MUST BE FILLED OUT

59
A note about tone

• Reviewer attitude in report decisions is
  critical---
• Remember that you want to be as helpful as
  possible to the institutionstating the problem
  in clear objective languageavoid a God-like tone

60

• Reviewers are asked to accept the integrity of
  the program and review it for the purpose of
  determining if the information submitted meets
  the SPA standards
• Do not try to redesign the programs
• Objectively evaluate the QUALITY of the
  performance assessments and rubrics with respect
  to the alignment w/ SPA standards
• POSITIVE PERSPECTIVE
• Should include strengths and weaknesses
• Adapted from IRA- Guidelines for IRA/NCATE
  Program Reviewers October 4, 2006

61

• The BIG question!

62

• DOES THIS PROGRAM MEET THE STANDARDS
• ?

63
Q A
To pose a question Click hand
icon Push microphone icon to begin and
end audio Please state name and SPA
affiliation Questions will be taken in
the order they are received
64
Resources on NCATE web site

• www.ncate.org, click on Program Reviewers
• Program Report Forms
• Guidelines, Instructions
• Archived Web Seminars
• Mini-videos on how to use PRS
• Recently updated
• Added new ones
• Lead reviewers
• Audit Team
• Linking

65
(No Transcript)
66
(No Transcript)
67
NCATE Contact Info
Margie Crutchfield Associate Vice President
Program Review margie_at_ncate.org Robin
Marion Accreditation Associate Program
Review robin_at_ncate.org

• Sabata Morris
• Accreditation Associate
• Program Review
• Sabata_at_ncate.org
• Monique Thomason
• Accreditation Assistant, Program Review
• monique_at_ncate.org