NCATE Standard 1 and the New Program Review Process

1
NCATE Standard 1 and the New Program Review
Process

• Donna M. Gollnick
• National Council for Accreditation of Teacher
  Education
• September 2004

2
NCATE Standards

• Candidate Performance
• Candidate Knowledge, Skills, Dispositions
• Assessment System and Unit Evaluation
• Unit Capacity
• Field Experiences and Clinical Practice
• Diversity
• Faculty Qualifications, Performance, and
  Development
• Unit Governance and Resources

3
Candidate Knowledge,Skills, and Dispositions
Standard 1
4
1. Candidate Knowledge, Skills, and
Dispositions

• Candidates preparing to work in schools as
  teachers or other professional school personnel
  know and demonstrate the content, pedagogical,
  and professional knowledge, skills, and
  dispositions necessary to help all students
  learn. Assessments indicate that candidates meet
  professional, state, and institutional standards.

5
Dispositions - A Definition

• The values, commitments, and professional ethics
  that influence behaviors toward students,
  families, colleagues, and communities and affect
  student learning, motivation, and development as
  well as the educators own professional growth.

6
Dispositions (continued)

• Dispositions are guided by beliefs and attitudes
  related to values such as caring, fairness,
  honesty, responsibility, and social justice.
• For example, they might include a belief that all
  students can learn, a vision of high and
  challenging standards, or a commitment to a safe
  and supportive learning environment.

7
Dispositions for All Candidates
8
from INTASC.

• The teacher respects students as individuals with
  differing personal and family backgrounds and
  various skills, talents, and interests.
• The teacher is sensitive to community and
  cultural norms.
• The teacher makes students feel valued for their
  potential as people, and helps them learn to
  value each other.

9
Dispositions are

• demonstrated in P-12 classrooms, courses, papers,
  etc.

10
Exercise on Dispositions

• At your table, introduce yourselves.
• Share how you are assessing dispositions at your
  institutions.
• How effective are the assessments? What works?
  What would you differently?

11
AREAS FOR IMPROVEMENT DISPOSITIONS

• The unit does not clearly delineate dispositions
  expected of candidates.
• The dispositions identified by the individual
  programs are not clearly linked to the conceptual
  framework.
• Not all programs consistently evaluate the
  identified dispositions.

12

• Candidate dispositions are not clearly defined,
  resulting in a lack of program data on how
  candidates dispositions are assessed.
• Candidates are not able to articulate the
  dispositions stated in the conceptual framework.
• The units key dispositions are not integrated in
  the units portfolio and field-based assessments.
  
• Assessment of candidate dispositions is
  inconsistent across programs.

13

• Not all candidates, unit faculty, and public
  school personnel are familiar with the
  professional dispositions identified in
  institutional standards.
• The definition and assessment of dispositions are
  inconsistent across the unit.
• The unit has not developed clear dispositions
  that candidates are expected to demonstrate.

14
1. Candidate Knowledge, Skills, and
Dispositions

• Candidates preparing to work in schools as
  teachers or other professional school personnel
  know and demonstrate the content knowledge,
  skills, and dispositions

15
Content Knowledge for Teacher Candidates (Initial
and Continuing Preparation of Teachers)
16
Content Knowledge for Other Professional School
Personnel
17
State Licensure Requirements in NCATE
18

• 80 of candidates must pass the content test
  required by the state to meet Standard 1.
  (aggregated across programs)
• 80 of candidates in a program must pass the
  content test required by the state for a program
  to be nationally recognized by NCATE and SPAs.

19

• Precondition 7 requires an institution to meet
  the state pass rate to be eligible for an
  accreditation visit by NCATE.
• All accredited institutions are reviewed annually
  for compliance with the new Precondition 7.
  (using Title II data)

20
Percentage of Candidates Passing Praxis in
Academic Year 2003-2004
21
National Benchmarks

• Biology Education
• Elementary Education
• English Education
• Mathematics Education
• Social Studies Education

22
1. Candidate Knowledge, Skills, and
Dispositions

• Candidates preparing to work in schools as
  teachers or other professional school personnel
  know and demonstrate the content, pedagogical,
  and professional knowledge, skills, and
  dispositions

23
1. Candidate Knowledge, Skills, and
Dispositions

• Candidates preparing to work in schools as
  teachers or other profes-sional school personnel
  know and demonstrate the content, pedagogical,
  and professional knowledge, skills, and
  dispositions necessary to help all students
  learn.

24
Candidates

• Assess student learning
• Use of assessments in instruction
• Develop meaningful learning experiences for
  students based on their developmental levels
  prior experience.

25
(No Transcript)
26
Assessments

• Student work samples
• Case studies analyzing student learning
• Portfolio tasks on student assessments
• Student teaching and internship evaluations
• Etc.

27
(No Transcript)
28
Required Evidence

• Admissions Data
• Internship assessment data
• Licensure test data
• Follow-up studies

29
Performance data must be summarized and sampled

• Time does not allow BOE to view each candidates
  work- present data
• The unit is responsible for making links between
  evidence and standards
• BOE teams are making judgements about the unit
  and its programs, not individual students.

30
Which data will make your case?

• Assessments used by faculty in courses.
• Assessments used at transition points.
• Admissions
• Before entry into student teaching
• Student teaching
• Program completion

31
Program Reviews as Evidence of Meeting Standard 1
National Reviews by SPAs (Specialized Professional
Associations)
State Reviews by the State Agency Responsible for
Program Approval
32
(No Transcript)
33
New NCATE Program Review Process
34
Principles for the Reform of the Program Review
Process

• Continue to rely on expertise of SPAs
• Reduce costs for SPAs by transferring management
  to NCATE
• Reduce costs and burdens for institutions
• Consistency across SPAs
• Closer alignment to NCATE Standard 1

35

• Web-based program report and SPA report

36
First, the basics.

• Which institutions have to do NCATE program
  reports (based on state partnerships)
• NCATE terminology
• What programs are covered by NCATE standards
• When reports are due

37
Terminology

• SPA Specialized Professional Association that
  develops standards and conducts program reviews
• Program Report Online report submitted to NCATE
  by institution
• NCATE Program Recognition Report Report by
  which NCATE will convey the SPAs decision on
  national recognition

38
Program Recognition vs. Accreditation

• A program is awarded national recognition when it
  meets SPA standards and is accredited by NCATE
• An institution receives accreditation when it
  meets the NCATE unit standards
• The scope of NCATE accreditation includes all
  P-12 education programs

39
NCATE has standards for the following programs

• Early Childhood Education
• Educational Communications and Instructional
  Technology
• Educational Computing and Technology
• Educational Leadership
• Elementary Education (Initial)
• English as Second Language Education (Initial)
• English Language Arts Education (Initial)
• Foreign Languages Education (Initial)

• Health Education (Initial)
• Mathematics Education (Initial)
• Middle School Education
• Physical Education
• Reading Education
• School Library Media
• School Psychology
• Science Education (Initial)
• Social Studies Education (Initial)
• Special Education
• Technology Education (Initial)
• Initial standards are for programs leading to
  first teaching license

40
NCATEs New On-Line Program Review System
41
QUESTIONS FOR WHICH PROGRAM REVIEWS SHOULD FIND
ANSWERS

• Have candidates mastered the necessary knowledge
  for the subjects they will teach or the jobs they
  will perform?
• Do candidates understand teaching and learning
  and can they plan their teaching or fulfill other
  professional education responsibilities?
• Can candidates implement their conceptual plan
  with students and colleagues?
• Can candidates apply their knowledge in
  classrooms and schools?
• Do candidates meet state licensure requirements?

42
Components of the New Program Report

• Cover sheet
• Section I Contextual information
• Section II Assessments
• Section III Standards Assessment Chart
• Section IV Evidence of Meeting Standards
• Section V Use of Assessment Results to Improve
  Performance

43
Section IContextual Information

• Submit descriptions of the following
• Relevant state or institutional policies and
  practices affecting the program
• Field and clinical experiences
• Admission criteria, including GPA requirements
• The relationship of the program to the units
  conceptual framework
• The relationship of assessments used in the
  program to the units assessment system

44
Section I (continued)

• Attach the following
• The program of study
• Table with number of candidates and completers in
  the program
• Table on faculty expertise and experience in the
  specialty field and in schools

45
Section II Assessments Related Data
When Admin- istered
Scoring Guide/ Criteria
Type of Assess- ment
Name of Assessment
Data Table
Assess- ment
Licensure Test
1. 2. 3. 4. 5. 6. 7. 8.
? ? ? ? ? ? ? ?
? ? ? ? ? ? ? ?
? ? ? ? ? ? ? ?
46
6-8 Assessments The Rules

• Institution must submit a minimum of six
  assessments, unless the SPA specifies more than
  six required assessments
• Institution may submit additional assessments
  when SPA does not specify all eight assessments
• Five specific types of assessments are required
  by all SPAs

47
Section II Assessments Related Data
When Admin- istered
Scoring Guide/ Criteria
Type of Assess- ment
Name of Assessment
Data Table
Assess- ment
? ? ? ? ? ? ? ?
1. Praxis II-Elem Ed 2. Comprehensive Ex 3.
Lesson Plan Assign 4. Student Tchng Eval 5.
Student Work Sample 6. Case Study 7.
Diversity Project 8. IEP Project
? ? ? ? ? ? ? ?
? ? ? ? ? ? ? ?
? Test ? Exam ? Portfolio ? Eval ? Work
Sample ? Case ? Portfolio ? Portfolio
?Before ST ?End of Prog ?ED 320 ?During
ST ?During ST ?ED 330 ?ED 330 ?SPED 310
48
Section II Assessments Related Data
When Admin- istered
Scoring Guide/ Criteria
Type of Assess- ment
Name of Assessment
Data Table
Assess- ment
1. Praxis II-Elem Ed 2. Comprehensive Ex 3.
Lesson Plan Assign 4. Student Tchng Eval 5.
Student Work Sample 6. Case Study 7.
Diversity Project 8. IEP Project
? ? ? ? ? ? ? ? ?
? Test ? Exam ? Portfolio ? Eval ? Work
Sample ? Case ? Portfolio ? Portfolio
?Before ST ?End of Prog ?ED 320 ?During
ST ?During ST ?ED 330 ?ED 330 ?SPED 310
? ? ? ? ? ? ? ? ?
? ? ? ? ? ? ? ? ?
49
Examples of Tables for Aggregating Assessment
Data
50
Teacher Candidates in Science (45 Candidates)
51
Teaching Performance During Internship Final
Evaluation  
 
52
Effectiveness of the Program for the Initial
Preparation of Teachers in Developing Candidate
Knowledge, Skills, and Dispositions Scale 1
strongly disagree 2 moderately disagree 3
disagree 4 agree 5 moderately agree
6strongly agree
53
Section III Alignment of SPA Standards
Assessments
ALIGNMENT with UNIT STANDARD 1
SPECIALTY STANDARD
RELATED ASSESSMENTS
? Content ? Professional Pedagogical KSD ?
Student Learning
Standard 1 Standard 2
?1 ?2 ?3 ?4 ?5 ?6 ?7 ?8
? Content ? Professional Pedagogical KSD ?
Student Learning
?1 ?2 ?3 ?4 ?5 ?6 ?7 ?8
54
SECTION IV Organization of Proficiency Areas

• 1. Content knowledge
• 2. Pedagogical and professional knowledge,
  skills and dispositions
• 3. Effects on student learning

55
Section IV Write the following in 2 pages or less

• Indicate the assessment(s) from Section II that
  provides information about the area and describe
  how it addresses the applicable program
  standards.
• 2. Summarize the data presented in the table(s)
  related to the assessment submitted in Section II
  and interpret the results in terms of the
  standards.

56
1 (Required)-CONTENT KNOWLEDGE

• Data from licensure tests or professional
  exam-inations of content knowledge.

57
2 (Required)-CONTENT KNOWLEDGE

• Assessment of content knowledge in the
  discipline to be taught, teaching field, or other
  education professional field.

58
3 (Required)-PEDAGOGICAL PROFESSIONAL
KNOWLEDGE, SKILLS, AND DISPOSITIONS

• Assessment that demonstrates candidates can
  effectively plan classroom-based instruction, or
  fulfill professional responsi-bilities in other
  pro-fessional education roles.

59
4 (Required)- PEDAGOGICAL PROFESSIONAL
KNOWLEDGE, SKILLS, AND DISPOSITIONS

• Assessment that demonstrates candidates'
  knowledge, skills, and dispositions are applied
  effectively in clinical practice.

60
5 (Required)-EFFECTS ON STUDENT LEARNING

• Assessment that demonstrates candidate
  effects on student learning, or on providing
  supportive learning environments for student
  learning.

61
6 (Required)

• Assessment that demonstrates candidates meet
  standards.
• Examples of assessments include evaluations of
  field experiences, case studies, portfolio tasks,
  licensure tests not reported in 1, and follow-up
  studies.

62
7 (May be Optional)

• Assessment that demonstrates candidates meet
  standards.
• Examples of assessments include evaluations of
  field experiences, case studies, portfolio tasks,
  licensure tests not reported in 1, and follow-up
  studies.

63
8 (May be Optional)

• Assessment that demonstrates candidates meet
  standards.
• Examples of assessments include evaluations of
  field experiences, case studies, portfolio tasks,
  licensure tests not reported in 1, and follow-up
  studies.

64
Examples of SPA-specific Assessments
65
Example of a Unique SPA Assessment (ISTE)

• A hardware/software needs assessment survey in
  which the candidate gives the survey, analyzes
  the data collected, develops a budget for
  creating the environment using the suggested
  hardware/software, and develops a multimedia
  presentation that conveys the results to the
  educational community

66
Example of a Unique SPA Assessment (NSTA)

• Research assessmentrequiring candidates to
  design and conduct data on a problem in science,
  collect, process and report data, and report the
  research results in a manner consistent with
  procedures of the scientific community.
  Demonstrated practical experience in research may
  meet this requirement.

67
Example of a Unique SPA Assessment (ACTFL)

• Assessment that demonstrates candidates engage in
  professional development opportunities that
  strengthen their competencies, promote
  reflection, and enable them to become advocates
  for foreign language study

68
Science Assessment

• An assessment that demonstrates candidate
  knowledge of safety in the classroom, laboratory,
  preparation and storage areas and field (9b) of
  legal liability (9b) and of the use and care of
  living things (9c). This could include a score on
  a composite test covering these areas portfolio
  data or performance in a seminar specifically
  covering these topics.

69
Section VUse of Assessment Results

• Evidence must be presented in this section
  that assessment results have been analyzed and
  have been or will be used to improve candidate
  performance and strengthen the program.

70
(No Transcript)
71
Alignment of Program Review with Standard 1

• Content

• Rubric elements 1-2

Professional Pedagogical Knowledge, Skills,
Dispositions
Rubric elements 3-5
P-12 Student Learning
Rubric elements 6-7
72
Features of Program Report

• Limited to 35 pages of text plus specific
  attachments for each of the 6-8 assessments
• Consistent categories of evidence across programs
• Common report formats from institutions SPAs
• Will not include samples of candidate work

73
When are program reports due?

• Under old process 2-3 semesters before the
  NCATE visit
• Under new process one semester before the NCATE
  visit (if seeking first accreditation, two
  semesters before the visit has been proposed)

74
What if . . .

• the program is new or in redesign?
• the program doesnt have any candidates in the
  pipeline?
• the program is (very) small?
• the program doesnt lead to licensure or
  certification?
• the program is an endorsement or add-on?
• there are no NCATE standards for the program?

75
What Decisions Can Reviewers Make?

• The SPA reviewers will make one of the
  following decisions based on your program
  report
• The program is nationally recognized.
• The program is nationally recognized with
  conditions.
• A decision is deferred, pending submission of
  additional or clarifying information.
• The program is not nationally recognized a new
  program report may be submitted.

76
The NCATE Report on National Recognition

• Part A Recognition Decision
• A.1 SPA Decision
• A.2 Test Results
• A.3 Summary of Strengths
• Part B Status of Meeting the SPA Standards

77
The NCATE Report on National Recognition (cont.)

• Part C Evaluation of Program Report Evidence
• C.1 Content Knowledge
• C.2 Pedagogical Professional
• Knowledge, Skills,
• Dispositions
• C.3 Effects on Student Learning
• Part D Evaluation of the Use of Assessment Data

78
The NCATE Report on National Recognition (cont.)

• Part E Areas for Improvement
• Part F Additional Comments
• E.1 Comments on Context, etc.
• E.2 Concerns for Possible Follow-
• up by the BOE Team
• Part G Directions for Preparing a Rejoinder

79
Use of National Recognition/ State Approval by
BOE Teams

• An area for improvement will be cited for each
  program that is not nationally recognized or does
  not have full state approval.
• UAB may remove AFI if program has been recognized
  after the BOE visit, but before the UAB meeting.

80
Management by NCATE
Institutions submit electronic program reports
approximately six months before the visit. (One
year for new institutions.)
NCATE notifies three SPA reviewers that program
reports are available.
Each reviewer prepares a report with a
recognition recommendation.
81
Lead reviewer compiles the three reviews into
one report with a recommendation on national
recognition.
Single SPA report is electronically submitted by
lead reviewers to NCATE.
NCATE notifies institutions that their SPA
reports are available.
82
When the three reviewers can not agree on the
recognition decision, a SPA Audit Committee makes
the recognition decision.
Institutions may electronically submit a
rejoinder to their SPA reports if a program has
not been nationally recognized.
BOE team accesses SPA report for the on-site
visit.
83
Timeline for Implementation

• September 2004

• Pilot testing of new program review system

• Submission of
• program reports
• delayed by 5-7
• months to next cycle

• Program reports
• due September
• 2004 to February
• 2006

84
What if . . .

• the program is new or in redesign?
• the program doesnt have any candidates in the
  pipeline?
• the program is (very) small?
• the program doesnt lead to licensure or
  certification?
• the program is an endorsement or add-on?
• there are no NCATE standards for the program?

85
What ifs

• What if the program is offered at different
  degree/award levels?
• What if the same program is offered at different
  campus sites?
• How much data do we need?
• What if our current program assessments are not a
  close match to the 6-8 SPA assessments?

86
Next Steps

• Templates available on the web in September 2004.
• Piloting the new process in fall 2004 and spring
  2005

87
What does the BOE team see?

• In the exhibit room (paper or electronic)
• Program Reports for SPAs or states
• State or SPA Reports of Findings

88
Who to Contact

• Wendy Wiggins, Director of Program Reviews,
  wendy_at_ncate.org
• Margie Crutchfield, Associate Vice President for
  Program Reviews, margie_at_ncate.org

89
NCATEs Websitewww.ncate.org
Another Resource

90
(No Transcript)
91
(No Transcript)
92
(No Transcript)
93
The Electronic Submission and Review of Program
Reports

• NCATEs Accreditation Information Management
  System (AIMS)

94
Upon logging in, the system brings you to your
own summary home page. Different shared work
areas, called communities in AIMS, are setup for
each group of users each Institution, each SPA,
each Program under review, and for NCATE staff.
All the forms, reports, documents and
instructions you need are found in the
appropriate communities. You can only go to
shared work areas for the group(s) you belong to.
If you belong to multiple groups, your home page
shows a list of your communities with links, so
you can go to any of them.
95
(No Transcript)
96
(No Transcript)
97
(No Transcript)
98
The multi-paneled Program Report Submission
screen allows the Institution user to fill out a
series of forms, displayed in the same format as
the printed version of the form. The system
treats all sections as part of a single report.
99
Questions for the Program Report are determined
dynamically from NCATEs file of each SPAs
unique program requirements.
100
A SPAs home page a shared workspace for all of
a SPAs AIMS-related activities
Reference documents and Directory of institution
contacts who share this workspace, with links to
email them, and shared calendar.
101
Other Accreditation Issues

• Revision of Standards
• Decisions of Other Accrediting Agencies (e.g.,
  CACREP or ASHA)
• Accreditation Information Management System
  (AIMS)
• Distance Learning

102

• Dont stop compiling data for your NCATE visit
  even if you just had a visit.