Using K-12 Assessment Data from Teacher Work Samples as Credible Evidence of a Teacher Candidate - PowerPoint PPT Presentation

Loading...

PPT – Using K-12 Assessment Data from Teacher Work Samples as Credible Evidence of a Teacher Candidate PowerPoint presentation | free to download - id: 3fe852-NmZjN



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Using K-12 Assessment Data from Teacher Work Samples as Credible Evidence of a Teacher Candidate

Description:

Our Research Questions What are the gains in K-6 student achievement ... The Context of Teacher Work Sample ... (Controlling for Student Teacher Effects) ... – PowerPoint PPT presentation

Number of Views:159
Avg rating:3.0/5.0
Slides: 31
Provided by: TonyN73
Learn more at: http://www.wku.edu
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Using K-12 Assessment Data from Teacher Work Samples as Credible Evidence of a Teacher Candidate


1
Using K-12 Assessment Data from Teacher Work
Samples as Credible Evidence of a Teacher
Candidates Ability to Produce Student Learning
  • Presented by
  • Roger Pankratz
  • Tony Norman
  • Judy Pierce
  • of
  • Western Kentucky University
  • Presented at
  • The Making an Impact Conference
  • Kennesaw University
  • March 2007

2
Our Leading Question
  • Can P-12 assessment data on teacher work samples
    produced by student teachers provide credible
    evidence that Westerns graduates have the
    ability to impact learning with the students they
    teach?

3
Our Hypothesis
  • Work sample data can provide credible evidence of
    ability to impact learning provided unit learning
    objectives and assessments to measure learning
    meet credibility standards.

4
Our Research Questions
  1. What are the gains in K-6 student achievement
    reported by Westerns student teachers on teacher
    work samples?
  2. What is the quality of assessments and degree of
    alignment between unit objectives and assessments
    used in student teachers work samples at
    Western?
  3. To what extent do unit objectives in Westerns
    student teacher work samples address Kentuckys
    core content standards?

5
Our Rationale for Exploring the Use of Work
Sample Data as a Measure of Impact
  • Renaissance Partnership work (Denner et al.,
    2004) demonstrating relationship between overall
    TWS performance and quality of learning
    assessments
  • Institutional work (e.g., Idaho State and
    Longwood) suggesting that P-12 scores within the
    TWS could be used to ascertain impact on learning

6
What is a Teacher Work Sample?
  • Provides documentation of a teacher candidates
    ability to plan, deliver, and assess a complete
    standards-based unit of instruction
  • Provides instruction embedded evidence of the
    impact of teaching on student learning

7
Validity Reliability Evidence
  • Processes parallel targeted INTASC and Kentucky
    Teacher Standards
  • Experts affirm that processes describe
    behaviors/skills that are frequent, critical, and
    necessary for good teaching.
  • Multi-institutional studies (Denner et al., 2004)
    show that TWS can be scored reliably
  • Two WKU studies show between 81-90 inter-rater
    agreement. An additional study showed between
    67-95 agreement (faculty tended to score
    higher) but some training discrepancies were
    present.

8
The Context of Teacher Work Sample Use at Western
Kentucky University
  • Western Kentucky University has about 400 student
    teachers each year
  • 300 K-5
  • 100 6-12
  • Western has required TWS as a performance
    assessment in student teaching since 2001.

9
The Context of Teacher Work Sample Use at Western
Kentucky University
  • Student teachers upload electronic versions of
    their teacher work samples on the College of
    Educations data management system.

10
The Context of Teacher Work Sample Use at Western
Kentucky University
  • Western convenes teacher educators and school
    practitioners each year to independently score
    TWSs (semi-holistic and holistic).
  • University student teacher supervisors initially
    score TWSs for student teaching evaluation.
  • Performance data of TWSs are used for program
    improvement.

11
The Seven Teaching Processes Assessed in
Westerns Teacher Work Sample
  1. Use of teaching context
  2. Unit learning objectives
  3. Assessment plan
  4. Instructional design
  5. Instructional decision making
  6. Analysis of learning results
  7. Reflection on teaching and learning

12
The Design of our Initial Study of Teacher Work
Sample Assessment Data, Quality of Assessments
and Quality of Unit Objectives
  • Analysis of reported pre-post assessment student
    gains in 22 student teacher work samples produced
    in fall of 2006 (460 students)
  • Consensus rating of the quality of assessments
    developed and administered by each student teacher

13
The Design of our Initial Study of Teacher Work
Sample Assessment Data, Quality of Assessments
and Quality of Unit Objectives
  • Description of assessments used for each unit
    learning objective number and type (short answer,
    multiple choice, true/false, open response,
    matching)
  • Quick and dirty rating of the quality of
    instructional objectives
  • Alignment with Kentuckys content standards
  • Importance of objective related to content
    standards

14
Study Results Impact on Student Learning
Student Teacher N 22, P-12 Student N 464
Significant Student Gain (Controlling for Student
Teacher Effects)
15
Study Results Quality of Assessments Rubric
16
Study Results Quality of Assessments
  • All TWS had two levels of DOK with most having a
    lower level goal (DOK 1,2) and higher level goal
    (DOK 3,4).
  • Total QA Scores ranged from 9 to 15 (M 11.89,
    SD 1.73).
  • Holistic QA Scores Frequencies 2 14 (64),
    3 3 (27), and 4 2 (9)
  • No significant relationships between Faculty
    Holistic TWS Score and either Total QA Scores
    (Pearson r -.25) or Holistic QA Scores (r
    -.18)
  • No significant relationships between Total Gain
    Scores and either Total QA Scores (Pearson r
    .34) or Holistic QA Scores (r .31)

17
Study Results Description of Assessments for
10 Work Samples Selected at Random
1 Kindergarten 3 Science
3 Grade 1 4 Math
2 Grade 3 2 Health
2 Grade 4 1 Social Studies
2 Grade 6
18
Study Results Description of Assessments
  • Average number of assessment items per unit of
    instruction 15.5
  • Range of assessment items for a unit of
    instruction 8 30
  • Average number of assessment items per
    instructional objective 5
  • Range of assessment items for an instructional
    objective 1 - 13

19
Study Results Description of Assessments
  • Types of Assessments Used Overall
  • Short answer 53
  • Multiple choice 48
  • Matching 24
  • True/False 17
  • Open Response 13

20
Study Results Description of Assessments
Types of assessments used by unit objective Depth of Knowledge (DOK) Types of assessments used by unit objective Depth of Knowledge (DOK) Types of assessments used by unit objective Depth of Knowledge (DOK) Types of assessments used by unit objective Depth of Knowledge (DOK) Types of assessments used by unit objective Depth of Knowledge (DOK)
DOK1 DOK2 DOK3 DOK4
Short Answer 15 25 5 8
Multiple Choice 14 14 20 0
Matching 6 13 5 0
True/False 7 5 1 4
Open Response 3 5 4 1
21
Study Results Quality of Objectives
Alignment with Kentucky Core Content 4 0 3 4 2 3 1 2
Importance of objective relative to standards 4 1 3 7 2 1 1 0
22
CONCLUSIONS
  1. TWS assessment data have potential for
    credibility provided standards for instructional
    objectives and assessments are addressed.
  2. TWS show impressive average student gains but
    there is a large variability by individual
    student.

23
CONCLUSIONS
  1. The quality of assessments used looks promising
    but needs improvement.
  2. The number of assessment items per objective for
    the ten work samples reviewed is too small for
    any degree of reliability.

24
CONCLUSIONS
  1. The quality of instructional objectives for the
    ten work samples reviewed varies widely with a
    majority of objectives needing improvement in
    alignment and focus.
  2. More instruction is needed on matching
    assessments to objectives especially
    assessments for DOK 3 and 4 level objectives.

25
CONCLUSIONS
  1. With a sustained effort to provide instruction
    for teacher candidates in the development of high
    quality instructional objectives and assignments,
    data from work samples could be one source of
    evidence of teacher graduates potential for
    impact on student learning.

26
Next Steps
  • Conducting larger, more precise TWS studies to
    explore, such as
  • The extent to which all TWS scores can be
    accessed and matched to goals
  • The quality of the assessments (depth of
    knowledge, alignment to standards, adaptations to
    meet diverse needs of students)
  • The potential for differential impact based on
    student-teacher program and P-12 student
    characteristics
  • Working with state to connect TWS, as well as
    other program assessments, success to impact on
    P-12 student learning during teaching internship

27
Discussion Question 1
  • What are your reactions to and/or comments on
    Westerns exploration of teacher work sample data
    as one source of impact evidence?

28
Discussion Question 2
  • What suggestions do you have for next steps
    Western might consider in its effort to develop
    credibility in student teacher work sample data?

29
Discussion Question 3
  • What would you suggest as standards for
    assessments and learning objectives to improve
    credibility of work sample data?

30
Discussion Question 4
  • What actions should Western take to engage
    other institutions in development of work samples
    that produce credible data?
About PowerShow.com