Tennessee CTE Competency Attainment Rubric

1
Tennessee CTE Competency Attainment Rubric
Gay Burden, Ph.D. Tennessee CTE Summer
Conference\ July 2009
2

• Mans mind, once stretched by a new idea, never
  again regains its original dimension.
• Oliver Wendell Holmes

3
Old Way of Thinking Competency Profile Example
Standard 1.0 The student will analyze careers
in advertising and public relations.
Standard 2.0 The student will relate the
importance of advertising and public relations to
the marketing mix.
4
Its about

• Creating a valid and reliable instrument and
  putting the assessment process in the hands of
  teachers
• Affecting instructional change
• Raising expectations for CTE student success
• Providing a comprehensive description of
  proficiency
• Giving teachers an objective instrument for
  assessing students

5
Steps in Developing the Rubric

• Literature review
• Consultation REL - Appalachia, SREB/HSTW
• Developing a Vision Statement
• Alignment of proficiency with post-secondary and
  workplace readiness standards
• Defining proficiency and basic performance level
  descriptors

6
Understanding Why a
common vision

• Vision statement
• 2s1, Technical Skill Attainment
• The Dinosaur story

7
Rubric Vision Statement

• Course competency checklists in each CTE program
  of study will be modified to include four levels
  of performance at the end of each competency
• One common rubric will assist teachers in rating
  students on each competency in all CTE courses
• Rubric is grounded in the work of Norman Webbs
  Depth of Knowledge (DOK) Levels to help teachers
  assess proficiency levels based on the complexity
  of assignments/student work

Source Tennessee CTE Rubric Vision Statement
8
Purposes of the Rubric

• Facilitate student progression through a program
  of study
• Facilitate student progress into industry and/or
  post-secondary education
• Increase the reliability and validity of
  competency checklists
• Increase teacher consistency in using competency
  checklists and rating student competencies
• Generate electronic student and classroom data
  that can lead to instructional improvement

Source Tennessee CTE Rubric Vision Statement
9
The rubric is not

• Intended to be used with every assignment but
  rather a cumulative assessment instrument for the
  course competency checklist.

10
The rubric is

• Raising the level of expectations in CTE courses
  across the state
• Defining what it is to be a successful CTE
  student
• A tool to assess students and drive instructional
  improvement
• A way to increase consistency in completing the
  competency checklists across the state
• A part of the CTE plan to be a leader in high
  school redesign and the Tennessee Diploma Project

11

• When this rubric is placed in the hands of
  teachers, we want them to say, I wish I had this
  rubric last year because the competency checklist
  makes more sense and is more meaningful with the
  rubric.
• --Ralph Barnett, Assistant Commissioner

12
The Greatest Challenges Understanding the How?

• Change from a binary process (mastery vs.
  non-mastery) to addressing four levels
• Advanced
• Proficient
• Basic
• Below Basic
• Create a common understanding

13
Defining Proficiency Basic

• Proficient
• Meets post-secondary readiness standards.
• Meets career readiness standards at entry level.

• Basic
• Needs remediation at post secondary level.
• Requires some supervision/additional training to
  meet career readiness standards.
• Struggles using previous knowledge.

14
How would you define Proficiency?
15
How would you define Proficiency?
What is Proficient? Pair Share Create a
five-point bulleted list List one supporting
point for each category
16
New Way of Thinking Competency Profile Example
17
(No Transcript)
18
Class Roster Form Data Entry
19
RUBRIC
The Big Picture

• Rubric defines a
• successful CTE student
• Rubric is a cumulative
• assessment tool of
• student achievement
• of COMPETENCIES

COMPETENCIES

• Competencies define
• expectations in
• particular course
• Competencies drive
• instructional practices
• teacher assignments

EVIDENCE

• Student work
• Teacher assignment
• assessment instrument
• Teacher observation
• of Student
• Student behaviors
• Student progress

20
RUBRIC
COMPETENCIES
EVIDENCE
Quiz 2
Knowledge Attainment Technical Skills Problem
Solving Career Awareness Communication/Literacy
2.2 Know something 2.4 Show something
Exam 1
2.3 Do something well
In-class Work
Project 3
Improvement
Teamwork
21
The Bigger Picture
Instructional improvement occurs as teacher
assignments are aligned with competencies with
PROFICIENT in mind.
Competency profiles were developed prior to the
Rubric process. Future competency revisions and
development will align with RUBRIC.
RUBRIC
COMPETENCIES
EVIDENCE

• Student work
• Teacher assignment
• assessment instrument
• Teacher observation
• of Student
• Student behaviors
• Student progress

• Competencies define
• expectations in
• particular course
• Competencies drive
• instructional practices
• teacher assignments

• Rubric defines a
• successful CTE student
• Rubric is a cumulative
• assessment tool of
• student achievement
• of COMPETENCIES

22
Old vs. New Approach
23
Something old, something new

• There were no changes to the existing standards
  and competencies for each course
• There were changes to the format of the
  competency checklist, including formulas on the
  electronic version for data reporting

24
Competency Checklist
25
Common Understanding

• Rubric Terminology Glossary
• Authentic
• Technical
• Contexts
• Synthesize
• Emerging Technologies
• Disparate
• Multiple
• Technical
• Soft Skills
• Demonstrates
• Utilizes

26
Instructional Impact of Rubric

• Webbs DOK and rubric staff development

27
Webbs Depth of Knowledge (DOK)

• Created by Norman L. Webb, Ph.D.
  Wisconsin Center for Education Research
• A continuous scale of cognitive demand
• 4 levels of cognitive complexity

Source http//facstaff.wcer.wisc.edu/normw/
28
Webbs DOK

• Could be used with every assignment to raise the
  level of rigor in a course and help more students
  reach proficiency.
• Will be a focus of professional development based
  on data generated in the rubric.

29
Rubric Pilot 1 Participants

• 32 CTE Teachers representing 7 program areas
• 5 CTE Directors
• Geographic representation (E, M, W)
• Rural, Suburban, Urban
• Range in number of CTE teachers in the program
  (1-8)

30
Pilot 1 Spring 2009

• Apply the rubric during the remainder of the
  school year
• Continue assessing students using old form for
  CTE data reporting purposes (2S1) in 2008-2009
• Assess students using new form for pilot purposes
• Think about the differences between the old way
  vs. the new way of using competency profiles to
  assess student performance (bring thoughts to
  June 15-16 meeting)
• Complete an on-line survey to collect qualitative
  data based on pilot 1 participators experiences

31
Rubric Teacher Survey Results
32
Comments

• To me it makes the reporting easier since we now
  have a standard to compare student work to. If
  teachers are completely honest with it, it may
  prove more difficult to cover as many
  competencies.
• Much easier to determine competency.
• Much better as far as accountability goes. Not
  that it is easier, but being accountable is never
  easy.
• It allows for a better interpretation of
  mastery
• If you fill out competencies as you are teaching
  (either way) it is not hard to do either one.
  Many teachers are NOT going to like this new
  process b/c it is a lot of individual papers to
  keep up with (on computer or printed).

33
Pilot 1 Teacher Comments

• How did you use the rubric? What did your process
  look like?
• What evidence did you use to assess student
  proficiency based on the rubric?
• How did you feel about your experience using the
  rubric? (aha moments, unexpected experiences,
  etc.)

34
Rubric Teacher Survey Results
35
Comments

• Provides me with a clearer line of competency.
• This will definitely make my lessons more
  rigorous and meaningful. I'm ashamed to say how
  inept I feel as a lesson plan designer right now.
  My lessons are not challenging enough for the
  advanced or for the proficient student in some
  cases.
• It does help identify areas where I need to
  provide more opportunities for practice and
  achievement.

36
This too, shall pass!
37
Three Tier Change Process

• Stage 1 Initiate the changeIntroduce the
  innovation to the participants.
• Stage 2 Implement the changeApply the tools and
  techniques of the innovation.
• Stage 3 Institutionalize the changeEstablish
  accountability for continued use.

Michael Fullan
38
Rubric Teacher Survey Results
39
Rubric Teacher Survey Results
40
Comments

• Very much so - it made me focus more on the
  competencies and whether my assignments/assessment
  were truly showing mastery of these
  competencies.
• In some areas yes and others no. Depends on the
  specific competency and goal of the competency.
• I always do anyway but this makes it happen.
• I will need to link work more to projects rather
  than tests to show mastery. I need to increase
  my higher order thinking skills to application
  processes.

41
More Comments

• It forced me to look deeply into the standards
  and evaluate how completely I was covering them.
• Specifically, I found that I need to incorporate
  more requirements for reading/interpreting
  related materials into activities. The blue
  section on the rubric identified an area of
  weakness in its specific requirements of reading,
  interpreting, and analyzing. I will also consider
  revision of some of my existing project rubrics
  to incorporate more of the rigor and relevance in
  the overall rubric.

42
Steps to Improve Student Performance

• Visible standards--syllabus and classroom
• Exemplary work examples
• Student explanations of proficient
• Posted expectations
• Evaluation according to standards
• Redoing work that doesnt meet criteria

43
Competency/Rubric Data Tables

• State, District, School, Classroom Levels
• Sample Tables
• Data Training

44
Implementation Timeline

• January 20, Rubric Review Committee
• February 2, CTE Directors Information Session
• March 6, Pilot 1 Training (35 teachers, 5 CTE
  Directors)
• June 15-16, 2 Day Focus Group, Pilot 1
  Participants
• Summer 2009, CTE Conference
• Awareness Session for all program areas
• 1 day training for CTE Directors (TASL)
• September 2009, Pilot 2 Regional Training
• 45 LEA Teams (CTE Director and 2 CTE Teachers)
• June 2010, Focus Group, Pilot 2 Teacher Sample
• Summer 2010, Full implementation
• Focus of 2010 Summer Conference

45
Change

• Educational change depends on what teachers
  do and think. Its as simple and as complex as
  that.
• Ultimately, your leadership in a culture of
  change will be judged as effective or ineffective
  not by who you are but by what leadership you
  produce in others.
• Michael Fullan

46
QA
Send questions/comments togay.burden_at_tn.gov
47
Comments

• I say yes to this but we need to remember that
  training, implementation, and having teachers
  understand the flexibility of this will be key to
  success.
• One example I have used in the past is if you
  tie a floral bow - - -you can either complete the
  task in an appropriate fashion or you have a limp
  piece of ribbon.
• No, with the way I currently teach some
  competencies the rubric will not address
  assignments. This shows I will need to change
  some teaching techniques which is okay.
• I want to keep working with it to think about
  detail, but much better than what we have had in
  the past, which opens so much for debate, this
  will work better than a "test" and better than
  just saying that the material was covered.
• Most of the competencies are written so that
  students either demonstrated attainment in an
  assignment or daily practice or did not. The
  rubric makes this type of competency
  unnecessarily complicated to evaluate.