Depth of Knowledge

1
Depth of Knowledge

• a professional development session prepared by
• PREPS, Inc.
• November 2, 2005

2
PREPS

• GUIDE TO SERVICES PRODUCTS-2005
• Regional workshops free to member districts
• Targeted assistancecustomized for districts
• New partner with MDE on
• Mattie T Technical Assistance Center
• MDE Facilitation for ICAI and TST Three tier
  training

3
Our children are smarter than we think. We must
give them more responsibility as young people and
then expect more of them as they grow and
develop. Secretary of Education Richard
Riley State of American Education Address
February 18,1997
4
Session Goals are to

• Create knowledge base of definition of Depth of
  Knowledge (DOK)
• Increase awareness of alignment of MS state tests
  to DOK
• Establish a common vocabulary of key terms
• Promote understanding of critical linkages across
  curriculum standards, instruction, classroom
  assessment and large scale assessments (MCT,
  SATP, Writing)

5
The Changing Role of the Classroom Teacher
Teacher
Content Instructor
Reflective Practitioner
Facilitator of Learning
Teacher of Reading
Diagnostic Evaluator
6
Why Analyze the Data?

• Right thing to do
• Valuable teaching and learning tool
• Accountability
• State
• Local
• National (NAEP)
• Professional Credibility-
• Ability to communicate successfully with
  colleagues, students, parents, public, and
  policymakers

7
Begin with the End in Mind

• - Steven Covey
• Each and every student here has potential for
  greatness.
• Each deserves an equal opportunity to learn.
• Which student do you want as your doctor, pilot,
  your grandchildrens teacher?

8
Thats good, because education paysAnnual
earnings of 25-34 year-olds by attainment, 2001
Source U.S. Bureau of Labor Statistics and
Bureau of the Census, Current Population Survey,
March 2002.
9
Unfortunately, when these new freshmen arrive in
college, many must take remedial (high-school)
level courses
10
Big gap between what students, teachers think is
necessary and what colleges and employers need

• While parents, students and teachers continue to
  believe in the diplomas value, those who judge
  the quality of high school graduates most closely
  first-year college professors and employers
  express strong skepticism.

11
Both professors and employers have similar worries
Percentage of employers and professors rating
graduates skills as fair or poor

• Most employers and professors question whether
  high school graduates have the knowledge and
  skills required on the job or in the college
  classroom.

12
Mississippi vs. NAEP Proficiency Levels
13
(No Transcript)
14
(No Transcript)
15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
Increased rigor has major benefitsA strong h.s.
curriculum improves college completion and
narrows gaps
11
28
Completing at least Algebra II plus other
courses. Source Adapted from Adelman, Clifford,
U.S. Department of Education, Answers in the
Toolbox, 1999.
19
Increased rigor in the curriculum offerings for
this state has major benefits

• Better prepared students for higher level
  coursework
• Better prepared students for the postsecondary
  experience
• Better prepared students for workforce and
  technical related occupations

20
Monster.comWhat did YOU want to be?
21

• Teach Different

22
Reflection
Without reflection, it is possible to teach a
single year 25 times over.
23
Conceptual Model of Focused Instruction
Curriculum Frameworks and Test Alignment
NAEP

• MCT
• SATP

Mississippi Curriculum Frameworks
National Professional Organizations (IRA/NCTE)
Thinking Skills --Depth of Knowledge
24
McRel findings

• 200 standards and 3,093 benchmarks for 14
  different subject areas in national and state
  level documents
• Average time to teach these standards and
  benchmarks is 15,465 hours. Average school year
  is 180 days with 5.6 hours /day of instructional
  time
• Not all class time is used. Range of actual time
  used for instruction in a classroom is 21-69.

25
McRel findings (contd)

• If the high of 69 is used to determine amount of
  instruction that actually takes place in a given
  school year, the total of instructional hours
  would be 9,042.
• 15,645 hours are needed to teach all of the
  standards, yet best case scenario, teacher would
  have only 9,042 hours.
• In order to address all of the standards,
  students would have to go to school K-grade 21!

26
DOK Glossary Prepared by PREPS, Inc.

• Alignment
• Benchmarks
• Test Blueprint
• Disaggregated Data
• Skill
• Objective
• Proficiency Level

• Standards
• Cognitive Demand
• Opportunity to Learn
• Opportunity Gap
• Differentiated Instruction
• Strategic Extended Thinking
• Universal Design

27
Increased rigor may be implemented in a number of
ways there can be differences in how the content
can be tested or taught.

• In the present Mississippi Mathematics
  Curriculum Framework, students in the 5th grade
  are asked to identify, describe, compare and
  classify geometric figures.
• To master this skill, students would identify
  rectangles, triangles, circles, and other
  geometric shapes. They would have to describe
  them, compare them, and know what characteristics
  are used to group them in classifications.

28
Specific Example of Adding Rigor to the MS
Curriculum Frameworks

• In the revised Mississippi Mathematics (2006)
  Curriculum Framework, students must take that
  knowledge further by justifying why a shape is a
  rectangle or triangle by understanding the
  properties of the figures and how these figures
  can be described using coordinate geometry.

29
Why do we care about cognitive demand?

• Standards are big ideas!!!
• Modern theories of learning focus on developing
  deep understanding to facilitate transfer.
• Students cannot develop deep understanding unless
  they are provided opportunities on both learning
  and assessment tasks.

30
Analyzing Cognitive Demand Adapted from Webb,
Norman L., (1999). Alignment of Science and
Mathematics Standards and Assessments in Four
States.

• Higher Demand
• Strategic Thinking
• Extended Thinking
• Lower Demand
• Recall
• Basic Application of Skill/Concept

31
Blooms Taxonomy-the classic

• Knowledge usually rote memory of factual
  material
• Comprehension involves translating from one
  level of abstractions to another (e.g., give an
  original example of a concept or principle).
• Application - the ability to use learned material
  in novel situations
• Analysis - the ability to break down material
  into its component parts so that its
  organizational structure may be understood (e.g.,
  distinguishing fact from opinion)
• Synthesis - the ability to put parts together to
  form a new wholeusually involved producing a
  product.
• Evaluation - the ability to judge the value of
  material (statement, novel, poem, research
  report) for a given purpose. This is like
  analysis but with a logical and defensible value
  judgment

32
Depth of Knowledge Levels(Webb, 1999)

• Recall-recall of a fact, information, or
  procedure
• Basic Application of Skill/Concept-Use of
  information, conceptual knowledge, procedures,
  two or more steps, etc.
• Strategic Thinkingrequires reasoning, developing
  a plan or sequence of steps has some complexity
  more than 1 possible answer generally takes less
  than 10 minutes to do.
• Extended Thinkingrequires an investigation time
  to think and process multiple conditions of the
  problem or task and more than 10 minutes to do
  non-routine manipulations.

33
Expectations for Student Performance
Acquire
Use
Extend
Skill/Concept
Recall
Strategic Thinking
Extended Thinking
Memorize
Perform Procedures
Demonstrate Understanding
Conjecture, Generalize Prove
Solve non- routine/ make connections
Memorize
Conduct Investigations
Communicate Understanding
Analyze Information
Apply concepts /make connections
Recall
Demonstrate /Explain
Analyze/ Investigate
Evaluate
Generate /Create
34
Depth of Knowledge

• Two factors
• 1. Sophistication and complexity. Sophistication
  will depend on the abstractness of the activity,
  the degree to which simple knowledge and skills
  have to be recalled or drawn upon, the amount of
  cognitive processing required, the complexity of
  the content concepts used, the amount of content
  that has to be recalled or drawn upon, the lack
  of routine, and the need to extend knowledge
  meaningfully or produce novel findings.

35
Depth of Knowledge

• 2. Prior instruction. Test items that address
  complex knowledge can still have a low depth of
  knowledge level, if the required knowledge is
  commonly known and students with normal
  instruction at a grade level should have had the
  opportunity to learn how to routinely
  (habitually) perform what is being asked

36
Depth of Knowledge

• Level 1. Recall and Reproduction
• This level represents recall of information such
  as a fact, definition, term, or a simple
  procedure, as well as performing a simple process
  or procedure. Level 1 only requires students to
  demonstrate a rote response, use a well-known
  formula, follow a set procedure (like a recipe),
  or perform a clearly defined series of steps. A
  simple procedure is well-defined and typically
  involves only one step.

37
Depth of Knowledge

• Level 1. Recall and Reproduction - Examples
• Language Arts Which of these means about the
  same thing as the word exacerbate?
• Science What is the process called which plants
  use to manufacture sugar from sunlight?
• Mathematics Which of the following numbers,
  when rounded to the nearest thousand, becomes
  90,000?
• Social studies What was the main reason many
  leaders in Great Britain leaned toward supporting
  the Confederacy in the Civil War?

38
Depth of Knowledge

• Level 2. Basic Reasoning, Using Skills and
  Concepts
• This includes the engagement of some mental
  processing beyond recalling or reproducing a
  response. The content knowledge or process
  involved is more complex than in level 1. Items
  require students to make some decisions as to how
  to approach the question or problem. In science
  and mathematics, keywords that generally
  distinguish a Level 2 item include classify,
  organize, estimate, make observations,
  collect and display data, and compare data.
  These actions imply more than one step.

39
Depth of Knowledge

• Level 3. Complex or Strategic Thinking
• This level requires reasoning, planning, using
  evidence, and a higher level of thinking than the
  previous two levels. Students would go beyond
  explaining or describing how and why to
  justifying the how and why through application
  and evidence. The cognitive demands at Level 3
  are complex and abstract. The complexity does not
  result only from the fact that there could be
  multiple answers, a possibility for both Levels 1
  and 2, but because the multi-step task requires
  more demanding reasoning.

40
Depth of Knowledge

• Level 3. Complex or Strategic Thinking
  Examples
• Language Arts The style in this passage is
  characterized by similes like these smooth as
  pudding, crisp as a fresh apple, and rough as
  shredded coconut. Which of these best explains
  the authors purpose in using these similes?
• Science In a laboratory experiment, an enzyme
  is combined with its substrate at time zero. The
  absorbance of the resulting solution is measured
  at five-minute intervals. In this procedure, an
  increase in absorbance is related to the amount
  of product formed during the reaction. The
  experiment uses three preparations. (Students
  will see illustrations.) The most likely reason
  for the failure of the absorbance to increase
  significantly after 10 minutes in preparation III
  is that . . . . (multiple choice options)

41
Depth of Knowledge

• Level 3. Complex or Strategic Thinking Examples
• Mathematics Julie wants to fence in an area for
  her dog. She can afford only 36 feet of fencing.
  She is considering various shaped for the fenced
  in area, but she wants the shape to have four
  sides that are whole numbers lengths and contain
  four right angles. What is the largest area that
  Julie can enclose with 36 feet of fencing?
• Social Studies A newspaper prints a story that
  criticizes the current administrations policies.
  The Bill of Rights allows a government official
  to respond to this headline by . . . . (multiple
  choice options)

42
Depth of Knowledge

• Level 4. Extended Thinking or Reasoning
• This level represents tasks with high cognitive
  demands and that are very complex. Students are
  required to make several connectionsrelate ideas
  within the content area or among content
  areasand have to select or devise one approach
  among many alternatives on how the situation can
  be solved. Many on-demand assessments will not
  include any assessment activities that could be
  classified as Level 4. However, competencies and
  objectives can be stated in such a way as to
  expect students to perform extended thinking.
  Also, classroom assessments could require Level 4
  tasks.

43
Depth of Knowledge

• Level 4. Extended Thinking or Reasoning
  Examples
• Language Arts Your teacher has given you an
  assignment to write an informative essay.
  (Generation of a product planning, writing,
  editing)
• Science Your teacher has asked you to design and
  conduct a science investigation. (Generation of a
  product data collection, display,
  interpretation)

44
Depth of Knowledge

• Level 4. Extended Thinking or Reasoning
  Examples
• Mathematics Students are asked to identify a
  real world problem that requires the
  application of mathematics, describe the possible
  procedure(s) for solving, and explain the outcome
  and their reasoning.
• Social Studies Students are given the scenario
  of acid rain potentially causing problems in a
  specific farming community. Students are to
  define and describe the problem with supporting
  data, propose alternative solutions to the
  problem, select one solution, and explain why it
  would be the best.

45
Depth of Knowledge

• Assessments must be demanding cognitively as what
  students are expected to know and do as stated in
  the curriculum frameworks. This is one aspect of
  the alignment between assessments and curriculum.
• Each test question does not have to have
  precisely the same depth-of knowledge level as
  the corresponding competency or objective.
  However, the majority of test items on an
  assessment must correspond to the most common
  depth-of-knowledge level for competency and
  related objectives.

This information is adapted from Webb, Norman L.,
Research Monograph No. 8, Criteria for Alignment
of Expectations and Assessments in Mathematics
and Science Education, Council of Chief State
School Officers, 1997.
46
More than Magic!

• Diagnosing learning characteristics of students
  is one of a teachers greatest challenges, BUT
• Tackling the problem is a moral obligation that
  we as educators have.

47

Tell me I forget Show me I remember Involve
me I understand.
48

• What can we do to ensure that all students are
• prepared with the appropriate depth of knowledge
• to be successful beyond grade twelve?
• Analyze cognitive demand for district, and state
  instruction and assessment standards
• Make appropriate adjustments to address range of
  cognitive
• Choose essential standards to make it easier to
  move to challenging depths of knowledge.

49
Opportunity to learn has the strongest
relationship with student achievement of all
school level factors identified. (What
Works in Schools, Marzano, 2003.
50

• One of the most overlooked instructional tools is
  the CLASSROOM assessment.

51
It Only Happens WhenAll the Stars are Aligned
52
Data tell what isWe determine what will be.