Assessment Rubric for Critical Thinking Academic Round Table Presentation

1
Assessment Rubric for Critical ThinkingAcademic
Round Table Presentation
2
Class Without CT

• Didactic instruction where students are presented
  with factual information from a text book
• Assessment is primarily multiple choice items
  where students are expected to regurgitate
  factual information

3
Class With CT

• Interactive learning environment where students
  not only learn facts but the relationship between
  the facts and the application of that information
• Authentic assessment where students are able to
  model the applications of the discipline through
  simulations, projects, etc.

4
Authentic Assessments

• Authentic assessments serve dual purposes of
• encouraging students to think critically and
• providing assessment data for measuring improved
  student learning.
• These assessment techniques fall into three
  general categories
• criterion-referenced rubrics,
• student reports (reflection or self-assessments),
  and
• student portfolios.

5
Assessment Rubric for Critical Thinking

• A global rubric template developed to provide a
  snapshot view of how student learning is being
  affected by the critical thinking initiative.

6
Assessment Rubric for Critical Thinking

• Designed to be flexible enough to address a
  number of student project modalities including
  written and oral communications.
• Will evaluate the students use of critical
  thinking skills in the development of the paper
  as opposed to specifically evaluating the quality
  of students writing skills.

7
Assessment Rubric for Critical Thinking

• ARC was designed by the QEP staff and the Faculty
  Champions to
• Enhance the QEP
• Align with the Colleges definition of critical
  thinking
• Be flexible for use in multi-disciplines

8
Assessment Rubric for Critical Thinking
Performance Element Exemplary (4) Proficient (3) Developing (2) Emerging (1) Not Present (0)
I. Communication Define problem in your own words. Identifies the main idea or problem with numerous supporting details and examples which are organized logically and coherently. Identifies the main idea or problem with some supporting details and examples in an organized manner. Identifies the main idea or problem with few details or examples in a somewhat organized manner. Identifies the main idea or problem poorly with few or no details or states the main idea or problem verbatim from the text. Does not identify the main idea or problem.
II. Analysis Compare contrast the available solutions. Uses specific inductive or deductive reasoning to make inferences regarding premises addresses implications and consequences identifies facts and relevant information correctly. Uses logical reasoning to make inferences regarding solutions addresses implications and consequences Identifies facts and relevant information correctly. Uses superficial reasoning to make inferences regarding solutions Shows some confusion regarding facts, opinions, and relevant, evidence, data, or information. Makes unexplained, unsupported, or unreasonable inferences regarding solutions makes multiple errors in distinguishing fact from fiction or in selecting relevant evidence. Does not analyze multiple solutions.
III. Problem Solving Select defend your final solution. Thoroughly identifies and addresses key aspects of the problem and insightfully uses facts and relevant evidence from analysis to support and defend potentially valid solutions. Identifies and addresses key aspects of the problem and uses facts and relevant evidence from analysis to develop potentially valid conclusions or solutions. Identifies and addresses some aspects of the problem develops possible conclusions or solutions using some inappropriate opinions and irrelevant information from analysis. Identifies and addresses only one aspect of the problem but develops untestable hypothesis or develops invalid conclusions or solutions based on opinion or irrelevant information. Does not select and defend a solution.
9
Assessment Rubric for Critical Thinking
Performance Element Exemplary (4) Proficient (3) Developing (2) Emerging (1) Not Present (0)
IV. Evaluation Identify weaknesses in your final solution. Insightfully interprets data or information identifies obvious as well as hidden assumptions, establishes credibility of sources on points other than authority alone, avoids fallacies in reasoning distinguishes appropriate arguments from extraneous elements provides sufficient logical support. Accurately interprets data or information identifies obvious assumptions, establishes credibility of sources on points other than authority alone, avoids fallacies in reasoning distinguishes appropriate arguments from extraneous elements provides sufficient logical support. Makes some errors in data or information interpretation makes arguments using weak evidence provides superficial support for conclusions or solutions. Interprets data or information incorrectly Supports conclusions or solutions without evidence or logic uses data, information, or evidence skewed by invalid assumptions uses poor sources of information uses fallacious arguments. Does not evaluate data, information, or evidence related to final solution.
V. Synthesis Suggest ways to improve/strengthen your final solution. Insightfully relates concepts and ideas from multiple sources uses new information to enhance final solution recognizes missing information correctly identifies potential effects of new information. Accurately relates concepts and ideas from multiple sources uses new information to enhance final solution correctly identifies potential effects of new information. Inaccurately or incompletely relates concepts and ideas from multiple sources shallow determination of effect of new information on final solution. Poorly integrates information from more than one source to support final solution Incorrectly predicts the effect of new information on final solution. Does not identify new information for final solution.
10
Assessment Rubric for Critical Thinking
Performance Element Exemplary (4) Proficient (3) Developing (2) Emerging (1) Not Present (0)
VI. Reflection Reflect on your own thought process. What did you learn from this process? What would you do differently next time to improve? Identifies strengths and weaknesses in own thinking recognizes personal assumptions, values and perspectives, compares to others, and evaluates them in the context of alternate points of view. Identifies strengths and weaknesses in own thinking recognizes personal assumptions, values and perspectives, compares to others, with some comparisons of alternate points of view. Identifies some personal assumptions, values, and perspectives recognizes some assumptions, values and perspectives of others shallow comparisons of alternate points of view. Identifies some personal assumptions, values, and perspectives does not consider alternate points of view. Does not reflect on own thinking.
11
ARC Assignment Profile

• ARC Assignment Profile is designed to provide
  consistency and accuracy in the evaluation of the
  ARC at the institutional level as well as provide
  guidelines for the use of the assessment at the
  course level.
• The ARC is essentially a tool to evaluate
  critical thinking. For a tool to be effective it
  must be used in the correct situation or job.
• The purpose of the ARC Assignment Profile is to
  outline the most appropriate course assignment.

12
ARC Assignment Profile

• 1. Participating faculty should have one
  assignment during the course that can be
  evaluated using the ARC scoring rubric. The
  course assignment could be a graded homework
  assignment or a major assessment for the course.

13
ARC Assignment Profile

• 2. The course assignment for the ARC should
  include all of the elements of the rubric and
  should be aligned with the task outlined for each
  element. Assignments that only evaluate some of
  the elements or are not aligned with the specific
  ARC tasks will be considered incomplete and not
  used in the institutional analysis.

14
ARC Assignment Profile

• 3. Faculty may add additional discipline specific
  rubric elements (such as grammar and punctuation
  in a composition class), but must maintain the
  ARC elements as listed.

15
ARC Assignment Profile

• Students should be provided a copy of the
  assignment rubric (ARC and any additional
  discipline specific elements). The specific
  elements and tasks include
• Communication Define the problem in your own
  words.
• Analysis Compare contrast the available
  solutions within the scenario.
• Problem Solving Select one of the available
  solutions and defend it as your final solution.
• Evaluation Identify the weaknesses of your final
  solution.
• Synthesis Suggest ways to improve/strengthen
  your final solution (may use information not
  contained within the scenario).
• Reflection Reflect on your own thought process
  after completing the assignment.
• What did you learn from this process?
• What would you do differently next time to
  improve?

16
ARC Assignment Profile

• 5. The evaluating scenario (selected or created)
  should be stated in such a manner to allow the
  student to address each of the tasks. The QEP
  team is willing to assist with the creation of
  the scenario or identify possible sources of
  existing scenario that could be used.

17
ARC Assignment Profile

• 6. At the end of the semester, please send the
  completed student assignments to the Janice
  Thiel, QEP Director, TE 1-111 (X3110). Completed
  student assignments should include a copy of the
  scenario, the assignment provided to the student
  (with the rubric), the students work and the
  final graded rubric.

18
ARC Assignment Profile
Competency (KSA) Problem with Multiple Solutions Premise with Multiple Perspectives
Communication Define Problem Define Premise
Analysis Compare Contrast Solutions Compare Contrast Alternative Perspectives
Problem Solving Select Defend Final Solution Select Defend Final Perspective
Evaluation Identify Weaknesses Final Solution Identify Weaknesses Final Perspective
Synthesis Suggest Improvements Final Solution Suggest Improvements Final Perspective
Reflection Reflect on Thought Process Reflect on Thought Process
19
Sample Scenario (Deer)

• Three teenagers were seriously injured in a
  car accident when swerving to avoid a deer on a
  two-lane road near a small, rural town in
  Florida. The residents of the town have seen more
  and more deer enter the towns populated areas
  over recent years. Local law enforcement has been
  called numerous times this year to remove the
  animals from backyards and neighborhood streets,
  and one deer even caused considerable damage as
  it entered a restaurant in town. The mayor has
  been charged by the city leaders to keep the town
  residents safe.

20
Sample Scenario (Deer)

• Local crops have even been damaged by the
  animals. Some long time residents have requested
  that the hunting season and catch limits be
  extended in order to reduce the deer population.
  One city leader even proposed that the city
  purchase electronic devices to deter the deer
  from entering populated areas. Health concerns
  have recently been elevated as three deer
  carcasses were found at the edge of town and
  local law enforcement suspect that the animals
  had been poisoned.

21
Sample Scenario (Deer)

• Possible Solutions
• Some long time residents have requested that the
  hunting season and catch limits be extended in
  order to reduce the deer population.
• One city leader even proposed that the city
  purchase electronic devices to deter the deer
  from entering populated areas.
• Health concerns have recently been elevated as
  three deer carcasses were found at the edge of
  town and local law enforcement suspect that the
  animals had been poisoned.

22
Improving Scoring Consistency

• Provide rubric to students prior to assessment
• Anonymous papers
• Anchor papers defining levels of proficiency for
  reference
• Use of multiple scorers
• Interrater reliability statistics during training
  and grading

23
Assessment Rubric for Critical ThinkingAcademic
Round Table Presentation