GE ASSESSMENT, 20062007: Critical Thinking in GE

1
GE ASSESSMENT, 2006-2007Critical Thinking in GE

• Process, Results, Recommendations
• GE Critical Thinking Task Force
• http//www.csuchico.edu/vpaa/ued/GE_CT_Program20A
  ssessment_v2.pdf

2
GE ASSESSMENT Rationale, Structure and Process

• Need for Program assessment
• Mandate for Program assessment
• (EO-595, EM-99-05, MOU)
• Existing structures GEAC, AURA, disciplinary
  specialists
• Goals of GE Program assessment
• Participatory
• Outcomes based
• Unobtrusive (limited impact on workload, budget,
  etc)
• Meaningful, Manageable, Sustainable

3
GE Program Assessment

• GE Goals (EM 99-05)
• improve reading, writing, critical thinking,
  discussion and speaking skills, mathematical
  reasoning, analysis and problem solving
• GE Program Outcomes
• communicate thoughtfully and clearly, both
  orally and in writing
• think critically and constructively
• be conversant and skilled in the basic
  undergraduate understanding of mathematics
• Other Goals, Outcomes focused on science,
  literature, the arts, humanities, behavioral and
  social sciences integrative and
  interdisciplinary understandings, etc
• GE Program Assessment to date has focused on Core
  Skills Writing, Oral Communications,
  Quantitative Reasoning in AY 05-06, Critical
  Thinking in AY 06-07

4
Goals of Assessing Critical Thinking (CT) in GE

• Progress towards an acceptable definition of CT
  appropriate to our campus and GE
• Progress towards a means of assessing student
  competence in CT using methods
• Acceptable to faculty
• Generate useful, actionable data
• Based on embedded assessment of authentic student
  work across GE (not solely Area A-3)
• Sound, defensible results

5
Process For Assessing CT in GE

• Form Task Force with membership from AURA, GEAC,
  Provosts Office and disciplinary specialists
  (Philosophy)
• Margaret Owens, Assoc Dean CNS, AURA
• Gwen Sheldon, Sociology, GEAC
• Brooke Moore and Greg Tropea, Philosophy
• Bill Loker, Dean of Undergraduate Education

6
Process For Assessing CT in GE (contd)

• Task Force wrestled with definitions of CT
• Simultaneously, recruited faculty for
  participation in CT assessment
• Instructors of GE Courses
• Assignments that elicit CT in students
• Shared CT definition with faculty
• Created shared rubric for assessing CT

7
CT Definitional Issues

• Many definitions of CT in the literature
• CT a generalized skill or context specific?
• Task Force guided by EO-595, EM 99-05 which
  circumscribe CT in GE
• Focused on understanding and analysis of argument
  or problem in a text
• Recognize not appropriate for all GE courses or
  intellectual traditions
• E.g. literary criticism, creative writing
• Two dimensions identified reasoning and
  comprehension

8
Comprehension In texts and other forms of
discourse, students SLO 1. Can identify
issues Does the text address an issue or
problem? If so, what is it? SLO 2. Can
distinguish between clarification, argument,
persuasion and other ways of relating to an
issue How does the text address the issue or
problem? Does it merely raise it? Does it seek to
clarify it? Does it take a position on it? Does
it attempt to persuade an audience of that
position? Does it support or defend the position?
Does it relate to the issue in another
manner? SLO 3. Can recognize the difference
between conclusions and the arguments for them If
a conclusion has been reached, what is it? What
arguments have been given for that conclusion?
SLO 4. Can distinguish between factual
judgments and non-factual judgments Is the issue
a question of objective fact? Is it a normative
questioni.e., a question calling for a value
judgment? Is it purely subjective?
9
Reasoning In texts and other forms of discourse,
students SLO 5. Can distinguish between
inductive reasoning and deductive reasoning Do
given arguments support their conclusion
(induction) or do they demonstrate it
(deduction)? SLO 6. Can distinguish between
truth and logic Can students distinguish and
assess the truth of a statement from the quality
of the reasoning (i.e., logical structure) within
which that statement may function? Can they, for
example, avoid the errors of thinking that
reasoning with one or more false statements
entails bad logic? Or that reasoning that
consists entirely of true statements entails good
logic? SLO 7. Can determine whether a
consideration is relevant Is the discussion
relevant to the issue? Has rhetoric been offered
in place of evidence? SLO 8. Can recognize
questionable assumptions and missing
information Have questionable assumptions been
made? Is important information missing? SLO 9.
Can evaluate the credibility of statements and
sources Are sources and claims both credible?
SLO 10. Can identify ambiguity, vagueness, and
common fallacies in reasoning Are key passages
vague, ambiguous, or otherwise unclear? Are there
mistakes in reasoning?
10
CT Scoring Rubric Comprehension ( Reasoning,
next slide)
11
(No Transcript)
12
CT in GE Assessment Results

• 18 faculty teaching 21 courses volunteered to
  participate in assessment of these we ended up
  with 11 faculty and 8 courses where student work
  was assessed
• Courses from Areas A, B, C and D, Lower Division
  Core, Breadth and Upper Division Themes (see next
  slide)
• 186 pieces of student work, each item
  independently assessed by two readers for Overall
  Comprehension and Reasoning on a scale of 1-3 (1
  Poor, 3 Competent)

13
Statistical Snapshot of CT Materials, Students
14
Statistical Summary Comprehension and Reasoning
Scores, all students
15
Mean Comprehension, Reasoning Scores, all students
16
Mean Comprehension Scores Differ Significantly (t
1.89, p lt 0.05, one-tailed)
17
Mean Reasoning Scores Differ Significantly (t
1.81, p lt 0.05, one-tailed)
18
Results Other Statistical Analyses

• Weak, positive relationship between class level
  and performance on both Reasoning and
  Comprehension (gamma 0.16, P lt 0.05)
• No relationship detected between gender, degree
  sought, native vs. transfer status and CT scores

19
Critical Thinking in GE On Balance

• Assuming the soundness of the methodology pursued
  
• Students gain in CT skills over time, e.g.
  seniors vs. first- year students
• Level of CT performance at all class levels less
  than outstanding, e.g. as seniors only slightly
  more than half of the students reached the
  developing level (score of 2) on
  Comprehension and Reasoning
• Apparent lack of clear consensus as to what
  constitutes CT among faculty and how to elicit CT
  from students via assignments in GE courses

20
GE Assessment Two Years On Writing, Oral
Communication, Quantitative Reasoning a brief
review of results
21
GE Assessment Two Years on What have we
learned?

• Both Quantitative Reasoning (QR) and Critical
  Thinking face definitional issues on campus
  until we know what we want our students to know
  and do, it will be difficult to assess these and
  measure our success
• Both CT and Writing are highly valued by faculty
  but faculty need support in the pedagogy of
  both Writing and CT so students can practice and
  improve
• Both QR and Oral Communication are
  little-practiced in GE outside Areas A1, A4
  faculty need to revisit GE documents that
  encourage integration of all Core domains
  across GE curriculum walk the walk or dont talk
  the talk

22
GE Assessment Two Years on What have we
learned? contd

• Student performance on all core domains verges on
  the adequate with non-normed results of
  non-standardized assessment, student performance
  is impossible to compare across campuses and
  difficult to gauge but clearly there is room
  for improvement
• Campus desperately needs an extended discussion
  of our purposes and goals in GE, as well as
  assessment strategies to substantiate and assure
  the quality of the GE program
• Are we ready?