Assessing Critical Thinking: A Search for a Holy Grail - PowerPoint PPT Presentation

1 / 40
About This Presentation
Title:

Assessing Critical Thinking: A Search for a Holy Grail

Description:

'The ideal critical thinker is habitually inquisitive, well-informed, trustful of ... [1] American Philosophical Association, 'The Delphi Report: Research ... – PowerPoint PPT presentation

Number of Views:163
Avg rating:3.0/5.0
Slides: 41
Provided by: jason81
Category:

less

Transcript and Presenter's Notes

Title: Assessing Critical Thinking: A Search for a Holy Grail


1
Assessing Critical Thinking A Search for a
Holy Grail?
2
Critical Thinking Assessment Framework
  • Critical Thinking Skill
  • Analysis
  • Evaluation
  • Inference
  • Divergent Thinking
  • Instructor Identified
  • Assessment Measure
  • California Critical Thinking Skills Test
  • Writing Rubric
  • CCTST
  • ?
  • Instructor Responsibility

3
CCTST Basics
  • The test booklet is a comforting peach color.
  • A 34-item forced-choice measure of critical
    thinking skills
  • Normative data allow comparison to larger
    population
  • Measures multiple aspects of critical thinking
    including inductive and deductive reasoning

4
CCTST Basics
  • Our three domains analysis, evaluation, and
    inference
  • Peter Facione CCTST measures them all without
    question
  • Douma and Jones CCTST measures analysis and
    inference domains well
  • Douma and Jones Pete Facione is a nice guy,
    but we are probably right.
  • Douma and Greg Now in syndication on UPN

5
CCTST and Heuristics1
  • Influenced by Kahneman and Tverskys decision
    making research and theory
  • Heuristic
  • a problem solving method that provides a
    reasonable, probably correct decision
  • selecting among actions
  • multi-attribute alternatives
  • A very practical outcome of critical thinking

6
Heuristics and Decision Making
  • Heuristics and thinking errors
  • Availability heuristic
  • Representative heuristic
  • Base-rate fallacy

7
Heuristics and Decision Making
  • Availability heuristic
  • If a word of three letters or more is sampled
    from an English text, is it more likely that the
    word starts with r or has r as the third
    letter?

8
Heuristics and Decision Making
  • Representative Heuristic
  • Linda is 31-years old, single, outspoken and
    very bright. She majored in philosophy. As a
    student, she was deeply concerned with issues of
    discrimination and social justice, and also
    participated in anti-nuclear demonstrations.
  • Considering this description, which of the
    following statements is most likely?
  • A Linda is a bank teller.
  • B Linda is a bank teller involved in the
    feminist movement.

9
Heuristics and Decision Making
  • Base rates
  • A taxi-cab was involved in a hit-and-run
    accident. Two cab companies, the Green and the
    Blue, operate in the city. You are given the
    following data
  • Eighty-five percent of the cabs in the city are
    Green while 15 percent are Blue.
  • In court, a witness identified the cab in
    question as Blue.

10
Heuristics and Decision Making
  • The court tested the ability of the witness to
    identify cabs under a variety of conditions.
    When presented with a series of pictures of cabs
    in which half were blue and half were green, the
    witness correctly identified the cabs in 80 of
    cases.
  • What was the probability that the cab involved
    in the accident was Blue rather than Green?

11
Heuristics and Student Thinking
  • These errors are how typically people think.
  • Bad news This is how people typically think.
  • Good news Adults seem to naturally use
    heuristics when making decisions.
  • Challenge 1 Get students to use more valid
    heuristics.
  • Challenge 2 Measure students use of valid
    heuristics.

12
CCTST and Heuristics
  • Even as a special interest reporter and
    weatherman in Indiana, I realized the
    importance of assessing critical thinking. You
    there, the third person from the left in the
    second row, what do you wonder about the CCTST?

13
Critical Thinking Dispositions
  • Alas, a topic only an academician could love

14
Critical Thinking Dispositions
  • Suppose that you, taking a page from Super Dave
    Osbornes playbook, decided to teach your
    students every critical thinking skill known to
    humankind
  • every law of logic,
  • every handy heuristic,
  • every rule of rhetoric,
  • every competency a student needs in order to ace
    the CCTST
  • (which the Faciones insist has never been done)

Photo from http//www.tommcmahon.net/2004/04/bob_e
instein_ak.html
15
Critical Thinking Dispositions
  • would your CT course float?
  • Im not convinced.
  • (Paul, you agree?)
  • First, like Super Dave, you would hurt yourself.
  • Second, even if your students can think
    critically, they may not order their intellectual
    behavior so that they do think critically.

Photo from http//www.tommcmahon.net/2004/04/bob_e
instein_ak.html
16
Critical Thinking Dispositions
  • Is our ultimate goal to develop students who can
    think critically, or students who do think
    critically?
  • These characteristics are not identical
  • in recognition of this, the good folks at Insight
    Assessment have developed the CCTDI, as a
    companion to the CCTST, to measure student
    development toward the latter (greater?) end.

17
Critical Thinking Dispositions
  • The CCTDI measures intellectual behavior along
    seven scales
  • Truthseeking the intellectual integrity to
    follow reason and evidence wherever it may lead
  • Open-Mindedness tolerant of divergent views,
    self-monitoring for possible bias
  • Analyticity demanding application of reason and
    evidence, inclined to anticipate consequences
  • Systematicity valuing organization, persistence
    even in complex problems
  • CT Self-Confidence trusting ones own reasoning
  • Inquisitiveness eager to acquire knowledge,
    even when applications of the knowledge are not
    immediately apparent
  • Maturity of Judgement prudence in making,
    suspending, or revising judgement awareness of
    circumstances that require closure even in
    absence of complete knowledge
  • http//www.insightassessment.com/test-cctdi2.html

18
Critical Thinking Dispositions
  • The CCTDI measures intellectual behavior along
    seven scales
  • These categories were selected to correspond to
    the attributes of critical thinkers identified in
    a 1990 report from the American Philosophical
    Association
  • The ideal critical thinker is habitually
    inquisitive, well-informed, trustful of reason,
    open-minded, flexible, fair-minded in evaluation,
    honest in facing personal biases, prudent in
    making judgements, willing to reconsider, clear
    about issues, orderly in complex matters,
    diligent in seeking relevant information,
    reasonable in selection of criteria, focused in
    inquiry, and persistent in seeking results which
    are as precise as the subject and the
    circumstances of inquiry permit.
  • (P. Facione, Delphi Report, 1990, p. 3)

19
Critical Thinking Dispositions
  • Is this anything?
  • Consider the following
  • The CCTDI is normed 4, with descriptive and
    quantitative classifications

20
Critical Thinking Dispositions
  • Is this anything?
  • Consider the following
  • The CCTDI has been tested for reliability 2,
    with Cronbachs Alpha .90 for overall disposition
    and a range of .72 to .80 for each of the seven
    subscales.

21
Critical Thinking Dispositions
  • Is this anything?
  • Consider the following
  • In prior studies, r 2 values for the relationship
    between CT skills and CT dispositions were
    consistently below 0.16, with r 2 0.04 for the
    overall relationship between CCTST and CCTDI
    scores 2.

22
Critical Thinking Dispositions
  • Is this anything?
  • Consider the following
  • While the CCTST focuses largely on a particular
    set of critical thinking skills (its no surprise
    that r .708 for CCTST vs GRE Analytic), the
    CCTDI may allow us insight into other critical
    thinking characteristics. (Nancy, heres a
    partial answer to your question.)

23
Critical Thinking Dispositions
  • Fast-Breaking News
  • A new and improved dispositional inventory, the
    CM3, is now available. Preliminary studies on
    groups of adolescents suggest that the CM3 may
    provide greater reliability and factor stability
    than the CCTDI 3.

24
Critical Thinking Dispositions
  • Fast-Breaking News
  • The CM3 classifies CT disposition according to
    four factors 3
  • Learning Orientation disposed toward increasing
    ones knowledge, values learning process, seeks
    challenges
  • Creative Problem Solving solves problems with
    original ideas, engages in puzzles, desires to
    understand underlying function of objects
  • Mental Focus diligent, systematic, organized,
    clear-headed
  • Cognitive Integrity interacts with different
    viewpoints for the sake of seeking truth or
    making best decision, intellectually curious,
    fair-minded

25
Critical Thinking Dispositions
  • Two Guys and aWorkshop Recommend
  • If you are teaching a CT course, you might be
    well advised to consider using skills and
    disposition tests in tandem as a means of course
    assessment. (CCTSTCCTDI or TERCM3)

26
Course-Level Assessment of CT
  • What are the domains of critical thinking
    that we assessing?
  • What measures must you use as CT course to assess
    these domains?
  • Are you limited to using these measures to assess
    CT?

27
  • Excellent answer, my friend. You are a lot
    better than those suckers at State U. Now,
    please accept this toast on a stick as a reward
    for your superior insight and generally pleasant
    demeanor.

28
Critical Thinking Assessment Framework
  • Critical Thinking Skill
  • Analysis
  • Evaluation
  • Inference
  • Divergent Thinking
  • Instructor Identified
  • Assessment Measure
  • California Critical Thinking Skills Test
  • Writing Rubric
  • CCTST
  • ?
  • Instructor Responsibility

29
Assessing CT in Your Course
  • Ideas and examples
  • Start
  • Build on what you already do to teach and assess
    critical thinking
  • Start

30
University and General Education Level Assessment
  • A Modest Proposal
  • Administer the CCTSTCCTDI (or TERCM3) to a
    random sample of freshmen on Assessment Day, and
    readminister to the same sample (those
    persisting, matched pairs design) on Assessment
    Day in the junior year.

31
University and General Education Level Assessment
  • A Modest Proposal
  • This design would ostensibly allow us to measure
    and assess change in CT skills and dispositions,
    and would allow us to control for potential
    confounding (general change in maturity and
    intellectual skills/habits through first to years
    of college), as some fall semester
    juniorshopefullyhave not yet completed a CT
    course.

32
University and General Education Level Assessment
  • A Modest Proposal
  • By classifying data according to number and type
    of CT courses taken by junior year, we couldin
    principlediscern the impact of the designated CT
    course model on student CT skills and
    disposition.

33
University and General Education Level Assessment
  • A Modest Proposal
  • Warning This is not official policy or
    processjust an idea at this point.
  • Lots of people far more important than Jones and
    Douma need to weigh in on this first.

34
An Exercise in Understanding and Evaluating an
Argument in a Biased Context
  • http//www.crispinsartwell.com/meds/math.htm
  • In light of Dr. Sartwells article, should USF
    require all of its students to study mathematics,
    regardless of their major or career plans?
  • (assignment used in various forms in MAT151)

35
Sample CT Proposal and Course Syllabus
MAT151 Proposal MAT151 Syllabus (Fall
2002) MAT151 Assessment Report to APC MAT151
Final Report to Bush Foundation
36
If you need hep, here I am
Keith Jones Jason Douma at your service
37
Critical Thinking Dispositions
  • List of References
  • 1 American Philosophical Association, The
    Delphi Report Research Findings and
    recommendations prepared for the committee on
    pre-college philosophy, Critical Thinking A
    Statement of Expert Consensus for Purposes of
    Educational Assessment and Instruction, P.
    Facione (project director), 1990.
  • 2 Facione, P., N. Facione, and C. A. Giancarlo,
    The Disposition Toward Critical Thinking Its
    Character, Measurement, and Relationship to
    Critical Thinking Skill, Informal Logic (2000),
    20(1), 61-84.
  • 3 Giancarlo, C.A., S. Blohm and T. Urdan,
    Assessing Secondary Students Dispositions
    Toward Critical Thinking Development of the
    California Measure of Mental Motivation,
    Educational and Psychological Measurement (2004),
    64(2), 347-364.
  • 4 Giancarlo, C.A. and P. Facione, A Look
    Across Four Years at the Disposition Toward
    Critical Thinking Among Undergraduate Students,
    The Journal of General Education (2001), 50(1),
    29-55.

38
End Note
  • 1. The material on heuristics was adapted from
    Gilhooly, K. J. (1996). Thinking Directed,
    undirected, and creative. San Diego Academic
    Press.

39
CT Assessment Resources
  • http//www.criticalthinking.org/
  • This is the Foundation for Critical Thinking
    (Richard Pauls group).  Like a corporate
    conglomerate, they have merged with several other
    groups and changed some of their names
    (International Congress on Critical Thinking is
    now International Conference of Critical
    Thinking).  Still, I believe this group is one
    of the most listened-to voices in the critical
    thinking community.
  • http//innumeracy.com/criticalthink.htm
  • This is a nifty collection of information that is
    consonant with both the CCTST perspective as well
    as Richard Pauls group (indeed it includes links
    to both).  The emphasis here, as the innumeracy
    title suggests, is toward the quantitative,
    deductive, and analytic however, I believe some
    of this material is universal.  This is actually
    the site that received my praises in this
    afternoons conversation.  The alleged New
    Mexico site (see mea culpa below) is one of the
    dozens of links from this site which I had
    followed a few weeks ago.
  •  

40
CT Assessment Resources
  • http//admin.santafe.cc.fl.us/acres/ct/tutorial/
  • See how another school is using the CCTST.   
  • http//www.usiouxfalls.edu/faculty/Development/Cri
    tical20Thinking20Proposal20Guidelines.doc
  • Okay, you probably could have accessed this link
    yourself, but just for convenience heres the
    document that outlines current APC policy for
    approving designated CT courses.
Write a Comment
User Comments (0)
About PowerShow.com