On%20Scoring%20Guides

1
On Scoring Guides

• everything you were afraid to ask

PART TWO
2
Critical thinking asdistributed compentency?

• ?Generalized thinking (e.g., Bloom)
• explained to others in the institution?
• ?Generalized thinking explained to laypersons
  outside the institution?
• ?Integrative thinking?
• ?Thinking in the major or discipline?
• ?Thinking in a specific community?
• ?Some combination?
• ?Other

3
(No Transcript)
4
(No Transcript)
5
(No Transcript)
6
(No Transcript)
7
(No Transcript)
8
St. Olaf E-Portfolio Thinking

• ?Reflective Thinking
• ?Integrative Thinking
• ?Thinking in Context
• ?Thinking in Community

9
Specific and Flexible
10
(No Transcript)
11
(No Transcript)
12
(No Transcript)
13
(No Transcript)
14
(No Transcript)
15
(No Transcript)
16

• WPA Outcomes Statement for First-Year
  Composition
• Rhetorical Knowledge
• Critical Thinking, Reading, and Writing
• Processes
• Knowledge of Conventions

17
Rhetorical Knowledge By the end of first year
composition, students should --Focus on a
purpose --Respond to the needs of different
audiences --Respond appropriately to different
kinds of rhetorical situations --Use
conventions of format and structure appropriate
to the rhetorical situation --Adopt appropriate
voice, tone, and level of formality
--Understand how genres shape reading and
writing --Write in several genres
18
Rhetorical Knowledgefor ASU CompositionOur
writing courses will focus on helping students
develop and use a rhetorical framework to analyze
writing situations, in a number of ways. Students
will learn how to--use heuristics to analyze
places, histories, and cultures --be aware of
the components of argument and create their own
arguments in conversation with other members of
their discourse communities --synthesize and
analyze multiple points of view --use a variety
of argumentative strategies to write for a
variety of audiences --express a working
knowledge of key rhetorical features, such as
audience, situation, and the use of appropriate
argument strategies --adopt appropriate voice,
tone, and level of formality --use conventions
of format, structure, and language appropriate to
the purpose of the written texts --be able to
focus on a specific rhetorical purpose
19
So . . .

• PLAN
• COLLECT DATA
• USE RESULTS (aka, feedback loop)

20
UNC Charlotte English Education

• Portfolio
• Interview
• ?to assess readiness to student teach
• What is your favorite reading?
• What African American literature would you
  like to teach? What Native American literature
  would you like to teach?

21
Issues . . .

• why were students preferring shorter texts?
• why could they not identify more than a single
  text in a given field? Was this a problem?
• how did they perform on national tests?
• what feedback did we get from schools?

22
CHANGES

• Added a course in Native American Lit
• Added a course in African American Lit
• Required a course in Ethnic Literature
• Encouraged all faculty to use real texts

RESULTS? ?new issues!!
23
PORTLAND STATE CRITICAL THINKINGprogram design
METHOD
interdisciplinary, team-taught first year
seminar

• A random sampling of these portfolios that was
  stratified for each class formed the analysis
  set. Portfolios were selected using a random
  number generator and a numbered list of students
  from each Freshman Inquiry class. Five student
  names and one alternate (and instructions for
  choosing a second alternate, if necessary) were
  given to each instructor. Alternate names were
  used if one or more of the original five students
  chosen appeared on the class list, but did not
  complete the course.

24
The scoring guides (rubrics) used in the Summer
Portfolio Review were internallydeveloped. A
previous attempt to use an externally developed
rubric for critical thinkingwas not successful
because the rubric was not contextually relevant
to the PSU student work. The new rubric for
critical thinking was not completely developed by
the time ofthis summers review, and the review
itself stood as a development process for
thisrubric.
PORTLAND STATE AND CRITICAL THINKING
25
WHAT HAPPENED?

• Following the June 2000 review of Freshman
• Inquiry portfolios, each Freshman Inquiry
• team was required to review information from
• the portfolio review and, if available, from
• the end-of- year course evaluations. Each team
• reported to the Freshman Inquiry Coordinator,
• specific, planned course revisions based on the
• asessment information.

26
In the summer 2000 meeting, the Metamorphosis
team decided to focus on the critical thinking
goal. To this end, they instituted a major
re-working of the fall term course,revising the
texts and focusing more clearly on central
concepts. They also added morescience content in
the first and second quarter courses. This team
raised questions aboutthe critical thinking
rubric, which is scheduled for revision this
year.
GET SPECIFIC! WHAT DOES THIS MEAN?
27
Did you say technology?
28
What some of this means for guides . . .

• Scoring Guides Can Show Development, Achievement,
  or Both
• --What is the guide intended to document, and
  why?
• Text and Context Scoring Guides Dont Act Alone.
• --How will you introduce them to students? And
  for what purpose/s?
• Scoring Guides Signal the Philosophy of the
  Institution.
• --Is the model of learning showcased in the
  guide a deficit model of learning (think error
  avoidance or removal) or an asset inventory model
  (think building on strengths)?

29
Scoring Guides Inherently Enact a Visual
Rhetoric --Most scoring guides list a set of
elements organized into an analytical framework
or a holistic framework, suggesting that each
item is weighted equally. When the guide is put
into an institutional context and/or is used to
rate student work, often some items are more
valuable than others. Which items are the more
valuable, and why, and how can that be signaled
within the guide itself?Scoring Guides, as
Genre, Change --The scoring guide we would have
created for writing twenty years ago would
exclude much of what we do in writing today
critical thinking is likewise defined more
ambitiously today than in earlier times. The
scoring guide we create today should do the best
job it can do today. It will need to be changed
tomorrow.
30
How to begin?

• Gather some student work.
• (What counts as work?)
• Gather some model scoring guides.
• Get some food.
• Talk!
• Identify what you likethats vocabulary.
  Consume? Produce?
• Achievement? Development?
• Identify a structure.
• Give it a go, and call it a pilot.

31
Youre never really ready . . .

• Its Better to Start Small,
• but to Start . . .
• Call it what you willa pilot, a field
  teststart! Start small. Get a taste of success,
  and build on that.