Title: How Do We Really Know What Our Students Are Learning Teaching and Assessing for Deep Learning
1How Do We Really Know What Our Students Are
Learning?Teaching and Assessing for Deep Learning
- Douglas Eder
- deder_at_siue.edu
Southern Illinois University Edwardsville
2Asteroid Impact AssessmentAssumptions
Expectations?
3Assessment makes learning visible
- Assessment is the ongoing, systematic examination
of student learning and the learning environment.
4Direct vs. Indirect Assessment
- Direct assessment acquires evidence about student
learning and the learning environment Exams,
projects, logs, portfolios, observations.... - Indirect assessment acquires evidence about how
students feel about learning and their learning
environment Surveys, questionnaires, interviews,
focus groups, reflective essays....
5Assessment vs. Evaluation
- Assessment focuses on the student and the
learning environment. - Evaluation focuses on the professor and the
teaching performance.
6Assessment Questions Answers
7Assessment Asks Questions
- Bad questions take just as much energy to answer
as good questions do.
8What is the purpose of college?
9UTEP Goals, 2001-05Learning Teaching(President
s Page)
- Students will acquire tools to
- meet lifelong intellectual, ethical, and career
challenges, and - be leaders.
10The Basic Assessment Question
- What do we want to know about ourselves?
11The Basic Assessment Question Amplified
- Are we doing what we say were doing?
- Are we doing what we ought to be doing?
-
- Just what do we think were doing?
12Good Assessment Asks Good Questions
- What is your job description for your first year
students? For your second year students? How do
they differ? - Do we have a curriculum or a set of courses?
Which of these do our students experience? - How much and what kinds of writing do our
students do? What kinds of math? Are these
relevant to our curricular goals?
13Its human nature to respond to subtle cues in
the environment
14The Scholarship of Teaching
- Discovery
- Integration
- Application
- Teaching
-
- Public and peer reviewed
- Reflective
15Six Principles of Scholarship--Glassick et al.
- 1. Clear goals -- we identify the question
- 2. Adequate preparation -- we see the task
- 3. Appropriate methods -- we can do the task
- 4. Significant results -- we do care
- 5. Effective presentation -- we can escape
- 6. Reflective critique -- we can improve
16Principle 1
17Whats the quickest way to solve a maze?
18The First and Only Goal
- To teach for long-term retention of information
and application to new situations - --after Halpern Hakel
19- Deep (expert) learning is not a course function.
- It is a curriculum function.
Colleges and Universities are Systems
20Cognitive Neuroscience
- The more of the brain --sensory, motor, and
association pathways-- involved during the
learning process, the more effective is the
learning.
21P
s s s s s s s s s s s s s s s s s s s s s s s s s
s s s s s s s s s s s s s s s s s s s s s s s s s
s s s s s s s s s s
22P
s s s s s s
s s s s s s
s s s s s s
s s s s s s
s s s s s s
s s s s s s
s s s s s s
23s s s s s s s Ps s s s s s s s s
s s s s s s s s s s s s s
24P
s s s s s s s s s s s s s
s s s s s s s
25Why do students come to classroom?
26Why do students come to classroom?
Why should students come to classroom?
27Annotated Word Journal
- Read the assigned text and write one word that
captures the essence of what youve read and
summarizes your response to it. - ____________________
- Explain why you chose that word and how it
provides, in a capsule, your summary of the
reading. - ____________________________________
- ____________________________________
28The Importance of Goals
- Cheshire Puss..., asked Alice, would you tell
me, please, which way I ought to go from here? - That depends a good deal on where you want to
get to, said the Cat. - I don't much care where said Alice.
- Then it doesn't matter which way you go, said
the Cat.
29Principle 2
30Goals and Objectivesfor Students
- Goals express what we want our students to be.
- Objectives describe what we want our students to
do. - .....
- Objectives are indicators of goals.
31To be assessment friendly, objectives should
- Focus on students
- Make the learning goals visible (serve as
indicators) - Describe behaviors or products (doing, making)
that can be captured by assignments.
32UTEP Core Curriculum Objectives?
- Communications To enable students to communicate
effectively in clear and correct prose in a style
appropriate to the subject, occasion, and
audience. - Mathematics To develop a quantitatively literate
college graduateable to apply basic mathematical
tools in the solution of real world problems. - Natural Sciences To enable the student to
understand, construct and evaluate
relationshipsand to understand the bases for
constructing and testing theories. - Humanities, Visual Performing Arts To expand
students knowledge of the human condition and
human cultures. - US History, Political Science To expand
students knowledge of the origin and evolution
of US Texas history. - Social Behavioral Sciences To increase
students knowledge of how social and behavioral
scientists discover, describe, and explain
behavior. - Institutional Option To develop in students
the critical thinking tools necessary to become
effective learners.
33An Assessment Question How Do You Know...
- ...that students walk out your door looking like
you want them to? What behaviors have they
exhibited or products have they produced? What
are the indicators for your goals?
34Principle 3
35Some Assessment Ways and Means
- Assessment days and centers
- Case studies
- Classroom assessments
- Completion and retention studies
- Content analyses
- Debates
- Direct observations
- Focus groups
- Graduate success
- Internships and service learning
- Interviews (including videotapes)
- Exams for certification and licensure
- Matrices
- Performances
- Portfolios of several kinds
- Projects (Primary Trait Analysis)
- Questionnaires and surveys (Direct and
telephone employer, alumni, and student attitude
and satisfaction) - Reflective essays
- Study and activity logs
- Tests
- (Locally-developed and standardized)
- Transcript analyses
36- If all you have is a hammer, everything starts to
look like a nail.
37Course or Curriculum Alignment Grid
- What are your students focusing on?
38The Minute Paper
- 1. What is the most important thing you learned
here so far? - 2. What is your most important unanswered
question from our time together?
39Primary Trait Analysis...
- ...is a very strong link.
40What Were Really Looking for is
- Way Better than Good Enough
- Good Enough
- Not Good Enough
- -------------------------------
That's Enough!
41Principle 4
42Five Conditions That Foster Deep
LearningHalpern Hakel, Marchese, DeWinstanley
Bjork, Shpancer,National Research Council
- Engaging students actively
- Practicing retrieval and presentation in varied
environments - Linking new learning to prior learning
- Applying learning to new situations that students
care about - Receiving timely and relevant reinforcement
43What are the characteristics of
- first year writing --- Pink
- second year writing --- Yellow
- third year writing --- Green
- fourth year writing --- Blue
44Cognitive Stages of Knowing(Piaget, Perry,
Kitchener, Baxter-Magolda -- Summarized by Haynes
- Absolute Transitional
- Method of authority Answers come from
research - Uncertainty when Uncertainty when
- authority is answers are unavailable
unknown - Strategy Gain control Strategy Debate and of
information research
45Cognitive Stages of Knowing(Piaget, Perry,
Kitchener, Baxter-Magolda -- Summarized by Haynes
- Independent Contextual
- Many valid interpretations Evidence in context
- All opinions are good, Some opinions are
including ones own better than others - Strategy Introspection Strategy Values are
important
46Cognitive Stages of Knowing(Piaget, Perry,
Kitchener, Baxter-Magolda -- Summarized by Haynes
- Absolute Transitional Independent
Contextual - 1st 68 32 0 0
- 2nd 47 52 1 0
- 3rd 11 83 5 1
- 4th 2 80 16 2
- 5th -- -- 57 12
47Staged Writing Thinking...
- ...takes students from one cognitive level to the
next through incremental, not additive,
writing/speaking assignments. - e.g., Description/Narration
- Analysis
- Comparison/Contrast
- Integration
48Highest Priority Questions
- The three highest priority questions that I would
like to look into regarding student educational
performance at this institution are - 1.________________________________
- 2.________________________________
- 3.________________________________
49What is Good Effective Critical Thinking?
- Identifying
- Exploring
- Prioritizing
- Revisioning
- --Wolcott Lynch
50Critical Thinking in a Writing EnvironmentGradin
g and Assessing
- Traits 3 2 1
- Critical Thinking (after Wolcott Lynch)
- 1. Identifying __ __ __
- 2. Exploring __ __ __
- 3. Prioritizing __ __ __
- 4. Revisioning __ __ __
- Writing
- 5. Consistent focus on topic or issue __
__ __ - 6. Claims founded upon evidence __ __ __
- 7. Language appropriate for the audience __
__ __ - 8. Appropriate writing mechanics __ __ __
- 9. Scholarly bibliographic support __ __ __
- Scoring
- 27-23 Exceeds expectations
- 22-16 Meets expectations
- 15- 9 Does not meet expectations
51Critical Thinking in a Writing EnvironmentGradin
g and Assessing
- Traits 3 2 1
- Critical Thinking (after Wolcott Lynch)
- 1. Identifying __ __ __
- 2. Exploring __ __ __
- 3. Prioritizing __ __ __
- 4. Revisioning __ __ __
- Writing
- 5. Consistent focus on topic or issue __
__ __ - 6. Claims founded upon evidence __ __ __
- 7. Language appropriate for the audience __
__ __ - 8. Appropriate writing mechanics __ __ __
- 9. Scholarly bibliographic support __ __ __
- Scoring
- 27-23 Exceeds expectations
- 22-16 Meets expectations
- 15- 9 Does not meet expectations
52Critical Thinking in a Writing Environment Levels
of Performance
Exploring 2 3 Probes alternatives and presents
primary and secondary evidence in support. 2
Recognizes alternatives and acknowledges
existence of evidence in support. 1 Does not
recognize that alternatives may exist ignores
conflicting evidence.
Prioritizing 3 3 2 1
Consistent focus on topic or issue 5 3 2 1
53Critical Thinking in a Writing EnvironmentGradin
g and Assessing
- Traits 3 2 1
- Critical Thinking (after Wolcott Lynch)
- 1. Identifying _x_ ___ ___
- 2. Exploring ___ _x_ ___
- 3. Prioritizing ___ _x_ ___
- 4. Revisioning ___ ___ _x_
- Writing
- 5. Consistent focus on topic or issue _x_
___ ___ - 6. Claims founded upon evidence ___ ___ _x_
- 7. Language appropriate for the audience ___
_x_ ___ - 8. Appropriate writing mechanics ___ _x_ ___
- 9. Scholarly bibliographic support _x_
___ ___ - Score 19
- Scoring
- 27-23 Exceeds expectations
- 22-16 Meets expectations
- 15- 9 Does not meet expectations
54Critical Thinking in a Writing EnvironmentGradin
g and Assessing
- Traits 3 2 1
- Critical Thinking (after Wolcott Lynch)
- 1. Identifying 12 13 5
- 2. Exploring 11 15 4
- 3. Prioritizing 8 16 6
- 4. Revisioning 3 12 15
- Writing
- 5. Consistent focus on topic or issue 22 5
3 - 6. Claims founded upon evidence 14 12 4
- 7. Language appropriate for the audience 16
9 5 - 8. Appropriate writing mechanics 4 15 11
- 9. Scholarly bibliographic support 6 16
8 - Mean score 19.2 or 2.1 / 3 ? 2.8 / 4
- QUESTION What part(s) of the curriculum
- deserves special attention?
55Assessment is...
- ...more than finding out how many students passed
and what the average score is on a test of
competence.
56- Construct an effective 3- or 4-point rubric for
some aspect of writing or critical thinking. - Pay special attention to the level identified as
acceptable.
57An Important Lesson from the Farm
- A pig doesnt get any fatter merely by weighing
it.
58Closing Thought
- The enemy of the good is the perfect.