How do you know if learners are learning? Assessment and evaluation techniques

1
How do you know if learners are learning?
Assessment and evaluation techniques

• An Instructor Development workshop
• facilitated by Lee Winters

2
Agenda

• Introductions and overview
• Why learn about assessing and/or evaluating
  learning?
• My intended learning outcomes
• Your desired learning outcomes
• Guidelines and Requirements

3
Agenda Continued

• What do you do now to assess and/or evaluate
  learning?
• Assessment and evaluation techniques
• What will you do now to assess and/or evaluate
  learning?
• Feedback on workshop

4
Introductions and overview

• Letting students know your grading criteria
• Consistency in grading policy
• Syllabus and grading consistency
• Homework and study time requirements for your
  course
• prerequisites

5
Why learn about assessing and/or evaluating
learning?

• A method to evaluate students
• Assess their preparation for the course
• Continuous assessment of student learning
• Assess knowledge, abilities, skills, attitudes
• A method to evaluate our teaching
• A chance to assess the course and make changes
  as needed
• Consistency in grading

6
Todays In-service Goals

• By the end of the session you'll be able to
• Gauge learners progress in and reaction to your
  course
• Deal with assessment data you gather
• Test your teaching and their learning
• Develop appropriate course, class and homework
  assignments and tests
• Evaluate (and grade) student work

7
Your desired learning outcomes

• What are your course objectives?
• How do you know if the students learned the
  material?
• How do you know what works and what needs to be
  changed the next time you teach the course?
• Assessing the students along the way
• See Handout of Sample Action Verbs for objectives

8
What do you do now to assess and/or evaluate
learning?

• Dyads (10 minutes)
• Reporting what you do now or what has worked
  best for you

9
Immediate student reactions to your teaching

• Reading body language
• Reading words and tone of voice
• Absenteeism
• Assessing learning by questions asked
• Using Blooms and Andersons Taxonomy
• Student involvement and interaction

10
Using Blooms Taxonomy to Structure Questions in
the Classroom

• Refer to last two pages of the handout
• Memory or Knowledge

11
Translation or Comprehension
12
Interpretation
13
Inference or Application
14
Higher Blooms Levels

• Analysis

15
Synthesis
16
Evaluation
17
What are you assessing or evaluating?

• Intended Learning Outcomes
• Objectives

18
Whats an Intended Learning Outcome?

• Big-picture learning (summative)
• Most important, integrative concepts
• Most important, integrative processes
• Laying out a solid foundation for future course
  work
• Real world application

19
Whats an Objective?

• Small-picture learning (formative)
• Skills that build toward the Intended Learning
  Outcome

20
Kinds of objectives

• Knowledge (thinking)
• Skills (doing)
• Abilities (mastery)
• Attitudes (feeling)
• Motivation (initiative)
• Maturity (reasoning)
• Deciding (problem solving)

21
What level of thinking (cognitive)?

• Creation - highest order
• Evaluation - higher order
• Synthesis
• Analysis
• Application
• Comprehension
• Knowledge - lower order

22
(At least) three components of an Outcome or
Objective

• 1. Condition
• 2. Performance or behavior
• 3. Criteria or standard

23
Practice in writing an outcome and objectives

• Select a course youre currently teaching or
  getting ready to teach
• Whats the most important Concept or Process you
  want your students to learn by the end of the
  course? (Outcome)
• What are the knowledge, skills, or attitudes
  your students need to learn in order to learn
  that most important Concept or Process?
  (Objectives)

24
Whats an assessment or evaluation task or
technique?

• Task or technique you use to assess or evaluate
  whether or not, and how well, the student has
  learned the outcome or objectives

25

• Formative techniques
• Summative techniques

26

• Paper-pencil techniques
• Non paper-pencil techniques
• Performance techniques
• Classroom
• Authentic or real world

27
Authentic performance assessment

• Develop one authentic performance assessment
  task for your Intended Learning Outcome

28
Gauging learners progress in and reaction to
your course

• Formative techniques
• Assessing course-related knowledge and skills
• Assessing learner attitudes, values, and
  self-awareness
• Assessing learner reactions to instruction

29
Course-related knowledge and skills

• Prior knowledge, recall, and understanding
• Background knowledge probe (1, page 4.1)
• Focused listing (2, page 4.1)
• Misconception/preconception check (3, page 4.1)
• Empty outlines
• Memory matrix (5, page 4.1)
• Minute paper (6, page 4.2)
• Muddiest point (7, page 4.2)

30

• Skill in analysis and critical thinking
• Categorizing grid (8, page 4.2)
• Defining features matrix (8, page 4.2)
• Pro and con grid (10, page 4.2)
• Content, form, and function outlines
• Analytic memos

31

• Skill in synthesis and creative thinking
• One-sentence summary (13, page 4.3)
• Word journal
• Approximate analogies
• Concept maps
• Invented dialogues
• Annotated portfolios

32

• Skill in problem solving
• Problem recognition tasks
• Whats the principle?
• Documented problem solutions
• Audio- and videotaped protocols

33

• Skill in application and performance
• Directed paraphrasing (23, page 4.3)
• Application cards (24, page 4.3)
• Student-generated test questions (24, page 4.3)
• Human tableau or class modeling
• Paper or project prospectus

34
Learner attitudes, values, and self-awareness

• Students awareness of their attitudes and values
• Classroom opinion polls (28, page 4.4)
• Double-entry journals
• Profiles of admirable individuals
• Everyday ethical dilemmas
• Course-related self-confidence surveys (32, page
  4.4)

35

• Students self-awareness as learners
• Focused autobiographical essays
• Interest/knowledge/skills checklists (34, page
  4.4)
• Goal ranking and matching
• Self-assessment of ways of learning

36

• Course-related learning and study skills,
  strategies, and behaviors
• Productive study-time logs
• Punctuated lectures (38, page 4.5)
• Process analysis
• Diagnostic learning lots

37
Learner reactions to instruction

• Learner reactions to teachers and teaching
• Chain notes (41, page 4.5)
• Electronic mail feedback
• Teacher-designed feedback forms (43, page 4.5)
• Group instructional feedback technique (44, page
  4.6)
• Classroom assessment quality circles

38

• Learner reactions to class activities,
  assignments, and materials
• RSQC2 (Recall, summarize, questions, comment, and
  connect)
• Group-work evaluations
• Reading rating sheets (48, page 4.6)
• Assignment assessments (49, page 4.6)
• Exam evaluations

39
Dealing with assessment data you gather

• Page 5

40
Testing your teaching and their learning
41
Developing appropriate course assignments and
tests

• ESL-friendly (page 9.1-9.2)
• Evaluating fairly with tests (page 10)
• Developing test items (pages 11.1-11.5)

42
Evaluating (and grading) student work (page 8)

• Norm-referenced grading
• Criterion-referenced grading

43
Four levels of evaluating

• Reaction
• Learning
• Behavior
• Results
• Fifth Return on Investment (ROI)

44
What will you do now to assess and/or evaluate
learning?
45
Feedback on workshop