Aligning Assessment Methods with Learning Outcome Statements and Curricular Design

1
Aligning Assessment Methods with Learning Outcome
Statements and Curricular Design

• Presented at
• CCRI
• April 8, 2005
• PeggyMaki_at_aol.com

Material from Maki, P. (2004). Assessing for
Learning Building a Sustainable Commitment
Across the Institution. Stylus Publishing and
AAHE.
2

• Alignmentdegree to which learning outcome
  statements match how and what we teach degree to
  which assessment methods match or build upon our
  collective educational practices
• What methods of assessment capture desired
  student learning--methods that align with
  pedagogy, content, and curricular design?

3

• The tasks to which students are asked to
  respond on an assessment are not arbitrary. They
  must be carefully designed to provide evidence
  that is linked to the cognitive model of learning
  and to support the kinds of inferences and
  decisions that will be based on the assessment
  results.
• National Research Council. Knowing what
  students know The science and design of
  educational assessment . Washington, D.C.
  National Academy Press, 2001, p. 47.

4
Assumptions Underlying Teaching
Actual Practices
Assumptions Underlying Assessment Tasks
Actual Tasks
5
When Will You Seek Evidence?

• Formativealong the way?
• For example, to ascertain
• progress or development
• Summativeat the end?
• For example, to ascertain
• mastery level of achievement

6
What Tasks Elicit Learning You Desire?

• Tasks that require students to select among
  possible answers (multiple choice test)?
• Tasks that require students to construct answers
  (students problem-solving and thinking
  abilities)?

7
Direct Methods

• Focus on how students represent or demonstrate
  their learning (meaning making)
• Align with students learning and assessment
  experiences
• Align with curricular-and co-curricular design
  verified through mapping

8

• Invite collaboration in design (faculty,
  students, tutors)

9
Standardized Instruments

• Psychometric approachvalues quantitative methods
  of interpretation
• History of validity and reliability
• Quick and easy adoption and efficient scoring
• One possible source of evidence of learning

10
Do Not Usually Provide

• Evidence of strategies, processes, ways of
  knowing, understanding, and behaving that
  students draw upon to represent learning
• Evidence of complex and diverse ways in which
  humans construct and generate meaning
• Highly useful results that relate to pedagogy,
  curricular design, sets of educational practices

11
Authentic, Performance-based Methods

• Focus on integrated learning
• Directly align with students learning and
  previous assessment experiences
• Provide opportunity for students to generate
  responses as opposed to selecting responses

12

• Provide opportunity for students to reflect on
  their performancestrengths, weaknesses,
  repositioned learning

13
Do Not Provide

• Immediate reliability and validity (unless there
  has been a history of use)
• Usually do not provide easy scoring unless
  closed-ended questions are used.

14
Some Options for Alternative Methods

• E-Portfolios
• Capstone projects (mid-point and end-point)
• Performances, productions, creations
• Visual representations (mind mapping, charting,
  graphing)

15

• Case studies
• Disciplinary or professional practices
• Agreed upon embedded assignments
• Selection of assignments students hand in
• Writing to speaking to visual presentation

16

• Team-based or collaborative projects
• Internships and service Projects
• Oral examinations/questions
• Critical incidents

17

• Externally or internally juried review of student
  projects
• Externally reviewed internship
• Performance on a case study/problem
• Performance on case study accompanied with
  students analysis

18

• Locally developed tests
• Pre-and post-tests
• Learning Logs or Journals
• Videotaping over time

19

• Simulationsvirtual labs, scenarios that track
  decision making and actions
• Magic boxproblem solving over time

20
Indirect Methods-- May Be Combined with Direct
Methods

• Focus group (representative of the population)
• Interviews (representative of the population)
• Surveys
• Transcript analyses

21

• Other sources of information that contribute to
  your inference making CCSSE results, grades,
  participation rates or persistence in support
  services, course-taking patterns, majors

22
Identify Methods to Assess Outcomes

• Using the handout, identify both direct and
  indirect methods you might use to assess several
  of your outcomes. Determine the kinds of
  inferences you will be able to make based on each
  method.

23
Examples of Changes

• Increased attention to weaving experiences across
  the institution, a program, or a department to
  improve student achievement
• Changes in advising based on assessment results
• Closer monitoring of student achievement--tracking

24

• Faculty and staff development to learn how to
  integrate experiences that contribute to improved
  student learning
• Changes in pedagogy and curricular and
  co-curricular design
• Development of modules to assist learning use of
  technology self-paced learning, supplemental
  learning

25
Gather Evidence
Interpret Evidence
Mission/Purposes Learning Outcomes
How well do we achieve our outcomes?
Enhance teaching/ learning inform institutional
decision- making, planning, budgeting
26
What and how students learn depends to a major
extent on how they think they will be assessed.
John Biggs, Teaching for Quality Learning at
University What The Student Does. Society for
Research into Higher Education Open University
Press, 1999, p. 141.
27
Works Cited

• Biggs, J. (1999). Teaching for Quality
  Learning at University What The Student Does.
  Society for Research into Higher Education Open
  University Press, 1999, p. 141.
• Maki, P. (forthcoming, 2004., May). Assessing
  for Learning Building a Sustainable Commitment
  Across the Institution. Sterling, VA Stylus
  Publishing, LLC, and the American Association for
  Higher Education.
• National Research Council. 2001. Knowing What
  Students Know The Science and Design of
  Educational Assessment. Washington, D.C.
  National Academy Press