Writing a Program Level Assessment Plan

1
Writing a Program Level Assessment Plan
2
Student LearningOutcomes
3
Student Learning Outcomes

• Students will recognize, analyze, and interpret
  human experience in terms of personal,
  intellectual, and social contexts.

4
Student Learning Outcomes

• Students should be able to critically comprehend,
  interpret, and evaluate written, visual, and
  aural material.

5
Student Learning Outcomes

• Students should be able to
• ltltaction verbgtgt ltltsomethinggtgt

6
Student Learning Outcomes

• Learner Centered
• Specific
• Action oriented
• Cognitively appropriate for the program level

7
COMPREHENSION
EVALUATION
APPLICATION
ANALYSIS
SYNTHESIS
KNOWLEDGE
Associate Classify Compare Compute Contrast Differ
entiate Discuss Distinguish Estimate Explain Expre
ss Extrapolate Interpolate Locate Predict Report R
estate Review Tell Translate
Analyze Appraise Calculate Categorize Classify Com
pare Debate Diagram Differentiate Distinguish Exam
ine Experiment Inspect Inventory Question Separate
Summarize Test
Arrange Assemble Collect Compose Construct Create
Design Formulate Integrate Manage Organize Plan Pr
epare Prescribe ProducePropose Specify Synthesize
Write
Appraise Assess Choose Compare Criticize Determine
Estimate Evaluate Grade Judge Measure Rank Rate R
ecommend Revise Score Select Standardize Test Vali
date
Cite Count Define Draw Identify List Name Point Qu
ote Read Recite Record Repeat Select State Tabulat
e Tell Trace Underline
Apply Calculate Classify Demonstrate Determine Dra
matize Employ Examine Illustrate Interpret Locate
Operate Order Practice Report Restructure Schedule
Sketch Solve Translate Use Write
Lower division course outcomes
8
COMPREHENSION
EVALUATION
APPLICATION
ANALYSIS
SYNTHESIS
KNOWLEDGE
Associate Classify Compare Compute Contrast Differ
entiate Discuss Distinguish Estimate Explain Expre
ss Extrapolate Interpolate Locate Predict Report R
estate Review Tell Translate
Analyze Appraise Calculate Categorize Classify Com
pare Debate Diagram Differentiate Distinguish Exam
ine Experiment Inspect Inventory Question Separate
Summarize Test
Arrange Assemble Collect Compose Construct Create
Design Formulate Integrate Manage Organize Plan Pr
epare Prescribe ProducePropose Specify Synthesize
Write
Appraise Assess Choose Compare Criticize Determine
Estimate Evaluate Grade Judge Measure Rank Rate R
ecommend Revise Score Select Standardize Test Vali
date
Cite Count Define Draw Identify List Name Point Qu
ote Read Recite Record Repeat Select State Tabulat
e Tell Trace Underline
Apply Calculate Classify Demonstrate Determine Dra
matize Employ Examine Illustrate Interpret Locate
Operate Order Practice Report Restructure Schedule
Sketch Solve Translate Use Write
Upper division Course / Program outcomes
9
Possible Outcomes

• Students will appreciate.
• Students will be exposed to.
• Students will demonstrate knowledge of the
  history, theories and applications of
• Students will attend.
• Faculty will provide students with opportunities
  to.

10
Identify Components of the Outcomes
11
Components

• Define student learning outcomes
• Provide a common language for describing student
  learning
• Must be outcome specific
• Must be shared across faculty
• Number of components will vary by outcome

12
Components
Communication
Speak
Relate
Listen
Participate
Write
delivery
Eulogy
content
organization
13
Components
Communication
Speak
Relate
Listen
Participate
Write
mechanics
Letter
style
organization
14
Communication
Relate
Speak
Listen
Participate
Write
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
15
The Reality of Assessing Student Learning Outcomes

• Why you need common components

16
Course
Course
Course
Course
Course
Speaking
volume
eye contact
gestures
sources
transitions
poise
style
rate
examples
verbal variety
conclusion
appearance
evidence
organization
attention getter
17
Can our students deliver an effective Public
Speech?
volume
eye contact
gestures
sources
transitions
poise
style
rate
examples
verbal variety
conclusion
appearance
evidence
organization
attention getter
18
a little quiz
19
Example 1

• Gather factual information and apply it to a
  given problem in a manner that is relevant,
  clear, comprehensive, and conscious of possible
  bias in the information selected
• BETTER Students will be able to apply factual
  information to a problem
• COMPONENTS
• Relevance
• Clarity
• Comprehensiveness
• Aware of Bias

20
Example 2

• Imagine and seek out a variety of possible goals,
  assumptions, interpretations, or perspectives
  which can give alternative meanings or solutions
  to given situations or problems
• BETTER Students will be able to provide
  alternative solutions to situations or problems
• COMPONENTS
• Variety of assumptions, perspectives,
  interpretations
• Analysis of comparative advantage

21
Example 3

• Formulate and test hypotheses by performing
  laboratory, simulation, or field experiments in
  at least two of the natural science disciplines
  (one of these experimental components should
  develop, in greater depth, students laboratory
  experience in the collection of data, its
  statistical and graphical analysis, and an
  appreciation of its sources of error and
  uncertainty)
• BETTER Students will be able to test
  hypotheses.
• COMPONENTS
• Data collection
• Statistical Analysis
• Graphical Analysis
• Identification of sources of error

22
Identify Performance Characteristics
23
Performance Characteristics

• Scale or description for assessing each of the
  components
• Two to Five-point scales for each component
• Anchored with descriptions and supported by
  examples

24
Performance Characteristics

• Level or degree
• Accurate, Correct
• Depth, Detail
• Coherence, Flow
• Complete, Thorough
• Integration
• Creative, Inventive
• Evidence based, supported
• Engaging, enhancing

25
Performance Rubric
OUTCOME
Performance Anchors
Exceeds Expectations
Meets Expectations
Does not meet Expectations
Components
Descriptions of Performance
26
Performance Rubric
OUTCOME
Performance Anchors
Exemplary
Essential
Basic
Components
27
(No Transcript)
28
www.winona.edu/air/rubrics.htm
29
Student Learning Outcome
Components
Test Questions
1
1
2
3
4
2
5
6
7
8
9
3
10
11
12
4
17
13
14
15
16
18
30
Link Outcomes to Curriculum
31
Student Learning Outcomes
Course 1
Course 2
Course 3
Course 4
Course 5
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
32
Assessing Student Learning Outcomes

• Supported by core courses in the curriculum?
• Orphan outcomes?
• Empty requirements?

33
Student Learning Outcomes
Course 1
Course 2
Course 3
Course 4
Course 5
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
34
Student Learning Outcomes
Course 1
Course 2
Course 3
Course 4
Course 5
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
35
ImplementationStrategy
36
Implementation Strategy

• Identify Assessment Points

37
Student learning outcomes
Course 1
Course 2
Course 3
Course 4
Course 5
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
38
Implementation Strategy

• Phase in Assessment of learning outcomes

39
Phase 4
Student learning outcomes
Course 1
Course 2
Course 3
Course 4
Course 5
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
40
Implementation Strategy

• Collect and aggregate data

41
Communicate Effectively Demonstrate Oral
Communication Skills
PROGRAM WIDE COMPETENCY REPORT
Does not meet
Meets
Exceeds
Verbal Delivery
20
65
15
57
30
Nonverbal Delivery
13
58
Organization
24
18
Evidence
Transitions
42
Student Learning Outcome
Components
Test Questions
correct
1
1
2
3
4
78
2
5
6
7
8
9
56
3
10
11
12
82
4
42
17
13
14
15
16
18
43
Interpreting Data
44
Interpreting Data

• IMPORTANT!! The first data collection in not
  about the data per se, but instead about testing
  the methods and tools.

45
Interpreting Data

• Consistency - over time
• Consensus - different populations
• Distinctiveness - different situations/ variables
  / items

46
Closing the Assessment Loop
47
Closing the Loop

• DANGER!!!!!
• Dont be in a rush to close the loop!
• At the same time, dont let the process become
  gaseous

48
Closing the Loop

• What plans were implemented to address the
  concerns identified?

49
Closing the Loop

• Development
• Faculty, Staff, Student
• Infrastructure
• Policy, Process, Planning
• Curriculum
• Learning Opportunities

50
Linking Assessment to Teaching and Learning Using
the Seven Principles for Good Practice in
Undergraduate Education

• 1. Student-Faculty Contact
• 2. Cooperative Learning
• 3. Active Learning
• 4. (Prompt) Feedback
• 5. Time on Task
• 6. High Expectations
• 7. Respect for Diverse Talents and Ways of
  Learning