Student Outcomes Assessment: Strategies for Getting Started and Sustaining the Momentum

1
Student Outcomes AssessmentStrategies for
Getting Started and Sustaining the Momentum

• Presented by
• Peggy Maki, Ph.D.

2
Some Questions about Teaching/Learning

• What do you expect your students to know and be
  able to do by the end of their education at your
  institution?
• What do the curricula and other educational
  experiences add up to?
• What do you do in your classes or in your
  programs to promote the kinds of learning or
  development that the institution seeks?

3
Questions (cond)

• Which students benefit from which classroom
  teaching strategies or educational experiences?
• What educational processes are responsible for
  the intended student outcomes the institution
  seeks?
• How can you help students make connections
  between classroom learning and experiences
  outside of the classroom?
• How do you intentionally build upon what each of
  you teaches or fosters to achieve institutional
  educational objectives?

4
The Assessment Loop
Gather Evidence
Interpret Evidence
Mission/Purposes Educational Objectives
How well do we achieve our educational
objectives?
Enhance teaching/ learning inform institutional
decision- making, planning, budgeting
5
Principles of Assessment

• Successful assessment effort are compatible with
  the institutions mission and its available
  resources.
• Assessment is not a one-time activity rather, it
  is evolutionary, ongoing, and incremental. Over
  time assessment efforts should become more
  comprehensive, systematic, integrative, and
  organic.
• Assessments primary focus is on the
  teaching/learning experience.
• Institutional autonomy should be preserved and
  innovation encouraged to provide the highest
  quality education.

6
Infrastructure As It Relates to Student Outcomes
Assessment
Library and Information Resources
Student Services (Extra and Co-Curricular)
Methods of Teaching/ Learning
Technology
Mission and Purposes
Alumni Follow-up
Educational Opportunities
Admissions
Advising
Orientation
7
Identify Outcome or Significant Question You Want
to Ask about Student Learning?

• Based on the mission of your institution or the
  mission of your program, identify a core outcome
  of your program or of the institution.
• Or, identify a central question or statement
  about student learning that has been raised on
  your campus? (Students dont write as is well as
  we expect in upper level courses.)
• Clarify what people mean by the question or
  statement.
• Start small build over time.

8
Domains for Assessing Student Learning

• Basic (entry) skills
• College-level skills
• General Education
• The major or professional or vocational program
• Social, spiritual, ethical development
• Other?

9
Articulating Core Outcomes

• Select verbs that specify the trait, ability,
  behavior, habit of mind that you wish to assess.
  Use active verbs, such as
  
  
• Create
• Compose
• Calculate
• Build
• Develop
• Evaluate

Be specific avoid general descriptors
10
Compare

• Students will write
• effectively.

to

• Students will compose a range of
  professional documents designed to solve
  problems for different audiences and purposes.

11
Compare

• Students will write
• effectively

to
Students will summarize recent articles on
economics and identify underlying economic
assumptions.
12
Example from ACRL
Literate student evaluates information and its
sources critically and incorporates selected
information into his or her knowledge base and
value system.
ONE OUTCOME Student examines and compares
information from various sources in order to
evaluate reliability, validity, accuracy,
timeliness, and point of view or bias.
13
Articulating Criteria

• List the major criteria you will use to assess
  the trait, ability, behavior, habit of mind. That
  is, on what basis will you be able to ascertain
  how well a student is achieving or performing or
  using or integrating x?
• Criterion 1
• Criterion 2
• Criterion 3
• Continue listing essential criteria.

14
Identifying Level of Expected Performance

• Determine your expected level of student
  performance. Is that level established by the
  institution, a program at the institution, by a
  national body, etc.?

One way to develop expectations is to assemble a
cross-disciplinary group that will articulate
what the expected level of performance should be.
15
Venues for Assessment

• Course-embedded
• In-class
• Out-of-class
• Off-campus
• On-line

16
Who Does Assessment?

• Faculty
• Students
• Academic Support Staff
• Administrators
• Internship Advisor
• Employers
• Graduates
• Faculty from Other Institutions
• Others?

17
Some Methods That Provide Direct Evidence

• Student work samples
• Collections of student work (e.g. Portfolios)
• Capstone projects
• Course-embedded assessment
• Other observations of student behavior
• Internal juried review of student projects

18
Direct Evidence (cond)

• External juried review of student projects
• Externally reviewed internship
• Performance on a case study/problem
• Performance on national licensure examinations
• Locally developed tests
• Standardized tests
• Pre-and post-tests
• Essay tests blind scored across units

Strategy to get started Use what you are already
doing.
19
Some Methods That Provide Indirect Evidence

• Alumni, Employer, Student Surveys
• Focus groups
• Exit Interviews with Graduates
• Graduate Follow-up Studies
• Percentage of students who go on to graduate
  school
• Retention and Transfer Studies
• Job Placement Statistics

20
Methods That Do Not Provide Direct Evidence but
May Be Combined with Other Methods

• Faculty publications (unless students are
  involved)
• Courses selected or elected by students
• Faculty/Student ratios
• Percentage of students who study abroad
• Enrollment trends
• Percentage of students who graduate within
  five-six years
• Diversity of student body

21
Four Strategies for Effective Data Reporting

• Disaggregate the data.
• Report results using graphics and comparative
  formats. (Show trends over time, gender
  differences, differences by race or ethnic
  background.)
• Publish short, issue-specific reports or research
  briefs. (Organize presentation of results around
  issues of interest,not the format of the data.)
• Combine outcomes information with other data.
  (When relevant, incorporate statistics about
  course enrollments, FTE students, credit hours
  earned, number of majors.)
• M.Kinnick,Four Strategies.

22
Meaningful Use of Data

• Collect data from different sources to make a
  meaningful point (for example, samples from
  inside and outside of the classroom.
• Collect data you believe will be useful to
  answering the question you have raised.
• Organize reports around issues, not solely data.
• Interpret your data so that it informs pedagogy,
  budgeting, planning, decision-making, or
  policies.

23
Part I Determining Your Institutions
Expectations

• Articulate Expected Outcome
• Identify Where Expected Outcome is Addressed
• For example, in courses, programs, projects,
  services, work experience
• Identify and Collect Baseline Information
• For example,portfolios, performances, essays,
  tests.
• Determine Methods and Criteria by Which the
  Outcome Will be Assessed
• For example, grade on a case study based on
  agreed upon criteria
• Articulate Institutions Level of Expectation
• For example,quantitative reasoning

24
PART II Determining Timing, Identifying
Cohorts, Assigning Responsibility

• Establish a Schedule
• Upon matriculation, upon acceptance into a
  program
• At the end of a specific semester
• At the completion of a required set of courses
• Upon graduation
• Upon employment
• In graduate school
• Other times?
• Determine Whom You Will Assess
• All students
• Categories of students
• Determine Who Will Assess
• Outside evaluators (agencies, faculty, employers)
• Inside evaluators (interdisciplinary teams,
• Assessment committee, centers or offices at
  the institution)

25
Part III Sharing Results and Enhancing
Institutional Effectiveness

• Determine How and with Whom Results Will Be
  Shared
• For example, General Education Sub-Committee,
  departments, planning and budgeting groups, other
  decision-making bodies?
• Determine How Results Will Inform
  Teaching/Learning and Associated Decision Making
• For example, revise pedagogy, curriculum,
  sequence of courses,advisingensure collective
  reinforcement of knowledge, competencies, habits
  of mind address faculty and staff development,
  resources, support services, etc.
• Repeat the Assessment Loop

26
Improving Teaching, Learning, Institutional
Culture

• Revising teaching strategies to improve student
  achievement
• Revising course content to assure appropriate
  attention to areas that need increased attention
• Replacing or revising courses, programs, services
  
• Changing sequence of courses
• Adding a requirement or required course
• Enhancing the advising process
• Increasing focus throughout the co-curriculum
• Increasing support services

27
Characteristics of Effective Assessment Methods

• are structured, systematic, and on-going, not
  episodic.
• relate to other institutional strategic
  long-range plans and to planning and budgeting
  processes.
• emerge from and sustained by a faculty and
  administrative commitment.
• provide explicit and public statements about
  institutional expectations for student learning.

28
Characteristics (cond)

• enable the institution to determine the fit
  between expectations for student learning and the
  level of learning achieved.
• provide encouragement and the means to test
  changes that could improve learning.

29
Necessary Elements for Successful Integration of
Assessment into Your Culture

• Leadership commitment
• Faculty commitment
• Office or committee to oversee and coordinate
  efforts (houses assessment materials, offers
  workshops or expertise)
• Means by which results will be shared to inform
  pedagogy, budgeting, planning, decision-making
• Means to monitor and report effects of changes
  once they are implemented
• Financial support

30
Benefits of Assessment Done Right

• Better information
• More and better student learning
• Stronger programs
• Intellectual stimulation, faculty rejuvenation
• Enhanced collegiality
• Improved campus-wide communication
• Better administrative decisions
• Happier employers of your graduates
• Better public relations
• Enhanced fundraising opportunities