Understanding Student Learning Outcomes and Assessment - PowerPoint PPT Presentation

Loading...

PPT – Understanding Student Learning Outcomes and Assessment PowerPoint presentation | free to download - id: 6b0590-Mzk5Y



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Understanding Student Learning Outcomes and Assessment

Description:

Understanding Student Learning Outcomes and Assessment Bob Barr Andrew LaManque Office of Institutional Research and Planning Breakout Session, District Opening Day ... – PowerPoint PPT presentation

Number of Views:1
Avg rating:3.0/5.0
Date added: 13 September 2019
Slides: 45
Provided by: ftrapp
Learn more at: http://research.fhda.edu
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Understanding Student Learning Outcomes and Assessment


1
Understanding Student Learning Outcomes and
Assessment
  • Bob Barr
  • Andrew LaManque
  • Office of Institutional Research and Planning

Breakout Session, District Opening Day, September
21, 2006
Adapted from material prepared by Fred Trapp,
Long Beach CC, the RP Group, and James O.Nichols,
University of Mississippi
2
Session Outcomes
  • Describe a multi-step process for conducting
    learning outcomes assessment and understand the
    purpose of each step.
  • Distinguish among levels and types of outcomes
    and assessment work.

3
What Is Student Learning Outcomes Assessment?
  • An ongoing process of systematically, gathering,
    analyzing, and interpreting evidence aimed at
    enabling the institution to improve student
    learning and success.
  • Requires making learning expectations explicit
    and public.
  • Requires setting appropriate criteria and high
    standards for learning quality.

4
Why Do SLO Assessment?
  • Improve student learning and institutional
    support for student learning and success.
  • Provide for continuous reflection on improving
    teaching-learning and support environments.
  • Document and display our work for students, the
    public for Regional or program accreditation.

5
How Does SLO Assessment Differ From What We
Already Do?
  • Its a matter of unit of analysis and purposes.
  • It is assessment of our work rather than
    students work.
  • Its purpose is to enable us to improve the work
    we do (of enabling students to learn).
  • Data is assembled in the aggregate rather than
    for an individual student

6
How Does SLO Assessment Differ From What We
Already Do?
Purpose
Summative
Formative
Assessment for Evaluation
Assessment for Improvement
Institutional
Level
Assessment for Grades
Assessment for Learning
Individual
7
The Assessment Process
8
Key Questions for Each Assessment Process Step
  • Identify institutional mission institutional
    student learning outcomes link levels of
    outcomes
  • Where and how does this program fit into the
    mission and learning outcomes of the college?
  • Specify intended learning outcomes
  • What learning comes of completing the course,
    program, or degree?

9
Key Questions for Each Assessment Process Step
  • Identify means of assessment and criteria for
    success
  • How shall we determine what students have
    learned? How good is good enough?
  • Assemble the results and make meaning from them
  • How well did students do relative to our
    expectations?

10
Key Questions for Each Assessment Process Step
  • Use the results to redesign learning environments
  • What shall we do with these results? How shall
    we redesign our learning and support environments
    to make the more effective for student learning
    and success?

11
Levels of SLO Assessment Work
  • Institutional Outcomes GE outcomes or core
    competencies
  • Instructional or Student Services Program
    Outcomes
  • Course Outcomes

12
Step 1. Identify Mission, Institutional Outcomes
and Link Levels
  • Mission of the college
  • Broad statements of defining purposes and
    outcomes
  • GE, program, course purpose and outcome
    statements
  • Link the GE package and programs to the college
    mission and institutional outcomes
  • Link course outcomes to GE and program outcomes
  • Purpose of assessment ( self-study)
  • How well are we accomplishing our purposes and
    how can we improve?

13
Step 1. Example of Linking the Three Levels
  • College or GE Outcome
  • Students will be able to speak and write
    effectively.
  • Academic Program Outcome (Psychology)
  • Students can speak and write effectively using
    the principles and concepts of the discipline.
  • Course Outcome (Psych Measurement)
  • Students can prepare a written summary and
    interpretation of standardized test results.

14
Two Types of Educational Performance Outcomes
  • Student Achievement Outcomes
  • Success rates
  • Retention or persistence rates
  • Number and rates of graduates or transfers
  • Student Learning Outcomes
  • What the college intends students to know, value,
    or be able to do when they have completed a
    degree, program, or course.

15
Step 2. Specify Intended Learning Outcomes
  • Ideally, the curriculum and courses are designed
    by working backwards from the ultimate outcomes.
  • What roles will program graduates hold and what
    must they be able to do out there in the real
    world or in upper division work?
  • How will the outcomes be demonstrated or assessed
    in here?
  • What skills and concepts need to be learned?

16
Step 2. Specify Intended Learning Outcomes
  • Helpful hints
  • Use concrete, action verbs
  • See Blooms Taxonomy of Educational Objectives
  • Anticipate the deliverable
  • What is it that students must do to convince you
    that they have learned what you want them to
    learn?
  • Focus on terminal rather than enabling outcomes
  • Target key concepts or change points in the
    program.
  • Focus on major aspects of learning central to the
    discipline, program, or course.

17
Example Specify Intended GE Learning Outcomes
  • Be exposed to good writing examples. (in need of
    repair)
  • Upon completion of a degree, the student will be
    able to
  • Write effectively
  • Compose a range of written products designed for
    different audiences and purposes.

18
Example Specify Intended Psychology Program
Learning Outcomes
  • Hear and read about the scientific method of
    inquiry as used in the social sciences. (in need
    of repair)
  • Upon completion of the program, the student will
    be able to
  • Use critical thinking, skeptical inquiry, and,
    when possible, the scientific approach to solve
    problems related to behavior and mental processes.

19
Example Auto Tech Program Intended Student
Learning Outcomes
  • Graduates of the Automotive Technology Program
    will be successfully employed in the field.
    (achievement outcome, not a SLO)
  • Graduates of the Automotive Technology Program
    will be technically proficient.
  • Employers of Automotive Technology Program
    graduates will rate them competent based on the
    education received in the program and would
    employ others.

20
Example Specify Intended Student Services
Program Learning Outcomes
  • Know how to be a more prepared student. (in need
    of repair)
  • Upon completion of the EOPS summer transition
    program, the student will be able to
  • Develop and use a personal schedule that
    incorporates regular study time, regular study
    location and study groups to match their class
    schedule.

21
Characteristics of Good SLO Statements
  • Student-focused, not teacher focused
  • Focused on learning resulting from an activity,
    not the activity itself
  • Focused on important, non-trivial aspects of
    learning
  • Focused on skills abilities central to the
    discipline, based on professional standards of
    excellence

22
Characteristics of Good SLO Statements
  • General enough to capture important learning but
    clear and specific enough to be measurable
  • Focused on aspects of learning that will develop
    and endure but that can be assessed in some form
    now

23
Aligning Outcomes,Teaching-Learning and
Assessment
Outcome statements Methods of teaching/ learning Assumptions about teaching/ learning Methods of assessing Assumptions about methods
What do you expect students to know, understand, be able to do as a result of your teaching? What methods develop/ foster your primary outcomes? What assumptions underlie your methods? What is your theory of learning? What methods do you believe provide evidence of desired student outcomes? What assumptions underlie your belief that each method will elicit students demonstration of your desired outcomes?
24
Step 3. Identify Assessment Means
  • Levels of assessment
  • Institutional (GE, Core Abilities)
  • Program (Deg, cert, or course sequence)
  • Course
  • Types of evidence
  • Direct
  • Indirect

25
Step 3. Identify Assessment Means
  • Direct evidence of student learning outcomes
    (think in the aggregate)
  • Passage of license, certification or subject area
    exams
  • Results of general education exams
  • Capstone projects or performances
  • Employer or internship supervisor ratings of
    student performance
  • Class assignments, projects, etc.
  • Departmental exam results
  • Grades (sometimes) in the aggregate

26
Step 3. Identify Assessment Means
  • Indirect evidence of student learning outcomes
    (think in the aggregate)
  • Student self-reflective reports
  • Surveys of perception
  • Alumni, graduating or current students
  • Focus group interviews

27
Step 3. Identify Assessment Means
  • General Types of Learning Outcomes
  • Cognitive (knowledge, thinking disciplines)
  • Attitudinal (values, beliefs)
  • Behavioral/performance (use of knowledge, values,
    skills, or abilities)

28
Example Institutional, Program, Course
Assessment vs. Individual Student Assessment
Total down the column for individual grades.
Analyze across the row for assessment of intended
outcomes from the group.
James Nichols
29
Step 3. Identify Assessment Means
  • Embedded Assignments
  • Case study
  • Portfolio (paper or electronic)
  • Capstone project
  • Senior exit essay
  • Performances
  • Critical situation scenario
  • Debate
  • Demonstration
  • Oral presentation
  • Online monitoring
  • Tests
  • Standardized test
  • Homegrown test
  • In class objective test
  • In class essay
  • Student Centered
  • Exit interview
  • Focus group
  • Self-rating of progress
  • Archival
  • Library use, on-line activity
  • Transcript analysis
  • Transfer records
  • Stakeholders Perspective
  • Alumni, employee, employer surveys
  • Experience of faculty who inherit our students

Weve always done this, but now used
systematically to improve student learning rather
than only to assign grades.
30
Step 3. Identify Assessment Means Cognitive
Outcomes
  • Common course exams or questions
  • Sampled embedded class assignments
  • Portfolios

31
Step 3. Identify Assessment Means
Attitudinal/Values Outcomes
  • Surveys of students, graduates
  • Exam or task to explore attitudes and/or to apply
    values

32
Step 3. Identify Assessment Means Behavioral
and Performance
  • Case studies
  • Internships
  • Simulations
  • Projects
  • Capstone course work
  • Public performance

33
Step 3. Criteria for Success
  • Identify the criteria before assessment is done
  • How well ought the completers (on average)
    perform?
  • Criteria are a result of dialogue, professional
    judgment and are a collective responsibility

34
Step 3. Criteria for Success -Using Rubrics
  • Key Questions
  • What essential elements must be present in
    student work to ensure high quality?
  • What does student work look like at each level of
    mastery?

35
Step 3. Criteria for Success - Elements of a
Rubric
  • Dimensions/criteria of rating (the rows)
  • Levels of mastery (the columns) e.g.,
  • Beginner
  • Developed
  • Accomplished
  • Commentary describing features of work at each
    level of mastery (the cells)
  • Build from strengths and weaknesses teachers see
    in student work over the years

36
Step 3. Means of Assessment - Course Exams
  • Agreed upon course exam questions
  • Monitor student responses to identify topics of
    weakness
  • Item analysis to discern what groups are having
    difficulty with selected items
  • Revisit teaching approach to the topic

37
Step 3. Means of Assessment - Embedded
Assignments
  • Embedded narrative class assignments
  • Evaluated for the grade mark
  • Later a sample is re-inspected to
  • Assess program accomplishment against expected
    outcomes
  • Explore the learning process
  • Agreed upon performance criteria
  • The fruits of campus dialogue
  • Expert judgment of the faculty
  • Inter rater reliability

38
Step 3. Means of Assessment - Portfolios
  • A collection of student work
  • Selective or all inclusive
  • Displays what students can do
  • Best if student is required to reflect upon and
    evaluate their own work and learning
  • Caution in program use
  • Require the same work product from all students
    at pre-determined points in time.

39
Example Criteria for Success - Automotive
Technology Program

Means of Program Assessment and Criteria for
Success 2a. At the close of their final term,
90 of the graduates will be able to identify
and correct within a given period of time all of
the mechanical problems in five test cars that
have been "prepared" for the students by
Automotive Technology Program faculty. No single
automotive malfunction will fail to be identified
and corrected by more than 20 of students. 2b.
80 of Automotive Technology Program graduates
will pass the National Automotive Test. On no
subscale will students average missing 30 or
more of the items. 3. 80 of the respondents to
an Employer Survey conducted every 3 years by the
college will respond that they find graduates
competent and would employ others.
Program Intended Educational Outcomes 2.
Graduates of the Automotive Technology Program
will be technically proficient. 3.
Employers of the Automotive Technology Program
graduates will be rate them competent based on
the education received in the program.
40
Step 4. Assembling and Interpreting Assessment
Results
  • Assembling Results
  • Logistics of doing assessment
  • Recording results
  • Making meaning
  • The creative tension between ought and is

41
Example Recording ResultsGraduates of the
Automotive Technology Program will be
technically proficient.

Summary of Data Collected 2a. 79 overall
success rate. Electrical system malfunction
undetected by 34 of students. 2b. Pass
rate on National Automotive Test was 83
however, on hydraulic theory subscale students
missed an average of 34 of questions.
Means of Program Assessment and Criteria for
Success 2a. At the close of their final term,
90 of the graduates will be able to
identify and correct within a given period of
time all of the mechanical problems in five test
cars that have been "prepared" for the students
by Automotive Technology Program faculty. No
single automotive malfunction will fail to be
identified and corrected by more than 20 of
students. 2b. 80 of Automotive Technology
Program graduates will pass the National
Automotive Test. On no subscale will students
average missing 30 or more of the items.
42
Step 5. Using Results to Redesign Learning
Environments
  • Adjust what we teach
  • Adjust how we teach
  • Adjust the means of assessment
  • Adjust where we put the bar of success

43
Example Auto Tech Program(Using Results
Documenting Those Actions)

Use of Results 2a. Expanded electrical
trouble-shooting component of AT 202 to include
automotive electrical systems. 2b. Modified
means of teaching hydraulic theory during AT 102
(Basic Auto Systems). 3. Added body shop
representative to Advisory Committee and are
reviewing curriculum to determine if separate
program is needed.
Summary of Data Collected 2a. 79 overall
success rate. Electrical system malfunction
undetected by 34 of students. 2b. Pass rate
on National Automotive Test was 83 however, on
hydraulic theory subscale students missed an
average of 34 of questions. 3. 90 reported
willingness to employ graduates, but only 50 of
body shops.
44
The Bottom Line on Assessment
  • SLO assessment is about ourselves and our work
    because
  • A Learning College is a learning organization
    composed of learners intent improving its
    achievement of its mission i.e., intent on
    improving student learning and success.
About PowerShow.com