California Community Colleges Registrars and Admissions Officers CACCRAO Development of Meaningful L

1
California Community Colleges Registrars and
Admissions Officers (CACCRAO) Development of
Meaningful Learning Outcomes

• Dotti Cordell BS, MPH, BSN
• Director of Student Health Services
• San Diego City College
• May 4, 2009

2
A Challenge to ...

review
rekindle



relearn

recharge
recapture

REEXAMINE

REMEMBER

replenish

RECONNECT

rechallenge

revisit

recall

rejuvenate

restore


REDISCOVER

recommit
redirect

reorient

And Restart?...

3
Presentation Objectives

• Review the process of Outcomes Assessment
• Clarify misconceptions relating to outcomes in
  Student/Administrative Services
• Explain the Six Column Model utilized at SD City
  College
• Equip you with an organized systematic knowledge
  base in this sometimes unfamiliar or
  uncomfortable territory
• Outline potential trouble spots

4
Choice and Attitude

• I hope that you can turn what might be a negative
  external mandate into a positive internal
  challenge.
• I challenge you to view assessment as an
  opportunity for admission program improvement and
  a means to seek the best possible way to serve
  students.

5
If you do what you've always done, you'll get
what you've always gotten
6
Do You Feel A Time-Crunch?

• What do I need to know?
• What does my department need to know?
• What do I do?

7
Dont Panic

• You and your department are probably already
  doing it.
• You just may not know it.
• Or you may not be documenting the process.
• Or you think there is a perfect way to do it and
  you are afraid to jump in.
• Or it does not seem natural and seems to be a
  large additional work load.

8
What is YOUR Idea of a Quality Admissions
Department? Of quality student service?
9
Quality What Is It?

• Quality of service the concept does not possess
  substance or mass.
• Cannot place quality on the table to analyze or
  study.
• Any definition of quality almost always seems
  deficient
• Quality is in the eye of the beholder.
• Quality definitions reflect the tastes and biases
  of those doing the defining.

10
Quality

• A quality admissions program may be one in which
  services for students faculty and administration
  increases the likelihood of desired outcomes and
  is consistent with current professional knowledge

11
Performance

• Service

P R O C E S S
(-)

(-)
12
Quality

• Doing the right things and doing them well.

13
Quality

• Naked assertions of high quality admissions,
  records, evaluations, counseling or any college
  department or program is no longer sufficient.
  Consumers, society, accrediting agencies and
  campus administrators hold us accountable to
  provide objective, quantifiable standards for
  quality student and service outcomes.

14
(No Transcript)
15
Admissions Assessment and Improvement

• You know, WE know, that admission departments
  help maintain the institution and are ESSENTIAL
  to operations.
• Assessment will help demonstrate that each staff
  member in your admissions/ registration
  departments are endeavoring to do their jobs
  better every day.

16
Assessment Principles

• Assessment is NOT a one-time activity it is
  evolutionary, ongoing, incremental.
• Over time, assessment efforts should become more
  comprehensive, systematic and integrated.
• Assessment efforts must be compatible with the
  institution's mission
• Diane Ramirez of College of the Desert and
  Merrill Deming at Crafton Hills College

17
How Can You/We Do It All?
FERPA
Budget Crises
Increasing Demand for Services
Day-to-Day Operations
OUTCOMES MEASUREMENT?
18
(No Transcript)
19
What is Assessment?

• Assessment is an ongoing process aimed at
  understanding/improving student learning and
  campus services and processes
• It serves to clarify our values
• Sets criteria and standards for learning and
  services
• We gather data to see how well performance
  matches standards
• Assessment is deciding what YOU as THE subject
  matter experts value and wish to measure and then
  using the resulting information to document and
  improve performance

20
Key Questions
The answers to these questions form the essence
of what is called assessment.

• What are we trying to do?
• What is it that admissions should accomplish?
• How is success defined?
• How successful are our students?
• How well have they learned?
• How do we know?
• How well are we performing our functions?
• How do we know?
• How do we use that information to continually
  improve?
• Does the improvements we make work and do we
  maintain the gain?
• Adapted From M. Bresciani, Director of Assessment
  SDSU and from CUPR Guidelines

21
Student Learning OutcomesDefinition

• The knowledge, skills, abilities, attitudes,
  habits of mind or competencies that students have
  at the completion of a
• Course
• Program
• Activity or interaction
• The presence and quality of which can be
  measured and evaluated.

22
Categorization of Student Learning Outcomes

• Complex Cognitive Skills
• Knowledge Acquisition
• Interpersonal Development
• Social Development
• Academic Achievement
• Persistence
• Practical Competence
• Civic Responsibility
• National Assoc. of Student Personnel
  Administrators, Upcraft 1999

23
Assessment to Improve Admissions
Processes/Services

• Redesign services based upon best practices
• Use information technology to improve access to
  information and to support decision-making
• Improve workforce knowledge and skills
• Develop effective teams within and outside the
  department(s)
• Coordinate referrals
• Measure performance and outcomes

24
Why Do We Use AnAssessment Framework?

• Model provides common language and a consistent
  framework
• Allows celebration of small wins
• Demonstrates accomplishments
• Is readily explained, one page
• Model clearly shows linkages with larger
  institutional competencies and college mission

25
San Diego City Colleges Six Column Model

• Credit to Jim Nichols We have adapted his work
  with over 380 colleges and are using an expansion
  of his model.
• Columnar model--helps individuals see linkages
  between levels of organization.

26
City College Six Column Assessment Model

• Column 1 Statement of Institutional Purposefor
  City this is our Mission and our Institutional
  Priorities
• Column 2 Institutional Competencies
• Communication/Interpersonal Skills
• Critical thinking
• Analyses/Computation
• Cultural Sensitivity/Global Awareness
• Information Management/ Literacy
• Personal Responsibility
• Civic and Environmental Responsibility
• Column 3 Programmatic Student Learning Outcomes
• Column 4 Means of Assessment and the Criteria
  for Success
• Column 5 Summarizes data collected (Results)
• Column 6 Designated to demonstrate how you will
  use results to improve the program

Note Linkages between the 6 columns is essential
27
Mission

• Your departmental mission is your reason for
  being, the motivation for doing this work, the
  soul of the department

28
Column One

• To guide City faculty and staff in formulating
  outcomes, column one is comprised of the City
  College mission statement and our college
  priorities

1
29
Admissions/Registration 2009 Outcome
30
Column Institutional Competencies
2

• Communication/Interpersonal Skills
• Critical thinking
• Analyses/Computation
• Cultural Sensitivity/Global Awareness
• Information Management/ Literacy
• Personal Responsibility
• Civic and Environmental Responsibility

31
Admissions/Registration 2009 Outcome
32
Column Development of Departmental/Programmati
c SLOs
3

• Departmental staff members convene to discuss and
  reach consensus on the focus of assessment for
  the coming semester/year.

33
Developing Programmatic Student Learning OR
Administrative Unit Outcomes

• Collect and review documents that describe the
  program (catalog, your programs mission)
• Examine outcome statements from similar programs
  at other institutions
• Review assessment criteria of WASC accrediting
  body
• Review criteria from your professional
  organization--CACCRAO
• Collect and review departmental materials
• Brainstorm with your staff and utilize the short
  list/long list concept

34
Long and Short List Concept

• Now choose 1-2 to study this semester/year. Have
  your department collaboratively discuss and reach
  consensus on what is important THIS year or
  semester
• 1.
• 2.

• Make a LOOONG list of all the learning or
  administrative outcomes of importance to your
  department
• 1.
• 2.
• 3.
• ..
• 42.

35
Writing Outcomes

• Uses clear, observable and measurable verbs
• Compile Arrange
  Classify
• Analyze Identify Operate
• Design Solve Write
• Apply Differentiate
  Calculate
• Demonstrate Formulate
  Compose
• Explain Predict Assess
• Compare Estimate
  Critique
• The following verbs are relatively vague states
  of mind. They are very difficult/ impossible to
  measure
• Know Understand Appreciate Learn
• Credit to Gavilan College for this content

36
Admissions/Registration 2009 Outcome
37
Column 4Means of Assessment and Criteria for
Success

• Rubber hits the road in this column
• Here you are defining the specifics of
  measurement and precisely what you meant in
  column 3. You are describing the context for the
  outcome. What do you mean that you want the
  student to be better at availing him/herself of
  campus resources? Able to navigate the catalog?
• It is in this column you must be very clear on
  the specifics of measurement, and at what point
  you will consider yourself successful. 78 of
  the students will..etc..

38
Admissions/Registration 2009 Outcome
39
Columns 5 and 6

• Column 5 Log results obtained and compare to the
  criteria for success that the department or
  program set for itself
• Column 6 Compare how actual results stacked up
  against the criteria for success. Will the
  department rotate the outcome off the list as
  accomplished and not problematic, or does it
  see the issue as an opportunity for programmatic
  improvement?
• What are the departmental actions to improve?
• NOW you have closed the loop and the entire
  goal of this assessment effort comes full cycle
  you now have data and actions to support program
  improvement efforts.

40
Admissions/Registration 2009 Outcome
41
Column 6 Closing the LoopKey Questions

• How well are we achieving the admissions/registrar
  program or department goals?
• How do we know?
• How can we use data/information to continually
  improve the processes and systems of the
  department?

42
Student Learning Outcomes And Administrative Unit
Outcomes Clarification FAQs

• What is a student learning outcome
  (SLO)?Knowledge, skills, abilities, attitudes,
  habits of mind that students have at the
  completion of a series of activities or
  interactions--the presence and quality of which
  can be measured and evaluated
• What is an AUO (Administrative Unit Outcome)?An
  AUO describes what an Administrative Unit such
  as your Admissions Depts. will accomplish or do
  in the provision of their services. The focus of
  AUOs is services. How you know you are
  accomplishing the purpose of Admissions, Records,
  Registration? THE FOCUS IS ON IMPROVEMENT IN
  SERVICES PROVIDED TO SUPPORT STUDENT LEARNING.
• Why is it important to include student support
  services in this work?Key to understanding the
  importance of this initiative is the premise that
  the entire campus supports and is crucial to
  student learning. As such it requires a
  commitment of the entire campus as part of a
  comprehensive Institutional Effectiveness program
• Why should I engage in outcome work if Im not in
  an instructional area of the campus?Student
  learning experiences most definitely do not just
  occur in the classroom. Some of the most
  important learning on campus occurs outside of
  the classroom. It is important that the campus
  participate to demonstrate learning outside the
  classroom and the efficiency of campus operations.

43
Similarities between SLOs and AUOs

• Both focus on improvement (in either student
  learning or administrative functioning)
• Both contribute to and are part of the quality
  picture of the entire campus
• Both use assessment data to effect improvement

44
Examples Of Programmatic Outcomes Admissions

• Admissions will shorten the response time for
  _____from 3 days to 2.5 days .
• Over the counter requests for __________will be
  delivered in ____minutes/hours versus
  ____minues/hours.
• 94 of students will articulate a high level of
  satisfaction with the overall registration
  process as evidenced by an increase from 2.7 to
  3.5 on a 5 point satisfaction scale.
• Or 85 of students will demonstrate a high(
  remember to define what you mean by high) level
  of satisfaction on exit interviews from the
  admissions dept.
• Admissions office complaints or dissatisfaction
  reports will decrease from ___ per semester to lt2
  per semester.
• Admissions professionals will contribute to the
  overall campus commitment to a diverse student
  body by increasing_____________.
• The Admissions department will institute the
  following interventions this semester to increase
  the students having a valid ID card prior to
  x date of the semester from ___ to ___
• Admissions will demonstrate an increase in
  students successfully( how is successfully
  defined?) navigating the college catalogue from
  ___ to ___ by______________________________.
• To improve the learning outcome related to
  services referral and campus networking---As
  reported on an exit questionnaire from the
  Admissions department, students will accurately
  report 3 of 5 departments necessary to meet their
  needs and educational goals.
• Admissions will provide accurate class enrollment
  data_____________.

45
Key Foundational Assessment Factors at City
College

• Not all outcomes are student learning outcomes,
  some may be administrative outcomes which are
  essential to, and form the foundation upon which,
  student learning and success are built
• This whole process is directed towards
  incremental program improvement, not individual
  evaluation
• The program is as the department/area/service
  defines it.
• Key themes emerged at City on learning outcomes
  and Student Services
• Students learn about other campus resources from
  those of us in Student Services
• Students learn from Student Services
  professionals how to better advocate for
  themselves
• Self-advocacy
• Self-assertiveness
• Networking
• To have buy-in, sustainability, and assessment of
  value-- it must be an inclusive staff/faculty
  driven process a grassroots effort
• Values driven
• You and your department are the subject matter
  experts what do YOU feel is important to study
  in your department?
• Part of daily/weekly/monthly/semester work--not a
  set-aside project

46
Student Services

• The campus is our classroom.
• All of us are teachers and each of us is
  essential to the college mission and the work and
  learning that occurs daily with students.
• Assessment is good---it demonstrates for us the
  vital role that each of us play in the
  educational process.

47
How To Start

• What do you and your dept. value or feel is
  important to do well in your area?
• Seek consensus in your area on choosing this
  issue to study.
• Pick one for now.
• Keep it simple
• Write it down
• Decide on a measure
• Observe students?
• Survey?
• Collect data
• Act on the results.
• Write it down.

48
Progressive improvements beat postponed
perfection every time
49
Direct and Indirect Measures

• Satisfaction surveys
• Provide feedback on student opinions
• Indirect measure of assessment
• Measure what the student feels or thinks about
  the program
• Move away from satisfaction surveys
• Optimal to use satisfaction measures in
  conjunction
• with other measures
• The assessment effort seeks to find what the
  students know, have learned or can demonstrate
• Direct measures of learning

50
Tips in Creating a Survey

• Limit number of questions
• Include explicit directions
• Check reading level
• Eliminate having two questions built into one
• Pilot the survey
• Consider alternative methods of
  distribution/collection
• Consider whether anonymity is an issue

51
Lessons from the Field

• A thousand mile march really does begin with your
  first step.
• Change is rarely convenient. The only who likes
  it is a wet baby.
• Who are the assessment champions? Persons willing
  to be accountable to keep the process going
  forward? Set meetings? Keep notes? HOPE IS NOT A
  PLAN.
• Focus, Focus, Focus. Start with just one
  question, one form, one anything!
• KNOW how you will know that what you have done
  is an improvement. Design for incremental
  improvement, not publication or perfection.
• Dont get hung up on names outcomes, objectives,
  competencies, goals, SLOs, AUOs, program
  definition. You know what is important to your
  dept.
• Sometimes you just have to jump in and DO IT!
• You are doing assessment much more than you
  thinkyou just are not taking credit for it.
  

52
(No Transcript)
53
Contact Information

• Dotti Cordell
• San Diego City College
• 1313 Park Blvd. Rm. A116
• San Diego, CA 92101
• [phone]
• Email dcordell_at_sdccd.edu