Title: Using CAS Standards and Review Process for Improving Student Learning and Program Outcomes
1Using CAS Standards and Review Process for
Improving Student Learning and Program Outcomes
- Patricia Carretta
- George Mason University
- CAS Officer-at-Large
- NACE Representative to CAS
- NASPA 2008 International Assessment Retention
Conference
2Purpose
- CAS Standards and Review Process
- Integrating CAS into assessment of programs and
services and student learning outcomes
3Role and Use of CAS Standards
- Student Affairs / Student Services at the
division level? - Student Affairs / Student Services at the
departmental or program level? - Other?
- Have looked at the CAS Standards for at least one
functional area? - Have participated in a department/program review
process using CAS standards? - Have used CAS standards for other purposes?
4Participant Outcomes
- Define the role of CAS self-assessment / review
process within a comprehensive assessment plan - List the seven step process of conducting a CAS
review - Appreciate the value of engaging a variety of
institutional stakeholders in the review process - Strategize how to conduct a comprehensive
assessment of programs, services and student
learning using CAS Standards - Formulate an assessment plan
- Understand the importance of assessing and
reporting on input data as well as student
learning and development outcomes as a strategies
for improving learning and program quality
5About CAS
The mission of the Council for the Advancement
of Standards in Higher Education (CAS) is to
promote the improvement of programs and services
to enhance the quality of student learning and
development. CAS is a consortium of
professional associations who work
collaboratively to develop and promulgate
standards and guidelines and to encourage
self-assessment.
6CAS Resources
- CAS Professional Standards, 2006 edition
- Self-Assessment Guides
- CAS Statement of Shared Ethical Principles
- CAS Characteristics of Individual Excellence
- Frameworks for Assessing Learning and Development
Outcomes
7CAS Guiding Principles
- Students and Their Institutions
- Diversity and Multiculturalism
- Organization, Leadership, and Human Resources
- Health Engendering Environments
- Ethical Considerations
8Organization of CAS Standards
- Mission
- Program
- Leadership
- Human Resources
- Ethics
- Legal Responsibilities
- Equity and Access
- Diversity
- Organization and Management
- Campus and External Relations
- Financial Resources
- Technology
- Facilities and Equipment
- Assessment and Evaluation
9Student Learning Development
- Must promote student learning and development
outcomes - Must identify relevant and desirable student
learning development outcomes - Must assess outcomes
- Must provide evidence of their impact on student
learning and development - Must articulate how they contribute to or support
student learning and development
10CAS Student Learning Development Outcomes
- Knowledge acquisition, construction, integration
and application - Cognitive complexity
- Intrapersonal development
- Interpersonal competence
- Humanitarianism civic engagement
- Practical competence
11Student Learning Development Outcomes
- Consistent with the institutional mission,
programs and services must identify relevant and
desirable student learning and development
outcomes from among the six domains. When
creating opportunities for student learning and
development, programs and services must explore
possibilities for collaboration with faculty
members and other colleagues. - Programs and services must assess relevant and
desirable student learning and development
outcomes and provide evidence of their impact on
student learning and development. Programs and
services must articulate how they contribute to
or support student learning and development in
the domains not specifically assessed.
12Essential Elementsin the CAS Approach
- Culture that values involvement of all its
members - Quality indicators
- Use of standards and guidelines
- Willingness/capacity
- To examine itself and assemble results
- To report and use the results
13Uses of CAS Standards
- Credibility, accountability, improvement
- Program service improvement measures of
quality and effectiveness measures of impact on
learning - Design of new programs services
- Restructuring / reorganization
- Change in leadership
- Institutional self-studies
- Preparation for accreditation
- Staff development
- Academic preparation
14CAS Standards and Guidelines
- STANDARD (BOLD TYPE AUXILARY VERB MUST)
- To support the institutions mission and goals,
Career services must provide faculty and staff
and administrative units with information,
guidance, and support on career development and
employment issues and linkages with the broader
community. - GUIDELINE (LIGHT-FACED TYPE VERB SHOULD)
- Career Services should support faculty and staff
and administrative units by - Identifying and disseminating information on
employment trends, top employing organizations,
and co-op and internship sponsors.
15Comprehensive Assessment Plan
- Includes
- Assessment of programs and services
- Assessment of student outcomes, including student
learning and development
16Program Evaluation
- Is the program or service functioning effectively
to achieve its mission? - What evidence is available to support the
determination? - How is evidence used to make program decisions?
17Student Outcomes Assessment
- What is the effect of our work on students?
- How are they different as a result of interacting
with our programs and services? - How do we demonstrate their learning?
- How do we know? Measure?
18The CAS SAG A Self-Assessment Guide
- Translates CAS standards into an effective
workbook / evaluative format - Promotes program self-assessment and development
- Informs on program strengths and weaknesses
- Supports professional staff development
- Leads to enhanced programs and services that
benefit student learning and development
19CAS Self-Assessment Process
- Establish and prepare the Self-Assessment Team
- Initiate the Self Study
- Compile and review Documentary and Evaluative
Evidence - Identify Discrepancies
- Determine appropriate Corrective Action
- Recommend Steps for Program Enhancement
- Prepare an Action Plan
20The Self-Assessment Team
- Appoint and Train the Team
- Size of team
- Composition internal, external
- Coordinator/Leader
- Establish team ground rules
- Discuss meaning of each standard
- Establish teams inter-rater reliability
- Encourage team discussion expect disagreements
commit to consensual resolution
21Team Actions
- Decide whether to include guidelines or other
measures that go beyond the standards - Gather and analyze quantitative and qualitative
data - Individuals rate each and every criterion measure
- Obtain additional documentary evidence if
required to make an informed team decision - Complete the assessment, ratings action plan
worksheets
22Evaluative Evidence
- STUDENT RECRUITMENT MATERIALS
- Brochures other program information
- Participation policies procedures
- PROGRAM DOCUMENTS
- Mission statements program purpose philosophy
statements - Catalogs related materials
- Staff student manuals policies procedures
statements - ADMINISTRATIVE DOCUMENTS
- Organization charts student staff profiles
- Financial resource statements budgets
- Annual reports
- STAFF ACTIVITY REPORTS
- Curriculum vitae resumes professional activity
- Service to other programs, departments, or
community - STUDENT ACTIVITY REPORTS
- Portfolios, developmental transcripts, resumes
- Reports of student service
- RESEARCH EVALUATION DATA
- Needs assessments self-studies
23Quantitative Data
- Summary of objective responses on a questionnaire
or program evaluation - Statistics about use by students and various
population subgroups - Needs assessments, follow-up studies, and
self-study reports - Institutional research reports and fact books
- Comparative or benchmark data at local, regional
or national level
24Qualitative Data
- Focus group information
- Written summaries of responses to open-ended
questions in interviews and on evaluations - Client satisfaction surveys, self-reports, and
written comments, both solicited and unsolicited
25Using the SAGs
- SAGs offer a ready format for evaluation
- In most instances, there are multiple criterion
statements for each standard - Each criterion measure focuses on a particular
aspect of the standard, allowing raters to
express more detailed and specific judgments
26Criterion Measure Rating Scale
- ND 1 2 3 4 NR
- Not Done Not met Minimally Well
Fully Not Rated - Met Met Met
ND 1 2 3 4 NR Not Done Not met
Minimally Well Fully Not
Rated Met Met Met
- Using this scale, consider each criterion
statement and decide - The extent to which each criterion measure has
been met by the program or service
27Assessment Criteria Example
- Part 6 FINANCIAL RESOURCES
- Counseling Services (CS) must have adequate
funding to accomplish its mission and goals.
Funding priorities must be determined within the
context of the stated mission, goals, objectives,
and comprehensive analysis of the needs and
capabilities of students and the availability of
internal or external resources. - CS must demonstrate fiscal responsibility and
cost effectiveness consistent with institutional
protocols. - ND 1 2 3
4 NR - Not Done Not Met Minimally Met Well
Met Fully Met Not Rated - PART 6. FINANCIAL RESOURCES (Criterion
Measures) Rating Scale NOTES - 6.1 The program has adequate funding to
accomplish its mission and goals. ND 1 2 3
4 NR - 6.2 Funding priorities are determined within
the context of program mission, - student needs, and available fiscal
resources. ND 1 2 3 4 NR - 6.3 The program demonstrates fiscal
responsibility and cost effectiveness - consistent with institutional protocols. ND
1 2 3 4 NR - Part 6 Financial Resources Overview Questions
28CAS Work Form AAssessment, Ratings, and
Significant Items
CAS Work Form A Assessment, Ratings, and
Significant Items INSTRUCTIONS This work form
should be completed following individual ratings
of the participants. For each of the 13 Parts,
identify (circle) the criterion measure item
number(s) in the column labeled for which there
is a substantial rating discrepancy (two or more
ratings apart). Items not circled should reflect
consensus among judges that practice in that area
is satisfactory. Items where judgment variance
occurs need to be discussed thoroughly by team
members. Follow this action by determining which
practices (criterion measures) can be designated
as excellent or unsatisfactory and record
them in the Step One column. In Step Two, list
the items requiring follow-up action including
any criterion measure rated as being
unsatisfactory by any reviewer.
29CAS Work Form A continued
30CAS Work Form B Follow Up Actions
- INSTRUCTIONS
- The purpose of this work form is to begin the
planning for action to be taken on practices
judged to merit follow-up (See Step 1, Work Form
A). In Step Three, transfer short descriptions of
the practices requiring follow-up and detail
these items using the table format provided. - Step Three Describe the current practice that
requires change and actions to initiate the
change
31CAS Work Form CSummary Action Plan
- Step Four
- This form concludes the self-assessment process
and calls for action to be taken as a consequence
of study results. Write a brief action plan
statement in the spaces below for each Part in
which action is required.
32Closing the loop
- Purpose scope of study is defined
- Self-study team is trained
- Self-study is conducted
- Evaluative evidence is collected
- Discrepancies are identified
- Appropriate actions are determined
- Special actions for program enhancement are
recommended - Action plan is developed and communicated
33Student Outcomes Assessment
- What is the effect of our work on students?
- How are they different as a result of interacting
with our programs and services? - What have students learned?
- How have they developed?
34Why Focus on Student Learning?
- Accountability
- Alignment with institutions mission
- Consistent with language of the academy
- Quality improvement
- More effective learning
- Better defined and more effective planning and
strategy development
35Defining Learning Outcomes
- Outcomes usually identify growth in some
dimension of knowing, being or doing. - Susan Komives Sarah Schoper, Developing
Learning Outcomes, - Learning Reconsidered 2 (2006)
- Knowledge (cognitive)
- Attitude (affective)
- Skill (psychomotor)
36CAS Student Learning Development Outcomes
- Knowledge acquisition, construction, integration
and application - Cognitive complexity
- Intrapersonal development
- Interpersonal competence
- Humanitarianism civic engagement
- Practical competence
37Plan for Assessing Student Learning Outcomes
- Review/revise mission
- Identify major services, programs, activities
- Specify desired student outcomes
- Identify information needed to show student
learning - Identify existing and new data needed
- Develop process for measuring outcomes
- Identify ways to use findings to change/improve
38Writing Learning Outcomes
- Identify words/phrases that describe what an
ideal program participant or college graduate
would be able to - Know Be Do as a result of engaging with
your program, activity or service. - What would he/she learn from the experience?
39Student Outcomes Matrix
40Writing Learning Outcomes
- Describe the learner characteristics of
users/participants of programs and services. What
knowledge, skills, experiences do they start
with? - Determine acceptable levels of performance
- Consider how you would provide evidence of
achievement
41Ways to measure learning outcomes
- Surveys, questionnaires, self-report (indirect)
- Observations
- Interviews
- Focus groups
- Reflective journals
- Performance evaluations
- Pre-test/post-test
- Portfolios
- Capstone projects
- Review of project/performance by peers and
internal/external reviewers
42Tips, suggestions
- Identify all your program outcomes
- Rank outcomes based on importance
- Activity outcome
- Program outcome
- Institutional or division outcome
- Identify where learning opportunities are
occurring (mapping) - Design and implement assessment cycle one
outcome at a time
43Remember
- Start small
- Start doable
- Start narrow
- Start targeted
- Start!
- One program, one workshop, one desired outcome,
one measured tool
44Additional CAS Resources
- www.cas.edu
- Standards book
- Prologue Context with How to information
- FALDOs book and CD formats as package
- SAGs CD or individually