LESSONS LEARNED from a MULTI-CAMPUS DIVERSITY INITIATIVE

1
LESSONS LEARNED from a MULTI-CAMPUS DIVERSITY
INITIATIVE

• THE USE OF EVALUATION ORGANIZATIONAL LEARNING
  FOR EFFECTIVE DIVERSITY INITIATIVES

2
Organized Anarchy
3
The American college or university is a
prototypic organized anarchy. It does not know
what it is doing. Its goals are either vague or
in dispute. Its technology is familiar but not
understood. Its major participants wander in and
out of the organization. These factors do not
make a university a bad organization or a
disorganized one but they do make it a problem
to describe, understand, and lead and as a
result evaluate. -- Michael D. Cohen James
G. March in Leadership and Ambiguity (1974)
4
Assessment

• Systematic collection, review, and use of
  information about educational programs undertaken
  for the purpose of improving student and
  campus learning and development.
• Marchese, T. in Assessment Essentials,
• Palomba, C.A. Banta, T.W. (1999)

5
Empowerment Evaluation

• The use of evaluation concepts, techniques, and
  findings to foster improvement and
  self-determinationit is designed to help people
  help themselves and improve their programs using
  a form of self-evaluation and reflection.
• David Fetterman in Empowerment Evaluation (1996)

6
Purposes of Assessment

• Test Assumptions
• Our bridge program is highly successful
• We are / are not diverse because
• Diversity is central to our mission
• Initiate Guide Dialogue
• Identify Gaps, Disconnects Potential Solutions
• Develop a sense of ownership and empowerment
• Develop and sustain capacity for organizational
  learning

7
Steps in effective assessment of diversity
initiatives

• Define the Purpose
• Primary Audience
• Evaluation Team
• Identify Context
• Target Topic

• Formulate Questions
• Obtain Data
• Assess Data
• Analyze Data
• Report Findings

Source Assessing Campus Diversity
Initiatives A Guide for Campus Practitioners,
Garcia, M. et al. (2001)
8
Utilize Existing Data

• Institutional Research Office
• Vice-President for Planning / Research
• Varied Campus Offices
• Past Strategic Plans

• Self-Studies for Accreditation Agencies
• Past Diversity Initiatives
• Integrated Postsecondary Education Data System
  (IPEDS) www.nces.ed.gov/ipeds/

9
Benefits of Varied Types of Data

• Quantitative Data
• Helps us explore the what and the who
• (e.g. campus-based surveys)
• Qualitative Data
• Helps us understand the why and the how
• (e.g. focus groups interviews observations
  document analyses)
• Process Data
• Helps us understand and explore where pockets of
  resistance, inertia, or hope are located
• And, whether IT is resistanceinertiaor hope

10
Assess and Analyze the Data

• Response Rates
• Recognize and Qualify Data Limitations
• Background Variables
• (e.g. race/ethnicity, gender, socioeconomic
  status, major etc.)
• Disaggregation
• Guided by mission / purpose of program or
  initiative

11
Report the Findings

• Review target audience
• Format for dissemination
• Written reports oral presentations
  internal/external dissemination
• Guide the discourse
• Involve critics
• Balanced Analysis that emphasizes the good, the
  bad, and the ugly
• Connect findings with overall institutional
  mission

12
Research must function as a tool for
appropriating the codes and cultural symbols of
institutional power in an effort to transform
institutional environments in the interest of
cultural democracyIn this way, researchers who
carry out their work in the interest of cultural
democracy can function as social advocates,
facilitating a production of knowledge that is
committed to the creation of institutional
conditions where people find their voices and
their rightful places as full and equal
participants. -- Antonia Darder Institutional
Research as a Tool for Cultural Democracy
13
References

• Assessing Campus Diversity Initiatives A Guide
  for Campus Practitioners by Garcia, M. Hudgins,
  C.A. Musil, C.M. Nettles, M.T. Sedlacek, W.E.
  Smith, D.G.
• Leadership and Ambiguity by Cohen, M.D. March,
  J.G.
• Assessment Essentials Planning, Implementing,
  and Improving Assessment in Higher Education by
  Palomba, C.A. Banta, T.W.
• Empowerment Evaluation Knowledge and Tools for
  Self-Assessment Accountability edited by
  Fetterman, D.M. Kaftarian, S.J.
• Wandersman, A.
• Institutional Research as a Tool for
  Institutional Democracy by Darder A. in Studying
  Diversity in Higher Education edited by Smith,
  D.G. Wolf, L.E.
• Levitan, T.

14
iALTO!BREAK !!
15
Nuts and Bolts of Evaluation How to Use the
Process to Advance Institutional Learning about
Diversity
http//www.aacu.org/irvinediveval/index.cfm
16
GOALS AND PURPOSE
IRVINE FOUNDATION GOALS FOR THE CAMPUS DIVERSITY
INITIATIVE

• Increase the institutionalization of diversity in
  the context of educating all students for
  participation and leadership in a diverse
  society.
• Increase the educational success of disadvantaged
  and ethnic minority populations.

http//www.aacu.org/irvinediveval/index.cfm
17
RESULTS
http//www.aacu.org/irvinediveval/index.cfm
18
GOALS AND PURPOSE
PURPOSE OF AN EVALUATION EFFORT ORGANIZATIONAL
LEARNING

• Provide ongoing information about the
  implementation so that mid-course corrections can
  be instituted.
• Build the capacity of campuses to assess and
  learn from their own progress.
• Identity and provide opportunities for campuses
  to share problems and solutions.

http//www.aacu.org/irvinediveval/index.cfm
19
ORGANIZATIONAL LEARNING

• Using institutional data to inform progress
• Interrupting the usual
• Engage myths that become self-fulfilling
  prophecies
• Maintaining the link between campus diversity
  efforts and institutional goals for effectiveness
  and excellence
• Taking time for reflection and self-learning
• Keeping all campus constituencies informed
• Building synergy among many efforts on campus

http//www.aacu.org/irvinediveval/index.cfm
20
PRINCIPLES

• Approaches evaluation from an organizational
  learning point of view
• Manageable for campus and capable of being
  maintained
• Monitors key goals and elements of proposal
• Focuses on institutional issues/change, not
  simply project-specific issues
• Reveals success and problems along the way in
  both results and processes
• Guides interim reports to senior leadership,
  campus, board
  ? ?

http//www.aacu.org/irvinediveval/index.cfm
21
PRINCIPLES

• Takes into account
• Institutional differences and stages with respect
  to diversity
• That institutions vary in mission, needs, goals
  and culture
• That strategies, goals and emphasis differ
• The possibility of taking some risks and learning
  from them
• Differences within institutions (Disaggregation
  of information)
• Encourages institutional sharing
• Uses outside resources as appropriate

22
INSTITUTIONAL AUDIT
Climate and Intergroup Relations
Education and Scholarship
CONTEXT
Global
Local
Institutional Viability and Vitality
Access and Success
http//www.aacu.org/irvinediveval/index.cfm
23
ACCESS AND SUCCESS
Success of students -- graduation, persistence,
honors, performance
Undergraduate/ graduate population by fields and
levels
ACCESS and SUCCESS
Transfer among fields (esp. SMET)
Pursuit of advanced degrees
Disaggregated
http//www.aacu.org/irvinediveval/index.cfm
24
(No Transcript)
25
Four Six-year Graduation Rates by
Race/Ethnicity ()
26
(No Transcript)
27
(No Transcript)
28
CLIMATE AND INTERGROUP RELATIONS
Type and quality of interaction among groups
CLIMATE AND INTERGROUP RELATIONS
Perceptions of institution (climate, commitment,
engagement)
Quality of experience/ engagement on campus
Disaggregated
http//www.aacu.org/irvinediveval/index.cfm
29
Satisfied with Climate for Minorities by
Attendance in Racial Cultural Awareness Session

30
Satisfied with Climate for Minorities by
Attendance in Racial Cultural Awareness Session
and Race/Ethnicity
31
EDUCATION AND SCHOLARSHIP

• Presence of diversity-related courses,
  requirements
• Degree to which courses include diversity issues
  and the placement of such courses

• Course taking patterns of students

Experience
Availability
EDUCATION AND SCHOLARSHIP
Faculty Capacity
Learning

• Level of faculty expertise on diversity-related
  matters
• Level diversity of faculty participating in
  diversity efforts

• Quantity and substance of student learning about
  diversity

http//www.aacu.org/irvinediveval/index.cfm
32
INSTITUTIONAL VIABILITY AND VITALITY
Institutional history on diversity issues and
incidents
Institutional Strategies and dedicated resources
Diversity of faculty and staff by level
Centrality of diversity in the planning process,
mission statements
Public perceptions of the institution
INSTITUTIONAL VIABILITY AND VITALITY
Constituency perceptions of institutional
commitment to diversity
Perceptions of access, equity and inclusion among
all constituents
http//www.aacu.org/irvinediveval/index.cfm
33
RESULTS
http//www.aacu.org/irvinediveval/index.cfm
34
RESULTS
Faculty Turnover Quotient

• TQ 0 No Turnover
• TQ 100 100 of URM new hires replaced URM
  faculty who left the institution

http//www.aacu.org/irvinediveval/index.cfm
35
RESULTS
http//www.aacu.org/irvinediveval/index.cfm
36

• FACULTY
• Overall --DISAGGREGATED--over time
• New hires
• Retention/turnover--TQ
• Location --departments/fields

http//www.aacu.org/irvinediveval/index.cfm
37
EVALUATION PROCESS

• What has been the impact on for example,
  curriculum, student success, etc.?

Institutional Goals
REPORTING

• Interpretations/Perceptions
• Lessons Learned

• What has been done?
• What has happened?
• What is working?
• What is not working?

Process
Strategies (Objectives)
http//www.aacu.org/irvinediveval/index.cfm
38
iALTO!BREAK !!
39
Principles of Evaluation Institutional Learning

• Framework and Monitoring Progress
• Centrality to Mission
• Leadership Communication
• Organizational Learning
• Inclusive and Differentiated Approach
• Alignment