Bond Presentation

1
Assessing Institutional Effectiveness The
Mission Engagement Index (MEI) as a Measure of
Progress on Mission Goals
Ellen Boylan, Ph.D. ? Marywood University NEAIR
35th Annual Conference ? Providence, Rhode Island
? Nov. 1 4 , 2008
2
Scope of Research

• Purpose
• Develop a measure (inventory) of student
  perception of institutional mission.
• Test instrument reliability.
• Uncover constructs (factors) and observe them
  longitudinally.
• Exploratory multiple regression Develop a
  measure (index) that allows institutions to
  compare their predicted with actual outcomes on
  mission constructs.

• Research Questions
• 1. Is the Mission Perception Inventory
• (MPI) a valid, reliable measure of
• student perception of institutional
• mission?
• 2. What are the factors in the
• Mission Perception Inventory?
• 3. Do factors recur with equal reliability in
  repeat administrations of the MPI?
• 4. Does the Mission Engagement Index (MEI)
  describe causal relationships among variables
  that affect mission perception?

3
Study Design

• Select the National Survey of Student
• Engagement (NSSE) as the vehicle
• for delivery of the research
• questions.
• Assemble a NSSE consortium to
• jointly engage in research to explore
• student perception of mission.
• Develop consortium questions that
• are then given as an attachment
• to NSSE analyze results via reliability,
  factor analysis, and correlation.
• Repeat annually. Once a sufficient
• volume of institutional data is
• gathered, conduct regression analysis to
  develop the index (MEI).

4
Developing the question items

• Conduct a qualitative analysis of mission
  statements.
• Distill common constructs.
• Draft question items based on those constructs.
• Vet with participating institutions, experts, and
  prospective subjects.

The University roots itself in the principle of
justice and a belief that education empowers
peopleoffers students a welcoming and supportive
community that encourages men and women of all
backgrounds to shape their lives as leaders in
service to others. Proud of its liberal arts
tradition and host of professional disciplines,
challenging students to broaden their
understanding of global issues and to make
decisions based on spiritual, ethical, and
religious values
Concepts common to consortium colleges mission
statements, like this one, were drawn out,
reduced, and formulated into 20 question items.
5
Concepts became question items...
6
A peek at the questionnaire
See consortia questions by year
http//nsse.iub.edu
7
Administration
The NSSE survey with attached Mission
Perception Inventory (MPI) questions has been
administered 145 times to more than 30,000
first-year and senior students at 83 colleges
across the United States every year since
2004.
8
Is the Mission Perception Inventory (MPI)
valid and reliable?

• Reliability analysis of the mission questions
• (20 items) is performed.
• Low-loading
• items kicked
• out.
• A Mission Perception Inventory (MPI) is
• produced (17 or 18 items).

9
What are the factors in the Mission Perception
Inventory?

• Factor analysis of MPI is conducted each year.
• By year, three or four subscales are produced
• Sense of mission.
• Respect for diversity.
• Individual values.
• Religious practice.

10
Derivation of Mission Perception Inventory (MPI)
and subscales
11
Sense of Mission Scale (8 items ? .867)

• The mission of this institution is widely
  understood by students.
• Social and personal development is an important
  part of the mission.
• Ethical and spiritual development of students is
  important.
• This institution offers opportunities for
  volunteering and community service.
• This institution offers opportunities for
  developing leadership skills.
• There are opportunities for students to
  strengthen their religious commitment.
• This institutions religious heritage is evident.
• Professors here discuss the ethical implications
  of what is being studied.

12
Respect for Diversity Scale (5 items ? .854)

• The faculty, staff, and students here
• respect different religions
• respect different races and
• cultures
• Students free to express individual
  spirituality.
• Different sexual orientations are accepted.
• The environment encourages appreciation of
• diversity.

13
Count of institutions and respondentsby year
respondents
14
Do the factors recur in repeated administrations
of the revised Mission Perception Inventory (MPI)
15
Are the factors equally reliable over time?
16
Teagle Foundation Research Continues
2008

• Develop the Mission Engagement Index.
• Obtain NSSE 2008 consortia data
• Test using reliability analysis to
• produce factors
• Distribute MPI reports
• Compare consortia results
• Conduct regression analysis to create the new
  index
• Produce Mission Engagement Index (MEI) Reports
  for consortia institutions.

17
Does the Mission Engagement Index describe causal
relationships among variables that affect mission
perception?

• Dependent variable (Institution score)
• Mission Perception Inventory
• Sense of Mission scale
• Respect for Diversity scale
• Independent variables (need 15 cases per)
• Selectivity 15 institutions
• Enrollment 30 institutions
• Urbanicity 45 institutions
• Resident 60 institutions
• Female 75 institutions
• Another? 90 institutions

a recommended ratio of subjects to IVs of at
least 15 to 1 will provide a reliable regression
equation (Stevens, 1992).
18
Selecting IVs for regression analysis
19
Participating institutions by region
2008 2 consortia, 54 institutions
20
Urbanicity of participating institutions 2008
21
Participants by gender and class rank by year
22
Considering elegance

• Consistency of factors is affirmed.
• Institution sizes reasonably varied.
• Data sufficient to conduct analysis.
• Variables, initially selected
• on an a priori basis, tested.
• Variability investigated.
• Useful.

ßeta juice
23
Mock Mission Engagement Index Report

• The MEI will show an institutions actual versus
  predicted scores on mission constructs.
• Progress on mission effectiveness can be assessed
  by comparing MEI outcomes to institutional goals.

24
Assessing Institutional Effectiveness The
Mission Engagement Index (MEI) as a Measure of
Progress on Mission Goals Ellen Boylan,
Ph.D. Marywood University Scranton PA
eboylan_at_marywood.edu available http//www.maryw
ood.edu/instresearch/activity.stm
Supported by a grant from
25
(No Transcript)