Leveraging Institutional Data

1
Leveraging Comparative Analysis for Institutional
Decision Making
NJAIR Annual Conference April 17th, 2009 The
College of New Jersey Robert Miller, Centenary
College Chad May, The Richard Stockton College of
NJ
2
Leveraging Comparative Analysis for
Institutional Decision Making

• Benchmarking What is it and who do we compare to?

3
What Forces are Driving the Need for Using Data
for Institutional Decision Making

• Internal
• Finite Resources
• Competition for Students
• External
• Increased Accountability
• Increased Call for Transparency
• Students as Consumers

4
What is Benchmarking?

• Benchmarking is an ongoing, systematic process
  for measuring and comparing the work processes of
  one organization to those of another, by bringing
  an external focus to internal activities,
  functions, or operations (Kempner 1993).
• Practitioners at colleges and universities have
  found that benchmarking helps overcome resistance
  to change, provides a structure for external
  evaluation, and creates new networks of
  communication between schools where valuable
  information and experiences can be shared (AACSB
  1994).
• Benchmarking is a positive process, and provides
  objective measurements for baselining (setting
  the initial values), goal-setting and improvement
  tracking, which can lead to dramatic innovations
  (Shafer Coate 1992).

5
Why Benchmark?

• Identification of Best Practices
• Academic
• Operational
• Provide context for institutional data
• Goal setting and measurement
• Institutional Planning

6
How Do We Select the Institutions?
Adapted from (Teeter Brinkman 2003 in The
Primer for Institutional Research, AIR)
7
Types of Peers

• Definitional
• Have similar identifiers expressing the essential
  nature of the institution
• Informational
• Hold practical knowledge of a desired process,
  outcome, accomplishment
• Analytical
• Provide realistic and practical benchmarks for
  internal and external review
• Nonsensical
• Have no meaning or convey no intelligible
  ideasabsurd or contrary to good sense
• Websters Seventh New Collegiate Dictionary.
  G.C. Merriam Company, Springfield,
  Massachusetts. 1967.
• Adapted from a presentation given at the NEAIR
  2002 Conference

8
Reasons for PEER ANALYSIS ???

• Compare
• Complain
• Assess
• Boast
• Improve
• Fund
• Evaluate

9
Some Common Peer Characteristics

• Affiliation (Public vs. Private)
• Carnegie Classification
• Financials (endowment, tuition, assets,
  liabilities, expenses, revenue)
• Enrollment and Staffing Levels
• Selectivity (SAT, Acceptance rates)
• Academic Programs (majors and degrees)
• IPEDS PAS System can generate a comparison group
  automatically using the information above

10
Strategies of Developing Peer/Aspirant List

• Data Statistics Judgment (Hybrid approach)
• Data Statistics (Cluster Analysis)
• Data Judgment (Threshold Approach)
• Judgment (Panel Review)

(Adapted from Teeter Brinkman 2003 in The
Primer for Institutional Research, AIR)
11
Sources of Comparison DataTo Help Identify Peers

• Carnegie Foundation
• National Student Clearinghouse- StudentTracker
• Integrated Postsecondary Education Data System
  (IPEDS) Peer Analysis System, Dataset Cutting
  tool, Executive peer tool, etc.
• http//nces.ed.gov/IPEDS/

12
Leveraging Comparative Analysis for Institutional
Decision Making

• What Type of Comparison Data is Available?

13
Sources of Comparison DataRecruitment Retention

• Noel-Levitz National Enrollment Management Survey
• Consortium for Student Retention Data Sharing
• Data on retention rates graduation rates
• IPEDS Peer Analysis System
• The College Board Admitted Student Questionnaire
  (ASQ and ASQ plus)
• ACT, Inc

14
Sources of Comparison DataStudent Engagement

• National Survey of Student Engagement (NSSE)
• SPSS Syntax Files
• UCLAs Higher Education Research Institute
  Surveys
• Cooperative Institutional Research Program (CIRP)
• Your First College Year
• College Senior Survey
• Education Benchmarking Inc. (Resident Student
  Assessment, First Year Initiative Survey, etc.)

15
Sources of Comparison DataStudent Learning

• Collegiate Learning Assessment (CLA)
• ACT Collegiate Assessment of Academic Proficiency
  (CAAP)
• ETS Measurement of Academic Proficiency and
  Progress

16
Sources of Comparison Data Financial Operations

• NACUBO Endowment Study
• NACUBO Tuition Discounting Study
• Voluntary Support of Education
• Fundraising results
• IPEDS Finance Survey (Peer Analysis System)
• Guidestar (990 data for non-profits)

17
Sources of Comparison DataSatisfaction

• Student Satisfaction Surveys
• Noel-Levitz Student Satisfaction Inventory
• ACT Survey of Student Opinions
• In-house surveys
• Employee Satisfaction Surveys
• HERI Faculty Survey
• Harvard University (Collaborative On Academic
  Careers in Higher Education survey)
• Alumni Surveys
• ACT Alumni Survey and Alumni Outcomes Survey

18
Analytical Tools (Software/Services)

• Proprietary Software
• MS Excel
• SPSS/SAS/STATA and other Stat packages
• Rapid Insight Analytics / Data Integration
• Tableau- Visual Analysis Software
• Proprietary Services (Internet based
  applications)
• AGB Benchmarking Service
• Peer Analysis System (PAS)
• Dataset Cutting Tool
• Executive Peer Analysis Tool (create your own
  data feedback report)
• CUPA- Data on Demand Services
• Voluntary Support of Education (CAE)- Data tool
• AAUP Faculty Compensation data published in
  Academe
• JMA Associates

19
Big PictureInitiatives/Projects Available

• Council for Independent Colleges
• CIC KIT
• This tool provides information relating to
  enrollment, staffing, admissions, and financial
  aid.
• Key feature allows you to conduct comparative
  analysis using schools with similar financial
  resources.
• Sample of CIC KIT Tool
• http//www.cic.edu/projects_services/infoservices/
  kit.asp
• CIC FIT Tool
• While the KIT tool provides traditional
  indicators such as acceptance rate, yield rate,
  and faculty counts, the FIT tool provides
  detailed financial comparisons
• Ratio analysis for overall institutional health
• Sample of CIC FIT Tool
• http//www.cic.edu/projects_services/infoservices/
  fit/index.asp

20
Leveraging Comparative Analysis for
Institutional Decision Making

• Pseudo Case Study

21
Comparative Data for Internal Analysis Case
Study Example

• Using comparative data to answer institutional
  specific questions
• Common Question for IR professionals
• Who are students choosing over us and who are
  students choosing us over?(i.e. the win/loss
  question)

22
Using Admissions and FASFA Data

• Admissions interview data
• Extraction of enrolled and not enrolled students
• Analysis of fields to identify what other
  institutions students sent their FAFSA data to-
  they can list up to six
• Send batch files to the National Student
  Clearinghouse using the StudentTracker service
• Return file from NSC shows enrollment history of
  your non-enrolling admitted students
• Match NSC return file data to other institutional
  data

23
Internal Data Combined with Student Tracker
24
Example Output- Win/Loss Ratio
25
Hypothesis

• Are institutional aid policies in line with other
  institutions?
• Is there a significant difference in EFC of
  enrolling and non enrolling business students?
• Internal analysis
• Is there a significant difference in the
  institutional grant aid awarded to enrolling and
  non enrolling business students?
• Internal analysis
• How does grant aid compare between our
  institution and other institutions?
• IPEDS PAS

26
Average Aid by Institution
27
Leveraging Comparative Analysis for
Institutional Decision Making

• Integrating comparative Analysis with planning

28
Reporting Comparative Data

• Standard comparative reports
• Externally processed
• Faculty Compensation Report (Academe)
• IPEDS Feedback Report
• University of Delaware Study of Instructional
  Costs and Productivity
• NSSE, HERI, and other survey instruments
• Internally Processed
• Dashboards and/or report of Key Indicators
  report(s)
• Competitors report and Tuition/Fee Comparison
  report
• Other IR reports
• Ad-hoc comparative reports
• Retention- where are our students going?
• Graduation Rate Study
• Internal analysis of survey data (comparison of
  student satisfaction)

29
Yellow Bars- Represent Aspiration
Institutions Dark Blue Bars- Represent Peer-Like
Institutions Aqua Bars- Represent Peer- Below
Institutions Orange line across represents the
target institution
30
Example Institutional Dashboard Summary
Dashboard Fall 2008
Total Gifts
Gifts to Capital Endwmnt
FR Applicants
Endowment/Reserves
Full-Time UGs



15,000
15,000
10,000
20,000
UG Alumni Participation
Number 1,000 donors
Part-Time UGs
FR Acceptances
Return on Endowment /Reserves Portfolio
65
2,000
60


Graduate Students
Faculty
Gross Cost to raise 1 FY 2006
Yield ( Enrolled)
Spending Rate
UG Student/Faculty Ratio
1,000 (fall)
40
37.0

UGs in-State
H.S. Avg. Rank
Student Aid
Unrestricted Annual Fund Gifts (change)
Full-Time Faculty
97.6
Discount Rate
74ile
6-year Graduation Rate

Avg. SAT- Regular
UG Class Size gt30
Inst. Financial Aid as of Operating Budget
68
60
Positive Variance
1017
1250
Diversity Enrollment

UG Class Size lt10
Another Indicator
29
21
Student Revenue Reliance
of FT Students w/ Financial Need
First-year Retention
Taught by FT Faculty

60
81.6
74.7
Debt coverage ratio
SR Stdnt Satisfaction
Importance of Change Green better Red
worse Yellow neutral
KEY Change Higher Lower None

FT Faculty W/ Term. Deg.
85
85
Plant Reinvestment Rate (excludes current
construction projects)
Resident Stds. (FT)
FT Faculty w/ Tenure
1.5-2

Benchmark
65
55
50
Current
51.4
31
Conclusion/Discussion

• Comparative Analysis provides context for
  institutional data with respect to decision
  making/planning/and assessment.
• There is a significant amount of data already
  available. Much of which is almost ready-made
  for dissemination.
• If you do not do the comparative analysis someone
  else will. (students, government, parents, etc.)