Institutional Research and SAS: The Right Information, for the Right Decision, at the Right Time

1
Institutional Research and SAS The Right
Information, for the Right Decision, at the Right
Time
Dr. Joe DeHart Executive Director of
Institutional Effectiveness and Assistant to the
President Des Moines Area Community College
(DMACC)
2
Institutional Research and SASDMACC (Pronounced
Dee-Mack)

• 18,000 fall enrollment (PT gt FT)
• 600,000 service area population
• 6 campuses (I urban, 2 suburban, 3 rural)

3
Institutional Research and SASEnrollment
Management and Marketing

• Institutional research and data, both historical
  and real-time, have been essential to DMACCs
  ability to increase enrollment and retain
  students. 
• Being able to access data effectively provides
  insight into our core services and their impact
  on students lives
• and allows us to focus enrollment management and
  marketing efforts in ways that add value to our
  students experience at DMACC.
• This directly benefits the student and the
  College.
• Rob Denson, President

4
Institutional Research and SASAGENDA

• Evolution of Institutional Research
• Choosing a tool to facilitate this evolution
• SASs role in this evolution at DMACC
• IR Tool (examples)
• Business Intelligence Tool (examples)
• Predictive Analytics (examples)
• Conclusion/ Next Steps
• Questions?

5
Institutional Research and SASEvolution of
Institutional Research
Facilitation Model (Pre- SAS)
6
Institutional Research and SASEvolution of
Institutional Research
Savvy IR Model (SAS)
7
Institutional Research and SASEvolution of
Institutional Research
Business Intelligence Model (SAS BI)
8
Institutional Research and SASEvolution of
Institutional Research
Emerging Model (SAS BI, Predictive Analytics)
9
Institutional Research and SASWhy SAS to
Achieve our Goals?

• Explosion of BI companies and products.
• Oracle Discoverer, Pentaho, Web FOCUS, Business
  Objects, Crystal Reports, Cognos, MS Access,
  SPSS, Microsoft BI, ProClarity, SAP,
  MicroStrategy, etc., etc.
• How did DMACC come to settle on SAS for our
  solution?
• It met our predefined objectives and requirements
• I had used the software before

10
Institutional Research and SASDesired
Objectives and Requirements

• REQUIREMENTS
• Integrate with Active Directory
• Control content based on user
• Automation
• Web and email based
• No web programming
• Drill-down
• Appropriate for one-person
• Fits in the budget

• OBJECTIVES
• Better, consistent data
• Improved Access
• Improved User Experience
• Faster
• Improved Efficiency

KNOW THESE AHEAD OF TIME!
11
Institutional Research and SASGetting to the IR
Savvy Model

• SAS allows me to start with an issue and...
• Gather only the data needed from Banner
• Aggregate and analyze the data using descriptive
  and/or inferential statistics
• Quickly produce a final report
• (tables, graphs, etc) in Adobe,
• Word, Excel or html format
• However
• SAS learning curve
• Must know your data

12
Institutional Research and SASGetting to the IR
Savvy Model

• PC based (Windows)
• Access to Banner Oracle Database
• Access to all ODBC data
• Two options
• Base SAS (SAS code)
• SAS Enterprise Guide (GUI)
• USED EVERY DAY
• Enrollment Management Example

13
Institutional Research and SASGetting to the IR
Savvy ModelEnrollment Management Example

• Can you tell me how many students are enrolled
  for each Dean by their delivery method (online,
  HS, etc)?
• Typical
• Start to Finish
• Took 5-10 minutes to create
• Took 6 seconds to run

14
Institutional Research and SASGetting to the IR
Savvy ModelEnrollment Management Example

• Finished Output

15
Institutional Research and SASGetting to the IR
Savvy ModelEnrollment Management Example

• Semi-technical skills to be expected from your IR
  person/department
• One sql statement gather data from three banner
  tables
• One procedure to aggregate the data
• Output file creation, look and feel, titles,
  footnotes, etc.

16
Institutional Research and SASGetting to the IR
Savvy ModelEnrollment Management Example

• Once the program is written
• Reusable
• Easily modified to show data for previous years
• Captures multiple steps
• Can be upgraded to BI

17
Institutional Research and SASFrom IR Savvy to
BI

• BI options in use at DMACC
• Running a canned report
• Creating a report from a dataset
• Running an individualized report
• Predictive Analytics (infancy)

18
Institutional Research and SASFrom IR Savvy to
BI

• What if the user wants this report each semester?
• Ask me each term?
• Incorporate into BI
• Put report on server
• Available on demand
• Not person dependent
• Others might want the same info
• Point-and-Click user interface

19
Institutional Research and SASFrom IR Savvy to
BI

• User experience

20
Institutional Research and SASFrom IR Savvy to
BI

• Access to Data User-defined Queries
• Data sets created, maintained, updated daily
• Current term enrolled students, dropped students,
  all students (enrolled and dropped)
• Past census data (term and annual)
• Graduate follow-up data
• Recruitment data
• Single version of the truth!
• User saved reports (personal or shared)
• All faculty and staff (not everyone sees all
  fields)

21
Institutional Research and SASFrom IR Savvy to
BI

• Recruiter Example (Report Wizard)

22
Institutional Research and SASFrom IR Savvy to
BI

• Recruiter Example Output

23
Institutional Research and SASFrom IR Savvy to
BI

• Individualized Reports
• Because you log in, SAS knows who you are and
  retrieves information only for you
• Examples
• Class Lists
• Grade Distribution Comparison
• Retention Comparison
• Faculty In-service Totals

24
Institutional Research and SASBI and Predictive
Analytics

• Up to this point
• Current and historical data
• Predictive analytics lets us use past data to
  make better decisions regarding what is likely to
  happen in the future

25
Institutional Research and SASBI and Predictive
Analytics

• It is a set of rules that use different
  statistical methods to uncover hidden patterns in
  the data.
• Changes how research has traditionally been done.
• Book
• Competing on Analytics,
• Harvard Business School Press

26
Institutional Research and SASBI and Predictive
Analytics

• Community College use of predictive modeling?
• Recruitment- recruit students most likely to be
  successful
• Advising- ability to identify students who need
  academic advising before there is a problem
• Placement- students likely to struggle can be
  placed in various assistance programs
• Efficiency- provide services where they are
  needed most
• Early warning- identify at-risk students early
• Fundraising- predict past donors likely to give
  again

27
Institutional Research and SASBI and Predictive
Analytics

• Predictive modeling at DMACC
• What is the likelihood
• of a first-time student to be successful in their
  first term?
• of a first-time, full-time student to graduate in
  3-years?
• of a first-time student this fall to persist to
  the spring term?

28
Institutional Research and SASBI and Predictive
Analytics
Some of what we know about students before
classes begin
14. Campus (on application)
15. Days Between Accept and Registration
16. Type of Admission (Guest, New, etc)
17. Department (SH, BI, IT, etc)
18. Degree Sought (AA, AS, etc)
19. Program
20. First Generation Status
21. Single Parent Status
22. City
23. High School
24. Amount Offered for PELL
25. Age
26. Developmental Education
1. Previous Career Advantage Credits
2. Total Credits Enrolled in First Term
3. Student Type (new, return, etc)
4. Sex
5. Race/Ethnicity
6. Highest Level of Education
7. First Term Type (fall, spring, summer)
8. Student Intent (transfer, job skills, etc)
9. Days Between Registration and Start
10. Days Between Application and Accept
11. Days Between Accept and Start
12. Residency Status (on application)
13. ESL Status
29
Institutional Research and SASBI and Predictive
Analytics

• First-term Success 3 yrs, 30,000 students
• Graduation 3 cohort years, 7,000 students
• Persistence 4 yrs (fall), 18,000 students

30
Institutional Research and SASBI and Predictive
Analytics
Graduation- 3yrs
First-term Success
Persistence

• Mean 23 graduate
• Top decile 60
• lt 3rd decile Mean

• Mean 62 persist
• Top 3 deciles 80
• Bottom 2 deciles 30

• Mean 56 successful
• Top 2 deciles 80
• Bottom decile 20

31
Institutional Research and SASBI and Predictive
Analytics

• How to best serve 3,500 students

First-term Success N3,500
Top - probably successful with minimal services
Middle - Most likely to impact their success
Bottom- less likely to impact their success or
failure (Title 3)
32
Institutional Research and SASBI and Predictive
Analytics
First-term Success- Evaluation Predicted v.
Actual
Fall 2006 (In Model) Fall 2006 (In Model) Fall 2006 (In Model) Fall 2006 (In Model) Fall 2006 (In Model)
Number Number Percent Percent
Failure Success Failure Success
0 1 . 100.00 .
0.1 109 13 89.34 10.66
0.2 189 66 74.12 25.88
0.3 296 191 60.78 39.22
0.4 477 400 54.39 45.61
0.5 446 528 45.79 54.21
0.6 201 353 36.28 63.72
0.7 77 309 19.95 80.05
0.8 56 367 13.24 86.76
0.9 18 139 11.46 88.54
Fall 2003 (NOT in Model) Fall 2003 (NOT in Model) Fall 2003 (NOT in Model) Fall 2003 (NOT in Model) Fall 2003 (NOT in Model)
Number Number Percent Percent
Failure Success Failure Success
0 58 5 92.06 7.94
0.1 813 9 98.91 1.09
0.2 219 70 75.78 24.22
0.3 164 115 58.78 41.22
0.4 250 202 55.31 44.69
0.5 249 286 46.54 53.46
0.6 178 270 39.73 60.27
0.7 187 287 39.45 60.55
0.8 97 388 20.00 80.00
0.9 19 150 11.24 88.76
1 . 4 . 100.00
33
Institutional Research and SASBI and Predictive
Analytics
Graduation- 3yrs- Evaluation Predicted v. Actual
Fall 2001 Cohort (not in model) Fall 2001 Cohort (not in model) Fall 2001 Cohort (not in model) Fall 2001 Cohort (not in model) Fall 2001 Cohort (not in model)
Number Number Percent Percent
Failure Success Failure Success
0 78 20 79.59 20.41
0.1 540 131 80.48 19.52
0.2 510 125 80.31 19.69
0.3 242 102 70.35 29.65
0.4 114 62 64.77 35.23
0.5 36 36 50.00 50.00
0.6 14 17 45.16 54.84
0.7 6 13 31.58 68.42
0.8 2 7 22.22 77.78
0.9 1 2 33.33 66.67
Fall 2002 (Not in Model) Fall 2002 (Not in Model) Fall 2002 (Not in Model) Fall 2002 (Not in Model) Fall 2002 (Not in Model)
Number Number Percent Percent
Failure Success Failure Success
0 64 12 84.21 15.79
0.1 440 77 85.11 14.89
0.2 482 102 82.53 17.47
0.3 295 124 70.41 29.59
0.4 150 75 66.67 33.33
0.5 56 55 50.45 49.55
0.6 28 28 50.00 50.00
0.7 12 18 40.00 60.00
0.8 2 10 16.67 83.33
0.9 1 1 50.00 50.00
34
Institutional Research and SASBI and Predictive
Analytics
F2S Persistence- Evaluation Predicted v. Actual
Fall 2004 (Not in Model) Fall 2004 (Not in Model) Fall 2004 (Not in Model) Fall 2004 (Not in Model) Fall 2004 (Not in Model)
Number Number Percent Percent
Failure Success Failure Success
0 14 2 87.50 12.50
0.1 138 39 77.97 22.03
0.2 232 77 75.08 24.92
0.3 270 156 63.38 36.62
0.4 225 196 53.44 46.56
0.5 201 293 40.69 59.31
0.6 169 373 31.18 68.82
0.7 209 568 26.90 73.10
0.8 180 669 21.20 78.80
0.9 37 310 10.66 89.34
1 . 4 . 100.00
Fall 2007 (In Model) Fall 2007 (In Model) Fall 2007 (In Model) Fall 2007 (In Model) Fall 2007 (In Model)
Number Number Percent Percent
Failure Success Failure Success
0 5 . 100.00 .
0.1 74 22 77.08 22.92
0.2 166 68 70.94 29.06
0.3 246 135 64.57 35.43
0.4 250 184 57.60 42.40
0.5 217 260 45.49 54.51
0.6 205 395 34.17 65.83
0.7 246 633 27.99 72.01
0.8 208 814 20.35 79.65
0.9 67 408 14.11 85.89
1 4 14 22.22 77.78
35
Institutional Research and SASBI and Predictive
Analytics

• Once the Model is Shown to be Accurate
• Scoring Students

36
Institutional Research and SASBI and Predictive
Analytics

• Outstanding Issues with Predictive Modeling
• Profiling
• Diverse populations
• Proactive, not reactive (no denial of services)
• Interfacing with Enrollment Management
• Recruitment
• Advising/Counseling
• 3. Model Maintenance

37
Institutional Research and SASBI and Predictive
Analytics

• Next Steps
• Key Performance Indicators
• IR priorities or help
• Succession planning for IR
• Scaling
• Cluster Analysis

38
Institutional Research and SAS

• Thank you!
• Questions and Comments?
• Contact info
• Joe DeHart
• jcdehart_at_dmacc.edu
• [phone]