From Student Engagement Outcomes to Student Success Initiatives

1
From Student Engagement Outcomes to Student
Success Initiatives

• Bob Smallwood
  
  
  Texas State University-San Marcos

Northern Nevada Assessment Conference

February 6,
2004

Reno
2
Presentation Objectives
I. Overview of the NSSE
II. Strategies for Interpreting NSSE Outcomes
III. Importance of Disaggregating Outcomes
IV. Using Faculty Perceptions of Student
Engagement to Set Improvement Priorities
V. Generating Ideas and Initiatives to
Facilitate Student Success

• VI. NSSE as a Tool for Focusing on Best
  Practices

3
Overview of the NSSE
?What is the NSSE? ?What does the NSSE
measure? ?What do NSSE outcomes look like?
4
What is the NSSE?

• College student survey that assesses the extent
  to which students engage in educational practices
  associated with high levels of student learning
  and development.

5
What does the NSSE Measure?
-Ask questions in class -Work with a faculty
member on a research project -Amount of time
spent preparing for class

Student Behaviors
Institutional Actions Requirements

• -Amount of analysis or synthesis activities
  required in class
• Number of written assignments required
• How challenging examinations are

-Extent institution contributed to quantitative
skills,
critically thinking skills, working effectively
with others -overall evaluation of educational
experience -quality of relationships with
students, faculty staff

Reactions to College
Student Background Information

-full-time / part time status -residence
status -academic major
6
What do NSSE outcomes look like?
7
Institutional Outcomes Means Summary Report
(1Never, 2
Sometimes, 3Often, 4Very Often)

Nevada

- Reno
Doc-Ext NSSE 2003
Average Average Average


Asked questions in class or contributed to class
discussion
FY 2.75 2.64
2.84
SR 2.94 2.89
3.12
8
II. Strategies for Interpreting NSSE Outcomes
? Strategy 1 Norm Referenced Approach
?
Interpret outcomes with respect to
Carnegie equivalent norms and national norms
9
Institutional Outcomes Means Summary Report
(1Never, 2
Sometimes, 3Often, 4Very Often)
Nevada

- Reno
Doc-Ext NSSE 2003
Average Average
Average

Asked questions in class or contributed to class
discussion
FY 2.75 2.64
2.84
SR 2.94 2.89
3.12
10
II. Strategies for Interpreting NSSE Outcomes
? Strategy 1 Norm Referenced Approach
?
Interpret outcomes with respect to
Customized Comparison Groups
11
Customized Comparison Groups
12
II. Strategies for Interpreting NSSE Outcomes
? Strategy 1 Norm Referenced Approach
? Strategy 2 Criterion Referenced Approach
?
Interpret outcomes with respect to a designated
criterion for each engagement practice
13
Criterion Referenced Strategy
Seniors
Some Very


Never times Often Often Worked with
classmates
outside of class to
18 44 26 12
prepare assignments
First Year
Some Very


Never times Often Often Discussed
grades
or assignments
21 59 12
8 with an instructor

14
II. Strategies for Interpreting NSSE Outcomes
? Strategy 1 Norm Referenced Approach
? Strategy 2 Criterion Referenced Approach
??
Eliminate Nevers Reduce or eliminate reports
by students of never doing a specific
engagement activity
15
(No Transcript)
16
III. Importance of Disaggregating Outcomes
? Interest in levels of engagement within
academic departments and colleges
17
III. Importance of Disaggregating Outcomes
? Interest in levels of engagement within
academic departments and colleges
? Increase survey sample size by over-sampling
and by local sampling
18
(No Transcript)
19
III. Importance of Disaggregating Outcomes
? Interest in levels of engagement within
academic departments and colleges
? Increase survey sample size by over-sampling
and by local sampling
? Variability of engagement greater within the
institution than between institutions
20
(No Transcript)
21
III. Importance of Disaggregating Outcomes
? Interest in levels of engagement within
academic departments and colleges
? Increase survey sample size by over-sampling
and by local sampling
? Variability of engagement greater within the
institution than between institutions
? Improvement Initiatives more manageable at the
college and department level
22
IV. Using Faculty Perceptions of Student
Engagement to Set Improvement Priorities

• Sought Permission From NSSE
• To compare faculty student responses on the
  NSSE 2002 questions.

2. To determine if faculty differentially value
or place more importance on some engagement
activities than others.
23
NSSE Faculty Questionnaire
1. How important to you are each of the following
educational practices in your Psychology 3321
class, and how often do students in your
Psychology 3321 class engage in these practices?
Not Important
Very Often
Somewhat Important
Often
Important
Sometimes
Very Important
Never
24
Faculty-Student Comparisons
Percent Reporting Often and Very Often
Faculty Teaching Seniors
Seniors
25
Faculty-Student Comparisons
Percent Reporting Often and Very Often
Faculty Teaching Seniors
Seniors
26
Faculty-Student Comparisons
Percent Reporting Often and Very Often
Faculty Teaching Seniors
Seniors
27
Faculty-Student Comparisons
Percent Reporting Important and Very Important
28
Faculty-Student Comparisons
Percent Reporting Important and Very Important
29
Faculty-Student Comparisons
Percent Reporting Important and Very Important
30
Faculty-Student Comparisons
Percent Reporting Important and Very Important
31
Faculty-Student Comparisons
Percent Reporting Important and Very Important
32
Students Have Told Us How Often They Engage in
these Educational Practices
Faculty Have Told Us How Important these
Educational Practices Are To Them
Link student outcomes and faculty outcomes to
decide where to start Focus on those student
engagement activities that faculty find important
and what student report are not occurring very
often.
33
NSSE 2002 Faculty QuestionnaireAbove Average in
Faculty Importance Below Average in Reported
Frequency of Occurrence by Students
Freshmen Item (1.a) Ask questions in class or
contribute to class discussion (1.f) Come to
class having completed readings or
assignments (1.n) Discuss grades or
assignments with you Seniors Item (1.f) Come
to class having completed readings or
assignments (1.o) Talk about career plans with
you (1.p) Discuss ideas from readings or from
class with you outside of this class
34
(No Transcript)
35
V. Generating Ideas and Initiatives to
Facilitate Student Success
Once institutional engagement priorities are
established, how do you decide what to do?
How do we come up with ideas to facilitate a
particular engagement practice?
36
Answer Check out the literature

• www.assessment.swt.edu
• (Click on) NSSE at SWT
• (Click on) Annotated Bibliography

Answer Ask your faculty staff
Student Engagement Idea Database
37

• Exercise
• Examine Items 1a 1 v on the College Student
  Report 2003. Which one of these engagement
  practices do you think has the greatest impact on
  student learning?

2. Identify an action faculty could take to
enhance the selected engagement practice.
What might the faculty member do to maximally
impact this engagement activity?
38
VI. NSSE as a Tool for Focusing on Best
Practices
Ideally Quality and Effectiveness in Higher
Education measured in terms of knowledge and
skills, how much the student learns.
But No agreement on a measure of learning
39
NSSE as a Tool for Focusing on Best Practices
(US News World Report Ranking Approach)

• Six year graduation rate
• One year retention rate
• Academic reputation
• Classes with
• Classes with 50 students
• StudentFaculty ratio
• Full time faculty w/ terminal degree
• Middle 50 of SAT Range
• Top 25 of H.S. Class
• Admission acceptance rate
• Alumni giving rate

40
NSSE as a Tool for Focusing on Best Practices
NSSE Benchmark Outcomes Level of Academic
Challenge Active and Collaborative
Learning Student-Faculty Interaction Enriching
Educational Experiences Supportive Campus
Environment
41
NSSE as a Tool for Focusing on Best Practices
Principles for Good Practice in Undergraduate
Education (Chickering Gamson, 1987)
NSSE Benchmark Outcomes Level of Academic
Challenge Active and Collaborative
Learning Student-Faculty Interaction Enriching
Educational Experiences Supportive Campus
Environment

• Student-faculty contact
• Active learning
• Prompt feedback
• Time on task
• High expectations
• Respect for diverse learning styles
• Cooperation among students

42
From NSSE 2003 Annual Report NSSEs success is
best measured by people, on and off campus,
thinking and talking about quality in terms of
educational effectiveness..what students and
institutions do as contrasted with what rankings
emphasize, which is student test scores and an
institutions resources and reputation.
There are indications that NSSE is moving the
conversation (about educational quality) in the
right direction. This past year, for example,
NSSE was mentioned as a preferred alternative
approach to measuring quality in articles in the
New York Times, USA Today, The Atlantic Monthly,
The Washington Post, Christian Science Monitor,
Kaplan/Newsweek How To Get Into College,
Forbes, and the Chronicle of Higher Education.
NSSE data have been the topic of discussion on
scores of campuses, at faculty and governing
board retreats, and teaching and learning
workshops. Dozens of institutions are using
student engagement results for strategic planning
and accreditation self-studies
43
Follow-up NSSE Presentation 1245pm - 145pm ?
NSSE as an Institutional Measure of
Effectiveness ? FSSE as a Useful Companion
Measure ? Examples of Some Improvement
Initiatives
Follow-up CCSSE Presentation 200pm - 300pm ?
Overview of the CCSSE ? Key Findings from the
2003 Administration ? Using CCSSE Outcomes to
Prompt Improvements