Measuring and Managing Student Engagement: - PowerPoint PPT Presentation

Loading...

PPT – Measuring and Managing Student Engagement: PowerPoint presentation | free to download - id: 4ed7cb-N2ZmM



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Measuring and Managing Student Engagement:

Description:

Measuring and Managing Student Engagement: Why it Matters in the First Year of College Jillian Kinzie Indiana University Center for Postsecondary Research – PowerPoint PPT presentation

Number of Views:104
Avg rating:3.0/5.0
Slides: 73
Provided by: ETS68
Learn more at: http://www.uwsa.edu
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Measuring and Managing Student Engagement:


1
  • Measuring and Managing Student Engagement
  • Why it Matters in the First Year of College
  • Jillian Kinzie
  • Indiana University Center for Postsecondary
    Research

Wisconsin OIPD Spring Conference 2007 Partnering
for Student Success in the First Year Learning
About Learning
2
Current Concerns about College Student Learning
  • Colleges and universities, for all the benefits
    they bring, accomplish far less for their
    students than they should. -- Derek Bok
  • At a time when the quality of postsecondary
    education seems to be slipping, participating in
    engaged learning activities promises to prepare
    students for a lifetime of continuous learning so
    that they and the country stay competitive in the
    global marketplace -- George Kuh
  • The first year presents a unique opportunity to
    engage students in the habits of learning. --
    John Gardner

3
Students and the College Experience
  • College-going stakes higher today than at any
    point in history
  • 45 students in 2yr-colleges depart during their
    first year, 1 of 4 leave from 4-yr schools
  • 51 of high school grads have reading skills
    necessary for college 25 of students in 4-yr
    colleges need 1 yr of remedial coursework
  • Enrollment persistence rates of historically
    underserved students lagging

4
What We Know About the Undergraduate Experience
from NSSE
  1. Full-time students spend an average of 13 hrs per
    week studying (Hrs. recommended by faculty
    20-25)
  2. 45 of all college seniors took at least one
    course at another institution prior to enrolling
    at their current school swirling
  3. Three of ten first-year students reported working
    just hard enough to get by.
  4. Between 40 and 50 of first-year students never
    used career planning, financial advising, or
    academic tutoring services.
  5. 45 first-years and 30 seniors never discussed
    ideas with faculty outside class
  6. Faculty spend 42 of class time lecturing (FSSE)

5
Current Context for Concern
  • Engaged pedagogy is really the only type of
    teaching that truly generates excitement in the
    classroom that enables students and professors to
    feel the joy of learning -- bell hooks
  • Conclusion The first year experience is not
    being used as effectively as it could to engage
    students at high levels to develop habits of
    learning that insure student success

6
Principles of Good Practice for the First College
Year Recommendations
  • Institutional commitment by leaders, faculty,
    staff, and governing boards
  • Focus on student learning both inside outside
    the classroom
  • Encourage student affairs-academic affairs
    partnership
  • Offer challenge and support
  • Communicate high expectations
  • Foster an inclusive and supportive campus climate
  • Conduct systematic assessment
  • Create an atmosphere of dignity and respect for
    first-year students
  • Teach students strategies and skills to succeed
  • Get faculty involved
  • Encourage students to assume responsibility for
    their success
  • Challenging and Supporting the First-Year
    Student A Handbook for Improving the First Year
    of College By M. Lee Upcraft, John N. Gardner,
    Betsy O. Barefoot, and Associates, Jossey-Bass,
    2005 Pg. 515-517

7
To Ponder
  • What data do you use to inform your understanding
    of the first year experience? In teaching first
    year students?
  • In what curricular and co-curricular areas,
    specific courses, or transition places do first
    year students have the most difficulty? What
    measures do you use to monitor these issues?
  • What are your current strengths in the first year
    experience?

8
Student Success in College
  • Student success - defined in a broad,
    all-encompassing manner includes academic
    achievement, engagement in educationally
    purposeful activities, satisfaction, acquisition
    of desired knowledge, skills and competencies,
    persistence, attainment of educational objectives
    including graduation, and post-college
    performance.

http//nces.ed.gov/npec/pdf/Kuh_Team_Report.pdf
9
Study Habits Peer Involvement Interaction
with Faculty Time on Task Motivation Other
First Year Experience Academic Support
Campus Environment Time on Task Peer
Support Teaching Learning Approaches
Other
Pre-college experiences
10
What Really Matters in College Student
Engagement
  • The greatest impact appears to stem from
    students total level of campus engagement,
    particularly when academic, interpersonal, and
    extracurricular involvements are mutually
    reinforcing

Pascarella Terenzini, How College Affects
Students, 2005, p. 647
11
What Really Matters in College Student
Engagement
  • Because individual effort and involvement are
    the critical determinants of college impact,
    institutions should focus on the ways they can
    shape their academic, interpersonal, and
    extracurricular offerings to encourage student
    engagement.

Pascarella Terenzini, How College Affects
Students, 2005, p. 602
12
Student Success Lessons From the Research
  • What matters most is what students do, not who
    they are
  • A key factor is the quality of effort students
    expend
  • Educationally effective institutions channel
    student energy toward the right activities

13
Two Components of Student Engagement
  • 1. What students do -- time and energy devoted to
    educationally purposeful activities
  • 2. What institutions do -- using effective
    educational practices to induce students to do
    the right things

14
Foundations of Student Engagement
  • Time on task (Tyler, 1930s)
  • Quality of effort (Pace, 1960-70s)
  • Student involvement (Astin, 1984)
  • Social, academic integration (Tinto,1987, 1993)
  • Good practices in undergraduate education
    (Chickering Gamson, 1987)
  • College impact (Pascarella, 1985)
  • Student engagement (Kuh, 1991, 2005)

15
Good Practices in Undergraduate Education
(Chickering Gamson, 1987 Pascarella
Terenzini, 2005)
  • Student-faculty contact
  • Active learning
  • Prompt feedback
  • Time on task
  • High expectations
  • Respect for diverse learning styles
  • Cooperation among students

16
Lessons from Research on College Impact
  • Effective Educational Practices for Fostering
    Engagement
  • Academic Challenge
  • Active and Collaborative Learning
  • Student-Faculty Interaction
  • Enriching Educational Experiences
  • Supportive Campus Environments

17
Promise ofStudent Engagement
If faculty and administrators use principles of
good practice to arrange the curriculum and other
aspects of the college experience, students
would write more papers, read more books, meet
with faculty and peers, and use information
technology appropriately, all of which would
result in greater gains in such areas as critical
thinking, problem solving, effective
communication, and responsible citizenship.
Kuh, Kinzie, Schuh, Whitt Associates, Student
Success in College, 2005
18
National Survey of Student Engagement(pronounced
nessie)Community College Survey of Student
Engagement(pronounced cessie)
  • Undergraduate student surveys that assess the
    extent to which students engage in educational
    practices associated with high levels of learning
    and development

19
NSSE Project Scope
  • One million students from 1,100 different
    schools
  • Random sample of First-year and Seniors surveyed
    spring term
  • 50 states, Puerto Rico, Canada
  • 70 consortia WISCONSIN SYSTEM!
  • Related Initiatives
  • Faculty Survey of Student Engagement (FSSE)
  • Beginning College Survey of Student Engagement
    (BCSSE)

20
NSSE sightings all over Wisconsin!
21
Taking a look at NSSE
  • Based on effective educational practices
  • Designed and tested for high validity and
    reliability
  • Relatively stable over time
  • High credibility of self-reported data
  • Students will participate
  • Actionable data
  • Psychometric properties document on Web site

22
NSSE Survey
Student Behaviors
Student Learning Development
Institutional Actions Requirements

Reactions to People Environment
Student Background Information
23
NSSE Survey
  • Q.1 Academic activities
  • Q.2 Learning mental activities
  • Q.3 Reading writing
  • Q.4 Homework
  • Q.5 Academic challenge
  • Q.6 Co-curricular activities
  • Q.7 Enriching educational experiences
  • Q.8 Campus relationship
  • Q.9 Time usage
  • Q.10 Institutional emphasis
  • Q. 11 Gains
  • Q.12 14 Satisfactions

24
NSSEs Five Clusters ofEffective Educational
Practice
Enriching Educational Experiences
Supportive Campus Environment
Level of Academic Challenge
Active Collaborative Learning
Student Faculty Interaction
25
NSSE Results
  • A window into the undergraduate experience
  • Discover strengths and weaknesses in educational
    program
  • Identify areas that need attention to improve
    student learning and success
  • Help pinpoint aspects not in line with mission,
    or what institution expects
  • Link with other institutional data

26
  • What can we learn about first year student
    engagement from NSSE?

27
What do first-year students do?
1. What percent of full-time first-year students
study, on average, more than 20 hours per week?
(a) 12 (b) 18 (c) 30 (d) 41 (e) 50
b. 18 NSSE FY Wisconsin FY 16
28
Worrisome Gap? Time spent studying
  • First-year students average about 13-14 hrs. per
    week studying
  • Faculty Survey of Student Engagement (FSSE) data
    indicate that faculty expect students to spend
    more than twice that amount preparing (estimated
    24-30 hrs. a week for FT)
  • Entering first-year students EXPECT to study more
    than they actually do in college

29
What do first-year students do?
2. What percent of first-year students frequently
(very often often) asked questions in
class or contributed to class discussions?
(a) 28 (b) 35 (c) 47 (d) 58 (e) 65
d. 58 NSSE FY Wisconsin 51 WI Range 31 -
57
30
First Year Students Ask Questions in Class
31
Active and Collaborative Learning Scales
  • Collaborative Learning Scale
  • Worked with peers on projects during class
  • Worked with classmates outside of class to
    prepare assignments
  • Tutored/Taught other students
  • Discussed ideas from readings or classes with
    others outside of class (students, family)
  • Active Learning Scale
  • Asked questions in class
  • Make a class presentation
  • Participated in a community-based project
    (service learning) as part of a regular course

32
What do first-year students do?
3. True or False. Almost half of all students
spend no time on co-curricular activities.
True. 43 NSSE first-years, 48 seniors
report 0 hours
33
FY Students Time Spent on Cocurriculars
34
What do first-year students do?
4. What percent of first year students report
they frequently (often or very often) received
prompt feedback on their academic
performance? (a) 27 (b) 35 (c) 44 (d) 53 (e)
none of the above
d. 53 NSSE first-years, Wisconsin 48
35
FSSE NSSE comparison Prompt Feedback
Lower Division
Upper Division
FACULTY gave prompt feedback often or very often
93 93
1st yr. Students
Seniors
STUDENTS received prompt feedback often or very
often
64 76
36
What do first-year students do?
  • 5. What percent of first-years frequently
    prepared 2 drafts of a paper or assignment
    before turning it in?
  • ( very often and often)
  • (a) 65 (b) 56 (c) 43 (d) 31
  • b. 56 NSSE FY Wisconsin 55

37
Striving for Deep Learning
  • Deep learning is learning that takes root in
    our apparatus of understanding, in the embedded
    meanings that define us and that we use to define
    the world.
  • J. Tagg (2003). The learning paradigm college
    (p. 70). Bolton, MA Anker

38
Deep Learning Sub-ScaleIntegrative Learning
  • Worked on a paper or project that required
    integrating ideas or information from various
    sources
  • Included diverse perspectives (different races,
    religions, genders, political beliefs, etc.) in
    class discussions or writing assignments
  • Put together ideas or concepts from different
    courses when completing assignments or during
    class discussions
  • Discussed ideas from your readings or classes
    with faculty members outside of class
  • Discussed ideas from your readings or classes
    with others outside of class (students, family
    members, co-workers, etc.)

39
Deep Learning ItemIntegrative Learning Scale
  • Students indicate 1 never to 4 very
    often, they did the following during school
    year very often and often
  • Worked on paper/project that required integrating
    ideas, info. from various sources
  • Included diverse perspectives (different races,
    religions, genders, political beliefs, etc.) in
    class discussions or writing assignments
  • Put together ideas or concepts from different
    courses when completing assignments or during
    class discussions
  • Discussed ideas from readings or classes with
    faculty members outside of class
  • Discussed ideas from readings or classes with
    others outside of class (students, family
    members, co-workers, etc.)

NSSEville U FY - SR
85/92 /
79/82 /
61/80 /
25/40 - /
45/60 - /
40
Compensatory Effect of Engagement
  • - Student engagement positively related to FY
    and senior student grades and to persistence
    between the first and second year of college at
    the same institution
  • - Engagement has compensatory effect on FY
    grades and persistence to the second year of
    college.

41
(No Transcript)
42
(No Transcript)
43
(No Transcript)
44
Using NSSE results in your explorations of first
year student learning
  • Take a look at the instrument. What items or
    scales might you want to know how your first year
    students score?
  • - what comparison group (to seniors, peers at
    other similar institutions, among departments,
    criterion reference) is compelling?
  • What items do you want to know more about what
    students responses really mean?
  • What does prompt feedback mean to students?
  • What assignments do students think require them
    to synthesize ideas?
  • How else might you assess the effective
    educational practices on NSSE?
  • How might you use NSSE results to warrant or add
    to an inquiry project of interest to you?

45
Approaches to using NSSE in inquiry about
teaching and learning
  • Assess engagement in your class.
  • Add selected NSSE items to your course evaluation
    to obtain classroom level engagement data,
    compare to aggregate FY data
  • Use NSSE to pinpoint FY results deserving of
    additional inquiry
  • Ex FY student faculty engagement low. Conduct
    student faculty focus groups to discover
    obstacles to interaction
  • Examine group differences
  • Identify FY groups of interest (e.g., rural
    students, first-generation, majors, women in
    science) -examine difference
  • Assess impact of intervention
  • Ex Effect of FY seminar on engagement impact of
    service learning course on reflective learning
    and civic engagement outcomes

46
Localizing Findings
  • Variation in levels of student engagement within
    the university is greater than variation between
    universities
  • Improvement initiatives might best be designed
    and implemented at the college level (rather than
    based on institutional findings) to maximally
    impact overall student engagement on campus.

47
Localizing Findings contd
  • Even when institutional outcomes are
    disaggregated to the college levelless desirable
    engagement outcomes are attributed to
    shortcomings in another academic department
  • These less-engaged students are not my students
  • Many institutions have increased their
    institutional NSSE sample or done targeted,
    over-sampling efforts to specific majors.

48
Localizing Findings contd
  • Disaggregating NSSE outcomes to the department
    level often leads to the same series of
    observations that occurred at the college level
  • a faculty member within the department will often
    explain that these less-than-optimal levels of
    engagement are not characteristic of students in
    my classes, but rather must be in other course
    offerings within the department. These are not my
    students!

49
CLASSE (Classroom Survey of Student Engagement)
created by Bob Smallwood, Univ. North Florida and
Judy Ouimet, Univ. of Nevada, Reno
  • CLASSE is a pair of survey instruments that
    enable one to compare what engagement practices
    faculty particularly value and perceive important
    in a designated class with how frequently
    students report these practices occurring in that
    class.
  • CLASSEStudent is the survey instrument completed
    by each student enrolled in the designated class,
    while CLASSEFaculty is the survey instrument
    completed by the faculty instructor of the
    designated class.

50
Approaches to using NSSE in inquiry about
teaching and learning
  • Assess engagement in your class.
  • Add selected NSSE items to your course evaluation
    to obtain classroom level engagement data,
    compare to aggregate FY data
  • Use NSSE to pinpoint FY results deserving of
    additional inquiry
  • Ex FY student faculty engagement low. Conduct
    student faculty focus groups to discover
    obstacles to interaction
  • Examine group differences
  • Identify FY groups of interest (e.g., rural
    students, first-generation, majors, women in
    science) -examine difference
  • Assess impact of intervention
  • Ex Effect of FY seminar on engagement impact of
    service learning course on reflective learning
    and civic engagement outcomes

51
Study effective practices
  • Teaching Learning Center at University of
    Wisconsin-Stout interviewed students to develop a
    more contextualized understanding of student
    engagement
  • TLC staff distilled a list of effective
    educational practices from the interview data,
    particularly around the most important factor to
    students student-faculty relationships.

52
Focus on desired pedagogy
  • First-year students less involved in service
    learning than JMU desired.
  • Workshops conducted to encourage faculty to adapt
    courses to include service learning
  • Studied change in participation of students and
    instructional practice

53
Improving teaching learning
  • NSSE and CIRP pointed to problems with first year
    students academic engagement, but WTAMU desired
    more holistic picture of students experience
  • Conducted Student Engagement Audit Focus Groups
    2 focus groups per college to discover what
    faculty and students found educationally engaging
    and identify classroom experiences that were
    engaging and disengaging

54
Approaches to using NSSE in inquiry about
teaching and learning
  • Assess engagement in your class.
  • Add selected NSSE items to your course evaluation
    to obtain classroom level engagement data,
    compare to aggregate FY data
  • Use NSSE to pinpoint FY results deserving of
    additional inquiry
  • Ex FY student faculty engagement low. Conduct
    student faculty focus groups to discover
    obstacles to interaction
  • Examine group differences
  • Identify FY groups of interest (e.g., rural
    students, first-generation, majors, women in
    science) -examine difference
  • Assess impact of intervention
  • Ex Effect of FY seminar on engagement impact of
    service learning course on reflective learning
    and civic engagement outcomes

55
Gender Differences in Engagement by First-year
Students (source NSSE 2006)
Female Students More Engaged Male Students More Engaged
Community service or volunteer work Tutored or taught other students
Hours per week preparing for class Hours per week relaxing and socializing (watching TV, partying, etc.)
Hours per week providing care for dependents living with you (parents, children, spouse, etc.) Hours per week participating in co-curricular activities
Worked harder than you thought you could to meet an instructor's standards or expectations Exercised or participated in physical fitness activities
Used e-mail to communicate with an instructor Discussed ideas from readings or classes with faculty members outside of class
Foreign language coursework Number of problem sets (problem-based homework assignments) that take less than an hour to complete
Prepared two or more drafts of a paper or assignment before turning it in Research project with faculty member
Attended an art exhibit, gallery, play, dance, or other theater performance Worked with faculty members on activities other than coursework (committees, orientation, student life activities, etc.)
56
Approaches to using NSSE in inquiry about
teaching and learning
  • Assess engagement in your class.
  • Add selected NSSE items to your course evaluation
    to obtain classroom level engagement data,
    compare to aggregate FY data
  • Use NSSE to pinpoint FY results deserving of
    additional inquiry
  • Ex FY student faculty engagement low. Conduct
    student faculty focus groups to discover
    obstacles to interaction
  • Examine group differences
  • Identify FY groups of interest (e.g., rural
    students, first-generation, majors, women in
    science) -examine difference
  • Assess impact of intervention
  • Ex Effect of FY seminar on engagement impact of
    service learning course on reflective learning
    and civic engagement outcomes

57
Study effects of potentially High Impact
Practices
  • First-year seminars and experiences
  • Common intellectual experiences
  • Learning communities
  • Writing-intensive courses
  • Collaborative assignments
  • Science as science is done
  • Diversity/global learning
  • Service learning/community service
  • Internships
  • Capstone experiences/projects

58
UW Green Bay Fall 2006 Freshman Seminar Pilot
Study
  • Comparing Freshman Seminar students from
    non-Seminar students
  • Asked questions in class or contributed to class
    discussions.
  • Worked with other students on projects DURING
    CLASS.
  • Used e-mail to communicate with an instructor.
  • Talked about career plans with a faculty member
    or advisor.
  • Discussed grades or assignments with an
    instructor.
  • Coursework emphasized APPLYING theories or
    concepts to practical problems or in new
    situations.
  • Coursework emphasized MAKING JUDGMENTS about the
    value of information, arguments, or methods, such
    as examining how others gathered and interpreted
    data and assessing the soundness of their
    conclusions

59
Institutional ExamplesExploring the First Year
Experience
60
California State University,
  • Seven principles of Good Practice in
    Undergraduate Education
  • NSSE results indicated need to focus on
    first-year experience
  • Campus-wide task force used selected NSSE data to
    determine how well students were doing and to
    pinpoint areas that needed improvement
  • First step - philosophy of First Year Experience
    distributed to entire campus

61
Plymouth State University
  • Student Affairs reviews NSSE data to assess
    out-of-class support
  • NSSE and institutional survey data used to revise
    General Education program
  • NSSE results supported grant application for
    faculty development center

PSU
62
St. Olaf College
  • Committee on First-Year Experience (CoFYE),
    first-year curriculum faculty student life
    staff conducted self-study, used NSSE results to
    compare St. Olaf FY to peers on key indicators,
    conducted focus groups with students and faculty
    using questions similar to NSSE items to gain
    insight into their results.
  • Self-study report serves as baseline for
    assessing change and to launch revisions to
    First-Year Experience

63
Truman State University
  • NSSE results showed need for

improvement in academic advising
  • Implemented professional advisors in residence
    halls
  • Based on NSSE and internal survey, 50 faculty
    were interested in advising workshop
  • Recommendation to provide faculty with regular
    workshops and advising newsletter
  • VP of Academic Affairs to carry out comprehensive
    assessment of advising on campus

64
University of Akron
  • Distribute campus-wide philosophy
  • More emphasis on diversity in FY and Gen Ed.
    Courses
  • Reward system for faculty teaching FY courses
  • One Voice syllabi common components in FY
    curriculum
  • Promote service learning, mentoring,
    undergraduate research
  • Foster student-faculty interaction
  • Student Success Seminar

65
Worcester Polytechnic Institute
  • NSSE results showed FY students were less engaged
    than seniors
  • New FY interdisciplinary, inquiry-based seminars
    better integration of disciplines engaging
    introductory courses
  • Associate Dean appointed to Office for the First
    Year
  • Assessment plan in development with NSSE
    indicators as key component

66
FSSE Survey
  • Faculty perceptions of how often their students
    engage in different activities
  • The importance faculty place on various areas of
    learning and development
  • The nature and frequency of interactions faculty
    have with students
  • How faculty members organize class time

67
FSSE- NSSE inquiry Teaching and Learning
  • Use FSSE to explore pedagogical practices by
    school, department, lower division vs. upper
    division faculty
  • Compare NSSE and FSSE to examine gaps between
    faculty and student perceptions and behaviors

68
FSSE - Class time
69
NSSE FSSE Time Spent Preparing for Class
70
Beginning College Student Survey of Engagement
(BCSSE)
  • BCSSE measures
  • HS experiences Importance of college activities
  • Baseline self-assessment of knowledge and skills
  • Demographics
  • Use to examine gap between expectations and
    engagement
  • Study effect of students background on NSSE
    scores

71
New Student Expectations, Engagement and Success
in College
  • Path to student success begins pre-college,
    shaped by early experiences with institution
  • Overall pattern Students initial expectations
    for their academic involvement is less than
    faculty might wish, and their actual engagement
    is even less
  • Students set in place in their first semester of
    college the pattern of time allocation that will
    serve them across their years in college
  • Educators need to detail the specific activities
    that will lead to success in college

72
QA
  • For More Information
  • Email jikinzie_at_indiana.edu
  • NSSE Web site www.nsse.iub.edu
About PowerShow.com