Assessing the Effectiveness of Your Academic Advising Programs - PowerPoint PPT Presentation

Loading...

PPT – Assessing the Effectiveness of Your Academic Advising Programs PowerPoint presentation | free to view - id: 7a11b0-ZTQ2Y



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Assessing the Effectiveness of Your Academic Advising Programs

Description:

Title: Assessing the Effectiveness of Your Academic Advising Programs Author: Tom Grites Last modified by: Best Buy Created Date: 7/12/2009 1:13:17 PM – PowerPoint PPT presentation

Number of Views:88
Avg rating:3.0/5.0
Slides: 56
Provided by: TomG105
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Assessing the Effectiveness of Your Academic Advising Programs


1
Assessing the Effectiveness of Your Academic
Advising Programs
  • Session 4
  • Academic Advising Symposium
  • Innovative Educators
  • Thomas J. Grites

2
Overview
  • Terminology and Rationale
  • Assessment as a Process
  • Focus on the Contexts
  • Non-Assessment
  • Summary

3
Terminology
  • Assessment
  • Evaluation
  • Measurement (the tools)

4
Assessment(re academic advising)
  • Assessment is the process through which we
    gather evidence about the claims we are making
    with regard to student learning and the
    process/delivery of academic advising in order to
    inform and support improvement (Campbell,
    2008)

5
Evaluation
  • The systematic determination of merit, worth, and
    significance of something or someone
    (Wikipedia)
  • To ascertain or fix the value or worth (American
    Heritage Dictionary)
  • Simply putto judge

6
Uses/Contexts
  • Assessment tends to be more related to
    programmatic issues and outcomes
  • Evaluation tends to be more related to people
    (advisor) skills, performance, and outcomes
  • Its OK to use evaluation as part of the
    assessment process

7
Intentions (related to both)
  • Formative more associated with assessment
    includes a wider range of efforts requires more
    analysis provides a broader perspective focus
    on improvement
  • Summative more associated with evaluation more
    limited effort focus on Does it work? or How
    well was job performed?

8
The Rationale
  • a lack of assessment data can sometimes lead to
    policies and practices based on intuition,
    prejudice, preconceived notions, or personal
    proclivities none of them desirable bases for
    making decisions
  • (Upcraft and Schuh, 2002, p. 20)

9
More Rationale
  • In God we trust all others bring data.
  • An ounce of data is worth a pound of opinion.
  • (Magoon, c. 1975)

10
Other Reasons
  • Accountability
  • Effectiveness
  • Accreditation
  • Trustees/Regents
  • Legislators
  • Program Improvement (to monitor and improve
    student success) the most important reason

11
The Assessment ProcessA Cycle
  • Resources
  • Guide to Assessment of Academic Advising (2010)
    (CD available from NACADA via www.nacada.ksu.edu)
  • Assessment of Academic Advising Institute
  • (Feb 15-17, 2012 San Diego, CA)

12
(No Transcript)
13
Getting Started Identify Stakeholders
  • Complete set of advising constituents (students,
    staff and faculty advisors)
  • Broad range of key offices (Registrar, Enrollment
    Management, similar advising units, certain
    campus referral resources, IR office)
  • Critics, Antagonists, and Naysayers
  • FYIs Faculty Senate, Deans Council, Retention
    Committee, others as appropriate

14
The Assessment Process/Cycle
  • Alignment with institutional and unit
    missions/purposes
  • Specify goals and/or objectives
  • Identify the outcomes expected (student learning
    and/or programmatic)
  • Gather evidence (the measurements)
  • Share findings, interpretations, and
    recommendations
  • Begin implementation and re-start the cycle

15
Mission/Purpose
  • A working model
  • Academic advising is integral to fulfilling the
    teaching and learning mission of higher
    education. Through academic advising, students
    learn to become members of their higher education
    community, to think critically about their roles
    and responsibilities as students, and to prepare
    to be educated citizens of a democratic society
    and a global community. Academic advising engages
    students beyond their own world views, while
    acknowledging their individual characteristics,
    values, and motivations as they enter, move
    through, and exit the institution.
  • (Preamble, Concept of Academic Advising, NACADA,
    2006)

16
Goals/Objectives(how we intend to achieve our
mission)
  • These need to emanate from and reflect the nature
    of the unit to be assessed (total institution,
    Advising Center and its clientele, College Deans
    Office, etc)
  • Examples
  • To assist students to become independent and
    lifelong learners
  • To assist students in understanding the relevance
    of the total curriculum
  • To assist students in making good decisions based
    on their own evidence (e.g., selecting a major)

17
Identify Outcomes
  • Student Learning Outcomes examples
  • All students will select an appropriate major by
    the end of their third semester.
  • All students will become engaged in at least one
    co-curricular activity each semester.
  • All students will be able to identify and will
    select courses that enhance their human capital.
  • At least 30 of the students will choose to
    participate in a service learning course.

18
A Task For Youre Course Selection
  • How many courses are in your Catalog? (A)
  • How many courses are required to earn a degree
    from your institution? (B)
  • What percentage of what your institution offers
    do students actually take in order to earn a
    degree? (B/A)
  • Now, for each course a student takes, how many
    are eliminated?

19
Outcomes (continued)
  • Programmatic/Process Outcomes examples
  • As a result of our advising services, the
    retention/persistence rate of first-year students
    will increase by 10 in the next 3 years.
  • As a result of our intervention strategies, the
    percentage of students who are removed from
    academic probation will increase by 10 in the
    next academic year.
  • After two advising sessions, all students will
    come to their future sessions with a degree audit
    already run and with a plan for meeting
    outstanding requirements

20
Everybodys Favorite
  • All students will be able to understand,
    appreciate, and articulate the value of general
    education.

21
Gather Evidence
  • Mapping the Experience (Maki, 2004)
  • Not all outcomes will necessarily occur as a
    direct result of what we do as advisors, so we
    need to know what other learning opportunities
    exist in order for the students to meet our
    stated goals/objectives.
  • WHAT learning is to occur?
  • WHERE might it be learned?
  • By WHEN should it be learned?
  • This process can also inform the kinds of
    evidence that needs to be gathered for
    appropriate assessment.

22
(No Transcript)
23
Gather (Multiple) Evidence
  • Satisfaction Surveys (OK, but not enough)
  • Institutional Data (changes of major, drop/add
    transactions, grades in gateway courses,
    retention and graduation rates, use of services
    provided elsewhere, advisor advisee ratios,
    NSSE, etc)
  • Office Data (number of appointments vs. walk-ins,
    nature of sessions, results of sessions,
    transcript analyses, other advisor
    tasks/activities What did you learn?)
  • Focus groups (of clients, of faculty advisors,
    others a qualitative measure)
  • The Advising Syllabus can inform what evidence
    should be collected
  • http//www.nacada.ksu.edu/Clearinghouse/Advising
    Issues/syllabus101.htm
  • http//intraweb.stockton.edu/eyos/page.cfm?siteI
    D123pageID42syllabus

24
Share the Results
  • Tips
  • Be sure that the stakeholders you identified
    earlier are informed throughout the process in
    order to enable their support in the
    decision-making for implementation of your
    recommendations.
  • Academics have a preferred method of review, so
    it makes sense to conform to their expectations.

25
Sharing the Results(Format and Content)
  • These elements are often best provided in a
    standard research report or journal format
  • Purpose of the assessment project
  • Method of data collection
  • Results found
  • Interpretation of the results
  • Recommendations with timetable for and
    anticipated cost of implementation
  • Executive Summary

26
Sample Implementation Recommendations
  • Redesign the advising effort in the Orientation
    Program
  • Develop a peer advising/mentoring program
  • Streamline office procedures
  • Initiate proposals for policy changes
  • Improve communication with other service offices
    and personnel
  • Request/Reallocate resources (human, fiscal,
    and/or physical)

27
You Did It!
  • This will complete the assessment cycle, which
    provides the evidence for change and improvement.
  • Completion of the cycle may also provide new
    goals and objectives, new assessment strategies
    and tools, and other aspects that will be need to
    be included in beginning the next cycle.

28
Youve earned a break
  • Please take a few minutes to submit any questions
    you may have at this point via the Chat function.

29
Back to the Original Contexts
  • People
  • Academic advising, as a teaching and learning
    process, requires a pedagogy that incorporates
    the preparation, facilitation, documentation, and
    assessment of advising interactions. Although the
    specific methods, strategies, and techniques may
    vary, the relationship between advisors and
    students is fundamental and is characterized by
    mutual respect, trust, and ethical behavior.
  • (Concept of Academic Advising, NACADA, 2006)

30
Assessment (Evaluation) of Advisors
  • SELECTION
  • TRAINING
  • EVALUATION
  • RECOGNITION/REWARD

31
Selection of Academic Advisors
  • Use the best
  • Add from other resources/units
  • Target specific populations
  • Cross disciplinary lines
  • Develop mentors
  • Use other skills/expertise

32
Faculty vs.
Professional Staff Advisors
  • They are hired via a search process and have
    specific job descriptions they are selected
  • Their training is systematic, intentional, and
    ongoing staff development is expected
  • They are evaluated through annual performance
    reviews
  • They are rewarded with salary and benefits
  • Too often all are expected or required to advise,
    but also teach, publish, seek grants, etc no
    selection
  • Training ranges from near nothing to perhaps a
    day or 2, but usually only a few hours
  • Evaluation is not systematic
  • Recognition/Reward is very limited in the tenure
    and promotion process mostly intrinsic can also
    be a reverse structure (better more)

33
ASSESSMENT (Evaluation)
  • 37 OF ALL INSTITUTIONS HAD NO
    PERFORMANCE EVALUATION MEASURES FOR
    FACULTY IN THEIR ACADEMIC ADVISING ROLE
  • 44 in 2 yr public institutions
  • 25 in 4 yr public institutions
  • 39 in 4 yr private institutions
  • (Habley, 2004)

34
PARAMETERS (faculty advisors)
  • Faculty Contract
  • List of Responsibilities
  • Availability of Resources
  • Assignment of Advisees
  • Recognition/Reward

35
NACADA Core Values
  • Academic Advisors are responsible
  • to the individuals they advise
  • for involving others, when appropriate, in the
    advising process
  • to their institutions
  • to higher education in general
  • to their educational community
  • for their professional practices and for
    themselves personally

36
Tools for Assessment of Advisors
  • Self evaluation
  • Student surveys (locally designed)
  • Survey of Academic Advising (ACT)
  • Academic Advising Inventory (NACADA)
  • Student Satisfaction Inventory (Noel-Levitz)
  • NACADA Clearinghouse

37
Back to the Original Contexts
  • Program
  • a lack of assessment data can sometimes lead to
    policies and practices based on intuition,
    prejudice, preconceived notions, or personal
    proclivities none of them desirable bases for
    making decisions
  • (Upcraft and Schuh, 2002, p. 20)

38
Tools and Strategies
  • Satisfaction Surveys
  • Institutional Data
  • Office Data
  • Focus groups
  • The Advising Syllabus
  • External Reviews
  • Others

39
The CAS Standards
  • MISSION educational plans
  • PROGRAM student learning and development
    outcomes achieved through programs and services
  • LEADERSHIP appropriate credentials and
    experience
  • ORGANIZATION reporting structure

40
Standards (continued)
  • HUMAN RESOURCES adequate staffing appropriate
    credentials
  • FINANCIAL RESOURCES adequate funding to meet
    mission professional development
  • FACILITIES, TECHNOLOGY, EQUIPMENT
  • space, computing access and capability
  • LEGAL RESPONSIBILITIES knowledge of Federal,
    State, and institutional laws and policies

41
Standards (continued)
  • EQUITY AND ACCESS non-discriminatory policies
    practices, and procedures
  • CAMPUS AND EXTERNAL RELATIONS both on and off
    the campus
  • DIVERSITY recognize and nurture harmonious
    academic advising environment
  • ETHICS privacy, confidentiality, fairness,
    fidelity
  • ASSESSMENT EVALUATION why were here

42
CAS Assessment Worksheet
43
An Economic Model
  • Though not an outcomes-based model per se, this
    approach to assessment is a functional analysis
    based on the premise that every task an advisor
    performs and every operation that an advising
    unit conducts has some monetary value related to
    it.
  • The analysis results in a comparison of the
    fiscal expenditures required to perform the tasks
    to the cost benefits as results.
  • The model operates from the perspective of a
    threat to the existence of an advising unit,
    function or personnel.
  • A quick example

44
Determining Your Worth
  • Identify every function the unit performs
  • Identify all possible alternatives for each
    function, if the unit was dissolved
  • Determine the cost of those functions that cannot
    be replaced and who would perform them estimates
    will sometimes be required
  • Determine the cost of those that could be
    eliminated
  • (NACADA Clearinghouse)

45
Identify Functions (sample)
  • Transfer Credit Equivalencies
  • Graduation Certification
  • Probation Students
  • Assign Advisors
  • Develop/Distribute Advising Materials
  • Advise Prospective Students
  • Webpage, CAPP Audit, NJ Transfer
  • Conduct Advisor Training
  • Individual Advisees (c. 300)

46
Alternatives (to third function probation
students)
  • Shift to the faculty
  • Shift to another office
  • Abolish the policy
  • All options would likely result in less student
    satisfaction and higher attrition.
  • ??? What is the (dollar) value of a retained
    student at your institution???

47
Determining Your Worth (continued)
  • Determine the institutions (dollar) value of a
    retained student and use this figure as part of
    your worth if you can reasonably demonstrate that
    your unit, program, or intervention resulted in
    some retention of students
  • Total the dollar amounts for each function
  • Compare the budget allocated for the unit to the
    value of the costs of the reassignment of
    functions also include the retention amounts
    calculated
  • Answer the question Are you worth it?

48
Associated Dollar Amounts
  • 57,750 Transfer Credit Equivalencies
  • 20,250 Graduation Certification
  • 48,500 Probation Students
  • 2,340 Assign Advisors
  • 73,740 Develop/Distribute Advising Materials
  • 18,475 Advise Prospective Students
  • 122,700 Webpage, CAPP Audit, NJ Transfer
  • 16,600 Conduct Advisor Training
  • 33,000 Individual Advisees (c. 300)

49
Compare Value to Cost
  • Our total value for the fiscal year
    was 601,415
  • Our budget for the same fiscal year
    was 525,000
  • Ask yourself (and others)
  • Are We Worth It?

50
Another Brief Break
  • Please take a few minutes to submit any questions
    you may have at this point via the Chat function.

51
What Are We Not AssessingAnd Should We Be?
  • Student expectations, intentions
  • Whether advising strategies actually can be
    attributed to different types of student success
    (removed from probation, successful choice of
    major, overcome a skills deficiency or harmful
    social habit, etc)
  • Retention and graduation rates of transfer
    students

52
Expectations vs. Experience
  • Expect Experience
  • Be Undecided 7 20
  • Change Majors 12 65-85
  • Fail a course 1 16
  • Extra time to complete degree 8 60
  • Drop out 1 40
  • Transfer institutions 12
    28
  • Work while in school 36 60
  • Seek personal counseling 6
    27
  • Need tutoring 15 20
  • Seek career guidance 5 25
  • (Habley 2011)

53
Non-Assessment (continued)
  • Use and value of articulation agreements number
    of students who use them, are they updated
  • Currency of academic policies, e.g., course
    repeats, course pre-requisite criteria,
    drop/add/withdrawal processes, academic warning,
    probation, and suspension policies
  • Does advisor training result in better advising?

54
Summary
  • Assessment is a process, not an event
  • Collaboration and cooperation are necessary for
    productive assessment to occur
  • An ounce of data is worth a pound of opinion
    (Magoon, c. 1975) Avoid the N of 1 syndrome
  • The purpose and results of assessment should
    always be used for program and/or advisor
    improvement in order to realize maximum student
    development and success

55
More Questions???
  • THANK YOU!!!
  • Tom.Grites_at_stockton.edu
About PowerShow.com