Transfer Student Presentation - PowerPoint PPT Presentation

Loading...

PPT – Transfer Student Presentation PowerPoint presentation | free to view - id: 13fc54-ZjZiY



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Transfer Student Presentation

Description:

Measure educational outcomes to identify and enhance effective educational practices ... Student Learning Outcomes ... VSA Core Outcomes Evaluation was Premised ... – PowerPoint PPT presentation

Number of Views:31
Avg rating:3.0/5.0
Slides: 32
Provided by: LoriVa7
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Transfer Student Presentation


1
The VSA Responding to a New Era of Accountability
Accountability Today Tomorrow AACU, AASCU,
APLU Williamsburg, VA October 6-8, 2009
2
Voluntary System of Accountability (VSA)
  • Initiative by public 4-year universities to
    supply basic, comparable information on the
    undergraduate student experience to important
    constituencies through a common web report the
    College Portrait.

Sponsored by APLU AASCU. Funding from Lumina.
3
VSA Goals
  • Provide a useful tool for students during the
    college search process
  • Assemble information that is transparent,
    comparable, and understandable
  • Demonstrate accountability and stewardship to
    public
  • Measure educational outcomes to identify and
    enhance effective educational practices

4
Multiple Audiences
  • Prospective students and families
  • High school, community college counselors
  • Policymakers and elected officials
  • Campus faculty and staff
  • Accrediting associations, institution and state
    boards
  • Taxpayers and public-at-large

5
VSA Participation
  • 329 institutions from 49 states and Puerto Rico
    (10/05/09)
  • 63 of 520 APLU AASCU institutions
  • 250 College Portraits posted on
    www.collegeportraits.org

6
VSA Context
  • Increasing disinvestment in higher education
  • Perceived lack of useful and transparent data
    prevents institutions from demonstrating
    accountability and contribution to public good
    (Spellings Commission)
  • Policy-makers and employers demand evidence of
    educational outcomes particularly broad
    transferable skills

7
A Community Effort
  • Concept outlined in APLU white papers, refined
    through feedback from members
  • Strengthened by AASCU endorsement
  • Structure developed by task forces - presidents,
    provosts, student affairs, IR
  • 82 members from 70 institutions
  • Input from associations and larger HE community

8
Result The College Portrait
Free web report that communicates user-friendly,
consistent information about college campuses.
  • Student Family Information
  • Student Experiences Perceptions
  • Student Learning Outcomes

9
www.collegeportraits.org
No rankings, no spin ... just the facts!
10
Student Family Information
  • Institution description, highlights
  • Student characteristics
  • Cost of attendance, net cost calculator
  • Success progress rate
  • Degree programs, class sizes
  • Post graduation plans

11
Introduction to Campus Community
Menu links take users to each section and data
element
Campus Defined Buttons
12
Success Progress Rate
  • Developed as alternative to IPEDS graduation rate
  • Focuses on student success in higher education
    system
  • Tracks student progress across 2-year and 4-year
    institutions using NSC data
  • New cohort query available to all Student Tracker
    users

13
(No Transcript)
14
VSA College Cost Calculator (new version late
fall 2009)
Estimate of costs tuition/fees, housing choice,
books, etc
Estimate of awards Grants, student loans, work
study
NET PRICE
15
Student Experiences Perceptions
  • Selected senior responses from one of 4 surveys
    NSSE, CSS, CSEQ, UCUES
  • Snapshot of student experiences on campus within
    6 common areas
  • group learning
  • interaction with faculty
  • institutional commitment
  • active learning
  • diversity experiences
  • student satisfaction

16
6 Common Constructs
Selected responses from Seniors
17
Student Learning Outcomes
  • Pilot project to measure learning gains in broad
    cognitive skills at institution level
  • Skills critical thinking, analytic reasoning,
    problem solving, written communication
  • Institutional assessment as one part of
    comprehensive assessment plan
  • Combined with other types of assessments
    courses, programs, general education
  • Combined with other outcomes measures
    professional licensure exams

18
Student Learning Outcomes
Learning gains in critical thinking, analytic
reasoning, written communication measured at
institution level
Reported in a common format using CAAP, CLA or
MAPP
19
FIPSE Test Validity Study
  • Findings

20
Background
  • For VSA reporting we needed to measure
    value-added for the core learning outcomes of
    critical thinking and written communications.
  • Because many provosts strongly objected to using
    a single test to measure these outcomes at all
    universities, VSA taskforces identified three
    tests that measured the outcomes. The VSA
    Presidential oversight taskforce decide to permit
    participating universities to choose from the
    three the test that best suited their campus
  • The three companies that produce the tests agreed
    to adopt the same value-added computation and
    reporting technique and format so that reporting
    would be standardized across universities while
    controlling for entering student quality.

21
The FIPSE study
  • What remained was to ensure that the three tests
    actually produced consistent scores.
  • The only way to be sure of this was to have a
    large number of students take each of the three
    tests and observe the results.
  • Funding from the Fund for the Improvement of
    Post-Secondary Education made the study possible.

22
Study Parameters
  • 13 tests administered to 1,100 students at 13
    universities
  • 13 tests components of CLA, CAAP, MAPP
  • 4 tests of critical thinking
  • 2 tests of reading
  • 2 tests of mathematics
  • 4 tests of writing
  • 1 test of science

23
Participating Universities
  • University of Michigan, Ann Arbor
  • University of Minnesota, Twin Cities
  • University of Texas, El Paso
  • University of Vermont
  • University of Wisconsin, Stout
  • Alabama AM
  • Arizona State
  • Boise State
  • California State-Northridge
  • Florida State
  • MIT
  • Trinity College
  • University of Colorado-Denver

24
Sample Parameters
25
Construct Validity
  • Construct Validity is very high when the school
    is the unit of analysis
  • All tests rank schools similarly, regardless of
    the construct or the response format
  • Mean correlation .92 for 9 multiple choice
    tests
  • Mean correlation .84 for 4 constructed response
    measures
  • Mean correlation .85 for multiple choice test
    and constructed response measure of the same
    construct

26
Senior Freshman Score Differences
  • Score gains between class levels indexed to
    common measure of effect size to control for
  • score distribution differences across tests
  • differences in average ACT/SAT scores
  • Larger effect sizes indicate greater differences
    in freshman versus senior scores
  • Seniors had higher mean scores than freshmen on
    all tests except the CAAP mathematics exam

27
Effect Sizes
  • Effect sizes not systematically related to
    constructs tested, response format or test
    publisher
  • Effect sizes across the 12 tests range from .25
    to .5 standard deviation (CAAP math excluded)
  • Effect sizes
  • ACT .33 (excluding math test)
  • CLA .31
  • ETS .34

28
Reliability
  • Reliability is score consistency
  • if there is a 100 chance a student will receive
    the same score on different forms of a test, the
    reliability score is 1.0
  • if there is a 0 chance, the reliability score is
    0.0
  • When the school was unit of analysis, score
    reliability varied from .75 to .87 across the 13
    tests
  • Conclusion Score reliability is not a major
    concern

29
Overall Conclusions
  • Validity and Reliability are high across tests
    and should not affect choice of measure
  • Factors affecting adoption
  • Acceptance by students, faculty, administrators
    or other policy makers
  • Trade-offs in cost, easy of administration, etc
  • Utility of the test for other purposes such as
    supporting campus activities and services or
    providing guidance on improving learning

30
The Premise on Which VSA Core Outcomes Evaluation
was Premised is Sound
  • While the TVS study did not measure Value-Added
    using the VSA technique, it did find that the
    effect size was approximately the same regardless
    of the exam utilized.
  • The premise that portions of CLA, CAAP and MAPP
    can be used interchangeably to produce roughly
    consistent value-added learning outcomes
    measurements for universities with similar
    students is consistent with the VSA finding.
  • Thus VSA will continue to permit universities to
    choose to use either CLA, CAAP or MAPP.

31
Final Report
  • Two Reports are available on the VSA website
  • The complete (and technically complex) TVS full
    report as well as
  • An interpretative (and less technical) report
    Christine Keller and I authored especially for
    VSA participating schools can be found at
  • http//www.voluntarysystem.org/index.cfm
About PowerShow.com