But What Is Good Enough The National Conversation about Accountability and Student Learning - PowerPoint PPT Presentation


PPT – But What Is Good Enough The National Conversation about Accountability and Student Learning PowerPoint presentation | free to download - id: 23c65-MTAxM


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation

But What Is Good Enough The National Conversation about Accountability and Student Learning


The National Conversation about Accountability and Student Learning. Barbara ... Larry King? Parade Magazine? Are you kidding? respond to public need, interest ... – PowerPoint PPT presentation

Number of Views:91
Avg rating:3.0/5.0
Slides: 45
Provided by: bwri
Learn more at: http://www.aacu.org


Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: But What Is Good Enough The National Conversation about Accountability and Student Learning

But What Is Good Enough? The National
Conversation about Accountability and Student
  • Barbara D. Wright
  • Associate Director,
  • WASC
  • bwright_at_wascsenior.org

Our plan for the session
  • Some background, issues, opinions
  • Small group discussion
  • Reporting out, closing observations

My pet assessment peeve . . .
  • Lots of focus on how, on process
  • Some examples of use
  • Little or nothing on actual findings

Why not?
The pendulum has swung . . .
  • 1983-1990 heavy focus on assessment for
    accountability, ascertaining level of achievement
  • 1990-2003 more focus on assessment for quality
    improvement, less on quality assurance
  • 2003 present renewed focus on quality
    assurance, standard-setting, comparability

The context
  • Boomers the most highly educated generation in
    US history retiring
  • Replacement workers need higher educational
    levels for a knowledge-based global economy
  • College costs are rising, results declining
  • The education gap and ambition gap (Th.
    Friedman) put US at competitive disadvantage
  • Prosperity, national security are at risk

The Commission on the Future of Higher Education
Issue Paper 2
Accountability/Assessment(2006) by Charles
Miller Geri Malandra
  • National Assessment of Adult Literacy less than
    1/3 of college graduates could read complex
    texts, make inferences
  • National Survey of Americas College Students
    20-30 of college grads have only basic
    quantitative skills
  • Employers college grads lack workplace skills

Accountability/Assessment by Charles Miller
Geri Malandra
  • Colleges standards are fuzzy
  • HE is focused inward, doesnt communicate
    outcomes to the public
  • There are no commonly used tests or other
  • College courses are not designed to foster
    critical thinking and problem solving, which are
    hard to address with lecture formats and
    multiple-choice tests.

Accountability/Assessment by Charles Miller
Geri Malandra
  • Clearly lacking is a consumer-oriented,
    nationwide system for comparative performance
    purposes, using standard formats, designed for
    two primary audiences student (and parents) and
    policy makers.

Accountability/Assessment by Charles Miller
Geri Malandra
Promising efforts include
  • AACU approach (Our Students Best Work, 2004
    Liberal Education Outcomes, 2005)
  • Accreditations focus on student learning
  • SHEEOs endorsement of a focus on student
  • Business Higher Education Forums endorsement of
    multiple methods and linking with K-12 education
  • New instruments (NSSE, CLA, ETS MAPP)

The Commission on the Future of Higher Education
Issue Paper 12
Improving College Readiness and Success for All
Students A Joint Responsibility between K-12 and
Postsecondary Education (2006) by Michael W.
Kirst and Andrea Venezia
  • Few standards are developed to the 11th or 12th
    grades or connected to the academic expectations
    of colleges
  • It is up to higher education to provide clear
    signals about what students need to know and do
    to be ready for college-level coursework.
  • Grim data on remediation, retention, graduation

SREB Getting Students Ready for College and
Careers, Kaye, Lord, and Bottoms, 2006
  • All high school students need a rigorous course
    of study that leads to readiness for career or
  • Job 1 High school and college faculty need to
    define and agree on clear outcomes and standards
  • Senior-year experiences need to link directly to
    workplace and college requirements
  • Selected standardized tests are reviewed

Aligning Postsecondary Expectations and High
School Practices The Gap Defined, ACT, 2007
High school teachers in all content areas
(English/writing, reading, mathematics, and
science) tended to rate far more content and
skills as important or very important
Postsecondary instructors focused on essential
knowledge and skills instead of a broad array
concentrating on understanding and rigorously
applying fundamental principles.
NASULGC Discussion Paper Improving Student
Learning in Higher Education Through Better
Accountability and Assessment, McPherson and
Shulenburger, 2006
  • Improving student learning lies at the very
    core of the value system of the academy
  • A federally mandated testing system would be
  • Assessment should provide information that helps
    to improve learning
  • Any accountability system should allow for
    differentiation by type of institution (i.e.,
    mission, student characteristics)

NASULGC Discussion Paper, McPherson and
Shulenburger, 2006
  • Test scores should be related to the predicted
    outcomes of a group of students
  • Focus should be on a few key competencies
  • Data should be readily available, easily
    understood by students, parents, public
  • The voluntary system of accountability (VSA)
    responds to current demands

NASULGC Discussion Paper McPherson and
Shulenburger, 2006
  • AACUs work noted
  • NSSE, CIRP, CLA cited
  • Problems of descriptive data described
  • US News rankings criticized
  • Advantages of direct measures noted
  • Standarized tests not seen as answer (but later
    included in VSA)
  • Voluntary system of multiple measures focused on
    small number of simple, useful indicators is

The Spellings Commission Report --
A Test of Leadership. Charting the Future of U.S.
Higher Education
September 2006
Spellings Commission concerns . . .
  • HE grads too often deficient in
  • Reading and writing
  • Math
  • Critical thinking
  • Problem solving
  • Ability to be life-long learners
  • Other countries overtaking the US in
  • HS graduation rates
  • College-going rates
  • Baccalaureate graduation rates

Concerns, cont. . . .
  • Lack of alignment between HS and HE expectations
    for learning
  • Lack of information about costs, comparative
    quality of colleges
  • Achievement gaps (racial, ethnic, SES)
  • HE insufficiently innovative, adaptable,
    efficient, focused on outcomes
  • Accreditation part of the problem, not a solution

On quality, transparency, and accountability
  • Despite increased attention to student learning
    results by colleges parents and students have
    no solid evidence, comparable across
    institutions, of how much students learn in
    colleges or whether they learn more at one
    college than another Accreditation reviews are
    typically kept private, and still focus on
    process reviews more than bottom-line results for
    learning or costs (p.13-14).

Commissions Recommendations
  • Seamless alignment of high school and college
  • Performance outcomes as the core of
    accreditation, over inputs or process
  • Technology and new pedagogies used to get more
    learning at lower cost
  • A culture of accountability and transparency
    with consumerfriendly information

Recommendations, cont.
  • Institutions should measure and report
    meaningful student learning outcomes (CLA, NSSE,
    MAPP noted)
  • Availability of test scores, certification and
    licensure rates, time to degree, graduation
    rates, other data should be a condition of
  • User-friendly data should allow
    inter-institutional and interstate comparisons

Hart Survey of Employers Views on the
Accountability Challenge (1/08)
  • Satisfaction with
  • Entry-level skills
  • Teamwork, intercultural skills
  • Ethical judgment
  • Dissatisfaction with
  • Global knowledge
  • Self-direction
  • Writing
  • Ability to advance in the field

Employers views on assessment
  • Dismissed
  • Tests of general content knowledge, particularly
    multiple-choice tests
  • Comparisons of colleges to one another
  • Trusted
  • Real-world and applied approaches
  • Evaluations of internships, projects by faculty
    or supervisor
  • E-Portfolios
  • Essay tests

The challenge . . .
On our campuses, to find ways to assess student
learning and report the findings in ways that
support education and do no harm. To keep
abreast of and respond to national developments
in useful ways.
Discussion questions
  • How would you describe overall awareness of these
    issues on your campus? What is the level of
  • Who on your campus follows the national
    conversation? How is information communicated to
    the rest of the campus?
  • What policies or structures are in place to

Choice of assessment method matters.
  • Students value and learn what we teach and test.
  • How we teach and test matters as much as what
  • What and how we assess also matters.
  • We get more of what we test or assess, less of
    what we dont.

Higher-order thinking ( adapted from L.
Resnick, 1987)
  • Its nonalgorithmic, i.e., the path of action is
    not fully specified in advance.
  • Its complex, i.e., the total path is not
    visible from any single vantage point.
  • It often yields multiple solutions, each with
    costs and benefits.
  • It requires nuanced judgment and interpretation
  • It involves application of multiple criteria,
    which may conflict with one another

Higher order thinking, cont
  • It often involves uncertainty not everything
    about the task is known or can be.
  • It requires self-regulation someone else is not
    giving directions.
  • It involves making meaning, discerning patterns
    in apparent disorder.
  • It is effortful the elaborations and judgments
    required entail considerable mental work and are
    likely to take time.

Other ways to think about higher-level learning .
. .
  • Blooms taxonomy (knowledge, comprehension,
    application, analysis, synthesis, evaluation)
  • Perry Scheme of Intellectual Development
    (dualism, multiplicity, relativism, commitment)
  • Biggs, Entwistles surface vs. deep learning
  • Constructivism
  • ???

Methods for complex outcomes
  • are direct
  • are open-ended
  • focus on essentials, principles
  • pose authentic, engaging tasks
  • require meaning-making, judgment
  • require active expression
  • are scored for understanding, not just

Methods for complex outcomes include
  • Portfolios
  • Capstones
  • Performances
  • Common assignments, templates
  • Secondary readings
  • Course management programs
  • Local tests, comps in the major
  • The CLA

Four dimensions of learning --
  • What students learn (cognitive as well as
    affective, social, civic, professional, spiritual
    and other dimensions)
  • How well (thoroughness, complexity, subtlety,
    agility, transferability)
  • What happens over time (cumulative, developmental
  • Is this good enough? (the DE question)

Thinking about standards . . .
  • Absolute standards knowledge/skill level of
    champions, award winners, top experts
  • Contextual standards appropriate expectations
    for, e.g., a 10-year old, a college student, an
    experienced professional
  • Developmental standards amount of growth,
    progress over time, e.g., 2 years of college, 5
  • (Institutional, regional, national standards?)

We know what, but do we know how well, or
what is good enough? How might we know?
Once assessment is in place. . .
  • We need to not only use our findings to make
  • we need to reveal our findings
  • and use them to set inter-institutional standards
    for college-level learning

This is the next frontier in assessing and
improving students learning and development. If
we dont do it, others will.
Were not starting from zero. Assessment is a
foundation to build on
  • Outcomes pervasive
  • Gathering student work in process
  • Analysis and use coming along
  • Public sharing of findings ?
  • Institutional performance standards a handful?
  • Cross-institutional standards, rubrics and
    examples of student work not yet, but why not?

We all have implicit standards, e.g., for
  • Remedial writing, math
  • First-year writing, math
  • Course-level judgments about grades, especially
    Ds, Fs
  • Competency-based curricula

As we did with outcomes, now we need to make the
implicit explicit and collectively owned, openly
What do students know, and how do we all know
they know it? Together we What do students
know? Is it good enough? A course of action might
be to
  • Analyze student work for quality, proficiency
  • Determine the ceiling floor of our students
    learning at different points
  • Compare it to work at other institutions
  • Begin to reach a collective sense of
    college-level learning in a few key areas
  • And communicate that level of learning to
  • students, parents
  • high school faculty and students
  • policy makers
  • the general public

It wont be easy. Plenty of factors work against
the use of standards.
  • Traditional proxies requirements, credits, seat
  • Grade inflation
  • The unspoken faculty/student truce (I wont
    bother you if you dont bother me.)
  • Insufficiently rigorous AP and dual enrollment
  • Lack of shared culture, trust
  • Complexity of the issue what standards, for
    whom, applied how, with what consequences?

Are parents students really interested in
Research suggests otherwise College students
look upon knowledge and ideas less as ends in
themselves and more as a means toward
accomplishing other goals, such as success in
their careers Rather than measures of student
learning, indicators such as graduation rates,
employment rates, and cost of attendance are
the consumer information colleges and
universities should make readily available
(NASULGC, 2006).
Having those standards means
  • We can have substantive conversations with
    students, parents, employers, policy makers, and
    the public.
  • We have not just numbers (data) but specific,
    substantive, outcomes-based information to inform
    conversations across the educational spectrum,
    K-16 and beyond.

Larry King? Parade Magazine? Are you kidding?
No. Communication is essential. It will
  • respond to public need, interest
  • inform the public, build intellectual capital in
    the wider society and among policy makers
  • allow parents, high schools, others to
    internalize and reinforce our message

Communication, cont.
  • be educative for students, show them what they
    have mastered or still need to do
  • shift some responsibility for education to these
  • free us in HE from taking ALL the responsibility
    for a complex process that can never succeed if
    it is ours alone.

Why do this? To
  • improve education
  • provide useful information
  • pre-empt federal regulations
  • shape the debate
  • educate the public

Questions for discussion
  • How can your campus use assessment to set
    standards? What efforts can you build on?
  • How might commercial and local approaches be
  • What are the benefits/drawbacks of
  • Institutional or consortial standards?
  • State, regional, national standards?
  • Should we be doing this at all?
About PowerShow.com