Begin With the Bar Chart in Mind

1
Begin With the Bar Chart in Mind

• Assessment at
• Fitchburg State College

2
Misconceptions

• Fiction Assessment focuses on the negative.
• Fact Assessment focuses on all pertinent facts,
  but primarily on the positive.
• Fiction Assessment involves telling stories out
  of school that may make units less competitive
  for funding and other resources.
• Fact Assessment data can help build cases for
  more and better resources.

3
The basic question

• If you were telling the story of an academic unit
  (e.g., department, program, major) to someone
  outside that unit, what might count as evidence
  that completers are, for the most part,
  succeeding at what you hoped they would when they
  entered the unit?

4
People outside the unit

• Legislators
• Accreditors
• Outside evaluators and other colleagues
• Prospective students
• Current students
• Parents and the general public
• Grantors and other external funding sources

5
Types of evidence

• Indirect
• Provider-side books written, conferences
  attended, classes taught, grades awarded
• Consumer-side surveys and other self-reports
  (most) standardized tests
• Direct
• Student learning products, e.g., papers, lab
  reports, art exhibitions, play performances,
  etc., evaluated per transparent, credible
  standards.
• Employer/supervisor field reports
• Direct is best
• Increasing attention to collecting and reporting
  it since A Nation at Risk in 1983.

6
Types of usable data

• Ambient
• Survey results that might be disaggregated for
  your unit (e.g., NSSE, alumni surveys) test
  score results (GRE, MCAT, etc.) that may reflect
  on success in your unit etc.
• Unit-generated
• Classroom-embedded assessment of student
  assignments course/departmental portfolios
  departmental surveys, etc.

7
Assessment data

• Systematically collected and shaped to tell a
  particular story
• Always reflects good news
• Were doing great and heres the proof, or
• We havent been doing so great, but we spotted
  the problem and heres what were doing about it.
• Identify the question(s) you need answered, then
  design a system that answers the question(s)
  economically.

8
(No Transcript)
9
(No Transcript)
10
Objectives

• Must be observable to be measurable.
• Builds a ham radio in a laboratory situation
  using only a spoon, a diamond, and a steel pipe
  is NOT THE SAME AS possesses the ability to
  build a ham radio etc.
• Should reflect application of disciplinary
  content knowledge.
• If I pour a concrete walkway, I probably KNOW how
  to do it. But I can know how to pour concrete
  without necessarily being able to do it.

11
Reporting assessment results

• Use assessment data.
• Explore significant questions raised by your
  data. (E.g., are freshman and sophomore
  Communication scores close because students were
  taking the same lower-division classes? Yes, in
  this example.)
• Redesign the system or add layers, as results
  dictate.
• Record and assess your actions.

12
Questions?