Title: PowerPoint Presentation From Compliance to Improvement: Accountability and Assessment for California
1From Compliance to Improvement Accountability
and Assessment for Californias Community
Colleges
Higher Education Evaluation and Research
Group Spring, 2005
PLEASE COMPLETE THE PRE-SURVEY
2.15.05
2Introduction Choosing Improvement over
Compliance
3ComplianceImprovement
Compliance
Improvement
4ImprovementCompliance
Compliance
Improvement
5Saying YES to Assessment and Accountability
acknowledges community colleges appropriate
roles in equity, upgrade training, lifelong
learning, and other unconventional missions
gives faculty an appropriate voice in running
their institutions promotes a form of research
in teaching and the creation of improvements in
teaching provides a foundation for widespread
institutional improvement become more
effective learning environments
6Accountability is NOT new in CA
- California Community Colleges operate
- under at least four accountability systems
- PFE uses system level goals
- State Report Card assesses performance of all
publicly funded workforce preparation programs - Federal Vocational and Technical Education Act
- Workforce Investment Act
7 and now WASC
- WASC has been the last regional accreditation
commission to require colleges to develop
mechanisms of assessment and use of student
learning outcomes - WASC lets LOCALITIES choose which aspects of
SLOs to measure and how to measure them
8Well developed system of internal accountability
Ability to respond to external accountability
requirements
9Module I
Taking Stock
10(No Transcript)
11(No Transcript)
12(No Transcript)
13THE CRUCIBLE OF THE CLASSROOM
14?
Students
What do you know about your students attitudes
and beliefs about learning?
15What Weve Learned About Student Values and
Attitudes
- Students are credentialists, wanting credit/
credentials but not necessarily the learning the
credential signifies (grades matter more than
content) - Students are highly vocationalist, using college
as a route to employment (relevancy matters more
than intellectualism students continuously make
cost-benefit calculations)
16What Weve Learned (contd)
- Students constantly undermine learning outcomes.
They are often fearful, afraid of being caught
unprepared, isolated, intimidated by professors,
and they manage their fear in unproductive ways
(keeping quiet in class, avoiding hard classes,
scaling down ambitions, failing to submit work
even when its completed, dropping or stopping
out) - Students define learning as the accumulation of
facts
17Activity Students 1
- Into what key groups do you subcategorize your
students? - How do you identify the changing needs of your
student groups? - What are your students beliefs and values about
learning? - How do you know? What is the forum in which you
discuss categories, changing needs, and attitudes
of students?
18?
Instructors
What do you know about instructors attitudes,
beliefs and knowledge about teaching and
learning? Is teaching community property?
19ACTIVITY INSTRUCTORS 1
- What are faculty attitudes and knowledge about
learning, teaching, assessment, teaching as
community property, and continuous improvement? - Is there a forum for discussing examples and
reasons for student success or lack of success,
teaching ideas and methods? - How have faculty previously developed, shared and
implemented student learning outcomes and
assessments? -
20?
Curriculum
What are external influences on curriculum?
What is the consistency of curriculum? The
role of employer feedback
21ACTIVITY CURRICULUM 1
- Which of your critical curriculum is set by
external agencies? - What is the consistency in expectations across
sections of a course? - When/ where do instructors norm content and
assessment? - Do instructors collaborate on and/or do peer
review of learning outcomes for critical courses?
22- TAKING STOCK INSTITUTIONAL
?
Institutional Support
What regulations impact student learning
outcomes? How do local practices and policies
impact student learning outcomes and assessment?
23ACTIVITY INSTITUTION 1
- What federal, state, district, and/or
professional-trade-industry regulations impact
student learning outcomes, their assessment, and
improvement on your campus? - Local practice and policies (faculty time
professional development policies hiring,
promotion, tenure policies teaching credential
24- TAKING STOCK CONSISTENCY/ ALIGNMENT
?
Alignment/ Consistency
How consistent are facultys expectations of
student outcomes across sections of a course,
courses, general education alternatives,
certificates, and degrees? What are forums for
discussing expectations?
25ACTIVITY CONSISTENCY 1
- What is the consistency between the values of
students and instructors? - What is consistency between instructors use of
curriculum and her/his own values and beliefs
about teaching and learning? - What is articulation among sections of a course
and/or courses in a sequence?
26- TAKING STOCK
- EXISTING ASSESSMENTS
?
Existing Assessments
What assessments are in place now (placement
tests, capstone projects, portfolios,
paper-pencil tests, etc.?) How do those
assessments contribute to improving student
learning outcomes?
27ACTIVITY ASSESSMENTS 1
28- TAKING STOCK
- GOVERNANCE of ASSESSMENT
?
Governance
Who coordinates student learning outcomes,
assessments, improvement strategies and
continuous improvement?
29TAKING STOCK OF Assessment Governance
- If assessment is to be continuous, on-going and
stable, then it must be overseen by a group that
takes responsibility for all aspects of
assessments.
30In a self-reforming institution focused on
instruction, the Assessment Committee would be
the central committee in a college, so that
concern over the nature and effectiveness of
instruction drives all other aspects of a
college. In this way, the Assessment Committee
should have responsibility not only for creating
a series of assessments but also for overseeing
the subsequent stages in the assessment system.
31ACTIVITY GOVERNANCE 1Student Assessment
Leadership Team
32Reporting Out
- When we take stock
- student attitudes beliefs instructor
attitudes knowledge consistency of pedagogy
curriculum local practices policies existing
assessments local governance of SLOACs, - we believe we have strength from _____ and we
want to build __________.
33A Primer
- Setting and Assessing Student Learning Outcomes
34Definition SLO
- Robust student learning outcomes incorporate
- behavioral objective what a student should
know, value and be able to demonstrate/ perform - conditions under which performance will be
assessed simulation, lab, portfolio, writing
task - Criteria/ performance standards/ primary traits
for assessing student performance - Rubric for scoring student performance
1Adapted from Scroggins, B. (2003, 2004).
Targeting Student Learning. Modesto Junior
College. lthttpcai.cc.ca.us/workshops/SLOFocusOn
Results.docgt
35Confusion Terminology??
- Course Objectives
- versus
- Student Learning Outcomes
- Generally, in California, course objective states
what student will demonstrate, represent or
produce at end of course. - SLO also incorporates the conditions under which
assessment will occur (test, portfolio,
demonstration, etc) as well as evidence/criteria
36slo SLOA SLOAC
- Among departmental/ institutional faculty, NORM
- - objectives for student performance (what
should students know, be able to do, value) - - conditions under which performance will be
assessed (simulation, portfolio, lab experiment,
writing assignment) - - traits/ criteria for assessing student
performance and rubrics for scoring - Implement assessment plan
- Compile and analyze pattern of results from
scoring student performance - COLLECTIVELY SET IMPROVEMENT PLAN
- IMPLEMENT IMPROVEMENT PLAN
- CONTINUE CYCLE
37Norming1
- Nested discussions, decisions and actions
- Collaboratively authored and collectively
accepted expectations for student learning and
assessment - Norming does NOT mean identical learning
activities, emphases, pedagogy it means CC
1Maki, P.L. (2004). Assessing for Learning.
American Association for Higher Education,
Sterling, VA Stylus.
38Norming Higher Ed Culture
- SLOs, criteria/ primary traits, rubrics are set
collaboratively with full and adjunct professors - Outcomes/ examples of student work are shared and
peer-reviewed - Improvement alternatives are agreed upon
- Autonomy/ Academic freedom/ Professional
discretion and expertise
39Reliability Inter-rater or test-retest
Validity Instrument/ procedure measures what
it is intended to measure
T H E
B A S I C S
40Types of Data
- Quantitative
- Qualitative
- SYSTEMATIC
41References Norm Criterion
- Norm-referenced assessments measure individual
outcomes relative to the sample of people taking
the test- grading on curve - Criterion-referenced assessments measure
individual outcomes compared to certain norms or
criteria -mastery, licensure - Criterion-referenced assessments are
appropriate for measuring improvement in SLOs.
42Direct vs. Indirect Measures
- Direct measures are reasonable replications of
real world tasks authentic assessment- DO IT - Indirect measures are proxies for demonstrated
performance grades, persistence, transfer
(legislated measures are often proxies)
43External vs. Internal Accountability
- External accountability is used to meet
requirements of funding/ regulatory agencies - Internal accountability is used to improve
student learning within courses, programs or
degrees.
44Well developed cycle of internal
accountability Ability to respond to external
accountability requirements
45A S S E S S M E N T
F O R M S O F
46Assessments
- Capstone projects
- Demonstration
- Simulations
- Portfolios
AUTHENTIC ASSESSMENTS
- Criterion-referenced tests (licensure exams)
- Norm-referenced tests (curve)
47Embedding Assessments
- Assessment is woven into existing courses
- Identify SLO demonstration points
- Retain and analyze results
- SLOs are aligned with certificate or degree goals
48EXAMPLE Embedding Assessments in a Program
49Levels at which Outcomes Can be Measured
- Targeted population of students
- Lesson/ unit of study
- Program
- Occupational certificate
- Major
- Department/ Division
- Associate degree (A.A./A.S./A.A.S)
- Institutional
50Targeted Populations
- For VTEA and some special grants, a college may
wish to focus on retention, persistence and/ or
achievement for special populations of students - CalWorks
- First generation
- Limited English proficient
51Lesson/ Unit-Level Assessment
- Classroom assessment techniques
- (Cross Angelo)
- Systematic but informal, frequent gathering of
information about content and pedagogy - What was hard to understand today?
- How did this teaching method work for you?
52Course Level SLOs
- Most campuses are emphasizing course level
assessments as part of program level SLOACs.1
1 Friedlander, J. Serban, A. (2004). Meeting
the Challenges of Assessing Student Learning
Outcomes, in Friedlander, J. Serban, A.
Ed.Developing and Implementing Assessment of
Student Learning Outcomes. New Directions for
Community Colleges No. 126. San Francisco, Jossey
Bass.
53Program and Institutional Level SLOs
- Courses are aligned to meet program goals and
expectations. - Program/ major/ or general education goals are
aligned to meet institutional goals and
expectations.
54Norena Norton Badway, Ph.D. Principal
Phone 209-951-7477 home office 209-946-2168
University office 209-601-7121 Email
badway_at_aol.com nbadway_at_pacific.edu