Strategies for Implementing Reviews of Student Learning in a Decentralized Environment

1

• Strategies for Implementing Reviews of Student
  Learning in a Decentralized Environment
• Sharon A. La Voy
• Office of Institutional Research, Planning and
  Assessment

2
What Well Cover Today

• History of assessment at UM
• UM challenges
• Structure UM put in place
• Resources made available
• Development of process
• New campus context
• Successes and lessons learned

3
Previous Assessment at UM

• Specialized Accreditation
• Engineering ABET
• Education NCATE
• Business AACSB
• Campus Assessment Working Group (CAWG)
• Student satisfaction
• Describing student experiences

4
Data-Supported Decisions

• Program reviews
• Budget planning cycle
• Task forces
• Course evaluations

5
Middle States Developments

• PRR highlighted CAWGs efforts
• Teams evaluation encouraged UM to make progress
  on learning outcomes
• General lack of awareness of scope of this
  recommendation
• Middle States numbers Previous to new standard,
  20-30 further review post new standard, 70-80

6
What is the New Expectation?

• Standard 14 Assessment of Student Learning
• Assessment of student learning demonstrates that
  the institutions students have knowledge,
  skills, and competencies consistent with
  institutional goals and that students at
  graduation have achieved appropriate higher
  education goals.
• An accredited institution is

7
Characterized by

• Articulated expectations of student learning at
  various levels (institution, degree/program,
  course) that are consonant with the institutions
  mission
• A plan that describes student learning assessment
  activities being undertaken at the institution
• Evidence that assessment information is used to
  improve teaching and learning

8
Others Following Suit

• State pressures to conform to common standards
• Modifications of professional school standards
• Some at the University interested in telling our
  story in language other than input factors

9
Challenges to Overcome

• Few strong research university examples, for
  understandable reasons
• Threat of failed reaccreditation suspicious
• Size of the institution
• Common outcomes for Theater and Electrical
  Engineering?
• Decentralized culture

10
Many Tasks

• Institution-wide common outcomes
• And are these the same as our CORE general
  education program?
• Program-specific outcomes
• Course outcomes
• Assessment of all
• We began with what we could control

11
A Centrally Validated Structure
The Provosts Commission on Learning Outcomes
Assessment

• The Planning Team (two IR staff and four faculty
  administrators)
• The Deans Steering Committee
• The Faculty Working Group
• The College Coordinator Committee

12
The Deans Steering Committee

• Chaired by the Undergraduate Dean
• Deans of prominent colleges asked to serve
• We work out details with them
• Introduce decisions to Council of Deans and
  community with them already on board
• They nominated faculty to Working Group (we asked
  for strong faculty with vibrant research agendas)

13
The Faculty Working Group

• Chaired by the Undergraduate Dean
• Tasked with writing outcomes for Middle States
  Five Essential Elements of an undergraduate
  education (in addition to Gen Ed and disciplines)
• Met for a semester, reviewed other institutions
  goals, worked out language

14
Rolling Out to the Programs

• Could show progress at University level
• Provost distributed Faculty Working Group
  results, and said ALL programs must follow
• Back to Deans Steering Committee
• Process has to be owned in the Colleges
• I envisioned programs submitting centrally but
  that would not honor College control
• They appoint College Coordinator for each

15
College Coordinators

• Sharing of experiences
• Honest critique of all work
• Peer review
• Seminar-type discussions on issues
• Ground rules for privacy and sharing work

16
College Process

• Organize learning outcomes assessment process
  internally (College Coordinators and Deans)
• Consider whether there are common College-wide
  outcome goals
• Review department and program plans for
  consistency with College standards for quality
• Submit all assembled Assessment Plans under the
  signature of the Dean to the Provost

17
Resources Made Available

• Planning Team always available for presentations
  and consultations
• Registration for local (thankfully) Middle States
  assessment conference funded
• Learning outcomes workshops with nationally
  renowned speakers for Coordinators all faculty
  and staff invited to keynote addresses
• www.umd.edu/LearningOutcomes

18
(No Transcript)
19
Developments

• Program Plans submitted in Spring 2006
• 400 plans split up among teams of Coordinators
  and reviewed using rubric
• Coordinators provide written feedback instead of
  rubric results, deleting judgments
• Overall and program specific feedback sent to
  Deans from Provost

20
Developments

• Revisions to plans submitted in Fall
• Plans for assessments this academic year
• Deans Steering Committee Colleges decide how
  they do this within a 4 year cycle
• Results and projected curriculum changes
  submitted in March, after accreditation visit
• Reviewed by Coordinators, feedback given to
  colleges

21
Survey of Coordinators

• What was the most important experience you had in
  working with your college?
• The language of evaluation has changed in my
  college.
• There is a larger sense of a shared commitment to
  our students.
• There is a shared sense of the value of
  articulating the learning outcome goals.
• Faculty have been very cooperative.
• I have shared good ideas from other disciplines
  with my college.
• We can meet our own needs and the needs of
  accrediting agencies at the same time.

22
Survey of Coordinators

• What was most important to you in participating
  in the College Coordinator group?
• A well-directed and focused committee can get a
  lot done.
• A committed group can take on a challenging
  project, work hard, and succeed.
• Communication across campus with different
  disciplines increased my understanding.
• The assessment rubric for graduate programs that
  was distributed helps us a lot.

23
Survey of Coordinators

• What was most problematic for you in this
  process?
• Getting agreement on the assessment of graduate
  programs.
• Finding the time.
• Adding this to our workload.

24
Survey of Coordinators

• What is the overall result for you and your
  college?
• There have been significant new conversations
  about how to change teaching.
• Faculty have changed their syllabi.
• We wonder if the university will continue this
  process.

25
Successes and Milestones

• Utilizing groups to their utmost capacity
• College Coordinators act as community of scholars
• Reported brainwashing of some as they come to
  understand value of learning outcomes assessment
• CORE faculty committee saw benefits and
  established general education learning outcomes
  with little resistance

26
Examples of CORE outcomes

• Demonstrate critical analysis of arguments and
  evaluation of an arguments major assertions, its
  background assumptions, the evidence used to
  support its assertions, and its explanatory
  utility
• Understand and articulate the importance and
  influence of diversity within and among cultures
  and societies

27
CORE Assessment Milestones

• In 2005, faculty groups articulated student
  learning outcomes for all CORE categories.

• In 2006, faculty who teach CORE courses mapped
  their courses to published CORE outcomes by using
  checklists.

28
New Campus Context

• New programs
• New CORE courses
• Program Review
• Focus on graduate programs
• College-based development of assessment
  instruments and measures

29
Lessons Learned

• In one instance, not utilizing established
  structures for buy-in and gentle roll-out caused
  relative uproar
• Successes in one venue will influence others
• Giving up control of process is necessary
• Utilizing existing structures essential

30

• Questions, comments, and discussion welcome!