Ada Emmett and Judith Emde University of Kansas Libraries

1
Ada Emmett and Judith EmdeUniversity of Kansas
Libraries

• Assessing Chemical Information Literacy Skills
• using the ACRL Standards

ALA 2006 Annual Meeting, New OrleansSTS Research
Forum Sunday June 25th
2
Overview

• Chemistry bibliography course
• Assessment tool development to measure learning
  outcomes
• Assessment results and observations
• ACRL standards advantages

3
Bibliography of Chemistry CHEM 720

• One hour credit offered for graduate students
• Major chemistry and biomedical research tools
• Grading satisfactory/unsatisfactory

4
Questions

• What should students learn?
• Does teaching produce the desired learning
  outcomes?
• How can we assess student learning?

5
ACRLs Information Literacy Competency Standards
for Higher Education

• Used to develop
• Learning outcomes
• Teaching activities
• Assessment tool
• An information literate student

6
ACRLs Information Literacy Competency Standards
for Higher Education

• Alternatives to ACRL literacy standards
• ACS
• STS

7
ACRLs Information Literacy Competency Standards
for Higher Education

• 1. Determine the extent of information needed
• 2. Access the needed information effectively and
  efficiently
• 3. Evaluate information and its sources
  critically and incorporate selected information
  into his or her knowledge base and value system.
• 4. Use information effectively to accomplish a
  specific purpose
• 5. Understand the economic, legal, and social
  issues surrounding the use of information, and
  access and use information ethically and legally
  
• http//www.ala.org/ala/acrl/acrlstandards/informat
  ionliteracycompetency.htm

8
Year One 2004

• Methodology
• Developed assessment tool
• Conducted pre- and post-course assessment
  interviews
• Used control group
• Quantified data

9
Year One 2004

• Results

10
Year One 2004
Possible
Possible
Post-
Post-
Pre-
Pre-
11
Year Two 2005

• Assessment tool redesign and refinement
• SPSS Data Entry Builder to enter data
• One-on-one interviews
• Pre- and post-test given
• students 16
• small control group 5

12
Year Two 2005

• Results

13
Year Two 2005

• Results

14
Year Three 2006

• Used backward design starting with ACRL
  learning outcomes
• Develop assessment questions
• Design course lectures/assignments
• Create tool web-based questionnaire
• Run as pre- and post-test
• No controls

15
Year Three 2006

• Results
• Points Improved 27
• Percent Improved 57

Possible
Post-
Pre-
16
Comparison by Assessment Questions
17
2004-2006 Improvement

• 2004 44 improvement (control 16)
• 2005 62 improvement (control 17)
• 2006 57 improvement

18
Assessment questions grouped by standards

• Standard 2
• The information literate student accesses needed
  information effectively and efficiently.
• Performance indicator 5
• The information literate student extracts,
  records, and manages the information and its
  sources.
• Outcome e
• Uses various technologies to manage the
  information selected and organized.
• Assessment questions
• Select among the following tool(s) that manage
  references or citations and therefore assist in
  writing research papers.
• Describe two features of a software program that
  assist in managing citations.

19
Assessment questions grouped by standards
20
Students perceptions
21
Observations

• Mapping standards is subjective
• Consultation on mapping questions to standards
• Some standards such as 3 and 4 are difficult to
  assess
• Strong subject expertise needed
  (faculty/instructor collaboration)
• Mechanism of delivering assessment tool needs
  improvement
• Ongoing review of assessment tools
  strengths/weaknesses

22
Observations

• Assessing all learning outcomes difficult
• Keep it simple to several specific outcomes.
• Did the class and the test incorporate most
  important learning outcomes for that student
  group?
• Faculty/instructor collaboration

23
ACRL standards advantages

• Provides mechanism to assess information literacy
  skills via learning outcomes (using backward
  design)
• Assists in the development of course content
  through backward design

24

• Discussion, Questions and Comments

25
Contact information

• Ada Emmett
• aemmett_at_ku.edu
• 785-864-8831
• Judith Emde
• jemde_at_ku.edu
• 785-864-4931
• Supplemental information at http//www.people.ku.e
  du/jemde/

26
(No Transcript)
27
Mapping lectures / exercises to standards

• Potential learning outcome of lecture and
  exercise mapped to standards at beginning of
  semester
• Comparison of lectures/exercises to results of
  post assessment by grouped standards.

28
Mapping lectures / exercises to standards