Title: Library Instruction Assessment Project Spring 2003 Patty Cloud ReferenceCoordinator of Instructional
1Library Instruction Assessment ProjectSpring
2003Patty CloudReference/Coordinator of
Instructional Programming
2Project Overview
- Number of students who participated 175
- Number of classes participating 13
- 5- item pre-test, 5 item post-test
- Questions taken from TILT (Texas Information
Literacy Tutorial)
3Classes Participating/Spring 2003
- BGS
- BGS 201-L12 Pro Seminar in Critical Skills
- BGS 201-10 Pro Seminar in Critical Skills
- BGS 302-10 Methods of Critical Reasoning
- UNDERGRADUATE
- BIO 150 Science as a way of knowing
- ENG 102-05 Argument, analysis and research
- HIST 111 The world to 1500
- LIBS 201B-17 Writing social justice social
sciences - POS 340 The Presidency
- GRADUATE
- PADM 406-10 Research and evaluation methods
- PSYC 389 Seminar Personality research
- PROFESSIONAL
- TRDV 434-17 Methods of research for trainers
- CHS 425-10 Research Current Counseling Issues
- HOSM 290 Methods and Research in Hospitality
Management
4Pre-test and post-test
- Two quizzes were made available on
http//library.roosevelt.edu/instruction/ in php
format - Results were written to mySql database on our
remote UNIX server - Results were downloaded from remote server and
analyzed using MS Access
5Test content
- Test content was taken directly from the quizzes
used in the TILT (Texas Information Literacy
Tutorial) which is being used at RU - No direct comparison between test results for
this project and results saved to the TILT
database can be made as TILT is currently set up
comparison will be possible with rollout of the
revised version of TILT (scheduled Fall 2003)
6Project goals
- Gain experience in administration of assessment
instrument - Gauge impact of administration on the instruction
experience - Prepare for possible adoption of national
standards based assessment tools (e.g. SAILS) - Gather quantitative data on learning outcomes
7Gain experience in administration of assessment
instrument
- Three librarians participated
- Tests had to be readily available on browsers
- Instructions can be improved / some results were
lost - Tests should be refined / javascript validators
should be used - A variety of problems with the post test results
(data capture, id matches, question wording)
8Gauge impact of administration on instruction
experience
- Administration required changing instruction
patterns use of workstations, distribution of
materials, taking 5 minutes at start and end of
session - High level of faculty consultation required
- Test content influences instruction content
- Faculty response was universally positive and
interested - Changing process contributes to stress
9Prepare for possible adoption of national
standards based assessment tools (e.g. SAILS)
- National or proprietary systems will require web
administration, database connectivity, reporting
abilities, as this project has - All the process changes weve experienced with
this project will probably be required if we
adopt a national or proprietary assessment
10Gather quantitative data on learning outcomes
- No real useable data from the post-test
- Pretest results allow us to look at the level of
preparation our students bring to the instruction
session, across programs, schools and classes
11Question 1
- Question One Imagine you have an assignment to
write a paper based on scholarly information.
Which would be the most appropriate sources to
use? (Choose two.) - journalmagazinenewspaperperiodical
indexweb
12Question 2
- Question Two Why would you use a periodical
index? - To check your emailTo surf the webTo find
citations to magazines or journalsTo see which
magazines your library owns
13Question 3
- Question Three How can you tell you are looking
at a popular magazine? (Choose two.) - The articles are written for the general
public - The articles are in-depth and usually include
a bibliographyThey include lots of
photographsThere are few if any
advertisementsThey are usually published
quarterly (4 times a year)
14Question 4
- Question Four Some magazines contain in-depth
articles which may be good sources for research
papers. - TrueFalse
15Question 5
- Question Five Who publishes information on the
Web? (Choose all that apply.) - StudentsFacultyPeople in foreign
countriesLibrariesGovernment
agenciesUniversitiesCompaniesNon-profit
organizations
16Distribution of scores across all
Y axis is the frequency of the score
For example, the POS 340 class had 9 people who
scored 90 or above on the test
The x axis is total percent correct
17For 3 BGS Classes
18DistributionUndergraduate classes
19Distribution for professional programs
20Graduate classes
21Comparison of the shape of score distribution
across programs
Undergraduate
BGS
Graduate
Professional
22Classes ranked by average score
Highest average scores POS 340, HIST 111, BIO 150
Low average scores BGS, ENG 102
In the middle graduate, professional and LIBS 201
23Analysis by question
- Question One Imagine you have an assignment to
write a paper based on scholarly information.
Which would be the most appropriate sources to
use? (Choose two.) - journalmagazinenewspaperperiodical
indexweb
24Question 2
- Question Two Why would you use a periodical
index? -
-
- To check your emailTo surf the webTo find
citations to magazines or journalsTo see which - magazines your library owns
25Question 3
- Question Three How can you tell you are looking
at a popular magazine? (Choose two.) - The articles are written for the general
publicThe articles are in-depth and usually
include a bibliographyThey include lots of
photographsThere are few if any
advertisementsThey are usually published
quarterly (4 times a year)
26Question 4
- Question Four Some magazines contain in-depth
articles which may be good sources for research
papers. - TrueFalse
27Question 5
- Question Five Who publishes information on the
Web? (Choose all that apply.) - StudentsFacultyPeople in foreign
countriesLibrariesGovernment
agenciesUniversitiesCompaniesNon-profit
organizations
28Questions 1-5
29Some observations
- A significant majority of students across all
programs scored 70 or better on the pretest - Comparative numbers of low scorers in both the
BGS and undergraduate classes may be the result
of sample size, though BGS appears to be
bi-modal - Shape of distribution of scores is roughly
similar across programs
30Next steps
- Refine the instrument
- Evaluate influence of test content on instruction
content - Get automatic capture of quiz data from
TILT.Version2 - Consider carefully in anticipation of a wider
rollout sometime in the (perhaps not too distant)
future