Assessing Undergraduate Information and Technology Skills: Opportunities, Outcomes and Challenges Ar

1
Assessing Undergraduate Information and
Technology Skills Opportunities, Outcomes and
ChallengesArizona State University West

• Bee Gallegos
• Dennis Isbell
• Lisa Kammerlocher

2
Assessment Opportunities

• North Central Accreditation imperative to
  demonstrate learning outcomes
• AZ Board of Regents approves lower-division
  offerings
• Previous self-assessment pilot for information
  and technology skills entrée to developing
  performance-based assessment

3
Project Goals

• create a reliable web-based Information and
  Technology Skills Assessment (ITSA)
• pilot ITSA and its administration to determine
  if it would be manageable and sustainable as an
  ongoing program
• conduct pre and post-assessments of the
  first-year cohort, with the pre-assessment given
  early in the Fall 2001, and the post-assessment
  at the end of their academic year in the Spring
  2002
• establish a baseline for incoming first-year
  student skills to use in future comparisons
• determine if a commonly delivered library
  curriculum in UNI 101 and English (ENG) 102
  results in an increased level of information
  competency for first-year students
• determine if gender or race/ethnicity is a factor
  in the level of information competency gained by
  first-year students

4
Library Curriculum

• Virtual and Experiential Tour Assignment
• using services reference and circulation
• using collections reference, media, journals,
  and books
• using the catalog, a database
• Plagiarism Unit

5
Development of the ITSA

• Based on ASU West Information Competencies
• Used models from James Madison University(4),
  University of California Los Angeles(5), and
  California State Universities(6)
• Tested with library staff and students
• Engaged assistance from database specialist to
  design web-based interface
• Sought research expertise from colleagues across
  campus

6
Attributes Measured

• Recognizing catalog and database structures and
  records
• Identifying citations and attributing authorship
• Evaluating information sources
• Recognizing and applying search techniques
• Recognizing effective research plans and search
  strategies

7
Administration of the ITSA

• Negotiated access to UNI 101 courses for pre-
  assessment in early fall semester
• Attempted to assess whole population (with no
  control group) as all students were enrolled in
  UNI 101
• Negotiated access to ENG 102 courses for
  post-assessment in late spring semester

8
Demographics
(Arizona State University West, Institutional
Planning and Research, 2002)
9
a p lt .001, at 95 confidence level d p lt .001,
at 95 confidence level b p lt .019, at 95
confidence level e p lt .007, at 95 confidence
level c p lt .069, at 95 confidence level f p lt
.006, at 95 confidence level
10
Results Figure 1
11
Results Figure 2
12
Discussion

• Information competency improved overall and in 4
  of the 5 attributes
• Suggests emphasis on information competency in
  curriculum influenced improved student
  performance (2)
• Females increase in level of information
  competency was significant
• Gender gap in education is shrinking (3)
• Males increase in level of information competency
  not significant
• Sample was too small to be conclusive
• Ethnicity was not a factor in performance on the
  ITSA

13
Future Directions

• Continue to improve reliability of the ITSA
• Modify learning experiences in the attributes of
  citation and attribution, search techniques and
  research plans and search strategies
• Seek improved opportunities for administration of
  the ITSA
• Revisit how the ITSA will be used pre-post vs.
  measure of competency at various academic levels
• Establish score that defines information
  competency (4)

14
Bibliography

• Arizona State University West, Institutional
  Planning and Research. (2002, Feb.) Student
  Characteristics Summary, Fall - 2001.
• Arizona State University West. (2002). College
  Student Experiences Questionnaire, Cohort01-02,
  Posttest. data file
• Campbell, J.R., Hombo, C.M. Mazzeo, J. (2002).
  NAEP 1999 Trends in Academic Progress Three
  Decades of Student Performance. Washington,
  D.C. U.S. Department of Education. Office of
  Educational Research and Improvement. National
  Center for Education Statistics. (National
  Center for Education Statistics Report No. NCES
  2000-469).
• Cameron, L. Feind, R. (2001, March 15-18). An
  Online competency test for information literacy
  Development, implementation, and results. Paper
  presented at the Association of College
  Research Libraries Tenth National Conference,
  Denver, CO.
• Carvello, P.S., Herschman, J. Mitchell, E.
  (2000, March 15-18). Assessing the Information
  Literacy of undergraduates Reports from the UCLA
  Librarys information competencies survey
  project. Paper presented at the Association of
  College Research Libraries Tenth National
  Conference, Denver, CO.
• Dunn, K. (2002, Jan-March). Assessing information
  literacy skills in the California State
  University a progress report. Journal of
  Academic Librarianship, 28(1), 26-35.

15
Presentation Web Page

• http//westcgi.west.asu.edu/libcontrib/northumbria
  03
• Literature Review and Bibliography
• Access to Information and Technology Skills
  Assessment
• First-Year Library Assignments
• Information Competencies, Attributes and Fletcher
  Menu
• PowerPoint Presentation
• Contact Information