RealLife Experiences with PreUpgrade Database Cleanup Projects

1
Real-Life Experiences with Pre-Upgrade
Database Cleanup Projects

• John Whisler
• Eastern Illinois University
• March 20, 2006

2
Everything There Is to Knowabout pre-upgrade
database cleanup

• Is in two documents in your packets
• Processing the Output Logs (newly revised)
• Correcting Non-Filing Indicators
• Detailed background and explanations
• Detailed instructions and suggestions
• Sources for further information

3
But Wait, Theres More

• Not as difficult as they sound
• Potential benefits independent of upgrade
• Doing them now will save time and trouble after
  upgrade
• But timing is up to you

4
Processing the Output Logs

• I had about 200 errors from a db of about 600,000
  bibs
• I skipped Excel formatting
• I skipped much of the interpretation of codes,
  found errors by eye
• Finished the project in one afternoon last
  December

5
How I Cleaned Up

• By importing fresh bibs from OCLC
• Sometimes problem is still present in OCLC
• Used mostly for Leader 9 is not space
• By replacing authority controlled fields by
  copying from authority file
• By editing in Voyager Cataloging Client

6
Two More Chances

• CARLI is planning another pre-upgrade test
  conversion, with new log files to examine
• Log files will be created during the production
  conversion for post-conversion cleanup

7
Correcting Non-Filing Indicators

• There are two problem scenarios
• Indicator does not say to skip something it
  should
• 245 10 The problem of initial articles
• Indicator says to skip something it shouldnt
• 245 14 Indicators gone bad lost bibs

8
Correcting Non-Filing Indicators

• First scenario is findable and correctible in the
  Voyager Cataloging Client
• Title search on the, etc.
• MS Access queries identify problems of the second
  type
• Unicode diacritic handling requires special
  attention
• Lots of just plain wrong indicators in my db

9

• From OCLC BFS
• 245 1 0 War of the worlds
• 245 0 0 Diary
• 245 1 0 --as others see us
• 245 0 0 A to Z drug facts
• 245 0 4 The year book of medicine
• 245 0 5 The part of Pennsylvania
  that ... townships
• 245 1 2 A report to the legislature for the
  year .... 
• 245 1 2 L'enfant criminal
• 245 12 L'ete
• 245 1 3 An A--Z of employment law
• 245 1 5 The "winter mind"
• 245 1 5 "The eve that never sleeps ... "
• 245 1 6 --the serpent--snapping eye
• 245 1 8 The ... annual report to the Governor

10
How Long Did This Take?

• For me, not as long as the paper suggests
• Ran scans on 245 field in batches of 100,000 bib
  IDs at a time
• Started scans when faculty meetings began, done
  when I got out (ca. 2 hrs.)
• Tags with fewer bibs ran even faster
• Processing results generally easy but
  time-consuming

11
How Many?
12
Project is Repeatable

• Queries can be run multiple times
• Can be run after upgrade
• Can become a part of regular bibliographic
  maintenance routines