????????????: ???????? Recent Developments of Library Evaluation in China: Research, Practice and Directions

1
????????????????????Recent Developments of
Library Evaluation in China Research, Practice
and Directions

• ???
• ?????????
• Guangqin Zhang Ph.D, Associate Prof.
• Dept. of Information Management, Peking
  University
• zgq_at_pku.edu.cn

2
Term and Type

• ?????? What words to choose?
• ?????20??20??,???60??beginning at 1920s,
  developed in 1960s
• ?????????????????????????????????????
• library (performance) evaluation, assess,
  measurement, auditing, etc.

3
Term and Type

• ???? Type of Lib. Evaluation
• ????????????????
• by coverage/aim/method survey, standard
• ????????????????
• auditing reports external auditing and
  internal auditing
• ?????????
• manage statistics and performance evaluation
• ???? evaluation research

4
Documentary Statistics Research
5
Documentary Statistics Research
6
Documentary Statistics Research

•   ?????
  ??? 
• JOURNAL OF THE MEDICAL LIBRARY ASSOCIATION
  
   44 
• ELECTRONIC LIBRARY
  
   33 
• JOURNAL OF ACADEMIC LIBRARIANSHIP
  
   30 
• COLLEGE RESEARCH LIBRARIES
  
    26 
• JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION
  SCIENCE AND TECHNOLOGY 23 
• LIBRARY TRENDS  22 
• PORTAL-LIBRARIES AND THE ACADEMY  22 
• LIBRARY INFORMATION SCIENCE RESEARCH  20 
• HEALTH INFORMATION AND LIBRARIES JOURNAL  17 
• PROGRAM-ELECTRONIC LIBRARY AND INFORMATION
  SYSTEMS  16 
• INFORMATION PROCESSING MANAGEMENT  13 
• JOURNAL OF DOCUMENTATION  13 
• LIBRARY HI TECH  13 
• ASLIB PROCEEDINGS  12 
• INFORMATION TECHNOLOGY AND LIBRARIES  12 
• JOURNAL OF LIBRARIANSHIP AND INFORMATION
  SCIENCE  12 
• ONLINE INFORMATION REVIEW  11 
• INTERLENDING DOCUMENT SUPPLY  10 

7
Documentary Statistics Research
???? ??? ??
????????? Review 4
????????? Practice Review ?? ?? 1 1
????????? Theory Study Review ???? ?? ???? 1 1 1
????????? Evaluation Methods Study AHP? StatsQual LibQual ????? ???? ??????? ????? ????? ???????? 4 1 1 1 1 1 1 1 1
??????? Digital Library Evaluation ??????????? 7
???????? Foreign Evaluation Methods Introductory ?? ?? ???? ???? 3 1 1 1
??????? Organization and Operation ????(?????) ?1
????????? Indicator System Study ????(??????????????) ???? ?????? ???? ???? 31 6 2 2 1
???????????????? Practice Summary, Re-thinking on Evaluation Practice ????? ???? ???? ???? ???? ?? ?? ?? ?? ??? ??? ?????? ?????? ???? ???? ??????? 6 31 11 10 7 1 1 1 1 2 1 11 4 5 1 1
??? News Report 9
8
Documentary Statistics Research

• 2005??????????????????????
• Journals issued the paper on Lib. Evaluation and
  percentage

9
?? Definition

• ????????????????????????????????????????????????
  ???????????????????????????????
• There are 3 elements in this concept (1) Basic
  standards and goals must be included in this
  concept (2) a basic concept judged by value (3)
  to describe the process and result of library
  work based on facts and statistical data.
• ?????????????????????(ISO11620)??????????????
  ???????????????????????,????????????????(Effective
  ness)??????????????????(Efficiency)???????????????
  ??????????????????????????????????

10
?? Definition

• ???????????????????(1)????????????????????????,?
  ???????????????????(2)?????,??????????,?????????
  ???????????
• Two ways are used by defining the term (1)
  Comparing to the standards view point Lib.
  Evaluation is to collect data, information,
  compared to the goal and ideal standards. (2)
  Systematical View by considering a library as a
  system, it is measured by profits through
  comparing input with output.

11
??????Why Evaluate?

• Wallance and Van Fleet (2001)
• Account for how they use their limited resources
• Explain what they do
• Enhance their visibility
• Describe their impact
• Increase efficiency
• Avoid errors
• Support planning activities
• Express concern for their public
• Support decision making
• Strengthen their political position

• Weiss (1988)
• Determining how clients are faring
• Providing legitimacy for decisions
• Fulfilling grant requirements
• Making midcourse corrections in programs
• Making decisions to continue or culminate
  programs
• Testing new ideas
• Choosing the best alternatives
• Recording program history
• Providing feedback to staff
• Highlighting goals

12
???? What is good evaluation?

• Jose-Marie Griffiths?Don King ??????????????????
  ??
• ???????,??????????????????????,???????????????????
  
• Evaluation has its aims, and the result is not
  the ending flag of an assessment, but obtaining
  improvement is the ending flag.
• ???????????????????????????,?????????????
• It is no any significance unless the evaluation
  result could help libraries improve their
  weakness and resolve their problems found during
  the evaluation.
• ????????????????????,?????????????????????????????
  ???????
• Evaluation should deeply discover the tight
  relationship between libraries and their users
  and communities served by the libraries, besides
  description to facts, activities.
• ?????????????????????,????????????????????????????
  ??????
• Beyond the library internal work flows and its
  social impact, the evaluation results should also
  discover that information from the assessment
  could make librarians communicate each other and
  between librarians and users.

13
???? What is good evaluation?

• Jose-Marie Griffiths?Don King ??????????????????
  ??
• ??????????????,???????????????????????????????????
  ???????????????????????????
• Evaluation should be one continuing course, but
  not an occasional event just happening within a
  certain period. Continuing evaluation could have
  impact on libraries with monitoring the library
  process, analyzing the questions occurring from
  the activities, and improving management and
  service levels.
• ????????????????????????????????????
• Evaluation should become one tool which help
  libraries monitor activities, diagnose problems,
  and implement changes.
• ???????????,?????????????????????????
• Evaluation should be a dynamic process, and it
  should reflect the impact on libraries by new
  ideas and new environment.

14
?????? Model

• ???????????????????????,??????????????????????????
  ?????
• Evaluation models are used to assist in our
  understanding of the functions, methods, goals,
  relations among all kinds of models and services
  provided by a library.
• ????????
• Stage of importing concept in China
• ?????????????????????????????????,???????????????
  ?????????????????????PDCA??????????????,????????
• PDCA Plan, Do, Check, Act
• Foreign researchers developed there kinds of
  assessment models(1) model regarding a library
  as a whole unit
• (2) model concerning one course of
  libraries activities or certain evaluation
  model
• (3) PDCA model describing the general
  evaluation course.

15
?????? Model

• Blaise Cronin Model

• Scot Nicholson Model

User Management Sponsor
Cost
Effectiveness
Benefits
16
?????? Model

• Richard Orr Model

• John C. Bertot and Charles R. McClure

17
?????? Model

• Integrated Model

• Orrs model extension
• The Performance Spectrum

18
???????Evaluation Methods and Tools

• ???????????????????????????,????????????????????
  ??????????????CIPP???????????(Data Envelopment
  Analysis)?????????????????????,?????????????????
  ???????????????????
• Two kinds of types are presented by the
  evaluation one is focused on the whole library
  effectiveness and efficiency, such as input
  measurement, standards, task measurement,
  comprehensive measurement, systematic
  measurement, CIPP measurement(Context, Input,
  Process, Product), Data Envelopment Analysis the
  other is aimed at assessing one certain library
  business flows or functions, including
  benchmarking, business flow analyzing,
  unobtrusive method.
• Ronald R. Powell???????/????????????????(????????
  ?????)????????/???????(???????????????????)?????(?
  ????????)?????????????????????????????????????????
  ???
• Ronald R. Powell input measurement,
  output/performance measurement, impact/outcomes
  assessment, service quality assessment, process
  evaluation, benchmarking, standards, quantitative
  methods, qualitative methods, cost analysis,
  organizational effectiveness, program evaluation
  methods, and LIS-centered methods.

19
Current Assessment Tools ?????????

• Collections ??
• User surveys ????
• Focus groups ????
• Outside consultant (Special Collections)
  ?????(??)
• Services ??
• Statistics (gate counts, reference)??(????,?????)
• Web usability testing ???????
• Focus groups ????
• Information Literacy ????
• SAILS ???????????
• IL and Assessment Working Group??????????
• Assessment Librarian ????
• Review process ????
• Peer Review Committee (PRC) ???????
• ClimateQUAL

20
Tools by the Association of Research
Libraries?????????????StatsQUAL

• ARL Statistics ????????????
• LibQUAL ????????????????
• ClimateQUAL ??????????
• DigiQUAL ????????????????
• MINES for Libraries ?????????

21
???? Special Topics

• ??????,???????????????????
• The following special study areas are also the
  fields being often assessed by the libraries.
• ???? Collection
• ?????? Printed Collection
• ?????? Digital Collection
• ???? Service
• ????(?????) Technology (Acquisition, Cataloging)
• ???? Reference Service
• ???? Inter-library Loan
• OPAC
• ????? Library Web Site
• ???? Information System
• ?????? whole service quality
• ???? Valuation
• Administrative functions (human resources,
  marketing, financial services) are least
  evaluated 30 of survey respondents have not
  assessed these areas.
• ????(??????????????????)?????????-30????????????
  ???????

22
??????? Library Evaluation Practice

• ?????????????20??80?????????????????????
• Library evaluation practice started in the
  beginning of 1980s in China, and firstly carried
  out by Chinese Academy of Sciences (CAS)s
  library.
• ?????????????????????,????????,?????
• Characteristics of Chinese library
  evaluation government-sponsored, from top to
  bottom lack of initiative, sense deficiency.
• ????????????????????????????(????)
• Library evaluation follows such a trail
  trialextendencenon rationalityrationality
  (spontaneousconscious)

23
????????? College and University Library
Evaluation Practice

• ??????????????????????????
• Characteristic from nonexistence to
  existence, from the point to surface
  policy-oriented, teaching evaluation promoted
• 1985?5?27?,???????????????
• Decision on Educational System Reform by
  the Central Committee of CPC
• 1987?7?25?,???????????
• Library Regulation of College and
  University
• ?????????,????from nonexistence to existence,
  from the point to surface, in spontaneous state
• 1991?,????????????????????
• Suggestion on Initiating Library
  Evaluation for College and University
• ????,???????? developing all of the
  various aspects, government-oriented, assessing
  for single kind of library businesses
• 2002?2?26?,???????????(??)
• Library Regulation of College and
  University (Rev.)
• 2003?2?25?,?????????????(?????)
• Evaluation Index System of College and
  University Library (draft)
• ?2008?,????????????????,?????????????????????
  ???????????????(2007)??????????????????????????
  ?
• Till 2008, Steering Committee for
  Academic Libraries of China established College
  and University Library Evaluation and Statistics
  Regulation, including Evaluation Indictors for
  College and University Library and its assessing
  methods, Digital Resources Calculating Guide for
  College and University Library (2007), Suggestion
  on Library Collection Indicator in College and
  University Condition Standard.

24
????????? Public Library Evaluation Practice

• 1993?1?,?????????????????????,????????????????????
  ??????
• National Public Library Meeting sponsored
  by the Ministry of Culture in Liuzhou, Guangxi
  Province was held in Jan. of 1993. The meeting
  decided that public library evaluation all over
  the country should be taken and 5 provinces would
  be chosen as the pilots.
• 1994?3?9?,?????????????????????????,????????????
  ?????????
• On Mar. 9, the Ministry of Culture
  released Decision on Evaluation for Public
  Library above the county level, which showed
  the beginning of the whole national public
  library evaluation.
• ?1994???,?2009?,???????????? ?
• From 1994 to 2009, four times of public
  library evaluation were taken.
• ????????,??????????????????????6???,????????????
  ????????????????,??????????
• Public Library Evaluation Standard
  System were set up, consisting of 6 standards
  Evaluation Standards for public library or
  separate childrens library at the provincial
  level, prefecture-level, and county-level
  respectively.
• ?????????,2004????????????????????????
• The Library Society of China hosted
  2004-year public library evaluation , trusted
  by the Ministry of Culture.
• ????,?2009???????,??????????????????????????????
  ??
• Some indicators about National Project
  for the Sharing of Cultural Information and
  Resources were added into the evaluation
  standards in 2009-year public library evaluation.
• ????????????????????,?????????,?????????????
  ??(????????)?
• I seldom heard that some public
  libraries took evaluation by themselves for the
  purpose of inspecting their own businesses and
  library performance, valuation or outcomes except
  the four times of national general survey or
  census. ( Except for one or two libraries)

25
??????? Suggestion

• ??????????
• ???????
• ????????
• ??????
• ?????????????????
• ??????
• ????

• Back to basics of library evaluation
• Fulfill third party assessment
• Strengthen statistics
• Take into account of valuation
• Develop library evaluation guide for libraries of
  different types and sized
• Set up reward system
• Legal guarantee

26

• Thank you!