User needs

1

• User needs
• Vipavc Brvar Irena, ADP, Slovenia
  IASSIST/IFDO
  2005, 24 - 27 May, 2005

2

• Purpose EVALUATION
• The most general definition of evaluation is that
  it is a process of assessment of the value of a
  certain activity or objects.
• In the case of libraries they evaluate how well
  the library satisfy the needs of its users and
  basic limitations and weaknesses in the librarys
  operation. The standard ISO 11620 (1998) defines
  evaluation as a process of assessing
  effectiveness, efficiency, usefulness and
  relevancy of a certain services, equipment or
  facilities.

3

• Lancaster (1977) states that services (in our
  case also data archives services), can be
  evaluated from different viewpoints and that in
  the process we find
• How well a service realises the goals set, which
  usually means how well it satisfies the needs of
  the users.
• How efficiently a service, system realises the
  goals set (operating costs).
• Does the data archive justify its existence, that
  is what is the usefulness of the data archive
  service in light of operating costs (measuring
  cost-performance-benefits)
• Evaluation processes are important for
  decision making.

4

• Evaluation is a part of strategic planning
  process and has practical significance.
• It is in some cases also necessary because of
  political reasons (to justify the expenditure of
  public funding and to show sense in its
  spending).
• - gt The future of data archives does not lie in
  supporting generic, broadly useful services such
  as information access to large collections, but
  in supporting customization by community, i.e.
  the development of services tailored to support
  the specific, and real, practices of different
  user constituencies.

5

• In data archive area we do not have indicators
  that would measure our performance nor do we have
  standard questionnaire/questions that would
  measure satisfaction of our users with our
  services.
• There were some attempts to measure user
  satisfactions (UK DA, ICPSR, University of
  Tennessee, etc.).
• -gt Changes in IT brought to us new expectations
  from users. We experience change of practice in
  data distribution and collection, change of
  behaviour of users, and usability. Users want
  codebooks and data files to be easy available.

6
(No Transcript)
7

• Decision was made to conduct a survey and
  evaluate users needs and their satisfaction with
  offered services.
• We do want responses from different groups of
  users
• (researchers, users and potential users)

Banner on Faculties Web pages
Registered Slovene researchers in the field of
social science.
Known users- in our user database
Cover letter by email
8

• How to measure usability
• Analytically counter log analyzer, tracing
  paths,
• Web site usability evaluation, etc.
• Empirically survey...

9

• Information from Web statistics (log
  analysis)
• Number of visits, and number of unique
  visitors, Visits duration and last visits,
  Days of week and rush hours, Domains/countries
  of hosts visitors (pages, hits, KB, 269
  domains/countries detected, GeoIp detection),
  Hosts list, last visits and unresolved IP
  addresses list, Most viewed, entry and exit
  pages, OS used, Browsers used , Search
  engines, keyphrases and keywords used to find
  your site, Number of times your site is "added
  to favourites bookmarks". Screen size ......

10

• Analysis of log file from Nesstar
• Analysing user paths

Colleting Study. Info.
event
news
Particular study
EN
ADP
List of studies
SLO
Price list
Mail-contact
depositors
Order form
11

• Survey on user satisfaction
• Pretest (current stage)
• Define Sampling Frame and Contact Design
• Decide on data collecting method
• Web based survey
• Create separate URL addresses for each email
  address given. Demands for anonymity are
  satisfied.

12

• Web survey with PHPSurveyor
• PHPSurveyor is a multi-question surveying tool
  that allows you to develop, publish and manage
  surveys. PHPSurveyor script allows you to display
  surveys as single questions, group by group or
  all in one page or use a dataentry system for
  administration of paper-based versions of the
  survey. PHP Surveyor can produce "branching"
  surveys (set conditions on whether individual
  questions will display), can vary the look and
  feel of your survey through a templating system,
  and can provide basic statistical analysis of
  your survey results.
• -Conversion to SPSS data file was written.

13

• Most of current user satisfaction surveys ask
  users to tell them how much they are satisfied
  and how important particular mentioned topic is
  (5 point scale) .
• To shorten the questionnaire we decided to do it
  a little bit different.
• We inquire about overall satisfaction (on
  general, and about web page) and to evaluate
  particular services.
• Discriminant analysis!!
• -gt create indexes that measure satisfaction and
  importance

14
Relation between satisfaction with and importance
of particular element
Satisfaction
Importance
This need to be changed!!!!
15

• Questionnaire available at
• http//www.adp.fdv.uni-lj.si/publikacije/userneeds
  .html

16

• Literature
• AMBROIC, Melita (2003) A few countries measure
  impact and outcomes most want to measure at
  least something. In Performance Measurement and
  metrics. Vol.4, No.2, p. 64-78.
• BARNES, Stuart J. And VIDGEN, Richard (2001) An
  Evaluation of Cyber-Bookshops The WebQual
  Method. In International Journal of Electronic
  Commerce. Fall 2001, Vol.6, No.1., pp.11-30.
• BOLLEN, Johan and LUCE, Rick (2002) Evaluation
  of Digital Library Impact and User Communities by
  Analysis of Usage Patterns. In D-Lib Magazine.
  Vol. 8, No.6.
• COLEMAN, Anita and SUMNER, Tamara (2004) Digital
  Libraries and User Nees Negotiating the Future.
  In Journal of Digital Information. Vol 5, Issue
  3.
• HERNON, Peter and ALTMAN, Ellen (1998) Assessing
  service quality satisfying the expectations of
  library customers. Chicago, London American
  Library Association.
• NORLIN, Elaina (2002) Usability testing for
  library Web Sites. A hands-on guide. Chicago,
  London American Library Association.
• RUBIN, Jeffrey (1994) Handbook of usability
  testing. How to plan, design, and conduct
  effective tests. John Wiley Sons, USA.