Title: The SCONUL Value and Impact Measurement Programme VAMP: a progress report
1The SCONUL Value and Impact Measurement Programme
(VAMP) a progress report
- Stephen Town
- (Cranfield University)
- Project Manager, VAMP
- SCONUL Autumn Conference
- British Library Conference Centre28th November,
2006
2Introduction
- Advocacy Drivers VAMP
- Report on progress to date
- Definition of the next steps
3The University Context (from the Library
Assessment Conference, Charlottesville, Va,
September 2006)
- Universities have two bottom lines
- Financial (as in business)
- Academic, largely through reputation in
- Research (the priority in leading Universities)
- Teaching ( maybe Learning)
4Library Pressures for Accountability
- The need is therefore to demonstrate the Library
contribution in these two dimensions - Financial, through value for money or related
measures - Impact on research, teaching and learning
- This also implies that competitive data will be
highly valued
5The SCONUL Response
- The SCONUL Working Group on Performance
Improvement - Ten years of toolkit development to assist in
performance measurement and improvement (for both
management and advocacy) - SCONUL Top concern survey 2005, leading to VAMP
6Examples of tools developed 1
- Integration
- Efficiency Comparability
- Quality assurance Guidelines
- SCONUL Statistics interactive service
- HELMS national performance indicators
- E-measures project
- Benchmarking Manual
7Examples of tools developed 2
- SCONUL Satisfaction Survey
- SCONUL LibQUAL Consortium
- LIRG/SCONUL Impact Initiative
- Information Literacy Success Factors
8VAMP Objectives
- New missing measurement instruments tools
- A full coherent framework for performance,
improvement and innovation - Persuasive data for University Senior Managers,
to prove value, impact, comparability, and worth
9Missing methods?
- An Impact tool, for both teaching learning and
research (based on the LIRG/SCONUL Impact
Initiative?) - A robust Value for Money/Economic Impact tool
- Staff measures
- Process operational costing methods
10Benefits?
- Attainment retention of Library institutional
income - Proof of value and impact on education and
research - Evidence of comparability with peer institutions
- Justification of a continuing role for libraries
and their staff
11VAMP Project Structure
- Phase 1 (March-June 2006)
- Critical review
- SCONUL Member Survey
- Gap analysis synthesis
- SCONUL Conference Workshops
- Phases 2 3 (July 2006 - April 2007)
- Development of new measures techniques
- Review and re-branding of existing tools
- Web site development
- Dissemination maintenance strategy
12Critical Review
13Critical Review Method
- Review of
- SCONUL initiated or promoted services
- Other UK European initiatives
- Initiatives from other UK library sectors
- International initiatives
- starting from the perspective of The Effective
Academic Library, 1995
14Review Findings
- The Impact Initiative work will be key, but needs
to be solidified and embedded - Eight JISC/EU projects in the last eight years
relevant to the assessment of value and impact,
many relating to e-resources - Significant work in the USA, Australia and South
Africa
15Review Conclusions
- Vast amount of relevant work, but without wide
take up - More critical analysis required of most products
and tools - Further development and simplification required
to create credible and applicable instruments for
SCONUL members
16SCONUL Member Survey
17Member Survey Findings
- 38 respondents 27 of population
- 70 undertaken value or impact measurement
- Main rationales are advocacy, service
improvement, comparison - Half used in-house methodologies half used
standard techniques - Main barrier is lack of tools, making time an
issue - Buy-in of stakeholders is an issue
18Member Survey Conclusions
- There is a need to demonstrate value and that
libraries make a difference - Measurement needs to show real value
- Need to link to University mission
- Libraries are, and intend to be, ahead of the
game - Impact may be difficult or impossible to measure
- All respondents welcomed the programme, and the
prospect of an available toolkit
19Synthesis, Expert Commentary, SCONUL Conference
Workshops
20Synthesis
- Some terminological confusion?
- Is impact effect or to outcome?
- Higher order effects and level
- Individual, course, institutional, vocational,
societal, national, international - Value and impact are not an item?
- Value, adding value, value for money,
cost-effectiveness
21Expert Comment
- Return to why?
- Advocacy or management?
- Critical Gap in measurement
- Effect of the Library on educational attainment
- Effect of the Library on research attainment
- Robust and simple tools
- Only a few existing tools effective, so simplify
and focus the range of offerings
22SCONUL Conference Workshops
- Accountability to a variety of structures and
individuals - therefore a range of approaches required
- SCONUL Statistics heavily used
- Directors want help with all VAMP lines
- Pedagogic Big project needed?
- Re-engineer processes rather than measure!
23Overall conclusions of Phase 1
- Wider sectoral involvement?
- Health Evidence-based methods
- National MLA Measures
- British Library Contingent Valuation
- Not only new measures
- but also supporting directing processes
24VAMP Phases 2 3
25Next Steps 1
- Content Products
- 2.1 Value Impact Guidelines
-
- 2.1.1 Institutional Value (eg VFM Economic
Impact) - 2.1.2 Impact on Teaching Learning
- 2.1.3 Impact on Research
26Next Steps 2
- Content Products
- 2.2 Staffing Operational Measures Guidelines
-
- 2.2.1 Staff Costing
- 2.2.2 Staff Added Value measures
- 2.2.3 Other operational costing methods
- 2.3 Re-branding packaging of existing tools
27Next Steps 3
- Process Products
- 3.1 Web Site
-
- 3.2 Community of practice establishment
- 3.3 Maintenance sustainability strategy
28Progress on Content 1
- 2.1.1 Institutional Value (eg VFM Economic
Impact) - No source identified for VFM tool
- Contingent Valuation method in negotiation
- 2.1.2 Impact on Teaching Learning
- 2.1.3 Impact on Research
- Tools for both areas commissioned (Information
Management Associates)
29Progress on Content 2
- 2.2.1 Staff Costing
- 2.2.2 Staff Added Value measures
- 2.2.3 Other operational costing methods
- Staff operational costing method in negotiation
- No source identified for Staff Added-Value tool
- 2.3 Re-branding packaging of existing tools
- Mainly included within the Web Site development
30Existing or Identified Methods/Tools
- Benchmarking
- Charter Mark
- Critical Success factors
- Economic Impact
- HELMS
- Impact Measurement
- Information Literacy Measurement
- Investors in People
- JISC Studies
- KPIs
- LibQUAL
- Process Costing
- Quality Assurance
- Quality Maturity Model
- Research Impact
- SCONUL Satisfaction Survey
- SCONUL Statistics
- Teaching Learning Impact
- Transparency Costing
31Progress on Process
- 3.1 Web Site
- Web Site commissioned (Cranfield)
-
- 3.2 Community of practice establishment
- Invitations in progress
32Communities of Practice
- groups of people who share a passion for
something that they know how to do,and who
interact regularly to learn how to do it better - coherence through mutual engagement
- Etienne Wenger, 1998 2002
33Community of Practice
Techniques
Members Forum (Blog?)
VAMP Home Page
Simple Introductions
Detailed Techniques
Techniques in Use (Wiki?)
34Acknowledgments
- Angela Conyers, Evidence Base, UCE
- Claire Creaser Suzanne Lockyer, LISU,
Loughborough University - Professor Peter Brophy, Manchester Metropolitan
University - The VAMP Subgroup of SCONUL WGPI
- Maxine Melling, Philip Payne, Rupert Wood
35(No Transcript)