Measurement practice Barbara Lrincz, Capgemini barbara'lorinczcapgemini'com Diane Whitehouse, Deloit

1
Measurement practiceBarbara Lörincz, Capgemini
(barbara.lorincz_at_capgemini.com)Diane
Whitehouse, Deloitte(with Indigov)(diane.whiteho
use_at_thecastlegateconsultancy.com)?
2
Roadmap

•  Proxi  scenarios
• eGovernment  before and after 
• Measurement practice  before and after 
• Europe  before and after 

3
eGovernment before and after

• After
• Technology on its own cannot compensate for the
  mistakes of bureaucrats and politicians
• eGovernment for better governance knowledge
  brings powerand makes maladministration harder
• eGovernment is a fast-moving target, and ad hoc
  measurements will fail to capture its impact
• Absolute quality is not the same as user
  satisfaction
• Communicating globally (open source) social
  networking locally (à la MySpace and YouTube
  generation)

• Before
• ICT enable existing administrative processes
• eGovernment for efficiency gains
• Ad hoc measurements
• Arrogance not to understand or inability of
  governments to meet user requirements
• Silo-government

4
Measurement practice before and after
Changing eGovernment Issues Over Time (based on
Heeks, 2006)?
5

Twofold Accenture Core Measure 1) 50 service
maturity 2) 50 customer service maturity

• Subdimensions of service maturity
• service maturity breadth (number of services)?
• service maturity depth (with maturity stages
  publish, interact, transact)?

Subdimensions of customer service maturity
6
US, Canada, Singapore, Australia ahead of EU
Member States
(Accenture, 2006)?
7
New emerging service paradigms
(Accenture, 2006, 2007)?
8
(UN 2008)?
United Nations eGovernment Readiness
Service maturity embedded in contextual
indicators
9
Nordics in the lead
Surprising results for eParticipation
(UN 2008)?
10
Capgemini
What about comparability over time?
Capgemini (2007)?
11

• Legally binding eID
• Number of data fields
• Multi channel access
• Compliance with accessibility standards

Pilot indicators success story?
Capgemini (2007)?
12

• Nr of 20 e-services accessible through portal
• Personalization
• Different entry modes according to user profile
• Consistency of lay-out and branding

How to define dimensions of composite indicators?
Capgemini (2007)?
13

American Customer Satisfaction Index
Correlation and causality please
Drivers of satisfaction search, navigation,
content, site performance Channel loyalty
(likelihood to recommend, return to the site and
to use it as a primary resource)?
14
Benchlearning
Leading partner
Pilot
Pilot 1 Efficiency gains
Pilot 2 Administrative burden
Pilot 3 User centric impact
Focus
GIS and Cadastre
Service for companies vs. citizens
National eGOV Portals

• Agenzia del Territorio (Italian National
  Cadastre)?
• Oficina Virtual del Cadastro (Cadaster Virtual
  Office of Spain)?
• Lantmäteriet (National Land Survey of Sweden)?
• Regional agencies (Emilia-Romagna, Catalonia)?

• Project Observers
• Fedict, Dutch Ministry of Interior, Greek
  Information Society Observatory
• Pilot agencies G2B
• Belgium FPS Economy
• Slovenia Ministry of Public Administration
• Pilot agencies G2C
• Greece e-taxes
• Belgium FPS Finances

• DirectGov (British eGovernment portal)?
• Service-public.fr (French eGovernment portal)
• eCroatia (Croatian eGovernment portal)?

Agencies
Countries
14
15
And implications for Member States?
16
Study on user satisfaction and impact on EU27

• Starting point 87 projects, initiatives,
  reports, websites
• Good practices selective list
• Member States level information (gathered with
  input of the i2010 eGovernment subgroup)?
• Most mentioned survey Eurostat ICT-usage by
  households
• Good practices highlighted
• Belgium Fed-eView longitudinal approach
  (panel, 3 waves), wide scope
• Italy Revenue Agency case one issue, focus on
  satisfaction
• Netherlands Citizens Services Code
• Slovenia strong methodology, large toolset
• United Kingdom multichannel approach, large
  toolset

17
User satisfaction and impact on EU27 main
findings

• From 27 European Member States cases
• Limited experience
• Lack of standardization
• Lack of transparency
• Variation in progress
• Starting points with regard to common
  experiences
• Handbooks and common guidelines (Germany and
  United Kingdom) and survey tools (sets of
  questionnaires)?
• Experiments in satisfaction measurement
  (Belgium, Netherlands, Slovenia)?
• Broader theoretical frameworks (eGEP)?
• Real-time evaluation (Top of the Web and others)?
• User centricity in the eGovernment benchmark
• Novel approaches Web metrics/tracking, mystery
  shopping.

18
Ten key lessons learned (1)?

• No standardization
• From e to inclusive Government (important on all
  political agendas)?
• Holistic framework
• eGovernment take-up
• User expectations
• Channel preferences
• Perceived benefits
• Future use and priorities for service
  improvements
• User satisfaction user expectations and
  perceptions of quality (survey concept/samples)?
• Profiling citizens (e-skills, attitudes, use of
  ICT, social groups, customer segments)?
• Focus of measurement eGovernment, stages of
  delivery, life events

19
Ten key lessons learned (2)?

• Different levels of take-up
• Longitudinal systematic monitoring
• ACSI/AeGSI (US-model) is the most developed and
  applied model today
• CMT (Canada) can provide us with a set of
  questions, database, evaluation/question toolkit
• Until now, we did not find an umbrella example
  within the EU.
• Canada and the US may provide the most
  interesting international examples
• Typologies of services, user definitions,
  possible approaches ( life events) are very much
  present in different EU studies (eUser, eGEP)?
• How to find a balance between standard solutions
  and the Member States installed base?

20
Selected Member States for survey

• Austria
• Belgium
• France
• Germany
• Hungary
• Italy
• Netherlands
• Spain
• Sweden
• United Kingdom

21
Europe before

• Growing, by leaps and bounds
• Fragmented
• Huge emphasis on subsidiarity
• Gaps between local, city, regional, and national
  levels
• Diversity of approaches huge spread and
  north-south divide
• Focus on 21 EU languages
• European and international institutions working
  in an uncoordinated way
• Cautiousness and timidity relevant lack of
  willingness to work together

22
Europe after hopes and expectations

• Continued growth
• More solid view of Europe as a union/a unified
  whole/a single market
• Increasing view of Europe and Europeans facing
  its/their mutual, shared challenges
• Lessening emphasis on subsidiarity
• Positive approach to diversity created by
  multitude of different local, city, regional,
  and national levels
• Whatever the diversity, points of single entry
  and one-stop shops
• Continued diversity of languages (including those
  of immigrants) but at the same time, a
  concentration on major languages
• Greater coherence in approaches lessening spread
  and lessening north-south divide
• A more joined-up European Commission (and
  international associations) if not yet joined-up
  European institutions

23
Next steps

• Open to discussion