Tools Strategies and Methodologies for measuring impact of ICTs in EDucation

1
Tools Strategies and Methodologies for measuring
impact of ICTs in EDucation

• Simon Ellis
• UNESCO Institute for Statistics

2
Some key observations

• Impact studies
• Every country has a pilot impact study on a few
  schools
• Few (no?) countries have nationwide impact
  studies
• The major determinants of school achievement are
• Intelligence IQ
• Education of parents
• NOT computers
• ICT financing
• OECD ICTs are a marginal cost so ICTs are
  ubiquitous in home and school
• Developing countries introducing ICTs in ALL
  schools in developing countries requires MAJOR
  investment
• Measuring impact is a major issue in developing
  countries

3
Asia

• Many Asian countries are on the threshold of
  ubiquitous ICT in particular through wireless
• Asian countries recognise that technological
  revolution is necessary for growth and this has
  to start in schools
• ICTs, especially wireless networks, can help
  bridge urban/rural divide
• BUT introducing ICTs everywhere remains a very
  big investment
• Asian pedagogy often consists of direct
  instruction rather than situational or
  constructivist learning

4
What must be done

• Prove benefit of national programme to put ICTs
  in all schools given that
• Marginal effect on exam results
• Ubiquitous ICTs necessary for growth (having a
  computer at home more important than having one
  at school)
• Need to
• Justify major financial investment
• Encourage further investment

5
Mobilising a partnership

• UNESCO Bangkok/UIS
• KERIS
• World Bank
• OECD
• EU

6
UNESCO Institute for Statistics (UIS) global
mandate

• WSIS 2003/5 set up Partnership for Measurement of
  ICTs for Development
• UNCTAD, ITU, UNESCO, WBank, OECD, EU, UNSD
• Partnership assigned global leadership for ICTs
  in Education to UIS
• UNESCO intersectoral meeting July 2008 (CI and
  Ed) assigned UNESCO lead on measuring ICTs to UIS

7
UIS value-added

• The official international lead agency for
  education indicators for EFA and MDGs
• Existing global data collection
• In support of EFA
• Put ICTs in context of education system
  indicators
• Able to develop and apply global international
  statistical standards
• Eg ISCED, FCS, core ICT indicators all for
  endorsement by UNESCO General Conference and UN
  Statistics Commission after extensive
  international consultation
• Strong network of national agencies officially
  responsible for managing education indicators

8
Levels of ICT indicator

• Infrastructure
• Presence of ICT (
• only used by headmaster/accounts
• Access
• Availability
• Time use
• How many hours per week per pupil
• Shared PCs
• email
• How used?
• Skills
• Type of use research, report preparation,
  presentations

9
Measuring Level 3 skills and impact

• Use of email (student output onto the internet)
• Embedding of downloaded image and sound in
  written work and presentations (use of new media)
• Posting of web pages, Facebook, Blogs etc
  (pro-active internet-based work)
• Overall level of generic skills in use of
  technology
• Communications skills with ICTs
• Problem-solving skills with ICTs

10
Survey instruments

• Admin data
• Used to manage education system, assess spread of
  new technology, costs
• School survey
• Skills assessment eg PISA
• School conditions eg SPS
• Household survey
• Children not in school
• Home environment
• Pc, internet access, parents literacy
• Eg. national census, LAMP

11
UIS actions in Asia

• Organizes annual/6 month meetings of regional EFA
  Co-ordinators and partners (with UNESCO APPEAL
  team)
• Worked with APEID on previous UNESCO Bangkok
  initiative on ICT in Education statistics
• WISE group (in Asia, Thailand, Korea, Malaysia)
  implementing new survey to collect 48 indicators
• Long-term statistical capacity building
  activities
• Lao, Brunei, Cambodia, Myanmar, India, Nepal,
  Bangladesh, Bhutan
• Data planning and DQAF
• EMIS and data collection techniques
• Data collected Admin data levels 1 and 2

12
KERIS initiative

• Data collected from Thailand, Philippines,
  Vietnam, Sri Lanka, Mongolia
• Includes more contextual data
• More analytical approach

13
Risks

• Duplication two or more teams collecting same
  data
• Conflict between two national teams over
  official mandate
• Response fatigue means no-one gets data
• Human/financial resources spent twice to get same
  data
• Confusion unclear whether data requested by
  KERIS or UIS
• Questionnaire sent to wrong agency
• Inconsistency slightly different data sent to
  UIS and KERIS
• More recent data
• Errors due to response fatigue
• Different reference period
• National or international definitions (eg ISCED
  mapping)

14
Co-ordination

• More international co-ordination of work on ICT
  in Education is needed to achieve
• Sustainability
• National capacity building
• New indicator development

15
How to ensure sustainable impact of KERIS and UIS
programmes?

• National ICT indicators (levels 1 and 2) must be
  embedded in EMIS systems and collected regularly
  to monitor
• progressive expansion of ICT use in schools
• integration of ICTs in teacher training
• BUT EMIS cannot capture integration into teaching
  practices

16
How to ensure that KERIS/UIS build national
capacity to monitor ICTs?

• Co-ordinate all regional programmes
• to develop nationally implementable measures of
  ICT impact
• To help countries to collect data collection to
  meet national needs
• Co-ordinate interventions in particular countries
  to build on each others successes and not
  duplicate same activity