CWIQ looped presentation

1
The Core Welfare Indicators Questionnaire (CWIQ)
2
CONTENTS
1. What is the CWIQ and how does it
work? 2. Overview of the Pakistan pilot survey
3. Planning the full scale CWIQ
3
Part 1 What is the CWIQ and how does it work?
4
Features of the CWIQ

Fixed core, flexible modules
Quick data entry validation
Simple reporting
Easy data collection
Short questionnaire
Large sample
5
Features of the CWIQ

Fixed core, flexible modules
Quick data entry validation
Simple reporting
Easy data collection
Short questionnaire
The CWIQ has been built around several key ideas
That all work to complement each other
Large sample
6
Features of the CWIQ

Fixed core, flexible modules
Quick data entry validation
Simple reporting
Easy data collection
Short questionnaire
Large sample
7
Features of the CWIQ

Fixed core, flexible modules
Needed to provide estimates at sub-national
levels and for particular target populations.
Quick data entry validation
Simple reporting
Easy data collection
Short questionnaire
Large sample
8
Features of the CWIQ

Fixed core, flexible modules
Quick data entry validation
Simple reporting
Easy data collection
Short questionnaire
Large sample
9
Features of the CWIQ
The core questionnaire consists of 9 pages (5
sheets).

Fixed core, flexible modules
Quick data entry validation
Simple reporting
Easy data collection
Short questionnaire
Large sample
10
Features of the CWIQ

Fixed core, flexible modules
Quick data entry validation
Simple reporting
Short questionnaire
Easy data collection
Large sample
11
Features of the CWIQ

Fixed core, flexible modules
Quick data entry validation
A single interview takes about 40 minutes to
complete
Simple reporting
Short questionnaire
Easy data collection
Large sample
12
Features of the CWIQ

Fixed core, flexible modules
Simple reporting
Quick data entry validation
Short questionnaire
Easy data collection
Large sample
13
Features of the CWIQ
The combination of short questionnaire, scanning
and predefined validation procedures reduces
processing time and reduces the range and number
of non-sampling errors

Fixed core, flexible modules
Simple reporting
Quick data entry validation
Short questionnaire
Easy data collection
Large sample
14
Features of the CWIQ

Fixed core, flexible modules
Simple reporting
Quick data entry validation
Short questionnaire
Easy data collection
Large sample
15
Features of the CWIQ
Basic tables are automatically tabulated to show
frequency distributions for all the key
indicators disaggregated by urban and rural,
region and socio-economic grouping.

Fixed core, flexible modules
Simple reporting
Quick data entry validation
Short questionnaire
Easy data collection
Large sample
16
Features of the CWIQ

Fixed core, flexible modules
Simple reporting
Quick data entry validation
Short questionnaire
Easy data collection
Large sample
17
Features of the CWIQ

Fixed core, flexible modules
Simple reporting
Quick data entry validation
A fixed core for rapid reporting. Flexible
modules for customizing to country needs
Short questionnaire
Easy data collection
Large sample
18
Features of the CWIQ

Fixed core, flexible modules
Simple reporting
Quick data entry validation
Short questionnaire
Easy data collection
Large sample
19
Features of the CWIQ
The CWIQ is not a complicated survey. It
incorporates a package of features, which, when
taken together, ensure wide coverage and a rapid
turnaround time.

Fixed core, flexible modules
Simple reporting
Quick data entry validation
Short questionnaire
Easy data collection
Large sample
20
Part 2 The Pakistan Pilot Survey
21
OBJECTIVES
The CWIQ pilot objectives

• Objectives of the pilot survey include
• Testing and refining the modified questionnaire
  and software system
• Training field and data processing staff in use
  of the CWIQ
• Gathering operational data for planning full
  scale survey
• Providing end-users with examples of CWIQ output.

22
OBJECTIVES
The CWIQ modifications

CWIQ is a generic package including computer
software. Specialist software used for both
scanning and data processing must be
modified. Questionnaire is combination of the
generic CWIQ with modifications based on
recommendations of technical committee.
23
OBJECTIVES
The CWIQ modifications- the wider picture

• Scanned forms for capturing data from schools and
  health facilities have also been developed and
  tested. The tabulation system is being prepared.
• First time the CWIQ has been used in Asia.
• Strong emphasis on creating an instrument for
  Pakistani requirements.
• New features developed here should contribute to
  the ongoing development of the CWIQ.

24
THE TIMETABLE
The CWIQ pilot timetable
3-6 November Training course in Islamabad 7-15
November Fieldwork in 4 provinces 20 November All
data cleaned and tabulated 21 November Presentatio
n workshop
25
THE QUESTIONNAIRE
The questionnaire

• some questions relate to the household as a unit
  of study (asset ownership, access to services
  etc.)
• other questions relate to each household member
  (demographic information, education, health and
  employment)
• additional data for children under five years old
  (immunisation history)

26
THE QUESTIONNAIRE
Welfare indicators The questionnaire captures
data for estimating

• reporting diminishing/increasing assets (land
  and livestock)
• of literate adults
• Housing (mean number of persons per room).
• of adults unemployed in the last 7 days.

27
THE QUESTIONNAIRE
Access, utilisation and satisfaction The
questionnaire captures data for estimating

• Access to clean water
• Access to sanitation facilities
• Access to pre-primary, primary and secondary
  school.
• Net pre-primary, primary and secondary enrolment
  rates (by gender)
• Primary and secondary drop out rates (by gender)
• Type of birth supervision for children less than
  five.
• Satisfaction rating with primary and secondary
  school services.
• Access to local health centre.
• Frequency of use of local health centre.
• Completeness of vaccination record for children
  under five years.
• Satisfaction rating with local health services.

28
THE QUESTIONNAIRE
Designed for scanning
29
THE TRAINING COURSE
The training course

• 4 day training course which included
• detailed presentation on the questionnaire and
  interview technique
• role play interviewing in class
• field interviews
• exercises to introduce techniques needed for
  scanned forms

30
Training
GOOD DATA QUALITY
31
GOOD DATA QUALITY
Preventing data quality problems

• The scanner and computer software is critical to
  the training.
• All practice interview forms are scanned and
  validated.
• Errors and problems are identified and feedback
  is provided to trainees.
• The system can be used for final selection of
  competent interviewers.

32
GOOD DATA QUALITY
Data quality is achieved through

• thorough training of interviewers to ensure an
  in-depth understanding of the CWIQ
• close supervision in the field
• interviewers checking each others work
• rapid data loading with extensive computerised
  validation checks and
• early feedback to interviewers in the case of
  problems.

33
THE SAMPLE FOR THE PILOT SURVEY
The sample

• 48 clusters each with 20 households
• total sample size of 960 households
• sample was selected on a purposive basis,
  principally to test the methodology. Care should
  be exercised with the results from this survey.

34
THE SAMPLE FOR THE PILOT SURVEY
The sample Number of households by province
35
THE FIELDWORK
The CWIQ pilot fieldwork

• 27 interviewers with 8 supervisors
• just over 9 days field work

36
THE FIELDWORK (continued)
4 interviews per enumerator per day should be
possible
37
THE FIELDWORK (continued)
Mean interview duration was about 40 minutes
38
THE FIELDWORK (continued)
Mean household size was 6.6 persons
39
THE DATA PROCESSING

• team of 4 FBS staff plus 1 consultant
• uses standard Windows-based computers
• requires two key software packages
• TELEform for image processing of the scanned
  forms, converting the marked areas into data
  values
• Microsoft Access to build data structures,
  validate, correct, summarise and tabulate the
  data
• objective is to start data processing as soon as
  possible after the start of fieldwork.

40
THE DATA PROCESSING
41
THE DATA PROCESSING - stage 1
Visual checks on questionaires
Prepare forms for scanning
Scan forms to create digital images in the
computer
Automatic creation of data records. Some user
verification may be required
42
THE DATA PROCESSING - stage 1 (continued)
This shows how character recognition is verified
and corrected using TELEform software
43
THE DATA PROCESSING - stage 1 (continued)
An evaluated form showing the original on the
right hand side
44
THE DATA PROCESSING - stage 1 (continued)
Week 1 waiting for forms to arrive. Week 2
over 250 forms on 18th
45
THE DATA PROCESSING - stage 1 (continued)
A single 9 page questionnaire can be scanned and
converted into the database format in just over 1
minute. Three people in the data processing team
can handle 300 questionnaires per day using the
equipment used for the pilot survey. Faster
scanners and computers as well as more staff
would be used in the full scale survey. Capacity
would be matched to the rate of fieldwork. The
package includes comprehensive documentation on
all aspects of the data processing.
46
THE DATA PROCESSING - stage 2
Once scanned the data are transferred to standard
database package (MS-Access) which has been
configured to

• perform a number of validation checks to test
  logical consistency of the data
• provide a means of editing erroneous records
• generate derived variables
• produce a range of formatted outputs including
  descriptive statistics, multi-way tables and
  standard errors and
• export the data to other data analysis packages.

47
THE DATA PROCESSING - stage 2
THE DATA PROCESSING - stage 2 (continued)
Transfer data to Access
Run validation checks
Enter corrections
Examine error listing and determine corrections
to be made
Summarise and generate tables
48
EXAMPLES OF CWIQ STANDARD OUTPUT TABLES

• A small selection of the standard output of the
  CWIQ tabulation system has been provided as a
  handout.
• Please note the request that these results not be
  used for any purpose other than demonstrating the
  CWIQ system.

49
Part 3 Scaling up to a national CWIQ survey
50
MOVING TO THE NATIONAL CWIQ
The pilot survey has, hopefully, achieved its
objectives. The pilot survey needs to be
thoroughly reviewed, both internally and
externally, in order to plan ahead. Prompt action
is required in order to maintain momentum. The
full set of modified tables and review document
should be produced as quickly as possible.
What are the next steps?
51
MOVING TO THE NATIONAL CWIQ
First thoughts what has worked?

• The technology has worked well. The modified
  software has been successful as has optical
  scanning for data loading.
• Most interviewers conducted the interview
  satisfactorily.
• The FBS team who carried out the scanning and
  cleaning have proved they can do this task.
• Data from 960 households across Pakistan have
  been collected, scanned, cleaned and tabulated in
  less than 2 weeks.

52
MOVING TO THE NATIONAL CWIQ
First thoughts what needs to be improved?

• The training course was probably too short.
• Some of the fieldwork and supervision was of poor
  quality. This is one of the most serious
  challenges.

53
MOVING TO THE NATIONAL CWIQ
Review and revision of the questionnaire

• Thorough review of output and field experience of
  interviewing teams.
• Comments of potential data users.
• Formalisation of lessons learned at each stage of
  the pilot exercise identifying best practice.
• Further adaptation of the CWIQ to Pakistans
  requirements? depends on the review.

54
MOVING TO THE NATIONAL CWIQ
The course ahead will be determined by the
answers to a number of questions.

• Does the output of the CWIQ meet the requirements
  of the data users (in terms of relevance,
  accuracy, timeliness etc.)?
• What are the resource implications?
• What is required for the FBS to implement this
  survey instrument in the time frame currently
  envisaged? (completed by Sept 2004)
• How does it fit into the programme of other
  activities for the FBS, especially the PIHS?

55
MOVING TO THE NATIONAL CWIQ
Proposal for moving to a national CWIQ tasks...

• consolidation of lessons learned and revision of
  questionnaire and data processing system
• logistics and planning for a major exercise
• preparation of detailed budget for additional
  capital and recurrent expenditure
• procurement
• technical assistance for the first full survey
• recruitment and training of interviewers

56
MOVING TO THE NATIONAL CWIQ
Extending the capacity within the FBS One
possible example

• A complement of 300 interviewers
• Say 4,000 interviews per week allowing time for
  travel, listing etc.
• A sample of 80,000 would take around 6 months,
• Data processing can keep pace with interview rate
  leading to early availability of results.

57
MOVING TO THE NATIONAL CWIQ
Linking CWIQ to policy?

• How do we ensure that the CWIQ contributes to
• Performance Related Budgeting
• Monitoring the impact of expenditures on priority
  areas
• Cross-checking facility (education and health)
  data with the HMIS and EMIS
• Easier to achieve above objectives when CWIQ is
  firmly established in the FBS and fully owned by
  stakeholders.

58
The End