Title: Monitoring and Evaluation: Calculating and Interpreting Coverage Indicators
1Monitoring and Evaluation Calculating and
Interpreting Coverage Indicators
2Learning Objectives
- By the end of the session, participants will be
able to - Identify sources of data for calculating coverage
indicators - Estimate denominators for routine coverage
estimates - Calculate and interpret coverage indicators from
routine data - Use online resources for estimating coverage
indicators - Assess the quality of relevant data sources
- Reconcile coverage estimates from different data
sources
3Maternal Health Coverage Indicators
- Proportion of pregnant women who received at
least two antenatal care visits - Proportion of deliveries occurring in a health
facility - Proportion of deliveries with skilled attendant
at birth - Proportion of women attended at least once during
postpartum period (42 days after delivery) by
skilled health personnel for reasons related to
childbirth
4Why Coverage Indicators Are Important
- Understand how effective program is
- See if one target group is reached more
effectively than another - Identify underserved area/regions
5Child Health Coverage Indicators
- Immunization Programs
- DTP3 vaccine coverage
- Measles vaccine coverage
- BCG vaccine coverage
- OPV3 coverage
- HepB3 coverage
- Fully immunized child
- Nutrition programs?
- Control of diarrheal disease programs?
6Coverage Indicators for HIV/AIDS Care Treatment
Programs
- Number of clients receiving public/NGO VCT
services - Number of clients provided with ARVs
- Percent of children in need receiving
cotrimoxazole prophylaxis - Percent of HIV patients receiving DOTS
- Coverage of PMTCT programs?
7Where Do We Get the Data?
- Censuses
- Surveys
- Registrations
- Health management information systems
- Program statistics
- Patient registers
8Estimating Coverage From Routine Data
9Indicators From Program Statistics Numerators
- HMIS and routine reports give information on
numerators - Numerators number of deliveries in health
facilities, measles vaccinations, pills
distributed, voluntary counseling and testing
clients etc. - Denominators ?
10ExampleImportance of denominator
- Town A vaccinated 200 infants
- Town B vaccinated 400 infants
- Town C vaccinated 600 infants
- Population size
- Town A 10,000
- Town B 30,000
- Town C 60,000
11Indicators From Program Statistics What
Denominators Are Needed?
- Denominators population composition
- Population composition
- How many women are of childbearing ages?
- How many children are under five?
- How many adolescents? 15-19? 20-24?
- How many men are 15-59 years?
- How many children are of school going age?
- How many infants are there?
- How many babies are born each year?
12How Do We Get Denominators?
- Population registers
- Censuses
- Population projections
- Population growth rate (r)
- Rate of natural increase crude birth rate (CBR)
minus the crude death rate (CDR) - Net migration rate inmigration - outmigrants per
1000 population - CBR no. of births per 1000 population in 1 year
- CDR no. of deaths per 1000 population in 1 yr
- Population growth rate of natural increase
net migration rate
13Spectrum Model
- DemProj projects population of country/region by
age and sex based on assumptions about fertility,
mortality, and migration - Urban and rural population projections can also
be prepared - EasyProj supplies data needed to make a
population projection from estimates provided by
the Population Division of the UN - www.tfgi.com
14Spectrum
15Calculating Denominators
- Population at time t P(t) P(0) exp(rt),
where - P(t) is the population size after t years
- P(0) is the population size at the last census
- Example
- 300,000 people at census
- Growth rate 3 (0.03),
- What is the population after 10 years?
- 404,958 people
16Estimating Number of Live Births
- Where data on the number of live births are
unavailable - Total expected births Total population x crude
birth rate - Where the crude birth rate (CBR) is unknown
- Total expected births Total population x 0.035
Source WHO 1999a WHO 1999b
17Estimating Number of Surviving Infants
- Target population for childhood immunization
- Surviving infants lt12 months of age in a year
- Where data on the number of surviving infants are
unavailable - Total expected number of surviving infants
- Total population x CBR x (1 infant mortality
rate)
18Estimating Number of Surviving Infants CBR Known
- Total population 5,500,000
- CBR 30/1000
- Infant mortality rate (IMR) 80/1000
- Number of surviving infants
- Total population x CBR x (1 IMR)
- 5,500,000 x 30/1000 x (1 - 0.080)
- 5,500,000 x 0.030 x 0.920
- 151,800
Source Immunization Essentials A Practical
Field Guide (USAID, 2003)
19Estimating Number of Surviving Infants CBR
Unknown
- Where data on the number of surviving infants,
CBR or IMR are unavailable, multiply total
population by 4 - Expected no. of surviving children lt 12 months
- Total population x .04
- If the total population is 30,000, then the
number of children under one year 30,000 x
4/100 1200
Source WHO, 2002b
20Estimating the Monthly Target Population
- Monitoring immunization and vitamin A coverage
should - be done monthly at the facility and district
levels, - requiring estimations of the monthly target
population - Monthly target population Estimated number of
children under 1 year of age divided by 12 - Example
- Annual target population of children lt 12 months
1200 - Monthly target 1200/12 100
21Example Immunization Coverage From Routine Data
- Total population of district in 1990 99,000
- CBR 40 per thousand
- IMR 80 per thousand
- Population growth (r) 3 per year
- 3,000 measles vaccinations were given to infants
in district in 1998 - What is the measles coverage rate for 1998?
- Numerator No. immunized by 12 months in a given
year - Denominator Total no. of surviving infants lt 12
months in same year
22Immunization Coverage From Routine Data Answer
- Estimate district total population in 1998
- Pop1998 99,000 exp(.038) 125,410
- Estimate number of surviving infants in 1998
- 125,410 x (40/1000) x (1 - .080) 4615
- Estimate measles coverage rate
- Measles coverage 3000/4615 x 100 65
23Case Study 1 Immunization Coverage from Facility
Data
- Estimate total population in 2003
- Calculate coverage for DTP1, DPT3, and measles
vaccine in 2003 - Evaluate trends in coverage
- Estimate drop-out rates
- Analyze the problems in 2003
- Is coverage low or falling?
- What are possible causes?
- What are the differences in coverage in different
areas? - What action can managers take if coverage
- data indicate problems?
24Challenges in Estimating Coverage from Routine
Data
- Limited knowledge of target pop/denominators
- Low timeliness completeness of reporting
- Poor data quality
- Lack of written standard reporting procedures
- No systematic supervision on data management
- Dual reporting systems (EPI, HMIS)
- Inclusion of data from private sector
25Assessing Reliability of Routine Coverage
Indicators
- Understand how denominators are derived
- Understand the process of collecting the
information - Look for inconsistencies and surprises
26Assessing Reliability of Routine Coverage
Indicators
- Look for reliable data from other sources to use
as a basis for comparison - Cross-check
27Estimating Coverage from Survey Data
28Survey Tools for Coverage Estimation
- WHO-EPI surveys
- Lot quality coverage surveys
- Large-scale population-based surveys
- USAID Demographic and Health Surveys
- UNICEF Multiple Indicator Cluster Survey
- Arab League PAPCHILD surveys
- CDC Reproductive Health Surveys
- Seventy-five household survey
- Knowledge-Practice-Coverage Surveys
- Other local surveys
29How Do Administrative Data Compare With Survey
Data?
30Reconciling Coverage Estimates From Different
Data Sources
- Age group geographic scope
- Health cards versus recall
- Different sources for different purposes
- Not all coverage data can be compared in
constructive way - Differences in inclusion of private sector
- Selectivity
31On-line Resource STATcompiler
- Innovative online database tool
- Allows users to select numerous countries and
hundreds of indicators to create customized
tables that serve specific needs - Accesses nearly all population and health
indicators published in DHS final reports - http//www.measuredhs.com/statcompiler
32STATcompiler
33On-line Resource DOLPHN
- DOLPHN Data Online for Population, Health and
Nutrition - Online statistical data resource
- Quick access to frequently used indicators from
multiple sources, including - DHS, BUCEN, CDC, UNAIDS, UNESCO, UNICEF, World
Bank, WHO - www.phnip.com/dolphn
34Advantages and Disadvantages of Routine-based
Coverage
- Advantages
- Provides information on more timely basis
- Makes use of data routinely collected
- Can be used to detect and correct problems in
service delivery - Disadvantages
- Denominator errors
- Poor quality reporting
35Advantages and Disadvantages of Survey-based
Coverage
- Advantages
- Avoids problems with denominators
- Includes information from non-reporting
facilities - Disadvantages
- Coverage survey has low precision
- Larger standard errors at sub-national levels
- Irregular and expensive
- Survey timing may affect coverage rates
36Case Study 2 Estimating Vitamin A Coverage
- Calculate coverage from routine data
- Use tally sheets to determine number of children
who received vitamin A compared to target
population - Compare coverage estimates from routine data
with estimates from survey data - Estimate missed opportunities
37References
- WHO. 1999a. Indicators to Monitor Maternal
Health Goals Report of a Technical Working
Group, Geneva, 8-12 November 1993. Division of
Family Health Geneva WHO. - WHO. 1999b. Reduction of Maternal Mortality A
Joint WHO, UNFPA, UNICEF, World Bank Statement.
Geneva WHO. - WHO (2002) Increasing Immunization at the Health
Facility Level. Geneva, Switzerland World
Health Organization