Arianna Legovini, Lead Impact Evaluation Specialist

1

• Arianna Legovini, Lead Impact Evaluation
  Specialist
• Edit V. Velenyi, MIEP Coordinator
• Africa Impact Evaluation Initiative (AIM)
• Malaria Impact Evaluation Program (MIEP)

2

• The impact of malaria extends far beyond the
  realm of public health and exacts a heavy toll on
  human and economic prosperity...
• Malaria kills an estimated 1 million people every
  year
• Costs Africa US12 billion annually in lost GDP
• 90 percent of malaria deaths worldwide are in
  Africa
• Yet malaria is both preventable and treatable
• Universal coverage would bring the scourge to a
  halt

3

• Strong epidemiological experimental evidence
  prevention and treatment work
• Lacking behavioral and economic evidence
• What will induce people to have their children
  sleep under a net? Or take the child for
  treatment within 24 hours?
• What are the socio-economic returns to malaria
  prevention and treatment?
• Labor participation and productivity, child
  cognitive development
• Which interventions are most cost-effective?
• Private delivery mechanisms
• School based malaria interventions

4

• AIM - Africa Impact Evaluation Initiative
• 80 Experimental or Quasi Experimental
  Evaluations
• MIEP - Malaria Impact Evaluation Program
• Inaugural Workshop, Cape Town May 2007
• Second Workshop Asmara Feb 2008
• Countries Eritrea, DRC, Kenya, India, Nigeria,
  Senegal, Zambia
• Areas of investigation
• Alternative delivery mechanisms and PPP
• Complementary interventions (IRS, test kits)
• Preventive treatment (IPTs)
• Innovative financing for ACTs

5

• Improve quality of operations
• Separate process from quality of intervention
• Test alternatives and inform design in real time
• Increase project effectiveness
• Answer the so what questions
• Build relationship with client
• Come with options not solutions
• Find out together what works best
• Assist client adopt better way of doing business
  and taking decisions

6

• Quality assurance
• Client-driven operationally relevant
• Feedback loop into policy design

Lead researcher Field Coordinator
Working Groups
Biometrics
Technical Advisory Group Researchers Specialists
Cognitive
MIEP Research Team
Socio-Economic
Knowledge, Attitudes and Practice
MBP Project Team
Coordinating Unit IE Leads Sector Lead Program
Coordinator Team Support
Client Government
Cost-Effectiveness
7

• Alternative delivery mechanisms and PPP
• Complementary interventions (IRS, test kits)
• Preventive treatment (IPTs)
• Innovative financing for ACTs

8
MIEP Portfolio BNPP Recipients

• DRC
• Nigeria

9

• Number one public health problem in Nigeria
• 99 of the population in endemic areas (MARA/ARMA
  2002)
• High Federal and States Governments commitment
  to controlling the disease
• 2004 National Economic Empowerment and
  Development Strategy
• development of strategic plans for malaria
• Presidential Initiative for Accelerated
  Achievement of the MDGs
• malaria control as a key component of
  health-related MDGs
• 2004 National Health Policy
• malaria control as a priority health program

10
(No Transcript)
11
(No Transcript)
12

• Explore innovative modes of delivery and
  complementary interventions to address
  limitations of the health system
• Weak / insufficient public sector delivery
• Human resource constraints
• Heterogeneity in the quality of private sector
  delivery
• Insufficient coordination between the public and
  private sectors

13

• Consultations and Capacity Building National
  and States
• Cape Town Inaugural Workshop May 2007
• Abuja Design Workshops
• MBP7 August 2007
• MBP 4 September 2008
• State Field Visits
• Anambra October 2008
• Gombe October 2008
• Akwa Ibom December 2008
• Jigawa December 2008

14
Nigeria Malaria IE Cycle and Timeline
15
(No Transcript)
16
JIGAWA
GOMBE
ANAMBRA
AKWA IBOM
17
1. FMOH / NMCP Program Coordinator
STEWARDSHIP National Policy Making Resource
Allocation System Harmonization
Evidence Based Policy Making
2. STATE MOH Program Managers
State-Level Policy Making Implementation
Decision
Supply
3. LGA MBP Focal Person
Program Implementation Service Delivery
Reporting
Wards WDC
Wards WDC
Wards WDC
ACT Supply
(LLIN BCC)
ME
Public Facility (K)
Private PMV T1
Community RMM T2
Vulnerable Target Group U5 Children Pregnant
Women
IE
18

• Counterfactual Public Sector (K)
• Private Sector Intervention Arm PMV (T1)
• What is the added value of using trained Patent
  Medicine Vendors (PMVs) to increase timely
  treatment of uncomplicated malaria with ACTs?
• Community-Based Intervention Arm RMM (T2)
• What is the added value of using trained Role
  Model Mothers (RMMs) to increase
• increase timely treatment of uncomplicated
  malaria with ACTs?
• effective coverage of LLINs by the vulnerable
  target groups?
• Cross-Over / Factorial Design (T1 T2)
• What is the joint effect of using trained PMVs
  and RMMs ?

19

• Uniform Intervention Across States
• Supply Side Training
• Variations
• Supply Side Incentives for PMVs and RMMs
• Monetary v. Non-Monetary
• Demand Side Approach Communication
• BCC Component in Training for RMMs to Improve Use
  LLINs
• BCC (Community-Based Orientation)

20

• Effect of Trained PMVs and RMMs on coverage and
  use of ACTs
• Target Population - U5 Children 470,770
• MB Program Funds 6 million
• Design Covers whole state, including all 11 LGAs
  and 114 Wards
• Intervention Arms 1) PMV 2) RMM 3) PMV RMM 4)
  Control
• Intensity 114 Trained PMVs and 114 Trained RMMs
• HH Survey 2,280 HH Community Survey 228
  Facility Survey MIS

RMM in 57 Communities
PMV in 57 Wards
PMV in 57 Wards RMM in 57 Communities
Pure Control in 57 Wards
21

• Effect of Trained PMVs RMMs on coverage and use
  of ACTs, and RRMs on LLIN use
• Target Population U5 (836,400) and Pregnant
  Women (209,100)
• MB Program Funds 9.5 million
• Design for State (21 LGAs and 329 Wards)
• Intervention Arms 1) PMV 2) RMM 3) PMV RMM 4)
  Control
• Intensity 172 Trained PMVs and 172 Trained RMMs
  (2 per Ward)
• HH Survey 2,064 HH Community Survey 172
  Facility Survey MIS

RMM in 43 Wards
PMV in 43 Wards
PMV RMM in 43 Wards
Pure Control of 43 Wards
22

• Effect of incentives for PMVs RMMs on coverage
  and use of ACT, and RRMs on LLIN use.
• Effect of BCC on coverage and use of ACTs, and
  RMMs on LLIN use.
• Target Population U5 (784,042) Pregnant Women
  (196,010)
• MB Program Funds 8.5 million
• Design for State (31 LGAs and 329 Wards)
• Arms 1) Incentives 2) BCC 3) Control
• 329 Trained PMVs and 329 Trained RMMs (Using 1
  each for Focal Village per Ward)
• HH Survey 3,290 HH Community Survey 329
  Facility Survey MIS

Incentives 110 Focal Villages (1 FV per Ward)
Control (Training Only) 109 Villages (1FV per
Ward)
BCC 110 Focal Villages (1 FV per Ward)
23

• Upgrade Routine MIS Case Management Reporting
• Training and Rollout of Case Management Tool to
  Health Facilities
• Management Tool (McKinsey)
• Costing Tool
• Cost of Standard Delivery
• Cost of Innovative Arms (Incremental Costs)
• System Start Up (Intervention Specific ME Tool)
• Intervention Cost (Commodity and Training)
• Management Cost (Operating the New System)
• Third Party Audit
• Focal Person and State Manager
• Contracting of NGO/CSO

24

• Routine Reporting
• Upgraded MIS
• Customized MIS Tools
• Random Audits
• Household Survey
• Baseline CWIQ (November - December 2008)
• Developed Malaria Section for CWIQ
• Follow up Dedicated Survey (November December
  2009)
• Community Surveys
• Component of HH Survey
• Facility Surveys
• SFH Facility Census (November 2008)
• Facility Survey (November 2009)