QHR Conference: Banff, Canada: 2004 The quality of sustainable development:

1
QHR Conference Banff, Canada 2004 The quality
of sustainable development

• Evaluation at the edge of chaos
• Oliver Slevin
• University of Ulster, UK

Evaluation of a PPP healthcare project
2

• Addresses
• The evaluation of an essential health services
  package in rural Bangladesh that utilized Public
  Private Partnership (PPP) working
• Realistic evaluation as a context-bound approach
  that incorporates quantitative and qualitative
  methodologies
• The qualitative dimension in such research
• And is accompanied by a paper providing detail
  of the presentation content with all references

3
Words and numbers are of equal value, for, in
the cloak of knowledge, one is warp and the other
woof. It is no more important to count the sands
than it is to name the stars. Therefore let both
kingdoms live in peace. Juster, N. (1962) The
phantom tollbooth. London Collins. p.74
4
Between heart and mind We approach the
problems of human psychology as humans, and it
seems a pity to waste that advantage. Midgley,
M. (1981). Heart and mind the varieties of moral
experience. London Methuen.
5
Narratives not only help to humanize aliens,
strangers and scapegoats but also to make each
one of us into an agent of love sensitive to
the particular details of others pain and
humiliation. Rorty, R. (1991). On
ethnocentrism A reply to Clifford Geertz. In R.
Rorty, Objecivity, relativism and truth
philosophical papers, Vol. I. New York Cambridge
University Press .
6

• TOWARDS DEVELOPMENT AID EFFICIENCY The PPP
  project
• Investment in specific approaches that will
  establish sustainable health improvement, in the
  sense that following the initiative the
  arrangements will be economically viable after
  aid is no longer in place.
• An emphasis on partnership working between the
  public, private and voluntary (including
  Non-Government Organisations NGOs), so that
  resources are pooled and more effectively
  targeted.
• A higher degree of technical assistance and
  monitoring during implementation.
• A greater emphasis on evaluation, that is
  extreme in its adherence to a logical framework
  That is, evaluates in a logical and linear
  fashion the achievement of interim and ultimate
  goals in terms of objective criteria (thus the
  term LogFrame analysis).

7

• Bangladesh is
• The most densely populated country in South Asia
• 50 of the 125m population are at or below the
  poverty line and 35 are described as in extreme
  poverty or ultra poor
• Maternal Mortality rate is one of highest in
  world at 392 per 100, 000 births
• Infant Mortality Rate is 66 per 1000 live births
• Only 12 births are attended by trained personnel
• Low birth weights are second only to India
• Stunted growth in the under-5s is second only to
  North Korea

8
Population characteristics are enjoined by an
unfriendly environment with high levels of
pollution and risks of disaster particularly from
flooding. Most of the pernicious tropical
diseases (including Malaria) are endemic AIDS is
a threat and old diseases such as TB and Leprosy
are on the increase.
9
The development response an Essential Services
Package (ESP)

• Reproductive health care
• Child health care
• Communicable diseases control
• Limited curative care
• Behavioural change communication
• Delivered on principles of Efficiency, Safety,
  Equity and Resilience (sustainability)

10

• PPP AT START
• A PILOT SCHEME TO DEVELOP PARTNERSHIP BETWEEN
• THE PUBLIC (HEALTH) SERVICES
• And
• PRIVATE FOR-PROFIT PROVIDERS

11
But Private provider problems

• An amorphous group
• Mainly unqualified (pharmacists, quacks)
• Mainly men (so women treated through male
  proxies)
• Interested only in for-profit
• Providing services of dubious quality

12
And Public limitations

• Services mainly absent, many facilities unused
• High levels of unofficial absence (as high as 74
  of time for doctors)
• Lack of qualified personnel
• Nurses (mainly women) excluded
• Demand for unofficial fees the norm

13
The PPP Project therefore developed instead as a
community empowerment initiative - described
within the project as a Public-Community
Partnership or also as Grassroots PPP. The
community would, with technical support from
Nicare (the facilitating Development agency), set
up its own local healthcare services best suited
to its needs.
14
PPP in Operation There is only one operational
model. The model incorporates ? Community
Schemes ? Funding and Commissioning
Partnerships ? Health Provider
partnerships It is being introduced with the
support of one of three groupings - Donor/contrac
tors (Nicare) - NGOs - Local Government

15
LIMITED EVALUATIVE RESEARCH
16
Evaluation overview
(Annexes are contained only in the full PPP
Policy Review report)
17
(No Transcript)
18

• EVALUATION OUTCOMES
• In Logical Framework (LogFrame) terms, the
  project failed to meet some of its main
  sustainability, management, quality and
  partnership criteria.
• 2. The evaluations that had previously taken
  place, limited to LogFrame orientation, valued
  only the imported goals
• 3. Because of this, wider contextual issues had
  not been taken fully into account, so that
  unrealistic expectations were not met.
• The exclusion of voices within the context
  resulted in a project that, though modified to
  some extent in response to emerging local
  circumstances, lacked participation in terms of
  project design, delivery and evaluation.
• Significant voices excluded from the scheme
  were women, the ultra-poor, and private providers
  (the latter being the main traditional source of
  healthcare).
• Significant voices included were more affluent
  men (who dominated local schemes through
  political capture, the PPP Project Team, and
  Government officials (again largely men).

19
(No Transcript)
20
EXCLUDING STAKEHOLDER VOICES FROM
DESIGN EXCLUDING STAKEHOLDER VOICES FROM
IMPLEMENTATION EXCLUDING STAKEHOLDER VOICES FROM
EVALUATION EQUALS FAILURE! AN EXTENSION OF
EVALUATION, IN TERMS OF THEORY AND ACTION, WAS
THEREFORE REQUIRED
21
RELEVANT THEORETICAL UNDERPINNINGS CAN BE SEEN TO
EMERGE AS DRIVING FORCES
22
I The idea of critical consciousness and
community empowerment as a process of
conscientization, that liberates the voice of the
previously unheard. Aware-ness Freire, P.
(1993). Pedagogy of the oppressed. London
Penguin
23
II The idea that within each situation there are
interpretive communities, each attributing
meaning and with different values and
goals Other-ness Yanow, D. (2000) Conducting
interpretive policy analysis. Thousand Oaks, CA
Sage Publications.
24
III The idea that through dialogue , and as
co-equals, differences can be acknowledged and
consensuses reached Together-ness Habermas, J.
(1987). The theory of communicative action.
Boston Beacon.
25
Adapted from Yanow (2000)
26
SEEKING THE UNHEARD VOICES a movement from the
homophonic voice (in this case, we might term
this a Western lens) to a polyphonic voice that
orientates itself responsibly toward the words
and voices of others the extent, in short, to
which it adopts otherness as a value. Tarulli,
D. (2000). Identity and otherness. Narrative
Inquiry, 10, 1, 111-126.
27

• Richard Kearney on stories
• Plot (Mythos) The human existence and experience
  that seeks a narrative.
• Re-creation (Mimesis) The verbal recounting of
  what is experienced, in terms of its eidetic or
  essential elements.
• Release (Catharsis) The way in which the story
  transports the listener into sympathetic
  alignment with the teller.
• Wisdom (Phronesis) The practical wisdom the
  listener gains as a consequence of the cathartic
  alignment.
• Ethics (Ethos) The moral call from the story,
  that demands an ethical response or indeed a
  decision of non-response.

28
THE LISTENING GUIDE Plot What is taking place
in the story. Self How the individual as a
feeling, thinking, acting I is enclosed within
the story. Supportive others The positive
sustaining relationships with sympathetic
others. Devaluing others The relationships that
would and oppress. Brown, L. Mikel and Gilligan,
C. (1992). Meeting at the crossroads womens
psychology and girls development. New York
Ballentine Books.
29
MARRIED AND AT HOME IN RURAL BANGLADESH She is
one of the ladies in one of our community schemes
she is unwell, she would call for help but
cannot (Mythos) Her husband has been purchasing
douches from the male village quack. She now
attends the health clinic. The health assistant
diagnoses thrush and prescribes one suppository
and sends her home. Her actual malady is
severe uterine prolapse (Mimesis) We experience,
from our contact with her, the magnitude of her
plight (Catharsis) It becomes clear to us that
the system put in place does not address the
social and cultural influences that construct
such circumstances (Phronesis) By becoming
aware, there is an immediate ethical demand to
respond appropriately to explore the situation,
to address the quality deficit (Ethos)
30
Life is pregnant with stories. It is a nascent
plot in search of a midwife. For inside every
human being there are lots of little narratives
trying to get out. Kearney op cit. (p. 130)
31
(No Transcript)
32
Nature is not just like a book nature itself is
a book, and the manmade book its analogue.
Reading the man-made book is an act of midwifery
it is an act of incarnation. Reading is a
somatic, bodily act of birth attendance
witnessing the sense brought forth by all things
encountered by the pilgrim through the
pages. Illich, I. (1993). In the vineyard of
the text. Chicago University of Chicago Press.