Use of mixed methods in the evaluation of suicide prevention strategies and interventions - PowerPoint PPT Presentation


PPT – Use of mixed methods in the evaluation of suicide prevention strategies and interventions PowerPoint presentation | free to download - id: 52a5bd-ODQ3Z


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation

Use of mixed methods in the evaluation of suicide prevention strategies and interventions


Use of mixed methods in the evaluation of suicide prevention strategies and interventions Stephen Platt Qualitative research and suicide Seminar, Cardiff University ... – PowerPoint PPT presentation

Number of Views:76
Avg rating:3.0/5.0
Slides: 46
Provided by: EmmaHa3
Learn more at:


Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Use of mixed methods in the evaluation of suicide prevention strategies and interventions

Use of mixed methods in the evaluation of suicide
prevention strategies and interventions
  • Stephen Platt
  • Qualitative research and suicide
  • Seminar, Cardiff University, 2 July 2007

Structure of presentation
  • Mixed methods research
  • Suicide in Scotland trends and context
  • Choose Life strategy and action plan
  • Evaluation plan
  • Selected main findings
  • Selected recommendations

Mixed methods research
  • Adoption of a research strategy involving more
    than one type of research method
  • May be mix of qualitative and quantitative
    methods or mix of quantitative methods or mix of
    qualitative methods
  • Increasing use of mixed methods strategies,
    especially combining qualitative and quantitative
  • Many reasons why this is occurring

Why the increasing popularity of mixed methods
research? (1)
  • Opportunity for skills enhancement
  • Broadening methodological repertoire mitigates
    vs. trained incapacities (Reiss)
  • Encourages thinking outside the box
  • Cross-national research (e.g. EU) provides
    increased opportunities for mixed methods

Why the increasing popularity of mixed methods
research? (2)
  • Fits with political currency accorded to
    practical enquiry that speaks to policy/ makers
    and informs practice
  • Whole industry (Brannen) of mixed methods
    research created around evidence-based policy and
    in policy evaluation
  • Increasing emphasis upon dissemination
  • Researchers need to communicate in double speak
    (Brannen) technical/specialised language of
    research and popular language that can easily
    communicate findings/messages to users
  • Words as important as numbers in writing up

Misconceptions of the other
  • Quantitative attitudes ? qualitative
  • Too context specific
  • Selection of data to fit preconceptions
  • Unrepresentative samples/examples
  • Unwarranted claims
  • Qualitative attitudes ? quantitative
  • Overly simplistic
  • Decontextualised
  • Reductionist in terms of generalisations
  • Failing to capture subjective meaning

Rationales underlying choice of method(s) (1)
  • Paradigms/philosophical assumptions
  • Qualitative and quantitative research seen as
    intrinsically different (particularly with regard
    to philosophical traditions)
  • But surveys not necessarily conducted on basis of
    positivist assumptions and qualitative
    researchers using participant observation often
    work in realist tradition
  • Micro-level emphasises subjective
    interpretations. Macro-level concerned with
    larger patterns/ trends and seeks structural
    explanations. But all aim to understand
    individuals in society. Methods need to be
    congruent with this quest.

Rationales underlying choice of method(s) (2)
  • Pragmatics
  • Commonplace to argue that methods should be
    appropriate (and subordinate) to research Q
  • But usually many research Qs. Some may be
    underpinned by realist assumptions, others by
    interpretevist assumptions.
  • And practicalities of research process may change
    original intention anyway, with outcomes of
    research taking precedence
  • Mixed methods believed/claimed to produce
    better outcomes than reliance on single method
    (pragmatic orientation linked to emphasis upon
    policy/practice application)

Rationales underlying choice of method(s) (3)
  • Politics
  • Political rationales for using mixed methods
  • Example addressing condition of women in society
    requires use of large-scale quantitative data
    (structural) as well as in-depth qualitative data
    (personal experience/perspective) in order to
    understand/expose gendered inequalities
  • Crucial issue is the purpose to which methods are
    put rather than the methods per se

Combining methods in research process context of
  • Stage of data analysis and interpretation
  • Data derived from different methods cannot be
    added together to produce unitary reality
  • Possible outcomes when methods are combined
  • Corroboration (triangulation) (same findings
    from different methods)
  • Elaboration (one method exemplifies how findings
    apply in particular cases)
  • Complementarity (findings differ but together
    synergistically they generate insights)
  • Contradiction (findings from different methods

Mixed methods designs key dimensions
  • Logic of enquiry inductive (aimed at discovery)
    or deductive (aimed at hypothesis testing)
  • No one-to-one correspondence between one
    particular logic of enquiry and one type of
  • (If mixed method approach is warranted) the
    ordering of methods needs to be considered
    sequential or simultaneous?
  • How dominant is a particular method going to be
    (consider scarce resources)?

Possible permutations of mixed methods designs
  • Examples of simultaneous designs
  • QUAL quan
  • QUAN qual
  • QUAL qual
  • Examples of sequential designs
  • Qual ? QUAN
  • QUAL ? quan
  • Quan ? QUAL
  • QUAN ? qual
  • Qual ? QUAL

Suicide rates across the world (2002)
Age-standardised suicide rates, by country,
1991/93-2002/04, males
Age-standardised suicide rates, by country,
1991/93-2002/04, females
Intentional self harm undetermined deaths,
Scotland, 15 years, 1970-2005
Scotlands national suicide prevention strategy
action plan Choose Life
  • Launched in December 2002
  • Major element of Scottish Executives work on
    health improvement and mental health
  • Plan being implemented in phases
  • Budget for phase 1 (April 2003-March 2006) was
  • Additional 8.4m allocated for first two years of
    phase 2 (2006-08)
  • Overall aim to reduce suicide rate in Scotland
    by 20 in 2013 (cf 2002)

Choose Life national and local infrastructure
  • Designated National Implementation Support Team
    (NIST) co-ordinates and supports national
    development and implementation
  • NISTs core functions include awareness
    raising/campaigning working with the media
    development/dissemination of information and
    knowledge and guiding and supporting local
  • In each local authority Choose Life action plans
    have been developed by the Community Planning
    Partnership (CPP)
  • National and local budgetary allocation

Evaluation main objectives
  • Assess whether sustainable infrastructure is
    being developed nationally and locally to support
    achievement of Choose Life strategy
  • Measure and review progress towards
    implementation of Choose Life milestones
  • Examine whether and how Choose Life is
    stimulating effective forms of practice
  • Provide detailed recommendations to guide the
    next phase of the action plan

Theory-based evaluation
  • Theory-driven approaches are intended to address
    needs of programme implementers
  • Aims to articulate and test the explicit and
    implicit theories that shape the design and
    planned implementation of a programme
  • Attempts to determine
  • whether a programme has been delivered as
  • what aspects of the programme work, for whom and
    in what circumstances/contexts/settings
  • Theory of change a systematic and cumulative
    study of the links between activities, outcomes
    and contexts (Weiss)

Theory-based evaluation the role of context
  • Context as a powerful ingredient in programme
    evolution and success not a variable to be
    controlled for
  • Social programmes cant be tested/rolled out in
    laboratories, so getting to grips with the
    interface between context and intervention is of
    fundamental importance
  • Qualitative approaches can do this to a degree
    but rarely in a way that address concerns about
    programme impact

Theory-based evaluation the process
  • Through a collective/collaborative process, the
    evaluator encourages programme stakeholders to
    articulate at the earliest possible stage
  • The programmes rationale
  • Its intended outcomes
  • The activities that will be implemented to
    achieve these
  • Contextual factors and their influence
  • The ensuing theory of change can be used to
    improve programme planning and of make evaluation

Main methods
  • Two electronic surveys of local co-ordinators
  • Two rounds of interviews with key informants at
    national level (including NIST)
  • Detailed and in-depth exploration of theories of
    change in 8 selected local areas (using
    interviews, workshops, observation, documentary
  • Two workshops with local coordinators and
    national informants

Co-ordinator surveys
  • Covering range of areas
  • (Progress towards) Local vision for change
  • Progress in the development of the local
  • Resource allocation and generation
  • Examples of innovative and effective practice
  • Monitoring and evaluation
  • Sustainability and mainstreaming
  • Collection of data on suicide and deliberate
  • Reflections on national support and on local
  • Open and closed questions
  • Rating scales to measure satisfaction with
    national action towards achievement of milestones
    and to review local implementation progress

Interviews with key informants
  • Key elements of semi-structured interviews
  • Considering progress towards the achievement of
    the national milestones set out in Choose Life
  • Understanding the criteria applied at the
    national level to assess local implementation
  • Considering the quality of collaboration between
    the NIST and major national agencies for the
    achievement of Choose Life objectives.
  • Interviews with members of NIST and sample of key
    national informants (e.g. ChildLine, Samaritans,
    SAMH, NUJ)
  • Additional component exploration of the NIST
    story via individual interviews and joint

Local area case studies (1)
  • Case studies were main vehicles for
  • exploring implementation process
  • identifying overall theoretical framework within
    which local suicide prevention teams worked
  • Representative sample selected by
  • Geographical type (rural/remote, urban and mixed)
    1º criterion
  • Local suicide rate, focus on priority groups,
    interaction between national and local levels,
    and approaches to coordination 2º criteria
  • Two fieldwork visits 6 months apart

Local area case studies (2)
  • Interviews with case study informants
  • Purposive sampling 4 key informants involved in
    decision-making process (e.g. members Choose Life
    partnership responsible for priority setting) and
    2 representatives involved in Choose Life funded
  • 16 project activities selected for more detailed
    examination (2 per case study site)
  • Intended to provide representative selection
    across Choose Life priority groups and objectives
  • Selection negotiated with coordinators
  • Interviews conducted with project representative,
    usually project lead/manager

Local area case studies (3)
  • Observational activities and collection of
  • Observation of Choose Life events (usually
    partnership meetings but also training days,
    evaluation days and practitioner fora)
  • Key documents included minutes of Choose Life
    partnership meetings, locality reports on
    progress, reports of previous needs assessment
  • Evaluation team aimed to understand theories of
    change at overall programme level as well as at
    project level

Local area case studies (4)
  • Changes to approach
  • Workshops replaced individual interviews with
    stakeholders at second fieldwork visit
  • Provided opportunity for joint testing of, and
    reflection on, local area theories of change
  • Local participants encouraged to assess progress
    towards Choose Life objectives and milestones,
    and how this could be demonstrated
  • Problems created as result of mixed levels and
    responsibilities of participants (e.g. strategic
    versus operational)

National workshops (1)
  • Two workshops one year apart brought together
    evaluation team, NIST, local coordinators and
    several key stakeholders
  • 1st workshop aimed to
  • develop understanding of the different models of
    national and local actions and activities being
    put in place
  • explore the evolving relationship between the
    approaches of the centre and of local areas
  • identify ways in which progress could be measured.

National workshops (2)
  • 2nd workshop aimed to
  • Review progress and learning in relation to
    objectives of Choose Life
  • Test out key themes emerging from evaluation
  • Identify future priorities for development,
    support required, measures of progress and
    outcome, and implications for information
    collection and research

Data analysis (1)
  • Continuous (iterative) process throughout study
  • Evaluation database designed and used to store
    (and retrieve) data on all 32 local areas
  • Findings from each element of data collection
    (case studies, workshops, national interviews,
    surveys) written up in detailed reports which
    were then used for comparative analysis
  • Data analysed according to predefined themes
    (e.g. sustainability, partnerships)
  • and themes also developed from inductive
    analysis conducted at each phase of evaluation

Data analysis (2)
  • Analytic framework developed to guide team
    through the evaluation
  • This was expanded and changed according to themes
    emerging from data
  • Analysis primarily drew on charting method of
    systematically handling complex datasets by
    drawing out dimensions relating to each theme
    across all cases

Evaluation main findings
  • Sustainable infrastructures for implementation
  • Allocation and use of resources
  • Innovative practice and use of evidence
  • Sustainability
  • Decision making processes and learning

Sustainable infrastructures for implementation
  • Demonstrable progress made by NIST
  • CPP successes but not as evident in less
    mature partnerships and in engaging with
    clinical services
  • Various models of local coordination developed
  • Preference for a dedicated (full-time)
    coordination post
  • but evaluation unable to demonstrate the
    superiority of this model

Allocation and use of resources
  • CPPs have attracted considerable additional
    investment at local level and in-kind
  • but areas have not been equally successful in
    raising additional funding
  • and there has been unnecessary duplication of
    effort at local level
  • Choose Life has stimulated a considerable amount
    of activity relating to self-harm
  • but local areas have different understandings
    of high risk suicidal behaviour and have
    adopted different responses to address the

Innovative practice and use of evidence
  • Many examples of locally defined innovative
  • and multiple sources of information and
    evidence used to inform local planning and
  • but research rarely used systematically

  • NIST identified several achievements in building
    a sustainable infrastructure for suicide
  • At local level, most success achieved in
    mainstreaming training activities

Decision making processes and learning
  • Local stakeholder consultation key approach to
    set implementation priorities
  • Short timescale to develop first action plan was
    a major challenge
  • National support for learning has been delivered
    through diverse routes
  • NIST has highlighted a strong commitment to
  • but a national framework for evaluation remains
    to be completed
  • In local areas different levels of priority and
    attention have been attached to evaluation.

Recommendations mainstreaming at national level
  • Incorporate Choose Life objectives and priorities
    into other policy streams/initiatives
  • Involve clinical services in population-based
    suicide prevention activities
  • Involve national voluntary sector organisations
    in awareness raising and campaigning
  • Engage in purposive innovation to test out,
    evaluate, learn and implement

Recommendations mainstreaming at local level
  • Using intelligence from a range of sources, as
    tools in planning for sustainability
  • Building in mechanisms to track and review
    progress towards objectives across policy areas
  • More focused targeting of action is required

Recommendations self-harm
  • More consideration to be given in phase 2 to the
    integration of self-harm into Choose Life
  • The strategy should continue to encompass high
    risk self-harm
  • but the less serious component of self-harm
    cannot be ignored

Recommendations CPP
  • The CPP remains the most appropriate vehicle for
    developing strategy and overseeing delivery in
    relation to Choose Life at the local level
  • But its limitations should be recognised
  • Need to examine the necessary partnerships that
    have yet to be put in place
  • Priority should be given to establishing/building
    on effective links with clinical drug/alcohol
  • NIST should continue to work closely with CPPs to
    ensure that Choose Life budgets are fully spent
    on suicide prevention activities

Recommendations central coordination body
  • Some type of central coordination body will
    continue to be required in the immediate future
  • Key tasks provide national oversight, assess and
    support performance and ensure accountability at
    local level, promote learning and effective
    knowledge transfer, and co-ordinate action.
  • There should be a review of how the central
    coordinating function is delivered and where it
    is situated

Choose Life evaluation (first phase)research
  • University of Edinburgh
  • Stephen Platt (RUHBC)
  • Emma Halliday (RUHBC)
  • Margaret Maxwell (General Practice)
  • Scottish Development Centre for Mental Health
  • Joanne McLean
  • Allyson McCollam
  • Amy Woodhouse
  • London School of Economics
  • Dave McDaid (Health Social Care)
  • Glasgow University
  • Mhairi Mackenzie (Public Health Health Policy)
  • Avril Blamey (Public Health Health Policy)