Use of mixed methods in the evaluation of suicide prevention strategies and interventions - PowerPoint PPT Presentation

Loading...

PPT – Use of mixed methods in the evaluation of suicide prevention strategies and interventions PowerPoint presentation | free to download - id: 52a5bd-ODQ3Z



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Use of mixed methods in the evaluation of suicide prevention strategies and interventions

Description:

Use of mixed methods in the evaluation of suicide prevention strategies and interventions Stephen Platt Qualitative research and suicide Seminar, Cardiff University ... – PowerPoint PPT presentation

Number of Views:76
Avg rating:3.0/5.0
Slides: 46
Provided by: EmmaHa3
Learn more at: http://www.cardiff.ac.uk
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Use of mixed methods in the evaluation of suicide prevention strategies and interventions


1
Use of mixed methods in the evaluation of suicide
prevention strategies and interventions
  • Stephen Platt
  • Qualitative research and suicide
  • Seminar, Cardiff University, 2 July 2007

2
Structure of presentation
  • Mixed methods research
  • Suicide in Scotland trends and context
  • Choose Life strategy and action plan
  • Evaluation plan
  • Selected main findings
  • Selected recommendations

3
Mixed methods research
  • Adoption of a research strategy involving more
    than one type of research method
  • May be mix of qualitative and quantitative
    methods or mix of quantitative methods or mix of
    qualitative methods
  • Increasing use of mixed methods strategies,
    especially combining qualitative and quantitative
    approaches
  • Many reasons why this is occurring

4
Why the increasing popularity of mixed methods
research? (1)
  • Opportunity for skills enhancement
  • Broadening methodological repertoire mitigates
    vs. trained incapacities (Reiss)
  • Encourages thinking outside the box
  • Cross-national research (e.g. EU) provides
    increased opportunities for mixed methods
    research

5
Why the increasing popularity of mixed methods
research? (2)
  • Fits with political currency accorded to
    practical enquiry that speaks to policy/ makers
    and informs practice
  • Whole industry (Brannen) of mixed methods
    research created around evidence-based policy and
    in policy evaluation
  • Increasing emphasis upon dissemination
  • Researchers need to communicate in double speak
    (Brannen) technical/specialised language of
    research and popular language that can easily
    communicate findings/messages to users
  • Words as important as numbers in writing up
    research

6
Misconceptions of the other
  • Quantitative attitudes ? qualitative
  • Too context specific
  • Selection of data to fit preconceptions
  • Unrepresentative samples/examples
  • Unwarranted claims
  • Qualitative attitudes ? quantitative
  • Overly simplistic
  • Decontextualised
  • Reductionist in terms of generalisations
  • Failing to capture subjective meaning

7
Rationales underlying choice of method(s) (1)
  • Paradigms/philosophical assumptions
  • Qualitative and quantitative research seen as
    intrinsically different (particularly with regard
    to philosophical traditions)
  • But surveys not necessarily conducted on basis of
    positivist assumptions and qualitative
    researchers using participant observation often
    work in realist tradition
  • Micro-level emphasises subjective
    interpretations. Macro-level concerned with
    larger patterns/ trends and seeks structural
    explanations. But all aim to understand
    individuals in society. Methods need to be
    congruent with this quest.

8
Rationales underlying choice of method(s) (2)
  • Pragmatics
  • Commonplace to argue that methods should be
    appropriate (and subordinate) to research Q
  • But usually many research Qs. Some may be
    underpinned by realist assumptions, others by
    interpretevist assumptions.
  • And practicalities of research process may change
    original intention anyway, with outcomes of
    research taking precedence
  • Mixed methods believed/claimed to produce
    better outcomes than reliance on single method
    (pragmatic orientation linked to emphasis upon
    policy/practice application)

9
Rationales underlying choice of method(s) (3)
  • Politics
  • Political rationales for using mixed methods
  • Example addressing condition of women in society
    requires use of large-scale quantitative data
    (structural) as well as in-depth qualitative data
    (personal experience/perspective) in order to
    understand/expose gendered inequalities
  • Crucial issue is the purpose to which methods are
    put rather than the methods per se

10
Combining methods in research process context of
justification
  • Stage of data analysis and interpretation
  • Data derived from different methods cannot be
    added together to produce unitary reality
  • Possible outcomes when methods are combined
  • Corroboration (triangulation) (same findings
    from different methods)
  • Elaboration (one method exemplifies how findings
    apply in particular cases)
  • Complementarity (findings differ but together
    synergistically they generate insights)
  • Contradiction (findings from different methods
    conflict)

11
Mixed methods designs key dimensions
  • Logic of enquiry inductive (aimed at discovery)
    or deductive (aimed at hypothesis testing)
  • No one-to-one correspondence between one
    particular logic of enquiry and one type of
    method
  • (If mixed method approach is warranted) the
    ordering of methods needs to be considered
    sequential or simultaneous?
  • How dominant is a particular method going to be
    (consider scarce resources)?

12
Possible permutations of mixed methods designs
  • Examples of simultaneous designs
  • QUAL quan
  • QUAN qual
  • QUAL QUAL
  • QUAL qual
  • Examples of sequential designs
  • Qual ? QUAN
  • QUAL ? quan
  • Quan ? QUAL
  • QUAN ? qual
  • Qual ? QUAL
  • QUAL ? QUAL
  • QUAL ? QUAL

13
Suicide rates across the world (2002)
14
Age-standardised suicide rates, by country,
1991/93-2002/04, males
15
Age-standardised suicide rates, by country,
1991/93-2002/04, females
16
Intentional self harm undetermined deaths,
Scotland, 15 years, 1970-2005
17
Scotlands national suicide prevention strategy
action plan Choose Life
  • Launched in December 2002
  • Major element of Scottish Executives work on
    health improvement and mental health
  • Plan being implemented in phases
  • Budget for phase 1 (April 2003-March 2006) was
    12m
  • Additional 8.4m allocated for first two years of
    phase 2 (2006-08)
  • Overall aim to reduce suicide rate in Scotland
    by 20 in 2013 (cf 2002)

18
Choose Life national and local infrastructure
  • Designated National Implementation Support Team
    (NIST) co-ordinates and supports national
    development and implementation
  • NISTs core functions include awareness
    raising/campaigning working with the media
    development/dissemination of information and
    knowledge and guiding and supporting local
    implementation
  • In each local authority Choose Life action plans
    have been developed by the Community Planning
    Partnership (CPP)
  • National and local budgetary allocation

19
Evaluation main objectives
  • Assess whether sustainable infrastructure is
    being developed nationally and locally to support
    achievement of Choose Life strategy
  • Measure and review progress towards
    implementation of Choose Life milestones
  • Examine whether and how Choose Life is
    stimulating effective forms of practice
  • Provide detailed recommendations to guide the
    next phase of the action plan

20
Theory-based evaluation
  • Theory-driven approaches are intended to address
    needs of programme implementers
  • Aims to articulate and test the explicit and
    implicit theories that shape the design and
    planned implementation of a programme
  • Attempts to determine
  • whether a programme has been delivered as
    intended
  • what aspects of the programme work, for whom and
    in what circumstances/contexts/settings
  • Theory of change a systematic and cumulative
    study of the links between activities, outcomes
    and contexts (Weiss)

21
Theory-based evaluation the role of context
  • Context as a powerful ingredient in programme
    evolution and success not a variable to be
    controlled for
  • Social programmes cant be tested/rolled out in
    laboratories, so getting to grips with the
    interface between context and intervention is of
    fundamental importance
  • Qualitative approaches can do this to a degree
    but rarely in a way that address concerns about
    programme impact

22
Theory-based evaluation the process
  • Through a collective/collaborative process, the
    evaluator encourages programme stakeholders to
    articulate at the earliest possible stage
  • The programmes rationale
  • Its intended outcomes
  • The activities that will be implemented to
    achieve these
  • Contextual factors and their influence
  • The ensuing theory of change can be used to
    improve programme planning and of make evaluation
    decisions

23
Main methods
  • Two electronic surveys of local co-ordinators
  • Two rounds of interviews with key informants at
    national level (including NIST)
  • Detailed and in-depth exploration of theories of
    change in 8 selected local areas (using
    interviews, workshops, observation, documentary
    analysis)
  • Two workshops with local coordinators and
    national informants

24
Co-ordinator surveys
  • Covering range of areas
  • (Progress towards) Local vision for change
  • Progress in the development of the local
    infrastructure
  • Resource allocation and generation
  • Examples of innovative and effective practice
  • Monitoring and evaluation
  • Sustainability and mainstreaming
  • Collection of data on suicide and deliberate
    self-harm
  • Reflections on national support and on local
    progress
  • Open and closed questions
  • Rating scales to measure satisfaction with
    national action towards achievement of milestones
    and to review local implementation progress

25
Interviews with key informants
  • Key elements of semi-structured interviews
  • Considering progress towards the achievement of
    the national milestones set out in Choose Life
  • Understanding the criteria applied at the
    national level to assess local implementation
  • Considering the quality of collaboration between
    the NIST and major national agencies for the
    achievement of Choose Life objectives.
  • Interviews with members of NIST and sample of key
    national informants (e.g. ChildLine, Samaritans,
    SAMH, NUJ)
  • Additional component exploration of the NIST
    story via individual interviews and joint
    workshop

26
Local area case studies (1)
  • Case studies were main vehicles for
  • exploring implementation process
  • identifying overall theoretical framework within
    which local suicide prevention teams worked
  • Representative sample selected by
  • Geographical type (rural/remote, urban and mixed)
    1º criterion
  • Local suicide rate, focus on priority groups,
    interaction between national and local levels,
    and approaches to coordination 2º criteria
  • Two fieldwork visits 6 months apart

27
Local area case studies (2)
  • Interviews with case study informants
  • Purposive sampling 4 key informants involved in
    decision-making process (e.g. members Choose Life
    partnership responsible for priority setting) and
    2 representatives involved in Choose Life funded
    activities
  • 16 project activities selected for more detailed
    examination (2 per case study site)
  • Intended to provide representative selection
    across Choose Life priority groups and objectives
  • Selection negotiated with coordinators
  • Interviews conducted with project representative,
    usually project lead/manager

28
Local area case studies (3)
  • Observational activities and collection of
    documentation
  • Observation of Choose Life events (usually
    partnership meetings but also training days,
    evaluation days and practitioner fora)
  • Key documents included minutes of Choose Life
    partnership meetings, locality reports on
    progress, reports of previous needs assessment
  • Evaluation team aimed to understand theories of
    change at overall programme level as well as at
    project level

29
Local area case studies (4)
  • Changes to approach
  • Workshops replaced individual interviews with
    stakeholders at second fieldwork visit
  • Provided opportunity for joint testing of, and
    reflection on, local area theories of change
  • Local participants encouraged to assess progress
    towards Choose Life objectives and milestones,
    and how this could be demonstrated
  • Problems created as result of mixed levels and
    responsibilities of participants (e.g. strategic
    versus operational)

30
National workshops (1)
  • Two workshops one year apart brought together
    evaluation team, NIST, local coordinators and
    several key stakeholders
  • 1st workshop aimed to
  • develop understanding of the different models of
    national and local actions and activities being
    put in place
  • explore the evolving relationship between the
    approaches of the centre and of local areas
  • identify ways in which progress could be measured.

31
National workshops (2)
  • 2nd workshop aimed to
  • Review progress and learning in relation to
    objectives of Choose Life
  • Test out key themes emerging from evaluation
  • Identify future priorities for development,
    support required, measures of progress and
    outcome, and implications for information
    collection and research

32
Data analysis (1)
  • Continuous (iterative) process throughout study
  • Evaluation database designed and used to store
    (and retrieve) data on all 32 local areas
  • Findings from each element of data collection
    (case studies, workshops, national interviews,
    surveys) written up in detailed reports which
    were then used for comparative analysis
  • Data analysed according to predefined themes
    (e.g. sustainability, partnerships)
  • and themes also developed from inductive
    analysis conducted at each phase of evaluation

33
Data analysis (2)
  • Analytic framework developed to guide team
    through the evaluation
  • This was expanded and changed according to themes
    emerging from data
  • Analysis primarily drew on charting method of
    systematically handling complex datasets by
    drawing out dimensions relating to each theme
    across all cases

34
Evaluation main findings
  • Sustainable infrastructures for implementation
  • Allocation and use of resources
  • Innovative practice and use of evidence
  • Sustainability
  • Decision making processes and learning

35
Sustainable infrastructures for implementation
  • Demonstrable progress made by NIST
  • CPP successes but not as evident in less
    mature partnerships and in engaging with
    clinical services
  • Various models of local coordination developed
  • Preference for a dedicated (full-time)
    coordination post
  • but evaluation unable to demonstrate the
    superiority of this model

36
Allocation and use of resources
  • CPPs have attracted considerable additional
    investment at local level and in-kind
    contribution
  • but areas have not been equally successful in
    raising additional funding
  • and there has been unnecessary duplication of
    effort at local level
  • Choose Life has stimulated a considerable amount
    of activity relating to self-harm
  • but local areas have different understandings
    of high risk suicidal behaviour and have
    adopted different responses to address the
    problem

37
Innovative practice and use of evidence
  • Many examples of locally defined innovative
    practice
  • and multiple sources of information and
    evidence used to inform local planning and
    activity
  • but research rarely used systematically

38
Sustainability
  • NIST identified several achievements in building
    a sustainable infrastructure for suicide
    prevention
  • At local level, most success achieved in
    mainstreaming training activities

39
Decision making processes and learning
  • Local stakeholder consultation key approach to
    set implementation priorities
  • Short timescale to develop first action plan was
    a major challenge
  • National support for learning has been delivered
    through diverse routes
  • NIST has highlighted a strong commitment to
    evaluation
  • but a national framework for evaluation remains
    to be completed
  • In local areas different levels of priority and
    attention have been attached to evaluation.

40
Recommendations mainstreaming at national level
  • Incorporate Choose Life objectives and priorities
    into other policy streams/initiatives
  • Involve clinical services in population-based
    suicide prevention activities
  • Involve national voluntary sector organisations
    in awareness raising and campaigning
  • Engage in purposive innovation to test out,
    evaluate, learn and implement

41
Recommendations mainstreaming at local level
  • Using intelligence from a range of sources, as
    tools in planning for sustainability
  • Building in mechanisms to track and review
    progress towards objectives across policy areas
  • More focused targeting of action is required

42
Recommendations self-harm
  • More consideration to be given in phase 2 to the
    integration of self-harm into Choose Life
  • The strategy should continue to encompass high
    risk self-harm
  • but the less serious component of self-harm
    cannot be ignored

43
Recommendations CPP
  • The CPP remains the most appropriate vehicle for
    developing strategy and overseeing delivery in
    relation to Choose Life at the local level
  • But its limitations should be recognised
  • Need to examine the necessary partnerships that
    have yet to be put in place
  • Priority should be given to establishing/building
    on effective links with clinical drug/alcohol
    services
  • NIST should continue to work closely with CPPs to
    ensure that Choose Life budgets are fully spent
    on suicide prevention activities

44
Recommendations central coordination body
  • Some type of central coordination body will
    continue to be required in the immediate future
  • Key tasks provide national oversight, assess and
    support performance and ensure accountability at
    local level, promote learning and effective
    knowledge transfer, and co-ordinate action.
  • There should be a review of how the central
    coordinating function is delivered and where it
    is situated

45
Choose Life evaluation (first phase)research
team
  • University of Edinburgh
  • Stephen Platt (RUHBC)
  • Emma Halliday (RUHBC)
  • Margaret Maxwell (General Practice)
  • Scottish Development Centre for Mental Health
  • Joanne McLean
  • Allyson McCollam
  • Amy Woodhouse
  • London School of Economics
  • Dave McDaid (Health Social Care)
  • Glasgow University
  • Mhairi Mackenzie (Public Health Health Policy)
  • Avril Blamey (Public Health Health Policy)
About PowerShow.com