The Most Significant Change Technique (MSC)

1
The Most Significant Change Technique (MSC)

• Dr Jessica Dart
• Clear Horizon

2
MSC

• Form of qualitative, participatory ME
• Based on stories of significant change
• Developed by Davies 1996 - Bangladesh
• Now used in numerous development programs and in
  the public sector

3
Qualitative vs quantitative monitoring

• Quantitative
• Focus on measurement
• Closed questions
• About proving
• Easy to aggregate
• Deductive
• Static
• Goal displacement can be a problem

• Qualitative
• Focus on questioning
• Open questions
• About learning
• Hard to aggregate
• Inductive
• Dynamic
• Goal displacement is not an issue

4
Limitations of indicator based monitoring

• Goal displacement
• Creaming
• Not about learning
• Dont tell you what you dont know you need to
  know

5
Qualitative monitoring

• Can be used in conjunction with conventional
  output monitoring
• Is usually more aimed at learning than
  accountability

6
Why stories?

• People tell stories naturally - indigenous
• Stories can deal with complexity and context
• People remember stories
• Stories can carry hard messages /undiscussables
• But stories not known for accuracy/truth

7
Use of stories in MSC

• Collection of stories systematic, collective
  interpretation storytelling can be effectively
  harnessed for participatory evaluation
• Because interpretations tell another story
  process has beneficial outcomes for evaluation
  utilisation

8
Overview of MSC

• 1. Determine sorts of change to monitor
• 2. Collect stories
• 3. Review filter stories regularly
• 4. Collate selected stories for funders review
• 5. Monitor the process and verify the stories

9
Overview of MSC

• 1. Determine sorts of change to monitor
• 2. Collect stories
• 3. Review filter stories regularly
• 4. Collate selected stories for funders review
• 5. Monitor the process and verify the stories

10
Example

• Target 10 Dairy Extension Project
• Four regions in Victoria, 50 staff
• 1999-2000 trail of the approach
• Still continues today

11
Step 1- Selection of domains of change

• 3 broad domains of changes to be monitored at
  the project level
• Changes in on-farm practice
• Changes in farmer-decision making skills
• Changes in profitability
• Any other type of change
• Not precisely defined

12
Step 2 - Collect stories

• During the last month, in your opinion, what do
  you think was the most significant change that
  took place as a result of the project?
• The respondent (farmer, extension worker or
  industry rep) answers in 2 parts
• 1) descriptive 2) explanatory

13
Step 3 Review filtering process

• The stories were reviewed by
• The regional committees (every 2-3 months)
• Statewide Executive (every 2-3 months)
• The stories are reviewed using a facilitated
  process at the state and funder levels

14
Funder meeting
State meetings
flow of stories
feedback
Region 1
Region 2 Region 3 Region 4
Story tellers
15
Step 4 - Collate review selected stories

1. In total 134 stories were collected - 80 from
   extension staff
2. A booklet containing 24 selected stories
   accompanied by the selection criteria comments
3. Purchasers provide feedback to project after
   reviewing booklet

16
Step 5 Monitor process verify stories

• A database was developed to keep track of all
  stories
• Secondary analysis at end of reporting period
• In this case selected stories were not verified
• Storytellers were asked to check final stories in
  report

17
Impact of MSC

• Staff gained more fully shared vision
• Process boosted their morale
• Process saw farmers, staff, collaborators sitting
  together and interpreting qualitative data
  casting evaluative judgements
• Project committees became better at
  conceptualising impact

18
Use of stories

• In addition to reporting, stories were used
• To improve planning
• To help explain a point to a farmer
• To recruit new participants
• To help explain a point to another member of
  staff
• For PR

19
MSC vs quantitative monitoring

• MSC
• Focus on measurement
• Closed questions
• Project out
• About proving
• Deductive
• Static
• Inclusive
• Central tendencies

• Qualitative
• Focus on questioning
• Open questions
• Context in
• About learning
• Inductive
• Dynamic
• Selective
• Outer edges of experience

20
Program out
Context in
Goal-based evaluation
Goals
Extent to which they were achieved
From the view point of the program staff
consultation
Program
From the viewpoint of the Participants
21
Purpose of MSC in ME

• Primary purpose to facilitate improvement by
• focusing direction of work towards explicitly
  valued directions
• eg. what do we really want to achieve and how
  will we produce more of it?
• Contributes to summative evaluation
• Information about unexpected outcomes
• Performance information concerning very best
  success stories
• Can inform criteria used to judge projects

22
MSC

• Creates space for stakeholders to reflect, to
  make sense of complex changes
• Provides dialogue to help make sense of each
  others values
• Facilitates dynamic dialogue ie. what do we
  really want to achieve and how will we produce
  more of it?
• Excellent for participatory programs with
  diverse, complex outcomes, multiple stakeholders