Judges

1
Malcolm BaldrigeNational Quality Award
Baldrige National Quality Program

• Judges Survey Results 2003-2008

2
Topics for Review

• Response Rates
• Applicant Characteristics
• Satisfaction with
• - Criteria
• Feedback Reports
• Overall Process
• Potential Changes to Survey

3
MBNQA Judges SurveyResponse Rates
27 of 60
34 of 64
48 of 86
55 of 84
56 of 85
4
MBNQA Judges SurveyWho Responded?
Total Responses N 56
Stage Applied Responded
Consensus 72 47
Site Visit 10 7
Award Recip. 3 2
Sector Applied Responded
HC 43 29
ED 11 5
SB 7 5
SV 5 4
MF 3 2
NP 16 11
5
MBNQA Judges SurveyS/L Programs an Important
Partner
6
MBNQA Judges Survey

• What prompted your organizations interest in
  using
• the Criteria? (could choose more than one
  response)
• State or Local Award Participation (n35)
• 2 Contact with Baldrige Award recipient (n26)
• 3 Word of mouth from other organizations (n25)
• 4 Quest for Excellence/Regionals (n22)
• 5 Baldrige Examiner in organization (n17)

7
MBNQA Judges Survey What was your
organizations primary goal in applying for the
Baldrige Award?
The options validate progress made on
improvements and customer requirement were
added in 2005
8
MBNQA Judges SurveyThe Criteria Items Are Clear
(21)
Top Box
(21)
(23)
(29)
(21)
9
MBNQA Judges SurveyThe Criteria are Relevant
(76)
Top Box
(62)
(48)
(51)
(68)
10
MBNQA Judges SurveyInteraction with NIST and ASQ
11
MBNQA Judges Survey Overall Satisfaction with
the Feedback Report
Top Box
(40)
(50)
(34)
(55)
(35)
12
MBNQA Judges SurveyTimeliness of the Feedback
Report
(48)
Top Box
(54)
(55)
(50)
(40)
13
MBNQA Judges Survey Objectivity of the Feedback
Report
Top Box
(46)
(35)
(54)
(47)
(26)
14
MBNQA Judges SurveyRelevance of Feedback Report
(50)
(41)
(46)
(55)
(37)
Top Box
15
MBNQA Judges Survey Value of Feedback Relative
to Investment
Top Box
(45)
(42)
(35)
(55)
(49)
16
MBNQA Judges SurveyNew Feedback Report Items
Accuracy of Strengths
Accuracy of OFIs
17
MBNQA Judges SurveyNew Feedback Report Items
Usefulness of Key Themes
Clarity of Strengths
Clarity of OFIs
18
MBNQA Judges Survey Likelihood Youll Reapply
19
MBNQA Judges Survey Would you recommend to
others?
(67)
(72)
(68)
(73)
(69)
Definitely
20
MBNQA Judges Survey Overall Satisfaction with
Award Process
Top Box
(24)
(37)
(57)
(33)
(41)
21
Issues with Satisfaction Surveys

• Too many surveys, too many questions
• Employees dont know how to take corrective
  action
• Scores dont link to behaviors
• Surveys confuse transactions with relationships
• Satisfaction surveys dissatisfy customers
• -- From The Ultimate Question
• F. Reicheld

22
Issues with Judges Survey

• Response rates should be higher
• We already know the answer to many of the
  questions
• Confounding results
• Dont currently link results to behaviors
• Not enough actionable feedback
• Much of the data doesnt really align with
  Customer Engagement

23
Potential Changes

• Move to a Net Promoter like metric
• How likely is that you would recommend X to a
  friend or colleague?
• Extremely Not at All
• Likely Likely
• 10 9 8 7 6 5
  4 3 2 1 0

Promoter
Passive
Detractor
24
Potential Changes

• NPS Promoter - Detractor
• Select NPS Stars Most Recent
• BNQP NPS
• Harley Davidson 81
• Costco 79 53
• SAS 66
• Intuit 58
• Southwest Air. 51

25
Potential Changes

• How likely is it that you would recommend use of
  the Baldrige Criteria for organizational
  improvement to a professional colleague?
• What is the most important improvement that
  would cause you to rate this closer to a 10?

26
Potential Changes

• How likely is it that you will reapply to the
  Award Program in the next two years?
• What is the most important improvement that
  would cause you to rate this closer to a 10?