Has the SEI Gone Too Far or Not Far Enough a Panel Discussion

1
Has the SEI Gone Too FarorNot Far Enough?a
Panel Discussion
2
Panelists

• Madhumita Sen IBM India
• Bill Deibler Software Systems Quality
  Consulting
• John Maher Organization Process Improvement,
  Inc.
• Ron Radice Software Technology Transition

3
Approach and Intent

• The panelists will discuss the Pros and Cons on a
  number of questions, which were supplied to each
  panelist in advance. The questions were offered
  to the full LA, including HMLA, community to rank
  preference.
•  
• The intent of the discussion is not to criticize,
  but to offer an open forum on issues and concerns
  within the LA, HMLA, and appraisal sponsor
  community, such that the SEI might benefit in how
  it will address their needs.
• The intent is to collaborate with the SEI on
  addressing pressing issues and concerns.

4
How Many Responded?Limited to Lead Appraiser
Community

• 437 LA/HMLAs in population set
• 13 Fatal/Blocked/Failed
• 166 Scored Responses 95 CL and 6 CI
• 56 HMLAs responded, 36 LAs, 25 of SEI LAs
• 18 of the 36 questions rated highest become
  candidates for the panel

5
Most Wanted Discussion Topics

• 1. Does the current process for HMLA
  certification address the concern regarding the
  validity of high maturity appraisals?
• 2. Should the SEI announce when LAs, including
  HMLAs, have their authorizations removed? What
  criteria are used to determine the necessity for
  removal and how often has it been necessary?
• 3. How do you feel about the lack of visibly
  defined criteria for when the authorizations
  should be removed, and the impact on appraisals
  that were done by a person/persons removed?
• 4. Should sample rates have guidance beyond the
  current MDD 3 project statements? Should
  guidance for larger OUs consider sampling theory
  for defined confidence levels?
• 5. Will the SCAMPI BOK meet its intended
  objective? How do we avoid falling into the Take
  a test to be certified trap similar to PMI and
  testing certifications that test memorization but
  not reasoning and judgment ability?
• 6. Should the entry criteria for Lead Appraisers
  be strengthened (proactive rather than reactive),
  particularly in the area of industry experience?

6
Most Wanted Discussion Topics

• 7. Are guidelines for the ADS Level 4/5
  Additional Information sufficient, considering
  the wide variation in current submissions?
• 8. Should the SEI support delta appraisals,
  similar to certification schemes like ISO 9001
  appraisals? What are the potential cost savings
  associated with such a strategy?
• 9. The quality guru of the 1980s, Dr. Edward
  Deming, warned against applying SPC to
  sub-processes by themselves he believed these
  actions could lead to optimizing the sub-process,
  possibly at the expense of the system. What might
  this suggest when SEI insists that sub-processes
  are to identified and managed?
• 10. What QA should the SEI be doing on Level 5
  activities? Does it really look at how Level 5
  activities build on Level 4 outputs?
• 11. How much of advanced statistical techniques
  should be needed for an organization to be Level
  4/5 using e.g., ANOVA, Monte Carlo Simulation,
  DOE?
• 12. Should all projects in an organization be
  required to be using the same advanced
  statistical techniques and models? How much is
  enough for any project to use or need?

7
Most Wanted Discussion Topics

• 13. How many sub-processes should be required for
  an organization to be a Level 4?
• 14. Is SCAMPI really for small organizations?
  Why is not in SEIs view for organizations that
  are large enterprises? Is SPC strictly being
  applied to software projects as was intended for
  example with manufacturing? What are the
  limitations of SPC in software projects using
  advanced statistical techniques?
• 15. What evidence exists that SPC can be used for
  software processes that are aperiodic in nature?
  What research exists that shows it is applicable
  and appropriate?
• 16. Has the SEI modified its interpretation of
  Level 4 without sufficient transfer of
  expectations to LAs and appraised organizations?
  
• 17. Why has SEI recently chosen to elevate
  subpractices in level 4 from informative to
  normative components? Should the SEI rewrite the
  model to take into account their latest position
  on subpractices in Levels 4 and 5?
• 18. Who must the SEI serve its sponsor the
  worldwide constituency who has embraced the CMMI,
  both? How well is the SEI doing in any regard?

8
What Panelists Have Been Given

• List of topics rated highest by respondents
• Textual remarks (attribution removed) from those
  respondents who gave such input (28 gave textual
  remarks also or exclusively)

9
Some Curious Remarks

• From a non-respondent HMLA at SEI Youre
  titling at windmills.
• What do you mean by "on a scale of 1 through 5,
  where 5 is high, just like in appraisals". Where
  is a 5 point scale used in an appraisal?

10
General Reactions From Respondents

• Great Idea, Really Needed, Great Questions,
  and Best of Luck
• So, we have 4 Don Quixotes with you today on the
  panel
• How many in the community want to ride this
  horse?
• Any Sancho Panzas?
• Hopefully, well all have some fun, learn, and
  give some needed help for SEI focus areas

11
Rules

• One of the top rated discussion topics is
  selected by the panel chair
• One panelist will staty discussio others will
  respond
• Starting panelist has 2 minutes MAX to reply
• Each of the other there has one minute each MAX
• We have time at the end to take 15 minutes of
  questions from the audience

12
You, the Audience, Will Have Access To

• Full set of topics provided to respondents
• Inputs given to panelists
• This introduction
• Summary of responses, etc.
• Go to www.stt.com ? Papers Presentations
• after March 30th