Standard CMMI Appraisal Method for Process Improvement

1
Standard CMMI Appraisal Method for Process
Improvement

• (SCAMPI)

2

• ARC Class A Compliant
• Version 1.1 in 2002
• Version 1.2 in 2006

3
Approach

• Discovery
• Verification

4
(No Transcript)
5
SCAMPI Phase Structure

• Phase I Plan and Prepare For Appraisal
• Phase II Conduct Appraisal
• Phase III Report Appraisal Results

6
(No Transcript)
7
(No Transcript)
8
(No Transcript)
9
(No Transcript)
10
(No Transcript)
11
Key Preparation Questions

• Should we have an appraisal?
• Should we do it now or later?
• Should SE and SW be appraised together with
  separate results, or should their appraisals be
  integrated?
• Which appraisal class should be used?
• What should be included in each appraisal?

12
Preparation Considerations

• Organization scope
• Model Scope
• Size
• Disciplines
• Improvement Status

13
Successful appraisal teams require

• a plan, including objectives, constraints, scope,
  outputs, tailoring, resources, cost, schedule,
  logistics, risks, and commitment
• clearly defined goals
• clearly defined roles
• established rules of engagement
• well-defined decision procedures
• understanding of the group process
• clear communications
• proper team behavior
• balanced participation
• training

14
Decision Making - Consensus

• Decision making on appraisal teams is done by
  consensus, not unanimous vote nor majority vote
• Consensus is finding a proposal acceptable enough
  that all team members can support it or live with
  it, and no team member opposes it
• The consensus process can require several cycles
  through the decision cycle as the team continues
  to gather and analyze factual information bearing
  on a decision
• If consensus cannot be obtained, a decision
  should not be forced, e.g., through majority vote
  - the item under consideration must be left not
  rated

15
(No Transcript)
16
(No Transcript)
17
Method Definition Document
18
MDD 1.2 Outline
19
(No Transcript)
20
SCAMPI MDD V1.2 Processes - 1

• 1 Plan and Prepare for Appraisal
• 1.1 Analyze Requirements
• 1.1.1 Determine Appraisal Objectives
• 1.1.2 Determine Appraisal Constraints
• 1.1.3 Determine Appraisal Scope
• 1.1.4 Determine Outputs
• 1.1.5 Obtain Commitment to Appraisal Input
• 1.2 Develop Appraisal Plan
• 1.2.1 Tailor Method
• 1.2.2 Identify Needed Resources
• 1.2.3 Determine Cost and Schedule
• 1.2.4 Plan and Manage Logistics
• 1.2.5 Document and Manage Risks
• 1.2.6 Obtain Commitment to Appraisal Plan
• 1.3 Select and Prepare Team
• 1.3.1 Identify Appraisal Team Leader
• 1.3.2 Select Team Members
• 1.3.3 Prepare Team
• 1.4 Obtain and Inventory Initial Objective
  Evidence

21
SCAMPI MDD V1.2 Processes - 2

• 2 Conduct Appraisals
• 2.1 Prepare Participants II-65
• 2.1.1 Conduct Participant Briefing
• 2.2 Examine Objective Evidence
• 2.2.1 Examine Objective Evidence from Documents
• 2.2.2 Examine Objective Evidence from Interviews
• 2.3 Document Objective Evidence
• 2.3.1 Take/Review/Tag Notes
• 2.3.2 Record Presence/Absence of Objective
  Evidence
• 2.3.3 Document Practice Implementation
• 2.3.4 Review and Update the Data Collection Plan
• 2.4 Verify Objective Evidence
• 2.4.1 Verify Objective Evidence
• 2.4.2 Characterize Implementation of Model
  Practices
• 2.5 Validate Preliminary Findings
• 2.5.1 Validate Preliminary Findings
• 2.6 Generate Appraisal Results
• 2.6.1 Derive Findings and Rate Goals
• 2.6.2a Determine Process Area Capability Level

22
SCAMPI MDD V1.2 Processes - 3

• 3 Report Results
• 3.1 Deliver Appraisal Results
• 3.1.1 Deliver Final Findings
• 3.1.2 Conduct Executive Session(s)
• 3.1.3 Plan for Next Steps
• 3.2 Package and Archive Appraisal Assets
• 3.2.1 Collect Lessons Learned
• 3.2.2 Generate Appraisal Record
• 3.2.3 Provide Appraisal Data Package to CMMI
  Steward
• 3.2.4 Archive and/or Dispose of Key Artifacts

23
(No Transcript)
24
(No Transcript)
25
Determination of Practice Implementation

• The basic approach to determination of practice
  implementation is to
• characterize the degree to which the practice is
  implemented, and
• note any factors which should be considered in
  addition as the instantiation data is aggregated
  (strengths, weaknesses, etc.).

26
(No Transcript)
27
(No Transcript)
28
Characterizing Practice Implementation -1

• Assign characterization values reflecting the
  extent of practice implementation for each
  instance
• Aggregate practice characterizations to
  organizational unit level using defined method
  aggregation rules
• Iterate and focus revisions to data collection
  plan
• Generate findings based on aggregation of
  weaknesses and strengths

29
(No Transcript)
30
(No Transcript)
31
(No Transcript)
32
(No Transcript)
33
Rating Goals

• Goal ratings are a function of the extent to
  which the corresponding practices are present in
  the planned and implemented processes of the
  organization.
• Team judgment is used to rate goals where the
  objective evidence doesnt render the outcome
  plainly obvious.
• If all practices are FI, then the goal must be
  satisfied.
• If all practices are NI, the goal cant be
  satisfied.
• The conditions between the two extremes require
  professional judgment.

34
(No Transcript)
35
Data Collection and Rating Concepts

• Corroboration
• Must have direct artifacts, combined with either
  indirect artifact or affirmation
• Coverage
• Must have sufficient objective evidence for
  implementation of each practice, for each
  instance
• Must have face-to-face (F2F) affirmations (avoid
  paper-only appraisals)
• At least one instance for each practice (one
  column)
• At least one practice for each instance (one
  row)
• Or 50 of practices for each PA goal, for each
  project, have at least one F2F affirmation data
  point

36
(No Transcript)
37
(No Transcript)
38
(No Transcript)
39
(No Transcript)
40
Completion of Pre Assessment Questionnaires - 1

• Site Information Questionnaire
• The Site Information Questionnaire helps the
  assessment team to rapidly understand the culture
  and language of the site's software practitioners
  and the context within which they operate
• Change Readiness Questionnaire
• The Change Readiness Questionnaire assists an
  organization in determining its readiness for
  change

41
Completion of Pre Assessment Questionnaires - 2

• Organizational Structure Questionnaire
• What is the real structure that gets things done?
  What is the communication structure? What is the
  relationship with the clients? How are the
  requirements gathered? Are there multiple
  customers? Are the customers internal or
  external? Are there end users involved?
• The Project Questionnaire
• The Project Questionnaire is intended to help
  collect information for use in selecting projects
  or other work efforts. It helps the assessment
  team organize contextual background information.

42
Completion of Pre Assessment Questionnaires - 3

• Documented Process Questionnaire
• The Documented Process Questionnaire looks at all
  of the documented policies, procedures,
  standards, guidelines, and templates as well as
  the artifacts that exist from the use of those
  defined processes and maps them to the
  requirements of the CMMI

43
Reviewing - 1

• Schedule 1-2 days to review documentation
  entire assessment team is present
• Request that process owners or at least process
  lead developers present or walk through the
  specific process area processes

44
Reviewing - 2

• Assessment team members may ask the following
  questions during the presentation
• Would you please clarify or expand on a point?
• Would you please jump to the referenced
  procedure, guideline, template, or checklist?
• Would you please show project examples that have
  followed that procedure or used that template?
• Why was this information placed in this document
  and not another one that is more closely aligned
  to the CMMI way of organization?
• Would you please print a hard copy of that
  section of the procedure or please print out the
  entire procedure?

45
Reviewing - 3

• Assessment team members can ask interview-like
  questions to gain more insight as to how these
  processes are actually being used on the projects
• Having experts describe their documented
  processes and other site assessment team members
  witnessing the answers eliminates or reduces the
  risk of long debates over the value of the
  documented processes in the later phases of the
  assessment

46
Consolidating - 1

• Each assessment team member reviews his/her own
  notes and makes observations
• These observations may make use of the sticky
  note format or may simply be placed on a sheet of
  paper with appropriate notations that will later
  be shared with the rest of the assessment team
• After each assessment team member has had a
  chance to develop individual observations, the
  Assessment Team Leader leads or facilitates the
  team in a brainstorming process to get the
  observation candidates out in front of the team
• A category is picked. The facilitator asks one
  person to offer a strength or weakness for that
  category

47
Consolidating - 2

• An observation is offered and the facilitator
  writes it on a flip chart or white board
• The facilitator asks the next person for an
  input
• This next person looks at what has already been
  placed on the flip chart or white board and what
  observations they have written down for
  themselves
• If an observation already written on the flip
  chart or white board is close enough, they either
  pass to the next person or offer an observation
  for that category that they feel is different
  enough to be considered by their teammates
• The facilitator continues in a Round Robin
  fashion. (each person gets a turn to offer one
  input for a category and this process continues
  until everyone has run out of new ideas)

48
Consolidating - 3

• The assessment team then takes a step back and
  reviews all of the observations again that are in
  front of them
• Like observations may be further consolidated
  with the consensus of the assessment team

49
(No Transcript)
50
(No Transcript)
51
(No Transcript)
52
(No Transcript)
53
(No Transcript)
54
(No Transcript)
55
(No Transcript)
56
(No Transcript)