Practical Software and

1
Practical Software and Systems Measurement A
foundation for objective project management
COCOMO Forum October 24, 2001
2
Outline

• Objectives of Measurement and Analysis
  Initiatives
• Practical Software and Systems Measurement
• ISO/IEC 15939, Software Measurement Process
• CMMI Measurement and Analysis Process Area
• ISO 9000-3, Guidelines for Software

3
Objectives of Measurement Initiatives

• Experienced-Based Measurement to Support
  Management and Engineering Requirements
• Consistent Measurement Process Across Measurement
  Initiatives
• Consistent Measurement Terminology

4
(No Transcript)
5
Measurement Process Model
6
Practical Software and Systems Measurement (PSM)
7
What is PSM?

• A Primary Measurement and Analysis Process Used
  by DoD, Government, and Commercial Programs
• Technology Based on Actual Experience on Real
  Programs
• A Fully Integrated Approach Including Guidance,
  Tools, Training, Implementation Workshops,
  Support Products, and a Qualified Technical Team

8
Key Measurement Concepts

• Measurement is a Process - Not a Pre-Defined List
  of Graphs or Reports
• The Measurement Process is Flexible - Adapted To
  Meet Specific Project and Organizational
  Objectives and Information Needs
• The Measurement Requirements are Integrated Into
  the Project Technical and Management Processes

9
Common Information Needs

• Schedule and Progress
• Resources and Cost
• Product Size and Stability
• Product Quality
• Process Performance
• Technology Effectiveness
• Customer Satisfaction

10
Problem Report Status Priority 1, 2, and 3
Problem Reports
Build 1
1000
900
800
700
600
Discovered
500
Number of Problem Reports
Closed
400
300
200
100
0
30 Aug 98
11
Types of Analysis
12
Estimation
13
Integrated Analysis Model
14
Integrated Analysis Example
15
Multi-Level Requirements
16
PSM Project Strategy
17
New Technology Areas

• Measurement and New Technologies
• Object-oriented developments
• Spiral/evolutionary approaches
• Interoperability
• Product lines
• Initial Recommendations Were Draft Drafted in
  Early 2001
• Field Trials Are Underway - Additional Trials
  Will be Conducted in 2002

18
PSM Participating Organizations

• DoD and Government
• US Air Force AFMC
• US Air Force SSSG
• US Air Force STSC
• US Air Force CRSIP
• US Air Force STRATCOM
• US Army AMC
• US Army ATEC
• US Army CECOM
• US Army ODISC4
• US Army SMDC
• US Army TACOM - ARDEC
• USMC MCTSSA
• US Navy AEDC
• US Navy ASN RDA
• US Navy FMSO
• US Navy NAVAIR
• US Navy NAVSEA
• US Navy NAWC

• Overseas
• Defence Materiel Org. (Australia)
• LiveWare (Argentina)
• NRC (Canada)
• Software and Systems Engineering (Israel)
• Software Improvements (Australia)
• Software Quality Institute (Australia)

• US Navy NUWC
• US Navy OPTEVFOR
• ODUSD (ST)
• OSD C31
• OSD PAE
• Aerospace Corp.
• DCMA
• DFAS
• DISA
• DLA
• DSMC
• FAA
• IDA
• MITRE Corp.
• NDU IRM College
• NASA
• National Park Service
• Sandia National Lab
• SEI

19
PSM Participating Organizations

• Industry
• ACS GSG
• American Sys. Corp.
• Averstar
• Bloodworth Int. Tech.
• Carnegie Mellon Univ.
• CSC
• CTA
• Distributive Software
• Federal Reserve Bank
• General Dynamics
• General Scientific Corp.
• Graeme Garland
• GTE
• Hawaiian Electric
• IEEE
• IFPUG
• IITRI
• INCOSE

• Independent Engineering
• ITABHI
• James Gregory Assoc.
• Kodak Health Imaging
• Litton/TASC
• Lockheed Martin
• L3 Communications
• Northrop Grumman
• OAO Corporation
• PRC - Litton
• PRC/TASC
• QSM
• Quality Plus Tech.
• Raytheon - Hughes
• Robbins, GIOIA
• Rockwell Collins
• SAIC

• Sallie Mae
• Sentel
• Software Metrics
• SPC
• Sverdrup Tech.
• Tecolote Research
• TeraQuest Metrics
• Tivoli
• TRW
• United Defense
• Univ. Southern CA
• User Trust Network
• US West
• VPI - State University
• West Virginia High Tech. Cosortium
• West Virginia University

20
(No Transcript)
21
ISO/IEC 15939, Software Measurement Process
22
ISO/IEC 15939

• PSM Was Base Document for This Standard
• Specifies Required Activities and Tasks of a
  Measurement Process
• FCD Ballot Passed in May 2001 With Only 1
  Dissenting Vote - All Comments Resolved at ISO
  Plenary
• Terminology Was Adjusted for Consistency With
  ISO/IEC 9126, 15498, and 14143 Series
• DIS Ballot Will be Released in September 2001 -
  Ballot Will Close in November 2001 -
  International Standard in Late 2001

23
Basic Measurement Architecture
ISO/IEC 15939
Measurement Information Model
Measurement Process Model
Information Driven Measurement Requirements
Consistent, Tailorable Implementation Process
24
Measurement Information Model
25
Information Needs
Measure that Provides an Estimate or Evaluation
With Respect to Information Needs
Indicator
Algorithm Combining Measures and Decision Criteria
Analysis Model
Quantity Defined as a Function of Two or More
Measures
Derived Measure
Derived Measure
Algorithm Combining Two or More Measures
Meas. Function
Base Measure
Base Measure
Measure of a Single Attribute Defined By a
Specified Measurement Method
Operations Mapping an Attribute to a Scale
Meas. Method
Meas. Method
Entities
Property Relevant to Defined Information Needs
Attribute
Attribute
26
Information Product
Information Needs
Information Product
Measure that Provides an Estimate or Evaluation
With Respect to Information Needs
Decision Criteria
Indicator
Algorithm Combining Measures and Decision Criteria
Analysis Model
Base Measure
Derived Measure
27
CMMI - Measurement and Analysis (MA)
28
CMMI MA

• ISO/IEC 15939 Was Base Document for the
  Measurement and Analysis Process Area
• CMMI MA Describes How to Evaluate a Measurement
  Process
• MA Is a Level 2 Process Area
• Pilot Assessments Are Being Conducted in 2001

29
ISO 9000-3Quality management and quality
assurance standards - Part 3 Guidelines for the
application of ISO 9001 to the development,
supply, installation, and maintenance of computer
software
30
ISO 9000-3

• 9000-3 Is Being Updated by SC7 as Part of the ISO
  90002000 Update
• Part 8, Measurement, Analysis and Improvement,
  Was Updated for Consistency With 15939
• Standard Is at Working Draft Stage - Expected
  Publication Date Is 2002/2003

31
For More Information
PSM Support Center Cheryl Jones - PSM Project
Manager TACOM - ARDEC AMSTA-AR-QAT Building
62 Picatinny Arsenal, NJ 07806-5000 (973)
724-2644 (Voice) [phone]
(FAX) cljones_at_pica.army.mil PSM Home Page
http//www.psmsc.com