Understanding the Business Case for Best Practices - PowerPoint PPT Presentation


PPT – Understanding the Business Case for Best Practices PowerPoint presentation | free to view - id: 1ce17-ZmYyM


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation

Understanding the Business Case for Best Practices


for Best Practices 'Best Practices - the road to higher ... meant by 'best practices' ... Implementation and use of best practices are recognized as a ... – PowerPoint PPT presentation

Number of Views:113
Avg rating:3.0/5.0
Slides: 34
Provided by: jimwellsl


Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Understanding the Business Case for Best Practices

Understanding the Business Case for Best
"Best Practices - the road to higher performance
and quality"
Module Objectives
  • Explain why many projects are not successful
  • Define what is meant by best practices
  • Identify examples of Return On Investment (ROI)
    resulting from the use of best practices using
    examples from both inside and outside of SSC
  • Explain that most of the ROI is achieved through
    the identification of defects early in the life

Do Projects Deliver Quality Products on Time and
within Budget?
  • Nearly 1/3 of information technology projects
    were cancelled before completion
  • Average overrun of project budgets was 189
  • The average schedule overrun for projects that
    were in difficulty was 222
  • On average, the delivered product contained only
    61 of the originally-specified features.
  • Only 35 (1 of 3) software projects were
    completed on time, on budget (2) (improved
    from 1 in 6 due to best practice
    usage) Charting the Seas of Technology The
    CHAOS Study The Standish Group, January 1995-
    Updated 2006 (2)
  • - Report of the Defense Science Board Task Force
    on Defense Software, November 2000

Traits of Successful Organizations
  • Research of advanced / higher maturity
    organizations (CMM/CMMI) found similarities among
  • Implementation and use of best practices are
    recognized as a standard business practice, not a
    voluntary exercise, and included in all strategic
    and business plans as an organizations goal
  • Vigorous management commitment, support, and
    involvement are obvious at all levels at all
  • Measured progress is reported to management
  • A dedicated, motivated, respected, and
    experienced implementation and support team is in
  • BP implementation is run as a project (plan and
    track, allocate resources, reviews, etc.) at the
    organization level
  • In every case, dollar Return on Investment (ROI)
    was realized, yearly reduction in time-to-market
    and post-release defects was achieved, and
    customer and employee satisfaction increased

Best Practices are Proven Ways to Help Ensure
  • They are effective, high-leverage technical
    and management processes that have been
    implemented and proven on successful projects
  • Example BPs Benefits
  • Project Planning Establish schedule, resources,
  • Measurement and Control Provide visibility into
  • Peer Reviews Identify and eliminate defects
  • Risk Management Reduce likelihood of surprises
  • Configuration Management Control work products
    through lifecycle
  • Quality Assurance Verify adherence to
  • Status Reporting Keep stakeholders informed
  • Effective Meetings Make meetings productive
  • Standards Strategies Provides common life cycle
    frameworks and lessons learned

The Business Case
  • A report by DoD Data Analysis Center for
    Software (DACS) found
  • Application of PI to Example organization with
    example projects
  • Development costs Reduced 73
  • Rework costs Reduced 96
  • Average schedule length Reduced 37
  • Post-release defects Reduced 80
  • Weighted risk likelihood Reduced 92
  • Return on Investment 211
  • - A Business Case for SPI Revised - Measuring
    ROI from Software Engineering and Management.
    DACS, September 1999 see http//www.dacs.dtic.mil

Tangible Benefits at Boeing
Planning was more accurate.
Defects could be detected earlier.
Product quality increased with rising maturity
ROI for the F/A-18 Advanced Weapons Lab, China
  • Major upgrades to 10 million words of code in 40
    different processors
  • Adopted CMM and achieved Level 4
  • Changes over latest 4 major upgrades over 10
  • Reduced cycle time from 56 months to 38 months
  • Reduced schedule slips from 12 months to on time
  • Decreased rework rate from 201 to 31
  • Decreased regression testing from 70 to 20
  • Decreased redundant testing from 100 to 10
  • Reduced defect density from 13.5 defects per
    KSLOC to 3.8
  • Reduced life-cycle cost from 1,170 per SLOC to
  • Reduced test flights from 3.1 per KSLOC to 0.6
  • No fleet problems reported in three years
  • Named one of Top 5 Quality Software Projects in
    Jan 2002 CrossTalk
  • Very well planned and managed - Capers Jones,
  • Size, complexity, and number of systems involved
    represent a significant technical challenge in
    itself - Watts Humphrey, Judge
  • See http//www.stsc.hill.af.mil/crosstalk/2002/jan

ROI Data from Ft. Sill - CMMI L5
  • Life cycle maintenance engineering center for
    U.S. Army CECOM, staffed by 30 government and 300
  • Fielded seventy-three software versions,
    transitioned eight weapons systems, developed
    twenty new fire support systems
  • Joint teams handle over 9.5 million SLOC (80
    new development, 20 defect repair)
  • Tangible / Quantifiable Benefits
  • 1.5M to 9.5M SLOC with NO staff increase needed
  • 250 improvement in ability to accurately predict
    size of product for staffing and cost estimates
  • 83.57 decrease in defects per KLOC
  • From 1.40 to .23 defects per KLOC (from last to
    current version)
  • LOC per hour rate has increased by 48
  • Using average inflation over thirteen years, if
    old processes were kept, operating cost would be
    59M per year
  • Continuous improvement has made the average
    operating cost 30M per year

Excerpts from An Analysis of Ft. Sills CMMI
ROI/Benefits Sandy VanDensen, SSC Pacific Code
ROI Data from Ft. Sill - CMMI L5 (2)
  • Intangible / Qualitative Benefits
  • Creativity and ingenuity encouraged from staff
  • Staff appreciates formal inspection process,
    which in turn contributed to early defect
  • Reduced post-delivery defects gives engineers
    time for follow-on efforts instead of rework and
  • A shared development platform has allowed staff
    to move easily between projects
  • Staff liked not having to guess at how to do
  • People who use the processes defined the
    processes, giving pride of ownership and buy-in

Excerpts from An Analysis of Ft. Sills CMMI
ROI/Benefits Sandy Van Densen, SSC Pacific
Code 53
Other Intangible Benefits
  • At Ogden Air Logistics Center
  • Positive influence on working environment no
    constraint on creativity
  • Beneficial structure provided to the development
  • Fewer surprises and last-minute glitches, fire
    drills reduced
  • Better quality software overtime and unhappy
    customers reduced - CrossTalk, May 1999
  • At Boeing Space Transportation Systems
  • Employee satisfaction up from 74 to 96
  • Employees motivated to eliminate defects, improve
  • -IEEE Software, Sept/Oct 1999
  • At SSC Pacific
  • Better management control over the project
  • Better overall performance of the software
  • Improved morale of team members, less overtime,
  • Better communication among the team
  • Increased competitive advantage and repeat
    business - Costs and Benefits of SPI, Karen
    D. Prenger
  • At multiple sites
  • Improved customer satisfaction
  • Improved professional staff - DACS Business
    Case for SPI Revised

Benefits Reported by Past SSC Pacific Projects
  • Timely deliveries to Fleet CDLMS
  • Developers now understand the requirements DCT
  • Realistic schedules, understanding of
    requirements ATDS
  • Increased sponsor confidence, improved
    coordination KSA
  • Morale of team improved TBMCS
  • Peer Review caught code defects before
    release SWaT
  • We delivered on-time and within budget WebSKED
  • Team members now understand how their roles
    interact CoRE
  • Less crisis management GCCS-M
  • Reduced risk in the product release cycle Tact.
    Comm. Sol.
  • Significant reduction of defects in testing Time
    Crit. Strike
  • We are SO much more organized than we used to
    be! JSIMS-M

More Benefits Reported at SSC Pacific
  • Some testimonials from Project Managers
    regarding results of PI
  • We produced more complex builds in less time
  • Implementing Peer Reviews and other process
    improvements significantly reduced the problems
    found and the testing efforts (e.g., reduced
    trouble reports by 71, time to conduct tests by
    33, time to fix all trouble reports by 70)
  • Through process maturity we have observed
    increased productivity
  • The project people have told me they would not
    work on another project without defined processes
  • I travel a lot and having defined processes makes
    turnover completely seamless between me and my
  • We are now consistently producing builds with
    zero defects
  • We have better communication across the team, and
    people know what they are supposed to be doing
  • I feel I am a much better project manager
  • We have fewer surprises, last minute glitches,
    and fire drills
  • We have fewer risks this year because we learned
    from our Risk Management Plan from last year
  • We have been awarded new work based on our
    process improvement efforts

From former SSC SD SW-CMM L3 projects
Benefits at SSC Pacific the MATCALS project
  • Life-Cycle Maintenance of 174K SLOC. Achieved
    CMM Level 2 in January 1999 Level 3 in October
  • Improvements and desk-top procedures were
    implemented in Design, Code, Unit Test, and Peer
    Reviews over 2.5 years for the Version M release.
    A comparison of results in the Unit Integration
    Testing of SCPs, SEPs, and Trouble Reports (TRs)
  • Measure Ver. L Ver. M Change
  • a. SCPs and SEPs implemented 16 16 same
  • b. TRs written and fixed in testing 221 64 71
  • c. Total KSLOC modified 26 28 similar
  • d. Staff months to conduct tests 12 8 33 fewer
  • e. Staff months to fix all TRs 57.8 17.3 70
  • de Staff months for unit integration tests
    69.8 25.3 64 fewer
  • e/b Staff months to fix each TR 0.26 0.27 similar
  • b/d TRs written per staff month testing
    18.1 8.0 56 fewer
  • b/c TRs written per KSLOC modified 8.5 2.3 73
  • (de)/c Staff months per KSLOC modified
    2.7 0.9 67 fewer
  • Conclusion Implementing Peer Reviews and other
    process improvements significantly reduced the
    problems found and the efforts expended in testing

Benefits at SSC Pacific Notable CMMI Data
Tactical Communication Solutions (Code 53)
  • Center based product development organization
    relying on sales (no sponsor) of Data Link
    Gateway, TADIL J Host Simulator, etc.
  • Valuable TCS measurements show improvements
    (since CMMI in 2000) have resulted in
  • Avg STR closure time decreased by 80
  • Average yearly business income increases of 22.3
  • Significantly low or decreases in number and
    severity of STRs
  • Pri 1 consistently under 5
  • Pri 2 from 97 to 20 and Pri 3 from 140 to 20

Tactical Communications Solution (TCS) CMMI…ROI?
presentation, dated October 2003
Example of How Best Practices Can Save
Money Cost to Repair Design Defects
Phase when design defect was corrected Source
SEPG Conference, 1999
SEI found Rework is 40 - 50 of project
costs High-maturity organizations can get
rework to (Paulk, 1999)
Defect Detection Changes as Process Maturity
Typical Roadblock Categories to Best Practices
  • Resistance to Change
  • 1. We tried process improvement before, and it
    didnt buy us anything.
  • 2. CMM/CMMI and best practices are just another
  • 3. We have processes and plans, but in a crunch
    we send in a Tiger Team.
  • Lack of commitment and resources
  • 4. Whos paying for it? If it isnt paid for,
    were not doing it.
  • 5. Im retiring/transferring from this job
    soon, I dont have to get involved.
  • 6. We are too busy now to have time for process
  • Sponsors are inflexible with respect to changes
    to project budgets and schedules
  • 7. Our schedule is fixed, our resources are
    fixed, our requirements are fixed, our quality
    level is fixed.
  • 8. Our sponsor is happy now, and only
    interested in short-term results.
  • 9. Our sponsor doesnt understand or care about
    process improvement.
  • 10. Our project is different (its RAD, its
    maintenance, its a prototype) so this process
    doesnt apply to us.
  • Management not fully engaged
  • 11. Managing requirements is a waste of time
    requirements keep changing.
  • 12. Discipline always interferes with

Advantages of Best Practices
  • More business!
  • a. We won more business because of our maturity
  • b. We have lowered our life-cycle costs.
  • c. Surviving in our changing environment
    requires constant adaptation.
  • Higher probability of meeting budgets, schedules,
    quality goals
  • d. We now have visibility into our development
  • e. Building quality into products of large size
    and complexity requires engineering.
  • f. Process makes a difference in the quality of
    the activities and products.
  • Smoother relationships among sponsors,
    management, teams, contractors
  • g. We have better estimates of our budgets,
    schedules, and technical requirements.
  • h. Our success is dependent on other groups and
  • i. One person cant track all the details, and
    error detection is more probable when the work is
    examined by more than one person.
  • Fewer crises reduced overtime
  • We plan because we dont have time for rework.
  • We can now capture and pass on our lessons
  • l. We have fewer fire drills and less overtime.
  • Higher team morale
  • m. We are more proud of our products, and more
    satisfied with our work.

Commitment Taking an Active Role
  • It is not enough that top management commits
    itself for life to quality and productivity.
    They must know what it is that they are committed
    to - that is, what they must do. These
    obligations cannot be delegated. Support is not
    enough. Action is required. - Dr. W.
    Edwards Deming
  • They watch your feet, not your lips. -
    Dr. Tom Peters

Do Best Practices Guarantee Success?
  • No, but if you fail, dont fail because of
  • Lack of stated project goals and purpose or
    failure to tie goals to business objectives (line
    of sight)
  • Poor/lack of project planning
  • Poor/lack of estimation
  • Poor requirements definition/No requirements
  • Unable to deliver product within budget and
  • Poor contractor management
  • Poor/lack of attention to COTS integration issues
  • Lack of training
  • No/poor risk management
  • No/poor configuration management
  • No/poor quality assurance
  • Inadequate resources and unrealistic expectations
  • Poor/lack of communication
  • Best Practices keep you from failing for
    preventable reasons

Understanding Best Practices Summary
  • You should now be able to answer the following
  • Which SSC Pacific organizations have reported
    benefits from Best Practices? Name some benefits
  • Which other DoD organizations have reported
    benefits from Best Practices? Name some benefits
  • When is the most expensive time to correct a
  • List three advantages of using best practices.
  • What happens if we dont use Best Practices?

Please fill out your evaluation form for this
Additional Material
  • Additional Material
  • Tangible benefits
  • Transitioning to CMMI
  • Costs of not using best practices

Cost Efficiency Improvements Software Maintenance
Activities at SSC Pacific
1 Mil 0.9 Mil 0.8 Mil 0.7 Mil 0.6 Mil 0.5
Mil 0.4 Mil 0.3 Mil 0.2 Mil 0.1 Mil
Near Future
Early 90s
Late 90s

Lines of Code Under Maintenance

1 Mil 2 Mil 3 Mil 4 Mil 5 Mil
6Mil 7 Mil 8 Mil 9 Mil 10 Mil
Baseline Revision Costs
Note Data based on COCOMO models derived from
actual projects.
Tangible Benefits at Boeing
Projects operating at Maturity Level 3 increased
productivity by 62 ...
While cycle times improved 36.
Reference Scott Griffin, Boeing CIO, SEPG
Conference 2000
Benefits at Lockheed Martin

Transitioning to CMMI Benefits and Return on
  • Recently reported CMMI data shows quantitative
    and qualitative benefits may be demonstrated
  • Switching from previous process improvement
    models (e.g. SW-CMM, SE-CMM, EIA-731) to CMMI
  • New or modified processes
  • Broadened organizational scope of improvement
    efforts to include systems engineering, hardware
    and related disciplines
  • Summary of data reports cost savings, and
    improvements in schedule, quality and customer
    satisfaction measures
  • Cost reductions in intermediate and final work
    products and processes employed to produce
  • Schedule improvements in predictability and cycle
  • Quality improvements via reduction in defects

Lockheed Martin - Management Data
Services System Software Integration
  • From SW CMM ML2 (1993) to ML 3
    (1996) and then to CMMI ML5 (2002)
  • Data from their continuing improvement program
  • Increased software productivity by 30
  • Decreased unit software cost by 20
  • Decreased defect find and fix costs by 15
  • Reduced overhead costs of 5

Demonstrating the Impact and Benefits of CMMI An
Update and Preliminary Results, SEI Special
Report, CMU/SEI-2003-SR-009
Northrop Grumman Information Technology
  • Defense Enterprise Solutions received CMMI ML 5
    in December 2002 (consolidation of various units
    using SW-CMM at ML2 5, and SE-CMM)
  • Increased quality - only 2 of defects found in
    fielded system
  • Customer satisfaction increases as defects
  • Causal analysis cycles showed ROI of 131
    calculated as defects avoided per hour in

Demonstrating the Impact and Benefits of CMMI An
Update and Preliminary Results, SEI Special
Report, CMU/SEI-2003-SR-009
Other Notable CMMI Benefits Data
  • General Motors Information Systems and Services
    integrates supplier and GM products
  • Met schedules more consistently after transition
    to CMMI
  • Increased number of project milestones from 50
    to 85
  • Reduced number of days late from 50 to less than
  • Boeing, Australia Ltd. systems development for
    commercial and aerospace markets
  • Focus project transitioning from SW-CMM and EIA
    731, to CMMI
  • Quantitative
  • 33 decrease in the average cost to fix a defect
  • Turnaround time for releases cut in half
  • 60 reduction in work from Pre-Test and Post-Test
    Audits passed with few outstanding actions
  • Qualitative
  • Increased focus on product quality
  • Increased focus on eliminating defects
  • Developers seeking improvement opportunities

Examples of Return on Investment - CMM
  • Software Engineering Institute 1994 study of 13
    organizations engaged in PI showed average of 51
  • Navy FMSO reported a savings of over 2 million
    through use of Formal Inspections alone (used
    SEPOs processes as foundation of PI effort)
  • Air Logistics Center reported a 7.51 ROI with
    tenfold increase in productivity
  • Raytheon achieved a 7.71 ROI with 21
    productivity gains, defect rate reduced by 4.2X,
    reduced testing effort by half. Received 9.6M
    bonus for early delivery
  • PRC reduced defects in documentation by 78,
    defects in code by 70, defects found
    operationally by 60, ability to meet monthly
    cost goals increased by 40
  • Boeing found cycle time was reduced up to 50,
    productivity was increased by 240, and realized
    a cost-to-benefit ratio of 17
  • Ogden Air Logistics Center spent 5M to reach
    Level 5 received over 100M in new work (19-to-1

Process Improvements at Motorola
1996 1997
1998 SEI CMM Level 2 3 4 Inspection Efficiency
1 1.68 2.00 Defect Density, per
KSLOC 1 .54 .47 Build Cycle Time NA 1 .19 Test
Rate, KLOC/Days 1 4.7 20 Test Rate,
reqts/week 1 1 2.7 Test Productivity, LOC/Stf
Wk 1 1.66 3.07 - Motorola, Systems
Solutions Group, SPIN 9/5/98
What is the Real Cost of not Using Best Practices?
About PowerShow.com