Earned Value Management and Past Performance

1
Earned Value Management and Past Performance

• NCMA Suncoast Chapter
• November 30, 2006

David Muzio DLM Consulting
2
Why Earned Value Management?

• Fluid Project Baselines
• Continuing problem
• Poor budget discipline
• FASA, Title V
• Establish Cost, Schedule and Performance Goals
  for Major Acquisitions and achieve, at least, 90
  of the goals.
• If project not within 90 of goals, Agency Head
  shall (1) determine if there is a continuing
  need for the project, and (2) identify suitable
  actions to be taken, including termination.
• Clinger-Cohen Act of 1996
• OMB - Establish the processes for executive
  agencies to analyze, track, and evaluate the
  risks and results of major investments in IT
• Report on the net program performance benefits
  achieved by agencies
• Agency Requirements
• Implement OMB policies through effective capital
  planning and performance and results based
  management

3
Why Past Performance?

• Improves contract administration and the ability
  to select the best contractors for future work.
• Provides a formal mechanism for, at least, an
  annual discussion between the customer and
  supplier on contract performance and provides a
  record of that discussion.
• Provides the Government customer a standard way
  to annually rate a suppliers performance on a
  contract.
• Allows the Government to input contract
  performance information into a central database
  for use by all other government contracting
  officers in making source selection decisions for
  new contracts.
• Encourages outstanding performance throughout the
  life of a contract.

4
Earned Value Management

• A project management tool that integrates the
  project scope of work with schedule and cost
  elements.
• ANSI/EIA Standard 748-1998-A, Earned Value
  Management Systems. Reaffirmed Aug 28, 2003
• Copies from Global Engineering Documents
• (1-800-854-7179)
• EVM information at
• www.pmi-cpm.org
• http//acc.dau.mil/evm
• http//www.ndia.org

5
History of EVM (CSCSC)

• DOD developed CSCSC early 60s
• Air Force initially developed system for the
  Minuteman Missile program. Lt. Gen. Driessnack
• Reason Major Contract Overruns, and lack of
  government visibility into contract status.
• Contractors reluctant acceptance
• concerns Government imposed, complicated and
  cumbersome 35 Criteria
• Boeing used modified CSCSC in Commercial Airplane
  Division on development of 747
• Raytheon 1974 First Multi-Plant/System
  Certification

6
History of EVM (CSCSC)

• April 1976 First OMB Circular on program
  management A-109
• 1969 Commission on Government Procurement deep
  concerns over management effectiveness of major
  systems
• Requirement to assess acquisition cost, schedule
  and performance experience against predictions,
  and provide such assessments to the agency head
  at key decision points.
• DOD adopted A-109 policies, continued to require
  CSCSC, other agencies did not.

7
History of EVM (CSCSC)

• OMB Response to FASA and CCA in 1996 - complete
  rewrite of Circular A-109
• Team of 82 people from 14 Major Agencies and GAO
• DOD working with NDIA to convert Government CSCSC
  to Industry EVM.
• DOD using draft EVM until ANSI/EIA Standard 748
  issued in 1998.
• Programs reviewed quarterly at DOD
• Programs held to 1 to 2 overruns during late
  1990s
• GAO reviewed use of EVM in State and local
  government and private industry.
• Motorola used EVM on 4.6 Billion Iridium Program
• Model for good project management with EVM used
  on fixed price subcontracts, including with
  Russia and China.
• Virginia used on construction

8
OMB Policy Developed 1995-1997

• Included in Circular A-11 - Preparation,
  Submission, and Execution of the Budget 1996
• Eight (8) Parts Issued annually
• Part 7 Planning, Budgeting, Acquisition and
  Management of Capital Assets
• Instructions on budget justification and
  reporting requirements
• requirement for a business case
• Supplement to Part 7 Capital Programming Guide
  version 1, May1997, Version 2, June 2006
• Specific program and contract management
  guidance, which is flexible policy, unless
  deviation is granted by OMB.
• Original guidance EVMS or Similar
• 2002 EVMS only to manage program and report
  status to management and OMB.

9
(No Transcript)
10
Good Policy - No Real Management By OMB No Real
Progress

• Executive Branch Management Scorecard Started
  2002 on Presidential Management Agenda. (See
  www.results.gov or www.whitehouse.gov/omb
• Good intentions and good beginnings are not the
  measure of success. What matters in the end is
  completion performance and results.
• To ensure accountability, a traffic light
  grading system is used to track how well agencies
  are executing the management initiatives and
  where they stand at any point in time.

11
(No Transcript)
12
(No Transcript)
13
Success Factors

• Justify projects with complete business cases
  adequately addressing, expected life-cycle costs,
  acquisition strategy, performance standards tied
  to agency strategic goals, alternatives analysis,
  risk management, cost and schedule analysis
  (EVMS) and
• Demonstrate management of projects using EVMS
  achieving, on average, 90 of cost, schedule and
  performance goals and for IT
• Secure at least 90 of operational systems and
• Have an effective Enterprise Architecture and
• Avoid redundant or agency unique systems

14
EIA 748 is a Guideline

• High level and goal oriented
• States qualities and operational considerations
  of an integrated management system using EV
  analysis methods
• Does not mandate detail system characteristics
• Process must fit firms management style and
  business environment.
• One size does not fit all contracts

15
EVMS Principles

• Plan all work scope for the program to completion
• Break down the program work scope into finite
  pieces that can be assigned to a responsible
  person or organization for control of technical,
  schedule and cost objectives.
• Integrate program work scope, schedule and cost
  objectives into a performance measurement
  baseline plan against which accomplishments may
  be measured.
• Control changes to the baseline.

16
EVMS Principles (Cont)

• Use actual costs incurred and recorded in
  accomplishing the work performed
• Objectively assess accomplishments at the work
  performance level
• Analyze significant variances from the plan,
  forecast impacts, and prepare an estimate at
  completion based on performance to date and work
  to be performed
• Use EVMS in the Companys Management processes.

17
EVMS Is A Central Key to Project Management

• EVMS required on all contracts with development
  effort
• Application of EVM principles (use WBS) begins
  during the program definition to ensure that
  technical requirements and schedule and cost
  estimates are properly integrated to support
  program authorization.
• Program baselines are finalized and risks
  identified by ensuring the contractor or in-house
  project is using an ANSI-748 System and through
  the completion of an Integrated Baseline Review
• Project management is accomplished through
  implementation of EVMS and the resulting
  reporting of achievement or variations from cost,
  schedule and performance goals.
• Variations from plan are expected, it is how they
  are handled that is important.
• Do not use incentives to achieve 1.0 CPI and SPI

18
CONTRACT WBS (CWBS)
LEVEL 1 LEVEL 2 LEVEL 3 LEVEL 4 LEVEL 5
COST ACCOUNTS
19
WHY HAVE A WORK BREAKDOWN STRUCTURE (WBS)?

• Provides a product framework for organization and
  management of the project
• provides a common framework for planning and
  controlling cost, schedule, and work scope
• allows work to be broken down into cost accounts
  --- the lowest level of visibility

20
Responsibility Assignment Matrix
The control account level is determined by the
responsibility assignment matrix (RAM). The RAM
is defined by combining the contract WBS with the
organizational structure.
21
A COST ACCCOUNT IS MADEUP OF WORK PACKAGES
In-Progress Work Package
50
65
80
Planned Work Package
50
Completed Work Package
25
70
30
J F M A M J J A
S O N D
22
ESTABLISH THE BASELINE

• 1. DEFINE THE WORK

2. SCHEDULE THE WORK

3. ALLOCATE BUDGETS
100
TIME
60
80
20
40
15
40
23
The Time-Phased Spend Plan


Years
24
Earned Value Measurement
PLANNED
ACTUAL COST
PERFORMED
2
2
2
2
15
10
10
5
5
5
5
5
5
5
5
25
3
3
3
3
BUDGET 50
EARNED VALUE 35
COST 40
STATUS Schedule Variance Earned - Budget -
15 Unfavorable Cost Variance
Earned - Actual - 5 Unfavorable
25
5 PERFORMANCE INDICATORS
26
COST PERFORMANCE REPORTINGKEY DATA ELEMENTS
27
What Capital Investments Require a Business Case
(Exhibit 300) ?

• All major capital investments
• Land, structures, equipment, intellectual
  property (e.g., software), information technology
  (including IT service contracts), environmental
  restorationSee appendix 1 of the CPG)
• All major IT investment business cases must be
  submitted to OMB with the Budget
• For all other types of capital assets OMB shall
  request a sample of the business cases (Section
  II.9.2. CPG)
• Important note All information necessary to
  complete an Exhibit 300 already exists as part of
  project specific documentation. Materials used
  to populate the Exhibit 300 should be readily
  available to OMB upon request.

28
Section C Acquisition/Contract Strategy

• 1. Complete the table
• Contract of Task Order Number
• Type of contract/task order
• Has the contract been awarded (Y/N)
• Date of award, if not awarded, planned date
• Start and end date of contract/task order
• Total Value of Contract/T.O. (includes all
  options)
• Is this an interagency acquisition?
• Is it performance-based?
• Competitively awarded? (Y/N) (many T.O.s will
  not be).
• Alternative financing? (ESPC, UESC, EUL, N/A)
• Is EVM in the contract? (Y/N)
• Does the contract include the required security
  and privacy clauses? (Y/N)

29
Section C Acquisition/Contract Strategy

• Name of Contracting Officer
• CO contact information (phone/email)
• CO Certification Level. ( 1, 2, 3, N/A)
• If N/A, does the CO have the competencies and
  skills necessary
• 2. If EVM is not required or will not be a
  contract requirement for any of the
  contracts/T.O.s, explain why
• Do the contracts ensure Section 508 compliance?
• Is there an acquisition plan which has been
  approved in accordance with agency requirements
• If yes, what date?
• If no, will an acquisition plan be developed?If
  no, explain why

30
Acquisition Plan

• Actual plan developed must meet the requirements
  in
• FAR Part 7, Acquisition Plans
• FAR Part 34, Major System Acquisition
• OMBs Capital Programming Guide
• Phase 1 Planning and Budgeting
• Phase II Acquisition

31
FAR 7.105(b)(10) Management Information
Requirements.

• If an EVMS is to be used, discuss the methodology
  the Government will employ to analyze and use the
  EV data to assess and monitor contract
  performance.
• Discuss how the Offerors/Contractors EVMS will
  be verified for compliance with ANSI/EIA 748 and
  
• The timing and conduct of IBRs (whether prior to
  or post award).

32
7.105(b)(3) Source Selection Procedures

• When EVM is required (see FAR 34.202(a)) and a
  pre-award IBR is contemplated, the acquisition
  plan must discuss
• How the pre-award IBR will be considered in the
  source selection decision
• How it will be conducted in the source selection
  process (see FAR 15.306) and
• Whether offerors will be directly compensated
  for the costs of participating in a pre-award
  IBR.

33
CPG Contracting

• 1.5.5.5 Integrating EV into Acquisition Strategy.
  
• All contracts with EVM are required to have an
  IBR pre- or post-award to finalize the agreement
  on the baseline and ensure the risks are
  identified and understood.
• Depending on the risk to establishing an
  achievable performance measurement baseline at
  time of contract award, the use of an IBR before
  or after award must be determined.
• Nevertheless, agencies are expected to achieve at
  the completion of the contract at least 90
  percent of the cost, schedule and performance
  goals established at the time of contract award.

34
Part 34, Major System Acquisition

• 34.2, Earned Value Management Systems,
• (a) An EVMS is required for major acquisitions
  for development, IAW OMB A-11, The Government may
  also require an EVMS for other acquisitions, IAW
  agency procedures.
• (b) Offerors that propose to use a system not in
  compliance with 748 must submit a comprehensive
  plan for compliance
• (e) COs will determine the adequacy of the
  proposed EVMS plan prior to contract award
• (c) At a minimum require monthly EVM reports
• (d) EVMS requirement will be applied to
  subcontractors using the same rules as applied to
  the prime contractor.

35
Two Provisions, One Clause for EVM

• 52.234-2 Notice of EVMS Pre-Award IBR
• 52.234-3 Notice of EVMS- Post-Award IBR
• The above are the same, except the Pre-Award IBR
  provision has a paragraph (d) The Government
  will conduct an IBR, as designated by the agency,
  prior to contract Award.
• 52.234-4 Earned Value Management System
• (a) The contractor shall use an EVMS that has
  been determined by the Cognizant Federal Agency
  (CFA) to be compliant with the guidelines in
  ANSI/EIA 748 (current version at time of award)
  to mange the contract. (See NDIA EVM Systems
  Acceptance Guide)
• (d) The CO may require an IBR at
• Exercise of significant options, or
• Incorporation of major modifications.
• CPG I.5.5.5 - An IBR must be accomplished
  whenever there are major changes to the baseline.
  

36
CPG Contracting

• TM contracts should only be used in the planning
  phase and only when there is insufficient
  knowledge about the requirement to be able to use
  a cost reimbursement contract.
• Agencies should only require in the contract
  sufficient goods or services to result in the
  agency receiving complete useful assets. Do not
  incentive contractors to provide more than needed
  to meet strategic goals.
• EVM is normally used on Fixed-price incentive
  contracts and cost reimbursement contracts. EVM
  shall also be used on firm-fixed price and any
  other type of contract that meets the major
  acquisition threshold if that contract contains a
  significant amount of development work.

37
Part II-Section B Risk Management

• Expected to have performed a risk assessment
  during the early planning and initial concept
  phase, developed a risk adjusted life-cycle cost
  estimate and a plan to eliminate, mitigate or
  manage risk, and be actively managing risk
  throughout investments life-cycle
• 1. Does the investment have a Risk Management
  Plan?
• If yes, date
• Has the plan been significantly changed since
  last years submission to OMB?
• If yes, describe significant changes.
• 2. If no plan When?
• 3. Briefly, describe how investment risks are
  reflected in the life-cycle cost estimate and
  investment schedule
• Use appendix 5, Risk Management, as the basis for
  your risk management process
• Use appendix 9, Cost Estimating, as the basis for
  your cost estimating process
• Agencies are expected to award contracts that
  have a high probability of achieving at least 90
  of the cost, schedule and performance goals
  established in the planning phase.

38
CPG Risk Principles

• Risk management leads to the development of a
  Risk-Adjusted Program Budget and Risk-Adjusted
  Schedule (See NDIA EVM Application Guide)
• The IPT must ensure that the proposals and
  in-house estimates clearly recognize the amount
  and impact of risk on cost, schedule and
  technical effort.
• New technology should be subject to Technical
  Readiness Level (TRL) and Degree of Difficulty
  (RD3) reviews to help determine the risk and
  necessary reserves.
• High risk should be accepted only if it can be
  justified by high expected returns, and only if
  program failure can be absorbed by the agency
  with loss of service capability or significant
  effect on the budget.
• Decision thresholds should be set for cost,
  schedule and performance expectations of
  development projects beyond which the ROI becomes
  so low that the project should be cancelled.

39
Part II-Section C Cost and Schedule Performance

• EVM is required only on DME (development)
  portions of investments
• For mixed life-cycle, OM milestones should still
  be included in the table (Comparison of Initial
  Baseline and Current Approved Baseline.)
• Does the EVMS meet the criteria in 748? Y/N
• What is
• PV, EV, AC
• What costs are included, Govt. Only/Contractor
  Only/Both?
• SPI
• SV
• CPI
• CV
• Is the CV or SV greater than /- 10
• Explain variance
• Use the EVM system to identify the specific work
  packages where problems are occurring.
• Corrective Actions, with risk included
• Current EAC
• Any significant changes to baseline in past year?
• If yes, when approved by OMB?

40
(No Transcript)
41
EVMS Data Indicated EVM Not Being Implemented
Properly

• In 2005 many agencies submitted EVM data showing
  SPI and CPI close to 1
• Agencies at same time asking for budget increase.
  
• For the 2006 Budget, OMB requested agencies to
  first self-evaluate their business cases, and
• Before beginning development, ensure cost,
  schedule and performance goals are independently
  validated for reasonableness.
• Inspector Generals, current IVV reviewers, or
  any other source internal or external to the
  agency, including another agency.
• Pre-Award Integrated Baseline Reviews may be
  substituted for a independent validation.

42
Requirements for Fully Implementing EVMS

• 1. Comprehensive agency policies by December 31,
  2005 New or Verified
• --Use NDIA EVMS Application Guide and those
  listed below
• 2. EVMS requirements in contracts or agency
  in-house project charters
• 3. Compliance Reviews of agency and contractor
  EVM systems
• - Use NDIA Intent Guide and Systems Acceptance
  Guide

43
Requirements for Fully Implementing EVMS for IT
Projects

• 4. Periodic system surveillance reviews to ensure
  the EVMS continues to meet the guidelines in
  ANSI-748
• -- Use NDIA Surveillance Guide
• 5. Integrated Baseline Reviews
• --Use NDIA Program Managers Guide to the
  Integrated Baseline Review.

44
Benefits of EVMS

• Reduces propensity of customers/boss to add work
  without adding budget
• Ties budget directly to work
• Requires all work transfers to include associated
  budget
• Requires all budget transfers to include
  associated work
• Fosters management decisions within a framework
  of reality, rather than latent unease.

45
Questions ?

• Agencies have flexibility in how they implement
  the key principles and concepts in the CPG.
• However, the key principles and importance of
  thorough planning, risk management, full funding,
  portfolio analysis, performance-based acquisition
  management, accountability for achieving the
  established goals will not change
• In General, OMB will only consider recommending
  for funding in the Presidents Budget priority
  capital asset investments that comply with good
  capital programming principles.

46
Past Performance

• The Federal Acquisition Streamlining Act of 1994,
  mandated the collection and use of contractor
  performance information.
• FAR Subpart 42.15 Contractor Performance
  Information
• FAR 42.1501
• Past performance information is relevant
  information, for future source selection
  purposes, regarding contractor actions under
  previously awarded contracts.

47
FAR 42.1501

• Past performance information includes
• Contractors record of conforming to contract
  requirements and standards of good workmanship
• Record of forecasting and controlling costs
• Adherence to contract schedules, including the
  administrative aspects of performance
• History of reasonable and cooperative behavior
  and commitment to customer satisfaction
• Generally, the business-like concern for the
  interest of the customer

48
FAR 42.1502 Policy

• Evaluations are required for each contract in
  excess of 100,000.
• At time of contract completion
• Contracts exceeding one year in duration,
  including options require interim evaluations
• Evaluation is generally for the entity, division,
  or unit that performed the contract
• Each agency establishes procedures
• Generally use scale of 1 to 5 with narrative

49
FAR 42.1503

• Technical office, contracting office and end
  users input to the evaluations
• Evaluations are provided to the contractors
• Contractors have 30 days to respond
• Disagreements reviewed at a level above the
  Contracting Officer
• Marked Source Selection Information
• Evaluations revealed to other Government COs and
  the Contractor.
• Evaluations retained for three (3) years

50
Past Performance Information Retrieval
System(PPIRS)

• Web-enabled Government-wide application which
  currently collects information from
• Past Performance Data Base NASA
• Past Information Management System ARMY and
  DISA
• Contractor Performance Assessment Rating System
• Navy, USMC, Air Force, DLA and other Defense
  Agencies
• Construction Contractor Appraisal support System
• Contractor Performance System - NIH
• HHS, Agriculture, Treasury, Commerce, Justice,
  Energy, Interior, Labor, VA, SSA, USAID, EPA,
  FEMA GSA, Transportation, Education, State,
  Export-Import Bank and the Architect of the
  Capital
• All of the above subsystems will be turned off
  next year?

51
New Guide and FAR Rule

• Contractor Performance in the Acquisition Process
  
• Draft open for comment at http//www.acquisition.
  gov
• Comments due January 16, 2007
• FAR rule currently being processed and will be
  issued for comment at a later date

52
Chapter 1- Pre-Solicitation Use of Performance
Information

• Designing Factors and Subfactors
• Should be designed to evaluate the key
  performance requirements of the solicitation.
• Whether done as a stand-alone evaluation factor,
  or integrated with other factors, the
  solicitations should request the offerors record
  for
• On-time delivery
• Technical quality and accomplishments
• Cost control
• Business relations
• Performance on subcontracting plans/programs.

53
Exhibit 3-1
54
Contractor Performance Information Report Summary
Exhibit 3-11
55
Contract Management at its Simplest

• What is the Government getting for the money it
  is spending?
• If it was your money would you spend it?
• Questions