DoD Systems Engineering and Management Implications for Evolutionary Acquisition of Major Defense Systems - PowerPoint PPT Presentation

Loading...

PPT – DoD Systems Engineering and Management Implications for Evolutionary Acquisition of Major Defense Systems PowerPoint presentation | free to download - id: 82ef86-N2I5Z



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

DoD Systems Engineering and Management Implications for Evolutionary Acquisition of Major Defense Systems

Description:

DoD Systems Engineering and Management Implications for Evolutionary Acquisition of Major Defense Systems A DoD SERC Quick-Look Study and CSER 2010 Invited Presentation – PowerPoint PPT presentation

Number of Views:90
Avg rating:3.0/5.0
Slides: 34
Provided by: Richard2200
Learn more at: http://csse.usc.edu
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: DoD Systems Engineering and Management Implications for Evolutionary Acquisition of Major Defense Systems


1
DoD Systems Engineering and Management
Implications for Evolutionary Acquisition of
Major Defense Systems
  • A DoD SERC Quick-Look Study
  • and CSER 2010 Invited Presentation
  • Barry Boehm, Jo Ann Lane, USC-CSSE
  • March 17, 2010
  • Charts with notes

2
Summary Evolutionary Acquisition (EvA)
  • EvA is the preferred DoD strategy for MDAPs (DoDI
    5000.02)
  • It is more responsive to DoDs current and future
    challenges
  • There are several forms of EvA
  • With different strengths and shortfalls
  • EvA requires significant advances in current
    practices
  • In systems engineering (SE) and development
    processes, particularly to avoid unscalable,
    easiest-first, sunny-day architectural
    commitments
  • In rapid, adaptive integration of SE and
    acquisition management (AM)
  • In financial and human resource allocation
  • In workforce capability and empowerment
  • In research and technology priorities and
    transition speedup
  • Initiatives are needed to provide these advances
    in time for EvA to succeed

3
What is Evolutionary Acquisition?
  • An acquisition approach that involves
  • Delivering mission capability in increments
  • Continuing reassessment of future-increment
    priorities
  • It departs from the outdated assumptions
    underlying traditional acquisition
  • System requirements can be specified in advance
  • System requirements are largely stable
  • Full operational capability cost estimation is
    feasible
  • Full-development, up-front Integrated Master
    Plans, Integrated Master Schedules, and Earned
    Value Management Plans are feasible
  • Its successful application requires rethinking
    traditional systems engineering (SE) and
    acquisition practices

4
Future DoD Challenges Asymmetric Conflict and
OODA Loops
  • Adversary
  • Picks time and place
  • Little to lose
  • Lightweight, simple systems and processes
  • Can reuse anything
  • Defender
  • Ready for anything
  • Much to lose
  • More heavy, complex systems and processes
  • Reuse requires trust

Orient with respect to stakeholders priorities,
feasibility, risks
Observe new/updated objectives, constraints,
alternatives
Decide on next-cycle capabilities, architecture
upgrades, plans
  • Act on plans, specifications

Development Commitment Milestone for Cycle
5
Average Change Processing Time Two Complex
Systems of Systems
  • Average workdays to process changes

Incompatible with turning within adversarys OODA
loop
6
Summary Evolutionary Acquisition (EvA)
  • EvA is the preferred DoD strategy for MDAPs (DoDI
    5000.02)
  • It is more responsive to DoDs current and future
    challenges
  • There are several forms of EvA
  • With different strengths and shortfalls
  • EvA requires significant advances in current
    practices
  • In systems engineering (SE) and development
    processes, particularly to avoid unscalable,
    easiest-first, sunny-day architectural
    commitments
  • In rapid, adaptive integration of SE and
    acquisition management (AM)
  • In financial and human resource allocation
  • In workforce capability and empowerment
  • In research and technology priorities and
    transition speedup
  • Initiatives are needed to provide these advances
    in time for EvA to succeed

7
There is No One-Size-Fits-All EvA Model
Type Examples Pros Cons
Prespecified Sequential Platform base plus PPPIs Prespecifiable full-capability requirements, scalability when stable Emergent requirements or rapid change, architecture breakers
Evolutionary Sequential Small Agile Larger Rapid fielding Adaptability to change, need for usage feedback Easiest-first late, costly fixes SysE time gaps Slow for large systems
Evolutionary Overlapped Stable development Maturing technology Mature technology upgrades Emergent requirements or rapid change SysE time gaps
Evolutionary Concurrent Rapid, emergent development Systems of systems Emergent requirements or rapid change, SysE continuity Overkill on small or highly stable systems
Time phasing terms Scoping Architecting
Developing Producing Operating
(SADPO)   Prespecified Sequential SA DPO1
DPO2 DPO3 Evolutionary Sequential
SADPO1 SADPO2 SADPO3 Evolutionary
Overlapped SADPO1
SADPO2
SADPO3
Evolutionary Concurrent SA D1 PO1
SA2
D2 PO2
SA3 D3 PO3
8
Evolutionary Acquisition (EvA) Decision Table
Type Stable, prespecifiable requirements? OK to wait for full system to be developed? Need to wait for next-increment priorities? Need to wait for next-increment enablers?
Single Step Yes Yes
Prespecified Sequential Yes No
Evolutionary Sequential No No Yes
Evolutionary Overlapped No No No Yes
Evolutionary Concurrent No No No No
Example enablers Technology maturity
External-system capabilities Needed resources
9
Evolutionary Overlapped Form per DoDI 5000.02
Defers the start of a new increment until its
technology is mature
Figure 2. Requirements and Acquisition Process
Flow.
10
Evolutionary Concurrent Rapid Change with High
Assurance
Unforeseeable Change (Adapt)
Rapid Change
Agile Rebaselining for Future Increments
Future Increment Baselines
Short Development Increments
Deferrals
Foreseeable Change (Plan)
Short, Stabilized Development of Increment N
Increment N Transition/ Operations and Maintenance
Increment N Baseline
Stable Development Increments
Artifacts
Concerns
High Assurance
Future VV Resources
Verification and Validation (VV) of Increment N
Current VV Resources
Continuous VV
11
Evolution and Maintenance
  • EvA often includes maintenance of fielded
    increments
  • Changes to fielded increments must be coordinated
    with development of new increments
  • Even if a separate organization handles
    maintenance
  • As above, there is no one-size-fits-all process
    for this
  • Can involve an evolutionary next-increment with a
    Milestone B gate
  • As in USD(ATL) Designation of MDAP Subprograms
    memo Carter, 2009
  • Can also involve pools of maintainers performing
    low levels of as-needed maintenance not requiring
    Milestone Bs
  • Cant have predetermined process for change
    traffic
  • Best available process shown in next chart
  • Situation even more complex with systems of
    systems
  • May need to integrate all types of evolution
    processes

12
Agile Change Processing and Rebaselining
13
Current and Future DoD Challenges Systems of
Systems
SoS SE Level
Constituent System n (pre-existing)
? ? ?
? ? ?
Increment m
Increment m1
? ? ?
Constituent System B (pre-existing)
? ? ?
? ? ?
Increment n-1
Increment n
Increment n1
MS C
MS A
MS B
PD
OS
New System A
? ? ?
? ? ?
EMD
TD
MSA
DoD Systems Engineering Guide for SoS, 2008
14
Summary Evolutionary Acquisition (EvA)
  • EvA is the preferred DoD strategy for MDAPs (DoDI
    5000.02)
  • It is more responsive to DoDs current and future
    challenges
  • There are several forms of EvA
  • With different strengths and shortfalls
  • EvA requires significant advances in current
    practices
  • In systems engineering (SE) and development
    processes, particularly to avoid unscalable,
    easiest-first, sunny-day architectural
    commitments
  • In rapid, adaptive integration of SE and
    acquisition management (AM)
  • In financial and human resource allocation
  • In workforce capability and empowerment
  • In research and technology priorities and
    transition speedup
  • Initiatives are needed to provide these advances
    in time for EvA to succeed

15
EvA Differences SE and Development Processes
  • EvA not a one-size-fits-all process
  • As shown in charts above
  • No clear boundary between development,
    operations and maintenance
  • Dont release your key SE people at PDR
  • Short prioritized increments Schedule as
    independent variable (SAIV)
  • Architect to add or drop borderline-priority
    features to meet schedule
  • Exceptions for non-subsettable deliverables
  • Earlier testing, verification, and validation
    (VV)
  • Architecture, infrastructure VV to avoid
    unscalable easiest-first increments
  • Earlier/continuous test, VV
  • Rapid, concurrent, pro-active SE and acquisition
    management
  • Of product and process requirements and
    solutions development and evolution
    hardware/software/human factors
  • Proactive vs. reactive with respect to
    technology, NDI, external interfaces
  • Evidence/risk as decision criterion, first class
    deliverable
  • Process for evidence preparation
  • Risk/opportunity-driven avoid overkill,
    underkill (balance risk, opportunity)
  • Competitive prototyping as a way of buying
    information to reduce risk

16
The SAIV Process ModelCross Talk, January 2002
(http//www.stsc.hill.af.mil/crosstalk)
  • 1. Shared vision and expectations management
  • 2. Feature prioritization
  • 3. Schedule range estimation and core-capability
    determination
  • - Top-priority features achievable within fixed
    schedule with 90 confidence
  • 4. Architecting for ease of adding or dropping
    borderline-priority features
  • - And for accommodating post-IOC directions of
    growth
  • 5. Incremental development
  • - Core capability as increment 1
  • 6. Change and progress monitoring and control
  • - Add or drop borderline-priority features to
    meet schedule
  • Schedule As Independent Variable Feature set as
    dependent variable
  • Also works for cost, schedule/cost/quality as
    independent variable
  • Also known as timeboxing, time-determined
    development, time-certain development

17
Rapid, Adaptive Acquisition Management
  • Need to rework traditional guidance documents
  • No longer prespecified, total-package, sequential
    IMP, IMS , WBS, EVMS
  • Concurrently engineered with evidence-based
    decision milestones
  • No one-size-fits-all contracting
  • E.g., for build-to-spec, agile rebaselining, VV
    teams
  • SE budgets a function of system size,
    criticality, requirements volatility
  • Award fees rewarding collaborative, effective
    change adaptation
  • Continuous monitoring INCOSE leading indicators
    SERC effectiveness measures, or equivalent
  • Evidence preparation schedule exit criteria
  • Success-critical stakeholders participation
    validation by experts
  • Competitive prototyping critical success factors
  • Acquisition corps empowerment
  • Next generation of tools, education, and
    training career paths

18
Types of Milestone Reviews
  • Schedule-based reviews (contract-driven)
  • Well hold the PDR on April 1 whether we have a
    design or not
  • High probability of proceeding into a Death March
  • Event-based reviews (artifact-driven)
  • The design will be done by June 1, so well have
    the review then
  • Large Death by PowerPoint and UML event
  • Hard to avoid proceeding with many unresolved
    risks and interfaces
  • Evidence-based commitment reviews (risk-driven)
  • Evidence provided in Feasibility Evidence
    Description (FED)
  • A first-class deliverable
  • Based on concurrently engineered ConOps, specs,
    and plans
  • Shortfalls in evidence are uncertainties and
    risks
  • Should be covered by risk mitigation plans
  • Stakeholders decide to commit based on risks of
    going forward

19
Content of Evidence-Based Reviews
  • Evidence provided by developer and validated by
    independent experts that
  • If the system is built to the specified
    architecture, it will
  • Satisfy the specified operational concept and
    requirements
  • Capability, interfaces, level of service, and
    evolution
  • Be buildable within the budgets and schedules in
    the plan
  • Generate a viable return on investment
  • Generate satisfactory outcomes for all of the
    success-critical stakeholders
  • Shortfalls in evidence are uncertainties and
    risks
  • Should be resolved or covered by risk management
    plans
  • Assessed in increasing detail at major anchor
    point milestones
  • Serves as basis for stakeholders commitment to
    proceed
  • Serves to synchronize and stabilize concurrently
    engineered elements

Can be used to strengthen current schedule- or
event-based reviews
20
Scalable remotely controlled operations
21
Total vs. Incremental Commitment 41 RPV
  • Total commitment
  • Agent technology demo and PR Can do 41 for 1B
  • Winning bidder 800M PDR in 120 days 41
    capability in 40 months
  • PDR many outstanding risks, undefined interfaces
  • 800M, 40 months halfway through integration
    and test
  • 11 IOC after 3B, 80 months
  • Incremental commitment number of competing
    teams
  • 25M, 6 mo. to MDD 4 may beat 12 with agent
    technology, but not 41
  • 75M, 8 mo. to Milestone A 3 agent technology
    may do 11 some risks
  • 225M, 10 mo. to Milestone B 2 validated
    architecture, high-risk elements
  • 675M, 18 mo. to IOC 1 viable 11 capability
  • 11 IOC after 1B, 42 months

22
Finance, Human Resource Allocation
  • Need balance of short-term, long-term budget
    commitments
  • Budget for long-term continuity but build in
    short-term off-ramp options
  • E.g., for technology maturity as in the DoDI
    5000.02 example of EvA in chart 9
  • Prespecified total-package capabilities, budgets,
    and schedules generally infeasible
  • Higher up-front SE schedules, costs for larger,
    more critical programs
  • Earlier infrastructure and test plans and
    preparations
  • Heavier penalties for late rework generally high
    payoffs for competitive prototyping
  • Program-specific budgets, schedules, staffing
    established via mix of parametric models and
    expert judgment
  • Need to address total cost of ownership,
    lifecycle ROI (DOTMLPF)
  • Lifecycle success-critical stakeholder
    involvement
  • Mission effectiveness analysis with respect to
    alternatives
  • Use to help prioritize increments
  • Use as basis for monitoring progress vs. plans
  • Continuing update of resources required

23
Risk of Delaying System-Specific Knowledge?
Blanchard-Fabrycky, 1998, pg. 37
Improvement Opportunities
24
Effect on Size on SE Investment Payoff
25
Effect of Volatility and Criticality on SE
Investment Payoff
26
COSYSMO Operational Concept
Requirements Interfaces Scenarios
Algorithms Volatility Factor Reuse Factor
Size Drivers
COSYSMO
Effort
Effort Multipliers
  • Application factors
  • 8 factors
  • Team factors
  • 6 factors
  • Schedule driver

Calibration
WBS guided by ISO/IEC 15288
27
Workforce, Capabilities, And Empowerment
  • Increasing gaps in DoD SE Knowledge, Skills, and
    Abilities (KSAs)
  • Workforce gaps domain experts retiring
  • Needed balance of specialists and systems
    thinkers
  • Hardware/software /human factors combined
    expertise needs
  • Needed to avoid commitments to unscalable ,
    easiest-first EvA
  • Needs for life-long learning learning how to
    learn
  • Applies to acquisition corps as well as SEs
  • Need to anticipate trends in workforce
    capabilities
  • Tool empowerment
  • Life cycle coverage, interoperability, domain
    tailorability
  • Tailorability to new and wider ranges of
    warfighter skills

28
EvA Research and Technology Needs
  • Incremental vs. start-over tools and methods
    (e.g., for formal methods)
  • Ambiguity-tolerant methods, processes, and tools
  • Powerful, flexible, composable models and
    simulations
  • Rapid change-impact analysis
  • Scalable methods, processes, and tools
  • Concurrent engineering support
  • Collaboration technology quality attribute
    tradeoff analysis tools
  • Value-based architecting and design methods,
    processes, and tools
  • Continuous VV early, continuous testing
    reliable autonomy
  • Use of multi-core computing processors for rapid
    multiple-option analysis
  • Integration of hardware, software , and human
    factors solution approaches and architectures
  • Next-generation process maturity models
  • Evolutionary acquisition contract and incentive
    structures
  • Rapid technology maturation and adoption

29
System/Software Architecture Mismatches- Maier,
2006
  • System Hierarchy
  • Part-of relationships no shared parts
  • Function-centric single data dictionary
  • Interface data flows
  • Static functional-physical allocation
  • Software Hierarchy
  • Served-by relationships layered multi-access
  • Data-centric class-object data relations
  • Interface protocols concurrency challenges
  • Dynamic functional-physical migration

30
Examples of Architecture Mismatches
  • Fractionated, incompatible sensor data management
  • Touch Football interface definition earned
    value
  • Full earned value taken for defining interface
    dataflow
  • No earned value left for defining interface
    dynamics
  • Joining/leaving network, publish-subscribe,
    interrupt handling, security protocols, exception
    handling, mode transitions
  • Result all green EVMS turns red in integration

31
Summary Evolutionary Acquisition (EvA)
  • EvA is the preferred DoD strategy for MDAPs (DoDI
    5000.02)
  • It is more responsive to DoDs current and future
    challenges
  • There are several forms of EvA
  • With different strengths and shortfalls
  • EvA requires significant advances in current
    practices
  • In systems engineering (SE) and development
    processes, particularly to avoid unscalable,
    easiest-first, sunny-day architectural
    commitments
  • In rapid, adaptive integration of SE and
    acquisition management (AM)
  • In financial and human resource allocation
  • In workforce capability and empowerment
  • In research and technology priorities and
    transition speedup
  • Initiatives are needed to provide these advances
    in time for EvA to succeed

32
References - I
  • Baldwin, C. and Clark, K., Design Rules The
    Power of Modularity, MIT Press, 2000.
  • Blanchard, B. and Fabrycky, W., Systems
    Engineering and Analysis. Upper Saddle River
    Prentice Hall, 1998.
  • Boehm, B., Ingold, D., Dangle, K., Turner, R., P.
    Componation et al., Early Identification of
    SE-Related Program Risks, SERC Technical Report
    and TR USC-CSSE-2009-518, Sept. 2009.
  • Boehm, B., Port, D., Huang, L., and Brown, A.W.,
    Using the Spiral Model and MBASE to Generate New
    Acquisition Process Models SAIV, CAIV, and
    SCQAIV, Cross Talk, January 2002, pp. 20-25.
  • Boehm, B., Brown, A. W., Basili, V, and Turner,
    R. Spiral Acquisition of Software-Intensive
    Systems of Systems, CrossTalk, May 2004, pp.
    4-9.
  • Boehm, B., Valerdi, R., and Honour, E., "The ROI
    of Systems Engineering Some Quantitative Results
    for Software-Intensive Systems," Systems
    Engineering, Fall 2008, pp. 221-234.
  • Boehm, B., and Lane, J., "Guide for Using the
    Incremental Commitment Model (ICM) for Systems
    Engineering of DoD Projects, v0.5,
    USC-CSSE-TR-2009-500.
  • Carter, A. , Designation of Subprograms for
    Major Defense Acquisition Programs, USD(ATL)
    Memorandum, June 23, 2009.
  • Cusumano, M., and Selby, R., Microsoft Secrets,
    Free Press, 1995.
  • Department of Defense (DoD), Defense Acquisition
    Guidebook, version 1.6, http//akss.dau.mil/dag/,
    2006.
  • Department of Defense (DoD), Systems Engineering
    Guide for System of Systems, June 2008.
  • Department of Defense (DoD), Instruction 5000.02,
    Operation of the Defense Acquisition System,
    December 2008.
  • Fortune, J., Estimating systems engineering
    reuse with the constructive systems engineering
    cost model (COSYSMO), USC Ph.D. Dissertation,
    December 2009.

33
References - II
  • Gelosh, D., Systems Engineering Workforce
    Development Update, Proceedings, NDIA SE
    Conference , October 2009.
  • Hall, E.T., Beyond Culture, Anchor
    Books/Doubleday, 1976.
  • INCOSE Systems Engineering Handbook v.3.1
    INCOSE-TP-2003-002-03.1, 2007.
  • Johnson , J., My Life Is Failure, Standish Group,
    2006.
  • Maier, M., System and Software Architecture
    Reconciliation, Systems Engineering 9 (2), 2006,
    pp. 146-159.
  • Maranzano, J., et al. 2005. Architecture Reviews
    Practice and Experience. IEEE Software, Mar./Apr.
    2005.
  • Parnas, D., Designing Software for Ease of
    Extension and Contraction, IEEE Trans. SW Engr.,
    March 1979, pp. 128-137.
  • Pew, R. W., and Mavor, A. S., Human-System
    Integration in the System Development Process A
    New Look, National Academy Press, 2007.
  • Schroeder, T., Integrating Systems and Software
    Engineering Observations in Practice, 22nd
    International Annual Forum on COCOMO and
    Systems/Software Cost Modeling, 2007.
  • U.S. Congress, Weapon System Acquisition Reform
    Act of 2009, January 6, 2009.
  • U.S. General Accountability Office, Defense
    Acquisitions Assessments of Selected Major
    Weapon Programs, March 2008.
  • Valerdi, R, Systems Engineering Cost Estimation
    with COSYSMO, Wiley, 2010 (to appear).
  • Young, J., Prototyping and Competition,
    USD(ATL) Memorandum, September 19, 2007
About PowerShow.com