Constructive System of Systems Integration Cost Model (COSOSIMO) ****************** Tutorial

1
Constructive System of Systems Integration Cost
Model (COSOSIMO)Tutorial

• Jo Ann Lane, jolane_at_usc.edu
• USC Center for Systems Software Engineering
• http//csse.usc.edu23 October 2006

2
Overview

• COSOSIMO Background
• System of Systems (SoS) and SoS Engineering
  (SoSE) Environment
• Current COSOSIMO Cost Estimation Approach
• Conclusions
• References

3
COCOMO Cost Model Suite Overview
Barry Boehm, Ricardo Valerdi, Jo Ann Lane, and
Winsor Brown, COCOMO Suite Methodology and
Evolution, CrossTalk, April 2005.
4
USC-CSE Modeling Methodology
Analyze existing literature Step 1
Concurrency and feedback implied
Perform Behavioral analyses Step 2
Identify relative significance Step 3
Perform expert-judgment Delphi assessment,
formulate a-priori model Step 4
Gather project data Step 5
Determine Bayesian A-Posteriori model Step 6
Gather more data refine model Step 7
Boehm, et. al., Software Cost Estimation with
COCOMOII, 2000.
5
Goal of Research

• Develop a cost model (COSOSIMO) to
• Support the estimation of effort associated with
  System-of-System Engineering (SoSE)
• May be performed by one or more Lead System
  Integrator (LSI) organizations
• Complement the other USC CSE cost models for
  software development, system engineering (SE),
  and Commercial-Off-the-Shelf (COTS) integration,
  leading toward a more comprehensive and unified
  cost model to support the much broader system of
  interest life cycle

COSOSIMO will not estimate the total SoS
development costs, but rather just the SoSE
costs at the SoS level
6
History of COSOSIMO Model
Early 2003 Potential need for SoSE cost model identified
Fall/Winter 2003 Initial model developed based on software size
Fall 2004 Early design model based of SoS architecture characteristics (not software size)
Spring/Summer 2005 EIA 632-based survey conducted to determine SoSE differences from traditional systems engineering
Fall 2005 SoSE WBS analysis
Fall/Winter 2005 2-submodel version of COSOSIMO investigated
Spring/Summer 2006 SoSE-specific characteristics captured from SoSE conferences/workshops
Spring 2006 3-submodel version of COSOSIMO proposed
7
What is a System-of-Systems?

• Very large systems developed by creating a
  framework or architecture to integrate component
  systems
• SoS component systems independently developed and
  managed
• New or existing systems
• Have their own purpose
• Can dynamically come and go from SoS
• SoS exhibits emergent behavior not otherwise
  achievable by component systems
• SoS activities often planned and coordinated by a
  Lead System Integrator (LSI)
• Typical domains
• Business Enterprise-wide and cross-enterprise
  integration to support core business enterprise
  operations across functional and geographical
  areas
• Military Dynamic communications infrastructure
  to support operations in a constantly changing,
  sometimes adversarial, environment
• INCOSE Handbook Definition Systems of Systems
  are defined as an interoperating collection of
  component systems that produce results
  unachievable by the individual systems alone.
  (Krygiel 1999)

8
What is a Lead System Integrator?

• Organization (or set of organizations) selected
  to accomplish the definition and acquisition of
  SoS components, and the continuing integration,
  test, and evolution of the components and SoS
• Typical activities
• Lead concurrent engineering of requirements,
  architecture, and plans
• Identify and evaluate technologies to be
  integrated
• Conduct source selection
• Coordinate supplier activities and validate SoS
  architecture feasibility
• Integrate and test SoS-level capabilities
• Manage changes at the SoS level and across the
  SoS-related IPTs
• Manage evolving interfaces to external systems
• Typically do not develop system components to be
  integrated (possible exception SoS
  infrastructure)

9
What is SoSE

• USAF SAB Report on SoSE for Air Force Capability
  (USAF 2005) The process of planning, analyzing,
  organizing, and integrating the capabilities of a
  mix of existing and new systems into a
  system-of-systems capability that is greater than
  the sum of the capabilities of the constituent
  parts. This processes emphasizes the process of
  discovering, developing, and implementing
  standards that promote interoperability among
  systems developed via different sponsorship,
  management, and primary acquisition processes.
• National Centers for Systems of Systems
  Engineering (NCOSOSE) The design, deployment,
  operation, and transformation of metasystems that
  must function as an integrated complex system to
  produce desirable results. These metasystems are
  themselves comprised of multiple autonomous
  embedded complex systems that can be diverse in
  technology, context, operation, geography, and
  conceptual frame. (http//www.eng.odu.edu/ncsose/
  what_is_SOSE.shtml)

10
What is SoSE (continued)

• Wikipedia (http//en.wikipedia.org/wiki/System_of_
  Systems_Engineering) SoSE is a set of developing
  processes and methods for designing and
  implementing solutions to System-of-Systems
  problems. SoSE is relatively new term being used
  in Department of Defense applications, but is
  increasingly being applied to non-military/securit
  y related problems (e.g. transportation,
  healthcare, internet, search and rescue, space
  exploration). SoSE is more than systems
  engineering of complex systems because design for
  System-of-Systems problems is performed under
  some level of uncertainty in the requirements and
  the constituent systems, and it involves
  considerations in multiple levels and domains.
• SoSE and Systems Engineering are related but
  different fields of study. Where as systems
  engineering addresses the development and
  operations of products, SoSE addresses the
  development and operations of programs. In other
  words, traditional systems engineering seeks to
  optimize an individual system (i.e., the
  product), while SoSE seeks to optimize network of
  various systems brought together to meet specific
  program's (i.e., the SoS problem's) objectives.
  SoSE enables decision-makers to understand the
  implications of various choices thus, SoSE
  methodology seeks to prepare the decision-makers
  for effective architecting of System-of-Systems
  problems.
• Due to varied methodology and areas of
  applications in existing literature, there is no
  unified consensus for processes involved in
  System-of-Systems Engineering. One of the
  proposed SoSE frameworks, by Dr. Daniel A.
  DeLaurentis, recommends a three-phase method
  where a SoS problem is defined (understood),
  abstracted, modeled and analyzed for behavioral
  patterns.

11
SoSE Compared to Traditional SE Activities

• Traditional SE Activities (EIA/ANSI 632)
• Acquisition and supply
• Product Supply
• Product Acquisition
• Supplier Performance
• Technical management
• Process Implementation Strategy
• Technical Effort Definition
• Schedule and Organization
• Technical Plans
• Work Directives
• Progress Against Plans and Schedules
• Progress Against Requirements
• Technical Reviews
• Outcomes Management
• Information Dissemination
• System design
• Acquirer Requirements
• Other Stakeholder Requirements

• Traditional SE Activities (continued)
• Product realization
• Implementation
• Transition to Use
• Technical evaluation
• Effectiveness Analysis
• Tradeoff Analysis
• Risk Analysis
• Requirements Statements Validation
• Acquirer Requirements Validation
• Other Stakeholder Requirements Validation
• System Technical Requirements Validation
• Logical Solution Representations Validation
• Design Solution Verification
• End Product Verification
• Enabling Product Readiness
• End Products Validation

12
SoSE Compared to Traditional SE Activities
(continued)

• Key Areas Where SoSE Activities Differ From
  Traditional Systems Engineering
• Architecting composability vs. decomposition
  (Meilich 2006)
• Added ilities such as flexibility,
  adaptability, composability (USAF 2005)
• Net-friendly vs. hierarchical (Meilich 2006)
• First order tradeoffs above the component systems
  level (e.g., optimization at the SoS level,
  instead of at the component system level) (Garber
  2006)
• Early tradeoffs/evaluations of alternatives
  (Finley 2006)
• Human as part of the SoS (Siel 2006, Meilich
  2006, USAF 2005)
• Discovery and application of convergence
  protocols (USAF 2005)

13
SoSE Compared to Traditional SE Activities
(continued)

• Key Areas Where SoSE Activities Differ From
  Traditional Systems Engineering (continued)
• Organizational scope defined at runtime instead
  of at system development time (Meilich 2006)
• Dynamic reconfiguration of architecture as needs
  change (USAF 2005)
• Modeling and simulation, in particular to better
  understand emergent behaviors (Finley 2006)
• Component systems separately acquired and
  continue to be managed as independent systems
  (USAF 2005)
• Intense concept phase analysis followed by
  continuous anticipation aided by ongoing
  experimentation (USAF 2005)

14
SoSE Compared to Traditional SE Activities
(continued)

• Key Challenges for SoSE
• Business model and incentives to encourage
  working together at the SoS level (Garber 2006)
• Doing the necessary tradeoffs at the SoS level
  (Garber 2006)
• Human-system integration (Siel 2006, Meilich
  2006)
• Commonality of data, architecture, and business
  strategies at the SoS level (Pair 2006)
• Removing multiple decision making layers (Pair
  2006)
• Requiring accountability at the enterprise level
  (Pair 2006)
• Evolution management (Meilich 2006)
• Maturity of technology (Finley 2006)

For the most part, SoSE appears to be SE
15
Sample Dynamic SoSMetropolitan Area Crisis
Management System
Net-Centric Connectivity
Net
-
Centric SoS
16
Sample Steady-State SoS Enterprise Wide
Integration of Core Business Applications
17
System of Systems Cost Estimation
Activity Levels Cost Model
SoS Lead System Integrator Effort (SoS scoping, planning, requirements, architecting source selection teambuilding, re-architecting, feasibility assurance with selected suppliers incremental acquisition management SoS integration and test transition planning, preparation, and execution and continuous change, risk, and opportunity management) Level 0, and other levels if lower level systems components are also SoSs (e.g., S2) COSOSIMO
Development of SoS Software-Intensive Infrastructure and Integration Tools Level 0 COCOMO II
System Engineering for SoS Components Levels 1-n COSYSMO
Software Development for Software-Intensive Components Levels 1-n COCOMO II
COTS Assessment and Integration for COTS-based Components Levels 1-n COCOTS
18
System of Systems Cost Model
COSOSIMO
Size Drivers
SoS Definition and Integration Effort
Cost Drivers
Calibration

• Characteristics of SoSs supported by cost model
• Strategically-oriented stakeholders interested in
  tradeoffs and costs
• Long-range architectural vision for SoS
• Developed and integrated by an LSI
• System component independence
• Size drivers and cost drivers
• Based on product characteristics, processes that
  impact LSI effort, and LSI personnel experience
  and capabilities

19
Proposed Size Drivers

• Number of SoS-related requirements
• Number of of distinct interface protocols to be
  provided by the SoS framework
• Number of independent system component
  organizations that are providing system
  components that will operate within the SoS
  framework
• Number of SoS user scenarios
• Number of unique component systems

S2
S1
S4
Each weighted by complexity
S3
20
Conceptual LSI Effort Profile

• LSI activities focus on three somewhat
  independent activities, performed by relatively
  independent teams
• A given LSI may be responsible for one, two, or
  all activity areas
• Some SoS programs may have more than one
  organization performing LSI activities

21
COSOSIMO Reduced Parameter Sub-Model Overview
Planning, Requirements Management, and
Architecting (PRA)
Size Drivers
SoS Definition and Integration Effort
Source Selection and Supplier Oversight (SO)
Cost Drivers
SoS Integration and Testing (IT)
22
COSOSIMO PRA Sub-Model

• Size Drivers
• SoS-related requirements
• SoS interface protocols

Planning, Requirements Management, and
Architecting
LSI PRA Effort

• Cost Drivers
• Requirements understanding
• Level of service requirements
• Stakeholder team cohesion
• SoS team capability
• Maturity of LSI processes
• Tool support
• Cost/schedule compatibility
• SoS risk resolution

23
COSOSIMO PRA Effort Estimation
m
n SoS PRAPM APRA? CREQi ?
CIPjBPRA
i1 j1
Where PRAPM LSI Planning, Requirements
Management, and Analysis effort in
person-months APRA Constant derived from PRA
historical data CREQi Complexity factor
associated with the ith SoS requirement CIPj
Complexity factor associated with the jth SoS
interface protocol m Number of SoS-related
sea-level requirements n Number of interface
protocols supported by the SoS architecture BPRA
Effort exponent based on the PRA exponential
scale factors. The geometric product of the scale
factors results in an overall exponential effort
adjustment factor to the nominal PRA effort
24
COSOSIMO SO Sub-Model

• Size Drivers
• independent component system organizations

Source Selection and Supplier Oversight
LSI SO Effort

• Cost Drivers
• Requirements understanding
• Architecture maturity
• Level of service requirements
• SoS team capability
• Maturity of LSI processes
• Tool support
• Cost/schedule compatibility
• SoS risk resolution

25
COSOSIMO SO Effort Estimation
n SoS SOPM
ASO? CSCOjBSO
j1
Where SOPM LSI Source Selection and Supplier
Oversight effort in person-months ASO Constant
derived from SO historical data CSCOj Complexity
factor associated with the jth SoS component
system organization n Number of organizations
providing independently developed and maintained
system components for the SoS BSO Effort
exponent based on the SoS SO exponential scale
factors. The geometric product of the scale
factors results in an overall exponential effort
adjustment factor to the nominal SO effort
26
COSOSIMO IT Sub-Model

• Size Drivers
• SoS interface protocols
• SoS scenarios
• unique component systems

SoS Integration and Testing
LSI IT Effort

• Cost Drivers
• Requirements understanding
• Architecture maturity
• Level of service requirements
• SoS team capability
• Maturity of LSI processes
• Tool support
• Cost/schedule compatibility
• SoS risk resolution
• Component system maturity and stability
• Component system readiness

27
COSOSIMO IT Effort Estimation
q
r s SoS
ITPM AIT? CIPi ? CSCENj ? CSCOkBIT

i1 j1
k1
Where ITPM LSI Integration and Test effort in
person-months AIT Constant derived from IT
historical data CIPi Complexity factor
associated with the ith SoS interface
protocol CSCENj Complexity factor associated
with the jth SoS interface protocol CSCOk
Complexity factor associated with the kth SoS
component system organization q Number of
interface protocols supported by the SoS
architecture r Number of SoS scenarios s Number
of organizations providing independently
developed and maintained system components for
the SoS BIT Effort exponent based on the IT
exponential scale factors. The geometric product
of the scale factors results in an overall
exponential effort adjustment factor to the
nominal IT effort
28
COSOSIMO Total SoSE Effort Estimation
SoSEPM PRAPM SOPM ITPM
Where PRAPM LSI Planning, Requirements
Management, and Analysis effort in
person-months SOPM LSI Source Selection and
Supplier Oversight effort in person-months ITPM
LSI Integration and Test effort in person-months
29
SoS Schedule Estimation
30
Conclusions

• Traditional systems engineering takes too long
  and too much effort
• LSIs are finding better ways to engineering SoSs
  (SoSE)
• Many combine agile with traditional approaches
• Increases concurrency
• Reduces risk
• Compresses schedules
• Reduced-parameter set COSOSIMO captures effects
  of new processes in three key areas
• Planning, requirements management, and
  architecting
• Source selection and supplier oversight
• SoS integration and testing
• Sub-models have fewer parameters that are more
  tailored to associated SoSE activities
• Allows LSIs to estimate areas of interest and
  conduct what ifs comparisons of different
  development strategies

31
Conclusions (continued)

• With the addition of a new COSOSIMO cost model to
  existing cost model tools, it will be possible to
  get more complete estimates of the SoS
  development effort
• Key to this process is
• Having an SoS architecture sufficiently defined
  so that component system modifications to support
  operation in the SoS environment can be made with
  few dependencies on other SoS development efforts
  
• Structuring the WBS so that
• SoS and component system tasks can be decomposed
  into parts that can be estimated using the
  existing cost model tools
• Parts not covered by cost models can be clearly
  identified and estimated using non-parametric
  methods
• Expected COSOSIMO availability Fall 2007

All models are wrong, but some of them are
useful (W. E. Deming)
32
What is Needed to Support Fall 2007 Availability

• Participation in current SoSE surveys
• Data from both SoS and SE programs
• Process descriptions to help understand the
  differences between SoSE and SE
• Effort data to calibrate COSOSIMO (either
  standalone model or special calibration of
  COSYSMO)

For those organizations that provide SoSE effort
from at least 3 SoS projects, a local calibration
will be provided
33
COSOSIMO-Related References

• Boehm, B., et al. (2000) Software Cost
  Estimation with COCOMO II Prentice Hall
• Boehm,B., Valerdi, R., Lane, J., and Brown, W.
  (2005) COCOMO Suite Methodology and Evolution
  CrossTalk, Vol. 18, No. 5 (pp. 20-25)
• Boehm, B., and J. Lane (2006) 21st Century
  Processes for Acquiring 21st Century Systems of
  Systems CrossTalk Vol. 19, No. 5 (pp. 4-9)
• Lane, J. (2005) System of Systems Lead System
  Integrators Where do They Spend Their Time and
  What Makes them More/Less Efficient
  USC-CSE-TR-2005-508
• Lane, J. (2005) Factors Influencing
  System-of-Systems Architecting and Integration
  Costs Conference on Systems Engineering Research
  
• Lane, J (2006) COSOSIMO Parameter Definitions,
  USC-CSE-TR-2006-606
• Lane, J and Boehm, B. (2006) Synthesis of
  Existing Cost Models to Meet System of Systems
  Needs Conference on Systems Engineering Research
• Lane, J and Boehm, B. (2006) System-of-Systems
  Cost Estimation Analysis of Lead System
  Integrator Engineering Activities InterSymposium
  Symposium on Information Systems Research and
  Systems Approach
• Lane, J and Valerdi, R (2005) Synthesizing SoS
  Concepts for Use in Cost Estimation IEEE
  Systems, Man, and Cybernetics

34
SoSE-Related References

• Carlock, P.G., and R.E. Fenton, "System of
  Systems (SoS) Enterprise Systems for
  Information-Intensive Organizations," Systems
  Engineering, Vol. 4, No. 4, pp. 242-261, 2001
• DiMario, Mike (2006) System of Systems
  Characteristics and Interoperability in Joint
  Command Control, Proceedings of the 2nd Annual
  System of Systems Engineering Conference
• Electronic Industries Alliance (1999) EIA
  Standard 632 Processes for Engineering a System
• Finley, James (2006) Keynote Address,
  Proceedings of the 2nd Annual System of Systems
  Engineering Conference
• Garber, Vitalij (2006) Keynote Presentation,
  Proceedings of the 2nd Annual System of Systems
  Engineering Conference
• INCOSE (2006) Systems Engineering Handbook,
  Version 3, INCOSE-TP-2003-002-03
• Krygiel, A. (1999) Behind the Wizards Curtain
  CCRP Publication Series, July, 1999, p. 33
• Maier, M. (1998) Architecting Principles for
  Systems-of-Systems Systems Engineering, Vol. 1,
  No. 4 (pp 267-284)
• Meilich, Abe (2006) System of Systems
  Engineering (SoSE) and Architecture Challenges in
  a Net Centric Environment, Proceedings of the
  2nd Annual System of Systems Engineering
  Conference
• Pair, Major General Carlos (2006) Keynote
  Presentation, Proceedings of the 2nd Annual
  System of Systems Engineering Conference
• Proceedings of AFOSR SoSE Workshop, Sponsored by
  Purdue University, 17-18 May 2006
• Proceedings of Society for Design and Process
  Science 9th World Conference on Integrated Design
  and Process Technology, San Diego, CA, 25-30 June
  2006
• Siel, Carl (2006) Keynote Presentation,
  Proceedings of the 2nd Annual System of Systems
  Engineering Conference
• United States Air Force Scientific Advisory Board
  (2005) Report on System-of-Systems Engineering
  for Air Force Capability Development Public
  Release SAB-TR-05-04