NDIA Systems Engineering Division Modeling and Simulation Committee Study Task Report M

1
NDIA Systems Engineering DivisionModeling and
Simulation CommitteeStudy Task ReportMS
Support to theNew DoD Acquisition Process

• James W. Hollenbach
• Simulation Strategies, Inc.
• MS Committee Co-Chair

2
Aug 03 Request fm PDUSD(ATL)DS
provide advice to USD(ATL) Defense Systems and
its partners on how modeling and simulation could
effectively be applied to develop and analyze
operational and systems architectures and to
implement systems engineering of integrated
systems.
3
MS Cmte Response

• Two-day workshop in Oct 03
• 50 people participated in workshop and
  subsequent reviews
• 28-page report developed from workshop
  conclusions, follow on research and several
  committee reviews over next four months
• 13 findings regarding systems engineering process
• 35 findings regarding MS
• Four recommendations regarding MS use
• 12 recommendations to better enable MS use
• Four appendices, with Appendix C identifying
  potential MS uses throughout the SoS
  development/evolution cycle
• A majority of findings and recommendations
  concern the DoD corporate level, where the most
  profound changes in roles and responsibilities
  occur

4
Systems Engineering Findings (1 of 4)

• Integrated architectures are a logical facet of
  good systems engineering
• The types of analyses which can be supported
  expand as an architecture becomes more detailed

5
Systems Engineering Findings (2 of 4)

• Integrated architecture development is SoS
  engineering that frames the systems engineering
  of individual systems
• Integrated architectures can serve as agreements
  between the FCBs/DoD leadership and individual
  PMs
• Receiving a reasonably complete architecture
  would decrease costs and risks for individual
  programs

Figure 2. Systems Engineering Process with SoS
Extensions Note The V depiction of SE is not
meant to imply a waterfall, step-by-step
approachall SE activities are iterative,
recursive spirals with ample feedback
opportunities.
6
Systems Engineering Findings (3 of 4)

• DoD has proposed to incrementally implement the
  new acquisition approach, learning as it goes
• DoD is currently embarked on a series of first
  order analyses
• DoD has proposed to foster coherence among the
  various FCB-managed architectures by developing
  an integrated framework and a core systems
  dataset
• DoD tentatively plans to delve more deeply into
  the SoS engineering task by conducting 2nd and
  3rd order analyses
• All integrated architectures will change over time

7
Systems Engineering Findings (4 of 4)

• There are many challenges to extending the
  architectures to the degree of detail needed for
  thorough SoS engineering
• Info required, people involved, complexity,
  management challenges
• Personnel and funding demands of SoS engineering
  will grow significantly
• Some possible sources of required human resources
  are
• Govt or military personnel added to current
  organizations or a new organization/defense
  agency
• FFRDC or UARC
• Defense OEM (prime contractor)
• First-tier SE contractor
• Govt and industry staffs of individual
  acquisition programs on an additional duty basis
• Some mix of above (e.g., a national team)

8
Modeling and Simulation Findings (1 of 8)

• General
• MS can facilitate the development and analysis
  of operational and systems architectures and the
  effective systems engineering of integrated
  systems of systems
• Opportunities for benefiting from MS exist
  throughout an SoS lifecycle
• Appendix C lists
• A large variety of models and simulations will
  continue to be used in the development of
  individual systems
• A means to inter-relate the various
  representations is necessary for effective
  collaboration

9
MS Findings (2 of 8)

• Architecture definition
• Architecture views in DoDAF are simple, static
  models
• DoDAF provides no guarantee its views are
  coherent
• Other architecture specification frameworks may
  offer viable alternatives to the DoDAF for
  realizing DoD goals
• e.g., OMG Model Driven Architecture (MDA), ISO
  Reference Model for Open Distributed Processing
  (RM-ODP)
• Architecture definition tools (modeling
  environments) can potentially help address DoDs
  architecting challenge
• e.g., systems engineering tools like Slate, Core
• DoDAF does not specify a standard notation for
  its views, complicating dispersed, model-centric
  collaboration
• All of todays architecture definition tools have
  shortfalls

10
MS Findings (3 of 8)

• Operational Concept Assessment
• Simulations that immerse warfighters into the
  operational environment can help assess validity
  of operational concepts
• A wide variety of existing simulations at the
  mission and campaign levels can be adapted to
  provide such an operational concept assessment
  capability
• Functional Capability Assessment
• Functional capability simulations are necessary
  to examine FC performance attributes and can also
  help assess impact/risk of the lack of a
  functional capability
• The range of simulations adaptable to provide
  this capability is narrower but still substantial

11
MS Findings (4 of 8)

• Integration and Testing
• Distributed simulation offers a practical,
  cost-effective way to integrate and test systems
  of systems
• Intermix real systems on instrumented ranges, lab
  hardware and software assets, simulations, etc.
• JDEP is promising example, but is under-resourced
• Investment optimization
• Credible cost models would be a great boon for
  making cost-performance trade-offs
• Poor understanding of cost-estimating
  relationships for latest technology, net-centric
  SoS makes it impractical to expect much fidelity
  from these models

12
MS Findings (5 of 8)

• Tool Standards
• Open standards can allow MS tools to be flexibly
  used together
• Application-level standards (OSI Model Level 7)
  are the critical area of interest for MS
• Several helpful application-level standards are
  in place
• IEEE 1516 HLA, XML, XMI, XSLT, UML, ISO STEP,
  DDMS
• Several categories of needed application-level
  standards are still absent
• e.g., logical architecture model notations,
  systems engineering data interchanges standards,
  lineage metadata interchange standards, etc.
• Several ongoing, emergent or potential standard
  development efforts could help address these
  shortfalls
• e.g., SysML, STEP AP 233 and 239, GEIA-927,
  extensions to DDMS
• Tool vendors have not solved, and seem unlikely
  to solve, the information interchange problem

13
MS Findings (6 of 8)

• Standard Tools
• At present, collaborative architecture
  development is difficult unless all the players
  use the same tool
• Requiring a common architecture definition tool
  is impractical in the long term
• There are insufficient reasons for standardizing
  on a comprehensive MS tool suite across the
  systems acquisition arena
• A desire to understand what the other guy is
  seeing will drive a desire for a limited core
  suite of MS tools

14
MS Findings (7 of 8)

• MS Coherency
• The various MS-based analyses of each FCB need
  to be coherent to maximum practical extent
• Information Sharing
• Information sharing is critical to MS
  effectiveness
• Protection of proprietary information is a major
  concern
• Services, PMs, and contractors are very reluctant
  to share information
• Costs to understand a request, fill it, and
  answer questions
• Info may be misunderstood, misused, used against
  them

15
MS Findings (8 of 8)

• Contracting policy
• Reluctance of commit to GFE/GFI is at odds with
  new acquisition strategys need to
  cost-effectively understand a systems SoS
  environment
• Resource Contention
• There will be resource contention across FCBs and
  among the various Service staffs, program offices
  and prime contractors
• Organizational Support
• No DoD organization is focused on providing MS
  support to the FCBs and WCAID

16
MS Recommendations

• Use data-driven architecture definition tools to
  facilitate collaborative development of
  integrated SoS architectures
• Involve government and industry staffs of
  individual programs
• Standardization on a single tool may be necessary
  in short term
• Examine the Model Driven Architecture and
  Reference Model for Open Distributed Processing
  for potential leverage and appropriate role
  vis-à-vis the DoDAF.
• Initiate a verification use of simulation to
  support the assessment of operational concepts or
  functional capability
• Increase support for the maturation of JDEP

17
MS Recommendations
Immersive simulations
JDEP
Architecture definition tools
Remove MS impediments
Remove MS impediments
18
MS Enabling Recommendations (1 of 4)

• For whatever architectural framework adopted,
  develop a composite logical schema of the
  information required
• Document the above logical information schema in
  multiple open formats
• As of a specified future date, require that a
  standardized XML file structure be used to
  describe any DoD integrated architecture
• Develop/provide a utility tool for the DoD
  community that will take as input two or more
  standardized XML architecture description files
  and analyze their structures to identify possible
  points of overlap and interface

19
MS Enabling Recommendations (2 of 4)

• Endorse and support the technical evolution and
  tool support of SysML
• Participate with industry in funding, developing
  and implementing systems engineering-related data
  interchange standards and an associated framework
• Establish a Systems Engineering Community of
  Interest to address system development
  information metadata extensions to the DDMS

20
MS Enabling Recommendations (3 of 4)

• Establish a DoD-wide policy on the sharing of
  system information
• Define what fundamental system info each program
  will be obliged to share with others having a
  valid need-to-know
• Establish a directory service to identify
  authoritative sources
• Define responsibilities and liabilities of both
  parties for reuse
• Strategy for protection of proprietary
  information, possibly via Application Service
  Provider approach
• Business model (who pays)
• Standard method to request info and adjudicate
  denials
• Establish a similar policy on the sharing of MS
  tools

21
MS Enabling Recommendations (4 of 4)

• Conduct basic research to capture and refine the
  cost-estimating relationships of net-centric
  systems of systems
• Provide PMs and industry with means of entry to
  the adjudication process for access to scarce,
  joint-use MS resources
• Reprioritize DMSO and JDEP Technical Support Team
  efforts towards the satisfaction of FCB and WCAID
  MS needs

22
Conclusion

• The committee has been pleased to contribute to
  DoD deliberations on implementation of its new
  acquisition process
• We hope this report will serve to better leverage
  the potential of MS to support that process
• The several examine and establish policy
  recommendations are fertile ground for additional
  dialogue

Report available at http//www.ndia.org/divisions/
modeling