MILSTD498

1
MIL-STD-498 IEEE/EIA J-STD-016

• CFICSE 2002 ST04
• Michelle L. Crane

2
References

• James W. Moore, Software Engineering Standards A
  Users Road Map, IEEE Computer Society, 1998.
• www.software.org/quagmire
• Paul A. Szulewski and David S. Maiblor.
  MIL-STD-498 Whats New and Some Real Lessons
  Learned. CrossTalk. March 1996.
• Maj. George A. Newberry, USAF. Changes from
  DOD-STD-2167A to MIL-STD-498. CrossTalk. April
  1995.

3
Outline

• MIL-STD-498 vitae
• J-STD-016 vitae
• why create 498 and its benefits
• organization
• differences between DOD-STD-2167A and MIL-STD-498
• miscellaneous

4
Standards We Will Love
DoD-Std 2167A
SW CMM
EIA/IEEE J-Std-016
Mil-Std 498
IEEE/EIA Std 12207
ISO 9000 series
ISO/IEC 12207
Reference http//www.software.org/quagmire/
5
MIL-STD-498

• Framework Name
• Software Development and Documentation
• Published 5 Dec 94
• approved for an interim period of two years
• interim to allow a commercial equivalent to be
  developed
• Cancelled Jun 98

Reference http//www.software.org/quagmire/
6
Purpose

• to establish uniform requirements for software
  development and documentation

Reference http//www.software.org/quagmire/
7
Description

• specifies activities and products that may be
  required
• products are described in detail in 22 Data Item
  Descriptions (DIDs)
• intended to be applicable to any type of software
  project and independent of any particular
  methodology

Reference http//www.software.org/quagmire/
8
Relation to Other Frameworks

• superseded DOD-STD-2167A, DOD-STD-7935A, and
  DOD-STD-1703
• cancelled when IEEE/EIA 12207 was released

Reference http//www.software.org/quagmire/
9
EIA/IEEE J-STD-016

• Framework Name
• Standard for Information Technology, Software
  Life Cycle Processes, Software Development,
  Acquirer-Supplier Agreement
• Published 30 Sep 95, as an Interim Standard

Reference http//www.software.org/quagmire/
10
Description

• joint standard for the software development
  process
• establishes requirements for acquiring,
  developing, modifying, and documenting software
• establishes activities, tasks, and products for a
  software development or maintenance project
• intended to be applicable to any type of software
  project and independent of any particular
  methodology

Reference http//www.software.org/quagmire/
11
Relation to Other Frameworks

• commercialized version of MIL-STD-498
• differences are minor
• therefore, well just focus on 498

Reference http//www.software.org/quagmire/
12
Why MIL-STD-498?

• 1) Improve compatibility with Incremental and
  Evolutionary development models
• 2) Decrease dependence on formal reviews and
  audits
• 3) Decrease emphasis on preparing documentation
• 4) Improve compatibility with computer-aided
  software engineering (CASE) tools

13
Why MIL-STD-498? (contd)

• 5) Improve links to systems engineering
• 6) Include the use of software management
  indicators
• 7) Improve coverage of modification, reuse, and
  reengineering
• 8) Increase emphasis on software supportability

14
Why MIL-STD-498? (contd)

• 9) Provide a clearer distinction between
  requirements and design
• 10) Improve compatibility with non-hierarchical
  design methods
• 11) Improve coverage of database development
• 12) Improve the criteria used for software
  product evaluation

15
Why MIL-STD-498? (contd)

• 13) Eliminate confusion between software quality
  assurance and software product evaluation
• 14) Extend configuration management concepts to
  in-process work products
• 15) Clarify relationships to other standards
• 16) Provide a means to order software via the
  Contract Data Requirements List (CDRL)
• 17) Eliminate inconsistencies and holes in the
  DIDs

16
Claimed Benefits of 498

• Is usable with any development strategy
• Is usable with any development methods
• Is compatible with CASE tools
• Provides requirements for software reuse
• Supports the use of software measurement
• Emphasizes software supportability
• Provides a role for software in the system

17
Organization of MIL-STD-498

• Section 1 Scope (including purpose and
  applicability)
• Section 2 Referenced Documents
• Section 3 Definitions
• Section 4 General Requirements
• Section 5 Detailed Requirements
• Appendixes
• Appendix A Acronyms
• Appendix B Interpreting MIL-STD-498 for
  Incorporation of Reusable Software Products
• Appendix C Category and Priority Classifications
  for Problem Reporting
• Appendix D Software Product Evaluations
• Appendix E Candidate Joint Management Reviews
• Appendix F Candidate Management Indicators
• Appendix G Guidance on Program Strategies,
  Tailoring, and Build Planning
• Appendix H Guidance on Ordering Deliverables
• Appendix I Conversion Guide from DOD-STD-2167A
  and DOD-STD-7935A
• Index to the standard and DIDs

18
Changes From 2167A
19
Removing the Waterfall Bias

• 2167A never dictated the use of a specific
  software development technique
• but perceived by many to impose the Waterfall
  model
• could only be used in a restrictive manner
• developers felt they could only
• perform each step of the software development
  process once
• perform the steps in sequence
• finish each step before beginning the next
• to add more flexibility to the development
  process, iterative techniques began to be used
  that involved prototyping and incremental
  development

Reference Maj. Newberry. Changes from
DOD-STD-2167A to MIL-STD-498.
20
Removing the Waterfall Bias (contd)

• 498 describes software development in one or more
  incremental builds
• each build implements a specified subset of the
  planned capabilities
• process steps are repeated for each build
• within each build, steps may be overlapping and
  iterative
• CAUTION
• no default process to follow
• skill level required to use it is higher

Reference Maj. Newberry. Changes from
DOD-STD-2167A to MIL-STD-498.
21
Alternatives to Formal Reviews Audits

• 2167A imposes formal reviews and audits that
  emphasize the waterfall model and are often
  non-productive dog and pony shows
• developer spends thousands of hours preparing
  special materials for these meetings, and the
  acquirer is overloaded

Reference Maj. Newberry. Changes from
DOD-STD-2167A to MIL-STD-498.
22
Alternatives to Formal Reviews (contd)

• 498 relaxes the formality
• simply requires joint technical and management
  reviews
• reviews should be frequent and informal and focus
  on the projects status, approach, and risks
• biggest change is emphasis on using existing work
  products instead of special materials
• objective is to move the communication from
  formal presentations to a continuous exchange of
  information

Reference Maj. Newberry. Changes from
DOD-STD-2167A to MIL-STD-498.
23
Compatibility with Nonhierarchical Methods

• 2167A shows software development as a top-down
  functional decomposition
• computer software configuration items (CSCIs) are
  decomposed into computer software components
  (CSCs), which are decomposed into other CSCs,
  etcdown to computer software units (CSUs)
• design, testing, configuration management, etc.
  are based on this decomposition
• caused some trouble for developers, especially
  when performing object-oriented analysis and
  design

Reference Maj. Newberry. Changes from
DOD-STD-2167A to MIL-STD-498.
24
Nonhierarchical Methods (contd)

• 498 removes the limitation
• CSCIs are decomposed into software units, which
  may or may not be related to each other in a
  hierarchical manner
• design, testing, CM, etc. based on
  developer-designated software units
• more flexibility to use methods best suited to
  the specific project

Reference Maj. Newberry. Changes from
DOD-STD-2167A to MIL-STD-498.
25
Less Emphasis on Documentation

• 2167A written in terms of producing documents
• implication was that the developer needed to
  prepare and deliver a series of documents
  relating to every piece of the project
• standard has been interpreted to discourage the
  use of computer aided software engineering (CASE)
  tools by referencing only traditional documents
• data item descriptions (DIDs) also reinforced
  this interpretation

Reference Maj. Newberry. Changes from
DOD-STD-2167A to MIL-STD-498.
26
Less Documentation (contd)

• 498 written in terms of defining and recording
  information
• info may or may not be in the form of a
  traditional document
• may or may not be deliverable
• wording in standard and DIDs encourages the use
  of CASE tools
• specifies required information, not the form
• objective is to reduce the cost by simplifying
  how information is recorded
• eliminate unnecessary documentation

Reference Maj. Newberry. Changes from
DOD-STD-2167A to MIL-STD-498.
27
Improved Links to Software Engineering

• 2167A assumes software is embedded in a
  hardware-software system
• assumes that someone else is performing
  system-level activities
• does not acknowledge software engineers
  participation in systems engineering

Reference Maj. Newberry. Changes from
DOD-STD-2167A to MIL-STD-498.
28
Links to Software Engineering (contd)

• 498 acknowledges both software-only systems and
  systems that contain software as one element
  (i.e., embedded systems)
• contains system-level requirements for
  software-only systems
• requires participation of software engineer in
  system-level activities for embedded systems

Reference Maj. Newberry. Changes from
DOD-STD-2167A to MIL-STD-498.
29
Use of Software Management Indicators

• 2167A does not require use of software management
  indicators and offers no guidance on the subject
• 498 requires the developer to define and apply
  software management indicators
• provides a set of candidate indicators to serve
  as a starting point

Reference Maj. Newberry. Changes from
DOD-STD-2167A to MIL-STD-498.
30
Improved Coverage of Databases

• 2167A was developed for use on weapons systems
• difficult to use on automated information systems
  because it largely ignores databases
• 498 written for use on development of all systems
• defines software as computer programs and
  computer databases
• adds a database design description DID
• uses the term implementation vice coding to
  include data
• new standard covers databases in all stages
  requirements, design, implementation

Reference Maj. Newberry. Changes from
DOD-STD-2167A to MIL-STD-498.
31
Better Coverage of Modification, Reuse, and
Reengineering

• 2167A written in terms of new development
• interpretation and intense tailoring to apply the
  standard to modification, reuse, and
  reengineering
• requires the developer to consider incorporating
  non-developmental software, it leaves unclear
  what criteria to use in the consideration or even
  how the standard is to be applied when
  incorporating reusable software

Reference Maj. Newberry. Changes from
DOD-STD-2167A to MIL-STD-498.
32
Modification, Reuse, and Reengineering (contd)

• 498 explicitly acknowledges that each step may
  involve modifying, reusing, or reengineering
  existing items vice new development
• expands the reuse requirement to cover all
  software products (e.g., reusable architectures)
  not just the software
• mandatory and non-mandatory criteria for
  evaluating items for reuse
• model that shows how to apply the standard to a
  reengineering project

Reference Maj. Newberry. Changes from
DOD-STD-2167A to MIL-STD-498.
33
Increased Emphasis on Supportability

• 2167A strong on supportability but some loopholes
• written as if testing is the final activity
• fails to clarify the tasks of preparing the
  completed software for delivery and the support
  agency
• doesnt distinguish clearly between preparing for
  software use and preparing for software transition

Reference Maj. Newberry. Changes from
DOD-STD-2167A to MIL-STD-498.
34
Emphasis on Supportability (contd)

• 498 requires identification of all resources used
  or generated during development that will be
  needed by the support agency
• covers hardware, software, data, documentation
  necessary for support and requires a
  demonstration be conducted showing that the
  delivered software can be supported given
  identified resources
• requires the rationale for key decisions be
  recorded
• includes separate activities to prepare for
  software use and transition and distinguishes
  tasks for each

Reference Maj. Newberry. Changes from
DOD-STD-2167A to MIL-STD-498.
35
Improved Evaluation Review Criteria

• 2167A defines criteria for software product
  evaluation but applies these only to deliverables
• relies on MIL-STD-1521B for formal review criteria

Reference Maj. Newberry. Changes from
DOD-STD-2167A to MIL-STD-498.
36
Evaluation Review Criteria (contd)

• 498 strengthens the criteria for software product
  evaluations by making them applicable to
  in-process work products
• uses same criteria for evaluations and joint
  technical reviews, thus integrating both
  activities

Reference Maj. Newberry. Changes from
DOD-STD-2167A to MIL-STD-498.
37
Clearer Distinction Between Requirements and
Design

• 2167A defined
• requirements as what the system or software must
  do
• design as how it is implemented
• traditional distinction led to argument,
  confusion, numerous program delays
• 498 seeks to eliminate confusion by modifying
  definitions
• requirements are defined as what the acquirer
  cares enough about to make conditions for
  acceptance (what or how)
• design is defined as the set of decisions made by
  the developer in response to the stated
  requirements (what or how)

Reference Maj. Newberry. Changes from
DOD-STD-2167A to MIL-STD-498.
38
Inclusion of Software Quality Assurance

• 2167A requires the developer to perform software
  product evaluations
• relies on DOD-STD-2168 for software quality
  assurance (SQA)
• 2168 info was in 2167 but separated during
  development of 2167A
• done so SQA info could be incorporated with
  another standard (but never occurred!)
• separate quality standard interpreted by
  developers to mean separate quality organization
• but the governments intent was to simply have a
  group of individuals who were not part of the
  development team evaluate the development effort

Reference Maj. Newberry. Changes from
DOD-STD-2167A to MIL-STD-498.
39
Software Quality Assurance (contd)

• 498 returns to a self-contained document (wrt
  quality)
• still requires the developer to perform software
  product evaluations and incorporates key points
  from 2168 regarding SQA
• eliminates inference regarding separate SQA
  organization
• clarifies scope of SQA so that overlap with
  software product evaluations is removed

Reference Maj. Newberry. Changes from
DOD-STD-2167A to MIL-STD-498.
40
Clarification of CM Requirements

• 2167A uses the concept of developmental
  configuration
• does not acknowledge that computer files are
  often the entities placed under CM, rather than
  CSUs, which may be conceptual rather than
  physical
• limits configuration control to deliverables and
  to just before delivery

Reference Maj. Newberry. Changes from
DOD-STD-2167A to MIL-STD-498.
41
CM Requirements (contd)

• 498 eliminates concept of developmental
  configuration
• requires identification of entities at the level
  at which they will actually be controlled (e.g.,
  computer files)
• does not dictate how the developer organizes
  their CM function
• objective is to allow the developer to use their
  own CM organization and not require them to
  provide a separate one just for DoD projects

Reference Maj. Newberry. Changes from
DOD-STD-2167A to MIL-STD-498.
42
CM Requirements (contd)
43
Applicability to More Types of Projects

• 2167A written in terms of government vs.
  contractor
• confusion as how to apply to government in-house
  development or us in contract to subcontractor

Reference Maj. Newberry. Changes from
DOD-STD-2167A to MIL-STD-498.
44
More Types of Projects (contd)

• 498 clarifies by using acquirer and developer
• in-house development efforts should find this
  easier as it defines contractual terms in ways
  usable in the absence of a contract
• clarifies the applicability in prime contractor
  to subcontractor relationships by generalizing
  usability of the standard

Reference Maj. Newberry. Changes from
DOD-STD-2167A to MIL-STD-498.
45
Improved Treatment of the Software

• 2167A assumes all software to be ordered using a
  contract line item number (CLIN), which is not
  the case
• offers no means to order the executable software,
  source code, or data files via a contract data
  requirements list (CDRL)
• incorrectly mimics hardware development by
  treating the final design as the end product of
  software development

Reference Maj. Newberry. Changes from
DOD-STD-2167A to MIL-STD-498.
46
Improved Treatment of Software (contd)

• 498 established a software product specification
  DID as a means to order the executable software
  and source code and data files via a CDRL
• treats the software, not the final design, as the
  final product of software development

Reference Maj. Newberry. Changes from
DOD-STD-2167A to MIL-STD-498.
47
Miscellaneous
48
Players and Agreements
User
Acquirer
Support Agency
Statement of Work (SOW)
Contract (Agreement)
Developer
Contract (Agreement)
CLIN or CDRL
Subcontractors
Subcontractors
49
The Importance of Plans
50
MIL-STD-498 Requirements (I)

• 4.1 Establish a software development process
  consistent with contract requirements
• 4.2 General requirements for software
  development
• Use systematic, documented methods
• Develop and apply standards for representing
  requirements, design, code, and test information
• Reuse software products as appropriate
• Evaluate reusable software products for use in
  fulfilling contract requirements incorporate
  those that meet the criteria in the Software
  Development Plan

51
MIL-STD-498 Requirements (II)

• 4.2 General requirements for software development
  (contd)
• Identify opportunities for developing software
  products for reuse notify the acquirer of those
  that have cost benefits and are consistent with
  program objectives
• Establish and apply strategies for handling
  critical requirements, such as those with safety,
  security, or privacy implications
• Analyze and fulfill the computer hardware
  resource utilization requirements (such as memory
  reserves)
• Record rationale for key decisions for use by the
  support agency
• Provide the acquirer access to developer and
  subcontractor facilities

52
Summary

• MIL-STD-498 vitae
• J-STD-016 vitae
• why create 498 and its benefits
• organization
• differences between DOD-STD-2167A and MIL-STD-498
• miscellaneous

53
?