CBS Process WG

1
CBS Process WG
2
Participants

• David Carney (chair)
• M. Al-Said (scribe)
• Tony Jordano
• Kyung Whan Lee
• Jeffrey Poulin
• David Klappholz
• Glenn Berg
• Jongmoon Baik

• Rami Razouk
• George Huling
• Tim Spinney
• Steve Cross
• Mike Moore
• Co-chairs
• Betsy Clark
• Dan Port
• Tricia Oberndorf

3
Working Approach

• Homework come in with 3 issues
• Brainstormed, collapsed, clustered
• Prioritized by importance (practitioner
  researcher) difficulty
• Formulated process-oriented statements
• Discussed prospective CeBASE Top 10 List

4
Issue 1 CBS Lifecycle Models and Processes - 1

• Is there a new lifecycle model for CBSs?
• Can one (or more) of the existing lifecycle
  models be tailored to CBSs?
• How do we define a CBS life-cycle process?

5
Issue 1 CBS Lifecycle Models and Processes - 2

• Both of these questions in light of
• A requirements-driven (COTS-supported) vs.
  COTS-driven paradigm
• Strategies for handling different kinds of market
  segments
• CBS time to market pressures
• Level of service quality needs
• .
• How do you identify - metrics group and factor
  in multiple CBS cost drivers?

6
Issue 2 Post-Deployment Process

• In what ways does the CBS post-deployment process
  differ from
• the CBS development process
• the custom post-deployment process?
• In what ways do the differences affect cost
  estimation for CBSs?

7
Issue 3 Release Planning

• What are the process implications of planning for
  system releases where COTS product
• releases are not synchronized with each other
• releases are not synchronized with your system
• end-of-life occurs during your system life
• Planning includes cost estimation, scheduling,
  determining system release content, etc.

8
Issue 4 Assessment, Evaluation Testing Process

• How and when do you assess, evaluate, and test
• COTS products
• COTS-based systems
• When includes timing (e.g., multiple points in
  the process) and system-independent certification
  of COTS products.
• How includes process, techniques, tools, roles,
  and metrics.
• Related to determination of CBS requirements.

9
Issue 5 COTS Market and Identification

• What are the processes for
• Acquiring and maintaining market knowledge
• Analyzing the forces in a market segment
• Forecasting trends in both market segments and
  specific products
• Obtaining disseminating COTS product
  information and product-specific experiences
• Establishing and maintaining vendor relationships
• Both the active (e.g., influencing) and passive
  aspects of these should be considered.

10
Issue 6 Surprise!

• How does a CBS process accommodate/address large
  discontinuities stemming from COTS product
  surprises? Surprises result from releases or
  patches
• that are not backward compatible
• that fail to perform
• whose fundamental properties change
• Exacerbated by inadequate documentation.
• - architecture issue concerning product
  characterization

11
Issue 7 Organizational Assessment Capability

• How does an organization assess its capability to
  be successful with CBSs?
• How does an organization improve its CBS
  capability? This includes such issues as
• relationship between CMMI and CBS processes
• metrics for CBS process maturity
• Applies to capabilities of CBS acquirers,
  developers, and maintainers.

12
Other Issues

• Dealing with extra features
• Managing customer expectations (post-selection)
• World-class SE organizations becoming COTS
  integrators
• Licensing
• Systematic way of deriving system challenges
• In-place transition of baselines for
  uninterruptible systems
• Metrics for degree of product system risk

13
COTS Process Issue Priorities 1a
14
COTS Process Issue Priorities 1b
15
COTS Process Issue Priorities 2a
16
COTS Process Issue Priorities 2b
17
COTS Process Issue Priorities 3a
18
COTS Process Issue Priorities 3b
19
Release, Surprise Votes
20
Post-deployment, AET Votes
21
Assessment, COTS Id. Votes
22
COTS-supp/driven Votes
23
Priorities (Diff. Imp.)
8
24
Priorities (Diff/Imp 1)
8
25
Prospective CeBASE Top 10

• General reactions
• Use of list as guidance
• Good (only?) as top 10 list of things CeBASE will
  research
• Use of unfamiliar and imprecise terminology
• Is it valid to have hypotheses on this list?

26
Specifics -1

• 1. Valid within a limited scope only.
• 2. What else is new? So does everything else!
• Add . Just like traditional
• Or Cost schedule overruns are as common with
  COTS as with custom development and they (CBSs)
  can cost as much.

27
Specifics -2

• 3. What is CBS type?
• Domain?
• COTS-driven vs COTS-supported?
• COTS product type?
• ..?
• 4. Should be studied, but we dont believe the
  hypothesis as posed
• How can a parametric model account for the
  situation in which the selected products cant be
  integrated?

28
Specifics -3

• 5. False, this is one of many parameters that
  influence the estimate.
• Prefer Architectural mismatch will affect CBS
  cost.
• 6. just as with traditional systems
  exaggerated with COTS products.
• Change to CBS post-deployment costs dominate

29
Specifics -4

• 7. Intuitively agree defects will be in
  mismatches between products, not something
  visible by inspection of line of code.
• A (ADDITION) Writing glue code requires greater
  skill than traditional coding. Often need to
  discover lots of things.

30
Specifics -5

• 8. Agree with frequently far worse than linear,
  question use of square heavily dependent on
  other factors (e.g. use of standards, quality of
  products, degree of integration)
• 9. Question twice.
• Replace withRisk specific to CBSs must be
  managed CBS risks (and their mitigations) are
  different risk doesnt disappear.
• Current statement is not a useful hypothesis.
• 10. Disagree with percentage agree vaporware is
  one of many factors.

31
Conclusions

• Process is foundational for CBSs.
• Risk management is still key - and will be
  different for CBSs.
• It was cold, but we had fun anyway!