Selecting%20COTS%20Products%20Using%20a%20Requirements-Based%20Approach

1
Selecting COTS Products Using a
Requirements-Based Approach

• Jaelson Castro, Carina Alves
• Centro de Informática
• Universidade Federal de Pernambuco

2
Motivations

• Development of large and complex systems
• Reusable components or COTS

The potential benefits of using COTS are
increased product reliability and stability, at
shorter development time and reduced cost.
3
What is COTS?

• Commercial-Off-The-Shelf
• There is no agreed definition
• COTS are products
• that are sold, leased or licensed to the
  general public
• that is usually available without source code
• that is supported and evolved by the vendor
  who returns the intellectual property rights
• Software Engineering Institute

4
COTS-Based Development (CBD)

• Large incentives from industry and academia
• Improved productivity and quality of software
  development
• Emphasizes the acquisition and integration of
  COTS components over in-house development

5
Main Characteristics of CBD

• The nature of COTS suggest new life cycle models
• The success of COTS-based systems largely depends
  on the successful selection of software products

6
COTS-based systems life cycle
7
The Evaluation Process

• Consists of determining the quality of a product
  in the context of its intended use
• Decision making
• Must accommodate uncertainty
• Evaluation activities can span the entire
  lifetime of systems

8
The Selection Process

• Critical activity for COTS-Based systems
• Oriented by evaluation criteria
• COTS candidates must satisfy user requirements
• Includes an accurate understanding of the
  features of each product

9
The Adaptation Process

• COTS are Plug and Pray
• Development of all necessary software adapters
  and enhancements to the selected COTS
• Component Wrapping includes a software barrier
  around the components limiting what it can do

10
The Integration Process

• Includes all development efforts required to
  interconnect different COTS into a single
  integrated system
• Consists of developing additional parts not
  supported by available products
• Conventional development efforts

11
The Update Process

• Like other systems, COTS-based systems need
  successive updates
• Repair an error
• New required functionality
• Acquisition of new releases

12
Current Problems with COTS-Based Development

• Products from different vendors have to be
  integrated and tailored to provide complete
  system functionality
• Customers have limited access to products
  internals design
• COTS lifecycle is determined by vendor

When using COTS products, customers are placed in
a situation over which they have no control
13
The impact of these problems can be minimized if
adequate attention is paid to the process of
COTS evaluation and selection
14
The importance of the Selection Process

• Includes the understanding of user requirements
• Careful analysis of the capabilities and
  limitations of each COTS candidate
• Assessment of products quality

15
Main Dimensions of the Selection Process
16
Selection Process Challenges

• Lack of well defined process
• Use of inappropriate evaluation criteria
• Back-box nature of COTS components
• Unclear system expectations
• Rapid rate of changes of COTS

This means that any evaluation will typically
have some degree of error
17
Related Works
Product Identification Requirements Acquisition Non-functional requirements description Product evaluation Decision making analysis
OTSO (Off-The-Shelf Option) ? - - ? ?
STACE (Social-Technical Approach to COTS Evaluation) ? - ?
PORE (Procurement-Oriented Requirements Engineering) ? ? ? ?
? Address fully Deals with the issue but not
fully does not deal with the issue
18
The lack of a careful consideration of
non-functional requirements increases the risks
of COTS failure and costs of the final system
19
Our Contribution

• An approach that addresses the lack of
  requirements engineering methods during COTS
  evaluation/selection
• Focus on non-functional requirements analysis to
  assist the evaluation of COTS products

20
The CRE Method

• A systematic, repeatable and requirements-driven
  approach
• Iterative process of requirements acquisition/
• modeling and product evaluation/selection

21
CRE Iterative Process

• The selection is made by rejection, products that
  do not meet user requirements are eliminated

22
CRE Main Features

• Goal oriented - each phase is oriented by
  predefined goals
• CRE provides templates that include some
  guidelines
• Interactive phases The order is not rigid

23
The CRE Templates
Template 1
Goals Final result Information and requirements to be acquired Steps / sequence Important considerations
24
The Phases

• Identification identify core requirements and
  candidates COTS products
• Description Describe each product and user
  requirements
• Evaluation Analyze products compliance with
  requirements
• Acceptance Verify legal issues

25
Identification

• Define selection goals, include analysis of
  influencing factors

26
Identification

• Evaluation criteria definition
• Elicitation of critical requirements
• Interviews and questionnaires techniques
• COTS product searching
• Possible sources in-house libraries, Internet,
  magazines, conferences, vendors.
• Final result - list with all COTS candidates

27
Description

• Evaluation criteria must be elaborated in detail
• Non-functional requirements play an important
  role to discriminate between similar functional
  products (ex. flexibility, reliability,
  portability)
• It is usually difficult to check if a product
  satisfies non-functional requirements
• Use NFR Framework to refine these requirements

28
Description

• Feedback mechanism information exchange between
  requirements process and products description

29
Description

• Requirements document is elaborated containing
  (all) relevant information about stakeholders
  needs

Requirements Document
Req_ID ltunique identifiergt Type ltfunctional, non-functionalgt Description ltinformation about requirementgt Priority ltvary from 1 to 4gt Contributions ltcan interact in synergy and conflictgt Comments ltadditional observationsgt
30
Description

• CRE method provides situation rules to deal with
  requirements conflict
• IF ltconditiongt THEN ltactiongt

• If Strong_Confl Req1,Req2 and Req1_Prio gt
  Req2_Prio
• Then Deal with Req1
• Else If Strong_Confl Req1,Req2 and Req2_Prio gt
  Req1_Prio
• Then Deal with Req2

31
Description

• Checklist to help the elicitation of product
  information

Checklist Checklist
Products Aspects Vendor Aspects
Price Maturity
Conformance with quality standards Time delivery
Capacities Stability
Benefits Training
Constraints Reputation
Version control Support quality
32
Evaluation

• Use of appropriate techniques to assist
  decision-making process
• WSM (Weighted Scoring Method)
• Simple but results are not accurate
• AHP (Analytic Hierarchy Process)
• Complex use, provides more confidence in decisions

33
WSM - Weighted Scoring Method
Where a represents an alternative and n the
number of criteria
Conformance Score Priority Weight
Does not meet the requirement 0 Low 1
Meets with restrictions 1 Medium 2
Partially meets 2 High 3
Meets 3 Very High 4
34
AHP (Analytic Hierarchy Process)
35
Evaluation

• Perform product demonstration sessions to obtain
  detailed information about COTS features and
  limitations
• Obtain products compliance score with evaluation
  criteria
• Important step during decision-making process

36
Evaluation

• The cost of each COTS alternative extends the
  acquisition price
• Apply cost estimation models
• COCOTS proposed by B. Boehm
• Provides a model to estimate the associated costs
  of COTS integration

37
Acceptance

• Negotiation of legal issues with COTS vendors
• A license should minimally specify
• License grant
• Payments to the supplier
• Who owns the licensed product
• The risks and liability each party assumes

38
Main Contributions

• An effective approach to guide the selection of
  COTS products
• An iterative process of requirements acquisition
  and product evaluation
• Including a detailed description of
  non-functional requirements
• Support for decision-making analysis

39
Future Work

• Detailed treatment of requirements prioritization
  and negotiation
• The interplay between software architecture and
  the selection of multiple COTS

40
Non-Functional Requirements (NFR)

• Address important issues of quality for software
  systems
• Related to constraints on system services
• Play critical importance during system
  development
• Have a global impact on systems
• Referred as ilities

41
NFRs Main Features

• Subjective
• interpreted and evaluated differently by
  different people
• Relative
• importance may vary depending on the particular
  system
• Interacting
• Attempts to achieve one NFR can hurt or help the
  achievement of other

42
Non-functional requirements can be difficult to
deal with, yet dealing with NFRs can be vital
for the success of a software system
43
Non-Functional Requirements

• There is not a unique definition or complete list
  of non-functional requirements
• Different researchers use different terminology
• Previous works
• Bohem, 1976
• Roman, 1985
• Keller, 1990
• Standards ISO, ANSI, IEEE

44
Types of Non-Functional Requirements
sommerville 92
Non-functional requirements
External requirements
Product requirements
Process requirements
Legal constraints
Reliability requirements
Economic constraints
Safety requirements
Interoperability requirements
Efficiency requirements
Performance requirements
Capacity requirements
45
The NFR Framework

• Proposed by Chung, University of Toronto
• Systematic and global representation of NFRs
• Process-oriented approach
• Qualitative approach
• Represents NFRs explicitly as softgoals

46
Main Characteristics

• Softgoals - are the basic unit for representing
  non-functional requirements
• Interdependencies - state interrelationships
  among softgoals
• Methods - offer operationalization techniques
• Correlations - provide catalogues for inferring
  possible interactions

47
The notion of Softgoals

• A goal that has no clear definition
• Qualitative reasoning
• Degrees of satisfaction
• Interact in synergy or conflict
• Decomposed through AND or OR relationship
• AND - the softgoal is satisfied if all of its
  sub-goals are
• OR - the softgoal is satisfied if any of its
  sub-goals are

48
NFR Framework can be used to

• Acquire knowledge about
• the particular domain
• functional requirements
• particular kinds of NFRs
• Identify particular NFRs for the domain
• Decompose NFRs
• Identify operationalizations (satisfice)

49
Using the NFR Framework (cont.)

• To deal with
• ambiguities
• tradeoffs and priorities
• interdependencies among NFRs
• Select operationalizations
• Support decisions (design rationale)
• Evaluate the impact of decisions

50
Catalogues

• Present knowledge about NFRs
• Sources of knowledge specialists, developers,
  textbooks, developer guides, etc.
• Provide a broad range of expertise
• Types of catalogues
• NFR types (organize NFRs into specialized
  hierarchies)
• method (refine NFRs considering
  operationalizations)
• correlation (show implicit interdependencies)

51
Catalogue of some NFR types
52
Softgoal Interdependency Graph (SIG)
Secure system
AND contribution
Confidentiality of system
Availability of system
Integrity of system
OR contribution

Operationalization
Authorization of User
Identification of User
53
Implicit Interdependency SIG - Softgoal
Interdependency Graph
54
Dealing with Priorities

• Priority softgoals can be identified as
• Critical vital for the success of the system
• Dominant deal with a significant part of the
  organization workload
• Make appropriate tradeoffs among softgoals

55
Identifying a Softgoal as a Priority SIG -
Softgoal Interdependency Graph
User-friendly system
Secure system
Confidentiality system
Simplicity interface
Accessibility capacities
Learnability user
Integrity system
Availability system

-

Identification user
Authorization user
!
Simplicity interface
Priority Softgoal
56
Recording Design Rationale

• Design decisions should be supported by
  well-justified arguments
• Reasons can be stated for making refinements, for
  selecting an alternative, etc.
• A Claim softgoal can rationalize tradeoffs

57
Recording Design Rationale SIG - Softgoal
Interdependency Graph
58
Selecting among alternatives

• The refinement process continues until the
  possible solutions are sufficiently detailed
• Evaluate the impact of decisions
• Consider operationalizations and decide whether a
  chosen alternative meets a softgoal sufficiently

59
Evaluating the Impact of Decisions

• Bottom-up process
• Evaluation of softgoals are represented by labels
  (such as ? and X)
• Positive contribution
• A satisficed offspring results in a satisficed
  parent
• A denied offspring results in a denied parent
• Negative contribution
• A satisficed offspring results in a denied parent
• A denied offspring results in a satisficed parent

60
Identifying a Softgoal as a Priority - SIG
User-friendly system
Secure system
Confidentiality system
Simplicity interface
Accessibility capacities
Learnability user
Integrity system
Availability system

?
X
-
?


Identification user
Authorization user
!
Simplicity interface
?
Claim User authorization will not hurt system
simplicity much