Lecture Summary 1-11 Objectives, Projects, Requirements, Design, Prototyping, Architecture, Estimation, Risk, Usability, Construction, and Testing

1
Lecture Summary 1-11Objectives, Projects,
Requirements, Design, Prototyping, Architecture,
Estimation, Risk, Usability, Construction, and
Testing
CS 540 Quantitative Software Engineering

2
History Software Crisis Persists

• The software crisis, NATO conference, autumn
  1968, Garmisch, Germany
• 250B spent on 175,000 projects with
• Average cost of projects large company 2.3M,
  medium 1.3M small .4M
• Almost a third will be cancelled before they are
  completed
• Over half will cost twice their current estimates
• Only 15 will be finished on time and on budget
• FAA Air Traffic Control project,
• Mars missions
• London Ambulance Dispatch System
• Airbus

3
Customers do not get what want
The National Software Council found this
situation largely unchanged in 1995 and 2005.
200,000 Used after rework
Abandoned or reworked
100,000 Used as delivered
Delivered but not used
A 1979 U.S. Government Accounting office report
(FGMSD-80-4), which breaks down results from
6.8 million in nine federal software projects,
shows that only 2 of the software was used as
delivered.
INFORMATION PROVIDED BY ACM SICSOFT SOFTWARE
ENGINEERING NOTES, VOL 10 NO. 5, OCTOBER 1985.
4
Views of Software Engineering

• SEI
• Engineering is the systematic application of
  scientific knowledge in creating and building
  cost-effective solutions to practical problems in
  the service of mankind.
• Software engineering is that form of engineering
  that applies the principles of computer science
  and mathematics to achieving cost-effective
  solutions to software problems.

5
Quantitative Software Engineering

• Quantitative Software Engineering is an
  analytical approach to producing reliable
  software products within budget and on time -
  Stevens QSE program
• Which matches the IEEE definition
• The application of a systematic, disciplined,
  quantifiable approach to the development,
  operation and maintenance of software that is
  the application of engineering to software

6
Software Engineering as a Profession

• Organizations
• IEEE, ACM, ISO, CMU SEI
• Standards
• IEEE SWEBOK
• Ethical Standards
• Best Practices

7
Software Engineering Knowledge (SWEBOK)

• Software requirements analysis
• Software design
• Software construction
• Software testing
• Software maintenance
• Software configuration management
• Software quality analysis
• Software engineering management
• Software engineering infrastructure
• Software engineering process

8
Software Process Models

• Hacking
• Waterfall
• Prototyping
• Incremental
• RAD
• Spiral
• Agile

9
Royces Waterfall Model

• Phases in lockstep
• Encourages narrow focus
• Mistakes found late
• Generates tightly coupled systems

10
Waterfall document-driven milestones

• Baselined Requirements Specification
• Baselined Test Plan
• Baselined Design Specification
• Baselined Code
• Test Results report

11
Waterfall process

• Change intolerant
• Document-driven
• Customer must state all requirements upfront, but
  fully 40 of requirements evolve
• Features unavailable until implementation phase
• Strong distinction between Development and
  Maintenance
• Came from an era when coding was difficult and
  computing was expensive
• Still in use

12
Development approach comparison
TRADITIONAL SOFTWARE PROCESS
PROTOTYPE BASED PROCESS
Systems Engineering Prototype
Systems Engineering
20
Final Development
Design, Develop, Test, Install
80
Deployment
Final Development, Deployment
40 REDUCTION
40
13
Incremental

• Functionality of system is produced and delivered
  in small increments
• prototyping waterfall - but focuses on
  delivery of operational product
• Focuses on assigning priorities to features for
  each release - Rolling Stones
• Especially useful with small staff, to manage
  technical risks and to build to current
  capability (e.g., hardware)
• Not good when delivery is expensive

14
Rapid Application Development (RAD)

• Incremental development
• Focus is on time to market
• JRPs (Joint Requirements Planning) - requirements
  triaged and JADs (Joint Application
  Design)-developers and users work together
  through prototyping to a finalized design
• Product developers are SWAT (Skilled with
  Advanced Tools) team
• Cutover- final testing of system takes place,
  users trained, system installed
• Best used in information systems where technical
  risks are low
• Typically 60-90 days

15
RAD advantages

• Customer involvement
• Tools reduce cycle time
• Project team usually knows problem domain, key
• Developers are willing to dive deeply into domain
  - key success factor in any model
• Time-box, usually 60 days, bounds development
• Installation and user training are an explicit
  part of the process

16
Boehms Spiral Model
Analysis
Design

• Incremental and iterative
• Evolutionary
• Prototyping quick feedback
• Continuous integration

Coding
Testing
17
Spiral advantages and disadvantages

• Advantages
• Risk analysis may uncover show stoppers early
• Chunks development so that it is affordable
• Waterfall like characteristics add some
  discipline, management control
• Lots of feedback from all stakeholders
• Disadvantages
• Expensive for small projects
• Requires risk assessment expertise
• Development is on again/off again so the other
  stages can be accomplished - in prototyping
  development is continuous.

18
Agile processes

• The processes of specification, design and
  implementation are concurrent. There is no
  detailed specification and design documentation
  is minimized.
• The system is developed in a series of
  increments. End users evaluate each increment and
  make proposals for later increments.
• System user interfaces are usually developed
  using an interactive development system.

19
Requirements issues(
Requirements Water Proto Spiral RAD Inc
Well known - - -
Defined early - -
Change often - - -
Proof of concept - -
Complex system - -
Early Functionality -
20
Carnegie Mellon SEI CMM (Paulk, 1999)
LEVEL FOCUS KEY PROCESS AREAS
5 Optimizing Continual Process Improvement Defect prevention, Technology change management, Process change management
4 Managed Product and process quality Quantitative process management,Software quality management
3 Defined Engineering processes and organizational support Organization process focus, Organization process definition, Training program, Integrated software management, Software product engineering, Intergroup coordination, Peer reviews
2 Repeatable Project management processed Requirements management, Software project planning, software project tracking and oversight, Software subcontract management, Software QA, Software configuration management
1 Initial Competent people And heroics
21
CS 540 QSE SWEBOK Software Engineering
Management KA (Knowledge Area)

• Why is software engineering any different than
  any other engineered product?
• Difficult for clients to appreciate
  complexity/value
• Software engineering processes generate change
• Iterative by nature
• Novelty and complexity is often extremely high
• Elements of creativity and discipline interact
• Rapid change in underlying platforms
• Often integrated in to enterprise portfolio
• Production Cost Structure is front loaded!

22
CS 540 QSE SWEBOK Software Engineering
Management KA Sub Areas

• Initiation and Scope Definition
• Determination/Discovery/Invention/Negotiation of
  Requirements
• Feasibility Analysis (Cost Evaluation,
  Investment, Failure Risk)
• Process for Review and Revision of Scope
• Software Project Planning
• Process Planning
• Deliverables
• Effort, Schedule, Cost Estimation
• Resources Allocation
• Risk Management
• Quality Management
• Plan Management
• Software Project Enactment/Implementation
  Monitor and Control
• Review and Evaluation

23
CS 540 QSE SWEBOK Software Engineering
Management KA Sub Areas

• Software Project Enactment
• Implementation Monitor and Control
• Supplier Contract Management
• Implementation of Measurement Process (earned
  value, schedule variance)
• Monitor Process
• Control Process
• Reporting
• Review and Evaluation
• Satisfaction with requirements (including
  performance, usability, and contractual terms)
• Reviewing and Evaluating measures of
  effectiveness (validity)
• Closure
• SW Engineering Measurement Metrics include
  cost, schedule compliance, effectiveness,
  quality, etc.

24
Trends in Software Expansion
Expansion Factor
The ratio of Source line of code to a machine
level line of code
Order of Magnitude Every Twenty Years
Technology Change
Machine Instructions
High Level Languages
Macro Assemblers
Database Managers
On-Line Dev
Prototyping
Subsec Time Sharing
Object Oriented Programming
Large Scale Reuse
Regression Testing
4GL
Small Scale Reuse
Each date is an estimate of widespread use of a
software technology
25
Software Requirements Process

• Requirements Elicitation
• Requirements Analysis
• Use Cases
• Requirements Specification
• Prototype/Modeling
• Requirements Management

26
What is a requirement?

• A property that must be exhibited by a system to
  solve some problem.
• Requirements may be
• Functional providing product capabilities
• Non-Functional constraining the implementation

27
Managing requirements

• Create and invoke the MOV (Measurable Operational
  Value)
• Establish, update and model the business case .
• Strategic linkages to business and technology
  organizations to AVOID SHELFWARE
• Continuous customer agreement on requirements
• Requirements agreement used as a basis for
  estimating, planning, implementing and tracking
• FORMAL COMMITMENT PROCESS

Source Andriole, Stephen J., Managing System
Requirements, Methods, Tools, and
Cases McGraw-Hill, 1996
28
Requirements Elicitation Techniques

• Asking interview, questionnaire, structured
  interview, Delphi (group based)
• Task analysis hierarchical decomposition
• Scenario based analysis instances of tasks,
  use-case (not only for OO)
• Ethnography studying folks in natural setting
• Walking around
• Models

29
Requirements Elicitation Techniques

• Form analysis existing forms
• Natural language descriptions training, manuals,
• Derivation from existing system
• Domain analysis study existing systems w/in
  domain, reusable components
• Project future business environment from PMO and
  systems

30
Summary Requirements 9/13/2007

• Software Requirement property which must be
  exhibited by software developed or adapted to
  solve a particular problem
• Fundamentals Functional vs. non-functional,
  Quantifiable vs Qualifiable, Emergent vs. Basic
• Four Phases Elicitation, Analysis,
  Specifications, Validation
• Elicitation sources and techniques
• Analysis Classification, Modeling, Design, and
  Negotiation
• Specifications System Definition, Requirements
  Specifications
• Validation Requirements Reviews, Prototyping,
  Model Validation, Acceptance
• Practical Considerations Iteration, Change
  Management, Attributes, Traceability, Measurement

31
Software Prototyping Review

• Types Throwaway, Evolutionary, Incremental
• Advantages and Disadvantages
• When to Use
• Methods DSDM, Evolutionary, Operational, SCRUM
• Tools Screen Generators, design, tangible
  architect, Visual Basic, REE, LYMB
• Simulation technologies

32
Evolutionary Prototyping Advantages

• Accelerated delivery of the system
• Rapid delivery and deployment are sometimes more
  important than functionality or long-term
  software maintainability
• User engagement with the system
• Not only is the system more likely to meet user
  requirements, they are more likely to commit to
  the use of the system

33
Evolutionary Prototyping Challenges

• Management problems
• Existing management processes assume a waterfall
  model of development
• Specialist skills are required which may not be
  available in all development teams
• Maintenance problems
• Continual change tends to corrupt system
  structure so long-term maintenance is expensive
• Contractual problems

34
On prototyping

• Evolutionary versus throwaway prototypes
• Prototyping takes advantage of high level
  languages, sacrifices efficiency for speed
• Great when few initial requirements
• People (dev and users) like prototypes
• Danger of feature creep
• Documentation, performance of final system may
  suffer - perceived lack of discipline
• Customer and management may think it is done
• Quality can go either way
• Requires experienced developers

35
User interface prototyping

• It is impossible to pre-specify the look and feel
  of a user interface in an effective way
• UI development consumes an increasing part of
  overall system development costs
• User interface generators may be used to draw
  the interface and simulate its functionality with
  components associated with interface entities
• Web interfaces may be prototyped using a web site
  editor

36
What to look for in an Architecture

• The purpose the system is intended to satisfy.
• The major functional components and/or platforms.
• The relationship between the components.
• The fit to function.
• The dynamic interplay of control and
  communication between these components.
• The systems ease of use.
• The data storage and flow of data among these
  components.
• The resources which are consumed by each
  component in the performance of its task.

37
What is Software Architecture?

• The software architecture of a program or
  computing system is the structure or structures
  of the system, which comprise software
  components, the externally visible properties of
  those components, and the relationships between
  them.
• The term also refers to documentation of a
  system's software architecture. Documenting
  software architecture facilitates communication
  between stakeholders, documents early decisions
  about high-level design, and allows reuse of
  design components and patterns between
  projects.Len Bass, Paul Clements, Rick Kazman
  2003

38
What are Software Architecture Views?

• Software architecture is commonly organized in
  views, which are analogous to the different types
  of blueprints made in building architecture.
  (ANSI/IEEE 1471-2000)
• Views are instances of viewpoints, where a
  viewpoint exists to describe the architecture in
  question from the perspective of a given set of
  stakeholders and their concerns.
• Some possible views are
• Functional/logic view
• Code view
• Development/structural view
• Concurrency/process/thread view
• Physical/deployment view
• User action/feedback view

39
What is Software Architecture?

• TSQTSE software architecture is the body of
  instructions written in a specific coding
  language that controls the structure and
  interaction of software modules
• SWEBOK software architectural design (top level
  design) describing the top-level structure and
  organization and identifying the various
  components
• Pressman architectural design represents the
  structure of data and program components that are
  required to build a computer-based system

40
Future Trends in Architectural Design SOA, Web
Services, EA

• Service-Oriented Architecture (Gartner)
  Service-oriented architecture is a client/server
  software design approach in which an application
  consists of software services and software
  service consumers (also known as clients or
  service requesters). SOA differs from the more
  general client/server model in its definitive
  emphasis on loose coupling between software
  components, and in its use of separately standing
  interfaces."
• SOA Infrastructure components
• Developer Suites JAVA, J2EE, XML, HTML
• Business Process Execution Languages
• File Content Management
• Business Process Management
• Enterprise Service Bus integration

41
Benefits of Good Software Architectures

• Helps identify and isolate reusable components
  that can speed implementation and improve system
  quality.
• Assists in measuring project impacts of
  inevitable ongoing technical and business
  decisions and compromises.
• Leads to clearly defined organizations with
  inherent project efficiencies, good
  communications and decision making.

42
Software Design Definition

• the process of defining the architecture,
  components, interfaces, and other characteristics
  of a system or component. SWEBOK
• life cycle activity in which software
  requirements are analyzed in order to produce a
  description of the softwares internal structure
  that will serve as the basis for its
  construction.
• fits between requirements and construction

43
Software Design Knowledge Areas (SWEBOK)

• Fundamentals concept, context, process, and
  enabling techniques
• Key Issues
• Software Structure micro-architecture, patterns
• Design Quality Analysis and Evaluation
• Software Design Notation
• Strategies and Methods

44
Project Metrics Why Estimate?

• Cost and schedule estimation
• Measure progress
• Calibrate models for future estimation
• Manage Project Scope
• Make Bid/No Bid decisions
• Make Buy/Build decisions

45
Specification for Development Plan

• Project
• Feature List
• Development Process
• Size Estimates
• Staff Estimates
• Schedule Estimates
• Organization
• Gantt Chart

46
Popular Methods for Effort Estimation

• Parametric Estimation
• Wideband Delphi
• Cocomo
• SLIM (Software Lifecycle Management)
• SEER-SEM
• Function Point Analysis
• PROBE (Proxy bases estimation, SEI CMM)
• Planning Game (XP) Explore-?Commit
• Program Evaluation and Review Technique (PERT)

47
Sizing Software Projects

• Effort (productivity)-1 (size)c
• productivity staff-months/kloc
• size kloc

Staff months
500
Lines of Code or Function Points
48
Understanding the equations

• Consider a transaction project of 38,000 lines of
  code, what is the shortest time it will take to
  develop? Module development is about 400
  SLOC/staff month
• Effort (productivity)-1 (size)c
• (1/.400 KSLOC/SM) (38 KSLOC)1.02
• 2.5 (38)1.02 100 SM
• Min time .75 T (.75)(2.5)(SM)1/3
• 1.875(100)1/3
• 1.875 x 4.63 9 months

49
Function Point (FP) Analysis

• Useful during requirement phase
• Substantial data supports the methodology
• Software skills and project characteristics are
  accounted for in the Adjusted Function Points
• FP is technology and project process dependent so
  that technology changes require recalibration of
  project models.
• Converting Unadjusted FPs (UFP) to LOC for a
  specific language (technology) and then use a
  model such as COCOMO.

50
Adjusted Function Points

• Now account for 14 characteristics on a 6 point
  scale (0-5)
• Total Degree of Influence (DI) is sum of scores.
• DI is converted to a technical complexity factor
  (TCF)
• TCF 0.65 0.01DI
• Adjusted Function Point is computed by
• FP UFP X TCF
• For any language there is a direct mapping from
  Function Points to LOC
• Beware function point counting is hard and needs
  special skills

51
Initial Conversion
http//www.qsm.com/FPGearing.html
Language Median SLOC/function point
C 104
C 53
HTML 42
JAVA 59
Perl 60
J2EE 50
Visual Basic 42
52
COCOMO

• COnstructive COst MOdel
• Based on Boehms analysis of a database of 63
  projects - models based on regression analysis of
  these systems
• Linked to classic waterfall model
• Effort is number of Source Lines of Code (SLOC)
  expressed in thousands of delivered source
  instructions (NCKSLOC) - excludes comments and
  unmodified software
• Original model has 3 versions and considers 3
  types of systems
• Organic - e.g.,simple business systems
• Embedded -e.g., avionics
• Semi-detached -e.g., management inventory systems

53
Business Realities of Software Estimation

• Customer Affordability/Willingness to pay
  design the system to win the business
• Conflict of Interests Project Manager
  (affordability and profit) vs. Development/Test
  (pad budgets)
• Estimation with uncertainty the more you know
  (better understood)the higher the estimate
• Personality Traits risk aversion, tolerance for
  ambiguity
• Staffing Issues sometimes any business is better
  than no business
• Opportunity Cost Winning a bid prevents you from
  working on other deals
• Strategic Interests Losing is sometimes ok

54
Software Risk

• Risk is the possibility that an undesirable event
  in the life of a software project can happen
• Requires uncertainty and loss
• Project Risk cost, schedule, completion
• Technical Risk feasibility, viability, etc.
• Business Risk contractual compliance, payment,
• Legal Risk damages, liability

55
Software Risk Mitigation Summary

• Project Risk cost, schedule, completion
• Estimation, simplifications, staffing changes
• Technical Risk feasibility, viability, etc.
• Prototyping, experimentation, invention, multiple
  path
• Business Risk contractual compliance, payment,
• Contractual mitigation time and materials,
• Legal Risk damages, liability
• Limitations of liability, termination for
  convenience

56
RISK Top 10 risk factors

• Personnel shortfall
• Unrealistic schedule or budget
• Wrong functionality
• Wrong user interface
• Gold plating (fun and games)

• Requirements volatility
• Bad external components
• Bad external tasks (subcontracts)
• Real time shortfalls
• Capability shortfall (bleeding edge)

57
Risk Tracking and Control

• Use Action Item system
• Track status of risks and actions taken to
  address them
• Define Risk Exposure
• RE Prob(risk) x Cost(risk)
• Review ten risks with highest RE at every
  project meeting

58
Usability ISO Definitions

• The document ISO 9126 (1991) Software Engineering
  Product Quality, issued by the International
  Organization for Standardization, defines
  usability as
• A set of attributes that bear on the effort
  needed for use, and on the individual assessment
  of such use, by a stated or implied set of users.
  
• The document ISO 9241-11 (1998) Guidance on
  Usability, also issued by the International
  Organization for Standardization, defines
  usability as
• The extent to which a product can be used by
  specified users to achieve specified goals with
  effectiveness, efficiency and satisfaction in a
  specified context of use.

59
Usability Operational Definitions

• Learnability (e.g. intuitive navigation)
• Efficiency of use
• Memorability
• Few and noncatastrophic errors
• Subjective satisfaction

60
Usability Considerations

• Who are the users, what do they know, and what
  can they learn?
• What do users want or need to do?
• What is the general background of the users?
• What is the context in which the user is working?
  
• What has to be left to the machine? What to the
  user?

61
Why spend effort on the Usability?

• Increased efficiency
• Improved productivity
• Reduced errors
• Reduced training - strive for game-like training
• Improved acceptance

62
SWEBOK Knowledge Areas Software Construction
Fundamentals

• Minimizing Complexity
• Anticipating Change Generalize,
• Construction for Verification
• Code reviews, unit test, automated testing,
  restrict language features
• Standards in Construction
• Communication methods (document format), tools
  (flow notations), platform calls
• Construction Models (process models like
  waterfall, Extreme programming, prototyping,
  Agile, etc.)
• Construction Planning
• Construction Measurement
• Code developed, modified, reused
• Code inspections completed
• Unit testing fault and fix rates

63
SWEBOK Knowledge Areas Practical Considerations
Software Construction

• Construction Design
• Construction Languages configuration languages,
  toolkit languages, programming languages
• Coding control structures, error handling,
  security, organization, documentation
• Unit Testing
• Reuse

64
Configuration Management

• Not a simple task!
• Different versions of software usually is in the
  field during the life cycle
• Different parts of the team are on different
  versions of the software and documents
• The same release of a software product may have
  multiple versions consisting of different
  combinations of software components
• Configuration management is both a development
  and production issue

65
Baseline

• IEEE - reviewed and agreed upon basis for
  further development which can be changed only
  through formal control procedures
• Contained in the baseline are configuration
  items source, objects, requirements
• Configuration management maintains integrity of
  these artifacts
• Major error- retrace steps through code, design
  documents and requirements specification
  -TRACEABILITY

66
Approaches to Quality
Conform Improve
Product ISO 9126 Best Practices
Process ISO 9001 CMM
67
Software Quality vs. Software Testing

• Software Quality Management (SQM) refers to
  processes designed to engineer value, functional
  conformance, and minimize faults, failures, and
  defects
• Includes processes throughout the software life
  cycle (inspections, reviews, audits, validations,
  etc.
• Software testing is an activity performed for
  evaluating quality (and improving it) by
  identifying defects and problems (SWEBOK)

68
SWEBOK Software Testing

• Software testing consists of dynamic
  verification of the behavior of a program on a
  finite set of test cases, suitably selected from
  the usually infinite executions domain, against
  the expected behavior
• Dynamic (means software execution vs. static
  inspections, reviews, etc.)
• Finite (Trade-off of resources)
• Selected Techniques vary on how they select
  tests (purpose)
• Expected behavior functional and operational

69
SWEBOK Software Testing Topics

• Fundamentals
• Definitions, standards, terminology, etc.
• Keys issues looking for defects vs. verify and
  validate
• Test levels
• Unit test? Beta test
• Objectives conformance, functional, acceptance,
  installation, performance/stress, reliability,
  stress, usability, etc
• Test Techniques
• Ad-hoc, exploratory, specification-based,
  boundary-value

70
SWEBOK Software Testing Topics

• Test Techniques
• Ad-hoc
• Exploratory
• Specification-based
• Boundary-value analysis
• Decision table
• Finite state/Model
• Random Generation
• Code based (control flow vs. data flow)
• Application/Technology based GUI, OO, Protocol,
  safety, certification,

71
SWEBOK Software Testing Topics

• Test Effectiveness Metrics
• Fault types and categorization
• Fault density
• Statistical estimates of find/fix rates
• Reliability modeling (failure occurrences)
• Coverage measures
• Fault seeding

72
Testing Metrics

• Test Case Execution Metrics
• Percent Planned, Executed, Passed
• Defect Rates
• Defect rates based on NCLOC
• Predicted defect detection (upper/lower control
  limits)
• Fault Density/fault criticality (software control
  board)
• Fault types, classification, and root cause
  analysis
• Fault on fault, breakage, regression test
  failures
• Reliability, Performance Impact
• Field faults/ Prediction of deficiencies

73
Types of Tests

• Unit
• Interface
• Integration
• System
• Scenario
• Reliability
• Stress
• Verification
• Validation
• Certification

74
QSE 540 Objectives 1-5

• Knowledge of industry practices (SWEBOK, etc.),
  organizations, ethics, etc.
• Software development process models and business
  models
• Requirements/Specification processes and
  importance
• Project management options costs, resources,
  schedules, features, etc.
• Software attributes functionality, reliability,
  fault tolerance, other ilities

75
QSE 540 Objectives 6-10

• Software metrics for projects, requirements,
  design, architecture, estimation, risk, testing,
  quality, etc.
• Methodologies for estimation of size, effort, and
  time requirements
• How life cycle methodologies vary with
  technology, domain, and business conditions
• Apply project management principles, risk
  analysis, etc. to software development problems
• Understand organizational, technological, and
  architectural issues associated with software
  engineering
• Keep current with professional advances.