Evolution of

1
Evolution of Software/System Quality Second
Asia-Pacific Conference on QUALITY SOFTWARE
Dr. Raymond A. Paul December 11, 2001
NOV 2000
2
Definition

• QUALITY is the characteristic that distinguishes
  the grade of excellence or superiority of a
  process, product, or service.
• In general usage, QUALITY means different
  things.Meaning of QUALITY varies considerably
  across specific disciplines and applications.

• What is Quality?

3
Historical Aspects of QUALITY Evolution

• Edward Deming
• Postulated Statistical QUALITY Control Principles
• Famous 14 Points of QUALITY Management
• A Subset of these Principles successfully adapted
  by Japanese Manufactures
• William Crosby
• Emphasized Humanistic Behavioral Aspects of
  QUALITY Improvement
• Becoming More Important Now

4
Juran
TOTAL STATISTICAL QUALITY MANAGEMENT

• QUALITY Trilogy
• A. QUALITY Planning
• Set of QUALITY Goals
• Set Plans for Operations Based on these Goals
• B. QUALITY Control
• Responsible for Meeting QUALITY Goals
• Prevent Adverse Changes
• Set and Observe
• Performance Measures
• Compare with Industry Standards
• Benchmarking
• Compare with Deming, CMM
• C. QUALITY Improvement
• Moving from Current Level to the Next Higher
  Level
• Organize Teams, Train Operators to identify and
  Correct QUALITY Problems

5
Six Sigma Initiatives

• GE and Motorola
• Recognized Widely by Manufacturing Industry
• Adopted in Various Ways by Software Industry
• Major Results
• Total QUALITY Management (TQM)
• ISO Certification
• Malcolm Baldridge Nation of QUALITY Award (Dept.
  of Commerce)
• Six Sigma Means no more that 3.4 defects per
  million opportunities (six standard deviations
  away from the average of an assumed normal
  distribution)
• IBM (late Harlan Mills) Clean Room follows this
  Metric

6
Perspectives on QUALITY (Garvin 1984)
VIEWS

• Transcendented View
• Something we recognize but cannot define
• EXAMPLE Beauty, Aristotle's Concept of Form
• User View
• Fitness of Purpose
• Manufacturing (Design) View
• Conformance to Specification
• Product View
• Tied to inherent product functions and
  characteristics
• Value-Based View
• Depends on the price the customer is willing to
  pay

• What is Quality?

7
QUALITY Models
(A LOOSE INTERPRETATION)

• McCalls QUALITY Model (1977)
• Shows relationships between external QUALITY
  Factors such as correctness, reliability,
  usability, testability, etc. and Product QUALITY
  Criteria such as traceability, completeness,
  error tolerance, etc.
• PROBLEM Difficult to measure these QUALITY
  Factors
• No Standards, No Methods, No Tools

• What is Quality?

8
Boehm's QUALITY Model (1986)

• Asserts that QUALITY software satisfies the needs
  of the users, designers, testers, and maintainers
• Relates softwares general utility to
  maintainability, reliability, testability, etc.,
  to device independence, completeness,
  accessibility, etc.
• Very difficult to apply in practice, but easy to
  understand and learn
• Parts of it can be supported by tools therefore,
  useful in a restricted sense

• What is Quality?

9
ISO 9126 QUALITY Model (1991)

• Consolidates many views of software QUALITY
• ISO hierarchy is strict and non-overlapping
  unlike McCalls and Boehm's
• EXAMPLE In the model, maintainability depends
  ONLY on analyzability, changeability, and
  stability. Whereas, in Boehms model,
  maintainability and understandability depend on
  structuredness, legibility, etc., in an
  overlapping way.
• Like Boehms model, it does not tell us how to
  measure characteristics such as security,
  accuracy, and interoperability.

• What is Quality?

10
Software QUALITY Characteristics Desired
(MOST MODELS INCLUDE THESE)

• Functional Correctness
• Functions and their specified properties
• Reliability
• Attributes maintained by software under stated
  conditions
• Usability
• Effort needed for users
• Efficiency
• Software performance with a given set of
  resources
• Maintainability
• Effort needed to make specified modifications
• Portability
• Effort needed to transfer software from one
  environment to another specified environment

• What is Quality?

11
Comments on the Software QUALITY Models

• Some important characteristics such as
  evolvability, safety, and security are left out
• Dromeys Model (1996) tries to address and serve
  some of these issues but not satisfactory

• What is Quality?

12
MATURITY Models (MM)

• Specification Process Product
• Complete MM emphasizes
• Process MATURITY
• What is MATURITY?
• Methods of continuously improving a procedure,
  activity, or process by planned measurements and
  feedbacks.
• In essence, it emphasizes incremental but
  positive improvement of an activity via repeated
  use and measurement.

(Design andImplementation)

• Measuring Quality

13
Analysis of Current MATURITY Models

• One Dimensional
• Does Not Include Items such as
• Maintenance
• Application
• Customer Satisfaction
• Correctness with Respect to Specifications
• Ease of Use, Ease of Change (maintenance)
• No Common Set of Assumptions
• No Common Set of Measurements (no
  standardization)
• No Financial Business Considerations
• No Interoperability and Scalability
  Considerations
• The emphasis in all these models
  INCREMENTAL!
• Incremental advances based on current experience.
• NO LEAP FROGGING!!!

• Measuring Quality

14
Pauls Call

• MATURITY means growing older and wiser
• Collecting and retrieving experiences
• Measure, observe, gather feedback from the
  product and process, update lessons learned from
  the product and process
• Maturity techniques continuously strive to
  improve processes, procedures, and activities by
  repeated use, planned measurements, and feedback
• In essence, maturity emphasizes incremental but
  positive improvement of an activity via repeated
  use, measurement, analysis of results of the
  product, and feedback refinement

• Measuring Quality

15
Questions

• What to measure? When to measure? Where to
  measure?
• How to measure? How to analyze and interpret
  results?
• Remember the problems of measurement...
• Assumptions
• Assessment
• Analysis
• have to be standardized
• One Method GQM Paradigm ( )
• Goal, Question, Metric (BASILI, et al.)
• Not Very Effective
• Have to customize the product design goal from
  designers that could be different from users

goal-primary-samegoal-primary-different

• Measuring Quality

16
Hierarchy of Maturity Models

• We Propose a Hierarchy of MATURITY Models for
  Improvement of IT Processes and Products

IT Maturity Model (ITMM)
Computing MM
Communications MM
Hardware MM
Software MM

• Measuring Quality

17
Assistive Technology to Improve QUALITY of Life

• These technologies assist users with physical
  disabilities. Quality of Life Improvement
• 47 million Americans are in some form disabled
• 75 of Americans with major disabilities are
  unemployed (President Clinton) Major Disability
  Vision Impairment World is so Visual
• Key Augmenting Human Productivity
• Designers do not have people with disabilities in
  mind when they develop their products
• Philosophy Design products that work with more
  people
• Methods of Design
• Universal Designs
• Specialized Designs

• Measuring Quality

18
QUALITY Parameters

• Usage Based Utilitarian
• Satisfaction of Users Needs
• Functionality
• Ease of Use and Handling
• Troubleshooting
• Restart and Recovery
• Maintenance and Analyzability
• Technology-Based Aspects
• Design Complexity
• Maintainability
• Evolutionary Flexibility
• Reconfigurability
• Agility
• Growth Enhancement

• Measuring Quality

19
QUALITY Parameters (cont.)

• Economic Considerations
• Cost to the Seller, and Price to the Buyer
• Competitiveness with Similar Products
• Ease of Manufacture, Reuse, and Maintenance
• Aesthetic Considerations
• Form and Function
• Appearance and Content
• Appropriateness for the Purpose
• Color, Packaging, Appearance
• Attractiveness to Users
• Attention-Grabbing Features
• Entertaining, Interesting, Attention Keeping,
  e.g. TV/Websites
• Mass Appeal
• Class/Culture Appeal
• Individual Appeal

• Measuring Quality

20
QUALITY Parameters (cont.)

• Personality/Individualization
• To support and help physical preferences and
  disabilities
• EXAMPLES
• Right vs. Left Handedness
• Poor Vision
• Hearing Impairment
• Web Browsing Personalization
• My YAHOO
• My EXCITE
• Personalization for
• News Junkies
• Financial Analysts
• Customized Information

• Measuring Quality

21
Another QUALITY Life Cycle Model
Another QUALITY Life Cycle Model
Another QUALITY Life Cycle Model
Another QUALITY Life-Cycle Model
Builtto Last
DemandGoodies
Make it Work
New Functions/Enhancements
Activities
Processes
Products
Services
Enterprises

• Life-Cycle Objectives of a Designer

Works the First Time
WorksMany Times
Use BecomesHabit Forming
Establishes UsersDependency and Brand Loyalty
(SONY)

• Achieving Quality

22
Designing Methodologies for Embedding
QUALITY-Based (QUALITY SUPPORTING) Functions

• Major Characteristics of These Designs
• Reliability, Fault Tolerance via
• Redundancy
• Highly Reliable Proven Components/Parts,
  Interconnections, Networks
• Design Methods
• Flexibility - Reconfigurability
• Dynamic Introduction of
• Standby Optional Functions
• Universal Designs
• Component-Based, Easy-Assembly Design
• Building Block Based Lego like
• Assembly Disassembly Principle

• Achieving Quality

23
Designing Methodologies (cont.)

• The Methods Presented Give Easy-Assembly
  (Disassembable) Subsystems
• Good for Maintenance (Repair, Modification)
• Subsystems and their integral components are
  loosely coupled. This helps system
  understandability, analyzability, and reuse
• Helps to add on or enhance functions
• Complex Systems are Synthesized by
• Layered Architectures each layer assembled from
  components
• Ring-Based Architectures
• Good for Security

• Achieving Quality

24
Designing Methodologies (cont.)
To Improve Performance Characteristics (in
real-time) the System can be Tightly Coupled

• Components (Building Blocks)
• Assemblers
• Subsystems
• System (made of loosely coupled subsystems)

SYSTEM A
For Better System Performance, Real-Time
Applications use Tight-Coupling Principles
(SYSTEM B). SYSTEM A is used as a Prototype.
A more compact high performance design can
be generalized by coalescing the layers (rings),
i.e., tight coupling of some layers SYSTEM B
SYSTEM B
In coalescing several small components into a
bigger element, we reduce the inter-component
overhead. More importantly, we eliminate or
reduce the conflicts among assumptions made
during the design of the components.

• Achieving Quality

25
Distributed System and Reconfiguration Design

• QUALITY implied here includes
• Fault Tolerance
• Control (Concurrency and Event Control)
• Reconfigurability (Part of Control)
• Heterogeneity
• EXAMPLES
• CORBA Components and Distributed Control
  Technologies
• Object-Oriented Design Components
• NASA Space Shuttle Avionics
• Hardware and Software
• Structured (Preferred) Reconfiguration Philosophy
• Very Simple Software Reconfigurations

• Achieving Quality

26
Universal Designs

• Primarily for Mass-Produced, Batch-Processed
  Hardware Systems
• Holds True for Software Systems as well
• PRINCIPLE
• The System Includes Many Functions Primary,
  Auxiliary, and Supporting Functions
• User/Supplier triggers as activities those
  supporting and auxiliary functions needed for
  that specific application or circumstance along
  with the primary functions
• EXAMPLE
• Pentium II MMX
• Has 57 supporting instructions (functions) to
  improve multimedia application processing
• Underlying Philosophy
• Design products that work for most applications
  (or people)
• INSIGHT Easier to mask out unwanted functions
  from a universal design (cheaper)
• than to build specialized functions from scratch
  (more expensive, time
• consuming, and labor intensive)

• Achieving Quality

27
QUALITY Manager Concept

• QUALITY embraces many aspects, for example
• Dependability, Fault Tolerance
• Maintainability
• Information Hiding
• QUALITY is a dynamic property at anytime.
  Different aspects of quality become more
  important depending on the circumstance and needs
  of the USER at that moment.
• QUALITY manager is a watch dog function that
  detects and supplies the quality aspect needed
  by the system at that moment.
• A method of implementing the quality manager
  function
• Remember that computer power is like cash. It can
  be used to buy
• Fault Tolerance
• Multi-Precision Computation
• Support or Take Over Failed Computing Functions

Use a high-performance computing function whose
excess computing power can be used for quality
manager functions that can invoke and support
real-time quality needs.

• Achieving Quality

28
Projected Improvement in QUALITY
EFFORT
AUTOMATION
Industry Growth
Time

• Service Industry Growing Very Rapidly (80 of
  Workers in US)
• Service Industrys Growth can be Traced to
  Increased Automation and Decreasing
  Labor-Intensive (manual) Functions
• Increased Automation means better QUALITY of
  Products and Services

• Evolution of Quality to a User Perspective

29
QUALITY Pyramid
ENTERPRISE
SERVICES
Increasing Complexity
Quality Input
PRODUCTS
PROCESSESS
ACTIVITIES

• Evolution of Quality to a User Perspective

30
Customer Priorities
Legacy Dependency - Afraid to Change
Make it Work
Failure No Option
Part of Lifestyle
A
B
C
D
E
Works the First Time All the Time
Work Horse
Make it Last
Legacy
Users Dependency
Usage History and QUALITY is Assessed
Customer is Prepared to Change
Customer is Reluctant to Change
A
B
C
D
E
Reliability isTaken for Granted
Maintenanceand Cost are No Issue
Afraid of Anything New Not Similar to the
One Used
Usage Life Cycle
C
D
E
Make it Last Forever
Demand

• Changes in Quality

• Evolution of Quality to a User Perspective

31
Evolution of QUALITY Consciousness and Measures
(HARDWARE, SOFTWARE, INFORMATION TECHNOLGY, AND
SERVICE)

• A
• B
• C
• D
• E

INDUSTRIAL REVOLUTION Assembly Line and Mass
Production, Productivity Consciousness - Time
studies and ELI Whitneys Standards and
InterchangeableParts, Fords Assembly Line
(Productivity Emphasis) Early Operations Research
ProductivityPerformance
Machines can be produced rapidly, and they are
complex -need for improved reliability. EXAMPLE
Steam Engines, Automobiles, Airplanes
Reliability
Statistical Quality Control- Edward
Deming,Shewhert
SQC
Man is not a machine. Quality can be improved by
focusing on human behavior and industrial
psychology. Rise of industrial unions, human
factors, and ergonomic emphasis improve quality
and productivity by creating better working
environments. ( Crosby)
BehavioralConsiderations
Total QualityManagementand Control
TQM C J. Juran Trilogy of Quality
Management, Most Popular and Important

• Evolution of Quality to a User Perspective

32
Evolution of QUALITY Consciousness and Measures
(cont.)
F G H I J K
SIX SIGMA QualityInitiatives
GE and Motorola - Moved from mostly mechanical
parts to electronic parts and assemblies - Jack
Welch of GE
SoftwareQuality Initiatives and Methodologies
Structured Programming, High-level languages,
object-oriented design, and implementation
languages
IBM Clean Room Methods
Structured Programming and Statistical Testing
Harlan Mills, Mike Fagan Inspection Procedures
Watts Humphrey, Bill Curtis
CMM of SEI
Initiative of DARPA for Communications Networks
QoS
To Be Developed
IT MM

• Evolution of Quality to a User Perspective

33
QUALITY Concerns Change with

• User Needs
• Technology Evolution
• Product and Service Sophistication
• EXAMPLE

HARDWARE
HARDWARESOFTWARE
HARDWARESOFTWARE COMMUNICATION
EMBEDDEDDISTRIBUTEDSYSTEMS

• Evolution of Quality to a User Perspective

34
QUALITY Concerns with Change
HIGH
Degradation of QUALITY Due to Change
LOW
LOW
HIGH
Complexity of Change

• Evolution of Quality to a User Perspective

35
Service Triplets

• PROCESS PRODUCT SERVICE
• Current Software Quality FADs Emphasize Process
  Aspects Only
• CMM Emphasizes Continuous Improvement in Software
  Development Processes (Well-Defined Activities)
• Specification Process Software Product
• CMM Emphasizes Process Maturity

• Evolution of Quality to a User Perspective

36
Change of Customer QUALITY Priorities with
Product (Software) Life Cycle
Make it Work (Works the First Time and All the
Time)
Users Dependencyon Product
Users Expectation (Product WorkHorse Ethic)
Builtto Last
User Afraidto Change LegacyMentality
PRODUCT USAGE LIFE CYCLE

• Customer Expectation Increasing Dependency
  on the Product
• The Ultimate Milestone of QUALITY

• Evolution of Quality to a User Perspective

37
QUALITY Customer Priorities
(EVOLUTION OF QUALITY CONSCIOUSNESS)

• Usage Life Cycle How QUALITY is assessed by the
  customer

A
B
C
D
E
Make it Work- Works the First Time EVERYTIME
Users Dependency
Work HorseEthic
Builtto Last
User Afraidto Change, LegacyMentality
Reliability Reliability and Maintainability
Failure No Option Failure No Option Refining,
Enhancing User Preferred Features
A
B
C
D
E

• Evolution of Quality to a User Perspective

38
Quality of Databases

• Databases are used widely and extensively in all
  kinds of enterprises and their use will only
  increase.
• But what is the view of database quality?
• From a database manager point of view, database
  quality is terms of database size, transaction
  speed, data reliability, scalability and data
  integrity.
• However, from a software designer point of view,
  quality may mean the maintainability, code
  reliability, and software architecture
  attributes.
• However, from the enterprise point of view,
  quality may mean the contribution of a database
  to the enterprise mission.

• Evolution of Quality to a User Perspective

39
Quality from a Higher Level Perspective

• We need to think in terms of a high level
  perspective from the user point of view rather
  than a design or process points of view
• Design view such as redundancy for fault
  tolerance, buffers for scalability.
• Process views such as following certain
  techniques, work flow, or using specific kinds of
  tools such as configuration control.
• Furthermore, we should not think in terms of
  system requirements only, we need to think about
  the overall mission of the enterprise, operation,
  enterprise capabilities, and enterprise evolution
  point of view.

• Evolution of Quality to a User Perspective

40
Enterprise Evolution

• Technology is changing rapidly at the speed of
  the Internet.
• Enterprise mission also changes, but at a much
  slower speed.
• Enterprise capability also needs to change with
  knowledge and technology changes .
• This is the trade-off between budget growths at
  X per year and Moores law IT technical
  capability grows at 67 per year.
• An orderly transition of enterprise capabilities,
  while taking advantage of technology change, is
  badly needed.
• This also assumes that an enterprise has a sound
  strategy to know what is needed and how to
  incorporate new technologies and new knowledge
  growth plans into the existing infrastructure and
  superstucture.

• Evolution of Quality to a User Perspective

41
Summary

• QUALITY - particularly software quality - has to
  be more adaptive
• Products and processes grow with their
  application and human environments. Quality
  criteria have to change with the situation,
  circumstance, and the environment. These are
  dynamic characteristics. Therefore, quality has
  to be dynamic and adaptive.
• Quality aspects change with application and
  product life cycles. Software is the right place
  to put quality adoption and evolution.
• The new IT environment will be multidisciplinary,
  fast changing with rapid diversification and
  globalization.
• THE QUALITY JOURNEY CONTINUES, BE PREPARED

• Summary

42
LEADERSHIP

• The position or office of a leader
• Capacity or ability to lead
• A continuum of good decisions
• Provide direction
• To influence

43
HOPE

• Michelangelo lived a few days shy of 89 years,
  still sculpting, painting, writing, and designing
  in an age when 90 was about 60 years beyond
  normal life expectancy was speaking to this idea
  of having high hopes and aims in this quotation
  (next slide) of his.
• I recall being in front of the statue of David
  in Florence and being transfixed. The size, the
  majesty, the spirit that seemed to jump right out
  of the marble was Michelangelo saying to all of
  us, Aim high.
• High aim indeed. Look at the at the Sistine
  Chapel where Michelangelo painted the ceiling by
  lying on his back working every day for 4 years.
  Lesser artists considered the task impossible,
  yet Michelangelo took it on and many more and
  aimed high

44
HOPE

• The greater danger for most of us
• is not that our aim is too high
• and we miss it,
• but that it is too low
• and we reach it.
• Michelangelo
• (1475-1564)

45
HOPE cont.

• No one knows enough to be a pessimist!!!
• Michelangelos advice is just as applicable today
  as it was almost 500 years ago. Never listen to
  those who try to influence you with their
  pessimism. Follow these guidelines
• Refuse to listen to or internalize the
  proclamations of those who point to your
  limitations. You must always remember
• when you argue for your limitations, the only
  thing you get are limitations
• imagine Michelangelo telling you that you can
  create the quality you desire, and the great
  danger is not in having too much hope, but in
  reaching what you have perceived as hopeless