International Institute of Business Analysis Software Testing

1
International Institute of Business
AnalysisSoftware Testing

• Vision The world's leading association for
• Business Analysis professionals
• Mission To develop and maintain standards for
  the practice of business analysis and for the
  certification of its practitioners

2
Agenda

• Overview of Testing
• Challenges of Testing
• Goals and Objectives
• Test Maturity / Process
• Key Indicators
• Requirements Impact
• Benefits
• Technology Considerations

3
Industry Report Card

• Project performance research groups, such as the
  Standish Group via their CHAOS Report, have
  reported through the years
• Over half of implemented software projects dont
  meet end user expectations when initially
  implemented.
• Many projects are cancelled before implementation
  after, at times, considerable expenditure of
  resources.
• Rework is still the main avenue projects use to
  achieve software user acceptance.

4
The Balancing ActA Balance Of Features And
Constraints
5
Typical Problems for Software Testing

• Wide net of Responsibilities Testing management
  and staff is involved in testing all aspects of
  software development as well as process
  improvement across all SDLC disciplines.
  Business and IT organizations have a hard time
  understand this wide coverage and defining their
  needs.
• Reactionary Organization request Testing (QC)
  a reaction to existing deliverables.
• Under Budget Organizations assume others on
  team can pick up QC tasks
• Identification of Accountability Testing often
  blamed for Production problems.
• Difficult to show value Difficult for
  organizations to understand and measure value of
  QA\QC.
• Good formal training is rare Many QA\QC
  Personnel have learned on the job

6
Testing Methods Types
7
The V-Model for Software Development
User Acceptance and System Testing Does the
software do what it is supposed to do?
user requirements
,acceptance testplanning
verified by
acceptance testing
verified by
functional requirements
,system test planning
system testing
verified by
,integration test planning
architecture design
integration testing
verified by
detailed design
,unit test planning
unit testing
Unit Testing and Integration Testing Does it
work?
coding
8
Testing Definitions

• Acceptance Testing
• Acceptance testing is black-box testing
  performed on a system prior to its delivery. In
  some engineering sub-disciplines, it is known as
  functional testing, black-box testing, release
  acceptance, QA testing, application testing,
  confidence testing, final testing, validation
  testing, usability testing, or factory acceptance
  testing.
• System Testing
• System testing of software or hardware is
  testing conducted on a complete, integrated
  system to evaluate the system's compliance with
  its specified requirements. System testing falls
  within the scope of black box testing, and as
  such, should require no knowledge of the inner
  design of the code or logic.
• Integration Testing
• Integration testing (sometimes called
  Integration and Testing, abbreviated IT) is the
  phase of software testing in which individual
  software modules are combined and tested as a
  group. It follows unit testing and precedes
  system testing. Integration testing takes as its
  input modules that have been unit tested, groups
  them in larger aggregates, applies tests defined
  in an integration test plan to those aggregates,
  and delivers as its output the integrated system
  ready for system testing.

9
Testing Definitions, Cont.

• Unit Testing
• Unit testing is a test that validates that
  individual units of source code are working
  properly. A unit is the smallest testable part of
  an application. In procedural programming a unit
  may be an individual program, function,
  procedure, etc., while in object-oriented
  programming, the smallest unit is a method, which
  may belong to a base/super class, abstract class
  or derived/child class.
• Performance Testing
• Performance Testing covers a broad range of
  engineering or functional evaluations where a
  material, product, or system is not specified by
  detailed material or component specifications
  Rather, emphasis is on the final measurable
  performance characteristics.
• For instance, in the computer industry,
  performance testing is used to determine the
  speed or effectiveness of a computer, network,
  software program or device. This process can
  involve quantitative tests done in a lab, such as
  measuring the response time or the number of MIPS
  (millions of instructions per second) at which a
  system functions. Qualitative attributes such as
  reliability, scalability and interoperability may
  also be evaluated.

10
Testing Skills
11
Skills Required
Business Domain Knowledge
Acceptance Testing
System Testing
Integration Testing
Skills
Unit / Performance Testing
Test Infrastructure Support Test Lead Management,
Test Automation, Process Design, Metrics
Reporting

Technical Skills
12
Domain Knowledge
Domain Knowledge
Strategic
Software Quality Assurance
QA Management
Software Quality Control Software Testing
Quality Assurance Software Testing Quality
Control
QC Leads
QC Leads QC Analysts
Tactical
13
QA / QC Disciplines
QA\QC has four disciplines within a testing
organization. Within each discipline there is a
continuum of roles. Some individuals have
skills in various roles and across multiple
disciplines.

• Testing Disciplines
• Manual Testing
• Automated Testing
• Data Analysis\Testing
• Performance Testing

• Testing Roles
• Tester
• Test Analyst
• Team Lead
• Manager

14
Skills Functions

• Team Lead
• Develop Implement Project Specific Test
  Strategy
• Implement Best Practices
• Plan and Manage Test Effort
• Mentoring
• Manager
• Assess and Implement QA/QC Process Improvement
• Design Test Automation Center of Excellence
• Design Performance Center of Excellence

• Tester
• Test Execution
• Defect Reporting
• Analysts
• Build and Execute Test Cases
• Technical Testing
• Defect Reporting
• Data Mapping
• Systems Integration Testing
• Build Run Test Automation
• Build and Run Performance Tests

15
Quality Control to Quality Assurance
16
Moving From QC to QA
17
What Are The Pieces?
Software Testing Quality Assurance Validation
with structure, metrics, moving upstream in the
development lifecycle with requirements reviews,
walkthroughs, etc.
Software Quality Control Validation, inspection
of Products against defined requirements. Lacks
formal process and controls.
Software Testing Quality Control Validation, with
structure, formal process, procedures, etc.)
Software Quality Assurance Focused on the overall
Project Management, Development Processes, i.e.
Configuration Management, Requirements
Management, Source Code/Version Control, etc.
18
QA/QC Definitions

• Software Quality Assurance
• Monitoring of the software development to ensure
  the software process and product conform to
  standards and procedures. (CMM)
• Software Quality Control
• Features and characteristics of a product,
  process or service that bear on its ability to
  satisfy stated or implied needs. (ISO8402)
• Software Testing Quality Assurance
• The process of verifying quality control
  standards and working to improve the processes
  that are used for producing the application and
  its components, infrastructure and content.
• Software Testing Quality Control
• The directed use of testing to measure the
  achievement of a specified
• standard the measurement and enforcement of a
  benchmark that says "the product must be this
  good."

19
How To Move Towards Quality Assurance
Software Testing Quality Assurance (Testing
Processes and Standards incorporated in
Development Lifecycle.)
Software Quality Control (QC Used to Validate
Product Features Against Requirements.)
Software Quality Assurance (Full QA of Project
Lifecycle.)
Software Testing Quality Control (The use of
Testing Processes to Validate Product Features
Against Requirements.
Validation Metrics provided to QA from QC
Validation Metrics provided to QA from QC
Validation Metrics provided to QA from QC
Validation Metrics provided to QA from QC
QA Uses Metrics and Works with Project Teams to
Improve Processes
20
Development Process Optimization
21
What Is the Outcome?
Software Quality Control
Software Testing Quality Control
Software Testing Quality Assurance
Software Quality Assurance
Validation Metrics
Verification Metrics
Validation Metrics
Verification Metrics
GOAL!
22
Goal Moving Up to the Right
SW Quality
Process Maturity
23
What are the metrics?

• Defect efficiency removal rating
• Number of software requirements containing
  defects
• Where in the development lifecycle defects are
  found
• Number of outstanding defects by severity
• Structured release process

24
Example Defect Removal Velocity Report
25
Process Maturity
26
Life Cycle Maturity Model
27
Process Maturity

• Level 1 Software Quality Control
• Validation, inspection of products against
  defined requirements. Lacks formal process and
  controls.
• Level 2 Software Testing Quality Control
• Validation, with structure, formal process,
  procedures, etc.
• Level 3 Software Testing Quality Assurance
• Validation with structure, metrics, moving
  upstream in the development lifecycle with
  requirements reviews, walkthroughs, etc.
• Level 4 Software Quality Management
• Focused on the overall Project Management,
  Development Processes, i.e. Configuration
  Change Management, Requirements Management, etc.

28
Characteristics of a Mature Organization

• Process
• Processes are defined, documented and used
• Deliverables are identified and produced
• Plans communicated and monitored
• Product and process are measured
• Quality, cost, and schedule are predictable
• People
• Roles and responsibilities are clear
• Technology
• Technology is planned, incorporated and used
  effectively

Continuous improvement is a way of life
Source SEI Capability Maturity Model (CMM)
29
Process Overview

• Process Highlights
• Process is tailored to Corporate SDLC
• Process is generic enough to adapt to project
  team practices
• QA\QC Deliverables are aligned with Project
  Deliverables
• Roles and Responsibilities are defined
• Key Performance Indicators are identified
• Processes have minimum standards to ensure
  portability across the enterprise

SAMPLE
30
From Needs and Preferences to Code and Tests
31
From Needs and Preferences to Code and Tests
32
ReworkReason

• 30-50 of a typical products development time is
  spent in correcting errors not discovered in the
  beginning of the product
• Software Project Survival Guide, Steve McConnell
• Most rework is related to inadequate,
  inconsistent, imprecise requirements and changed
  requirements
• Fixing a defect late in the development lifecycle
  can cost 30-1000 times more

33
Collaborative Technology Enables Efficiency
34
Integrate People, Process and Technology

• Processes
• Customer Needs Management
• Requirements Engineering
• Change Management
• Quality Management
• Test Management
• Project Management
• Issue Management
• Risk Management
• Model Management

• Tools
• Customer Needs Management
• Model Management
• Requirements Management
• Test Case Generation
• Test Management
• Project Management
• Issue Management
• Risk Management

• Techniques
• Needs Analysis
• Use Case Identification
• Use Case Diagramming
• Use Case Analysis
• Prototyping
• Requirements Decomposition
• Process Modeling
• Data Modeling

• Problem Analysis
• Feasibility Assessment
• Risk Analysis
• Risk Mitigation
• Trade Off Analysis
• Stakeholder Profiling
• Test Case Analysis
• Test Case Design
• Test Case Execution

35
IIBA International Business Development

• Contact Information
• Don McElwee
• 817.915.9600
• don.mcelwee_at_orasi.com
• www.orasi.com