System Testing Using the Rational Unified Process as a Guideline

1
System Testing Using the Rational Unified Process
as a Guideline

• Dr Tamar ZemelSoftware and Systems Engineering
• Arik Rokach Eli ShalhonPrinting Workflow
  SolutionsCreoScitex

2

• The transition program
• The RUP Process
• The Standard Processes
• The System Testing Standard Process
• Results
• Plans for the future

3
The Printing Workflow Solutions SPIRE Product Line

• Develop HW and SW to improve printing quality and
  rate
• A number of systems being developed in parallel
  using an existing common core
• Incremental development
• Small difference between systems
• A number of engineering groups work concurrently

4
The Transition Program

• Improve Development Process using the CMM as a
  guide line
• Involve all development groups SW, HW,
  Marketing and System Testing

5
The Rational Unified Process
6
Rational Unified Process - Characteristic

• Object-Oriented driven
• Iterative and Incremental
• Architecture and Use case driven
• Component based development

7
RUP

• A process that consist of a number of cycles
• Every cycle ends with a product release to the
  customer

8
The phases of a RUP Cycle

• Phases
• Inception - Definition of the system capabilities
• Elaboration - Design of the system architecture
• Construction - Building the system
• Transition - Deployment of the system
• In every phase a number of iterations

9
RUP- The Distribution of the Main Activities
Inception
Elaboration
Construction
Transition
Requirement Analysis Design Implementation Te
st
10
RUPS Models

• Requirements - Use-case Model(actors and use
  cases)
• Analysis - Analysis Model (Classifiers and use
  case realizations) - Conceptual
• Design - Design Model - Blue print for
  implementationdeployment Model
• Implementation - Implementation Model (system
  components)
• Testing - Testing Model (test cases)

11
The Standard Processes
12
Standard Process

• Policy
• Roles
• Method
• Phases
• Training

13
Phase Definition

• Activities
• Entry criteria
• Applicable infrastructure
• Work products
• Measurements
• Verification and reviews
• Exit criteria

14
The Standard Process

• Product Development Standard Process
• Sub-processes
• Supporting Activities

15
Product Development Standard Process
Management
A-Level Plans
Inception Feature list identification A-Level
Planning
Feature List
B-Level Plans
Customer Requirements
Elaboration Feature Specification GUI
Specification System Design HW Architecture
Design B-Level Planning
Feature Spec
System Design
Construction HW CI Construction SW CI
Construction System Testing Design Beta-site
Testing Design System Integration System Testing
Product
Transition Beta-Site Testing Product Release
Configuration Management Quality
Assurance System Engineering
Quality Assurance Configuration
Management Configuration Management
16
Product Development Standard Process The
Sub-Processes

• Product Marketing
• SW
• HW
• System Testing
• Every Sub-process defines the roles and
  activities of a specific group

17
Product Development Standard Process Supporting
Activities

• Program Management
• Configuration Management
• Quality Assurance
• Measurements
• Support activities are applicable to all groups
  involved in the development process

18
Program Organization
System Testing Group Leader
19
The System Testing Standard Process
20
The System Testing Standard Process Policy

• The system testing group is involved in the
  program from its inception phase and participates
  in technical and managerial reviews
• System testing activities are planned based on
  the standard process
• Tests cases and plans are feature oriented

21
The System Testing Standard Process Policy
(Continued)

• Plans and tests are reviewed
• The system testing documentation, inputs files,
  output files and SW are under configuration
  management.
• System testing plans and test cases are changed
  when requirements are changed

22
System Testing Standard Process
Management
A-Level Plans
Inception A-Level Planning
Feature List
B-Level Plans
Customer Requirements
Elaboration B-Level Planning
Construction System Testing Design Beta-Site
testing Design System Testing for a Release
Product
Transition Beta-Site Testing Product Release
System Design
Feature Spec
Configuration Management Quality
Assurance System Engineering
23
The System Testing Standard Process Roles -
System Testing Leader

• System Requirement review - participant
• A-Level system testing plan
• Feature specification and Graphical User
  Interface (GUI) definition reviews - participant
• B-Level system testing plan
• Re-planning
• System testing task progress meetings
• Updating system testing progress

• Managing system testing risks
• Reporting system testing status
• System testing design
• System testing design review
• Identifying required test cases for a release
• Approving testing environment
• Reporting faults detected
• Analysis of system testing results
• System testing result review
• Configuration management

24
The System Testing Standard Process Roles -
System Testing Engineer

• System testing test case detailed design
• System testing
• System testing results recording
• Report System testing task status
• Reporting system faults detected during testing

25
The System Testing Standard Process
Phase/Activities
26
The System Testing Standard Process Inception
Phase

• Evaluate testability of features
• A-level planning

27
The System Testing Standard Process Inception
Phase - A-Level Planning

• Purpose - Preliminary planning
• Input - Program A Level Plan
• Activities
• Estimation of effort and schedule
• Identification of risks
• Identification of testing facilities and
  equipment
• Output - A-Level system testing plan (STP)
• Reviews - A-Level Program Management Review

28
The System Testing Standard Process Elaboration
Phase

• Evaluate testability of features specified in
  feature specs and GUI definitions
• B-Level Planning

29
The System Testing Standard Process Elaboration
Phase - B-Level Planning

• Purpose - Detailed planning of testing process
• Inputs
• A-level plans
• Feature specs
• System design
• GUI definitions

30
The System Testing Standard Process Elaboration
Phase - B-Level Planning

• Activities
• Define System testing content
• Estimate effort and size
• Define process
• Define schedule
• Risk management
• Facilities
• Review - B-Level PMR

31
The System Testing Standard Process The
Construction Phase

• System testing design
• System testing
• Beta-site testing design
• If required

32
The System Testing Standard Process Construction
Phase - System Testing Design

• Purpose - Design test case before testing begins
  based on system requirements
• Input
• Feature specifications and GUI definitions
• System design
• B Level STP
• Existing system testing documentation

33
The System Testing Standard Process Construction
Phase - System Testing Design

• Activities
• Identify test cases
• Write test procedures
• Evaluate coverage
• Define testing sequence
• Define regression testing
• Output - System Testing Design document
• Reviews - System testing design review

34
The System Testing Standard Process Construction
Phase - System Testing for a Release

• Purpose - Test a release of the system according
  to the STD
• Input
• RD release note
• STD
• System CD

35
The System Testing Standard Process Construction
Phase - System Testing for a Release

• Activities
• Identify required tests for the release
• Test and record results
• Evaluate results
• Compute testing metrics
• Output - System testing results report
• Review -
• System testing progress
• System testing results review

36
The System Testing Standard Process Construction
Phase - Beta-Site Testing Design

• Purpose - Design the test cases for the beta-site
  testing and their distribution between testing
  sites
• Input
• System Testing
• Feature specifications and GUI definitions
• System design
• Existing system testing documentation

37
The System Testing Standard Process Construction
Phase - System Testing Design

• Activities
• Identify test cases
• Specify test cases
• Map test cases to the sites
• Output - System Testing Design document
• Reviews - Beta-Site Testing Design Review

38
The System Testing Standard Process Training

• System Testing Leader
• Management
• Testing
• System Testing Engineer
• testing
• STSP
• Other
• Testing orientation

39
The System Testing Standard Process Measurements

• Number of test cases
• Number of system testing cycles
• Number of changes in test cases
• Number of faults in test cases
• Number of faults detected
• Number of crashes
• Rate of fault detection
• Testing coverage

40
The System Testing Standard Process Tailoring
Guidelines

• A new product with one release
• A new product with several releases
• A new product within a product line
• An upgrade of an existing product

41
Infrastructure to Support System Testing

• System testing procedure
• A-Level and B-level system testing plan templates
• System Testing Description template
• Test Director
• Automatic testing (WinRunner)
• Bug Tracking tool

42
Management of the System Testing Standard Process

• Monitoring
• Task meetings
• Professional - status meetings
• Projectile - progress meetings
• Control
• Program Management Review

43
Quality Assurance for the System Testing Standard
Process

• Participate in planning
• Support system testing design
• Review work products
• Audit process

44
Configuration Management for System Testing

• All work products are placed under configuration
  control
• Test scripts and simulators are placed under
  configuration control

45
Results - Achievements

• Improved group morale
• System testing engineers are involved in the
  program from the beginning
• The role of the STEs is defined
• The required activities are defined
• Templates improve efficiency of testing design

46
Results - Achievements (continued)

• Improve testing capabilities
• Tests are planned and documented in advance
• Tests fits the features
• Test case undergo peer reviews
• System testing documentation is under
  configuration control

47
Results - Difficulties

• Existing set of tests do not fit new format
• No time to redesign existing testware
• Short time for testing cause frustrations

48
Plans for the Future

• Improve existing set of tests
• Identify and use regression tests
• Increase automatic testing portion
• Apply system testing measurements