NITEL Testing Introduction

1
NITEL Testing Introduction

• xyz

2
NITEL Testing Strategy

• xyz

3
NITEL Testing Strategy

• to ensure

4
NITEL Testing Strategy

• to ensure
• EXCELLENT QUALITY

5

• xyz

6
My Name is

• xyz

7
My Name is

• Dr. Wolfgang J. Schneider

8
My Name is

• Dr. Wolfgang Schneider

9
My Name is

• Dr. Wolfgang Schneider

10
My Name is

• Wolfgang Schneider

11
My Name is

• Wolfgang Schneider

12
My Name is

• WOLFGANG Schneider

13
My Name is WOLFGANG

• xyz

14
My Name is WOLFGANG

• and I am very proud to work for

15
My Name is WOLFGANG

• and I am very proud to work for

16
Wolfgang from Austria

• I am from
• AUSTRIA

17
Wolfgang from Austria

• I am from
• AUSTRIA

18
Wolfgang from Austria

• I am from
• AUSTRIA

19
Wolfgang from Austria

• AUSTRIA is a small country...

20
Wolfgang from Austria

• AUSTRIA is a small country...
• Only 8 million people

21
Wolfgang from Austria

• ...quite a way to go...

22
My Task is to help
23
My Task is to help
24
My Task is to help
25
My Task is to help

• NITEL
• to establish a
• TEST TEAM
• to be managed by NITEL people

26
My Task is to help

• NITEL
• to establish a
• TEST TEAM
• to be managed by NITEL people

27
My Task is to help

• NITEL
• to establish a
• TEST TEAM
• to be managed by NITEL people

28
My Task is to help

• NITEL
• to establish a
• TEST TEAM
• to be managed by NITEL people

29
Together

• we will test
• a new
• BILLING SYSTEM

30
Together

• we will test
• a new
• BILLING SYSTEM

31
Together

• we will test
• a new
• BILLING SYSTEM

32
Together

• we will test
• a new
• BILLING SYSTEM

33
Together

• we will test
• a new
• BILLING SYSTEM

34
Together

• we will test
• a new
• BILLING SYSTEM
• NIIMA

35
The Name of our ProjectisNIIMA

• xyz

36
The Name of our ProjectisNIIMA

• Nitel Interconnect, ITU, Mediation and Activation
  xyz

37
The Name of our ProjectisNIIMA

• Nitel Interconnect, ITU, Mediation and Activation
  xyz

38
The Name of our ProjectisNIIMA

• Nitel Interconnect, ITU, Mediation and Activation
  xyz

39
The Name of our ProjectisNIIMA

• Nitel Interconnect, ITU, Mediation and Activation
  xyz

40
The Name of our ProjectisNIIMA

• Nitel Interconnect, ITU, Mediation and Activation
  xyz

41
The Name of our ProjectisNIIMA

• Nitel Interconnect, ITU, Mediation and Activation
  xyz

42
The Name of our ProjectisNIIMA

• Nitel Interconnect, ITU, Mediation and Activation
  xyz

43
Our new System

• will be provided
• by
• INTEC

44
Our new System

• will be provided
• by
• INTEC

45
Our new System

• will be provided
• by
• INTEC

46
Our new System

• will be provided
• by
• INTEC

47
Our new System

• will be provided
• by
• INTEC

48
(No Transcript)
49
Our job will be

• to test the new system.
• We will not test against
• INTEC
• We will work together

50
Our job will be

• to test the new system.
• We will not test against
• INTEC
• We will work together

51
Our job will be

• to test the new system.
• We will NOT test AGAINST
• INTEC
• We will work together

52
Our job will be

• to test the new system.
• We will NOT test AGAINST
• INTEC
• We will work together

53
Our job will be

• to test the new system.
• We will NOT test AGAINST
• INTEC
• We will work together

54
Phases of Testing
55
Phases of Testing
56
Phases of Testing
57
Phases of Testing
58
Phases of Testing
59
Phases of Testing
60
Preparation Phase
61
Preparation Phase

• Evaluation of Requirements

62
REQUIREMENTS

• Compliance List
• Technical Knowledge
• Requirement

63
REQUIREMENTS

• Compliance List
• Technical Knowledge
• Requirement

64
REQUIREMENTS

• Compliance List
• Technical Knowledge
• Requirement

65
REQUIREMENTS

• Compliance List
• Technical Knowledge
• Requirement

66
Preparation Phase

• Evaluation of Requirements
• Establishing Test Methodology

67
Preparation Phase

• Evaluation of Requirements
• Establishing Test Methodology
• Development of Test Scenarios

68
Preparation Phase

• Evaluation of Requirements
• Establishing Test Methodology
• Development of Test Scenarios
• Definition of Test Cases

69
Preparation Phase

• Evaluation of Requirements
• Establishing Test Methodology
• Development of Test Scenarios
• Definition of Test Cases
• Establishing Test Data Requirements

70
Preparation Phase

• Evaluation of Requirements
• Establishing Test Methodology
• Development of Test Scenarios
• Definition of Test Cases
• Establishing Test Data Requirements

71
Preparation Phase

• The workload of this phase is depending on

72
Preparation Phase

• The workload of this phase is depending on
• Complexity of the Requirements

73
Preparation Phase

• The workload of this phase is depending on
• Complexity of the Requirements
• Quality Expectations

74
Preparation Phase

• The workload of this phase is depending on
• Complexity of the Requirements
• Quality Expectations
• Test Methodology Standards

75
Preparation Phase

• The workload of this phase is depending on
• Complexity of the Requirements
• Quality Expectations
• Test Methodology Standards
• Data Volume should not be a crucial issue

76
Pre-Production Testing Phase
77
Pre-Production Testing Phase

• Functional Test

78
Pre-Production Testing Phase

• Functional Test
• Acceptance Test

79
Pre-Production Testing Phase

• Functional Test
• Acceptance Test
• Integration Test

80
Pre-Production Testing Phase

• Functional Test
• Acceptance Test
• Integration Test
• Volume Test

81
Pre-Production Testing Phase

• Functional Test
• Acceptance Test
• Integration Test
• Volume Test

82
Pre-Production Testing Phase
83
Pre-Production Testing Phase

• Running the Test Cases

84
Pre-Production Testing Phase

• Running the Test Cases
• Evaluation of Test Results

85
Pre-Production Testing Phase

• Running the Test Cases
• Evaluation of Test Results
• Reporting of Errors

86
Pre-Production Testing Phase

• Running the Test Cases
• Evaluation of Test Results
• Reporting of Errors
• Review of Requirements

87
Pre-Production Testing Phase

• Running the Test Cases
• Evaluation of Test Results
• Reporting of Errors
• Review of Requirements
• Review of Test Cases

88
Pre-Production Testing Phase

• Running the Test Cases
• Evaluation of Test Results
• Reporting of Errors
• Review of Requirements
• Review of Test Cases
• Development of Work Arounds

89
Pre-Production Testing Phase

• Running the Test Cases
• Evaluation of Test Results
• Reporting of Errors
• Review of Requirements
• Review of Test Cases
• Development of Work Arounds

90
Pre-Production Testing Phase

• The workload of this phase is depending on

91
Pre-Production Testing Phase

• The workload of this phase is depending on
• Quality of Preparation

92
Pre-Production Testing Phase

• The workload of this phase is depending on
• Quality of Preparation
• Complexity and Number of Test Cases

93
Pre-Production Testing Phase

• The workload of this phase is depending on
• Quality of Preparation
• Complexity and Number of Test Cases
• Quality of the Software

94
Pre-Production Testing Phase

• The workload of this phase is depending on
• Quality of Preparation
• Complexity and Number of Test Cases
• Quality of the Software
• Test Data Volume

95
Pre-Production Testing Phase

• The workload of this phase is depending on
• Quality of Preparation
• Complexity and Number of Test Cases
• Quality of the Software
• Test Data Volume

96
Production Phase
97
Production Phase

• Administration of Change Requests

98
Production Phase

• Administration of Change Requests
• Release Testing

99
Production Phase

• Administration of Change Requests
• Release Testing
• Testing of Hotfixes

100
Production Phase

• Administration of Change Requests
• Release Testing
• Testing of Hotfixes
• Development of Quality Management

101
Production Phase

• Administration of Change Requests
• Release Testing
• Testing of Hotfixes
• Development of Quality Management

102
Production Phase

• The workload of this phase is depending on

103
Production Phase

• The workload of this phase is depending on
• Quality / Stability of the System

104
Production Phase

• The workload of this phase is depending on
• Quality / Stability of the System
• Customer Expectations

105
Production Phase

• The workload of this phase is depending on
• Quality / Stability of the System
• Customer Expectations
• Release Schedule

106
Production Phase

• The workload of this phase is depending on
• Quality / Stability of the System
• Customer Expectations
• Release Schedule

107
Production Phase

• How many
• Test Environments
• will I need
• ???

108
Production Phase

• Of course, a Production Environment is needed.

109
Production Phase

• Of course, a Production Environment is needed.
• It has to be completely separated from the
• Test Environments

110
Production Phase

• Of course, a Production Environment is needed.
• It has to be completely separated from the
• Test Environments

111
Production Phase

• You will need a Test Environment for the
• next Release

112
Production Phase

• V 1.0
• Production Release

113
Production Phase

• V 1.0 V 1.1
• Production Release New Release

114
Production Phase

• But what will happen if problems have to be
  fixed in the Production Release ?

115
Production Phase

• But what will happen if problems have to be
  fixed in the Production Release ?
• Then we have to test the fixes...

116
Production Phase

• But what will happen if problems have to be
  fixed in the Production Release ?
• Then we have to test the fixes...
• In a separate Test Environment

117
Production Phase

• V 1.0 V 1.1 V 1.0
• Production New Production
  Release Release Release

118
Production Phase

• V 1.0 V 1.1 V 1.0
• Production New Production
  Release Release Release
• This is the MINIMAL Requirement !!!

119
Production Phase

• IMPLICATIONS
• We need 1 set of Test Environments per
  Version / Release
• We never must mix Components
• Reference Tables have to be tested, too

120
Production Phase

• IMPLICATIONS
• We need 1 set of Test Environments per
  Version / Release
• We never must mix Components
• Reference Tables have to be tested, too

121
Production Phase

• IMPLICATIONS
• We need 1 set of Test Environments per
  Version / Release
• We never must mix Components
• Reference Tables have to be tested, too

122
Production Phase

• IMPLICATIONS
• We need 1 set of Test Environments per
  Version / Release
• We never must mix Components
• Reference Tables have to be tested, too

123
Production Phase

• IMPLICATIONS
• We need 1 set of Test Environments per
  Version / Release
• We never must mix Components
• Reference Tables have to be tested, too

124
Methodology Experience

• Good Methodology and General Testing
• Experience can help to compensate for Lack of
• Special Knowledge (System / Business).
• Specific Testing Experience can help to
• compensate for Flaws in Methodology.
• Involving the User Departments too little or too
• late is causing a Chain of Change Requests.

125
Methodology Experience

• Good Methodology and General Testing
• Experience can help to compensate for Lack of
• Special Knowledge (System / Business).
• Specific Testing Experience can help to
• compensate for Flaws in Methodology.
• Involving the User Departments too little or too
• late is causing a Chain of Change Requests.

126
Methodology Experience

• Good Methodology and General Testing
• Experience can help to compensate for Lack of
• Special Knowledge (System / Business).
• Specific Testing Experience can help to
• compensate for Flaws in Methodology.
• Involving the User Departments too little or too
• late is causing a Chain of Change Requests.

127
Methodology Experience

• Good Methodology and General Testing
• Experience can help to compensate for Lack of
• Special Knowledge (System / Business).
• Specific Testing Experience can help to
• compensate for Flaws in Methodology.
• Involving the User Departments too little or too
• late is causing a Chain of Change Requests.

128
Methodology Experience

• Good Methodology and General Testing
• Experience can help to compensate for Lack of
• Special Knowledge (System / Business).
• Specific Testing Experience can help to
• compensate for Flaws in Methodology.
• Involving the User Departments too little or too
• late is causing a Chain of Change Requests.

129
xyz

• TEST DATA

130
Synthetic Data vs. Real Data

• What type of data to be used for
• Pre-Production Operational Testing
• Acceptance Testing
• ???

131
Synthetic Data vs. Real Data

• Lets have a look at
• Advantages
• Disadvantages

132
Synthetic Data vs. Real Data

• xyz

• xyz

133
Synthetic Data vs. Real Data

• Predictable Results

134
Synthetic Data vs. Real Data

• Predictable Results

• Surprising Results

135
Synthetic Data vs. Real Data

• Predictable Results
• Easy Evaluation

• Surprising Results

136
Synthetic Data vs. Real Data

• Predictable Results
• Easy Evaluation

• Surprising Results
• Time-Consuming

137
Synthetic Data vs. Real Data

• Predictable Results
• Easy Evaluation
• Formal Approach

• Surprising Results
• Time-Consuming

138
Synthetic Data vs. Real Data

• Predictable Results
• Easy Evaluation
• Formal Approach

• Surprising Results
• Time-Consuming
• Empiric Approach

139
Synthetic Data vs. Real Data

• Predictable Results
• Easy Evaluation
• Formal Approach
• Unrealistic

• Surprising Results
• Time-Consuming
• Empiric Approach

140
Synthetic Data vs. Real Data

• Predictable Results
• Easy Evaluation
• Formal Approach
• Unrealistic

• Surprising Results
• Time-Consuming
• Empiric Approach
• Pragmatic

141
Synthetic Data vs. Real Data

• Predictable Results
• Easy Evaluation
• Formal Approach
• Unrealistic
• Expectation-Oriented

• Surprising Results
• Time-Consuming
• Empiric Approach
• Pragmatic

142
Synthetic Data vs. Real Data

• Predictable Results
• Easy Evaluation
• Formal Approach
• Unrealistic
• Expectation-Oriented

• Surprising Results
• Time-Consuming
• Empiric Approach
• Pragmatic
• Result-Oriented

143
Synthetic Data vs. Real Data

• Predictable Results
• Easy Evaluation
• Formal Approach
• Unrealistic
• Expectation-Oriented

• Surprising Results
• Time-Consuming
• Empiric Approach
• Pragmatic
• Result-Oriented

144
Synthetic Data vs. Real Data

• Synthetic data is very useful for

145
Synthetic Data vs. Real Data

• Synthetic data is very useful for
• Functional Tests

146
Synthetic Data vs. Real Data

• Synthetic data is very useful for
• Functional Tests
• Testing Rules and Algorithms

147
Synthetic Data vs. Real Data

• Synthetic data is very useful for
• Functional Tests
• Testing Rules and Algorithms
• Alternate Flows

148
Synthetic Data vs. Real Data

• Synthetic data is very useful for
• Functional Tests
• Testing Rules and Algorithms
• Alternate Flows

149
Synthetic Data vs. Real Data

• Synthetic data is very useful for
• Functional Tests
• Testing Rules and Algorithms
• Alternate Flows
• but...

150
Synthetic Data vs. Real Data

• Synthetic data will not tell you anything
• about

151
Synthetic Data vs. Real Data

• Synthetic data will not tell you anything
• about
• Entries made by the average user

152
Synthetic Data vs. Real Data

• Synthetic data will not tell you anything
• about
• Entries made by the average user
• Unclean data Garbage

153
Synthetic Data vs. Real Data

• Synthetic data will not tell you anything
• about
• Entries made by the average user
• Unclean data Garbage
• Migrated data in general

154
Synthetic Data vs. Real Data

• Synthetic data will not tell you anything
• about
• Entries made by the average user
• Unclean data Garbage
• Migrated data in general

155
Real Data whats that ?

• xyz

156
Real Data whats that ?

• What we call Real Data will comply with
• one of the following requirements

157
Real Data whats that ?

• What we call Real Data will comply with
• one of the following requirements
• Entries from Migration (Conversion)

158
Real Data whats that ?

• What we call Real Data will comply with
• one of the following requirements
• Entries from Migration (Conversion)
• Entries by the On-Line System (GUI)

159
Real Data whats that ?

• What we call Real Data will comply with
• one of the following requirements
• Entries from Migration (Conversion)
• Entries by the On-Line System (GUI)
• Entries from External Sources

160
Real Data whats that ?

• What we call Real Data will comply with
• one of the following requirements
• Entries from Migration (Conversion)
• Entries by the On-Line System (GUI)
• Entries from External Sources
• Definitely not data entered directly into the DB

161
Real Synthetic Data
162
Real Synthetic Data

• Synthetic data can be considered real
• if it is entered through the gateways we
• just have defined.

163
Real Synthetic Data

• Synthetic data can be considered real
• if it is entered through the gateways we
• just have defined.
• This will improve the value of the tests
• in a significant way.

164
New Data whats that ?

• xyz

165
New Data whats that ?

• What we call New Data is everything entering
• the system in another way than Migration
• (Conversion) from the legacy system.

166
New Data whats that ?

• What we call New Data is everything entering
• the system in another way than Migration
• (Conversion) from the legacy system.
• In general, synthetic data will be new.

167
Migrated Data vs. New Data

• What type of data to be used for
• Pre-Production Operational Testing
• Acceptance Testing
• ???

168
Migrated Data - Development

• Assumptions for Cancellation Rates
• Low 0.25 per month
• High 2.50 per month

169
New Entries - Development

• Assumptions for Growth Rates
• Slow 10 5 - 4 3 2 - 2 per month
• Fast 20 15 - 10 5 - 4 3 2 - 2 per month

170
Migrated Data vs. New Data

• Even with a very high subscriber turnover,
• the percentage of migrated data will remain
• high for many months

171
Migrated Data vs. New Data

• For both
• Pre-Production Operational Testing
• Acceptance Testing
• migrated data should be
• definitely more than 50
• of the test population.....

172
Migrated Data vs. New Data

• ...but,
• this data is to be carefully selected to obtain
• a good Test Case Coverage

173
Migrated Data vs. New Data

• ...but,
• this data is to be carefully selected to obtain
• a good Test Case Coverage
• ...and, of course, there is an
• Exception of the Rule...

174
Migrated Data Call Details

• Exception of the Rule
• Call Details

175
Migrated Data Call Details

• Exception of the Rule
• Call Details
• Why ???

176
Migrated Data Call Details

• Migrated CDRs (Call Detail Records) will be used
• only once
• in the Life-Cycle of the system
• In the first Bill Run

177
Migrated Data Call Details

• Migrated CDRs (Call Detail Records) will be used
• only once
• in the Life-Cycle of the system
• In the first Bill Run

178
TEST DOCUMENTATION
179
TEST DOCUMENTATION

• Primary Documentation

180
TEST DOCUMENTATION

• Primary Documentation

• Primary Documentation is also known as

181
TEST DOCUMENTATION

• Primary Documentation

• Primary Documentation is also known as
• RAW Documentation

182
TEST DOCUMENTATION

• Primary Documentation

183
TEST DOCUMENTATION

• Primary Documentation
• Secondary Documentation

184
TEST DOCUMENTATION

• Primary Documentation
• Secondary Documentation

• Secondary Documentation is also known as

185
TEST DOCUMENTATION

• Primary Documentation
• Secondary Documentation

• Secondary Documentation is also known as
• EDITED Documentation

186
Primary Documentation

• Input Files
• Output Files
• Reports
• Screenshots
• Data Bases

187
Primary Documentation

• Input Files
• Output Files
• Reports
• Screenshots
• Data Bases

188
Primary Documentation

• Input Files
• Output Files
• Reports
• Screenshots
• Data Bases

189
Primary Documentation

• Input Files
• Output Files
• Reports
• Screenshots
• Data Bases

190
Primary Documentation

• Input Files
• Output Files
• Reports
• Screenshots
• Data Bases

191
Primary Documentation

• Input Files
• Output Files
• Reports
• Screenshots
• Data Bases

192
Primary Documentation

• Input Files
• Output Files
• Reports
• Screenshots
• Data Bases

193
Primary Documentation

• Input Files

194
Primary Documentation

• Input Files
• REPEATABILITY

195
Primary Documentation

• Input Files
• REPEATABILITY
• REGRESSION

196
Primary Documentation

• Input Files
• REPEATABILITY
• REGRESSION

197
Primary Documentation

• Output Files

198
Primary Documentation

• Output Files
• REVIEW

199
Primary Documentation

• Output Files
• REVIEW
• REGRESSION

200
Primary Documentation

• Output Files
• REVIEW
• REGRESSION
• Works only if Input has been saved

201
Primary Documentation

• Output Files
• REVIEW
• REGRESSION

202
Primary Documentation

• Reports

203
Primary Documentation

• Reports
• REVIEW

204
Primary Documentation

• Reports
• REVIEW
• READABILITY

205
Primary Documentation

• Reports
• REVIEW
• READABILITY

206
Primary Documentation

• Screenshots

207
Primary Documentation

• Screenshots
• GUI

208
Primary Documentation

• Screenshots
• GUI
• ERROR MESSAGES

209
Primary Documentation

• Screenshots
• GUI
• ERROR MESSAGES

210
Primary Documentation

• Data Bases

211
Primary Documentation

• Data Bases
• NOT RECOMMENDED

212
Primary Documentation

• Data Bases
• NOT RECOMMENDED
• but...

213
Primary Documentation

• Data Bases
• NOT RECOMMENDED
• but...
• Results of Queries are useful !

214
Primary Documentation

• Input Files
• Output Files
• Reports
• Screenshots
• Data Bases

215
Secondary Documentation
216
Secondary Documentation

• Extracts of Data Files
• Extracts of Reports
• Screenshot Traces
• Summary Sheets
• Combined Documents

217
Secondary Documentation

• Extracts of Data Files
• Extracts of Reports
• Screenshot Traces
• Summary Sheets
• Combined Documents

218
Secondary Documentation

• Extracts of Data Files
• Extracts of Reports
• Screenshot Traces
• Summary Sheets
• Combined Documents

219
Secondary Documentation

• Extracts of Data Files
• Extracts of Reports
• Screenshot Traces
• Summary Sheets
• Combined Documents

220
Secondary Documentation

• Extracts of Data Files
• Extracts of Reports
• Screenshot Traces
• Summary Sheets
• Combined Documents

221
Secondary Documentation

• Extracts of Data Files
• Extracts of Reports
• Screenshot Traces
• Summary Sheets
• Combined Documents

222
Secondary Documentation

• Sample of an Excel Sheet imported from a test
  output to be forwarded to an SAP application

223
Secondary Documentation

• Here, a spelling problem has been outlined in
  the converted report.

224
Secondary Documentation

• This comment has been entered in a
  higher-level acceptance run in 2001

225
Secondary Documentation

• Summaries can be put to the total page to make
  evaluation processes more obvious

226
Secondary Documentation

• First, the screenshot is put into the document

227
Secondary Documentation

• Then marks and texts are added

228
Secondary Documentation

• This is a sample of a spreadsheet to track the
  postings to an account

229
Secondary Documentation

• Objects can be combined

230
Secondary Documentation

• Data Integrity Check from 3 Sources

231
Secondary Documentation

• Here we checked the consistency of 3 reports

232
Secondary Documentation

• An Inconsistency was found

233
TEST COVERAGE
234
TEST COVERAGE

• What does it mean ?

235
TEST COVERAGE

• What does it mean ?
• TEST COVERAGE

236
TEST COVERAGE

• What does it mean ?
• TEST COVERAGE
• ?????

237
TEST COVERAGE

• How many
• Paths
• are there ???

238
TEST COVERAGE

• How many
• Paths
• are there ???

239
TEST COVERAGE

• Maybe 3 ?
• There are 3
• Terminators

240
TEST COVERAGE

• Maybe 6 ?
• There are 3
• Decisions

241
TEST COVERAGE

• Lets figure out

242
TEST COVERAGE

• This is Path
• Number 1

243
TEST COVERAGE

• This is Path
• Number 2

244
TEST COVERAGE

• This is Path
• Number 3

245
TEST COVERAGE

• This is Path
• Number 4

246
TEST COVERAGE

• To cover all 4 paths, four input records
• will be the absolute minimum
• It is Good Practice to use more data,
• e.g. 3 or more records per path

247
TEST COVERAGE

• So we had to follow the Diagram to find out
  about the Paths

248
TEST COVERAGE

• Do we know if the Flowchart represents the
  Source Code ?

249
TEST COVERAGE

• Can we be sure that the Source Code covers the
  Requirements ?

250
TEST COVERAGE

• Source Codes and Diagrams provided by
• the Supplier can only be Secondary Input
• for Testers
• Primary input are always the
• Requirements

251
TEST COVERAGE

• Source Codes and Diagrams provided by
• the Supplier can only be Secondary Input
• for Testers
• Primary input are always the
• Requirements

252
TEST COVERAGE

• Source Codes and Diagrams provided by
• the Supplier can only be Secondary Input
• for Testers
• Primary input are always the
• Requirements

253
TEST COVERAGE

• Population Selection by pickung every nth
  record
• Example
• n 10

254
TEST COVERAGE

• ADVANTAGE
• Easy Random Method
• Likely to give you Good Statistical
• Coverage
• This is considered Good Practice

255
TEST COVERAGE

• ADVANTAGE
• Easy Random Method
• Likely to give you Good Statistical
• Coverage
• This is considered Good Practice

256
TEST COVERAGE

• ADVANTAGE
• Easy Random Method
• Likely to give you Good Statistical
• Coverage
• This is considered Good Practice

257
TEST COVERAGE

• ADVANTAGE
• Easy Random Method
• Likely to give you Good Statistical
• Coverage
• This is considered Good Practice

258
TEST COVERAGE

• ADVANTAGE
• Easy Random Method
• Likely to give you Good Statistical
• Coverage
• This is considered Good Practice

259
TEST COVERAGE

• But, still.....

260
TEST COVERAGE
261
TEST COVERAGE

• But, still.....
• if your problem records are not selected,
• your test will succeed when it should
• better fail

262
TEST COVERAGE

• CONCLUSION
• If there are notorious problem records,
• add them always to your test population

263
Test Practice

• xyz

264
Test Practice Simple Things
265
Test Practice Simple Things

• Reports
• Check for
• Correct End-of-Page / Top-of-Page
• Level Breaks not only simple ones
• Page Number Page 0 follows 9
• Edit Masks overlapping / overwriting

266
Test Practice Simple Things

• Reports
• Check for
• Correct End-of-Page / Top-of-Page
• Level Breaks not only simple ones
• Page Number Page 0 follows 9
• Edit Masks overlapping / overwriting

267
Test Practice Simple Things

• Reports
• Check for
• Correct End-of-Page / Top-of-Page
• Level Breaks not only simple ones
• Page Number Page 0 follows 9
• Edit Masks overlapping / overwriting

268
Test Practice Simple Things

• Reports
• Check for
• Correct End-of-Page / Top-of-Page
• Level Breaks not only simple ones
• Page Number Page 0 follows 9
• Edit Masks overlapping / overwriting

269
Test Practice Simple Things

• Reports
• Check for
• Correct End-of-Page / Top-of-Page
• Level Breaks not only simple ones
• Page Number Page 0 follows 9
• Edit Masks overlapping / overwriting

270
Test Practice Simple Things

• Reports
• Check for
• Correct End-of-Page / Top-of-Page
• Level Breaks not only simple ones
• Page Number Page 0 follows 9
• Edit Masks overlapping / overwriting

271
Test Practice Simple Things

• Reports
• Check for
• Correct End-of-Page / Top-of-Page
• Level Breaks not only simple ones
• Page Number Page 0 follows 9
• Edit Masks overlapping / overwriting

272
Test Practice Simple Things

• Reports
• Check for
• Correct End-of-Page / Top-of-Page
• Level Breaks not only simple ones
• Page Number Page 0 follows 9
• Edit Masks overlapping / overwriting

273
Test Practice Simple Things

• Reports
• Check for
• Correct End-of-Page / Top-of-Page
• Level Breaks not only simple ones
• Page Number Page 0 follows 9
• Edit Masks overlapping / overwriting

274
Test Practice Simple Things

• Data Files
• Check for
• Overwriting instead of Inserting
• Updates in the GUI not made in DB
• Misrepresentation of DB Data on Screen
• Data Duplication in Output File

275
Test Practice Simple Things

• Data Files
• Check for
• Overwriting instead of Inserting
• Updates in the GUI not made in DB
• Misrepresentation of DB Data on Screen
• Data Duplication in Output File

276
Test Practice Simple Things

• Data Files
• Check for
• Overwriting instead of Inserting
• Updates in the GUI not made in DB
• Misrepresentation of DB Data on Screen
• Data Duplication in Output File

277
Test Practice Simple Things

• Data Files
• Check for
• Overwriting instead of Inserting
• Updates in the GUI not made in DB
• Misrepresentation of DB Data on Screen
• Data Duplication in Output File

278
Test Practice Simple Things

• Data Files
• Check for
• Overwriting instead of Inserting
• Updates in the GUI not made in DB
• Misrepresentation of DB Data on Screen
• Data Duplication in Output File

279
Test Practice Simple Things

• Data Files
• Check for
• Overwriting instead of Inserting
• Updates in the GUI not made in DB
• Misrepresentation of DB Data on Screen
• Data Duplication in Output File

280
Test Practice Simple Things

• Data Files
• Check for
• Overwriting instead of Inserting
• Updates in the GUI not made in DB
• Misrepresentation of DB Data on Screen
• Data Duplication in Output File

281
Test Practice Simple Things

• Manipulation of Data
• Q May I manipulate data ?
• A Without data manipulation, your testing
• performance will be rather poor !

282
Test Practice Simple Things

• Manipulation of Data
• Q May I manipulate data ?
• A Without data manipulation, your testing
• performance will be rather poor !

283
Test Practice Simple Things

• Manipulation of Data
• Q May I manipulate data ?
• A Without data manipulation, your testing
• performance will be rather poor !

284
Test Practice Volume Test
285
Test Practice Volume Test

• Requirement

286
Test Practice Volume Test

• Requirement
• 1 Million Records to be processed

287
Test Practice Volume Test

• Requirement
• 1 Million Records to be processed
• Time Window 8 hours

288
Test Practice Volume Test

• Requirement
• 1 Million Records to be processed
• Time Window 8 hours

289
Test Practice Volume Test

• Linear Model
• 125 K / h

290
Test Practice Volume Test

• Testers Assumption
• If I send 125,000 records through
• and they are finished in less than 1 hour
• the test has passed

291
Test Practice Volume Test

• Test Result
• The processing of 125,000 records takes
• 1 hour, 14 minutes and 31 seconds

292
Test Practice Volume Test

• Test Result
• The processing of 125,000 records takes
• 1 hour, 14 minutes and 31 seconds
• The test has failed

293
Test Practice Volume Test

• Next Step
• The provider of the program claims that
• the linear approach does not apply and
• asks for a test series with more records

294
Test Practice Volume Test

• Next Step
• The provider of the program claims that
• the linear approach does not apply and
• asks for a test series with more records.
• A new test with steps of 125,000 records is
• scheduled.

295
Test Practice Volume Test
296
Test Practice Volume Test

• Test Result
• The program is slow at initiation and speeds
• up later.
• A new model is created.
• The program is accepted and signed off.

297
Test Practice Volume Test
298
Test Practice Volume Test

• The First Live Run
• The program needed a couple of minutes more
• than 10 hours.
• The tester is working at a car wash now.

299
Test Practice Volume Test

• The First Live Run
• The program needed a couple of minutes more
• than 10 hours.
• The tester is working at a car wash now.

300
Test Practice Volume Test

• What happened ???

301
Test Practice Volume Test

• What happened ???
• The program is filling a table and gets
• significantly slower after ca. 505,000 records.

302
Test Practice Volume Test

• What happened ???
• The program is filling a table and gets
• significantly slower after ca. 505,000 records.
• It had never been tested with the full volume.

303
Test Practice Volume Test
304
Test Practice

• External Interfaces

305
Test Practice External Interfaces

• Common Examples
• CDRs
• Bank Files (Payments, Direct Debit...)
• Credit Card Interchanges
• EDIFACT
• Output to Accounting Systems

306
Test Practice External Interfaces

• Common Examples
• CDRs
• Bank Files (Payments, Direct Debit...)
• Credit Card Interchanges
• EDIFACT
• Output to Accounting Systems

307
Test Practice External Interfaces

• Common Examples
• CDRs
• Bank Files (Payments, Direct Debit...)
• Credit Card Interchanges
• EDIFACT
• Output to Accounting Systems

308
Test Practice External Interfaces

• Common Examples
• CDRs
• Bank Files (Payments, Direct Debit...)
• Credit Card Interchanges
• EDIFACT
• Output to Accounting Systems

309
Test Practice External Interfaces

• Common Examples
• CDRs
• Bank Files (Payments, Direct Debit...)
• Credit Card Interchanges
• EDIFACT
• Output to Accounting Systems

310
Test Practice External Interfaces

• Common Examples
• CDRs
• Bank Files (Payments, Direct Debit...)
• Credit Card Interchanges
• EDIFACT
• Output to Accounting Systems

311
Test Practice External Interfaces

• Common Examples
• CDRs
• Bank Files (Payments, Direct Debit...)
• Credit Card Interchanges
• EDIFACT
• Output to Accounting Systems

312
Test Practice External Interfaces

• General Rules
• Always ask for Official Descriptions
• Never use Obscure Private Documentation
• Try to get Authorized Samples

313
Test Practice External Interfaces

• General Rules
• Always ask for Official Descriptions
• Never use Obscure Private Documentation
• Try to get Authorized Samples

314
Test Practice External Interfaces

• General Rules
• Always ask for Official Descriptions
• Never use Obscure Private Documentation
• Try to get Authorized Samples

315
Test Practice External Interfaces

• General Rules
• Always ask for Official Descriptions
• Never use Obscure Private Documentation
• Try to get Authorized Samples

316
Test Practice External Interfaces

• General Rules
• Always ask for Official Descriptions
• Never use Obscure Private Documentation
• Try to get Authorized Samples

317
Test Practice External Interfaces

• General Rules
• Always ask for Official Descriptions
• Never use Obscure Private Documentation
• Try to get Authorized Samples

318
Test Practice External Interfaces

• General Rules
• Always ask for Official Descriptions
• Never use Obscure Private Documentation
• Try to get Authorized Samples

319
Test Practice External Interfaces

• General Rules
• Always ask for Official Descriptions
• Never use Obscure Private Documentation
• Try to get Authorized Samples

320
Test Practice External Interfaces

• Rules for Input Files

321
Test Practice External Interfaces

• Rules for Input Files
• Format Checks with Authorized Samples
• Functional Tests with your own Files
• Obscure Private Documentation
• Try to get Authorized Samples

322
Test Practice External Interfaces

• Rules for Input Files
• Format Checks with Authorized Samples
• Functional Tests with your own Files
• Obscure Private Documentation
• Try to get Authorized Samples

323
Test Practice External Interfaces

• Rules for Input Files
• Format Checks with Authorized Samples
• Functional Tests with your own Files
• Obscure Private Documentation
• Try to get Authorized Samples

324
Test Practice External Interfaces

• Rules for Input Files
• Format Checks with Authorized Samples
• Functional Tests with your own Files
• Obscure Private Documentation
• Try to get Authorized Samples

325
Test Practice External Interfaces

• Rules for Input Files
• Format Checks with Authorized Samples
• Functional Tests with your own Files
• Of course, you can use the Authorized Samples
• for Functional Tests, too.
• Obscure Private Documentation
• Try to get Authorized Samples

326
Test Practice External Interfaces

• Rules for Input Files
• Format Checks with Authorized Samples
• Functional Tests with your own Files
• Of course, you can use the Authorized Samples
• for Functional Tests, too.
• But do not use your own files for Format Checks
  !!!

327
Test Practice External Interfaces

• Rules for Output Files
• Perform Multi-Layered Format Checks
• Check for Hex contents with Hex Editor
• Write a small program for checksums etc.
• Load the output into a data base of your own
• tests with your own Files

328
Test Practice External Interfaces

• Rules for Output Files
• Perform Multi-Layered Format Checks
• Check for Hex contents with Hex Editor
• Write a small program for checksums etc.
• Load the output into a data base of your own
• tests with your own Files

329
Test Practice External Interfaces

• Rules for Output Files
• Perform Multi-Layered Format Checks
• Check for Hex contents with Hex Editor
• Write a small program for checksums etc.
• Load the output into a data base of your own
• tests with your own Files

330
Test Practice External Interfaces

• Rules for Output Files
• Perform Multi-Layered Format Checks
• Check for Hex contents with Hex Editor
• Write a small program for checksums etc.
• Load the output into a data base of your own
• tests with your own Files

331
Test Practice External Interfaces

• Rules for Output Files
• Perform Multi-Layered Format Checks
• Check for Hex contents with Hex Editor
• Write a small program for checksums etc.
• Load the output into a data base of your own
• tests with your own Files

332
Test Practice External Interfaces

• Rules for Output Files
• Perform Multi-Layered Format Checks
• Check for Hex contents with Hex Editor
• Write a small program for checksums etc.
• Load the output into a data base of your own
• tests with your own Files

333
Test Practice External Interfaces

• Rules for Output Files
• Perform Multi-Layered Format Checks
• Check for Hex contents with Hex Editor
• Write a small program for checksums etc.
• Load the output into a data base of your own
• tests with your own Files

334
Test Practice External Interfaces

• Rules for Output Files
• Perform Multi-Layered Format Checks
• Have the Output checked by Recipient
• tests with your own Files

335
Test Practice External Interfaces

• Rules for Output Files
• Perform Multi-Layered Format Checks
• Have the Output checked by Recipient
• tests with your own Files

336
Test Practice External Interfaces

• Rules for Output Files
• Perform Multi-Layered Format Checks
• Have the Output checked by Recipient
• If there is a small number of recipients, ask
• all of them for assistance

337
Test Practice External Interfaces

• DO NOT TRAP YOURSELF
• If you use your own output files for input
• what does this prove ?
• there is a small number of recipients, ask
• all of them for assistance

338
Test Practice External Interfaces

• DO NOT TRAP YOURSELF
• If you use your own output files for input
• what does this prove ?
• there is a small number of recipients, ask
• all of them for assistance

339
Test Practice External Interfaces

• DO NOT TRAP YOURSELF
• If you use your own output files for input
• what does this prove ?
• That you are using compatible formats
• all of them for assistance

340
Test Practice External Interfaces

• DO NOT TRAP YOURSELF
• If you use your own output files for input
• what does this prove ?
• That you are using compatible formats
• Maybe both of them are wrong !?!?

341
THE END

• xyz

342
THE END

• ready for questions.......