From Quality Control to Quality Assurance - PowerPoint PPT Presentation


PPT – From Quality Control to Quality Assurance PowerPoint presentation | free to download - id: 247c74-ZDc1Z


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation

From Quality Control to Quality Assurance


... a stepping stone to development (often still true today) ... Software testing today. Many teams using automation to produce consistent and frequent test results ... – PowerPoint PPT presentation

Number of Views:17
Avg rating:3.0/5.0
Slides: 22
Provided by: magd153
Learn more at:


Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: From Quality Control to Quality Assurance

From Quality Control to Quality Assuranceand
  • Alan Page
  • Microsoft

  • My testing career started in 1993
  • Testing always last
  • Testing often skipped
  • Anyone can test
  • Testing in quality
  • My role in testing has changed significantly in
    the past 13 years
  • Testing as a whole has matured
  • Disclaimer I know what I know and dont claim
    to know more (and are not representative of

Software testing 13 years ago
  • Anyone can do it
  • Technical knowledge not important
  • Test is a stepping stone to development (often
    still true today)
  • Testing occurs late in the product cycle

Software testing today
  • Many teams using automation to produce consistent
    and frequent test results
  • Often heavy emphasis on automation
  • Testers are generally focused on
  • Test authoring writing test cases and automated
  • Test execution running automated or manual test
  • Functional and non-functional testing
  • Other bug finding activities
  • Some analysis
  • Data analysis is often extensive, but is
    primarily based on defect data

Software testing tomorrow
  • Too many bugs getting to customers
  • Bugs need to be found sooner

Where are we going?
  • Better bug prediction
  • More testable software
  • Defect prevention
  • Inspections
  • How do we get there?

More testable software
  • Testability is the degree to which components and
    systems are designed and implemented to make it
    easier for tests to achieve complete and
    repeatable code path coverage and simulate all
    usage situations in a cost efficient manner. In
    other words, testability determines how
    inexpensive it is to test.
  • or
  • What is the cost of testing?

  • Testability is a design time attribute
  • but testers often drive testability into the
  • Ask questions and take action
  • How are we going to efficiently test this?
  • Add testability sections to design and functional
  • Hold testability reviews

SOCK for testability
  • Testability SOCK
  • Simple components are less expensive to test.
  • Can the hardware be simulated?
  • Can we test a multi-machine scenario on a single
  • Observable behavior and results are essential in
    order to determine pass or fail
  • Can I determine if the internal structures were
    updated correctly?
  • Can I tell which path the code took?

SOCK for testability
  • Control can I test every nook and cranny?
  • Can I change timeout values or other thresholds
    to simulate failures?
  • Can I simulate every failure that a customer may
  • Knowledge is needed to understand expected
    results and compare with actual results
  • What should happen in error cases?
  • What circumstances cause this error?

Defect analysis
  • Known defects need to be analyzed
  • Deep analysis can tell a lot, but its expensive
  • Too lightweight of an analysis may not tell you
  • Ask why
  • Analyze to the point of action Ask why until
    you have a sufficient root cause

Ask Why?
  • Why did the program crash?
  • The filename was longer than expected.
  • Why did the program not expect a longer filename?
  • The programmer was unfamiliar with file name
  • Why was the programmer unfamiliar with the
  • They were new to windows programming and hadnt
    been trained
  • or
  • Why was this missed during code review?
  • Code review rules were often skipped or rushed.

Defect prevention
  • Root cause analysis leads to defect prevention
  • What could have been done to keep this defect
    from ever happening?
  • Examples
  • Change compiler settings or use code analysis
    tools or scripts
  • Build code often (continuous integration)
  • Automatically run unit tests or verification
    tests at check-in

Software Inspections
  • Not typically considered a Test task but
    testers are well suited to drive this effort
  • Extremely effective method of removing defects,
  • Cost (training and time) tends to scare teams away

What happens when you inspect?
Measurements were performed on existing code
base, and again on subsequent releases
  • Example of changes from inspection

Inspection Metrics Existing code Release A Release B Release C
Inspection Effort 0 hrs 406 hrs 276 hrs 691 hrs
Total LOC 400K 7777 10904 49343
Inspection hrs / KLOC 0 52 25 14
Total Effort n/a 1062 hrs 1101 hrs 2169 hrs
Percent in inspection 0 38 25 32
Total hrs / KLOC n/a 137 101 44
Defects / KLOC (after check-in) 8.2 1.8 1 0.7
How to make inspections less scary
  • Team members do not know how to inspect
  • Train the team appoint a moderator
  • Inspections perceived to slow down the project
  • Schedule inspections measure and track progress
  • Team members fearful of inspections
  • Non-confrontational meetings remove threats use
    as a learning experience for author

What am I trying to say here?
  • We need to stop waiting for bugs!
  • Its great to find bugs before the customers do,
    but we need to find them earlier
  • We need more emphasis on early detection and
    ultimately, prevention

How can this be achieved?
  • Be an equal member of the engineering team
  • Software quality is a tough problem
  • Technical, creative people are needed to solve
    this problem
  • We need clearly defined career paths for both
    managers and non-managers in order to keep these
    types of people in test
  • Provide learning opportunities when needed

Key points
  • Software testing is a maturing profession
  • Need to move beyond quality control (writing test
    cases and running tests)
  • Test needs to do more to drive quality
    improvements throughout the engineering process
  • Technical skills are important in order to drive
    quality initiatives

  • Software Inspections
  • Tom Gilb and Dorothy Graham, Addison-Wesley
  • The Practical Guide to Defect Prevention
  • Harry Emil et al. Microsoft Press
  • Random thoughts on testing
  • http//