From Quality Control to Quality Assurance - PowerPoint PPT Presentation

Loading...

PPT – From Quality Control to Quality Assurance PowerPoint presentation | free to download - id: 247c74-ZDc1Z



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

From Quality Control to Quality Assurance

Description:

... a stepping stone to development (often still true today) ... Software testing today. Many teams using automation to produce consistent and frequent test results ... – PowerPoint PPT presentation

Number of Views:17
Avg rating:3.0/5.0
Slides: 22
Provided by: magd153
Learn more at: http://www.sasqag.org
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: From Quality Control to Quality Assurance


1
From Quality Control to Quality Assuranceand
Beyond
  • Alan Page
  • Microsoft

2
Introduction
  • My testing career started in 1993
  • Testing always last
  • Testing often skipped
  • Anyone can test
  • Testing in quality
  • My role in testing has changed significantly in
    the past 13 years
  • Testing as a whole has matured
  • Disclaimer I know what I know and dont claim
    to know more (and are not representative of
    microsoft)

3
Software testing 13 years ago
  • Anyone can do it
  • Technical knowledge not important
  • Test is a stepping stone to development (often
    still true today)
  • Testing occurs late in the product cycle

4
Software testing today
  • Many teams using automation to produce consistent
    and frequent test results
  • Often heavy emphasis on automation
  • Testers are generally focused on
  • Test authoring writing test cases and automated
    tests
  • Test execution running automated or manual test
    cases
  • Functional and non-functional testing
  • Other bug finding activities
  • Some analysis
  • Data analysis is often extensive, but is
    primarily based on defect data

5
Software testing tomorrow
  • Too many bugs getting to customers
  • Bugs need to be found sooner

6
Where are we going?
  • Better bug prediction
  • More testable software
  • Defect prevention
  • Inspections
  • How do we get there?

7
More testable software
  • Testability is the degree to which components and
    systems are designed and implemented to make it
    easier for tests to achieve complete and
    repeatable code path coverage and simulate all
    usage situations in a cost efficient manner. In
    other words, testability determines how
    inexpensive it is to test.
  • or
  • What is the cost of testing?

8
Testability
  • Testability is a design time attribute
  • but testers often drive testability into the
    design
  • Ask questions and take action
  • How are we going to efficiently test this?
  • Add testability sections to design and functional
    specifications
  • Hold testability reviews

9
SOCK for testability
  • Testability SOCK
  • Simple components are less expensive to test.
  • Can the hardware be simulated?
  • Can we test a multi-machine scenario on a single
    machine?
  • Observable behavior and results are essential in
    order to determine pass or fail
  • Can I determine if the internal structures were
    updated correctly?
  • Can I tell which path the code took?

10
SOCK for testability
  • Control can I test every nook and cranny?
  • Can I change timeout values or other thresholds
    to simulate failures?
  • Can I simulate every failure that a customer may
    hit?
  • Knowledge is needed to understand expected
    results and compare with actual results
  • What should happen in error cases?
  • What circumstances cause this error?

11
Defect analysis
  • Known defects need to be analyzed
  • Deep analysis can tell a lot, but its expensive
  • Too lightweight of an analysis may not tell you
    enough
  • Ask why
  • Analyze to the point of action Ask why until
    you have a sufficient root cause

12
Ask Why?
  • Why did the program crash?
  • The filename was longer than expected.
  • Why did the program not expect a longer filename?
  • The programmer was unfamiliar with file name
    limits
  • Why was the programmer unfamiliar with the
    limits?
  • They were new to windows programming and hadnt
    been trained
  • or
  • Why was this missed during code review?
  • Code review rules were often skipped or rushed.

13
Defect prevention
  • Root cause analysis leads to defect prevention
  • What could have been done to keep this defect
    from ever happening?
  • Examples
  • Change compiler settings or use code analysis
    tools or scripts
  • Build code often (continuous integration)
  • Automatically run unit tests or verification
    tests at check-in

14
Software Inspections
  • Not typically considered a Test task but
    testers are well suited to drive this effort
  • Extremely effective method of removing defects,
    but
  • Cost (training and time) tends to scare teams away

15
What happens when you inspect?
Measurements were performed on existing code
base, and again on subsequent releases
  • Example of changes from inspection

Inspection Metrics Existing code Release A Release B Release C
Inspection Effort 0 hrs 406 hrs 276 hrs 691 hrs
Total LOC 400K 7777 10904 49343
Inspection hrs / KLOC 0 52 25 14
Total Effort n/a 1062 hrs 1101 hrs 2169 hrs
Percent in inspection 0 38 25 32
Total hrs / KLOC n/a 137 101 44
Defects / KLOC (after check-in) 8.2 1.8 1 0.7
16
How to make inspections less scary
  • Team members do not know how to inspect
  • Train the team appoint a moderator
  • Inspections perceived to slow down the project
  • Schedule inspections measure and track progress
  • Team members fearful of inspections
  • Non-confrontational meetings remove threats use
    as a learning experience for author

17
What am I trying to say here?
  • We need to stop waiting for bugs!
  • Its great to find bugs before the customers do,
    but we need to find them earlier
  • We need more emphasis on early detection and
    ultimately, prevention

18
How can this be achieved?
  • Be an equal member of the engineering team
  • Software quality is a tough problem
  • Technical, creative people are needed to solve
    this problem
  • We need clearly defined career paths for both
    managers and non-managers in order to keep these
    types of people in test
  • Provide learning opportunities when needed

19
Key points
  • Software testing is a maturing profession
  • Need to move beyond quality control (writing test
    cases and running tests)
  • Test needs to do more to drive quality
    improvements throughout the engineering process
  • Technical skills are important in order to drive
    quality initiatives

20
Resources
  • Software Inspections
  • Tom Gilb and Dorothy Graham, Addison-Wesley
  • The Practical Guide to Defect Prevention
  • Harry Emil et al. Microsoft Press
  • Random thoughts on testing
  • http//blogs.msdn.com/alanpa/

21
Questions?
About PowerShow.com