Top 10 Challenges in the Field of Quality Today Panel Discussion

1
Top 10 Challenges in the Field of Quality
TodayPanel Discussion

• SWOSQG Meeting
• February 9, 2005
• Held at London Life, London Ontario

2
Agenda

• Test Environment Challenges of Today - Lynn
• Training for Testers Dan
• Tools for Testing - Dan
• The State of the Profession - Mike
• Requirements - Mike
• Theres not (never) enough time to test - Lynn

3
About the Panel

• Dan Corcoran CSQA, CSTE, Consulting Specialist,
  Software Development Quality Assurance from
  London Life
• Mike Cookson CSQA, Manager, Customer Delivery
  Team from MRO Software
• Lynn Larouche CSTE, Quality Assurance Senior
  Consultant also from London Life.

4
Agenda

• Test Environment Challenges of Today - Lynn
• Training for Testers Dan
• The State of the Profession - Mike
• Theres not (never) enough time to test - Mike
• Tools for Testing Open Discussion
• Requirements Open Discussion

5
Test Environment Challenges
6
Test Environment Challenges

• Testing is becoming much more challenging

7
Test Environment Challenges

• Types of Testing
• Load, Stress, Performance, Breakpoint,
  Authentication, Authorization, Boundary,
  Structure, Function, System, User Acceptance,
  Installation Verification, Unit, Integration,
  Regression, Risk, Configuration, Exhaustive,
  Path, Recovery, Usability, Validation,
  Verification, White Box, Black Box.
• Requires several different environments
• Requires different test data

8
Test Environment Challenges
Web Environment Hundreds of Pages
Mainframe Environment 24x7 ???
Middleware Environment Medium Complexity
9
Test Environment Challenges

• What do we need to do to make this work?
• Better tools
• Better processes
• Very good understanding of the environment(s)
• More automated testing techniques
• Better analytical tools and processes

10
Test Environment Challenges

• Installation Verification Testing (IVT)
• Record and playback
• Web Based
• XDE Tester from IBM Rational
• Works on/across web platforms
• Comprehensive and repeatable
• Function oriented versus performance oriented
• Some care and feeding required
• Dependant on backend not changing

11
Test Environment Challenges

• Discussion and Questions

12
Agenda

• Test Environment Challenges of Today - Lynn
• Training for Testers Dan
• The State of the Profession - Mike
• Theres not (never) enough time to test - Mike
• Tools for Testing Open Discussion
• Requirements Open Discussion

13
Training for Testers
February 9, 2005

• No training in the processes?
• Training beyond conference and course
• whats available cheap?
• Read any good books lately?

14
Training for Testers
No training in the processes?

• Software Development Processes
• do you follow a proprietory process?
• do you follow a software vendors process?
• are you following industry practices?
• Testing Processes
• do you follow a software vendors process?
• are you following industry practices?

15
Training for Testers
What is available cheap?

• Software Development Processes
• http//www.sei.cmu.edu/
• choose Books or Technical Reports
• http//www.asq.org/ed/
• choose e-learning centre
• http//www.computer.org/education/
• choose distance learning

16
Training for Testers
What is available cheap?

• Testing Processes
• http//landing.domainsponsor.com/?a_id794adultfi
  lterondomainnamestlabs.com
• http//www.riceconsulting.com/new/
• http//www.qaiusa.com/partners/index.htm
• choose QAI Online Courses
• http//www.tassq.org/
• choose Free Book in exchange for Book Review

17
Training for Testers
Beyond conference course...

• Software Development Processes
• http//www.experts-exchange.com/
• http//publibz.boulder.ibm.com/cgi-bin/bookmgr_OS
  390/Shelves
• http//www-306.ibm.com/software/sw-training/
• choose View Special Offers or Trials and Betas
• http//www.adtmag.com/section.asp?sectionprocess

18
Training for Testers
Beyond conference course...

• Testing Processes
• http//www.soft.com/Institute/HotList/
• http//www.mtsu.edu/storm/
• choose Software Testing Discussion and News
• choose Software Testing Literature
• http//www.stickyminds.com/
• http//www.cutter.com/itreports/

19
Training for Testers
Read any good books lately?

• The ROI from Software Quality An executive
  briefing
• Khaled El Emam
• K Sharp Technology Inc.
• http//www.ksharp.com
• choose Reports -gt bookshelf

20
Training forTesters

• Discussion and Questions

21
Agenda

• Test Environment Challenges of Today - Lynn
• Training for Testers Dan
• The State of the Profession - Mike
• Theres not (never) enough time to test - Mike
• Tools for Testing Open Discussion
• Requirements Open Discussion

22
Topics

• Management Doesnt Understand or care about
  testing or quality
• Lack of respect for testers and QA
• No standards and best practices
• Never Enough Time to Test Thoroughly

23
Management

• Management doesnt know or care about testing or
  Quality
• Partially correct!
• Management is interested in productivity and
  company results
• They only care about testing and productivity as
  it relates to those two items

24
Management
We do not relate testing and quality to these
concerns!
25
Management

• It is difficult to understand the relationship
  between quality testing, overall quality and
  performance of the company.
• The expense (or investment) of Quality Programs
  is definable the benefits sometimes are not as
  well defined.

26
Management

• Need to tie the two together
• Is there an SLA with penalties?
• Do you know the cost of losing 1 day / 1 hour of
  uptime? (website, internal app, bank app)
• Are there any direct competitors that are taking
  market share that are working on a formal
  certification program like CMM?
• Do you have the data to show the cost of fixing
  errors in production vs. test vs. development vs.
  design?

27
Management

• There are many books dedicated to presenting the
  ROI of testing and quality
• They are generic though and require an
  organization to do some investigation to make it
  meaningful to them

28
Lack of Respect

• Two ways as a profession or personal
• Profession no respect for testing in general
• Personal no respect for you as a tester
• Both will give you problems
• Ill only deal with the profession here

29
Lack of Respect

• Usually caused because the person(s) does not
  understand or has experienced the value of
  testing
• Sees it as a cost and an obstacle to getting
  projects done
• Testers seem to have a reputation for whining
  not fair, but there it is

30
Lack of Respect

• Have to understand why people feel this way.
• Explain what your goal is not to prove the
  application works, but find the bugs before
  customers
• Especially find the bugs that will cause the
  project to fail

31
Lack of Respect

• Testing brings value to a project!
• We need to be flexible to ensure we bring that
  value
• Testing for testing sake will not win you any
  friends

32
No Standards

• No standards or best practices
• Assume this to mean the company does not use
  standards or best practices because there are
  many standards and best practices available
• CMM, ISO, IEEE all have standards around
  documentation and processes
• Numerous textbooks on testing and testing
  methodology

33
No Standards

• Corporate standards are impossible for an
  individual to impose
• Have to start smaller get yourself and your
  team to create and follow a standard
• Test Plans
• Test Cases
• Test Reports

34
No Standards

• If you are not following them consistently, then
  no one else will either
• Once you do, you can begin to push on the
  interface areas to standardize as well
• We need a rough spec of how the new function
  will work an email is fine.
• We need more details for the new function can
  you put it in a document?

35
State of theProfession

• Discussion and Questions

36
Agenda

• Test Environment Challenges of Today - Lynn
• Training for Testers Dan
• The State of the Profession - Mike
• Theres not (never) enough time to test - Mike
• Tools for Testing Open Discussion
• Requirements Open Discussion

37
No Time

• Never enough time to test thoroughly
• Related to not having a process or standard
• Sometimes difficult to determine
• Clear end to design phase, clear end to coding,
  not so clear to testing
• What is thoroughly? What is enough?
• We can never test 100

38
No Time

• If you have a process that states what you are
  doing then it is easier to get time
• Sometimes say Go ahead and deploy then
• dont be a rubber stamp
• Try to get an agreement on what has to be tested
• These functions tested and no outstanding sev 1

39
No Time

• Have to sell what you do
• We do the testing
• Development puts the bugs in
• We are done when development has done the job
  right
• It takes one week to do a testing iteration
• Once we hit the target we are done

40
No Time

• Discussion and Questions

41
Agenda

• Test Environment Challenges of Today - Lynn
• Training for Testers Dan
• The State of the Profession - Mike
• Theres not (never) enough time to test - Mike
• Tools for Testing Open Discussion
• Requirements Open Discussion

42
Agenda

• Test Environment Challenges of Today - Lynn
• Training for Testers Dan
• The State of the Profession - Mike
• Theres not (never) enough time to test - Mike
• Tools for Testing Open Discussion
• Requirements Open Discussion

43
Final Call

• Discussion and Questions