TQS Teste e Qualidade de Software Software Testing and Quality Preamble Course Positioning

1
TQS - Teste e Qualidade de Software(Software
Testing and Quality) Preamble - Course
Positioning
João Pascoal Faria jpf_at_fe.up.pt
www.fe.up.pt/jpf
2
IndexCourse positioning with respect to ...

• Capability Maturity Model Integration for
  Software Engineering (CMMI-SW), Software
  Engineering Institute (SEI), Carnegie Mellon
  University (CMU)
• To help organize quality related activities and
  terminology
• Software Engineering Education Knowledge,
  Computing Curricula - Software Engineering
   (CC-SE), IEEE Computer Society / ACM
• To help organize and put in context this
  (educational) knowledge area
• Guide to the Software Engineering Body of
  Knowledge (SWEBOK), IEEE Computer Society
• To help organize and put in context this (4 years
  professional) knowledge area
• Certified Software Quality Engineer (CSQE) Body
  of Knowledge, American Society for Quality (ASQ)
• To help organize and put in context this (8 years
  professional) knowledge area

3
Capability Maturity Model Integration for
Software Engineering (CMMI-SW), Version 1.1

• Developed by the Software Engineering Institute,
  Carnegie Mellon University
• Designed to help organizations improve their
  product and service development, acquisition, and
  maintenance processes
• Judge the maturity of the software processes of
  an organization
• Identify the key practices required to increase
  the maturity of these processes
• Staged representation - measures
  process-improvement achievement across an
  organizational unit using maturity levels (as in
  traditional SW-CMM)
• At maturity level n, an organization has achieved
  all the specific goals of the process areas
  assigned to maturity levels ? n (and generic
  goals assigned to levels ? n)
• To achieve each goal a set of (best) practices
  must be implemented
• Continuous representation - measures
  process-improvement achievement within individual
  process areas (e.g. configuration management)
  using capability levels
• The CMMI models are the result of an effort of
  integration of the "traditional" CMM models
  (SW-CMM, etc.)

4
CMMI-SW, v1.1, staged versionMaturity Levels

• 1) Initial. The software process is characterized
  as ad hoc, and occasionally even chaotic. Few
  processes are defined, and success depends on
  individual effort and heroics.
• 2) Managed. Basic project management processes
  are established to track cost, schedule, and
  functionality. The necessary process discipline
  is in place to repeat earlier successes on
  projects with similar applications.
• 3) Defined. The organizations set of standard
  processes is established and improved over time.
  Projects and organizational units establish their
  defined processes by tailoring the organizations
  set of standard processes according to tailoring
  guidelines.
• 4) Quantitatively Managed. Detailed measures of
  the software process and product quality are
  collected. Both the software process and products
  are quantitatively understood and controlled.
• 5) Optimizing. Continuous process improvement is
  enabled by quantitative feedback from the process
  and from piloting innovative ideas and
  technologies.

5
CMMI-SW, v1.1, staged versionProcess Areas by
Maturity Levels and Process Area Categories
Level 4)Quantitatively Managed
Level 1) Initial
Level 5)Optimizing
Level 2) Managed
Level 3) Defined ()
Organizational Process Focus
Organizational Process Performance
Organizational Innovation and Deployment
Process Managem.
Organizational Process Definition
Organizational Training
Integrated Project Managementfor Integrated
Product and Process Development (IPPD)
Project Planning
Quantitative Project Management
Project Managem.
Project Monitoring and Control
Supplier Agreement Management
Risk Management
Requirements Management (REQM)
Requirements Development (RD)
Technical Solution (TS)
Engineering
Product Integration (PI)
Verification (VER)
Validation (VAL)
Configuration Management (CM)
Causal Analysis and Resolution (CAR)
Decision Analysis and Resolution (DAR)
Measurement and Analysis (MA)
Support
Process and Product Quality Assurance (PPQA)
() also includes practices to institutionalize a
defined process for each process area of level 2
6
CMMI-SW, v1.1Verification and Validation (VV)

• Verification - ensures that the specified
  requirements are satisfied
• Confirms that (intermediate and final) work
  products properly reflect the requirements
  specified for them
• Ensures that you built it right
• Occurs throughout the development of the product
  and work products, beginning with verification of
  the requirements, progressing through the
  verification of the evolving work products, and
  culminating in the verification of the completed
  product
• Validation - ensures that the product will
  fulfill its intended use
• Confirms that the (final) product, as provided,
  will fulfill its intended use
• Ensures that you built the right thing
• Applied mainly to the product and product
  components in its intended environment, but can
  also be applied to work products (e.g.,
  requirements, designs, prototypes) selected on
  the basis of which are the best predictors of how
  well the product and product component will
  satisfy user needs
• Both validation and verification activities often
  run concurrently and may use portions of the same
  environment

7
CMMI-SW, v1.1 Interactions Among VV and other
Engineering Process Areas
Requirements
Product and product
component requirements
Alternative
Product
solutions
Product
components
Customer
Require
-
ments
Product components, work products,
verification and validation reports
Customer needs
8
CMMI-SW, v1.1 Process and Product Quality
Assurance (PPQA)

• Quality Assurance - ensures that planned
  processes are implemented
• Involves objectively evaluating performed
  processes and work products against applicable
  process descriptions, standards, and procedures,
  and communicating and ensuring resolution of
  noncompliance issues
• Usually performed by a QA group that is
  independent of the project, or by peers not
  directly involved in the development or
  maintenance of the work product subject to
  evaluation
• Should begin in the early phases of a project
  (with the participation of the QA group) to
  establish plans, processes, standards, and
  procedures that will add value to the project and
  satisfy its requirements and the organizational
  policies and that will be useable for performing
  QA evaluations, and to designate the specific
  processes and work products that will be
  evaluated
• VV and PPQA may on occasion address the same
  work product but from different perspectives
  care should be taken to minimize duplication of
  effort

9
CMMI-SW, v1.1 Interactions Among PPQA and other
Process Areas
IPPD
Infrastructure
organizational environment for integration
Ability to develop
and deploy IPPD processes
and supporting assets
Process improvement
Integrated work
proposals
IPPD
environment and
knowledge
people practices
and skill
Defects and
needs
other problems
Quality and
Measurements,
noncompliance
analyses
issues
PPQA
Processes and work products,
Information
standards and procedures
needs
All process areas
Selected
issues
Configuration
Baselines,
Formal
items,
audit reports
evaluations
change
requests
10
CMMI-SW, v1.1Terminology - artifacts

• Product - any tangible output or service that is
  a result of a process and that is intended for
  delivery to a customer or end user
• A product is a work product that is delivered to
  the customer
• Product components - lower level components of
  the product, that are integrated to build the
  product
• There may be multiple levels of product
  components
• A product component is any work product that must
  be engineered (requirements defined and designs
  developed and implemented) to achieve the
  intended use of the product throughout its life
• Work product - any artifact produced by a
  process, including files, documents, parts of the
  product, services, processes, specifications, and
  invoices
• Examples of processes to be considered as work
  products include a manufacturing process, a
  training process, and a disposal process for the
  product

11
CMMI-SW, v1.1Terminology actors and activities

• Customer - the party (individual, project, or
  organization) responsible for accepting the
  product or for authorizing payment
• The customer is external to the project, but not
  necessarily external to the organization
• The customer may be a higher level project
• Customers are a subset of stakeholders
• Stakeholder - a group or individual that is
  affected by or in some way accountable for the
  outcome of an undertaking
• Stakeholders may include project members,
  suppliers, customers, end users, and others
• Project manager - the person responsible for
  planning, directing, controlling, structuring,
  and motivating the project and satisfying the
  customer
• Project - a managed set of interrelated resources
  that delivers one or more products to a customer
  or end user
• This set of resources has a definite beginning
  and end and typically operates according to a
  plan
• Such a plan is frequently documented and
  specifies the product to be delivered or
  implemented, the resources and funds used, the
  work to be done, and a schedule for doing the
  work
• A project can be composed of projects

12
Index

• Capability Maturity Model Integration for
  Software Engineering (CMMI-SW), Software
  Engineering Institute (SEI), Carnegie Mellon
  University (CMU)
• Computing Curricula - Software Engineering
   (CC-SE), IEEE Computer Society / ACM
• Guide to the Software Engineering Body of
  Knowledge (SWEBOK), IEEE Computer Society
• Certified Software Quality Engineer (CSQE) Body
  of Knowledge, American Society for Quality (ASQ)

13
Software Engineering Education Knowledge (SEEK),
Computing Curriculum Software Engineering
(CC-SE), 2004Knowledge areas and knowledge units
(1/2)

• Computing Essentials (172 hours)
• Computer Science foundations
• Construction technologies
• Construction tools
• Formal construction methods
• Mathematical Engineering Fundamentals (89
  hours)
• Mathematical foundations
• Engineering foundations for software
• Engineering economics for software
• Professional Practice (35 hours)
• Group dynamics / psychology
• Communications skills (specific to SE)
• Professionalism

• Software Modeling Analysis (53 hours)
• Modeling foundations
• Types of models
• Analysis fundamentals
• Requirements fundamentals
• Eliciting requirements
• Requirements specification documentation
• Requirements validation
• Software Design (45 hours)
• Design concepts
• Design strategies
• Architectural design
• Human computer interface design
• Detailed design
• Design support tools and evaluation

14
CC-SEKnowledge areas and knowledge units (2/2)
Process and Product
Assurance

• Software Quality (16 hours)
• Software quality concepts and culture (2 h)
• Software quality standards (2 h)
• Software quality processes (4 h)
• Process assurance (4 h)
• Product assurance (4 h)
• Software Management (19 hours)
• Management concepts
• Project planning
• Project personnel and organization
• Project control
• Software configuration management

• Software Verification Validation (42 hours)
• VV terminology and foundations (5 h)
• Reviews (6 h)
• Testing (21 h)
• Human computer UI testing and evaluation (6 h) ?
  IPC
• Problem analysis and reporting (4 h)
• Software Evolution (10 hours)
• Evolution processes
• Evolution activities
• Software Process (13 hours)
• Process concepts
• Process implementation

15
CC-SE / SEEK Software Verification Validation
(1/2)

• Software Verification Validation (42 hours) -gt
  20 h
• "Software verification and validation uses both
  static and dynamic techniques of system checking
  to ensure that the resulting program satisfies
  its specification and that the program as
  implemented meets the expectations of the
  stakeholders. Static techniques are concerned
  with the analysis and checking of system
  representations throughout all stages of the
  software life cycle, while dynamic techniques
  involve only the implemented system."

• VV terminology and foundations (5 h ) ? 2h
• Objectives and constraints of VV
• Planning the VV effort
• Documenting VV strategy, including tests and
  other artifacts
• Metrics Measurement (e.g. reliability,
  usability, performance, etc.)
• VV involvement at different points in the
  lifecycle

16
CC-SE / SEEK Software Verification Validation
(2/2)

• Reviews (6 h ) ? 1 h
• Desk checking
• Walkthroughs
• Inspections
• Human computer UI testing and evaluation (6 h) ?
  IPC
• The variety of aspects of usefulness and
  usability
• Heuristic evaluation
• Cognitive walkthroughs
• User testing approaches (observation sessions
  etc.)
• Web usability testing techniques for web sites
• Formal experiments to test hypotheses about
  specific HCI controls (D)
• Problem analysis and reporting (4 h) ? 3 h
• Analyzing failure reports
• Debugging/fault isolation techniques
• Defect analysis
• Problem tracking

• Testing (21 h ) ? 14h
• Unit testing
• Exception handling (writing test cases to trigger
  exception handling designing good handling)
• Coverage analysis (e.g. statement, branch, basis
  path, multi-condition, dataflow, etc.)
• Black-box functional testing techniques
• Integration Testing
• Developing test cases based on use cases and/or
  customer stories
• Operational profile-based testing
• System and acceptance testing
• Testing across quality attributes (e.g.
  usability, security, compatibility,
  accessibility, etc.)
• Regression Testing
• Testing tools
• Deployment process (D)

17
CC-SE / SEEK Software Quality

• Software Quality (16 hours) -gt 14 h
• "Software quality is a pervasive concept that
  affects, and is affected by all aspects of
  software development, support, revision, and
  maintenance. It encompasses the quality of work
  products developed and/or modified (both
  intermediate and deliverable work products) and
  the quality of the work processes used to develop
  and/or modify the work products. Quality work
  product attributes include functionality,
  usability, reliability, safety, security,
  maintainability, portability, efficiency,
  performance, and availability."

• Software quality concepts and culture (2 h) ?1h
• Definitions of quality
• Society's concern for quality
• The costs and impacts of bad quality
• A cost of quality model
• Quality attributes for software (e.g.
  dependability, usability, etc.)
• The dimensions of quality engineering
• Roles of people, processes, methods, tools, and
  technology

18
CC-SE / SEEK Software Quality

• Process assurance (4 h) ? 4
• The nature of process assurance
• Quality planning
• Organizing and reporting for process assurance
• Techniques of process assurance
• Product assurance (4 h) ? 3
• The nature of product assurance
• Distinctions between assurance and VV
• Quality product models
• Root cause analysis and defect prevention
• Quality product metrics and measurement
• Assessment of product quality attributes (e.g.
  usability, reliability, availability, etc.)

• Software quality standards (2 h) ? 2
• Quality Management Systems
• ISO/IEEE Standard 12207 Software Life Cycle
  Processes
• Organizational implementation of standards
• IEEE software quality-related standards (D)
• Software quality processes (4 h) ? 4
• Software quality models and metrics
• Quality-related aspects of software process
  models
• Introduction/overview of ISO 15504 and the SEI
  CMMs
• Quality-related process areas of ISO 15504
• Quality-related process areas of the SW-CMM and
  the CMMIs
• The Baldrige Award criteria as applied to
  software engineering (O)
• Quality aspects of other process models (O)

19
CC-SE / SEEK A proposed course on Software
Quality Assurance (1/2)

• Software Quality Assurance (37 hours) ? 34 h
• Broad coverage of software quality and testing.
• Course Description
• Quality how to assure it and verify it, and the
  need for a culture of quality. Avoidance of
  errors and other quality problems. Inspections
  and reviews. Testing, verification and validation
  techniques. Process assurance vs. Product
  assurance. Quality process standards. Product and
  process assurance. Problem analysis and
  reporting. Statistical approaches to quality
  control.
• Learning objectivesUpon completion of this
  course, students will have the ability to
• Conduct effective and efficient inspections
• Design and implement comprehensive test plans
• Apply a wide variety of testing techniques in an
  effective and efficient manner
• Compute test coverage and yield, according to a
  variety of criteria
• Use statistical techniques to evaluate the defect
  density and the likelihood of faults
• Assess a software process to evaluate how
  effective it is at promoting quality

20
CC-SE / SEEK A proposed course on Software
Quality Assurance (2/2)

• Suggested sequence of teaching modules
• Introduction to software quality assurance
• Inspections and reviews
• Principles of software validation
• Software verification
• Software testing
• Specification based test construction techniques
• White-box and grey-box testing
• Control flow oriented test construction
  techniques
• Data flow oriented test construction techniques
• Cleanroom approach to quality assurance
• Software process certification
• Sample labs and assignments
• Use of automated testing tools
• Testing of a wide variety of software
• Application of a wide variety of testing
  techniques
• Inspecting of software in teams comparison and
  analysis of results

21
Index

• Capability Maturity Model Integration for
  Software Engineering (CMMI-SW), Software
  Engineering Institute (SEI), Carnegie Mellon
  University (CMU)
• Computing Curricula - Software Engineering
   (CC-SE), IEEE Computer Society / ACM
• Guide to the Software Engineering Body of
  Knowledge (SWEBOK), IEEE Computer Society
• Certified Software Quality Engineer (CSQE) Body
  of Knowledge, American Society for Quality (ASQ)

22
Guide to the Software Engineering Body of
Knowledge (SWEBOK), Trial Version 1.0, 2001

• Developed by the IEEE Computer Society
• Established with the following objectives
• Promote a consistent view of software engineering
  worldwide.
• Clarify the placeand set the boundaryof
  software engineering with respect to other
  disciplines such as computer science, project
  management, computer engineering, and
  mathematics.
• Characterize the contents of the software
  engineering discipline.
• Provide a topical access to the Software
  Engineering Body of Knowledge.
• Provide a foundation for curriculum development
  and individual certification and licensing
  material.
• Describes knowledge expected after 4 years of
  experience as software engineer

23
SEWBOK - Knowledge areas (1/2)
24
SEWBOK - Knowledge areas (2/2)
25
SEWBOK Software Testing
26
SEWBOKSoftware Quality
27
SWEBOK Terminology

• Verification Validation versus Software Quality
  Assurance
• SQA governs the procedures meant to build the
  desired quality into the products by assuring
  that the process is well-planned and then applied
  as prescribed
• VV is aimed more directly at product quality, in
  that it is based on testing that can locate
  deviations and fix them. But it also validates
  the intermediate products and therefore the
  intermediate steps of the software engineering
  process
• SQA ensures that appropriate types of tests are
  planned, developed, and implemented, and VV
  develops test plans, strategies, cases and
  procedures
• Testing is a technique for evaluating product
  quality and also for indirectly improving it, by
  identifying defects and problems
• Software testing consists of the dynamic
  verification of the behavior of a program on a
  finite set of test cases, suitably selected from
  the usually infinite executions domain, against
  the specified expected behavior

28
Index

• Capability Maturity Model Integration for
  Software Engineering (CMMI-SW), Software
  Engineering Institute (SEI), Carnegie Mellon
  University (CMU)
• Computing Curricula - Software Engineering
   (CC-SE), IEEE Computer Society / ACM
• Guide to the Software Engineering Body of
  Knowledge (SWEBOK), IEEE Computer Society
• Certified Software Quality Engineer (CSQE) Body
  of Knowledge, American Society for Quality (ASQ)

29
Certified Software Quality Engineer (CSQE) Body
of Knowledge, American Society for Quality (ASQ)

• The CSQE is a professional who has comprehensive
  understanding of software quality development and
  implementation, a thorough understanding of
  software inspection, testing, verification, and
  validation, and can implement software
  development and maintenance processes and
  methods.
• Requires 8 years of experience (3 if master's or
  doctorate)

30
CSQE Body of KnowledgeI. GENERAL KNOWLEDGE,
CONDUCT, and ETHICS

• B. Standards, specifications, and models
  (Ap)Identify and use software process and
  assessment models, including ISO 9001, ISO 15504,
  IEEE software standards, IEEE/EIA 12207, SEI
  Capability Maturity Model Integrated (CMMI),
  etc., in a variety of situations.
• C. Leadership tools and skills (Ap)
• Organizational leadership
• Team management
• Team tools
• Facilitation skills
• Communication skills
• D. Ethical conduct and professional development
• ASQ Code of Ethics (E)
• Software liability and safety issues (Ap)
• Professional training and development (Ap)

• A. Quality philosophy and principles
• Benefits of software quality (C)Describe how
  software quality engineering can benefit an
  organization.
• Prevention vs. detection (C)Describe how quality
  engineering methodologies can reduce the length
  of time for testing and can influence other
  defect detection methods.
• Organizational and process benchmarking
  (An)Identify, analyze, and model best practices
  at the macro (organizational) and micro (process
  and project) levels. Identify and develop
  business objectives, use metrics to monitor their
  achievement, and provide feedback to close the
  process improvement loop.

31
CSQE Body of KnowledgeII. SOFTWARE QUALITY
MANAGEMENT - A

• A. Goals and objectives
• Quality goals and objectives (E)Describe,
  analyze, and evaluate quality goals and
  objectives for programs, projects, and products.
• Outsourced services (E)Define, analyze, and
  evaluate the impact of acquisitions,
  subcontractor services, and other external
  resources on the organization's goals and
  objectives.
• Planning (E)Identify, apply, and evaluate
  scheduling and resource requirements necessary to
  achieve quality goals and objectives.
• Software quality management (SQM) systems
  documentation (C)Identify and describe various
  elements related to SQM system documentation.
• Customer requirements (E)Analyze and evaluate
  customer requirements and their effect on
  programs, projects, and products.

32
CSQE Body of KnowledgeII. SOFTWARE QUALITY
MANAGEMENT - B

• B. Methodologies
• Review, inspection, and testing (E)Define,
  describe, evaluate, and differentiate between
  these defect detection methods.
• Change management methods (E)Identify and apply
  various methods appropriate for responding to
  changes in technology, organizations,
  environment, human performance, etc.
• Cost of quality (COQ) (An)Define, differentiate,
  and analyze COQ categories (prevention,
  appraisal, internal failure, external failure)
  and their impact on products and processes.
• Quality data tracking (E)Define, describe,
  select, and implement information systems and
  models used to track quality data in various
  situations.
• Problem reporting and corrective action
  procedures (E)Define, describe, analyze, and
  distinguish between these procedures for software
  defects, process nonconformances, and other
  quality system deficiencies.
• Quality improvement processes (E)Define,
  describe, analyze and distinguish between various
  defect prevention, detection, and removal
  processes, and evaluate process improvement
  opportunities in relation to these tools.

33
CSQE Body of KnowledgeII. SOFTWARE QUALITY
MANAGEMENT - C

• C. Audits
• Program development and administration
  (C)Identify roles and responsibilities for
  various audit participants, including team
  leader, team members, auditee, auditor, etc.
• Audit preparation and execution (C)Define and
  distinguish between various audit types,
  including process, compliance, supplier, system,
  etc. Define and describe various steps in the
  audit process, from scheduling the audit through
  the closing meeting and subsequent follow-up
  activities. Define and identify various tools and
  procedures used in conducting audits.
• Audit reporting and follow up (Ap)Identify,
  describe, and apply the steps of audit reporting
  and follow up, including the need for and
  verification of corrective action.

34
CSQE Body of KnowledgeIII. SOFTWARE ENGINEERING
PROCESSES

• A. Environmental conditions
• Life cycles
• Systems architecture
• B. Requirements management
• Requirements prioritization and evaluation
• Requirements change management
• Bi-directional requirements traceability
• C. Requirements engineering
• Requirement types
• Requirements elicitation
• Requirements analysis and modeling
• System and software requirements specifications

• D. Analysis, design, and development methods and
  tools
• Software design methods
• Types of software reuse
• Clean room and other formal methods
• Software development tools
• E. Maintenance management
• Maintenance types
• Operational maintenance

35
CSQE Body of KnowledgeIV. PROGRAM AND PROJECT
MANAGEMENT

• A. Planning
• Project planning elements
• Goal-setting and deployment
• Project planning tools
• Cost and value data
• B. Tracking and controlling
• Phase transition control techniques
• Interpreting and reporting cost of quality (COQ)
  data
• Tracking elements and methods
• Project reviews
• C. Risk management
• Risk management planning methods
• Risk probability
• Product release decisions
• Software security, safety, and hazard analysis
  issues

36
CSQE Body of KnowledgeV. SOFTWARE METRICS,
MEASUREMENT, AND ANALYTICAL METHODS A,B

• A. Metrics and measurement theory
• Definitions
• Basic measurement theory and techniques
• Psychology of metrics
• B. Process and product measurement
• Process, product, and resource metrics (Ap)
• Commonly used metrics (Ap)Define and use metrics
  to measure various aspects of software, including
  software complexity, lines of code (LOC),
  non-commented lines of code (NCLOC), design
  defects, requirements volatility, system
  performance, etc.
• Software quality attributes (C)Identify and
  describe various criteria for measuring
  attributes such as maintainability,
  verifiability, reliability, usability,
  reusability, testability, expandability, etc.
• Defect detection effectiveness measures
  (Ap)Define, describe, and use defect detection
  measures such as cost, yield, customer impact,
  etc., and track their effectiveness.

37
CSQE Body of KnowledgeV. SOFTWARE METRICS,
MEASUREMENT, AND ANALYTICAL METHODS - C

• C. Analytical techniques
• Data integrity (S)Define, use, and interpret
  various techniques to ensure the quality of
  metrics data, its accuracy, completeness,
  timeliness, etc.
• Quality tools (An)Define, select, and use
  quality analysis and problem-solving tools such
  as flow charts, Pareto charts, cause and effect
  diagrams, check sheets, scatter diagrams, control
  (run) charts, histograms, root cause analysis,
  affinity diagrams, tree diagrams, process
  decision program charts (PDPCs), matrix diagrams,
  interrelationship digraphs, prioritization
  matrices, activity network diagrams.
• Sampling theory and techniques (An)Describe,
  differentiate, and analyze various sampling
  techniques for use in auditing, testing, product
  acceptance, etc.

38
CSQE Body of KnowledgeVI. SOFTWARE VERIFICATION
AND VALIDATION (VV) - A

• A. Theory
• VV planning procedures and tasks (S)Identify
  and select various methods for verification and
  validation, including static analysis, structural
  analysis, mathematical proof, simulation, etc.
  Identify and analyze which tasks should be
  iterated as a result of proposed or completed
  modifications.
• VV program (An)Describe and analyze methods for
  managing and reviewing a VV program, including
  technical accomplishments, resource utilization,
  program status, etc.
• Evaluating software products and processes
  (S)Analyze and select various ways of evaluating
  documentation, source code, test and audit
  results, etc., to determine whether user needs
  and project objectives have been satisfied.
• Interfaces (C)Identify various interfaces used
  with hardware, user, operator, and software
  applications.
• Program performance and process effectiveness
  (An)Identify and use various methods of
  examining performance and effectiveness.

39
CSQE Body of KnowledgeVI. SOFTWARE VERIFICATION
AND VALIDATION (VV) - B

• B. Reviews and inspections
• Types (Ap)Define, describe, and use various
  types of reviews and inspections, including
  desk-checking, walk-throughs, Fagan and Gilb
  inspections, technical accomplishments, resource
  utilization, future planning, etc.
• Items (Ap)Identify, describe, and use various
  review and inspection items, including proposals,
  project charters, specifications, code, tests,
  etc.
• Processes (Ap)Define, describe, and use various
  review and inspection processes to examine
  objectives, criteria, techniques, methods, etc.
• Data collection, reports, and summaries
  (Ap)Define, describe, and use terms related to
  data collection, including preparation rates,
  defect density yield, phase containment, etc.

40
CSQE Body of KnowledgeVI. SOFTWARE VERIFICATION
AND VALIDATION (VV) - C

• C. Test planning and design
• Types of tests (S) Select, apply, and develop
  various types of test, including functional,
  performance, regression, certification,
  environmental load, stress, worst case,
  perfective, exploratory, etc.
• Test tools (C)Define and describe the
  application and capabilities of commonly used
  test tools such as acceptance test suites,
  utilities (for memory, screen capture,
  string-finding, file viewer, file comparison,
  etc.), and diagnostics (for hardware, software,
  configuration, etc.).
• Test strategies (S)Identify, analyze, and apply
  various test strategies, including top-down,
  bottom-up, black-box, white-box, simulation,
  automation, etc.
• Test design (Ap)Identify, describe, and apply
  various types of test design including fault
  insertion, fault-error handling, equivalence
  class partitioning, boundary value, etc.

41
CSQE Body of KnowledgeVI. SOFTWARE VERIFICATION
AND VALIDATION (VV) - C

• C. Test planning and design (cont.)
• Test coverage of specifications (S)Identify,
  apply, and develop various test coverage
  specifications, including functions, states, data
  and time domains, etc.
• Test environments (S)Identify various
  environments and use tools such as test
  libraries, drivers, stubs, harnesses, etc., in
  those environments, and describe how simulations
  can be used in test environments.
• Supplier components and products (Ap)Identify
  the common risks and benefits of incorporating
  purchased software into other software products.
  Use various methods to test supplier components
  and products in the larger system.
• Test plans (Ap)Identify, describe, and apply
  methods for creating and evaluating test plans
  including system, acceptance, validation, etc.,
  to determine whether project objectives are being
  met.

42
CSQE Body of KnowledgeVI. SOFTWARE VERIFICATION
AND VALIDATION (VV) - D

• D. Test execution and evaluation
• Test implementation (Ap)Define, describe, and
  use various implementation elements, including
  scheduling, freezing, dependencies, V-model,
  error repair models, acceptance testing, etc.
• Test documentation (Ap)Define, describe, and use
  various documentation procedures, including
  defect recording and tracking, test report
  completion metrics, trouble reports, input/output
  specifications, etc.
• Test reviews (S)Describe, develop, and analyze
  various methods of reviewing test efforts,
  including technical accomplishments, future
  planning, risk management, etc.
• Code coverage metrics (Ap)Define and apply
  various metrics including branch-to-branch,
  condition, domain, McCabe's cyclomatic
  complexity, boundary, etc.
• Customer deliverables (S)Identify and select
  various methods for testing the accuracy of
  customer deliverables, including packaged or
  downloaded products, license keys, user
  documentation, marketing and training materials,
  etc.
• Severity of anomalies (E)Identify and select
  various methods for evaluating severity of
  anomalies in software operations.

43
CSQE Body of KnowledgeVII. SOFTWARE
CONFIGURATION MANAGEMENT

• D. Configuration status accounting
• Status reporting
• Changes to configuration items and baselines
• Documentation control
• E. Configuration audits
• Functional configuration audit
• Physical configuration audit
• F. Release and distribution issues
• Product release process issues
• Packaging, production, and distribution

• A. Configuration infrastructure
• Configuration management
• Library/repository processes
• Defect tracking and library tools
• B. Configuration identification
• Configuration items
• Baselines
• Configuration identification methods
• Software builds
• C. Configuration control
• Item and baseline control
• Proposed modifications
• Review and configuration control boards (CCBs)
• Concurrent development
• Traceability
• Version control
• Configuration item interfaces

44
CSQE Body of KnowledgeLevels of Cognition
(Bloom's Taxonomy)

• Knowledge Level (K) - Being able to remember or
  recognize terminology, definitions, facts, ideas,
  materials, patterns, sequences, methodologies,
  principles, etc.
• Comprehension Level (C) - Being able to read and
  understand descriptions, communications, reports,
  tables, diagrams, directions, regulations, etc.
• Application Level (Ap) - Being able to apply
  ideas, procedures, methods, formulas, principles,
  theories, etc., in job-related situations.
• Analysis (An) - Being able to break down
  information into its constituent parts and
  recognize the parts relationship to one another
  and how they are organized identify sublevel
  factors or salient data from a complex scenario.
• Synthesis (S) - Being able to put parts or
  elements together in such a way as to show a
  pattern or structure not clearly there before
  identify which data or information from a complex
  set is appropriate to examine further or from
  which supported conclusions can be drawn.
• Evaluation (E) - Being able to make judgments
  regarding the value of proposed ideas, solutions,
  methodologies, etc., by using appropriate
  criteria or standards to estimate accuracy,
  effectiveness, economic benefits, etc.

45
References and further reading

• http//www.sei.cmu.edu/cmmi/
• Capability Maturity Model Integration for
  Software Engineering (CMMI-SW), Software
  Engineering Institute (SEI), Carnegie Mellon
  University (CMU)
• http//sites.computer.org/ccse/
• Computing Curricula - Software Engineering
   (CC-SE), IEEE Computer Society / ACM
• http//www.swebok.org/
• Guide to the Software Engineering Body of
  Knowledge (SWEBOK), IEEE Computer Society
• http//www.asq.org/cert/types/csqe/index.html
• Certified Software Quality Engineer (CSQE) Body
  of Knowledge, American Society for Quality (ASQ)