Course Topics

1
Course Topics
2
Topic Objectives

• By the end of this topic, you should be able to
• Define quality and identify standard quality
  attributes
• Describe usability objectives, importance, and
  design process
• Apply the accessibility litmus test
• Recognize the types and goals of technical
  reviews

3
Topic Outline

• The concept of quality
• Standard quality attributes
• Usability objectives, importance, and design
  process
• The importance of accessibility
• Quality review opportunities

4
How is Quality Defined?

• A characteristic or attribute of something a
  degree or grade of excellence
• Elusive term in the software industry
• Multiple definitions and multiple sources
• International Standards Organization (ISO)

5
Is Quality Defined By?

• User satisfaction
• Meets requirements delivered on time
  reasonable cost quality product
• Fitness of purpose
• End product is capable of performing the assigned
  task
• The implemented system works, and works correctly
  in all of the required situations
• Performs the tasks in specified manner and within
  the constraints of the specified resources

Source Glass, Robert L. Facts and Fallacies of
Software Engineering. Addison-Wesley
Professional, 2003.
6
Can Quality be Measured?

• U.S. Department of Defense study
• Bowen, Wigle and Tsai, 1985
• Many attributes comprise software quality
• Not all quality attributes are quantifiable

Source Glass, Robert L. Facts and Fallacies of
Software Engineering. Addison-Wesley
Professional, 2003.
7
Topic Outline

• The concept of quality
• The quality attribute standards
• Usability objectives, importance, and design
  process
• The importance of accessibility
• Quality review opportunities

8
McCalls Quality Attributes
Source McCall, Richards and Walters. Factors in
Software Quality. November 1977.
9
The ISO Standards on Quality

• Derived from McCalls model
• ISO/IEC 9126 is an international standard for the
  evaluation of software quality
• Divided into 4 parts
• Quality model - ISO/IEC 9126-1, 2001
• External metrics ISO/IEC 9126-2, 2003
• Internal metrics ISO/IEC 9126-3, 2003
• Quality in use metrics ISO/IEC 9126-4, 2004

10
The ISO Standards Quality Model

• ISO/IEC 9126-1 Quality Attributes
• Functionality
• Reliability
• Efficiency
• Maintainability
• Portability
• Usability

11
ISO 9126-1 Quality Attributes

• Functionality
• The degree to which the software satisfies stated
  needs
• Reliability
• The amount of time the software is available for
  use
• Efficiency
• The degree to which the software makes optimal
  use of the system resources

Source International Organization for
Standardization. ISO/IEC 9126-1 Quality Model
12
ISO 9126-1 Quality Attributes, cont.

• Maintainability
• The ease with which repair may be made to the
  software
• Portability
• The ease with which the software can be
  transposed from one environment to another

Source International Organization for
Standardization. ISO/IEC 9126-1 Quality Model
13
ISO 9126-1 Quality Attributes, cont.

• Usability
• The extent to which a product can be used by
  specified users to achieve specified goals with
  effectiveness, efficiency and satisfaction

Source International Organization for
Standardization. ISO/IEC 9126-1 Quality Model
14
Think Time Exercise

• Do you think ISO 9126-1 is the right list of
  software quality attributes?
• Do you think there is a priority order to those
  quality attributes?
• How does your organization define and measure
  software quality?

15
Topic Outline

• The concept of quality
• The quality attribute standards
• Usability objectives, importance, and design
  process
• The importance of accessibility
• Quality review opportunities

16
Usability Objectives

• Learnability
• Efficiency
• Effectiveness
• Memorability
• Error Handling and Recovery
• Satisfaction
• Flexibility
• Tailorabilty

Source Adapted from EVANTAGE Consulting.
Understanding Usability Objectives. September
2005.
17
Usability Objectives, cont.

• Learnability
• Users are able to learn the system quickly and
  gain knowledge about deeper functionality over
  time
• Efficiency
• The resources consumed to achieve those goals are
  at an acceptable and accurate level
• Effectiveness
• Users achieve the right goals they set out to
  achieve in the system

Source Adapted from EVANTAGE Consulting.
Understanding Usability Objectives. September
2005.
18
Usability Objectives, cont.

• Memorability
• Users can return from a break and still know
  where they are in the system and how to use it
• Error Handling and Recovery
• The system limits the errors a user encounters
  and helps them recover when they occur
• Satisfaction
• The system is engaging and users have a positive
  attitude

Source Adapted from EVANTAGE Consulting.
Understanding Usability Objectives. September
2005.
19
Usability Objectives, cont.

• Flexibility
• Sites/groups have the ability to customize the
  system (within established constraints) to
  accommodate differences
• Tailorabilty
• Users have the ability to customize the user
  interface (UI) to accommodate their specific work
  responsibilities and priorities

Source Adapted from EVANTAGE Consulting.
Understanding Usability Objectives. September
2005.
20
Importance of Usability Objectives

• Usability objectives are important because
  users fail when
• They cannot find the data they want to work with.
• They cannot navigate to the functionality they
  need.
• The functionality does not map to their own
  mental model of how a process should work.

Amateurish graphics and poor layouts are the
largest source of annoyance. A first first
impression is formed in the first 50
milliseconds. A lasting first impression is
formed in 500 milliseconds.
Source Bell, Kevin. Usability Services and
Features. National IT Managers Meeting. May 2008.
21
An Implicit Bargain with Your Users

• You want more quality data to drive
• Accountability
• Compliance
• Outcome measurement.
• Your staff doesnt like to do data entry but
  they agree to enter quality data to the extent
  they are not annoyed or confused.
• A usable system increases the amount of
  quality data you can get before annoyance and
  confusion set in.

Usability
Source Bell, Kevin. Usability Services and
Features. National IT Managers Meeting. May
2008.
22
The Bottom Line
One dollar spent on usability during design
returns at least ten dollars later, and sometimes
as much as one hundred dollars!
Source Bell, Kevin. Usability Services and
Features. National IT Managers Meeting. May
2008.
23
Getting to the Bottom Line

• Include Usability requirements in a Request for
  Proposal (RFP).
• Do not include bells and whistles that will be
  used by the minority.
• ACF assesses your system on the degree it is used
  to support case workers, not by the number of
  fancy features.
• Embed a formal usability design process in the
  software development lifecycle.
• Requires investment in additional steps
• Demands engagement from user community

24
A Usability Design Process

ANALYSIS PHASE
(evaluation)
1. Analysis of User
2. Analysis of Task
3. Usability benchmarks
DESIGN PHASE
(evaluation)
4. Conceptual Design
5. Detail Design
Usability Testing
Source IEEE Software Magazine. Usability Basics
for Software Developers. January/February 2001.
25
Design Process Analysis Phase

• User Analysis
• Identify needs, expectations, interests,
  behaviors and responsibilities
• Site visits, focus groups
• Record and organize findings

Source IEEE Software Magazine. Usability Basics
for Software Developers. January/February 2001.
26
Design Process Analysis Phase

• Task Analysis
• Describes a set of techniques people use to get
  things done
• Tasks become the basis for building the usability
  specifications and testing scenarios
• Tasks are used to drive and test the UI design
  throughout the development lifecycle

Source IEEE Software Magazine. Usability Basics
for Software Developers. January/February 2001.
27
Design Process Analysis Phase

• Usability Benchmarks
• Benchmarks become usability goals defined before
  system design begins
• Based upon the your usability objectives
• Define a set of benchmarks for each usability
  objective you want to evaluate
• Use the benchmarks to drive usability testing

Source IEEE Software Magazine. Usability Basics
for Software Developers. January/February 2001.
28
Sample Usability Benchmark Table
29
Sample Usability Benchmark Table
30
Sample Usability Benchmark Table
Source http//en.wikipedia.org/wiki/Flesch-Kincai
d_Readability_Test Source Adapted from EVANTAGE
Consulting. Understanding Usability Objectives.
September 2005.
31
Sample Benchmark Table, cont.
Source Adapted from EVANTAGE Consulting.
Understanding Usability Objectives. September
2005.
32
Design Process Design Phase

• Conceptual Design
• Defines the basic user-system interaction
• Findings of the user and task analysis are basis
  of conceptual design
• Deliverables are typically paper prototypes

Source IEEE Software Magazine. Usability Basics
for Software Developers. January/February 2001.
33
Design Process Design Phase

• Visual Design
• The UIs appearance is defined
• Layout of screens and dialog boxes
• Colors, graphics and icons
• Prototypes
• Scenarios
• Storyboards
• Snapshots

Source IEEE Software Magazine. Usability Basics
for Software Developers. January/February 2001.
34
Usability Design Process Summary
User Interface Design
User Task Analysis
Usability testing
Easy-to-Use System
Source Adapted from EVANTAGE Consulting.
Understanding Usability Objectives. September
2005.
35
Usability Features and Services
Source Bell, Kevin. Usability Services and
Features. National IT Managers Meeting. May
2008.
36
Think Time Exercise

• You are designing a User Interface (UI).
• You can pick 3 priority usability objectives.
• Learnability
• Efficiency
• Effectiveness
• Memorability
• Error Handling and Recovery
• Satisfaction
• Flexibility
• Tailorabilty
• Which 3 will you pick, and why?

37
Topic Summary

• Quality consists of several attributes.
• Usability objectives are key to forming a sound
  user strategy.
• Understanding the tasks that a user is expected
  to perform is essential in validating that the
  usability objectives were met.
• Usability design is an organizational process
  requiring additional investment.

38
Topic Outline

• The concept of quality
• The quality attribute standards
• Usability objectives, importance, and design
  process
• The importance of accessibility
• Quality review opportunities

39
Accessibility in Design Why?

• Rehabilitation Act Section 508
• Section 508 is a federal law
• Passed in August, 1998 and took effect June 2001
• Eliminate barriers in IT
• New opportunities for people with disabilities
• FACT 1 in 5 Americans have some form of
  disability
• Encourage technology development to achieve those
  goals

Source Administration for Children and
Families. http//www.acf.hhs.gov/acf_accessibility
.html Source U.S. General Services
Administration. http//www.section508.gov
40
Accessibility in Design What?

• What is accessibility?
• The degree to which software can be used
  comfortably by a wide variety of people,
  including those who require assistive
  technologies like screen magnifiers or voice
  recognition
• What is an accessible application?
• An accessible application can be used by anyone,
  regardless of disability.

41
Accessibility in Design Who?

• Who does it apply to?
• Government entities who procure, develop,
  maintain, or use electronic or information
  technology
• What is the intent?
• Must give disabled employees and the public
  access to information that is comparable to the
  access available to others
• Is there legal liability?

42
Accessibility in Design You?

• What is your States compliance requirement?
• States have adopted provisions of Section 508 for
  web based application development.
• State examples
• http//accessibility.gtri.gatech.edu/sitid/stateLa
  wAtGlance.php
• http//www.dir.state.tx.us/standards/srrpub11-acce
  ssibility.htm
• http//enterprise.state.wi.us/home/standards/std60
  5r.htm

43
The Spirit of Section 508

• Basic questions to ask
• Can you complete the tasks of the application
  without a mouse?
• Can you complete the tasks with your monitor
  turned off?
• Can you complete the tasks with the speakers
  turned off?

Source Administration for Children and Families.
http//www.acf.hhs.gov/acf_accessibility.html
44
Accessibility in Design Guidelines

• Publication
• Research-Based Web Design and Usability
  Guidelines
• Response to the Presidents Management Agenda and
  the E-government Act of 2002
• US Department of Health and Human Services
• www.usability.gov/pdfs/guidelines.html
• http//www-03.ibm.com/able/guidelines/

45
Accessibility in Design Testing

• Accessibility Checking Software
• Rational Policy Checker Accessibility Edition
  from IBM
• Formerly Bobby and WebXM from Watchfire
• InFocus from SSB Technologies
• The LIFT Machine from UsableNet
• Ramp Ascend from Deque
• WebKing from Parasoft

Source Web Accessibility Initiative Evaluation
and Repair Tools Working Group. http//www.jimthat
cher.com/testing2.htm
46
Topic Summary

• Accessibility is the law.
• Most states/counties have statutes, policies or
  guidelines.
• There are readily available design guidelines.
• Conformance to accessibility standards is
  critical.

47
Topic Outline

• The concept of quality
• Quality attributes and outcomes
• The importance of accessibility
• Quality review opportunities

48
Technical Reviews History

• Started about the same time as Systems
  Engineering, in 1960
• The concept was formalized in 1972.
• Still around because there is evidence that they
  help produce a quality system at less cost
• Many different types and names of reviews

49
Technical Reviews Benefits

• Ensure the requirements are correct, complete and
  verifiable
• Monitor program progress and risks
• Assess maturity level of the development effort
• Allow recommending start of next phase
• Facilitate approval for committing additional
  resources

50
Technical Reviews Format

• Reviews should be formal
• Results should be documented
• A senior analyst is generally responsible for
  initiating and conducting the reviews
• Conducted throughout the software development
  life cycle

51
Technical Reviews Pre-Design

• Project Definition Review (PDR)
• Project objectives (including scope, estimated
• resource needs, costs, and timeline)
• Top-level functional requirements
• Candidate architecture
• Measures of effectiveness
• Interfaces
• Alternative or unique concepts or requirements
• Anticipated risks

52
Technical Reviews During Design

• System Requirements Review
• System Functional Review
• Software Specifications Review
• Preliminary Design Review
• Critical Design Review

53
Technical Reviews During Design

• System Requirements Review (SRR)
• Requirements were analyzed and translated into
  system-specific functions and performance
• Technology validation and demonstration plans
  completed
• Critical technologies have been identified and
  assessed
• Risks are identified, quantified and risk
  mitigation is proceeding

54
Technical Reviews During Design

• System Functional Review (SFD)
• System and performance requirements have
  converged
• System physical architecture demonstrates it will
  meet expected deliverable
• Readiness to initiate preliminary design

55
Technical Reviews During Design

• Software Specifications Review (SSR)
• Conducted when the system-level software
  requirements have been defined and allocated
• Establishes a formal baseline for software
  configuration items
• Demonstrates adequacy of the software and
  interface requirements specifications
• Demonstrates adequacy of the operational concepts
  document

56
Technical Reviews During Design

• Preliminary Design Review (PDR)
• The design implements all the functions
• The system requirements are completely defined
• Sufficient design has been accomplished to verify
  the completeness and achievability of the
  requirements
• Risk handling approach is reasonable
• Criteria and metrics support continued effort

57
Technical Reviews During Design

• Critical Design Review (CDR)
• System design in full detail
• Resolution of technical problems
• Technical performance measures
• Development plan and timelines
• Interface design in detail

58
Technical Reviews Post Design

• Code Inspection or Walkthrough
• Involves going over software to identify errors
  or other problems
• Generally conducted by software developers for
  software developers
• Can remove up to 90 of errors from a software
  product

Source Glass, Robert L. Facts and Fallacies of
Software Engineering. Addison-Wesley
Professional, 2003.
59
Topic Summary

• Reviews keep tab on requirements and control
  scope.
• Reviews assist in assessing the design product
  and the design process.
• Reviews assist in achieving a high quality end
  product.
• Reviews are both technical and sociological.

60
Team Exercise Group 1

• Proper conduct of a technical review
• Did the facilitator (Annette) do her job of
  facilitating well?
• How should this review have been handled
  differently?
• What should Annette do now?

61
Team Exercise Group 2

• Software development as a team activity
• What was the purpose of Mikes code?
• What impact did Mikes code have on the rest of
  the system?
• How do the standards contribute to teamwork and
  systems development?

62
Team Exercise Group 3

• Technical reviews as a part of design
  verification
• What problems did the reviewers have with Mikes
  code?
• Do you believe that Mike was correctly following
  standards? Why or why not?