Barry Boehm University of Southern California Richard Turner OUSDAT

1
Barry BoehmUniversity of Southern
CaliforniaRichard TurnerOUSD(ATL)/DS/SE
(George Washington University)STC 2004
TutorialApril 19, 2004
Balancing Agility and Discipline Evaluating and
Integrating Agile and Plan-Driven Methods
Based on Balancing Agility and Discipline A
Guide for the Perplexed,B. Boehm and R. Turner,
Addison Wesley, 2004
2
Outline

• Tutorial learning objectives
• Survey of assumptions about participants
• General software trends and implications
• Sources of perplexity about agile, plan-driven
  methods
• Overview of agile and plan-driven methods
• XP, TSP days in the life
• Comparisons of differences, strengths, weaknesses
• Risk-based balance of agility and discipline
• Small, medium, large examples
• Observations and Way Forward
• Hands-on exercise
• Conclusions review of learning objectives

3
Tutorial Learning Objectives

• Practitioners and managers
• Understand agile, plan-driven method strengths,
  difficulties
• Learn risk-based approach to tailor hybrid
  methods
• Practice in formulating organizational strategy
• Researchers
• Understand research issues, opportunities
• Techniques for achieving rapid quality
• Empirical analyses
• Educators
• Understand differences in agile, plan-driven
  approaches
• Understand teaching challenges, opportunities
• Survey courses, project courses

4
Assumptions About Participants - Surveyed at
Tutorial

• Generally familiar with plan-driven methods
• Waterfall, incremental, spiral, RUP, PSP/TSP
• Software CMM, CMMI, SPICE, ISO I2207
• Less familiar with agile methods
• XP, Scrum, Crystal, DSDM, FDD
• Representative of mixed domains
• Large/small projects, research, education
• Variety of software applications domains
• Increasing need for high software dependability
• Increasing software complexity, speed of change

5
Information Technology Trends

• Current/Future Trends
• Everything connected (maybe)
• Rapid requirements change
• COTS capabilities determine rqts.
• No control over COTS evolution
• Ever-decreasing cycle times
• Outsourced jobs
• Failures critical
• Complex, adaptive, emergent systems of systems
• Adaptive process models

• Traditional Development
• Standalone systems
• Stable requirements
• Rqts. determine capabilities
• Control over evolution
• Enough time to keep stable
• Stable jobs
• Failures noncritical
• Reductionist systems
• Repeatability-oriented process, maturity models

6
Background

• Two approaches to software development
• Disciplined (SW-CMM, document-based, heavy
  process)
• Agile (XP, tacit knowledge, light process)
• Both have strengths and weaknesses
• Agile and disciplined proponents are believers
• Many of us are perplexed

7
Key Definitions

• Agile method
• one which fully adopts the four value
  propositions and twelve principles in the Agile
  Manifesto.
• Discipline (per Webster)
• control gained by enforcing obedience or order
• orderly or prescribed conduct or pattern of
  behavior
• self-control.
• Plan-driven
• a description for disciplined methods (order is
  often defined in plans)
• Plan (per Webster)
• a method for achieving an end (a process plan)
• an orderly arrangement of parts of an overall
  design (a product plan).
• We assume that such plans are documented.

8
The Agile Manifesto
We are uncovering better ways of developing
software by doing it and helping others do
it. Through this work we have come to value

• Individuals and interactions over processes and
  tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

That is, while there is value in the items on
the right, we value the items on the left more.
9
Plan-Driven Methods Are Evolving Too

• Evolutionary acquisition and spiral development
• U.S. DoD Directive 5000.1, Instruction 5000.2
• Risk-driven level of detail
• Of processes peer reviews vs. Critical Design
  Reviews
• Of products GUI prototypes vs. specifications
• Of heavyweight methods Team Software Process,
  Rational Unified Process

10
Sources of Perplexity

• Distinguishing method use from method misuse
• Citing method misuse vs. intended use
• Claiming XP use when simply not documenting
• CMM Level 4 Memorial Library of 99 2-inch
  binders
• Overgeneralization based on the most visible
  instances
• XP is Agile CMM is Plan-Driven
• Multiple definitions
• Quality customer satisfaction or compliance?
• Claims of universality
• The pace of IT change is accelerating and Agile
  methods adapt to change better than disciplined
  methods therefore Agile methods will take over
  the IT world.
• Software development is uncertain and the SW-CMM
  improves predictability therefore all software
  developers should use the SW-CMM

11
Sources of Perplexity (2)

• Early success stories
• Chrysler project that successfully birthed XP was
  later cancelled
• Cleanroom has never made it into the mainstream
• Purist interpretations (and internal
  disagreements)
• Don't start by incrementally adopting parts of
  XP. Its pieces fit together like a fine Swiss
  watch
• "An advantage of agile methods is that you can
  apply them selectively and generatively."
• "If you aren't 100 compliant with SW CMM Level
  3, don't bother to bid"
• "With the CMMI continuous interpretation, you can
  improve your processes in any order you feel is
  best."

12
Outline

• Tutorial learning objectives
• Survey of assumptions about participants
• General software trends and implications
• Sources of perplexity about agile, plan-driven
  methods
• Overview of agile and plan-driven methods
• XP, TSP days in the life
• Comparisons of differences, strengths, weaknesses
• Risk-based balance of agility and discipline
• Small, medium, large examples
• Observations and Way Forward
• Hands-on exercise
• Conclusions review of learning objectives

13
Various Agile Methods Available

• Adaptive Software Development (ASD)
• Agile Modeling
• Crystal methods
• Dynamic System Development Methodology (DSDM)
• eXtreme Programming (XP)
• Feature Driven Development
• Lean Development
• Scrum

14
XP Principles I

• Philosophy Take known good practices and push
  them to extremes
• If code reviews are good, well review code all
  the time
• If testing is good, well test all the time
• If design is good, well make it part of
  everybodys daily business

15
XP Principles II

• If simplicity is good, well always leave the
  system with the simplest design that supports its
  current functionality
• If architecture is important, everybody will
  work defining and refining the architecture all
  the time
• If integration testing is important, then well
  integrate and test several times a day

16
XP Principles III

• If short iterations are good, well make the
  iterations really, really short seconds and
  minutes and hours, not weeks and months and
  years
• If customer involvement is good, well make them
  full-time participants

17
XP The 12 Practices-Kent Beck, Extreme
Programming Explained, Addison Wesley, 1999

• The Planning Game
• Small Releases
• Metaphor
• Simple Design
• Testing
• Refactoring

• Pair Programming
• Collective Ownership
• Continuous Integration
• 40-hour Week
• On-site Customer
• Coding Standards

-Used generatively, not imperatively
18
XP Characteristics
19
The Team Software Process (TSP)-Watts Humphrey,
Introduction to the ISP, Addison Wesley, 2000

• Scale-up of Personal Software Process (PSP)
• To about 20-person teams
• Strong emphasis on planning and control
• 21 scripts, 5 roles/activities, 21 forms
• Risk-driven tailoring can make process more agile

20
Example TSP Role Script
21
TSP Characteristics
22
Days In The Life XP and TSP

• Comparison of two development teams
• Product
• Tool to process complex sales report generated by
  legacy system
• Prototype delivered
• Task is to enhance and port prototype
• 8 month projected schedule
• Estimated size 20,000 SLOC

23
Background Information

• TSP/PSP
• Training
• Each member has 125-hour PSP for Engineers
• 2 Members 5-day launch coach training
• Tools and environment
• Web-based TSP data collection tool
• OO tools
• Individual offices and conference room
• Project planning
• 4-day launch session
• Develop/tailored all processes
• 180 tasks identified and plan accepted by
  stakeholders

• XP
• Training
• 2 members attended 40-hour XP workshop
• In-house presentations and mentoring
• Tools and environment
• Automated test suite development tool
• OO tools
• Bullpen and Conference room/lounge
• Project planning
• 1-day customer exploration
• 2-day follow-up to prioritize
• Agreed on 2-week iteration length, 3-iteration
  release length, and release schedule

24
Normal and Crisis Days

• Used Friday as common day
• Crisis days are plan-breakers
• New requirements
• Short schedule
• Please read the handouts and well discuss

25
Comparison

• Differences
• Depth and scope of metrics
• Specificity of process
• Level and scope of reporting
• Customer interface
• Similarities
• Collaborative teams
• Well-defined roles
• Spiral, risk- and increment-driven process
• Measurement
• Test first (test-driven) approach
• Bottom Line
• Both were able to confidently push-back when
  necessary

26
Outline

• Tutorial learning objectives
• Survey of assumptions about participants
• General software trends and implications
• Sources of perplexity about agile, plan-driven
  methods
• Overview of agile and plan-driven methods
• XP, TSP days in the life
• Comparisons of differences, strengths, weaknesses
• Risk-based balance of agility and discipline
• Small, medium, large examples
• Observations and Way Forward
• Hands-on exercise
• Conclusions review of learning objectives

27
Finding Middle Ground

• A pragmatic view
• Both approaches have home grounds
• A broad range of environments and needs
• Trends require better handling of rapid change
• Processes should be the right weight for the job
• We believe risk-based analysis is the key to
  finding middle ground

28
Contrasts and Home Grounds

• Comparison is difficult, but we found 4 areas
  where there are clear differences
• Application
• Project goals, size, environment velocity
• Management
• Customer relations, planning and control,
  communications
• Technical
• How the software is developed
• Personnel
• The type and competency of developers and
  stakeholders

29
Application Characteristics

• Primary goals
• A Rapid value, responsiveness to change
• P Predictability, stability, high assurance
• Size
• A Small to medium
• P Large
• Environment
• A Turbulent, high change
• P Stable, requirements defined
• Project velocity
• A Code, learn, and fix
• P Architect for parallel, incremental development

30
Management Characteristics

• Customer Relations
• A Dedicated, collocated where feasible
• P Contractual increasingly evolutionary
• A Trust through working software and
  participation
• P Trust through process maturity evaluations
• Planning and Control
• A Means to an end
• P Anchor processes, communication
• Project Communications
• A Tacit, interpersonal knowledge
• P Explicit, documented knowledge

31
Technical Characteristics

• Requirements
• A Adjustable, informal stories
• P Formally baselined, complete, consistent
  specifications
• Development
• A Simple Design
• P Architecture-based design
• Testing
• A Automated, test-driven
• P Planned, requirements-driven

32
Technical Characteristics (2)Cost of Change
Beck, Li
Beck
Li
Li
33
Technical Characteristics (3) Cost of change
Poppendieck

• At least two cost-escalation curves
• Lean development Architect to defer high-stakes
  decisions e.g. COTS
• Easier to change an unmade decision

34
Technical Characteristics (4)Cost of Change TRW
Beck
Li
35
Technical Characteristics (5)How Much
Architecting?
Beck
10,000 KSLOC
Sweet spot
100 KSLOC
Liu
10 KSLOC
36
Personnel Characteristics

• Customers
• A CRACK customers throughout development
• P CRACK customers early
• CRACK Collaborative, Representative, Authorized,
  Committed, and Knowledgeable
• Developers
• A Heavy mix of high caliber throughout
• P Heavy mix early with lower capability later
• Culture
• A Many degrees of freedom (Thrives on chaos)
• P Clear policies and procedures (Thrives on
  order)
• Collocated customers are difficult to
  find/keep/keep current on either side

37
Personnel Characteristics (2)Modified Cockburn
Levels
Level Characteristics 3 Able to revise a method
(break its rules) to fit an unprecedented new
situation 2 Able to tailor a method to fit a
precedented new situation 1A With training, able
to perform discretionary method steps (e.g.,
sizing stories to fit increments, composing
patterns, compound refactoring, complex COTS
integration). With experience can become Level
2. 1B With training, able to perform procedural
method steps (e.g. coding a simple method, simple
refactoring, following coding standards and CM
procedures, running tests). With experience can
master some Level 1A skills. -1 May have
technical skills, but unable or unwilling to
collaborate or follow shared methods.
38
Summary of Home Grounds
39
Two Case Studies

• Show how that agile and plan-driven methods can
  be extended
• Provide examples of success and lessons learned
• ThoughtWorks Lease Management
• Agile extended with plan-driven
• CCPDS-R
• Plan-driven overlaid with agile concepts

40
Thoughtworks Lease Management

• XP replaced ineffective traditional development
• Problems when project moved beyond XP assumptions
• The effort to develop or modify a story really
  does not increase with time and story number
• Trusting people to get everything done on time is
  compatible with fixed schedules and diseconomies
  of scale
• Simple design and YAGNI scale up easily to large
  projects
• Disciplined practices enabled XP to scale up
• High-level architectural plans to provide
  essential big-picture information
• More careful definition of milestone completion
  criteria to avoid finishing but not finishing
• Use of design patterns and architectural
  solutions rather than simple design to handle
  foreseeable change

41
CCPDS-R

• USAF Command Center Processing and Display System
  Replacement for early missile warning
• Government contract environment required
  plan-driven approach, project needed agility
• Practices implemented to provide agility mapped
  well to the Agile Manifesto
• Individuals and Interactions over Processes and
  Tools
• Milestone content was redefined
• Architecture was organized around the performers
  skill levels
• Working Software over Comprehensive Documentation
• Later PDR demonstrated working software for
  high-risk areas
• Customer Collaboration Over Contract Negotiation
• Used COCOMO for cost and schedule negotiations
• Responding to Change Over Following a Plan
• Automated plans
• Architecture for re-use saved effort

42
CCPDS-R (2) Cost of Change
Beck
43
Five Critical Decision Factors

• Size, Criticality, Dynamism, Personnel, Culture

44
Comparing the Case Study Projects
45
Misconceptions
46
Outline

• Tutorial learning objectives
• Survey of assumptions about participants
• General software trends and implications
• Sources of perplexity about agile, plan-driven
  methods
• Overview of agile and plan-driven methods
• XP, TSP days in the life
• Comparisons of differences, strengths, weaknesses
• Risk-based balance of agility and discipline
• Small, medium, large examples
• Observations and Way Forward
• Hands-on exercise
• Conclusions review of learning objectives

47
Using Risk to Balance Discipline and Agility -
Overview
48
Risks Used in Risk Comparison

• Environmental risks
• Technology uncertainties
• Many stakeholders
• Complex system-of-systems
• Legacy compatibility
• Agility risks
• Scalability
• Use of simple design
• Personnel turnover
• Too-frequent releases
• Not enough agile-capable people
• Plan-driven risks
• Rapid change
• Need for rapid results
• Emergent requirements
• Not enough discipline-capable people

49
Three Examples

• Agent-based systems
• Small Event Managers application
• Medium SupplyChain.com application
• Large National Information System for Crisis
  Management (NISCM) application
• Each example results in a development strategy
  based on risk analyses

50
Event Managers Project

• Small startup company
• Diverse set of smaller agent-based planning
  applications
• This project support for managing the
  infrastructure and operations of conferences and
  conventions
• Widening variety of options and
  interdependencies, makes an agent-based approach
  highly attractive

51
Event Managers Profile
52
Event Managers Strategy
LCA
53
SupplyChain.com Profile

• Turnkey agent-based supply chain management
  systems
• Distributed, multi-organization teams of about 50
  people
• Parts of applications are relatively stable,
  while others are highly volatile
• Architectures are driven by a few key COTS
  packages that are also continually evolving

54
SupplyChain.com Profile
55
SupplyChain.com Strategy
LCO
LCA
56
NISCM Profile

• Broad coalition of government agencies and
  critical private-sector organizations
• Support cross-agency and public-private sector
  coordination of crisis management activities
• Adaptive mobile network, virtual collaboration,
  information assurance, and information
  integration technology
• Private-sector system-of-systems integration
  contractor

57
NISCM Profile
58
NISCM Strategy
LCA
LCO
59
Risk Profiles Across Examples
60
Outline

• Tutorial learning objectives
• Survey of assumptions about participants
• General software trends and implications
• Sources of perplexity about agile, plan-driven
  methods
• Overview of agile and plan-driven methods
• XP, TSP days in the life
• Comparisons of differences, strengths, weaknesses
• Risk-based balance of agility and discipline
• Small, medium, large examples
• Observations and Way Forward
• Hands-on exercise
• Conclusions review of learning objectives

61
Summary of Observations

• No silver bullet
• Clear agile and plan-driven home grounds
• Increased demands for agility, velocity, quality
• Balanced methods are emerging

• Build methods up vs. tailor down
• Methods are important people more so
• Values
• Communications
• Expectations management

62
No Silver Bullet

• Brooks werewolf concerns
• Complexity, conformity, changeability,
  invisibility
• Agile methods
• Handle changeability and invisibility
• Miss on complexity and conformity
• Plan-driven methods
• Handle conformity and invisibility
• Miss on changeability and complexity

63
Future Applications Need Both

• Historically
• Many small, non-critical, well-skilled, agile
  culture, rapidly evolving projects
• Many large, critical, mixed-skill, ordered
  culture, stable projects
• In the future
• Large projects are no longer stable
• Maintenance of extensive process and product
  plans will become too expensive
• Complexity and conformity werewolves are waiting
  for agile projects
• Attributes of both approaches will be needed

64
Balanced Methods are Emerging

• Agile methods
• Crystal Orange
• DSDM
• FDD
• Lean Development
• Plan-Driven methods
• Rational Unified Process
• CMMI
• Hybrid
• Boehm-Turner Risk-based
• Code Science/Agile Plus

65
Build up Dont Tailor Down

• Plan-driven methods traditionally
• Define heavyweight process guidelines
• Advocate tailoring
• Treat mass of processes as security blanket
• Agilists traditionally
• Begin with the minimum
• Add as needed (and justified by cost-benefit)
• Start small extend only where necessary

66
Maybe its not the methods!

• Agile movement has echoed a long line of warning
  calls
• Success of agile may be due as much to people
  factors as to technology
• Valuing people over processes is the most
  important factor in the manifesto

I know I saw something about that in the process
somewhere
67
People

• Of the people, by the people, for the people
• Separation of concerns is increasingly harmful
• A few quotes
• The notion of user cannot be precisely defined,
  and therefore it has no place in computer science
  or software engineering. Dijkstra
• Analysis and allocation of the system
  requirements is not the responsibility of the
  software engineering group, but it is a
  prerequisite for their work. The Capability
  Maturity Model for Software.
• Software engineering is not project management.
  Tucker

68
Values

• Reconciling values is a critical people-oriented
  task
• Stakeholders value different things
• Software engineering is usually value-neutral
• Every requirement, object, test case, defect
  equally important
• Process improvement and plan-driven methods are
  inwardly-focused
• Aimed at productivity improvement
• Not higher value to customer
• Agilists attention to prioritization and
  negotiation are promising

69
Communications

• Face it, most engineers cant talk, much less
  write
• Need to bridge the customer-developer
  communications gap
• IKIWISI reigns
• Rapid change increases need for solid
  communication
• Few sources of guidance
• Cockburns Agile Software Development is a good
  starting place

Ill know It When I See It
70
Expectations Management

• Differences between successful and troubled
  projects is often expectations management
• SW developers have problems with EM
• Strong desire to please
• Little confidence in prediction
• Over confidence in abilities
• Most significant common factor in highly
  effective plan-driven or agile teams
• EM means not setting up unrealistic expectations
• Process mastery
• Preparation
• Courage

Developers seem to like Sisyphean tasks
71
Research Challenges and Opportunities

• Techniques for achieving rapid quality
• Risk/value-driven process lightening
• Agile with lightweight VV overlay
• Capitalizing on domain knowledge
• Reuse, component-based approaches
• Automating quality enhancement
• Empirical analyses
• Distributed agile methods
• Hybrid agile/plan-driven methods
• Effects of individual practices
• Effects of scale, personnel turnover

72
Size Distribution

• 60 of projects in agile home ground
• 60 of effort in plan-driven home ground

73
Development Methods
74
Do Both Assure Success?

• TSP and agile methods are indeed silver bullets
  and always succeed spectacularly well.
• People tend to report success more than they do
  failures.
• The pioneer projects are being done by
  high-capability early adopters of new methods.
• The pioneer projects are experiencing some
  Hawthorne effects of performing extra well while
  being in the spotlight.
• The projects are being compared to particularly
  poorly performing past projects.
• High improvement numbers make options worth
  exploring

75
Reifer Scorecard for XP
76
Pair Programming Results

• Short studies are suspect pairs take about 10
  hours to jell

77
Educational Approaches

• Survey courses
• Materials here plus excursions into practice
• Try, compare inspections and pair programming
• Try, compare refactoring vs. use of design
  patterns
• Project courses
• Some practices need tailoring to student
  constraints
• Steep learning curve precede with survey course

78
Experimental XP-Like Course

• Goals
• Create an improved version of Agile COCOMOII
• Use XP practices supplement where needed
• Gather software process data
• Characteristics
• Part-time, non-collocated student team
• Part-time customer
• Personnel with moderate platform/applications
  experience

79
(No Transcript)
80
XP situational common sense gt Lightweight
method

• Planning variants
• Brainstorming modified win-win -gt e-Story/Task
  cards
• Bi-weekly meetings
• Stories prioritized -gt iteration requirements
• Design variants
• Brainstorming group input team buy-in
• Architecture determined from shortfalls of
  previous arch.
• Top level design iteration designs
• Iteration design fulfills iteration
  requirements
• Prototype GUI, draft class design pair-work

81
Electronic Story/Task Cards
82
XP situational common sense gt Lightweight
method

• Coding variants
• Coding standards drafted, team consensus, will
  evolve as needed
• Customer available bi-weekly for feedback/report
• Tests are being written before any code, all code
  is unit integration tested
• Quality guidelines
• Digital archives of artifacts
• Simple, short, risk-based documentation

83
Organizational Way Forward

• Convene key stakeholders to determine
  organizational goals and strategies
• Rebalance agility and discipline in this
  organizational context

84
Determining Organizational Goals and Strategies

• Current software organizations status and needs
• Key stakeholders needs and trends
• How to cope with future trends
• The increased pace of change and need for agility
• The increased concern with software dependability
  and need for discipline
• Your ability to satisfy your stakeholders
  evolving value propositions and to keep up with
  your toughest competitors
• The increasing gap between supply and demand for
  Cockburn Levels 2 and 3 people
• Your ability to cope with existing and emerging
  challenges
• COTS integration, evolving Internet/Web
  capabilities, distributed and mobile operations,
  agent coordination, multimode virtual
  collaboration, and outsourcing jobs

85
Collins Good to Great- HarperBusiness, 2001
86
Rebalancing Agility and Discipline
Highly mixed agile/plan driven profile
In agile or plan-driven home ground
87
Hands-on Exercise

• Graph your organizations typical project on 5D
  charts
• Now 5 years from now
• Identify major needs for organizational change
• With respect to balancing agility and discipline
• With respect to future trends, environmental
  risks
• Identify most critical first steps for
  improvement
• Identify pilot project for applying first steps
• Application, staffing
• Needs for education, teambuilding, culture change
• Summarize in bulleted strategic plan

88
Now
89
In 5 Years
90
Conclusions

• Plan-driven and agile methods both aim to
• Satisfy customers
• Meet cost and schedule parameters
• Home grounds exist, but the need/opportunity for
  integration is expanding
• We recommend a self-assessment of your
  organization and business
• Locate your place in the home ground space
• Identify areas of change and how they might move
  your location
• Look for ways to integrate methods to better
  respond to change
• People concerns dominate method concerns