Embracing Change An introduction to Agile XP Day 2006 Clarke Ching, Senior Consultant, VISION Consul

1
Embracing Change An introduction to Agile XP
Day 2006Clarke Ching, Senior Consultant,
VISION Consulting Chairman, AgileScotlandwww.cl
arkeching.comwww.RollingRocksDownhill.com
1
2
Where did we go wrong? The software paradox

• Software is all around us
• So we must be doing something right
• But,
• IT Projects have a history of failure
• 30 - 40 of systems projects fail prior to
  completion 1
• Half of all systems projects overrun their
  budgets and schedules by 200 or more 1
• Failed systems projects cost more than US100
  billion per year in the USA alone 2
• Most businesses have a growing backlog of
  software development projects and consider their
  profitability to be constrained by ITs capacity
  to deliver.
• Low trust between Customers and IT and within IT

1 B.P. Lientz and K.P. Rea, Breakthrough
Technology Project Management 2 Computerworld
3
What went wrong?

• ?
• We wrongly assume that we can get the spec right
  upfront.

4
NHS chaos exposed by new e-mails A COMPUTER
project costing 6.2 billion that is central to
Tony Blairs National Health Service reforms is
in grave danger of being derailed, leaked
Whitehall e-mails reveal. The warning has been
issued by Richard Granger, the 250,000-a-year
civil servant in charge of what has been billed
as the worlds biggest civil information
technology project. The scheme is central to the
governments plans to give patients wider choice
by allowing GPs to book hospital appointments
online with consultants throughout the country.
The problems have already caused a year-long
delay in the booking system and now threaten to
add millions to the cost of the project. To date
the system has made only about 20,000
appointments for patients. It was supposed to
have made 250,000 by December 2004. When it is
fully operational the system is meant to be
capable of making up to 9.5m first hospital
appointments a year. In the e-mail exchanges in
September, Granger blames a senior civil servant
in the Department of Health for the fiasco,
criticising her repeated last-minute changes and
failure to heed his advice. Granger censures
Margaret Edwards, the departments director for
access and patient choice, for adding numerous
new specifications to the booking programme,
known as Choose and Book. Granger writes
Choose and Books 20m IT build contract is now
in grave danger of derailing (not just
destabilising) a 6.2 billion programme. He
concludes Unfortunately, your consistently late
requests will not enable us to rescue the missed
opportunities and targets. Sir Nigel Crisp, the
NHS chief executive, was forced to admit to the
Commons health select committee two weeks ago
that the booking system was at least a year
behind schedule. However, he failed to mention
that the delay was having a serious impact on the
entire project. The National Audit Office has
identified changes to specifications after the
award of IT contracts as a key reason for regular
delays and overspends on government projects.
Grangers comments were triggered by an e-mail on
September 9 from Edwards marked Restricted
Policy which begins We have a problem! The
e-mail reveals that patients and their GPs still
cannot book treatment at any of the countrys 32
foundation trust hospitals by computer because
they are not on its choice menu. The 10
private sector treatment centres, set up by the
government to reduce waiting lists, are also
absent from the official list on the computer.
Edwards warns that the treatment centres and
foundation trusts will not be on the choice
menu until next summer. The delay places
hospitals at a financial disadvantage. Under the
governments payment-by-results regime, they are
supposed to compete with other NHS hospitals for
patients. Edwards admits We havent yet told
ministers that there is a problem. Granger was
incensed by the implied criticism of the booking
system and fired off a trenchant 11-point reply.
Although Edwardss original e-mail was encrypted
and her password protected, Granger decrypted it,
sent it out with his reply and widened the
distribution. Granger complains that the
project has been allowed to change beyond
recognition from the original specification. The
original request from your predecessor and
yourself was for an Electronic Booking System.
The change of this to Choose and Book occurred in
(the second quarter of) 2003. This was the first
of what are now recurrent major changes in your
requirements. The booking system has been
dogged with difficulties since its inception. GPs
have refused to use it and early pilot schemes
identified fundamental software design flaws.
Last week Granger, who insists that the booking
system now works, broke civil service protocol
and publicly blamed policy officials in the
Department of Health for failing to get GPs to
use the system. In an interview with Computing
Magazine, he said Low usage is not something I
can do anything about. Both the health
department and Grangers spokesman refused to
comment on the leaked e-mails.
Jonathon Carr-Brown, The Sunday Times, November
13, 2005 http//www.timesonline.co.uk/article/0
,,2087-1869851,00.html
5

• No Change!
• We are already running late.
• I need to meet my date.
• We worked hard to prevent change at the start.

Barry BoehmsCost of change
Customer
IT Project Manager
Promised date
6

• No Change!
• We are already running late.
• I need to meet my date.
• We worked hard to prevent change at the start.

Change Rework happens at the most expensive time
Barry BoehmsCost of change
Customer
IT Project Manager
Promised date
Spec signed off here
7

• Change!
• Our spec was a guess
• We learn by doing the project
• We need the best product

Don ReinertsonsLearning Curve
Customer
IT Project Manager
Promised date
Spec signed off here
8

• Change!
• Our spec was a guess
• We learn by doing the project
• We need the best product

Out of hundreds of projects, there is no case in
which requirements remained stable throughout the
design - Reinertson (1998) on Product Development
A typical software project experiences a 25
change in requirements - Boehm and Papaccio
(1988) on Software Development
Don ReinertsonsLearning Curve
This learning causes change
Customer
IT Project Manager
Medium to Large projects (1000 function points)
experience 25 35 requirements change - Jones
(1997) on Software Development
Promised date
Spec signed off here
Conclusion We cant successfully prevent change
9
The Project Managers Conflict
Successful Project
Meet Schedule
Best Product
No Change!
Learn Change!
Customer
IT Project Manager
10
The Project Managers Conflict

• Who is to blame?
• The customer? ?
• The project manger? ?
• An simple misunderstanding in the 70s
  ?

Successful Project
Meet Schedule
Best Product
No Change!
Learn Change!
Customer
IT Project Manager
11
A simple misunderstanding in the 1970s

• In the 1970s
• The software gurus borrowed quality thinking
  from Japanese manufacturing
• But,
• Software development is not manufacturing ?
• Software development is product development ?

12
Software Development is product development
Product Development Develops the product
specification, which is used by Manufacturing
Concept -gt Iterative Learning -gt Spec
Spec -gt Build to spec -gt Product
13
Software Development is (mostly) product
development

• Product development is a learning exercise
• Its meant to be iterative
• Testing is the best source of feedback and
  learning.
• Testing happens before the spec is finished.

14
But, we treat Software Development as
manufacturing
In other words We spend the first half of the
project putting defects in We spend the second
most expensive - half of the project reworking
them out.
1. Waterfall tries to prevent change by getting
spec right up front
2. We learn throughout the project request
change
3. Change causes REWORK at most expensive time
4. Testing duration unpredictable 50 of
total
5. We often de-scope or run-late
15
Agile Methodologies such as XP, DSDM, CBA and
Scrum

• are specifically designed to efficiently cope
  with change.
• They do this by building software iteratively and
  incrementally.

• The first Space shuttle control software was
  developed in 1970s using incremental methods
• US Department of Defence now demands that
  software is constructed using incremental and
  evolutionary techniques see MILSTD-498

16
Six principles of Agile Development
1 Customer lists known requirements (to a high
level), then prioritises them.
2 Deliver time-boxed increments of high-value,
well engineered, Working software often
3 The Customer can release the software at any
time they want.
All features
4 The Customer can add, delete or reprioritise
features at any time. i.e. this is how we
embrace change
Promised ShippingDate
5 We protect schedule commitments, despite change
6. We can review the project and the value it
delivers at the end of each increment
learn from the market
Working Software Potentially shippable
RELEASE
prioritised
Backlog
time
1 4 weeks
17
Benefit Reduced Waste quicker to market
All features
prioritised
Backlog
time
18
Agile is more efficient Many changes are free
If we are working on this increment then
All features
you can change any of these for FREE
prioritised
Backlog
19
Agile is more efficient Killer changes dont
always kill

• Working incrementally forces early, not late,
  failure
• Most changes are cheap

Barry Boehms original paper on CoC described 3
curves
Very few killer changes
Most changes
20
Agile is more efficient Quality practices
prevent speed rework
The sooner a defect is detected, the easier and
cheaper it is to fix
Prevention is best cheapest
but
All features

• Customer easily available prevents defects
• Write automate UAT and Unit tests before coding
  prevents defects
• Automated tests find regression defects earlier.
• Continuous integration finds defects earlier
• Pair Programming (XP) finds defects much earlier

prioritised
Backlog
21
Agile is more efficient consistently good design

• Agile builds working software on top of working
  software, on top of working software,
• Each increment aims to be
• Defect free potentially shippable
• Well engineered, good clean code

Good Design easy to change low cost of change
End with a good design
Make a Small change
Start with a good design
Agile Cost of Change
time
22
Major Benefit Faster cash flow and higher profits
You Single Release Revenue, starts month 24,
1,000,000 / month (12m / year
Both have development costs of 250,000 per
month
23
Major Benefit Faster cash flow and higher profits
Your competitor Two releasesRevenue, starts
month 9 500,000 / month (6m / year, Then
grows to 12M at month 24
You Single Release Revenue, starts month 24,
1,000,000 / month (12m / year
Both versions have development costs of
250,000 per month
24
Major Benefit Faster cash flow and higher profits
More realistic Your competitor learns from
customer feedback does multiple releases.
Your competitor Two releasesRevenue, starts
month 9 500,000 / month (6m / year, Then
grows to 12M at month 24
You Single Release Revenue, starts month 24,
1,000,000 / month (12m / year
Both versions have development costs of
250,000 per month
25
Benefit Making more money.
Total time to cross the bridge Queuing Time
Crossing time

• Before Agile
• Typically, the business produces ideas faster
  than IT delivers them.
• Big backlogs of queuing projects
• So, projects spend a long time waiting in a queue
  before being started
• After Agile
• IT is more productive.
• IT projects are finished faster.
• Backlog shrinks, projects start sooner. Queuing
  time reduced.
• Each idea is realized sooner.
• Cash and Profit both flow faster.

Crossing time
Bottleneck
Queuing time
26

• Agile is hard work but it does work

27
Clarke Ching, Senior Consultant, VISION
Consulting Chairman, Agile Scotland
27
28
Agile the new buzz word?

• Umbrella term to cover various methods/approaches
  
• Scrum iterative project management ?start here
• Self Managing teams
• ScrumMaster
• Daily Stand up
• Extreme Programming (XP) - iterative software
  development
• Test Driven Development (TDD) write test, write
  code, repeat
• Pair Programming
• Lean Software Development Why agile works
• Commitment-based Agile development making Agile
  sticky
• Agile manifesto
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan
• Individuals and interactions over processes and
  tools

http//www.agilemanifesto.org/
29
How Action Happens - ideally
Let me help you address your concern
Customer
Performer
30
How Action Happens in the typical project
1. vaguely
7. Both sides negotiate harder in future.
5. Hmmmm I dont trust your future promises
3. hope
Let me help you address your concern
Customer
Performer
2. Whats in the spec / contract
4. Everything going great then were Late /
Over budget / De-scoped / Cancelled
6. Better Spend more time getting spec right
upfront
31
How Action Happens in agile projects

• New rules
• You can change your mind but lower priority
  features will be dropped.
• You must actively engage in the project if you
  dont, then you get less features.
• We protect dates by dropping lower priority
  features

Let me help you address your concern
Customer
Performer