Code Sequence and Control

1
Minimising Risk on Large Scale Software Projects
Or Some practical advice on staying out of
trouble.
Mike White mike.white_at_AIPEX.com.au
2
Key reasons for FAILURE
Reasons for Software Project Failure

44
Source The Standish Group 1994
http//www.standishgroup.com/sample_research/chaos
_1994_4.php 1994
3
Rule 1
Take an Entrepreneurial Approach the Business
Understand
4
Quality is an exercise in risk mitigation. The
amount of Quality to be built in is determined by
the level of risk a business is willing to
accept But the Business need the facts to make a
determination.
5
Quality Mgr vs Testing Mgr
System Testing, Integration, End 2 End,
Conversion, Traceability, Data Integrity,
Performance, Security, .

• Financial Mgt
• Audit
• Governance (SOX)
• Vendor Mgt

• Process
• Build Mgt
• Deploy Mgt
• Defect Mgt
• Change Control
• Risk Issue Mgt
• Dev Mgt
• Req Mgt
• Config Mgt

• Unit Testing
• Code Reviews
• Requirements
• Infrastructure
• Methodology
• Tools

6
Entrepreneurial Approach

• What is the project/my Objective?
• What is the project/my Scope?
• What does success and failure look like?
  Measurement?
• What are the Execs Buttons? How do I Influence?
• How do I get/maintain a Quality profile?
• Whom are my Customers?

7
Entrepreneurial Approach

• What is my Budget?
• Schedule scope budget contingency?
• What are key dependencies / deliverables (in
  schedule?)
• Where can we save any costs (aka where can I
  create my contingency?)
• What are our strengths weaknesses?
• What are the risks and how are they managed?

8
Rule 2
Understand the Cost and Consequences of the
Projects Failure and Success
9
Global Problems
2005 2004

• FBI Virtual Case File project abandoned after
  US170M spent
• Hudson Bay Co Problem with Inventory system
  contributes to US 33M loss
• UK Inland Revenue Software errors contribute to
  US3.5B tax-credit overpayment
• Ford Purchasing System abandoned after US400M
  spent
• J Sainsbury Supply-chain Mgt system abandoned
  after US 527M
• HP ERP System Problems contribute to US160M
  loss
• Avis Europe ERP system cancelled after 54.5M
  spent
• ATT CRM Upgrade leads to revenue loss of
  US100M

Spectrum Magazine Sept 2005
10
Closer to Home
ANZ transactions in 10bn chaos A major computer
system implementation at ANZ Banking Group has
gone seriously wrong, throwing hundreds of
thousands of transactions worth an estimated 10
billion into confusion.The Financial Review
30/11/2002     NAB admits to 110m fees bungle
National Australia Bank managing director John
Stewart has admitted the cultural shake-up at the
bank is far from complete after a major bungle
involving 200,000 customer accounts cost it 110
million in repayments and costs.The Financial
Review 29/07/2005     Westpac pulls plug on
net banking as software fails Westpac Banking
Corporation, the country's second-largest
internet bank, yesterday pulled the plug on its
online service after being hit by a serious
software problem on its computer servers.The
Financial Review 15/08/2001  
11
More Recently
11/10/2005 Importers have warned of confusion
on the nation's docks by week's end with ships
unable to unload their cargo because of the
rushed introduction of a new cargo management
system. However it has been dogged with
technical difficulties that at last count have
pushed out the original expected cost to 188
million from 35 million. The computer software
pivotal to the new computer system has only
arrived in the offices of importers and brokers
in the past week. Executive director of the
Customs Brokers and Forwarders Council, Stephen
Morris, described the introduction of the new
system as an absolute stuff-up. A recent survey
by the council found 90 per cent of its more than
200 members were simply not ready for the change
to the new system. The AGE http//www.theage.com
.au/news/National/Confusion-looming-on-docks-indus
try/2005/10/11/1128796519415.html
12
Large Banking Project

• Create a Philosophy
• This project reflects the cost of staying in
  business.. Get it wrong and we are out of
  business. There is no fallback position.
• Use it at every opportunity
• It reinforces how serious you take your job and
  their career

13
Rule 3
Understand the Budget, the Funding Process and
Follow the Money (Find the business owner
establish a direct relationship)
14

• Without Funding you will have to deliver on
  someone elses perception of Quality
• Without Funding you will have to deliver to a
  schedule you cant control
• Without Funding your structure is already assumed
• Without Funding your resourcing will be assumed
  based on previous projects and the same-old
  same-old
• Without Funding you have no contingency to
  mitigate risks and issues
• Unless you can Reforecast REMEMBER
• 94 of Projects will be restarted

Chaos Report Standish Group Page 3
15
Helpful Guidelines

• Never accept the budget as given (94)
• Build your budget from the Ground up (easier to
  defend)
• Ensure it is a Quotation
• Ensure caveats are part of Quote
• Understand and match the risk. (no testing even)

16
Helpful Guidelines

• Expect to have to Defend your Quote
• A Methodical approach is necessary
• Build an Estimation Model on Effort
• Based on complexity, requirements, competency,
• technology used,
• Match manpower to a Resource Profile
• Effort Profile Doability (Man Days)

17
Rule 4
Shape the Schedule (the tail wagging the dog)
18
An AIPEX Case Study
Original Waterfall Plan
UAT

• 8 of Project time for Testing
• Compatibility Testing 1 test every 3 minutes
• Defects not found till end
• Cost of repair higher
• Little visibility for customer

24 July
19
Where AIPEX Helped
New Iterative Plan
UAT 0
Plan
Dev
Fix
Dev
Dev
Fix
Fix
Test
Test
Test
ReTest
Test
Production
ReTest

• 30 of Project time for Testing
• Defects found earlier
• Cost of Repair less
• Customer Feedback Earlier
• Increased Creative time
• Reduced Change Requests
• User Involvement increased by 10 fold
• Project delivered 2 weeks early
• Project Still Within Budget
• Highest Customer Feedback
• Reduced risk of failure
• Is Scalable

UAT 1
UAT 2
UAT 3
10 July
20
Remember!

• Try and break the schedule into chunks
• Chunk the schedule according to risk
• Test Early !! Test Often!! Minimise Risk
• Ensure you have at least 1 milestone before
  your major task
• Build Complete - Build Accepted
• Build Deployed - Testing Begins
• You run at least 30 of the schedule
• You are a Project Manager, Run it as a project

21
Remember 2!

• Watch for and plan how you will
• handle conflict
• You do NOT own the Risk! Therefore plan to Be
  Flexible (Entry/Exit) Be Traceable!
• Accept business decisions until you can no
  longer personally compromise.
• Have an End to End Model

22
Remember 3!

• You will be audited on large scale projects!
• Process Tools alone wont deliver!
• Ensure your own communication channel to the
  business create the relationship
• Look at innovative ideas (eg Scrums, might
  actually work here)..
• Prioritise the risk and Test in Production

23
Key reasons for FAILURE
Reasons for Software Project Failure

44
Source The Standish Group 1994
http//www.standishgroup.com/sample_research/chaos
_1994_4.php 1994
24
Lack of User Input or Get Partnering with the
Business
Rule 6
25
Partner? How?

• Get them involved early!
• Risk Reviews
• Useability
• Requirement Reviews
• Requirement Analysis Testing
• User Reviews
• UAT
• Training Reviews
• Defect Reviews
• Identify what they need for success
• PIR Plan how to measure Value.
• Get them Involved They want to be!

Build a Trust Relationship
26
Requirements Analysis Testing
Vendor
4xweeks
2-4xweeks
Dev
Test
4xweeks
Client
Test
UAT n

• Needed Early Identification of Problems
• Time lags too long to wait for feedback in UAT

27
Requirements Analysis Testing
Vendor
4xweeks
2-4xweeks
RAT Team consists of BAs, Test and Business
Dev
Test
RAT n
4xweeks
Client
Test
UAT n

• Early Identification of Problems
• Time lags reduced for feedback from Users

28
Expected Benefits

• Comfortable users
• Early Feedback to Development
• Positive Business Feedback to Project
• Build of Trust / Teamwork
• Better Budgeting
• Early Change Requests
• Build up to UAT
• Shorter UAT Length

29
Unplanned Benefits

• Understood real problems and challenges
• Run as a reward
• Positive Feedback to Branch Network (pls
  accelerate)
• Branch people felt involved not dumped on
• Different Geographical Business Processes
• Snr Mgt Buy-in
• Used as an example of Cultural Change
• Political Leverage and Business Support
• Quotes and survey stats used to reinforce success

30
Rule 7
It all starts with Requirements!
31
Reasons for FAILURE
Reasons for Software Project Failure
24.1

Source The Standish Group 1994
http//www.standishgroup.com/sample_research/chaos
_1994_4.php 1994
32
(No Transcript)
33

• Influence Requirements
• Once youve moved from Testing to Quality,
  you can get involved in all stages of the SDLC.
• Ensure Quality signs off on Requirements. This
  means getting Test Analysts or Requirements
  Analysts in early which means Budget (take a
  pragmatic approach initially). Assess
  Testability.
• This also means breaking the mould of perception
  when quality/testing gets involved.
• Testing can also Start Earlier

34
A Case Study

• Requirements within Bank N
• No Separate Competency
• Most Analysis done within Development Areas
• Duplication of Effort
• Unstructured
• Testing not Engaged Early Enough
• Testing Churned in Test Case Preparation
• Manual Traceability reports

35
Move the Bubble

• The following illustrates where the project
  bubbles occur during the
• traditional project life cycle

Design
Build
Test
Implementation
Analysis
Definition

• The following illustrates where the RM bubbles
  could occur during the traditional project life
  cycle

Design
Build
Test
Implementation
Analysis
Definition
Compliments of Phil Bowdery
36
An Approach

• Change to Quality
• Gain Competancy/Budget
• Develop a process
• Prove the process (Culturally)

• Develop the Model(s)
• Get Buy-in
• Change the SDLC
• Check Benefits
• Expand Capability

37
The Model

• Three models were developed and tried.
• The first model Testability / Sufficiency
• The second model expanded (1) but also included
  mentoring of the business and notional
  ownership of requirements management
• The third was performing the complete
  requirements function.
• A clear need for all three

38
Some Benefits

• Early involvement of Quality in requirements
  phase
• Better requirements produced
• Fewer Change Requests
• Grading acknowledges more work may be necessary
• Gets visibility of Quality earlier
• Better SMEs
• A nett saving of 10 in the Test Case
  preparation phase
• Fewer Defects in Requirements, Data, Test Cases
• Further savings in Traceability Matrices (auto)
• Can begin Testing the Requirements early
• Answers the question Are we there yet wrt
  req. (Silver)

39
Prioritise

• Prioritise the Requirements (for risk)
• Based upon
• Critical Business Functionality
• Architectural Significance

Focuses the Development Allows Iterative
development Enabler for Early Feedback Enabler
for Risk based testing Enabler for Core
Regression Tests Automation
40

• One Approach to Shifting the Risk back to the
  business and Partnering with the business on the
  risk.

41
Maintain Service Levels

• Q How to establish a software quality standard?
• A Start by prioritising functionality into
  critical and other either through Requirements or
  Test Cases
• Establish What will cause a Severity 1 or 2
  defect if we went into production.
• Get business to signoff

42
By Definition!!

• If it isnt in the Core regression suite it
  therefore cannot become a Sev 1 or 2
• defect in production (unless environmental)
• If a Sev 1 or 2 defect does occur we can trace
  backwards and understand why it occurred.
• Any change should have a minimum core
  regression run progression change.

43

• Was this Successful
• When the business start quoting Core Regression
  back to you.
• Its a good start.

44
80/20 Rule

• Rules of Thumb
• 80 of the critical Functionality of an
  Application lies in 20 of the Requirements
• 20 Test Cases cover 80 of Critical Functionality

45
The Benefits

• The business was fully engaged in establishing
  the RISK profile
• The business owned the risk
• The business felt more empowered
• The business understood what the core suite
  represented to their business
• The business was accountable for the quality
• It was measurable and the costs known/understood
• It ensures an understood level of quality
• Data requirements were understood and planned
• It forms a perfect basis for automation!

46
Some Final Thoughts

• Partner with the Business / Get them Involved
• Budget will Limit your effectiveness
• If you own part of the budget profile
• Always Prioritise Requirements/Test Cases
• Assess the Requirements for Testability
• Match the data matrix against vendor requests
• Get the business to signoff on priority

47
Above All

• Good people will remain your BEST asset
• Process Tools Alone wont Deliver a Project
• Find time to develop relationships

48
Rule 8
See Rule 1
49
(No Transcript)