Title: Software Reliability Observations for Software Products Relying on Agile Programming Practices
1Software Reliability Observations for Software
Products Relying on Agile Programming Practices
2Quick Overview
- Software Reliability
- Agile Programming
- Reliability Observations Made on Software
Products based on Agile Programming - Software Reliability models and which fit agile
programming best from observations deducted - Conclusion
3Introduction
- After working in the industry for 2 years Ive
found that full software process cycle is not
really practical in many software companies
because of time constraints and business needs. - Documentation, software specifications, project
design, full software testing are not fully
explored or completely omitted because of time
and this greatly hurts reliability of the system. - This paper aims to explore possible causes that
increase system unreliably and aims to suggest
measures to be taken into consideration under
these kind of development environments.
4Software Reliability
- Software reliability is defined as the
probability of failure-free software operation
for a specified period of time in a specified
environment. - It plays an important role in determining cost,
efficiency, accuracy and ultimately the success
of a software application. - It depends on two factors the faults that exist
in the software and the exposure of such faults
through activation of the software via testing,
usage, etc
5Fault vs. Errors
- It is important to properly recognize and
identify faults and errors to properly address
these issues. - A software error is defined as programmer action
or omission that results in a fault. - A fault is defined as a software defect, in
deviance from project specifications, which
causes failures or data corruption.
6Problems Reliability
- Reliability is not exact or total
- Current models make approximations and
assumptions - Software products characteristics,
implementation, and goals vary greatly from one
program to another - It is difficult to correctly and accurately
measure risk assessment - Reliability models arent complete they all have
things they measure and other things omit - These suggestions ONLY apply to non life-critical
systems.
7Benefits of Software Reliability
- Gives a quantitative measure of reliability over
time - Reduces escaped defects
- Reduces occurrences of late software deliveries
- Tells us know how many bugs to expect over a
period of time and effort required to fix a
designated number of them
8Software Reliability Metrics
- Software reliability metrics are used to identife
areas of the software requirements specification
and code that can potentially cause errors. - There are several metrics to take into
consideration when aiming at securing optimal
reliability on a system and those are
requirement reliability, design and code
reliability and testing reliability metrics.
9Agile Programming
- Agile software development is a conceptual
framework for software engineering that promotes
development iterations throughout the life-cycle
of the project. - Most minimize risk by developing software in
short amounts of time. - Each iteration is an entire software project
including planning, requirements analysis,
design, coding, testing, and documentation. - This new development approach is based on the
premise that if you hire competent developers,
presumably they know how to write code. - Their time is spent developing and addressing
issues organizing, communicating, designing and
carrying out the software product quickly.
10Agile Programming Principals
- Customer satisfaction by rapid, continuous
delivery of useful software - Working software is delivered frequently (weeks
rather than months) - Working software is the principal measure of
progress - Even late changes in requirements are welcomed
- Close, daily cooperation between business people
and developers - Face-to-face conversation is the best form of
communication (Co-location) - Projects are built around motivated individuals,
who should be trusted - Continuous attention to technical excellence and
good design - Simplicity
- Self-organizing teams
- Regular adaptation to changing circumstances
11Experiment
- Aims to see the impact of reliability on software
that performed agile programming practices - 4 application projects were evaluated
- These applications are developed to automate
business tasks for the many different company
departments. - The goal of all the applications were to
standardize a particular business process to make
it more cost efficient, reduce data corruption,
reduce human error, and enhance the business
model.
12General Project Details
- The software projects that are being analyzed
were done to assists the following company
departments marketing, appointment center,
sales and IT. - Development time for these applications lasted
between three months to 2 years, counting
maintenance. - User size vary from project to project. From 10
users to 300 users. - The products that will be explored in this paper
are Book Request Automation, SHIP Framework,
Pricebook and Lead Import.
13Book Request Automation
- Integrates three department marketing,
appointment center and sales department to
automate appointment scheduling flow. - Marketing determines the lead flow by market by
product by day according to how aggressive
theyre marketing the specific area and from
historical sales turnout. - Lead flow Information is sent to sales team so
that they can stock up a certain amount of sales
representative per city trained in the specific
products according to the lead flow. - The appointment center is then responsible for
scheduling appointments for properly scheduling
the lead of flow. - Automating this process allows appointment
specialists to properly book appointments
according to the sales representatives that are
working on each specific day for the specific
products they are trained on.
14Pricebook Editor
- Tool that manager national prices for all the
items sear home improvement sales/installs. - Allows management of price books which are a
collection of the available products/services for
each individual market and the price for those
products for that market. - Allows bulk import of prices, cut in dates for
prices, sales representative mileage commissions,
product discounts and their cut in dates.
15SHIP Framework
- Main library used for all SHIP software products.
- All common shared code is placed here to allow
developers the ability to properly develop and
solve complex programming issues. - It contains the following but its not limited to
permissions, data access connections, manager
base management, collections management file
compression, encryptions, GUI standard look and
feel for the applications, etc.
16Lead Import
- Back end process that sits on a web service that
receives lead information for vendors and from
our main site Searshomepro. - Processes leads into our system.
- The leads get posted through a Message Queue and
as the server responds/listens different actions
are taken on the leads info. Example new
customers are created, Appointment center gets
notified about customers interest and places the
leads in a call queue, if the lead requested an
appointment, the system syncs with Book Request
to search for an available opening and creates
the appointment, etc.
17Design Phase
- For most projects
- Project documentation was created after software
rolled out - Project design came together completely during
the development although it was conceptualize
before coding began. - Developers met directly with customers and
maintain an open line of communication all the
way through - Need changes were added as needed
- Weekly codes review meeting allowed for opening
questions to developers to share ideas and
collaborate.
18Testing Phase
- There are two SQL 2005 testing servers available
for testing - Servers scripts are ran to mini data, table
structures, and stored procedures. - Active directory permission groups were either
created or tested with an existing permissions
group. - Virtual machines are also set up with win2k to
test performance on the slowest of computers that
are available to sears associates. - Again because of time constraints testing happens
during the development process. As developers
are coding pieces they test the individual
methods to see if the are obtaining the expected
results. From this they make modifications until
they feel confident enough that their code is
performing as it should.
19Maintenance
- Customers put in software request changes,
enhancement or bug fix request through a ticket
queue system - Assigned developer takes care of the particular
problem. - Ticket gets assigned to project lead or to a
junior developer in charge of the ticket queue.
- Usually if it is a critical change and it is
during the deployment phase of the software where
bugs and changes are still being applied the
developer responsible for the application will
take all the tickets regarding his or her
product. - As applications became more reliable and big
fixes die down, enhancements and small ticket
request will be handled by a junior developer.
20Deployment
- Easy and fast
- Developers need to log every deployment and
obtain management permission to deploy - Dlls are copied into server machine
- Application updater is install on all machines
- When user reopens their application updater runs
before software starts and checks to see there
are new dll versions deployed and grabs puts them
into the clients machine - GAC dlls update automatically
21Experimental Process
- To examine the reliability of the software
instances, I looked into the history of the
tickets put in for bug changes, specification
changes and enhancements. - The request tickets were matched with the
deployment logs for each of the specified
programs. - Observations were done on queries to the data
deployment logs and on interviews/conversations
with developers.
22Observations
- As a project became more complex and big there
faults and errors were greater than the smaller
less complex projects. - These results were expected since rapid
development is best suited for projects that are
smaller and less complex, but what I was surprise
to see was that these results showed equal
regardless of whether a senior developer were
assigned to it or not. - Bugs also increased exponentially as less time
was given for development, it increased even more
the developers that worked over 60 hrs a week to
meet deadline. My guess is that due to fatigue
developers were more prone to making mistakes and
less likely to catch those mistakes during
testing because of exhaustion. - Programs that reuse code showed a slight better
rate of faults and errors than the ones that
didnt or reused very little code.
23Observations cont.
- To my surprise, developers with very little field
experience, experience more errors in their applications those with more
experience even if it was a few more years. - My take in this is that because of their
inexperience they cannot easily spot possible
problems during development and during testing. - In addition, when the less experience developers
try fixing or changing requirements for the later
versions, 28 of the time they ware found to
introduce another bug, as opposed to 4 rate of
errors for more experienced developers.
24Observations cont.
- Projects that had well define software
specifications from start had less
rewrites/redesign of processes and thus had more
time during the testing phase - When a new person was bought on to the project
after initial rollout or a developer left the
group, the rate of fault increased initially - Projects with greater number of users, though
initially displayed a greater number of bugs than
the other software products, they more quickly
stabilize and became more reliable.
25Reliability Models
- Software reliability models are used for the
estimation and prediction of software
reliability. - Selection of software reliability model for use
in a particular case has been an area of interest
for researchers in the field of software
reliability - It is important to point out models that might be
even more useful in fast paced development
environments
26Model Types
- Times between failures model it focuses on time
between failures. - Failure Count Model is interested in the number
of faults or failures in a specified time
interval rather than in times between failures.
The failures counts are assumed to follow a known
stochastic process with a time dependent discrete
or continuous failure rate. - Fault seeding model it seeds a known number of
faults in a program which is assumed to have an
unknown number of indigenous faults. The program
is tested and the observer number of seeded and
native faults is counted to make an estimate of
fault. - Input domain based model generates a set of test
cases from an input distribution which is assumed
to representative of the operational usage of the
program.
27Model Selection
- When selecting the proper reliability model it is
important to considered several factors such as
utility, applicability and validity. - For agile programming it is vital to distinguish
what might be the most useful methods under such
extreme time constraints. - Times between failure models
- Time failures over time for a given set of
observed values may not be the best approach due
to the fact that failure times are random. - As seen earlier having many users increases
initial rate of errors. - Doesnt seem applicable for environments that can
push out updates daily and even several times a
day - Fault Count models
- Are the cumulative number of failures versus time
curve that will eventually level off and this
model can take into account, calendar time, CPU
time, number of test cases ran and other relevant
metrics. Because this model
28Model Selection Cont.
- Fault seeding
- Requires seeding a number of faults in a program
and then testing the program and counting the
number of indigenous faults that were exposed. - Time consuming process
- Not great for development environments with no
time. - Im not saying that this model is not useful but
perhaps not very practical for extreme
programming. The same applies for input domain
base models.
29Conclusion
- Requirements reliability is very important.
- Best reliability models for agile programming
seemed to be failure count models. - Also when time not on your side it is important
to remember to increase reliability - Complex and big programs should be given to more
season developers, - Properly defined specifications is important
- Jr. Developers do a lot better with smaller less
complex programs where they can obtain the
necessary experience for fixing and deploying
bugs correctly - Good to main developers through the entire
project cycle - Of course all this observations need to be put to
test with new development projects to see how
well they hold.