Software Reliability Observations for Software Products Relying on Agile Programming Practices

1 / 29
About This Presentation
Title:

Software Reliability Observations for Software Products Relying on Agile Programming Practices

Description:

Reliability Observations Made on Software Products based on Agile Programming. Software Reliability models and which fit agile programming best from observations ... – PowerPoint PPT presentation

Number of Views:83
Avg rating:3.0/5.0
Slides: 30
Provided by: VALM3

less

Transcript and Presenter's Notes

Title: Software Reliability Observations for Software Products Relying on Agile Programming Practices


1
Software Reliability Observations for Software
Products Relying on Agile Programming Practices
  • Valery Martinez

2
Quick Overview
  • Software Reliability
  • Agile Programming
  • Reliability Observations Made on Software
    Products based on Agile Programming
  • Software Reliability models and which fit agile
    programming best from observations deducted
  • Conclusion

3
Introduction
  • After working in the industry for 2 years Ive
    found that full software process cycle is not
    really practical in many software companies
    because of time constraints and business needs.
  • Documentation, software specifications, project
    design, full software testing are not fully
    explored or completely omitted because of time
    and this greatly hurts reliability of the system.
  • This paper aims to explore possible causes that
    increase system unreliably and aims to suggest
    measures to be taken into consideration under
    these kind of development environments.

4
Software Reliability
  • Software reliability is defined as the
    probability of failure-free software operation
    for a specified period of time in a specified
    environment.
  • It plays an important role in determining cost,
    efficiency, accuracy and ultimately the success
    of a software application.
  • It depends on two factors the faults that exist
    in the software and the exposure of such faults
    through activation of the software via testing,
    usage, etc

5
Fault vs. Errors
  • It is important to properly recognize and
    identify faults and errors to properly address
    these issues.
  • A software error is defined as programmer action
    or omission that results in a fault.
  • A fault is defined as a software defect, in
    deviance from project specifications, which
    causes failures or data corruption.

6
Problems Reliability
  • Reliability is not exact or total
  • Current models make approximations and
    assumptions
  • Software products characteristics,
    implementation, and goals vary greatly from one
    program to another
  • It is difficult to correctly and accurately
    measure risk assessment
  • Reliability models arent complete they all have
    things they measure and other things omit
  • These suggestions ONLY apply to non life-critical
    systems.

7
Benefits of Software Reliability
  • Gives a quantitative measure of reliability over
    time
  • Reduces escaped defects
  • Reduces occurrences of late software deliveries
  • Tells us know how many bugs to expect over a
    period of time and effort required to fix a
    designated number of them

8
Software Reliability Metrics
  • Software reliability metrics are used to identife
    areas of the software requirements specification
    and code that can potentially cause errors.
  • There are several metrics to take into
    consideration when aiming at securing optimal
    reliability on a system and those are
    requirement reliability, design and code
    reliability and testing reliability metrics.

9
Agile Programming
  • Agile software development is a conceptual
    framework for software engineering that promotes
    development iterations throughout the life-cycle
    of the project.
  • Most minimize risk by developing software in
    short amounts of time.
  • Each iteration is an entire software project
    including planning, requirements analysis,
    design, coding, testing, and documentation.
  • This new development approach is based on the
    premise that if you hire competent developers,
    presumably they know how to write code.
  • Their time is spent developing and addressing
    issues organizing, communicating, designing and
    carrying out the software product quickly.

10
Agile Programming Principals
  • Customer satisfaction by rapid, continuous
    delivery of useful software
  • Working software is delivered frequently (weeks
    rather than months)
  • Working software is the principal measure of
    progress
  • Even late changes in requirements are welcomed
  • Close, daily cooperation between business people
    and developers
  • Face-to-face conversation is the best form of
    communication (Co-location)
  • Projects are built around motivated individuals,
    who should be trusted
  • Continuous attention to technical excellence and
    good design
  • Simplicity
  • Self-organizing teams
  • Regular adaptation to changing circumstances

11
Experiment
  • Aims to see the impact of reliability on software
    that performed agile programming practices
  • 4 application projects were evaluated
  • These applications are developed to automate
    business tasks for the many different company
    departments.
  • The goal of all the applications were to
    standardize a particular business process to make
    it more cost efficient, reduce data corruption,
    reduce human error, and enhance the business
    model.

12
General Project Details
  • The software projects that are being analyzed
    were done to assists the following company
    departments marketing, appointment center,
    sales and IT.
  • Development time for these applications lasted
    between three months to 2 years, counting
    maintenance.
  • User size vary from project to project. From 10
    users to 300 users.
  • The products that will be explored in this paper
    are Book Request Automation, SHIP Framework,
    Pricebook and Lead Import.

13
Book Request Automation
  • Integrates three department marketing,
    appointment center and sales department to
    automate appointment scheduling flow.
  • Marketing determines the lead flow by market by
    product by day according to how aggressive
    theyre marketing the specific area and from
    historical sales turnout.
  • Lead flow Information is sent to sales team so
    that they can stock up a certain amount of sales
    representative per city trained in the specific
    products according to the lead flow.
  • The appointment center is then responsible for
    scheduling appointments for properly scheduling
    the lead of flow.
  • Automating this process allows appointment
    specialists to properly book appointments
    according to the sales representatives that are
    working on each specific day for the specific
    products they are trained on.

14
Pricebook Editor
  • Tool that manager national prices for all the
    items sear home improvement sales/installs.
  • Allows management of price books which are a
    collection of the available products/services for
    each individual market and the price for those
    products for that market.
  • Allows bulk import of prices, cut in dates for
    prices, sales representative mileage commissions,
    product discounts and their cut in dates.

15
SHIP Framework
  • Main library used for all SHIP software products.
  • All common shared code is placed here to allow
    developers the ability to properly develop and
    solve complex programming issues.
  • It contains the following but its not limited to
    permissions, data access connections, manager
    base management, collections management file
    compression, encryptions, GUI standard look and
    feel for the applications, etc.

16
Lead Import
  • Back end process that sits on a web service that
    receives lead information for vendors and from
    our main site Searshomepro.
  • Processes leads into our system.
  • The leads get posted through a Message Queue and
    as the server responds/listens different actions
    are taken on the leads info. Example new
    customers are created, Appointment center gets
    notified about customers interest and places the
    leads in a call queue, if the lead requested an
    appointment, the system syncs with Book Request
    to search for an available opening and creates
    the appointment, etc.

17
Design Phase
  • For most projects
  • Project documentation was created after software
    rolled out
  • Project design came together completely during
    the development although it was conceptualize
    before coding began.
  • Developers met directly with customers and
    maintain an open line of communication all the
    way through
  • Need changes were added as needed
  • Weekly codes review meeting allowed for opening
    questions to developers to share ideas and
    collaborate.

18
Testing Phase
  • There are two SQL 2005 testing servers available
    for testing
  • Servers scripts are ran to mini data, table
    structures, and stored procedures.
  • Active directory permission groups were either
    created or tested with an existing permissions
    group.
  • Virtual machines are also set up with win2k to
    test performance on the slowest of computers that
    are available to sears associates.
  • Again because of time constraints testing happens
    during the development process. As developers
    are coding pieces they test the individual
    methods to see if the are obtaining the expected
    results. From this they make modifications until
    they feel confident enough that their code is
    performing as it should.

19
Maintenance
  • Customers put in software request changes,
    enhancement or bug fix request through a ticket
    queue system
  • Assigned developer takes care of the particular
    problem.
  • Ticket gets assigned to project lead or to a
    junior developer in charge of the ticket queue.
  • Usually if it is a critical change and it is
    during the deployment phase of the software where
    bugs and changes are still being applied the
    developer responsible for the application will
    take all the tickets regarding his or her
    product.
  • As applications became more reliable and big
    fixes die down, enhancements and small ticket
    request will be handled by a junior developer.

20
Deployment
  • Easy and fast
  • Developers need to log every deployment and
    obtain management permission to deploy
  • Dlls are copied into server machine
  • Application updater is install on all machines
  • When user reopens their application updater runs
    before software starts and checks to see there
    are new dll versions deployed and grabs puts them
    into the clients machine
  • GAC dlls update automatically

21
Experimental Process
  • To examine the reliability of the software
    instances, I looked into the history of the
    tickets put in for bug changes, specification
    changes and enhancements.
  • The request tickets were matched with the
    deployment logs for each of the specified
    programs.
  • Observations were done on queries to the data
    deployment logs and on interviews/conversations
    with developers.

22
Observations
  • As a project became more complex and big there
    faults and errors were greater than the smaller
    less complex projects.
  • These results were expected since rapid
    development is best suited for projects that are
    smaller and less complex, but what I was surprise
    to see was that these results showed equal
    regardless of whether a senior developer were
    assigned to it or not.
  • Bugs also increased exponentially as less time
    was given for development, it increased even more
    the developers that worked over 60 hrs a week to
    meet deadline. My guess is that due to fatigue
    developers were more prone to making mistakes and
    less likely to catch those mistakes during
    testing because of exhaustion.
  • Programs that reuse code showed a slight better
    rate of faults and errors than the ones that
    didnt or reused very little code.

23
Observations cont.
  • To my surprise, developers with very little field
    experience, experience more errors in their applications those with more
    experience even if it was a few more years.
  • My take in this is that because of their
    inexperience they cannot easily spot possible
    problems during development and during testing.
  • In addition, when the less experience developers
    try fixing or changing requirements for the later
    versions, 28 of the time they ware found to
    introduce another bug, as opposed to 4 rate of
    errors for more experienced developers.

24
Observations cont.
  • Projects that had well define software
    specifications from start had less
    rewrites/redesign of processes and thus had more
    time during the testing phase
  • When a new person was bought on to the project
    after initial rollout or a developer left the
    group, the rate of fault increased initially
  • Projects with greater number of users, though
    initially displayed a greater number of bugs than
    the other software products, they more quickly
    stabilize and became more reliable.

25
Reliability Models
  • Software reliability models are used for the
    estimation and prediction of software
    reliability.
  • Selection of software reliability model for use
    in a particular case has been an area of interest
    for researchers in the field of software
    reliability
  • It is important to point out models that might be
    even more useful in fast paced development
    environments

26
Model Types
  • Times between failures model it focuses on time
    between failures.
  • Failure Count Model is interested in the number
    of faults or failures in a specified time
    interval rather than in times between failures.
    The failures counts are assumed to follow a known
    stochastic process with a time dependent discrete
    or continuous failure rate.
  • Fault seeding model it seeds a known number of
    faults in a program which is assumed to have an
    unknown number of indigenous faults. The program
    is tested and the observer number of seeded and
    native faults is counted to make an estimate of
    fault.
  • Input domain based model generates a set of test
    cases from an input distribution which is assumed
    to representative of the operational usage of the
    program.

27
Model Selection
  • When selecting the proper reliability model it is
    important to considered several factors such as
    utility, applicability and validity.
  • For agile programming it is vital to distinguish
    what might be the most useful methods under such
    extreme time constraints.
  • Times between failure models
  • Time failures over time for a given set of
    observed values may not be the best approach due
    to the fact that failure times are random.
  • As seen earlier having many users increases
    initial rate of errors.
  • Doesnt seem applicable for environments that can
    push out updates daily and even several times a
    day
  • Fault Count models
  • Are the cumulative number of failures versus time
    curve that will eventually level off and this
    model can take into account, calendar time, CPU
    time, number of test cases ran and other relevant
    metrics. Because this model

28
Model Selection Cont.
  • Fault seeding
  • Requires seeding a number of faults in a program
    and then testing the program and counting the
    number of indigenous faults that were exposed.
  • Time consuming process
  • Not great for development environments with no
    time.
  • Im not saying that this model is not useful but
    perhaps not very practical for extreme
    programming. The same applies for input domain
    base models.

29
Conclusion
  • Requirements reliability is very important.
  • Best reliability models for agile programming
    seemed to be failure count models.
  • Also when time not on your side it is important
    to remember to increase reliability
  • Complex and big programs should be given to more
    season developers,
  • Properly defined specifications is important
  • Jr. Developers do a lot better with smaller less
    complex programs where they can obtain the
    necessary experience for fixing and deploying
    bugs correctly
  • Good to main developers through the entire
    project cycle
  • Of course all this observations need to be put to
    test with new development projects to see how
    well they hold.
Write a Comment
User Comments (0)