RUP and Iterative Development

1
RUP and Iterative Development

• A Software Lifecycle Automation Approach
• Tom Schultz CTO Codefast Inc.

2
What youll Learn

• Examine the challenges in adoption of process
  through the construction and transition phases
• Understand how to enhance RUP best practices in
  construction and transition
• Discuss the implementation of these best
  practices
• Learn the benefits of Codefast Lifecycle
  Automation

3
Why adopt a process
?
4
Why adopt a process

• why does it take so long to get programs
  finished?
• why do we have difficulty in measuring progress
  as software is being developed?
• why are development costs so high?
• why doesnt the software meet our needs?
• why cant we find all errors before we give the
  software to our customers?
• why do we spend so much money maintaining our
  software?

5
Reality Check

• Our cost structure for building software exceeds
  what we are given
• Our ability to build new programs cannot keep
  pace with the demand for new programs
• Our ability to maintain existing programs is
  threatened by poor design and inadequate
  resources
• Governmental and other compliance issues

6
Scale Adds Complexity

• Scale Forces More Variation
• More people do things different ways
• More customers have more varied needs
• Large apps vary by customer needs
• Large apps see generations of tools, technologies
• Tooling, versions, platforms all change over time
• Scale Forces More Complexity
• Large apps go through multiple versions
• More products, files, languages, platforms
• More added features, variants, special cases
• Large apps must interface to more customer systems

7
Lifecycle Processes

• Classic life-cycle (waterfall model)
• Prototyping
• Spiral or Iterative model
• eXtreme programming

8
Rational Unified Process

• RUP is a framework that emphasizes the adoption
  of certain best practices of modern software
  development, as a way to reduce the risk in
  development
• Develop iteratively
• Manage requirements
• Use component-based architectures
• Model visually
• Continuously verify quality
• Control change

9
RUP Phases and Iterations
10
RUP coverage on Projects
Inception Elaboration-Construction-Transition

• Back End Lifecycle
• Code generation
• Configuration Management
• Software Builds
• Software Integration
• Regression Tests
• Documentation
• Software Release
• Software Deployment
• Report Generation
• Web Deployment

• Front End Lifecycle
• Feasibility Study
• Analysis
• Requirements
• Use Cases
• System Modeling
• Architecture
• Database Schemas
• Code Design
• Coding
• Test Case Design

11
Construction - Transition

• The goal of Construction is to complete the
  development of the system
• Construction phase is, in some sense, a
  manufacturing process
• Is the transformation of Inception and
  Elaboration artifacts into deployable products

12
Essential Activites

• Manage resources and control process
• Everyone needs to know who will do what and when.
  Insure that the workload does not exceed your
  capacity and that work is to schedule.
• Develop and test components.
• You build the components required to satisfy the
  use cases, scenarios, and other functionality for
  the iteration
• You test them with unit and integration tests.
• Assess the iteration
• Satisfied the goals of the iteration or
  re-prioritize and manage the scope to meet your
  delivery date.

13
Issues in these Activities

• Build
• Package
• Test and Test Configuration
• Deployment
• Metrics

14
Build Issues

• Membership in the build
• What (past) version of the code are we building?
• What new features go into this release?
• What new patches go into this release?
• What code is used for this platform?
• We have to integrate all these pieces.
• Processes in the build
• Different EJB compilers for different app servers
• Different tooling for different platforms
• Different options for each operating system

15
Package Issues

• Membership in the package
• What applications should be packaged?
• What auxiliary data files should be packaged?
• Configuration files, data files
• Documentation files (internationalization)
• What test data/harnesses should be packaged?
• Processes in the package
• Package format varies by languages (eg Java vs C)
• Package tools vary by language, platform

16
Testing Issues

• Membership in the test run
• Different tests for application versions
• Different tests for new feature blocks
• Different tests for bug fixes
• Processes in the test run
• Deployment of application into test environment
• Initialization of database tables, test data
• Run tests, report, refresh test data, run more
  tests
• Test harnesses, tooling, tests, data all vary by
  language, platform, version, application features

17
Test Configuration Issues

• Multiple Test Configurations Are Normal
• Vary by platform (Linux, Solaris, AIX, Windows)
• Vary by container (Websphere, JBoss, Weblogic)
• Vary by database (Postgres, Oracle, DB2, MySQL)
• Vary by feature set (default, OEM, per-customer)
• 3 platforms 3 containers 4 DBs 36
  configurations
• Why They Dont Get Tested
• Too much work for humans to configure everything
• Setup, test, fail Fix, test, fail. Fix, test,
  fail.
• Picky little things make the test runs fail

18
Customer Deployment Issues

• Membership in the deployment
• What version is being deployed?
• What features are being enabled/deployed?
• Need adaptors to match customer environment?
• Processes in the deployment
• Install processes vary by version, features,
  platform
• Must connect application to customer environment
• Databases, hosts, ports, tooling, websites, etc
• Acceptance test processes vary in multiple ways

19
Software Development Value Chain
Developers
Projects
Customers

• Back End Lifecycle
• Process Code Generation
• Software Builds
• Software Integration
• Regression Tests
• Software Releases
• Software Deployment
• Test Database Setup
• Internal Documentation
• Web Generation
• Web Deployment

20
Need to Enhance best practices

• A number of new techniques, many also being
  adopted by the XP and Agile development movements
  can aid and increase the speed and quality of
  development
• RUP as a framework embraces these concepts

21
enhanced Best Practices

• Automate the Process
• Practice continuous integration and employ
  automated testing environments
• Release Software Early and Often

22
Automate the Process
Wall of Responsibility
Developer Responsible
System Responsible

• Lifecycle Automation System
• Back End Lifecycle
• Process code generation
• Software Builds
• Software Integration
• Regression Tests
• Software Release
• Software Deployment
• Test Database Setup
• Internal Documentation
• Web Page Generation
• Web Deployment

• Front End Lifecycle
• Feasibility Study
• Analysis
• Requirements
• Use Cases
• System Modeling
• Architecture
• Database Schemas
• Code Design
• Coding
• Test Case Design

23
Continuous Integration

• Practice continuous integration and employ
  automated testing environments
• Many more smaller incremental baselines
• Extend automated testing to both unit and
  functional testing

24
Release Early and Often

• Open source and Agile development both use
  shorter, more frequent release schedules.
• Delivers value to customers early on, rather than
  making customers wait until the full release
  before beginning to capture value.
• Early Release allows for much faster
  time-to-value

25
Issues with adoption

• Manual, personnel intensive area of lifecycle
• Vendor Tool support lacking
• Little vendor assistance in adoption

26
How to Technical Overview of Lifecycle Automation
27
Technical Objectives

• Automate the whole back-end lifecycle
• Under the previous assumptions
• Using smart, knowledge-based automation
• No human labor required
• Capture incremental human effort
• Efficient knowledge reuse
• 1 person creates, N people reuse at zero cost
• Integrate within existing customer environment

28
Functional Architecture
Abstraction
Integration
Knowledge Base
Orchestration
Generation
Execution
Reporting
29
Abstraction

• Collections and Collection Types
• Fundamental unit of manipulation
• Contain common properties
• Think objects and classes
• Stored in Knowledge System
• Knowledge base provides the structured knowledge
  to dynamically generate and execute build
  processes
• Text readable file format

30
Collection Model
31
Orchestration
Build
Traditonal concept of "build"
32
Generation

• Codefast generates all infrastructure process
  code
• Supports popular Scripting environments
• Ant
• Make
• Perl
• Why Process Code???

33
Root Cause Of Pain Process Code
Process Code Build
Source Files
Process CodeTest
Test Cases
Development Artifacts
Process Code Package
Packaging Specs
Process Code Deploy
Deployment Specs
Area Of Human Labor
34
What is Process Code?

• Code that
• is not managed or addressed by productivity tools
• is not part of the delivered application
• is in a 11 ratio with file-based source code
• is based on makefile-level thinking
• is complex, brittle, and labor intensive
• requires constant attention and maintenance
  effort
• Code that adds NO VALUE to end-users or customers

35
Root Cause Of Pain Process Code
Process Code Build
Source Files
Process CodeTest
Test Cases
Development Artifacts
Process Code Package
Packaging Specs
Process Code Deploy
Deployment Specs
Area Of Human Labor
36
The Solution Lifecycle Automation
Process Code Build
Source Files
Process CodeTest
Test Cases
Development Artifacts
Process Code Package
Packaging Specs
Process Code Deploy
Deployment Specs
Area Of Lifecycle Automation
37
Codefast Generation Execution
Analyze Source Collections
Analyze Knowledge Base
Generate Process Code
Process Code Generators
Execute Process Code
Scalable Execution System
38
Parallelism

• Codefast supports several layers of parallelism
• Orchestrated Tasks
• Build Order
• Dependencies
• Execution servers provide distributed parallel
  environment

39
Reporting

• Logging
• Robust logging of steps
• Notification
• Email notification
• Metrics
• Six Sigma, etc.

40
Integration
Developer Productivity
Unit Testing
Deployment
SCM
Defects
Functional Testing
Packaging
KB
Files
41
Current and planned Integration

• SCM
• Rational CC, Perforce, CVS, MKS
• IDE
• VS .Net 2005, Eclipse
• Testing
• Mercury, Junit, Rational
• Deployment
• Tivioli

42
Example Results of Codefast Method

• Perfect Builds
• 1M LOC, 20 platforms, new guarantee every hour
• Incremental, encapsulated process techniques
• 99.8 success rate, 4 failures in 2500 over 3
  months
• Perfect Tests
• 3132 test runs over 4 months
• 98.5 average success rate
• Perfect Builds Perfect Tests Perfect
  Releases
• No Human Labor Involved All Automated

43
How Development Should Be

• Developer Checks In Some Code Changes
• Codefast Software Lifecycle Automation
• Calculates and generates optimal build/process
  scripts
• For all variations of languages, platforms,
  versions, and tools
• Optimized for parallel, distributed builds
• And Automatically Executes a Build/Package/Test
  Cycle
• Builds applications across various target
  configurations
• Deploys applications onto every target test bed
  for testing
• On each target, executes the regression tests
• On each target, executes functional tests for the
  code changes
• Makes installation packages for the target
  configurations
• Reports results via email
• We can provide this TODAY!

44
Benefits

• Reduce labor costs and increase time to value
  through automation
• Enable adoption of modern software development
  tools and best practices
• Achieve compliance standards

45
Summary

• RUP can be enhanced in the construction and
  transition areas
• Adoption of these best practices enhances time to
  value, quality and reduces cost
• Codefast Software Lifecycle Automation provides
  this infrastructure

46
The Power of Automation

• Removes Human Knowledge Burden
• Removes Human Labor Burden
• Removes Human Responsibility
• Automation vs. Human Labor