Construction by Configuration: An opportunity for SE research

1
Construction by Configuration An opportunity
for SE research

• Prof. Ian SommervilleSt Andrews
  UniversityScotland

2
St Andrews

• Small Scottish town, on the north-east coast of
  the UK
• Home of golf
• Scotlands oldest university (founded in 1413)
• Small university focusing on research and
  teaching excellence

3
Software engineering 2008

• Over the past 10 years or so, there has been a
  radical shift in development strategy for
  large-scale organisational information systems
• Instead of developing software by programming in
  a conventional language, the primary development
  mode is now based around the reuse of
  off-the-shelf systems such as ERP systems or
  generic systems (COTS) for a specific application
  area
• Existing approaches to software engineering have
  to evolve to take into account this approach to
  systems engineering

4
Commercial Off-the-Shelf Systems (COTS)

• COTS-based reuse relies on adopting a generic
  system that has been developed to deliver some
  business function and adapting that systems to
  the needs of a particular organisation
• Increasingly, new systems are developed by
  integrating several existing COTS, sometimes with
  different owners. Systems are becoming systems of
  systems

5
COTS-based reuse

• COTS-based reuse is based around systems for a
  specific application domain (e.g. patient
  information system in healthcare).
• This incorporates generic functionality and
  abstractions from the domain which are configured
  for each specific customer. COTS are designed for
  configuration.
• This can be a (very) long-term process
• The current Spanish en-route air traffic control
  system is being reconfigured for use in the UK.
  The process is anticipated to take 6 years.

6
Construction by configuration (C-b-C)

• COTS and ERP systems include a set of reusable
  abstractions that represent an application domain
• The reusable abstractions have to be configured
  to adapt them to their local circumstances of use
• This can range from simple parameter setting
  through the definition of business rules
• Sometimes, configuration may require new,
  special purpose component development.
• C-b-C also may also include configuring the
  process as well as configuring the software that
  is the way people work has to change to meet the
  model assumed by the COTS system

7
Configuration

• I use the term configuration to cover both
  customisation and adapting software to a specific
  execution environment
• Adapting the system to reflect
• The specific needs of a customer (e.g. a
  hospital)
• The requirements of a user or group of users
  (e.g. maternity unit or physiotherapy or both)
• Interaction with other systems
• The characteristics of the platform on which the
  software executes.

8
Configuration activities

• Selecting required system modules
• Defining a data model
• Defining business rules
• Defining workflows
• Defining external interactions
• Defining the user interface
• Defining reports produced
• Setting platform parameters
• Data re-engineering
• Re-defining business processes

9
A patient management system

• We have followed the deployment of a patient
  management system (PIMS) for mental healthcare in
  Edinburgh, Scotland
• Based around a generic COTS-package developed for
  hospitals in England
• This was designed to be adapted for supporting
  different kinds of clinic including mental
  healthcare
• Scotland has its own legal system and laws and
  healthcare is a devolved responsibility. The
  Scottish government sets its own targets and
  priorities for healthcare

10
Procurement issues

• A major influence in choosing the particular COTS
  system was that it offered the opportunity for
  hospital managers (rather than clinical staff) to
  control how information was recorded
• The government placed a tight deadline on
  hospitals for reporting against a set of targets
• There was little time to carry out a detailed
  comparison of alternatives and this system had
  already been successfully deployed elsewhere

11
Software engineering

• Configuring the data model required for a
  particular set of clinics
• Configuring the menus for the particular type of
  clinic and the patient information that had to be
  recorded
• Configuring the reports to be generated by the
  system
• Configuring the rules that should be applied to
  the system data

12
Issues and problems

• Objectives conflict
• Management objective - provide reports against a
  set of headings defined by the Executive
• Clinical objectives - record patient information
  according to clinical classification
• Invalid configuration assumptions
• The language for defining the rules of the system
  was not expressive enough to cover the
  requirements of Scots law regarding the forced
  detention of patients who were a danger to
  themselves or others
• Failure to configure the process
• Local process differences meant that different
  clinics recorded different information about
  patients

13
The C-b-C process for COTS

• There is often no clear distinction between
  specification, design and development
• Systems are rarely completely configured before
  being put into use. The configuration process
  continues as the system is integrated with
  operational processes
• There can be extensive (uncontrolled) user
  configuration of the system
• The expectations of system stakeholders have to
  be configured to reflect the realities of he
  system

14
Process characteristics

• Two-stage system requirements process
• Identify general business requirements to chose a
  reusable system
• Identify specific requirements from the settings
  where the system will be used.
• Business process redesign
• The business processes have to be redesigned to
  r4flect the constraints of the selected system
• Co-design of process and software
• May be more active stakeholder involvement in the
  development process.
• Limited testing - often post-deployment testing
  to iron out problems
• Good practices such as configuration management,
  reviews, etc. are very difficult to implement

15
Choosing a COTS system

• The decision on which COTS system to use is
  rarely a transparent process, based on a detailed
  analysis of the requirements in a specific
  setting
• Issues that affect the decision include
• Political issues
• Platform issues
• Cost and schedule issues
• Availability of expertise
• Prejudice!

16
Co-design

• For C-by-C, separating requirements and
  development/deployment is unlikely to be
  successful
• Requirements compromises are essential because
  the stakeholders real needs need to be matched
  with what the system actually provides
• Configurability makes co-design, with stakeholder
  involvement in the design process, much easier
• However, users may misunderstand the (lack of)
  system flexibility

17
System testing

• Testing is a particular problem for COTS-based
  systems
• Systems are not designed for running automated
  test suites
• There is no specification that can serve as a
  basis for deriving tests
• Failures are often not observable by testers as
  they are failures in process support that can
  only be recognised by users
• Problems that arise are often a consequence of
  interactions between the process of use and the
  system rather than system failure.

18
System evolution

• Handover process from consultants to local IT
  staff. Change of system ownership
• Limited expertise with system - may be managers
  rather than software engineers
• Problems of change management exacerbated because
  of the system configurability
• Increasing system entropy as local configuration
  changes are implemented
• Evolution of underlying COTS system outside
  control of system owners

19
Configuration problems

• Understanding the configuration possibilities
• Knowing what can be configured is not easy,
  especially if more than one product is included
  in the system
• Understanding how to configure a system
• Typically, configuration requires both business
  knowledge and technical knowledge
• Predicting the consequences of configuration
  decisions
• It is often difficult to understand how changing
  a configuration will affect the use and behaviour
  of a system while it is in operation
• Understanding how a system is configured

20
Multiple configuration possibilities

• Ways in which MS Word can be configured (that Im
  aware of)
• Preferences screen
• Customisation screen
• Organiser screen
• Definition of templates
• Definition of styles
• Definition of macros
• Inclusion of add-ins (e.g. Endnote)

21
Understanding how to configure
22
Configuration predictability

• If you change a program, it is generally possible
  to hypothesise how this will affect the behaviour
  of the executing system
• When you change a configuration, the relationship
  is less obvious
• Change becomes a process of ad hoc
  experimentation. Gurus evolve who can suggest
  what to do but who cant explain why it should be
  done that way

23
Understanding the configuration

• Once a system has been configured, how can others
  understand the configuration.
• Requirements/design/code traceability is
  difficult enough in conventional systems but much
  harder in COTS where
• Requirements are not properly documented as a
  result of the co-design process
• Understanding requires knowledge of the COTS
  system knowledge of the configuration.
• Change costing and impact analysis is difficult
• E.g. in the healthcare system we looked at, it
  was originally estimated that changing menus
  would take a few days. It ended up taking several
  months.

24
Research challenges

• Design for configuration
• Design principles for configurability
• Configuration visibility
• Configuration description
• Configuration engineering
• Knowledge management for system engineering
• System modelling
• Methods and tools for COTS-based system testing
• Adapt supporting software engineering processes
  for C-by-C

25
Principles for configurable design

• We need a coherent set of tested principles in
  this area to guide design and implementation
• Possible examples of principles
• Principle of visibility - make configurations
  explicit?
• Principle of low coupling - reduce dependencies
  across configurations?
• Principle of scalability - separate configuration
  of system deployment from configuration of
  functionality?
• Principle of localisation - localise volatile
  configuration entities?
• The diversity of different approaches to C-b-C
  mean that identifying unifying principles across
  different systems is very difficult

26
Configuration visualisation

• In most current systems, configurations are
  opaque - it is difficult or impossible to view
  the configuration as a whole, at different levels
  of abstraction or to navigate through the
  configuration space
• There is a need for tools that allow engineers to
  see the configuration state of a system and to
  explore dependencies across that state
• What configuration information is most useful to
  system developers?
• What is the best way to present configuration
  views to system developers?

27
Configuration description

• Configuration policies
• There is sometimes a wide gap between
  organisational policy (especially, security
  policy) and the realisation of that policy as a
  configuration
• Configuration compilation
• Methods and tools are needed to allow higher
  level configuration specification and the
  automatic generation of a configuration from that
  specification

28
Knowledge management

• Configuration engineering requires a close
  collaboration between different classes of system
  user, business analysts and system developers
• Our experience has been that major problems arise
  because of failure to communicate across and
  within the different groups in an organisation
• Better knowledge management support is required
  to capture, classify and share knowledge about
  assumptions, problems, the organisation and the
  system itself

29
Configuration modelling

• Modelling notations such as the UML are focused
  on developing object-oriented models of the
  system
• While some UML models (such as Use-Case models)
  can be used for configuration engineering, the
  majority of the UML models are either
  inappropriate or cannot be applied in a natural
  way when configuring a generic system
• What domain-oriented modelling techniques are
  most useful, how can they be used and adapted to
  support understanding and analysis of systems
  being built using CbyC?

30
System testing

• Workflow testing
• How can workflow models built into the system be
  tested against the reality of work?
• Test management
• How can test coverage be assessed and improved?
• How can regression testing be supported?
• Test-first development
• Is this a valid approach for systems developed
  using CbC?

31
Supporting processes

• Users may be partly responsible for parts of the
  configuration process. This causes major problems
  with supporting processes
• Configuration management
• Existing tools are oriented to source code
• Change management
• Can users be stopped from making ad hoc changes?
  How do users find out about changes?
• Quality management
• What does a high-quality configuration mean?

32
Conclusions

• Construction by configuration is a reality for
  modern business software development
• Political, technical and business factors mean
  that this approach will be increasingly used for
  all types of system
• Our current ad hoc approaches are not good
  enough - we need to adapt software engineering
  methods and techniques for this approach.