The expert system development life cycle

1
The expert system development life cycle

2

• Distinctive features of expert systems - 1

3
Distinctive features of expert systems - 1

• To repeat points made in earlier lectures
• Expert systems require special approaches to
  systems analysis, especially to the collection of
  the data (or rather knowledge) on which the
  system is based.

4
Distinctive features of expert systems - 1

• The process of gathering the knowledge to stock
  the expert system's knowledge base - knowledge
  acquisition - has proved to be the most difficult
  component of the knowledge engineering process.
• It's become known as the 'knowledge acquisition
  bottleneck', and expert system projects are more
  likely to fail at this stage than any other.

5
Distinctive features of expert systems - 1

• Knowledge acquisition almost always involves
  extracting knowledge from someone who is expert
  in the field concerned - a domain expert. This
  process - knowledge elicitation - involves a
  variety of interview and non-interview
  techniques.

6

• Distinctive features of expert systems - 2

7
Distinctive features of expert systems - 2

• Expert systems require particular attention to
  the human-computer interface.
• User interfaces for expert systems are more
  troublesome, and harder to develop, than those of
  conventional pieces of software.
• This is because, for various reasons, the
  interactions between computer and user are more
  complex than those involved in a conventional
  piece of software.

8
Distinctive features of expert systems - 2

• For instance
• Conventional software typically performs some
  task, perhaps in interaction with the user.
  Expert systems typically assist with
  decision-making about how a task is to be
  tackled, and this means that the information that
  must be exchanged between the system and the user
  is more complex.
• Different users differ in the sorts of
  problem-solving style they prefer.

9

• Distinctive features of expert systems - 3

10
Distinctive features of expert systems - 3

• Expert systems projects require special
  approaches to software management.
• The methodologies used to build expert systems
  have been shaped by the problems with knowledge
  acquisition, described earlier.
• For a long time, the favourite development
  methodology was rapid prototyping. (Cyclical
  development means more or less the same thing).

11
Distinctive features of expert systems - 3

• In the mid-1980s, this approach came under
  criticism, because it appeared to have all the
  shortcomings of the unstructured approaches which
  had been used, with very poor results, in the
  early days of mainstream software.

12
Distinctive features of expert systems - 3

• But the Structured Systems Analysis Design
  methodologies did not seem to be appropriate,
  because of the differences between knowledge and
  data.

13
Distinctive features of expert systems - 3

• As a result, special-purpose development
  methodologies for knowledge engineering were
  developed. The most well known is KADS, which was
  developed at the University of Amsterdam, as part
  of the ESPRIT programme, in co-operation with
  several European partners.

14
Distinctive features of expert systems - 3

• Nevertheless, many practitioners have not heard
  of KADS, or have rejected it, and continue to use
  rapid prototyping.

15
Distinctive features of expert systems

• Both KADS and a more conventional approach are
  described below.

16

• Cyclical development

17
Cyclical development

• In the 1980s (and before), the favourite
  approach, was based on rapid prototyping (also
  known as cyclical development or evolutionary
  prototyping).

18
Cyclical development

• The essential principles
• get a prototype - a small, preliminary version of
  the final system - up running at an early
  stage
• present this to the domain expert, for criticism
• proceed to refine this prototype with repeated
  debugging knowledge accretion stages
• continue with this cycle until the knowledgebase
  is finished.

19
Cyclical development
Knowledge acquisition
Prototype critiquing
Prototype development
20
Cyclical development
Knowledge acquisition
Preliminary exploration of field - initial k.e.
interviews
Prototype critiquing
Prototype development
21
Cyclical development
Knowledge acquisition
Build a small system containing a few of the
features
Prototype critiquing
Prototype development
22
Cyclical development
Present the prototype to the domain expert for
him/her to criticise and improve
Knowledge acquisition
Prototype critiquing
Prototype development
23
Cyclical development
Knowledge acquisition
Correct expand the knowledge on the basis of
the D.E.s comments
Prototype critiquing
Prototype development
24
Cyclical development
Knowledge acquisition
Add more features, and debug existing features
Prototype critiquing
Prototype development
25
Cyclical development
Present the revised prototype to the domain
expert for him/her to criticise and improve
Knowledge acquisition
Prototype critiquing
Prototype development
26
Cyclical development
Knowledge acquisition
Correct expand the knowledge on the basis of
the D.E.s comments
Prototype critiquing
Prototype development
27
Cyclical development
Knowledge acquisition
Add more features, and debug exisiting features
Prototype critiquing
Prototype development
28
Cyclical development
Present the revised prototype to the domain
expert for him/her to criticise and improve
Knowledge acquisition
Prototype critiquing
Prototype development
29
Cyclical development

• This has the advantage that the KE is able to
  show early progress in the Knowledge elicitation
  task.

30
Cyclical development

• It also generates enthusiasm in the domain
  expert.

31

• Turbans account of the
• expert system development lifecycle

32
The expert system development lifecycle

• The expert system development lifecycle, as
  described by Turban
• Turban (1993) identifies the following phases and
  sub-phases in the development of an expert
  system. This is probably a good account of the
  way knowledge engineering projects are currently
  organised in America.

33
The expert system development lifecycle

• Phase 1 Project initialisation
• Problem definition
• Needs assessment
• Evaluation of alternative solutions
• Verification that an ES approach is appropriate
• Consideration of management issues

34
The expert system development lifecycle

• Comment on Phase 1
• it's important to discover what problem/problems
  the client expects the system to solve for them,
  and what their real needs are. The problem may
  very well be that more knowledge is needed in the
  organisation, but there may be other, better ways
  to provide it.
• 'Management issues' include availability of
  finance, legal constraints, and finding a
  'champion' in top management.

35
The expert system development lifecycle

• Phase 2 System analysis design
• Produce conceptual design
• Decide development strategy
• Decide sources of knowledge, and ensure
  co-operation
• Select computer resources
• Perform a feasibility study
• Perform a cost-benefit analysis

36
The expert system development lifecycle

• Comment on Phase 2
• the 'conceptual design' will describe the
  general capabilities of the intended system, and
  the required resources.
• The problems of selecting software, and finding a
  domain expert (and persuading him/her to
  co-operate) have been discussed in the last two
  lectures.

37
The expert system development lifecycle

• Phase 3 Prototyping
• Build a small prototype
• Test, improve and expand it
• Demonstrate and analyse feasibility
• Complete the design

38
The expert system development lifecycle

• Comments on Phase 3
• Comments See below on the question of what
  exactly this prototype is used for.
• It's important to establish the feasibility
  (economic, technical and operational) of the
  system before too much work has been done, and
  it's easier to do this if a prototype has been
  built.

39
The expert system development lifecycle

• Phase 4 System development
• Build the knowledge base
• Test, evaluate and improve the knowledge base
• Plan for integration

40
The expert system development lifecycle

• Comments on Phase 4
• See below on the question of how the knowledge
  base, as constructed, relates to the prototype.
• The evaluation of an expert system (in terms of
  validation and verification) is a particularly
  difficult problem.

41
The expert system development lifecycle

• Phase 5 Implementation
• Ensure acceptance by users
• Install, demonstrate and deploy the system
• Arrange orientation and training for the users
• Ensure security
• Provide documentation
• Arrange for integration and field testing

42
The expert system development lifecycle

• Comments on Phase 5
• If the system is not accepted by the users, the
  project has largely been a waste of time.
• Field testing (leading to refinement of the
  system) is essential, but may be quite lengthy.

43
The expert system development lifecycle

• Phase 6 Post-implementation
• Operation
• Maintenance
• Upgrading
• Periodic evaluation

44
The expert system development lifecycle

• Comments on Phase 6
• A person or group of people must be put in
  charge of maintenance (and, perhaps, expansion).
  They are responsible for correcting bugs, and
  updating the knowledgebase. They must therefore
  have some knowledge engineering skills.
• The system should be evaluated, once or twice a
  year, in terms of its costs benefits, its
  accuracy, its accessibility, and its acceptance.

45
The expert system development lifecycle

• Turban leaves open the options that the prototype
  which features in phase 3 might be an
  evolutionary prototype or a throwaway prototype.

46
evolutionary prototype or throwaway prototype

• In the 1st case, phase 4 would consist mainly of
  expanding this prototype, by adding more and more
  knowledge, until it became the knowledge base for
  the finished system.
• In the 2nd case, it would consist of drawing
  lessons from building the prototype and using
  these to assist in building the knowledge base
  from scratch, using a more appropriate tool.

47
evolutionary prototype or throwaway prototype

• In other words, the development lifecycle as
  described is either a disguised form of the rapid
  prototyping (cyclical development) procedure
  mentioned earlier, or it is a substantially
  different approach which is liable to produce a
  substantially different knowledge base.

48
evolutionary prototype or throwaway prototype

• The tendency among European knowledge engineers
  is to reject evolutionary prototyping (and rapid
  prototyping / cyclical development) in favour of
  throwaway prototyping. See next section for the
  reasons why.

49
evolutionary prototype or throwaway prototype

• Note that some knowledge engineers would object
  to Turban's sequence of steps on the grounds that
  the computer resources should not be finally
  selected until the precise nature of the
  knowledge had been established. i.e. not until
  after phase 3. Again, see next section.

50

• KADS

51
KADS

• KADS is a development methodology for
  knowledge-based systems.
• KADS was developed at the University of
  Amsterdam, as part of the ESPRIT programme, in
  co-operation with several European partners.

52
KADS

• KADS is
• A methodology for managing knowledge engineering
  projects
• A methodology for performing knowledge
  elicitation.

53
KADS

• KADS objections to rapid prototyping / cyclical
  development
• The interpretation of the verbal data is strongly
  influenced by the implementation formalism.
• i.e., if the shell which the KE has available is
  based exclusively on rules, then everything the
  expert says will be interpreted as rules, or
  discarded.
• This results in a system based on shallow
  knowledge.

54
KADS

• KADS objections to rapid prototyping / cyclical
  development
• Such "shallow" knowledge-based systems, which
  only contain representations of a limited amount
  of the knowledge in the domain, are unable to
  give acceptable explanations about their
  conclusions.

55
KADS

• KADS objections to rapid prototyping / cyclical
  development
• The knowledge in such a system is often
  unstructured, so that it's not clear where or why
  certain rules or guidelines have been included.
• It is generally difficult to adjust and to
  maintain such a system.

56
KADS

• KADS objections to rapid prototyping / cyclical
  development
• There is no way to decide how long the knowledge
  acquisition phase can be expected to last.
• The development tends to last as long as the
  budget permits, and the feasibility is not
  determined until the system is ready. Such
  projects are barely manageable.

57
KADS

• KADS objections to rapid prototyping / cyclical
  development
• Therefore, this approach is highly unsatisfactory
  for the development of commercial systems, where
  the goal is to build a system which can execute a
  predetermined task at a specified level, for a
  predetermined budget.
• For a commercial system, the feasibility and
  scale of the project must be estimated at an
  early stage.

58
KADS

• The KADS alternative
• Build a conceptual model of the domain expert's
  knowledge, including his/her problem solving
  strategies. Take it to its final, complete state
  before doing any program building.

59
KADS

• The KADS alternative
• When eliciting knowledge from the DE, use the
  KADS scheme which organises knowledge into
  several layers
• Facts about entities in the domain, and their
  relationships, are found in the bottom layer.
• Strategies for finding pieces of information (or
  other small-scale tasks) are found in the layer
  above.
• Strategies for performing larger tasks, such as
  doing a diagnosis, are found in the layer above.

60
KADS

• When structuring the domain knowledge in this
  way, be guided by the KADS library of
  Interpretation Models.
• These are descriptions of various different
  general problem-solving tasks. e.g. if you decide
  that the task your expert system must perform is
  "single-fault systematic diagnosis by causal
  tracing", there will be an interpretation model
  corresponding to this. It will describe the
  contents of the various layers, in general terms,
  and give you guidance as to what pieces of
  knowledge you should expect the DE to provide you
  with.

61
KADS

• Imposing a structure on the knowledge elicitation
  task in this way should ensure that
• the expert system that is subsequently built
  contains knowledge which has a suitable degree of
  depth

62
KADS

• Imposing a structure on the knowledge elicitation
  task in this way should ensure that
• the DE's knowledge is stored, in its final form,
  as a set of intermediate representations at a
  later date, a knowledge engineer can revise the
  system, or build a new system, from this stored
  knowledge, rather than having to work with the
  original DE.

63
KADS

• Imposing a structure on the knowledge elicitation
  task in this way should ensure that
• The knowledge elicitation task is shorter, and of
  predictable length.