EC: Lecture 17: Classifier Systems

1
EC Lecture 17 Classifier Systems

• Ata Kaban
• University of Birmingham

2
Classifier Systems

• Rule based systems with input and output interface

classifiersrules IF cond AND cond THEN action
ternary strings (0,1,)
Classifier List
Input Interface
Message List
Output Interface
database of messagesfactsbinary strings
3
The message list

• Finite memory (db) of messages or facts
• Messages are binary strings of fixed length
• Can be visible or not to the external world
• Messages can arrive from several modules
• Input interface
• Classifies
• Messages can be deleted by Output interface
  (after processing them)

4
The classifier list

• Finite size rule base
• Classifier production rule here
• IF cond1 AND cond2 AND condN THEN action
• Represented on fixed nos of words of the form
• Cond1, cond2, condN / action
• Nos conditions fixed
• Conditions can contain NOT
• Conditions are fixed length length of messages
• Over the alphabet 0,1,, where dont care
• Conditions are matched against messages in the
  message list (in terms of Hamming distance)

5
Matching

• Non-negated conditions are satisfied if there is
  at least a message in the message list whose
  Hamming distance from the conditions string is 0,
  disregarding characters
• E.g. message list a 0101, b 1010, c 1111
• condition 0101 satisfied (by a)
• 1101 not satis
• 101 satis (by a)
• 1 satis (by both
  b and c)
• 00 not satis
• satis (by a, b
  and c)
• Conditions with many tend to be unspecific

6
Matching

• Negated conditions are satisfied if no message in
  the message list matches them
• E.g. message list a 0101, b 1010, c 1111
• condition 0101 not satis (matched
  by a)
• 1101 satis
• 101 not satis (a)
• 1 not satis
  (b,c)
• 00 satis
• not satis
  (a,b,c)
• Negated conditions with many s tend to be very
  specific (I.e. can be satisfied by few messages)
• A cond with M hash symbols can be satisfied by
  2L-M messages, L being the message length

7
Actions

• Strings of fixed length over 0,1,
• When a message matches a classifier, the
  classifier(s) are activated a message is built
  as follows
• 0s and 1s in the action string are copied in
  the message
• s are substituted by the corresponding
  characters in the message that matches the first
  condition of the classifier.

8
E.g. actions

• Message list a 0101, b 1010, c 1111
• Classifier list i 11, 110 / 00
• ii 1, 110 / 0
• iii 1, 1110 / 00
• The following set of messages will be produced
• 0011 (posted by I, c matches cond1)
• 0100 (posted by ii, a matches cond1)
• 1110 (posted by ii, c matches cond1)
• 0010 (posted by iii, b matches cond1)
• 0110 (posted bt iii, c matches cond1)

9

• Many classifiers can be activated in parallel!
• Conflict resolution is only necessary if the
  active classifiers can produce more messages than
  entries in the message list

10
The Input Interface

• Initially the db is empty
• The input interface is a mechanism by which the
  CFS can obtain info about the environment,
  through messages
• These messages are often description of the state
  set of (binary) detectors that can sense various
  features of the environment
• E.g. indicate the position of a robot controlled
  by a CFS wrt some obstacles of environment

11
The Output Interface

• The output interface is a mechanism of using (and
  deleting) some messages in the message list
• usually represent (and control) the state of a
  set of effectors which act on the environment ---
  e.g. control the actions of a robot
• Must distinguish between output messages and
  other messages!
• Types of messages
• Input messages (posted by the input interface)
• Internal messages (posted by classifiers to be
  later matched and processed by other classifiers)
• Messages meant to be output messages
• A tag can be used to distinguish between types of
  messages

12
E.g.

• A CFS which has to control a robot might have
  some classifiers devoted to obstacle avoidance
• 1 / 00 0100 00
• 1 / 00 0001 00
• 1 / 00 1000 00
• 1 / 00 0010 00

tag
Conditions (any obstacles left, right, up down?)
action
13
The CFS Main Cycle

• The blocks indicated in the basic CFS diagram are
  activated according to the following main cycle
• Activate the Input Interface post the input
  messages it generates to the message list
• Perform the matching of all the conditions of all
  classifiers against the message list
• Activate classifiers whose conditions are
  satisfied and add their messages to the message
  list
• Activate the output interface, I.e. remove the
  output messages from the message list perform
  the actions they describe
• Repeat the previous steps

14
What is Missing?

• It has been proved that in the presence of
  repetition of the main cycle, using
  characters and at least 2 conditions per
  classifier, a CFS is capable of arbitrarily
  complex computations and of representing
  structures like stacks, lists, etc.
• CFSs can be programmed to do useful things
• BUT are quite difficult to program
• Need to add adaptation mechanisms to the basic
  architecture! so that they can learn to behave
  appropriately in the environment or to perform
  useful tasks.

15
The Need for Competition

• Some classifiers in the system are better than
  others
• i.e. the messages they produce lead to better
  actions
• E.g. for an autonomous agent a good action is the
  action that leads to getting some food.
• In its basic form, a CFS gives the same chances
  to all classifiers to post their messages
• We would like to prioritize good classifiers
• Classifiers should compete to post their
  messages!
• Based on some measure of quality

16
Quality of classifiers

• Usefulness, or strength capability of
  determining a good performance of the whole
  system
• Relevance to a particular situation, or
  specificity (L nos of symbols)/L (where L
  is the length of message)
• Classifiers that match a particular message (or
  few messages only) are specialists for that
  kind of situation --- these are preferred over
  classifiers that match many situations and
  therefore provide a kind of default behaviour for
  the system.
• This measure is needed to ensure the competition

17
Quality of classifiers

• Bid strength and specificity combined
• Bid const strength specificity
• (const is usually cca. 1/10)
• Notes
• To maintain paralelism, we must allow for more
  than one winner in the competition
• To avoid premature convergence, we use
  probabilistic competition, with bid-proportionate
  winning probability
• For a given classifier, specificity is a
  constant, so the strength can only be varied to
  influence the behaviour of a CFS.

18
How to adapt?

• by Credit assignment
• Info about the quality of the behaviour of the
  system comes from the environment in the form of
  rewards (e.g. 1,-1)
• Need to decide which classifiers are responsible
  and to which extent for the good or bad overall
  behaviour
• The Bucket Brigade Algorithm
• If there is a reward (pr punishment), add it to
  the strength of all classifiers active in the
  current major cycle
• Make each active classifier pay its bid to the
  classifiers that prepared the stage for it (I.e.
  posted messages matched by its conditions)
• In time, strength is propagated backwards and
  each classifier receives the correct share of
  credit for the good (or bad) behaviour of the
  system as a whole

19
How to adapt?

• by Rule Discovery
• Discovery of optimum classifiers by means of
  genetic algorithms
• Gas can be used in 2 ways
• Pittsburg approach considering the classifier
  list as a single individual, whose chromosome is
  obtained by concatenating the conditions and
  actions of all classifiers
• Different sets of classifiers compete
• Fitness of each CFS is determined by observing
  the syst for some time
• Michigan approach considering each classifier as
  a separate individual
• Different classifiers compete or cooperate
• Requires a fitness measure for each classifier
  bucket brigade algorithm can be used

20
Main Cycle of Learning Classifier Systems

• Read input messages from sensors
• Find the classifiers that are activated select
  those with highest importance (fitness)
• Reduce the fitness of the matching classifiers
• Clear the message list
• Use the matching classifiers
• Evaluate the resulting behaviour
• Punish or reward the classifiers that were active
  (reinforcement learning)
• Generate a new population of classifiers with an
  evolutionary algorithm
• iterate

21
Summary

• LCS
• Rule based system
• Evolving LCS
• Pitt approach every individual contains all
  rules
• Michigan approach every individual contains a
  single rule
• Fitness based on accuracy of strength value