Knowledge acquisition and processing: new methods for neuro-fuzzy systems

1
Knowledge acquisition and processing new methods
for neuro-fuzzy systems
Danuta Rutkowska Department of Computer
Engineering Technical University of Czestochowa,
Poland E-mail drutko_at_kik.pcz.czest.pl
2
Knowledge Acquisition and Inferencein the
Framework of Soft Computingand Computing with
Words
Cognitive Technologies
3
Soft Computing, Computing with Words, ...

• Soft computing
• Computing with words
• Perception-based systems
• Computational Intelligence
• Artificial Intelligence
• Cognitive sciences
• Neural networks
• Fuzzy systems
• Evolutionary algorithms
• Intelligent systems

4
Soft computing techniques
5
Cognition
The word cognition comes from the latin word
cognitio, which meansknowledge.
Cognitive sciences concern thinking, perception,
reasoning, creation of meaning, and other
functions of a human mind.
6
Soft computing and cognition
The principal aim of soft computing is to exploit
the tolerance of uncertainty and vagueness in
the area of cognitive reasoning.
Nauck D., Kruse R. NEFCLASS-J A
JAVA-Based Soft Computing Tool, In. B. Azvine et
al. (Eds.), Intelligent Systems and Soft
Computing, LNAI 1804, Springer-Verlag,
Heidelberg, New York (2000), pp. 139-160.
7
Artificial Intelligence and cognition
The aim of artificial intelligence is to develop
paradigms or algorithmsthat allow machines to
perform tasksthat involve cognition when
performedby humans
A.P. Sage (ed.), Coincise Encyclopedia
of Information Processing in Systems and
Organization Pergamon Press, New York, 1990
8
Perception and fuzzy systems
Perception is very important in human cognition
The systems that incorporate perceptions
expressed by words are fuzzy systems,
introduced by Prof. L.A. Zadeh.
9
Perception-based systems
Fuzzy systems are rule-based systems(knowledge-ba
sed systems) that can beviewed as
perception-based systems.
The rule base of a fuzzy system is composed of
fuzzy IF-THEN rules that are similar to the rules
used by humans in their reasoning.
10
Learning by examples
Learning by examples is one of the simplest
cognitive capabilities of a young child.
Artificial neural networks with an inductive,
supervised learning algorithm, imitate the
cognitive behaviour.
11
Machine learning
Machine learning research has the potential to
make a profound contribution to the theory and
practice of expert systems, as well as to other
areas of artificial intelligence. Its application
to the problem of deriving rule sets from
examples is already helping to circumvent the
knowledge acquisition bottleneck.
P. Jackson, Introduction to Expert
Systems, Addison Wesley, 1999, Chapter 20, p.399
12
Inductive learning
The most common form ofsupervised learning
taskis called induction.An inductive learning
programis one which is capable oflearning from
examples by a process of generalization.
P. Jackson, Introduction to Expert
Systems, Addison Wesley, 1999, Chapter 20, p.381
13
Neural network (MLP)
14
Model of an artificial neuron
15
RBF network
16
Gaussian function
17
Normalized RBF network
18
General neuro-fuzzy architecture
19
Fuzzy reasoning for k-th rule
consequent
antecedent
k-th rule
input variable
output variable
fuzzy relation
fuzzification
input value
k-th output fuzzy set
input fuzzy set
20
Aggregation and defuzzification
aggregation for Mamdani approach
aggregation for logical approach
T-norm
output fuzzy set for all N rules
S-norm
defuzzification
centre of consequent fuzzy set Bk
output value
21
Fuzzy implications Mamdani, logical
Mamdani approach
logical approach
22
An example of a neuro-fuzzy network
23
More general form of this network
24
Another example of the NF network
25
T-norm
A triangular norm T is a function of two
arguments T 0,10,1?0,1 which satisfies
the following conditions for a,b,c,d?0,1 Mono
tonicity T(a,b)T(c,d) ac bd Commutativity
T(a,b)T(b,a) Associativity T
(T(a,b),c)T(a,T(b,c)) Boundary conditions
T(a,0)0 T(a,1)a
26
T-conorm (S-norm)
A T-conorm (S-norm) is a function of
two arguments S 0,10,1?0,1, which
satisfies the following conditions for
a,b,c,d?0,1 Monotonicity S(a,b)S(c,d) ac
bd Commutativity S(a,b)S(b,a) Associ
ativity S (S(a,b),c)S(a,S(b,c))
Boundary conditions S(a,0)a S(a,1)1
27
Neuro-fuzzy inference systems (NFIS)
28
Fuzzy-logic inference system
29
Fuzzy-logic inference system fuzzifier
30
Fuzzy-logicinference systemfuzzy rule base
31
Fuzzy-logic inference system fuzzy inference
engine
32
Fuzzy-logicinference systemdefuzzifier
33
General architecture of Neuro-Fuzzy Inference
System
NFIS
34
Flexible neuro-fuzzysystemMamdani approach
35
Definition Fuzzy implication
A fuzzy implication is a function I0,12?0,1
satisfying the following conditions (I1) if
a1a3 then I(a1,a2)I(a3,a2), for all
a1,a2,a3?0,1 (I2) if a2a3 then
I(a1,a2)I(a1,a3), for all a1,a2,a3?0,1 (I3) I
(0,a2)1, for all a2?0,1 (falsity implies
anything) (I4) I(a1,1)1, for all
a1?0,1 (anything implies tautology) (I5) I(1,0
)0 (booleanity)
36
Fuzzy implications
37
Flexible neuro-fuzzysystemLogical approach
38
Flexible neuro-fuzzy system AND-type compromise
NFIS
39
Flexible neuro-fuzzy system OR-type compromise
NFIS
40
Flexible neuro-fuzzy system
L. Rutkowski and K. Cpalka Flexible Neuro-Fuzzy
Systems, IEEE Trans. Neural Networks, vol. 14,
pp. 554-574, May 2003
41
Flexible neuro-fuzzy system Soft NFIS (1/2)
42
Flexible neuro-fuzzy system Soft NFIS (2/2)
43
Flexible neuro-fuzzy system NFIS realized by
parameterised families of triangular norms (1/2)
44
Flexible neuro-fuzzy system NFIS realized by
parameterised families of triangular norms (2/2)
45
Flexible neuro-fuzzy system NFIS realized by
triangular norms with weighted arguments (1/2)
46
Flexible neuro-fuzzy system NFIS realized by
triangular norms with weighted arguments (2/2)
47
Flexible neuro-fuzzy system Glass
Identification experimental results
48
Flexibleneuro-fuzzysystem Glass
Identification weights representation
Weights representation in the Glass
Identification problem (dark areas correspond to
low values and vice versa)
49
Flexible neuro-fuzzy system Glass
Identification comparison table
50
Neuro-fuzzy relational system
51
Neuro-fuzzy relational system with fuzzy matrix R
52
Neuro-fuzzy connectionist system (basic
architecture)
53
Rule generation
The neuro-fuzzy networks reflect fuzzy IF-THEN
rules.
The network architectures are created based on
the rules.
How to get the rules ?
54
Basic questions

• How many rules ?

• What kind of the membership functions
• (Gaussian, triangular, trapezoidal, etc.) ?
• How to determine parameter values
• of the membership functions (centers, widths)
  ?

55
Many methods
There are many methods of rule generation.
However, most of the rules obtained by these
methods, when applied in neuro-fuzzy systems for
classification, result in some misclassifications.
56
Perception-based approach
This method generates fuzzy IF-THEN rules, from a
data set, by use of fuzzy granulation.
The neuro-fuzzy systems, which utilize these
rules, perform without misclassifications.
57
Multi-stage classification
The perception-based approach allows to generate
fuzzy rules and perform a multi-stage classificati
on without misclassifications.
This method will be illustrated on the IRIS
example.
58
IRIS data set
150 data items that contain measurements of iris
flowers from three species of iris Setosa,
Versicolor, and Virginica 50 data items for
each of the iris species.
The data include information about four features
of the iris flowers sepal length, sepal width,
petal length, petal width.
59
Ranges of the measurementsof iris flowers (in
centimeters)
60
Ranges within the classes
Setosa Versicolor Virginica
Sepal length 4.3 5.8 4.9 7.0 4.9 7.9
Sepal width 2.3 4.4 2.0 3.4 2.2 3.8
Petal length 1.0 1.9 3.0 5.1 4.5 6.9
Petal width 0.1 0.6 1.0 1.8 1.4 2.5
61
Granulated ranges of sepal length
62
Granulated ranges of sepal width
63
Granulated ranges of petal length
64
Granulated ranges of petal width
65
Linguistic labels for sepal length
66
Linguistic labels for sepal width
67
Linguistic labels for petal length
68
Linguistic labels for petal width
69
Rule 1
IF sepal is short or medium long and medium wide
or wide or very wide and petal is very short and
very narrow THEN Setosa
IF x1 is and x2 is and x3 is and x4
is THEN Setosa
70
Rule 2
IF sepal is medium long or long and very narrow
or narrow or medium wide and petal is medium long
or long and medium wide or wide THEN Versicolor
IF x1 is and x2 is and x3 is and x4
is THEN Versicolor
71
Rule 3
IF sepal is medium long or long or very long and
narrow or medium wide or wide and petal is long
or very long and wide or very wide THEN
Virginica
IF x1 is and x2 is and x3 is and x4 is
THEN Virginica
72
NF network for the iris classification
73
Results of the 1st stage classification
50 data vectors correctly classified to Setosa 32
data vectors correctly classified to
Versicolor 42 data vectors correctly classified
to Virginica 26 data vectors I do not know
decision
Versicolor or Virginica These data vectors
participate in the 2nd stage of the
classification.
74
2nd stage classification
Two fuzzy IF-THEN rules are formulated, based on
the granulated ranges, obtained for the data
vectors with the I do not know decision in the
1st stage.
The NF network in the 2nd stage is reduced to the
components associated with the Versicolor and
Virginica classes.
75
Results of the 2nd stage classification
12 data vectors correctly classified to
Versicolor 1 data vector correctly classified to
Virginica
13 data vectors I do not know decision
Versicolor or
Virginica These data vectors participate in the
3rd stage of the classification. Two new rules
are created.
76
Results of the 3rd stage classification
4 data vectors correctly classified to
Versicolor 5 data vectors correctly classified to
Virginica
4 data vectors I do not know decision
Versicolor or
Virginica These data vectors participate in the
4th stage of the classification. Two new rules
are created.
77
Results of the 4th stage classification
2 data vectors correctly classified to
Versicolor 2 data vectors correctly classified to
Virginica
All data vectors correctly classified after 4
stages of the classification.
No misclassifications !
78
IRIS data P1, P2
79
IRIS data P1, P3
80
IRIS data P2, P4
81
IRIS data P3, P4
82
Diagnosis of a tumor of mucous membrane of uterus

• Attributes
• period of time after menopause
• BMI (Body Mass Index)
• LH (luteinizing hormone )
• FSH (follicle-stimulating hormone )
• PRL (prolactin )
• E1 (estron)
• E2 (estradiol)
• Aromatase
• estrogenic receptor

9 attributes
Data 52 records of positive diagnosis 13 records
of negative diagnosis
Diagnosis negative (class 0), positive (class 1)
83
Ranges of the attribute values
0.5 - 34
20 - 46
0.5 120.3
1.36 155.4
2.4 128.1
156 - 542
0.04 1.48
2.28 11.85
0.72 3.85
84
Ranges within the classes
Class 0 Class 1
0.5 - 20 0.5 - 34
20 - 46 20 - 45
1.2 53.9 0.5 120.3
1.63 88.2 1.36 155.4
3.4 128.1 2.4 76.6
170 - 412 156 - 542
0.04 0.27 0.05 1.48
2.28 10.51 3 11,85
0.72 1.05 0.91 3.85
85
Rules for the medical diagnosis
86
NF network for the medical diagnosis
87
Results correct diagnosis
3 cases with the I do not know response after
the first stage of classification
62 correct diagnosis for all 65 input vectors.
(95.4 correct decisions, 4.6 I do not know )
The I do not know answers, which mean positive
or negative diagnosis, refer to the cases that
are difficult to be recognized, because they
belong to overlapping regions.
88
Conclusions (perception-based classification)
The perception-based approach allows to generate
fuzzy IF-THEN rules in the same way as humans do,
and perform the multi-stage classification without
misclassifications.
89
Final conclusions
Neuro-fuzzy systems are soft computing methods
utilizing artificial neural networks and fuzzy
systems. Various connectionist architectures of
neuro-fuzzy systems can be constructed. The
knowledge acquisition concerns fuzzy IF-THEN
rules, and is performed by a learning process.
The systems realize an inference (fuzzy
reasoning) based on these rules.