Title: Knowledge acquisition and processing: new methods for neuro-fuzzy systems
1Knowledge acquisition and processing new methods
for neuro-fuzzy systems
Danuta Rutkowska Department of Computer
Engineering Technical University of Czestochowa,
Poland E-mail drutko_at_kik.pcz.czest.pl
2Knowledge Acquisition and Inferencein the
Framework of Soft Computingand Computing with
Words
Cognitive Technologies
3Soft Computing, Computing with Words, ...
- Soft computing
- Computing with words
- Perception-based systems
- Computational Intelligence
- Artificial Intelligence
- Cognitive sciences
- Neural networks
- Fuzzy systems
- Evolutionary algorithms
- Intelligent systems
4Soft computing techniques
5Cognition
The word cognition comes from the latin word
cognitio, which meansknowledge.
Cognitive sciences concern thinking, perception,
reasoning, creation of meaning, and other
functions of a human mind.
6Soft computing and cognition
The principal aim of soft computing is to exploit
the tolerance of uncertainty and vagueness in
the area of cognitive reasoning.
Nauck D., Kruse R. NEFCLASS-J A
JAVA-Based Soft Computing Tool, In. B. Azvine et
al. (Eds.), Intelligent Systems and Soft
Computing, LNAI 1804, Springer-Verlag,
Heidelberg, New York (2000), pp. 139-160.
7Artificial Intelligence and cognition
The aim of artificial intelligence is to develop
paradigms or algorithmsthat allow machines to
perform tasksthat involve cognition when
performedby humans
A.P. Sage (ed.), Coincise Encyclopedia
of Information Processing in Systems and
Organization Pergamon Press, New York, 1990
8Perception and fuzzy systems
Perception is very important in human cognition
The systems that incorporate perceptions
expressed by words are fuzzy systems,
introduced by Prof. L.A. Zadeh.
9Perception-based systems
Fuzzy systems are rule-based systems(knowledge-ba
sed systems) that can beviewed as
perception-based systems.
The rule base of a fuzzy system is composed of
fuzzy IF-THEN rules that are similar to the rules
used by humans in their reasoning.
10Learning by examples
Learning by examples is one of the simplest
cognitive capabilities of a young child.
Artificial neural networks with an inductive,
supervised learning algorithm, imitate the
cognitive behaviour.
11Machine learning
Machine learning research has the potential to
make a profound contribution to the theory and
practice of expert systems, as well as to other
areas of artificial intelligence. Its application
to the problem of deriving rule sets from
examples is already helping to circumvent the
knowledge acquisition bottleneck.
P. Jackson, Introduction to Expert
Systems, Addison Wesley, 1999, Chapter 20, p.399
12Inductive learning
The most common form ofsupervised learning
taskis called induction.An inductive learning
programis one which is capable oflearning from
examples by a process of generalization.
P. Jackson, Introduction to Expert
Systems, Addison Wesley, 1999, Chapter 20, p.381
13Neural network (MLP)
14Model of an artificial neuron
15RBF network
16Gaussian function
17Normalized RBF network
18General neuro-fuzzy architecture
19Fuzzy reasoning for k-th rule
consequent
antecedent
k-th rule
input variable
output variable
fuzzy relation
fuzzification
input value
k-th output fuzzy set
input fuzzy set
20Aggregation and defuzzification
aggregation for Mamdani approach
aggregation for logical approach
T-norm
output fuzzy set for all N rules
S-norm
defuzzification
centre of consequent fuzzy set Bk
output value
21Fuzzy implications Mamdani, logical
Mamdani approach
logical approach
22An example of a neuro-fuzzy network
23More general form of this network
24Another example of the NF network
25T-norm
A triangular norm T is a function of two
arguments T 0,10,1?0,1 which satisfies
the following conditions for a,b,c,d?0,1 Mono
tonicity T(a,b)T(c,d) ac bd Commutativity
T(a,b)T(b,a) Associativity T
(T(a,b),c)T(a,T(b,c)) Boundary conditions
T(a,0)0 T(a,1)a
26T-conorm (S-norm)
A T-conorm (S-norm) is a function of
two arguments S 0,10,1?0,1, which
satisfies the following conditions for
a,b,c,d?0,1 Monotonicity S(a,b)S(c,d) ac
bd Commutativity S(a,b)S(b,a) Associ
ativity S (S(a,b),c)S(a,S(b,c))
Boundary conditions S(a,0)a S(a,1)1
27Neuro-fuzzy inference systems (NFIS)
28Fuzzy-logic inference system
29Fuzzy-logic inference system fuzzifier
30Fuzzy-logicinference systemfuzzy rule base
31Fuzzy-logic inference system fuzzy inference
engine
32Fuzzy-logicinference systemdefuzzifier
33General architecture of Neuro-Fuzzy Inference
System
NFIS
34Flexible neuro-fuzzysystemMamdani approach
35Definition Fuzzy implication
A fuzzy implication is a function I0,12?0,1
satisfying the following conditions (I1) if
a1a3 then I(a1,a2)I(a3,a2), for all
a1,a2,a3?0,1 (I2) if a2a3 then
I(a1,a2)I(a1,a3), for all a1,a2,a3?0,1 (I3) I
(0,a2)1, for all a2?0,1 (falsity implies
anything) (I4) I(a1,1)1, for all
a1?0,1 (anything implies tautology) (I5) I(1,0
)0 (booleanity)
36Fuzzy implications
37Flexible neuro-fuzzysystemLogical approach
38Flexible neuro-fuzzy system AND-type compromise
NFIS
39Flexible neuro-fuzzy system OR-type compromise
NFIS
40Flexible neuro-fuzzy system
L. Rutkowski and K. Cpalka Flexible Neuro-Fuzzy
Systems, IEEE Trans. Neural Networks, vol. 14,
pp. 554-574, May 2003
41Flexible neuro-fuzzy system Soft NFIS (1/2)
42Flexible neuro-fuzzy system Soft NFIS (2/2)
43Flexible neuro-fuzzy system NFIS realized by
parameterised families of triangular norms (1/2)
44Flexible neuro-fuzzy system NFIS realized by
parameterised families of triangular norms (2/2)
45Flexible neuro-fuzzy system NFIS realized by
triangular norms with weighted arguments (1/2)
46Flexible neuro-fuzzy system NFIS realized by
triangular norms with weighted arguments (2/2)
47Flexible neuro-fuzzy system Glass
Identification experimental results
48Flexibleneuro-fuzzysystem Glass
Identification weights representation
Weights representation in the Glass
Identification problem (dark areas correspond to
low values and vice versa)
49Flexible neuro-fuzzy system Glass
Identification comparison table
50Neuro-fuzzy relational system
51Neuro-fuzzy relational system with fuzzy matrix R
52Neuro-fuzzy connectionist system (basic
architecture)
53Rule generation
The neuro-fuzzy networks reflect fuzzy IF-THEN
rules.
The network architectures are created based on
the rules.
How to get the rules ?
54Basic questions
- What kind of the membership functions
- (Gaussian, triangular, trapezoidal, etc.) ?
- How to determine parameter values
- of the membership functions (centers, widths)
? -
55Many methods
There are many methods of rule generation.
However, most of the rules obtained by these
methods, when applied in neuro-fuzzy systems for
classification, result in some misclassifications.
56Perception-based approach
This method generates fuzzy IF-THEN rules, from a
data set, by use of fuzzy granulation.
The neuro-fuzzy systems, which utilize these
rules, perform without misclassifications.
57Multi-stage classification
The perception-based approach allows to generate
fuzzy rules and perform a multi-stage classificati
on without misclassifications.
This method will be illustrated on the IRIS
example.
58IRIS data set
150 data items that contain measurements of iris
flowers from three species of iris Setosa,
Versicolor, and Virginica 50 data items for
each of the iris species.
The data include information about four features
of the iris flowers sepal length, sepal width,
petal length, petal width.
59Ranges of the measurementsof iris flowers (in
centimeters)
60Ranges within the classes
Setosa Versicolor Virginica
Sepal length 4.3 5.8 4.9 7.0 4.9 7.9
Sepal width 2.3 4.4 2.0 3.4 2.2 3.8
Petal length 1.0 1.9 3.0 5.1 4.5 6.9
Petal width 0.1 0.6 1.0 1.8 1.4 2.5
61Granulated ranges of sepal length
62Granulated ranges of sepal width
63Granulated ranges of petal length
64Granulated ranges of petal width
65Linguistic labels for sepal length
66Linguistic labels for sepal width
67Linguistic labels for petal length
68Linguistic labels for petal width
69Rule 1
IF sepal is short or medium long and medium wide
or wide or very wide and petal is very short and
very narrow THEN Setosa
IF x1 is and x2 is and x3 is and x4
is THEN Setosa
70Rule 2
IF sepal is medium long or long and very narrow
or narrow or medium wide and petal is medium long
or long and medium wide or wide THEN Versicolor
IF x1 is and x2 is and x3 is and x4
is THEN Versicolor
71Rule 3
IF sepal is medium long or long or very long and
narrow or medium wide or wide and petal is long
or very long and wide or very wide THEN
Virginica
IF x1 is and x2 is and x3 is and x4 is
THEN Virginica
72NF network for the iris classification
73Results of the 1st stage classification
50 data vectors correctly classified to Setosa 32
data vectors correctly classified to
Versicolor 42 data vectors correctly classified
to Virginica 26 data vectors I do not know
decision
Versicolor or Virginica These data vectors
participate in the 2nd stage of the
classification.
742nd stage classification
Two fuzzy IF-THEN rules are formulated, based on
the granulated ranges, obtained for the data
vectors with the I do not know decision in the
1st stage.
The NF network in the 2nd stage is reduced to the
components associated with the Versicolor and
Virginica classes.
75Results of the 2nd stage classification
12 data vectors correctly classified to
Versicolor 1 data vector correctly classified to
Virginica
13 data vectors I do not know decision
Versicolor or
Virginica These data vectors participate in the
3rd stage of the classification. Two new rules
are created.
76Results of the 3rd stage classification
4 data vectors correctly classified to
Versicolor 5 data vectors correctly classified to
Virginica
4 data vectors I do not know decision
Versicolor or
Virginica These data vectors participate in the
4th stage of the classification. Two new rules
are created.
77Results of the 4th stage classification
2 data vectors correctly classified to
Versicolor 2 data vectors correctly classified to
Virginica
All data vectors correctly classified after 4
stages of the classification.
No misclassifications !
78IRIS data P1, P2
79IRIS data P1, P3
80IRIS data P2, P4
81IRIS data P3, P4
82Diagnosis of a tumor of mucous membrane of uterus
- Attributes
- period of time after menopause
- BMI (Body Mass Index)
- LH (luteinizing hormone )
- FSH (follicle-stimulating hormone )
- PRL (prolactin )
- E1 (estron)
- E2 (estradiol)
- Aromatase
- estrogenic receptor
9 attributes
Data 52 records of positive diagnosis 13 records
of negative diagnosis
Diagnosis negative (class 0), positive (class 1)
83Ranges of the attribute values
0.5 - 34
20 - 46
0.5 120.3
1.36 155.4
2.4 128.1
156 - 542
0.04 1.48
2.28 11.85
0.72 3.85
84Ranges within the classes
Class 0 Class 1
0.5 - 20 0.5 - 34
20 - 46 20 - 45
1.2 53.9 0.5 120.3
1.63 88.2 1.36 155.4
3.4 128.1 2.4 76.6
170 - 412 156 - 542
0.04 0.27 0.05 1.48
2.28 10.51 3 11,85
0.72 1.05 0.91 3.85
85Rules for the medical diagnosis
86NF network for the medical diagnosis
87Results correct diagnosis
3 cases with the I do not know response after
the first stage of classification
62 correct diagnosis for all 65 input vectors.
(95.4 correct decisions, 4.6 I do not know )
The I do not know answers, which mean positive
or negative diagnosis, refer to the cases that
are difficult to be recognized, because they
belong to overlapping regions.
88Conclusions (perception-based classification)
The perception-based approach allows to generate
fuzzy IF-THEN rules in the same way as humans do,
and perform the multi-stage classification without
misclassifications.
89Final conclusions
Neuro-fuzzy systems are soft computing methods
utilizing artificial neural networks and fuzzy
systems. Various connectionist architectures of
neuro-fuzzy systems can be constructed. The
knowledge acquisition concerns fuzzy IF-THEN
rules, and is performed by a learning process.
The systems realize an inference (fuzzy
reasoning) based on these rules.