Title: BISCDSS
1Natural Language Computing and Reasoning
2Symptoms X Diagnosis
Symptoms
Diagnosis
Test Attribute Set/ Question
3Symptoms X Diagnosis
- The use of Linguistic variables
- Simple relations between variables by fuzzy
conditional statement - Complex relations by fuzzy Algorithms
- IF Symptom (A1 ) is a11 and Symptom (A2 ) is
a12 and Symptom (An ) is a1n Then Diagnosis
(F1) is a1 - IF Symptom (A1 ) is a21 and Symptom (A2 ) is
a22 and Symptom (An ) is a2n Then Diagnosis
(F1) is a1 - Question
- IF Symptom (A1 ) is a1 and Symptom (A2 ) is a2
and Symptom (An ) is an Then Diagnosis (F1) is
b?
4Other Applications
Description
5(No Transcript)
6(No Transcript)
7(No Transcript)
8University admissions
Different admission rates and Varying criteria
depending on the University strategy e.g.
UC-Berkeley and Stanford University
9Outline
- BISC Decision Support System
- Neuro-Fuzzy-Evolutionary Computing NeF-ECom
- Multi-Criteria Decision Analysis with Uncertain
and Incomplete Information - Application Areas
- ASIS
10BISC- Decision Support System
BISC-DSS
Human Knowledge
HM
First Principle Models
Data Knowledge
11- OBJECTIVES
- Develop soft-computing-based techniques for
decision analysis - Tools to assist decision-makers in assessing
the consequences of decision made in an
environment of imprecision, uncertainty, and
partial truth and providing a systematic risk
analysis - Tools to assist decision-makers answer What if
Questions, examine numerous alternatives very
quickly and find the value of the inputs to
achieve a desired level of output - Tools to be used with human interaction and
feedback to achieve a capability to learn and
adapt through time
12DECISION ENVIRONMENT
- Information (Can be uncertain)
- Granular (Scale and Precision)
- Query (Can be imprecise)
- Measure (Similarity)
- Aggregation (Can be fuzzy)
- Ranking (Provide Alternatives)
- Optimization (Multi-Objective Multi-Criteria)
13BISC DSS Components and Structure
Model and Data Visualization
- Model Management
- Query
- Aggregation
- Ranking
- Fitness Evaluation
Evolutionary Kernel Genetic Algorithm, Genetic
Programming, and DNA
- Selection
- Cross Over
- Mutation
Input From Decision Makers
Experts Knowledge
Model Representation Including Linguistic
Formulation
Data Management
- Functional Requirements
- Constraints
- Goals and Objectives
- Linguistic Variables Requirement
14Query (Request) Q
- find if such query exists ? degree of match ?
rank ? decision ( i.e. resource allocation) - compare queries ? rank ? decision (task
allocation) - Use Fuzzy Min-Max with degree of preferences
15Objective function Cost Function/ Fitness
Function
This may involve multi-objective, multi-criteria
optimization with conflict and fuzzy variables.
Therefore, use fuzzy-GA to solve the objective
function.
16University admissions
Different admission rates and Varying criteria
depending on the University strategy e.g.
UC-Berkeley and Stanford University
17Actual Model Given Student Rate of Success
Predicted Model Using Fuzzy-GA
Initial GA Population of Models
18The BISC Decision Support System Conventional GA
Multi-Objective Multi-Criteria Optimization
Max
Preferences
Mean
Actual Prediction 0.5010 0.7961
0.5010 0.5176 0.5210 0.5686 0.4800
0.4588 0.5010 0.7176 0.5010
0.8588 0.5010 0.9490 0.5010 0.6980
0.5010 0.5922 0.5010 0.9373
0.5000 0.7412 0.5210 0.7608 0.5210
0.6353 0.5630 0.6784 0.5210
0.7490 0.5420 0.8667 0.5630 0.7843
Fitness
Min.
Std Dev.
Generation
19The BISC Decision Support System Interactive-GA
Multi-Objective Multi-Criteria Optimization
Max
Preferences
Preferences
Actual Predicted 0.5010 0.4609
0.5010 0.4907 0.5210 0.5712 0.4800
0.4709 0.5010 0.5381 0.5010
0.5106 0.5010 0.5513 0.5010 0.5469
0.5010 0.5161 0.5010 0.5061
0.5000 0.5106 0.5210 0.5701 0.5210
0.5425 0.5630 0.5469 0.5210
0.5370 0.5420 0.4444 0.5630 0.5017
Mean
Fitness
Min.
Std Dev.
Generation
20 Neuro-Fuzzy-Evolutionary ComputingMulti-Criteria
Decision Analysis with Uncertain and Incomplete
Information
BISC-DSS Software
NeF-ECom
21BISC DSS Software Architecture
- Aggregation operators
- Similarity measures
- Norm-Pairs
- Fuzzy sets
Application Template
Fuzzy Search Engine (FSE)
User Interface
Evolutionary Computing Kernel
DB
22Basic concepts
Fuzzy sets/ Membership Functions (MFs)
Low Medium High
Triangular
Gaussian
Low diversity diverse High
diversity
Trapezoidal
23Basic concepts
Fuzzy similarity measures
X and Y are fuzzy measures defined over the same
fuzzy sets with MFs µ1, µ2, , µm
Norm-Pair operators ? et ? (norm-conorm)
24Basic concepts
Norm-Pairs
Fuzzy AND ?
Fuzzy OR ?
x and y are MF values in 0,1.
25Basic concepts
Aggregation Operators
26Basic concepts
Weighted Aggregation Operators
27Advanced Multi-Aggregator Model
Basic concepts
- Parameters
- aggregators
- weights
- tree structure.
28Compactification Algorithm InterpretationA
Simple Algorithm for Qualitative AnalysisRule
Extraction and Building Decision TreeNikravesh
and Zadeh (2005)(Zadeh, 1976)
29Symptoms X Diagnosis
- The use of Linguistic variables
- Simple relations between variables by fuzzy
conditional statement - Complex relations by fuzzy Algorithms
- IF Symptom (A1 ) is a11 and Symptom (A2 ) is
a12 and Symptom (An ) is a1n Then Diagnosis
(F1) is a1 - IF Symptom (A1 ) is a21 and Symptom (A2 ) is
a22 and Symptom (An ) is a2n Then Diagnosis
(F1) is a1 - Question
- IF Symptom (A1 ) is a1 and Symptom (A2 ) is a2
and Symptom (An ) is an Then Diagnosis (F1) is
b?
30Symptoms X Diagnosis
Test Attribute Set
31Table 1 (intermediate results)
Group 1(initial)
Pass (1)
Pass (2)
Pass (3)
32MAXIMALLY COMPACT REPRESENTATION
33(No Transcript)
34Chromosome Representation
Fuzzy Label, Set Value, Scalar Series Input
- Composed of primitive statistical, fuzzy set,
aggregator, similarity, arithmetic, and signal
processing operators. - Each gene (or algorithm) is represented as a
tree, accepts both scalar and series input, and
outputs scalar features. - The chromosome produces a feature vector set.
Scalar Fuzzy Label Features
35Multi Aggregator Tree
Advanced Multi-Aggregator Model
- Parameters
- aggregators
- weights
- tree structure.
36H-DT3 Nikravesh Zadeh (2003)
Attrb.
BISC-DSS Nikravesh (2000)
Attrib./Feature Selection Nikravesh (2005)
C-Rules Zadeh (1976) Nikravesh (2003)
Transf.
RCR-PFRL Berenji (2003) Nikravesh (2003)
Compactification Zadeh (1976)
Signal
C-DT3 Zadeh (1976) Nikravesh (2003)
MA-DT3 Nikravesh (2003)
Cases
SVM, NN (RBF MLP), NF Nikravesh (2003)
37BISC-DSS Software
38BISC-DSS Software
39EC Genetic Algorithms
- Requirements
- - Individual problem representation
- - Fitness function for evaluation
- - Termination criterion
- Principle
- Create randomly an initial population of
individuals - Evolve the population
- evaluate and select individuals
- use them in genetic operators (crossover,
mutation) - generate new generation
- - Stop if termination criterion satisfied
40EC Genetic Algorithms
Genetic Operators
41EC Genetic Programming
- Individual Computer program
- Most common representation tree encoding
(nodes functions, leaves terminals) - Fitness function returned value by the root
node
Chosen node
Mutation
new individual
selected individual
resulting individual
42EC Genetic Programming
Crossover
43(No Transcript)
44BISC-DSS Interaction and Optimization
- Comparison, Aggregation, Scoring
- MODEL based on
- Aggregation operators,
- Similarity measures
- Norm-Pairs
- Fuzzy sets
DB
Fuzzy Search Engine (FSE)
QUERY
User Interface
ANSWERS
Evolutionary Computing Kernel
User preferences (re-ranking, selection)
OPTIMIZATION
45Multi-Criteria Decision Model (1)
- Multi-Attribute Query K attributes A1, A2,,AK
Scores
Database
Similarity calculation
Query
Query Answering Ranking based Selection
based (criteria number top answers) (criteria
threshold)
46Multi-Criteria Decision Model (2)
Query
Data
Fuzzification
Fuzzy sets
For each attribute
Norm-pairs ?,?
Fuzzy similarity calculation
Fuzzy similarity measures
Aggregation model
aggregation
Scoring
Ranking or Selecting Answers
47Multi-Criteria Decision Model (3)
- Data Xi (xi1, xi2, , xiK), Query Q
(y1, y2, , yk) - K attributes A1, A2,,AK
- For each attribute Aj
- rj fuzzy sets µ1(Aj,.), µ2(Aj,.),,µrj(Aj,.)
- sj similarity(xij, yj), j 1, 2, , K
- Score SIM(Q,Xi) Aggregation(s1, s2, , sk)
48First Order Aggregation Model (1)
- Norm-pair Min/Max
- Fuzzy similarity measure Jaccard
- Aggregation operator Weighted Mean
49First Order Aggregation Model (2)
- Aggregation model simple weighted aggregation
operator - user preferences attribute weighting
- (Degree of importance of each attribute)
- Aggregation model parameters weighting vector
Optimization process find the optimal weights
Using GA.
50First Order Aggregation Model (3)
- Model parameters learning using GA
- GA-based learning module
- - Individuals weight vectors
- - Genetic operators crossover, Mutation
- Fitness function
- Termination criterion
Specific fitness function
Problem specification
Optimal weights
51Advanced Multi-Aggregator Model (1)
- parameters
- similarity measures
- norm-pairs
- aggregation operators
- weights
- aggregation model structure
Representation of user/expert preferences
52Advanced Multi-Aggregator Model (2)
- Parameters
- aggregators
- weights
- tree structure.
53Advanced Multi-Aggregator Model (3)
- Model parameters learning using GP
Aggregators set, Attributes set, Model constraints
Specific DNA encoding
GP-based learning module
Problem specification
Specific fitness function
Optimal multi-aggregation model
54Basic concepts
Neural Network Aggregator
55Basic concepts
NeuroFuzzy Aggregator
56Committee Machine
Committee Machines
57Model evaluation
rejected
accepted
D
Similarity calculation
Score Ranking
- Fitness function combining
- accuracy rates to maximize
- distance D to maximize
- model structure size to minimize
58Other Applications
Description
59Issues with Missing Data and Uncertainties
UCI Repository
Nikravesh, 2004
60Automated Sensory Inspection System
BISC-DSS-ASIS Software
ASIS
61Applications
62Chromosome Representation
Genes
Chromosome
It is not part of current ASIS-BISC-DSS
y
Feature Vector, Automated
x1
xi
xn
Feature Vector, Expert Knowledge
x1E
xiE
xnE
63Primitive Operators
- Minimum
- Maximum
- Ratio of Means
- Add, Subtract, Multiply, Divide
- Subseries
- Subsampling
- Derivative Approximation
- Convolution Filtering
- Mean
- Standard Deviation
- Variance
- Skewness
- Kurtosis
- Integral
- Sum
- Linear Combination
It is not part of current ASIS-BISC-DSS
64Signal Attribute/ Character Analysis
- 1. Conventional Hilbert Attributes
- 2. FFT-based frequency-domain attributes/TFA
- 3. Empirical Correlation of Attributes
- 4. Clustering of Attributes
- 6. Multi-trace correlation/coherence attributes
- 7. Wavelet Character Analysis
- 7. Use Neural Networks for Integration
65Conventional Attributes
- Amplitude Envelope
- Ampl. Weighted Cosine Phase
- Ampl. Weighted Freq.
- Amplitude Weighted Phase
- Apparent Polarity
- Average Frequency
- Cosine Instantaneous Phase
- Derivative Inst. Amplitude
- Derivative
- Instantaneous Frequency
- Dominant Frequency
- Instantaneous Phase
- Integrated Abs. Amplitude
- Trace Integration
- Raw Acoustic Wave Trace
- 2nd Derivative Inst. Ampl.
- 2nd Derivative
66Chromosome Representation
Fuzzy Label, Set Value, Scalar Series Input
- Composed of primitive statistical, fuzzy set,
aggregator, similarity, arithmetic, and signal
processing operators. - Each gene (or algorithm) is represented as a
tree, accepts both scalar and series input, and
outputs scalar features. - The chromosome produces a feature vector set.
Scalar Fuzzy Label Features
67Front/Back-end Architecture (BISC-Zeus)
Time Series
Classifier Generator
Genetic Algorithm Feature Selection Based on ASIS
Feature Extraction
Front-end Stochastic Search
Classification
Support Vector Machine BISC-DSS
Back-end Classification
Time Series Classifier
68Classifier Architecture (BISC-Zeus)
Classifier Generator
Time Series
TIME SERIES CLASSIFIER
FEATUREEXTRACTOR
FEATURE SET
MODEL
Prediction Result
BISC-DSS
Feature Selection Based On ASIS
69Fitness In-sample Rate (BISC-Zeus)
N Features
K Features
Chromosome
Training Set
Processed Set
A
B
Feature Extractor Feature Selection Based On
ASIS
Model
C
E
- Steps
- Run Feature Extractor
- Produce Training Set
- Train SVM/BISC-DSS
- Produce Model
- Run Classifier
- Produce Result Set
- Calculate Score
Classifier
BISC-DSS
F
D
Score
Result Set
70Fitness N-Fold Cross Valid (BISC-Zeus)
N Features
K Features
Chromosome
Training Set
Processed Set
A
B
Feature Extractor Feature Selection Based On
ASIS
E
Model
D
- Steps
- Run Feature Extractor
- Produce Training Set
- Produce Testing Partitions
- Train on Complement
C
Testing Partitions
Classifier
F
E. Produce Model F. Predict Labels of
Test Set G. Score if finished, otherwise,
goto step D.
BISC-DSS
Result Set
Finished? No
Yes
Score
71(No Transcript)
72(No Transcript)
73BISC-DSS Clustering-Based ANSIS
Sets of Ranked Attributes AS1, AS2, AS3, AS4
Input All the Attributes
Final Set of Attributes selected using
Unsupervised Clustering
AS1
Att1, Att2, Att3, Att4,
Unsupervised Clustering K-Mean, PCA, SOM,
AS2
Att11, Att12, Att13, Att14
Modeling Techniques Principal Component Analysis
(PCA) Singular Value decomposition (SVD) Mahalanob
is Distance (MD) One Class Suppoer vector
Machine (1CSVM)
AS3
Att21, Att22, Att23, Att24, ..
Selected Attributes
AS4
Att31, Att32, Att33, Att34,
Cluster, Classification, Anomaly, Rules, Decision
Tree,
Final Set of Attributes selected using Modeling
Technique
Marked Data as Clusters, Anomaly,
In Progress
BISC-DSS Software
74(No Transcript)
75(No Transcript)
76(No Transcript)
77(No Transcript)
78One-Class ClassificationAnalysis of new data
79One-Class SVM - Experiments
80One-Class SVM Experiment 1
81One-Class SVM Experiment 2