CPE/CSC 481: Knowledge-Based Systems

1
CPE/CSC 481 Knowledge-Based Systems

• Dr. Franz J. Kurfess
• Computer Science Department
• Cal Poly

2
Course Overview

• Introduction
• Knowledge Representation
• Semantic Nets, Frames, Logic
• Reasoning and Inference
• Predicate Logic, Inference Methods, Resolution
• Reasoning with Uncertainty
• Probability, Bayesian Decision Making
• Expert System Design
• ES Life Cycle

• CLIPS Overview
• Concepts, Notation, Usage
• Pattern Matching
• Variables, Functions, Expressions, Constraints
• Expert System Implementation
• Salience, Rete Algorithm
• Expert System Examples
• Conclusions and Outlook

3
Overview Reasoning and Uncertainty

• Motivation
• Objectives
• Sources of Uncertainty and Inexactness in
  Reasoning
• Incorrect and Incomplete Knowledge
• Ambiguities
• Belief and Disbelief
• Probability Theory
• Bayesian Networks

• Dempster-Shafer Theory
• Certainty Factors
• \Approximate Reasoning
• Fuzzy Logic
• Important Concepts and Terms
• Chapter Summary

4
Logistics

• Introductions
• Course Materials
• textbooks (see below)
• lecture notes
• PowerPoint Slides will be available on my Web
  page
• handouts
• Web page
• http//www.csc.calpoly.edu/fkurfess
• Term Project
• Lab and Homework Assignments
• Exams
• Grading

5
Bridge-In
6
Pre-Test
7
Motivation
8
Objectives
9
Evaluation Criteria
10
Introductions

• reasoning under uncertainty and with inexact
  knowledge
• heuristics
• ways to mimic heuristic knowledge processing
• methods used by experts
• empirical associations
• experiential reasoning
• based on limited observations
• probabilities
• objective (frequency counting)
• subjective (human experience )
• reproducibility
• will observations deliver the same results when
  repeated

11
Dealing with Uncertainty

• expressiveness
• can concepts used by humans be represented
  adequately?
• can the confidence of experts in their decisions
  be expressed?
• comprehensibility
• representation of uncertainty
• utilization in reasoning methods
• correctness
• probabilities
• relevance ranking
• long inference chains
• computational complexity
• feasibility of calculations for practical purposes

12
Sources of Uncertainty

• data
• missing data, unreliable, ambiguous, imprecise
  representation, inconsistent, subjective, derived
  from defaults,
• expert knowledge
• inconsistency between different experts
• plausibility
• best guess of experts
• quality
• causal knowledge
• deep understanding
• statistical associations
• observations
• scope
• only current domain?

13
Sources of Uncertainty (cont.)

• knowledge representation
• restricted model of the real system
• limited expressiveness of the representation
  mechanism
• inference process
• deductive
• the derived result is formally correct, but wrong
  in the real system
• inductive
• new conclusions are not well-founded
• unsound reasoning methods

14
Uncertainty in Individual Rules

• individual rules
• errors
• domain errors
• representation errors
• inappropriate application of the rules
• likelihood of evidence
• for each premise
• for the conclusion
• combination of evidence from multiple premises

15
Uncertainty and Multiple Rules

• conflict resolution
• if multiple rules are applicable, which one is
  selected
• explicit priorities, provided by domain experts
• implicit priorities derived from rule properties
• specificity of patterns, ordering of patterns
  creation time of rules, most recent usage,
• compatibility
• contradictions between rules
• subsumption
• one rule is a more general version of another one
• redundancy
• missing rules
• data fusion
• integration of data from multiple sources

16
Basics of Probability Theory

• mathematical approach for processing uncertain
  information
• sample space setX x1, x2, , xn
• collection of all possible events
• can be discrete or continuous
• probability number P(xi)likelihood of an event
  xi to occur
• non-negative value in 0,1
• total probability of the sample space is 1
• for mutually exclusive events, the probability
  for at least one of them is the sum of their
  individual probabilities
• experimental probability
• based on the frequency of events
• subjective probability
• based on expert assessment

17
Compound Probabilities

• describes independent events
• do not affect each other in any way
• joint probability of two independent events A and
  BP(A ? B) n(A ? B) / n(s) P(A) P (B)
• where n(S) is the number of elements in S
• union probability of two independent events A and
  BP(A ? B) P(A) P(B) - P(A ? B) P(A) P(B)
  - P(A) P (B)
• where n(S) is the number of elements in S

18
Conditional Probabilities

• describes dependent events
• affect each other in some way
• conditional probability of event a given that
  event B has already occurredP(AB) P(A ? B) /
  P(B)

19
Advantages and Problems of Probabilities

• advantages
• formal foundation
• reflection of reality (a posteriori)
• problems
• may be inappropriate
• the future is not always similar to the past
• inexact or incorrect
• especially for subjective probabilities
• knowledge may be represented implicitly

20
Bayesian Approaches

• derive the probability of a cause given a symptom
• has gained importance recently due to advances in
  efficiency
• more computational power available
• better methods
• especially useful in diagnostic systems
• medicine, computer help systems
• inverse or a posteriori probability
• inverse to conditional probability of an earlier
  event given that a later one occurred

21
Bayes Rule for Single Event

• single hypothesis H, single event EP(HE)
  (P(EH) P(H)) / P(E)or
• P(HE) (P(EH) P(H) / (P(EH)
  P(H) P(E?H) P(?H) )

22
Bayes Rule for Multiple Events

• multiple hypotheses Hi, multiple events E1, ,
  Ei, , EnP(HiE1, E2, , En) (P(E1, E2, ,
  EnHi) P(Hi)) / P(E1, E2, , En)or
• P(HiE1, E2, , En) (P(E1Hi) P(E2Hi)
  P(EnHi) P(Hi)) / ?k P(E1Hk) P(E2Hk)
  P(EnHk) P(Hk)with independent pieces of
  evidence Ei

23
Advantages and Problems of Bayesian Reasoning

• advantages
• sound theoretical foundation
• well-defined semantics for decision making
• problems
• requires large amounts of probability data
• sufficient sample sizes
• subjective evidence may not be reliable
• independence of evidences assumption often not
  valid
• relationship between hypothesis and evidence is
  reduced to a number
• explanations for the user difficult
• high computational overhead

24
Dempster-Shafer Theory

• mathematical theory of evidence
• notations
• frame of discernment FD
• power set of the set of possible conclusions
• mass probability function m
• assigns a value from 0,1 to every item in the
  frame of discernment
• mass probability m(A)
• portion of the total mass probability that is
  assigned to an element A of FD

25
Belief and Certainty

• belief Bel(A) in a subset A
• sum of the mass probabilities of all the proper
  subsets of A
• likelihood that one of its members is the
  conclusion
• plausibility Pl(A)
• maximum belief of A
• certainty Cer(A)
• interval Bel(A), Pl(A)
• expresses the range of belief

26
Combination of Mass Probabilities

• m1 ? m2 (C) ? X ? YC m1(X) m2(Y) / 1- ?X ?
  YC m1(X) m2(Y) where X, Y are hypothesis
  subsets and C is their intersection

27
Advantages and Problems of Dempster-Shafer

• advantages
• clear, rigorous foundation
• ability oto express confidence through intervals
• certainty about certainty
• problems
• non-intuitive determination of mass probability
• very high computational overhead
• may produce counterintuitive results due to
  normalization
• usability somewhat unclear

28
Certainty Factors

• shares some foundations with Dempster-Shafer
  theory, but more practical
• denotes the belief in a hypothesis H given that
  some pieces of evidence are observed
• no statements about the belief is no evidence is
  present
• in contrast to Bayes method

29
Belief and Disbelief

• measure of belief
• degree to which hypothesis H is supported by
  evidence E
• MB(H,E) 1 IF P(H) 1 (P(HE) -
  P(H)) / (1- P(H)) otherwise
• measure of disbelief
• degree to which doubt in hypothesis H is
  supported by evidence E
• MB(H,E) 1 IF P(H) 0 (P(H) -
  P(HE)) / P(H)) otherwise

30
Certainty Factor

• certainty factor CF
• ranges between -1 (denial of the hypothesis H)
  and 1 (confirmation of H)
• CF (MB - MD) / (1 - min (MD, MB))
• combining antecedent evidence
• use of premises with less than absolute
  confidence
• E1 ? E2 min(CF(H, E1), CF(H, E2))
• E1 ? E2 max(CF(H, E1), CF(H, E2))
• ?E ? CF(H, E)

31
Combining Certainty Factors

• certainty factors that support the same
  conclusion
• several rules can lead to the same conclusion
• applied incrementally as new evidence becomes
  available
• Cfrev(CFold, CFnew)
• CFold CFnew(1 - CFold) if both gt 0
• CFold CFnew(1 CFold) if both lt 0
• CFold CFnew / (1 - min(CFold, CFnew)) if
  one lt 0

32
Advantages and Problems of Certainty Factors

• Advantages
• simple implementation
• reasonable modeling of human experts belief
• expression of belief and disbelief
• successful applications for certain problem
  classes
• evidence relatively easy to gather
• no statistical base required
• Problems
• partially ad hoc approach
• theoretical foundation through Dempster-Shafer
  theory was developed later
• combination of non-independent evidence
  unsatisfactory
• new knowledge may require changes in the
  certainty factors of existing knowledge
• certainty factors can become the opposite of
  conditional probabilities for certain cases
• not suitable for long inference chains

33
Fuzzy Logic

• approach to a formal treatment of uncertainty
• relies on quantifying and reasoning through
  natural language
• uses linguistic variables to describe concepts
  with vague values
• tall, large, small, heavy, ...

34
Get Fuzzy
35
Fuzzy Set

• categorization of elements xi into a set S
• described through a membership function m(s)
• associates each element xi with a degree of
  membership in S
• possibility measure Possx?S
• degree to which an individual element x is a
  potential member in the fuzzy set S
• possibility refers to allowed values
• probability expresses expected occurrences of
  events
• combination of multiple premises
• Poss(A ? B) min(Poss(A),Poss(B))
• Poss(A ? B) max(Poss(A),Poss(B))

36
Fuzzy Set Example
membership
tall
short
medium
1
0.5
height (cm)
0
0
50
100
150
200
250
37
Fuzzy vs. Crisp Set
membership
tall
short
medium
1
0.5
height (cm)
0
0
50
100
150
200
250
38
Fuzzy Inference Methods

• how to combine evidence across rules
• Poss(BA) min(1, (1 - Poss(A) Poss(B)))
• implication according to Max-Min inference
• also Max-Product inference and other rules
• formal foundation through Lukasiewicz logic
• extension of binary logic to infinite-valued logic

39
Example Fuzzy Reasoning
40
Advantages and Problems of Fuzzy Logic

• advantages
• general theory of uncertainty
• wide applicability, many practical applications
• natural use of vague and imprecise concepts
• helpful for commonsense reasoning, explanation
• problems
• membership functions can be difficult to find
• multiple ways for combining evidence
• problems with long inference chains

41
Post-Test
42
Evaluation

• Criteria

43
Use of References

• Giarratano Riley 1998
• Russell Norvig 1995
• Jackson 1999
• Durkin 1994

Giarratano Riley 1998
44
Important Concepts and Terms

• natural language processing
• neural network
• predicate logic
• propositional logic
• rational agent
• rationality
• Turing test

• agent
• automated reasoning
• belief network
• cognitive science
• computer science
• hidden Markov model
• intelligence
• knowledge representation
• linguistics
• Lisp
• logic
• machine learning
• microworlds

45
Summary Chapter-Topic
46
(No Transcript)