Resolving%20Underconstrained%20and%20Overconstrained%20Systems%20of%20Conjunctive%20Constraints%20for%20Service%20Requests

1
Resolving Underconstrained and Overconstrained
Systems of Conjunctive Constraints for Service
Requests

• Muhammed J. Al-Muhammed
• David W. Embley
• Brigham Young University

Sponsored in part by NSF (0083127 and 0414644)
2
The Problem Underconstrained

• I want a dodge a 2000 or newer. The Mileage
  should be less
• than 80,000 and the price should not exceed
  15,000

Solution Make Model Price Year Mileage
S1 Dodge Stratus 9,451.00 2004 35,808
S2 Dodge Stratus 14,995.00 2005 1,694
S3 Dodge Stratus 14,999.00 2005 27,543
S4 Dodge Stratus 2,555.00 1997 115,424
S5 Dodge Stratus 6,900.00 2001 70,000


S168 Dodge Stratus 9,975.00 2003 34,060
www.cars.com, November 2005
3
The Problem Overconstrained

• I want a dodge a 2000 or newer. The Mileage
  should be less
• than 80,000 and the price should not exceed
  4,000.

Sorry No car matches your criteria.
www.cars.com, November 2005
4
Key Observations

• Some (near) solutions are better than others
• People specify constraints on some concepts in a
  domain more often than other concepts

5
Fundamental Concepts reward, penalty, and
expectation

• A reward is a positive or zero real number given
  to a solution for satisfying a constraint
• A penalty is a negative real number given to a
  near solution for violating a constraint
• An expectation for a concept is the probability
  that people will specify constraint for the
  concept

6
Fundamental ConceptsPareto Optimality

• Based on dominance relations
• The reward for S1 is as high as the reward for S2
• For at least one reward S1 has a higher reward
• Dominated solutions are not Pareto optimal

7
Too Many SolutionsReward-Based Ordering

• Calculate rewards and combine them
• Order solutions, highest combined reward first
• Select the top-m Pareto optimal solutions
• Discard non-Pareto optimal solutions from the
  reward ordering
• Return the top-m for consideration

8
Example

• I want a dodge a 2000 or newer. The mileage
  should be less
• than 80,000 and the price should not be more than
  15,000.
• Solution Make Model Price Year Mileage
• --------------------------------------------------
  -------------
• S1 Dodge Stratus 13,999 2005
  15,775
• S2 Dodge Stratus 11,998 2004 23,404
• S3 Dodge Stratus 14,200 2005 16,008
• S4 Dodge Stratus 14,557 2005 16,954
  
• S5 Dodge Stratus 10,590 2003 38,608

9
Too Many Solutions Expectation-Based Constraint
Elicitation

• Associate expectations with domain concepts
• Order the concepts in a domain based on their
  expectations
• Most expected first
• Example Make gt Price gt Model gt
• Elicit additional constraints over unconstrained
  concepts
• Most expected first
• If no preferred make provided, ask for Make if
  no price, ask for Price

10
No Solution Penalty-Based Ordering

• Calculate penalties and combine them
• Order close solutions, lowest combined penalty
  first
• Select the top-m Pareto optimal near solutions
• Discard dominated near solutions from penalty
  ordering
• Return the top-m near solutions for consideration

11
No SolutionExpectation-Based Constraint
Relaxation

• Select the near solutions violating fewer
  constraints than a threshold
• Compute the relaxation cost rsi ?kekCk(si).
• Suggest constraints of the near solution with the
  least rsi for relaxation

penalty
expectation
12
Example
Can this constraint 100 PM or after be relaxed
to 1240 PM Can this constraint the 20th be
relaxed to the 19th
I want to see a dermatologist on the 20th, 100
PM or after. The dermatologist should be within 5
miles from my home and must accept my IHC
insurance.
Near Solution Insurance Distance Time Date
S1 IHC 16 100 PM the 19th
s2 IHC 18 110 PM the 19th
s3 IHC 4 1240 PM the 19th
s4 IHC 6 1250 PM the 19th
s5 IHC 20 300 PM the 19th
1240 PM
the 19th
Near Solution InsuranceIHC Expectation 0.4 Distance?5 Expectation 0.3 Time?100 PM Expectation 0.8 Datethe 20th Expectation 0.9 rsi
s1 0.000 ? 0.076 0.167 ? 0.250 ? 0.248
S2 0.000 ? 0.090 0.160 ? 0.250 ? 0.252
s3 0.000 0.007 ? 0.014 ? 0.250 ? 0.236
s4 0.000 ? 0.007 ? 0.007 ? 0.250 ? 0.233
s5 0.000 ? 0.102 0.083 ? 0.250 ? 0.256
13
Performance Analysis

• Tested on appointment and car purchase domains
• 16 human subjects
• The best-5 near solutions from 19 appointments
• The best-5 solutions from 32 cars
• Compare human selection with system selection
  with respect to the best-5

14
Performance Analysis
Human selection versus system selection
appointment
15
Performance Analysis
Human selection versus system selection car
purchase
16
Performance Analysis

• Inter-observer agreement test
• Results
• kappa 0.74 (appointment)
• kappa 0.67 (car purchase)
• Substantial agreement based on kappa values