A Theory of Theory Formation

1
A Theory of Theory Formation

• Simon Colton
• Universities of Edinburgh and York

2
Overview

• What is a theory?
• Four components of the theory of ATF
• Techniques inside the components
• Cycles of theory formation
• Case Studies
• Applications (briefly)
• Of both the theories and the process

3
What is a Theory?

• Theories are (minimally) a collection of
• Objects of interest
• Concepts about the objects
• Hypotheses relating the concepts
• Explanations which prove the hypotheses
• Finite Group Theory
• All cyclic groups are Abelian
• Inorganic Chemistry
• Acid Base ? Salt Water

4
So, We Require

Object Generator
Concept Generator
Hypothesis Generator
Explanation Generator
5
In Principle, These Could Be

Database, CAS, CSP, Model Generator
Machine Learning Program
ATP System, Pathway Finder, Visualisation
Data Mining Program
6
In Practice, Current Implementation

Database, Model Generator, (CAS, CSP nearly)
The HR Program
The HR Program
ATP Systems
7
Object Generation and Explanation Generation

• Object Generation
• Machine learning reading a file, database
• In Mathematics
• CSP (e.g., FINDER, Solver), CAS (e.g., Maple)
• Davis Putnam method (e.g., MACE)
• Resolution Theorem Proving (e.g., Otter)
• HR must be able to communicate
• Read models and concepts from MACEs output
• Read proofs and statistics from Otters output

8
Concept Generation

• Build a new concept from old ones
• 10 general production rules (demonstrated later)
• Produce both a definition and examples
• Throw away concepts using definitions
• Tidy definitions up
• Repetitions, function conflict, negation conflict
• Decide which concepts to use for construction
• Plethora of measures of interestingness
• Weighted sum of measures

9
Concept GenerationLakatos-inspired Techniques

• Monster Barring
• Remove an object of interest from theory
• Counterexample Barring
• Except a finite subset of objects from a theorem
• E.g., all primes except 2 are odd
• Concept Barring
• Except a concept from a theorem
• All integers other than squares have an
• even number of divisors
• Credit to Alison Pease

10
Hypothesis GenerationFinding Empirical
Relationships

• Equivalence conjectures
• One concept has the same examples as another
• Subsumption conjectures
• All examples of one concept are examples of other
• Non-existence conjectures
• A concept has no examples
• Assessment of conjectures
• Used to assess the concepts mentioned in them

11
Hypothesis GenerationExtracting Prime Implicates

• Extract implications, then prime implicates
• Equivalence conjectures are split
• A B C ? D E F becomes
• A B C ? D, A B C ? E, etc.
• Non-existence conjectures are split
• (A B C) becomes A B ? C, etc.
• Extract Prime implicates
• A B C ? D, try A ? D, then B ? D,
• C ? D, then A B ? D, etc.

12
Hypothesis Generation Imperfect Conjectures

• User sets a percentage minimum, say 80
• Near-subsumption conjectures
• E.g., primes ? odd (99 true)
• Also returns the counterexamples here, 2
• Near-equivalence conjectures
• Prime ? odd (70 true)
• Applicability conjectures
• A concept has a (small) finite number of examples
• E.g., even prime numbers 2 is only example

13
Cycles of Theory Formation

• How the individual techniques are employed
• Concept driven conjecture making
• Finding conjectures to help understand concepts
• Exploration techniques
• Conjecture driven concept formation
• Inventing concepts to fix faulty conjectures
• Imperfect conjectures, Lakatos techniques

14
Concept Driven Cycle (cut-down)
Invent Concept
Reject
Equivalence
Non Existence
New Concept
Subsumptions
Implications
15
Concept Driven Cycle Continued
Implications
Counterexample
Proof
Prime Implicates
Counterexample
Proof
16
Conjecture Driven Cycle
Invent Concept
Reject
Near Equivalence
Applicability
Near Subsumption
Monster Barring
Concept Barring
Concept Barring
Counterex Barring
New Concept
Counterex Barring
New/Old Concept
New/Old Concept
Equivalence
Implications
17
Case Study Groups

Given Group theory axioms
18
Case Study Groups

Davis Putnam Method
MACE model generator finds a model of size 1
19
Case Study Groups

HR Reads MACEs Output
Extracts concepts Element, Multiplication,
Identity, Inverse
20
Case Study Groups

Match Production Rule
Invents the concept idempotent elements (aaa)
21
Case Study Groups

Equivalence Finding
Makes Conjecture aaa ? a is the identity
element
22
Case Study Groups

Resolution Theorem Proving
Otter proves this in less than a second
23
Case Study Groups

Extracts Prime Implicates
aa a ? aidentity, aidentity ? aaa End of
cycle
24
Case Study Groups

Compose Production Rule
Later Invents the concept of triples of
elements (a,b,c) for which abc bac
25
Case Study Groups

Exists Production Rule
Invents concept of pairs (a,b) for which
there exists an element c such that abc bac
26
Case Study Groups

Forall Production Rule
Invents the concept of groups for which all
pairs of elements have such a c Abelian groups
27
Case Study Groups

Equivalence Finding
Makes the Conjecture G is a group if and only
if it is Abelian
28
Case Study Groups

Sorry
Otter fails to prove this conjecture
29
Case Study Groups

Davis Putnam Method
MACE finds a counterexample Dihedral Group of
size 6 (non-Abelian)
30
Case Study Groups

Assessment of Concepts
Concept of Abelian groups allowed into
theory Theory recalculated in light of new object
of interest
31
Case Study Goldbach

Given Integers 1 to 100, Concepts Divisors,
Addition
32
Case Study Goldbach

Split Production Rule
Invents Even Numbers (divisible by 2)
33
Case Study Goldbach

Size Production Rule
Invents Number of Divisors (tau function)
34
Case Study Goldbach

Split Production Rule
Invents Prime numbers (2 divisors)
35
Case Study Goldbach

Compose Production Rule
Half an hour later Invents Goldbach numbers
(sum of 2 primes)
36
Case Study Goldbach

Near Equivalence Finding
Conjectures Even numbers are Goldbach
numbers (with one exception, the number 2)
37
Case Study Goldbach

Counterexample Barring (Split)
Forces Concept of being the number 2
38
Case Study Goldbach

Counterexample Barring (Negate)
Forces concept Even numbers except 2
39
Case Study Goldbach

Subsumption Finding
Conjectures Even numbers except 2 are
Goldbach Numbers (Goldbachs Conjecture)
40
Case Study Goldbach

Absolutely No Chance
Passes the conjecture to an inductive theorem
prover?
41
Applications of Theories

• Puzzle generation
• Which is the odd one out 4, 9, 16, 24
• Which is the odd one out 2, 9, 8, 3
• Problem generation
• TPTP library find theorem to differentiate Spass
  E
• See AI and Maths paper
• Prediction tests (e.g., Progol animals file)
• P(mammal has_milk) 1.0
• P(mammal habitat(water)) 0.125
• Take average over all Bayesian probabilities

42
Applications of Theory Formation

• Identifying concepts (e.g., Michalski trains)
• Forward look ahead mechanism (see ICML-00 paper)
• Simplifying problems
• Lemma generation for ATP
• Constraint generation for CSP (see CP-01 paper)
• Identifying outliers
• How unique an object of interest is
• Inventing concepts
• Integer sequences (and conjectures),
• See AAAI-00 paper, Journal of Integer Sequences

43
Conclusions

• Presented a snapshot of the theory of ATF
• Autonomous
• Four components, numerous techniques
• Uses third party software
• Concept driven and conjecture driven cycles
• Applies to many machine learning tasks
• Concept identification, puzzle generation,
• Predictions, problem simplification

44
Welcome to the Next Level

• For any of the four components
• Substitute a human for interactive ATF
• Roy McCasland (hopefully), mathematician
• Work on Zariski spaces with HR
• For any of the four components
• Substitute another agent for multi-agent ATF
• Alison Peases PhD, cognitive modelling
• Lakatos style reasoning and machine creativity

45
Theory Formation in Bioinformatics?

• Can work with non-maths data
• Can form near-conjectures
• Needs to relax notion of equality
• Multi-agent approach definitely needed