Unsupervised Learning of Probabilistic Concept Hierarchies

1
Unsupervised Learning of Probabilistic Concept
Hierarchies

• Wayne Iba
• Pat Langley

Presentation by Michael Metternich 0012645
2
Unsupervised (Clustering) Methods

• Aim is to acquire concepts or categories
• Concept descriptions represent the variability in
  the natural world
• Concepts describe experience at different levels
  of generality
• The learning process is incremental

3
Unsupervised (Clustering) Methods

• Three main types of unsupervised methods
• K-means clustering
• Hierarchical clustering
• Self Organizing Maps (SOMs)

4
K-means clustering

• The basic steps to follow given a raw dataset
• 1. Initialization
• Define the number of clusters (k).
• Designate a cluster centre (a vector quantity
  that is of the same dimensionality of the data)
  for each cluster.
• 2. Assign each data point to the closest cluster
  centre.
• 3. Calculate the new cluster centre
• 4. Calculate the sum of within-cluster
  sum-of-squares.

5
Hierarchical clustering

• two types of hierarchical clustering
  agglomerative and divisive

6
Hierarchical clustering

• Basic Algorithm for Hierarchical Clustering
• Derive vector representations for each entity
• Compare every entity with all other entities by
  calculating a distance.
• Input that distance into a matrix.
• Group closest two entities together
• go back to step 2

7
Cobweb - goals

• Infer the values of attributes missing from test
  cases (flexible prediction)
• Hierarchical clustering over unlabeled training
  cases.
• Estimate the probability density function over
  the space of possible instances

8
Cobweb central notion

• Generate an arbitrary name C for each category
  and associate with C a descriptive summary.
• For each attribute A, this summary specifies a
  probability distribution over the values of A,
  conditioned on the category C

9
Cobweb - example
10
Cobweb - learning

• Extending downward (a)
• Creating a disjunct (b)
• Merging two categories (c)
• Splitting a concept (d)

11
Cobweb - assumptions

• At each level of the concept hierarchy attributes
  are conditionally independent given the category.
• Use of a hierarchical memory gives it the ability
  to represent complex target concepts.

12
Cobweb - evaluation

• Several testing techniques
• Instances with some attribute value omitted and
  letting the system predict the value.
• Labeled data, holding back the labels during
  hierarchy construction.
• Tests on specific parameters and characteristics.

13
Cobweb evaluation (continued)
14
Cobweb evaluation (continued)
15
Cobweb - extensions

• Arachne
• Twilix
• Oxbow
• Bridger
• Labyrinth
• Dædalus
• Exor

16
Cobweb closing remarks

• A selected sample of empirical studies was
  shown, that have established the frameworks
  robustness along several dimensions. Although
  these studies were generally encouraging, they
  also suggested some limitations and directions
  for improvement.
• Overview from http//www.ucl.ac.uk/oncology/Micro
  Core/HTML_resource/Unsupervised_Clustering_banner.
  htm