Infinite block models for belief networks, social networks, and cultural knowledge

1
Infinite block models for belief networks, social
networks, and cultural knowledge

• Josh Tenenbaum, MIT
• 2007 MURI Review Meeting
• Work of Charles Kemp, Chris Baker, Tom Griffiths,
  Pat Shafto, Vikash Mansinghka, Dan Roy

2
Goal

• Algorithmic tools for uncovering structure in
  belief networks, social networks, and joint
  structure (social-belief networks).
• Why?
• Joint social-belief structure culture
• Algorithms let us map cultural knowledge quickly
  and semi-automatically, detect changes and track
  dynamics.

3
Approach

• Data measurement matrices
• Peoples beliefs about things in their
  environment
• Relations between people
• Peoples beliefs about relations between things,
  or relations between people.
• Representation cluster-based models
• Clusters of things categories
• Clusters of people social groups
• Clusters of people who share similar beliefs
  about clusters of things or people cultural
  groups

4
Approach

• Learning Bayesian inference from data
• Relational models analyze arbitrary relational
  databases of beliefs, not just a single matrix
• Nonparametric models automatically determine
  complexity of representations
• Hierarchical models multiple levels of structure
• Nested models structures with structure
• Result a flexible toolkit that goes
  qualitatively beyond standard algorithms.
• e.g., ability to discover cultural groups
  characterized by a shared understanding of the
  environments relational structure.

5
Talk outline

• Classic cluster-based methods
• New methods
• Clustering with arbitrary relational systems
• Hierarchical relational clustering
• Cross-cutting clustering with nested models
• Cross-cutting relational clustering
• Guatemalan cultural data from Atran Medin
• Conclusions and future directions

6
Classic cluster-based methods

• Belief networks mixture models

7
Classic cluster-based methods

• Belief networks mixture models

8
Classic cluster-based methods

• Social networks block models

DefersTo(Pi, Pj)
9
Classic cluster-based methods

• Cultural knowledge (joint social/belief
  structure) cultural consensus model

Not cluster-based. SVD on matrix of people x
questions
10
Problems with classic methods

• No principled tools for discovering different
  cultural groups characterized by different belief
  networks.
• CCM not intended to find cultural groups, but
  rather to uncover (and test for) shared knowledge
  and authoritativeness in a single cultural group.
  Test theory without an answer key
• Can only represent simple forms of knowledge that
  fit into a single two-mode matrix.
• Cultural knowledge and social contexts are often
  much richer.

11
Talk outline

• Classic cluster-based methods
• New methods
• Clustering with arbitrary relational systems
• Hierarchical relational clustering
• Cross-cutting clustering with nested models
• Cross-cutting relational clustering
• Guatemalan cultural data from Atran Medin
• Conclusions and future directions

12
Clustering arbitrary relational systems
people
people
social relation
people
attributes

• Alyawarra tribe, central Australia (Denham)
• 104 individuals
• 27 binary social relations
• 3 attributes kinship class, age, sex (used
  only for cluster validation, not learning)

13
Clustering arbitrary relational systems
Infinite relational model (IRM) discovers 15
clusters
14
Clustering arbitrary relational systems

• International relations circa 1965 (Rummel)
• 14 countries UK, USA, USSR, China, .
• 54 binary relations representing interactions
  between countries exports to( USA, UK ),
  protests( USA, USSR ), .
• 90 (dynamic) country features purges, protests,
  unemployment, communists, languages,
  assassinations, .

15
(No Transcript)
16
Clustering arbitrary relational systems
concept
predicate
concept

• Data from UMLS (McCrae et al.) medical knowledge
  base
• 134 terms enzyme, hormone, organ, disease, cell
  function ...
• 49 predicates affects(hormone, organ),
  complicates(enzyme, cell function), treats(drug,
  disease), diagnoses(procedure, disease)

17
Clustering arbitrary relational systems
Conceptual clusters discovered
18
Concept maps
19
Hierarchical relational clustering
20
Hierarchical relational clustering
21
Cross-cutting clustering with nested models

• Models so far all learn a single system of
  clusters.
• We would like to be able to discover multiple
  cross-cutting systems of clusters.
• Within an individuals mind multiple mental
  models of a single complex domain.
• Across individuals in a population multiple
  cultural groups with different characteristic
  mental models.

22
Cross-cutting clustering with nested models
Conventional mixture model
23
Cross-cutting clustering with nested models
CrossCat model
24
Analysis of US Senate votes 1989-90

• 101 senators x 638 issues 10 systems of
  classes.

Environment agriculture
Hot-button social issues
Core democratic platform
Law and order
Military
25
Cross-cutting clustering with nested models
Nested relational model
Infinite relational model
people
people
relation
relation
people
people
26
Discovering cultural groups based on shared
relational knowledge

• Guatemala studies of Atran Medin
• Subjects
• 12 native Itza maya
• 12 immigrant Ladino
• 12 immigrant Qeqchi maya
• Questions
• Does plant i help animal j?

Nested relational model
animal
people
plant
27
Discovering cultural groups based on shared
relational knowledge
Clusters of people found

• Guatemala studies of Atran Medin
• Subjects
• 12 native Itza maya
• 12 immigrant Ladino
• 12 immigrant Qeqchi maya
• Questions
• Does plant i help animal j?

L1 L2 L3 L4 L5 L6 L7 L8 L9 L10 L11 L12 I6 I11
Q3 Q6 Q8 Q9 Q10 Q11 Q12
Q1 Q2 Q4 Q5 Q7
I1 I2 I3 I5 I7 I8 I9 I10 I12
I4
28
Talk outline

• Classic cluster-based methods
• New methods
• Clustering with arbitrary relational systems
• Hierarchical relational clustering
• Cross-cutting clustering with nested models
• Cross-cutting relational clustering
• Guatemalan cultural data from Atran Medin
• Conclusions and future directions

29
Conclusions

• A flexible toolkit for statistical learning about
  cultural knowledge and cultural groups.
• Relational models analyze arbitrary relational
  databases of beliefs, not just a single matrix
• Nonparametric models automatically determine
  complexity of representations
• Hierarchical models multiple levels of structure
• Nested models structures with structure
• Can automatically discover cultural structure in
  real-world data (Atran Medin, DARPA CPoF).

30
Ongoing and future work

• Algorithms that can scale to very large networks.
• More dynamic data and models.
• Second-generation Guatemala data
• Political data sets voting records,
  international relations
• Better statistical models for sparse networks.
• Integrated models of values and beliefs.
• More structured representations necessary to
  capture cultural stories grammars, logical
  schemas.
• Multi-level statistical models for learning about
  network structure from raw event data, or
  learning about networks with different forms of
  structure.

31
Learning network structure from raw event data
of samples 20 80
1000
Network N
edge (N)
Data D
Classes Z
1 2 3 4 5 6
7 8 9 10 11 12 13 14 15 16
class (Z)
Abstract Classes
c1

c2
c1
c2
h
0.4
0.0
c1

0.0
0.0
c2

edge (N)
Network N
Data D
(Mansinghka, Kemp, Tenenbaum, Griffiths UAI 06)
32
Learning network structure from raw event data
12
1
11
2
10
3
9
4
8
5
7
6
of samples 40 100
1000
Network N
edge (N)
Data D
Classes Z
c1
h
class (Z)
1 2 3 4 5 6 7 8 9 10 11 12
Abstract Classes
0.1
c1



c1
edge (N)
Network N
Data D
(Mansinghka, Kemp, Tenenbaum, Griffiths UAI 06)
33
Learning different forms of network structure
Dominance hierarchy Tree
Cliques Ring
Primate troop Bush administration
Prison inmates New Guinea islands
beats told likes
trades with