Causal Models, Learning Algorithms and their Application to Performance Modeling - PowerPoint PPT Presentation

Loading...

PPT – Causal Models, Learning Algorithms and their Application to Performance Modeling PowerPoint presentation | free to download - id: 11b051-ZTI1N



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Causal Models, Learning Algorithms and their Application to Performance Modeling

Description:

Variable becomes independent from all its non-descendants by conditioning on its ... Reductionism. Causal modeling = reductionism ... – PowerPoint PPT presentation

Number of Views:40
Avg rating:3.0/5.0
Slides: 30
Provided by: janle8
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Causal Models, Learning Algorithms and their Application to Performance Modeling


1
Causal Models, Learning Algorithms and their
Application to Performance Modeling
  • Jan Lemeire
  • Parallel Systems lab
  • November 15th 2006

2
Overview
  • I. Causal Models
  • II. Learning Algorithms
  • III. Performance Modeling
  • IV. Extensions

3
I. Multivariate Analysis
  • Variables
  • Experimental data

Probabilistic model of joint distribution? Relatio
nal information? A priori unknown relations
4
A. Representation of distributions
  • Factorization
  • Reduction of factorization complexity
  • Bayesian Network

Ordering 1
Ordering 2
5
B. Representation of Independencies
  • Conditional independence
  • Qualitative property P(rainquality of
    speech)P(rain)?
  • Markov condition in graph
  • Variable becomes independent from all its
    non-descendants by conditioning on its direct
    parents.
  • graphical d-separation criterion

6
Faithfulness
  • Independence-map
  • All independencies in the Bayesian network
    appear in the distribution
  • Faithfulness
  • Joint Distribution ? Directed Acyclic Graph
  • Conditional independencies ? d-separation
  • Theorem
  • if a faithful graph exists, it is the minimal
    factorization.

7
C. Representation of Causal Mechanisms
Model of the underlying physical mechanisms
  • Definition through interventions
  • causal model Conditional Probability
    Distributions
  • Causal Markov Condition Bayesian network

8
Reductionism
  • Causal modeling reductionism
  • Canonical representation unique, minimal,
    independent
  • Building block P(Xiparents(Xi))
  • Whole theory is based on this modularity
  • Intervention
  • change of block

9
Ultimate motivation for causality
  • If causal mechanisms are unrelated
  • model is faithful
  • Model canonical representation able to explain
    all qualitative properties (independencies)
  • close to reality

10
II. Learning Algorithms
  • Two types
  • Constraint-based
  • based on the independencies
  • Scoring-based
  • searches set of all models, give a score of how
    good they represent distribution

11
Step 1 Adjacency search
  • Property
  • adjacent nodes do not become independent
  • Algorithm
  • start with full-connected graph
  • check for marginal independencies
  • check for conditional independencies

12
Step 2 Orientation
  • Property
  • V-structure can be recognized
  • Algorithm
  • look for v-structures
  • derived rules

13
Assumptions
  • General statistical assumptions
  • No selection bias
  • Random sample
  • Sufficient data for correctness of statistical
    tests
  • Underlying network is faithful
  • Causal sufficiency
  • No unknown common causes

14
Criticism
  • Definition causality?
  • About predicting the effect of changes to the
    system
  • Faithfulness assumption
  • Eg. accidental cancellation
  • Causal Markov Condition
  • All relations are causal
  • Learning algorithms are not robust
  • Statistical tests make mistakes

15
Part III Performance Analysis
  • High-Performance computing

parallel system
1 processor
  • Performance Questions
  • Performance prediction
  • Parameter-dependency?
  • Reasons of bad performance?
  • System-dependency?
  • Effect of Optimizations?

16
PhD??
  • Causal modeling (cf. COMO lab, VUB)
  • Representation form
  • Close to reality
  • Learning algorithms
  • TETRAD tool (open-source, java)

17
Performance Models
  • Aim performance analysis
  • Support software developer
  • High-performance applications
  • Expected properties
  • offer insight into causes performance
    degradation
  • prediction
  • estimate effect of optimizations
  • reusable submodels
  • separate application and system-dependency
  • reason under uncertainty
  • causal models

18
Integrated in statistical analysis
  • Statistical characteristics
  • Regression analysis
  • Probability table compression
  • Outlier detection
  • Iterative process
  • 1. Perform additional experiments
  • 2. Extract additional characteristics
  • 3. Indicate exceptions
  • 4. Analyze the divergences of the data points
    with the current hypotheses

19
A. Model construction
  • Model of computation
  • time of LU decom-
  • position algorithm
  • elementsize (redundant variable) is sufficient
    for influence datatype -gt cache misses
  • regression analysis on submodels Xf(parents)
  • analysis of parameters

20
B. Detection of unexpected dependencies
  • Point-to-point communication performance
  • background communication

21
C. Finding explanations for outliers
Exceptional data in communication performance
measurements
Probability table compression
gt derived variable Interesting features
22
IV. Complexity of Performance Data
  • Mixture discrete and continuous variables
  • Mutual Information Kernel Density Estimation
  • Non-linear relations
  • Mutual Information Kernel Density Estimation
  • Deterministic relations
  • Augmented models Complexity criterion
  • Context variables
  • Work in progress
  • Context-specific independencies
  • Work in progress

23
A. Information-theoretic Dependency
  • Entropy of random variable X
  • Discretized entropy for continuous variable
  • Mutual Information

24
B. Kernel Density Estimation
  • See applets
  • Trade-off maximal entropy ltgt typicalness
  • Conclusions
  • Limited number data points needed
  • Discretization of continuous data justified
  • Form-free dependency measure

25
C. Deterministic relations
  • Yf(X)
  • Y becomes independent from Z conditioned on X
  • violation of the intersection condition (Pearl
    88)
  • Not faithfully describable

Solution augmented causal model - add
regularity to model - adapt inference algorithms
26
The Complexity Criterion
  • X Y contain equivalent information about Z
  • Select simplest relation

27
Augmented causal model
  • Restrict conditional independencies
  • Generalize d-separation
  • Reestablish faithfulness

  • Consistent models under
  • Complexity Increase assumption

28
Theory works!
Deterministic
B
Probabilistic
A
29
Conclusions
  • Benefit of the integration of statistical
    techniques
  • Causal modeling is a challenge
  • wants to know the inner from the outer
  • More information
  • http//parallel.vub.ac.be
  • http//parallel.vub.ac.be/jan
About PowerShow.com