Artificial Intelligence: Advanced Search and Machine Learning - PowerPoint PPT Presentation

1 / 23
About This Presentation
Title:

Artificial Intelligence: Advanced Search and Machine Learning

Description:

Lemur. Tiger. zebra. Traverse tree based. on values of features. Leaf node ... Rule: if small and green and flying OR large and red and non-flying THEN insect. ... – PowerPoint PPT presentation

Number of Views:41
Avg rating:3.0/5.0
Slides: 24
Provided by: supp161
Category:

less

Transcript and Presenter's Notes

Title: Artificial Intelligence: Advanced Search and Machine Learning


1
Artificial Intelligence Advanced Search and
Machine Learning
  • Lecture 15
  • Decision Tree Induction
  • Neural Nets
  • Evaluating Machine Learning Systems

2
Decision Tree Induction
  • Weve looked so far at how machine learning may
    be applied to classification problems.
  • So far just looked at how we can come up with
    very simple rules, based on features that should
    and should not be present, allowing us to do a
    yes/no (e.g., tiger/not tiger) classification.
  • But more useful structure for classification is
    the decision tree..

3
Decision Tree
Traverse tree based on values of features. Leaf
node gives classification. (equivalent to
logical formulae that may contain
disjunctions, but more natural representation,
easier for humans to use)
Stripy
yes
no
Long Tail
Big Teeth
no
no
yes
yes
Big Teeth
Lion
Rabbit
zebra
no
yes
Lemur
Tiger
4
Decision Tree Induction
  • Aims to come up with simplest decision tree
    covering the example data.
  • Simplest tree is more likely to be a general rule
    which works for new cases, rather than a complex
    formula that only applies to the given data.
  • Resulting tree is useful representation that can
    easily verified by humans and used by humans or
    programs.

5
A sample data set(with extra tigers)
6
Creating the Decision Tree
  • We create a decision tree top down looking for
    features to label nodes with.
  • The feature chosen should be a good
    discriminating feature, that helps split the
    examples into different classes (e.g, tiger vs
    not tiger).
  • For our tiger data set the best disciminating
    feature is big teeth..
  • We label the top node, and look at which examples
    go into yes and no branches for that feature.

7
Tiger Tree
Big Teeth?
yes
no
2, 6
1, 3, 4, 5
Yes branch has 3 tigers, 1 non tiger. No
branch just has non tigers. .. So we need to do
further work on yes branch, but no branch can
have label not tiger
8
Tiger Tree
  • We look for feature that best splits examples 1,
    3, 4, 5 into tigers and non-tigers.
  • Best feature is stripy

Big Teeth?
yes
no
Stripy
Not a tiger
Not a tiger
Tiger
9
In general..
  • To select a discriminating feature given a set
    of examples..
  • Pick feature that best splits examples into
    different result categories.
  • For each value of feature (e.g., yes/no)
  • Create branch in tree labelled with value of
    feature.
  • Find subset S of examples such that
    featurevalue.
  • If all examples in S are in same category, mark
    relevant node with that category. Otherwise
    repeat selection process using set S.

10
Summary Decision Tree Induction
  • Goal is to create decision tree allowing you to
    classify things based on features.
  • Do this by taking example data set, and
    repeatedly looking for most discriminating
    feature (that best splits data into different
    classes).
  • Works for multiple classes, and data where
    feature can have a number of possible values.

11
Neural Networks
  • Back to biologically inspired approaches..
  • Neural Networks are inspired by biological
    neurons..

Synapse
Soma
Axon
Axon
Dendrites
12
Biological Neuron
  • Human brain consists of thousands of millions of
    neurons.
  • Each connected to thousands of other neurons.
  • Neuron receives inputs from its neighbours - may
    become excited/activated, giving output in turn
    to other neurons.
  • Behaviour can change if connections (synapses)
    between neurons are weakened or strengthened.

13
Perceptron
  • The perceptron is a simple computational neuron.
  • It takes a number of inputs, with weights on the
    connections (cf strength of synapse)
  • if w1x1w2x2w3x3w4x4 gt thresholdthen output
    1else output 0

x4
w4
ouput
x3
w3
inputs
w2
x2
w1
x1
14
Perceptron
  • Example All the weights are 0.5
  • x1 1, x20, x30, x40, threshold 1.
  • Output 0
  • x11, x21, x31, x40, threshold 1.
  • Output 1
  • In general can of course have any number of
    inputs, and different weights on each.
  • Also may have many interconnected perceptrons.

15
Learning in Neural Nets
  • Machine Learning for Neural Nets means adapting
    the weights, given some examples.
  • We keep on adjusting the weights a bit, until the
    neural net gives the output the examples suggest.
  • Tiger example
  • Let each input in the network correspond to one
    feature (e.g., x1stripy?). Input has value 1 if
    feature has value yes.
  • We want an output of 1 if it is a tiger.

16
Learning in NNs
  • Basic learning rule is
  • See what the output is for an example.
  • If it is wrong, adjust the weights a bit.
  • Keep going with all the examples, and repeat
    until weights converge and right results
    obtained.
  • E.g., tiger 1, intitial weights

ouput
1
0.5
1
0.5
1
OK, dont change .
0.5
1
17
Learning tigers
  • Tiger 3 - Not quite right.
  • So increase weights on active connections

output
1
0.5
0
0.5
1
0.5
0
output
1
0.6
1
0.6
1
0.5
0
18
Learning tigers
  • Example 6 still not quite right.
  • Decrease weights on active connections.
  • End up with

output
1
0.6
0
0.5
1
0.4
0
19
Perceptron Learning
  • Repeat
  • For each example
  • If actual output is 1 and target is 0, decrease
    weights on active connections by small amount.
  • If actual output is 0 and target is 1, increase
    weights on active connections by small amount.
  • Until network gives right results for all
    examples.
  • (Active connections are those for which the input
    is 1).

20
NN Summary
  • Neural Networks provide alternative approach to
    learning, inspired by architecture of brain.
  • Based on repeatedly modifying weights in simple
    neurons.
  • Result is (often) a network that correctly
    classifies, but which is difficult for human to
    interpret.
  • Contrasts with decision trees, which produce
    human-readable output.

21
Evaluating Machine Learning
  • How do we test whether the rules/networks
    produced by a machine learning system are any
    good?
  • They may correctly classify all the training data
    .. But may lack generality and fail on any new
    unseen data. Consider
  • Data small green flying insect large red
    non-flying insect.
  • Rule if small and green and flying OR large and
    red and non-flying THEN insect.
  • Rule unlikely to work for new insects.

22
Evaluation
  • One way is to split the set of examples that a
    human has classified correctly into a training
    set and a test set.
  • Train on the training set (say, 50 examples) to
    obtain the rules/networks.
  • Then take the test set. Ignore for now the humans
    classification of the data. Test to see what the
    rules/networks predict.
  • Compare this with the actual (human)
    classification.
  • Score the system according to how many of the
    test case examples are correctly classified using
    the learned rules.

23
Summary
  • Inductive Learning systems use example data to
    produce general rules (or networks, or decision
    trees).
  • These rules can be used to classify new data.
  • Essentially a search/optimisation problem
    search through all possible rules to find set
    that covers the given data best.
  • Various approaches GA, NN, Decision trees.
Write a Comment
User Comments (0)
About PowerShow.com