NEURAL NETWORKS FOR DATA MINING - PowerPoint PPT Presentation

Loading...

PPT – NEURAL NETWORKS FOR DATA MINING PowerPoint presentation | free to download - id: 55997f-OWNjZ



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

NEURAL NETWORKS FOR DATA MINING

Description:

Chapter 8 NEURAL NETWORKS FOR DATA MINING Developing Neural Network Based Systems Data collection and preparation The data used for training and testing must ... – PowerPoint PPT presentation

Number of Views:72
Avg rating:3.0/5.0
Slides: 44
Provided by: Jud8187
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: NEURAL NETWORKS FOR DATA MINING


1
Chapter 8
  • NEURAL NETWORKS FOR DATA MINING

2
Learning Objectives
  • Understand the concept and different types of
    artificial neural networks (ANN)
  • Learn the advantages and limitations of ANN
  • Understand how backpropagation neural networks
    learn
  • Understand the complete process of using neural
    networks
  • Appreciate the wide variety of applications of
    neural networks

3
Basic Concepts of Neural Networks
  • Neural networks (NN) or artificial neural network
    (ANN)
  • Computer technology that attempts to build
    computers that will operate like a human brain.
    The machines possess simultaneous memory storage
    and works with ambiguous information
  • A brain metaphor for information processing.
    These models are biologically inspired rather
    than an exact replica of how the brain actually
    functions.
  • Very promising systems
  • Ability to learn from the data
  • Nonparametric nature(no rigid assumptions)
  • Ability to generalize

4
Basic Concepts of Neural Networks
  • Neural computing
  • An experimental computer design aimed at
    building intelligent computers that operate in a
    manner modeled on the functioning of the human
    brain. See artificial neural networks (ANN)
  • it is a pattern recognition methodology for
    machine learning. The resulting model from the
    neural computing is often called ANN.
  • Development history (ANN journey)
  • McCulloch and Pitts, 1943
  • 1950s-1960s popular
  • 1970s-1980s diminished
  • 1990s-, rational developing.
  • 2000s-, data mining,

5
Basic Concepts of Neural Networks
  • Biological and artificial neural networks
  • Perceptron (???)
  • Early neural network structure that uses no
    hidden layer
  • Neurons (???)
  • Cells (processing elements) of a biological or
    artificial neural network
  • Nucleus (???)
  • The central processing portion of a neuron
  • Dendrite (??)
  • The part of a biological neuron that provides
    inputs to the cell
  • Axon (??)
  • An outgoing connection (i.e., terminal) from a
    biological neuron
  • Synapse (??)
  • The connection (where the weights are) between
    processing elements in a neural network

6
Basic Concepts of Neural Networks
7
Basic Concepts of Neural Networks
8
Basic Concepts of Neural Networks
Biological Artificial
Soma Node
Dendrite Input
Axon Output
Synapse Weight
Slow Speed Fast Speed
Many neurons (10e9) Few neurons (10-100s)
9
Basic Concepts of Neural Networks
  • Elements of ANN
  • Topologies (architectures)
  • The type neurons are organized in a neural
    network
  • Backpropagation (????,????)
  • The best-known learning algorithm in neural
    computing. Learning is done by comparing computed
    outputs to desired outputs of historical cases
  • 1986?Rumelhart?McCelland?????

10
Basic Concepts of Neural Networks
  • Processing elements (PEs)
  • The PE in a neural network are artificial
    neurons.
  • each of the neurons receives input,
    processes them, and deliveries a single output.
    Show in figure 8.2.
  • Network structure (three layers)
  • Input
  • Intermediate (hidden layer)
  • Output
  • A hidden layer is a layer of neurons that takes
    input from the previous layer and converts those
    input into output for further processing.
  • Several hidden layers can be places between input
    and output layers.
  • Most of hidden layers process mechanisms are
    feature extraction.

11
Basic Concepts of Neural Networks
12
Basic Concepts of Neural Networks
  • Parallel processing
  • An advanced computer processing technique that
    allows a computer to perform multiple processes
    at oncein parallel
  • Resembles the way the brain works, it
    differs from the serial processing of
    conventional computing.

13
Basic Concepts of Neural Networks
  • Network information processing
  • Inputs each input corresponds t a single
    attribute. E.g., problem is approval or
    disapproval for a loan.
  • Numeric value, or representation, of an
    attribute is the input to the network.
  • Outputs contain the solution of a problem. For
    example, loan can be yes or no. Also, numeric
    value to the output, such as 0, or 1 for yes or
    no.
  • The purpose of ANN is to compute the
    values of the output. Thus, post-processing of
    output is required .

14
Basic Concepts of Neural Networks
  • Network information processing
  • Connection weights
  • The weight associated with each link in a neural
    network model. They are assessed by neural
    networks learning algorithms
  • Summation function or transformation (transfer)
    function
  • In a neural network, the function that sums and
    transforms inputs before a neuron fires. The
    relationship between the internal activation
    level and the output of a neuron

15
Basic Concepts of Neural Networks
16
Basic Concepts of Neural Networks
  • Sigmoid (logical activation) function
  • An S-shaped transfer function in the range of
    zero to one

17
Basic Concepts of Neural Networks
  • Threshold value
  • A hurdle value for the output of a neuron to
    trigger the next level of neurons. If an output
    value is smaller than the threshold value, it
    will not be passed to the next level of neurons
  • Hidden layer
  • The middle layer of an artificial neural network
    that has three or more layers
  • Theoretically, there can be 10-100 layers.
    However, more than three layers are seldom in
    commercial software.
  • The more of hidden layers, the more the time will
    be used for training.

18
Basic Concepts of Neural Networks
  • Neural network architectures
  • Common neural network models and algorithms
    include
  • Backpropagation
  • Feedforward (or associative memory)
  • Recurrent network

19
Basic Concepts of Neural Networks
20
Basic Concepts of Neural Networks
21
Learning in ANN
  • Learning algorithm ( also, training algorithm)
  • The training procedure used by an artificial
    neural network . Learning algorithm specify the
    process by which a neural network learns the
    underlying relationship between input and
    outputs, or just among the inputs.
  • There are hundreds of them.
  • It can be classified into to categories
    Supervised learning and unsupervised learning.

22
Learning in ANN
23
Learning in ANN
  • Supervised learning
  • A method of training artificial neural networks
    in which sample cases (training sets) are shown
    (teaching ) to the network as input and the
    weights are adjusted to minimize the error in its
    outputs. The training set and desired output is
    iteratively presented to the neural networks.
    Output of the network in its present form is
    calculated and compared to the desired output.
  • Backpropagation learning algorithm is
    popular supervised learning.
  • It is an iterative gradient-descent
    technique designed to minimize an error function
    between the actual output of the network and its
    desired output, as specified in the training data
    set.
  • Unsupervised learning
  • A method of training artificial neural networks
    in which only input stimuli are shown to the
    network, it organizes itself internally so that
    each hidden processing element responds
    strategically to a different set of input
    stimuli. No knowledge is supplied about which
    classification are correct.
  • So it called self-organizing or
    clustering its neurons related to the specific
    desired task.

24
Learning in ANN
  • Self-organizing
  • A neural network architecture that uses
    unsupervised learning
  • Adaptive resonance theory (ART)
  • An unsupervised learning method created by
    Stephen Grossberg. It is a neural network
    architecture that is aimed at being more
    brain-like in unsupervised mode
  • Kohonen self-organizing feature maps
  • A type of neural network model for machine
    learning

25
Learning in ANN
  • The general ANN learning process
  • The process of learning involves three tasks
  • Compute temporary outputs
  • Compare outputs with desired targets
  • Adjust the weights and repeat the process

26
Learning in ANN
27
Learning in ANN
  • The general ANN learning process
  • The process of learning involves three tasks
  • Compute temporary outputs
  • Compare outputs with desired targets
  • Adjust the weights and repeat the process

28
Learning in ANN
  • Pattern recognition
  • The technique of matching an external pattern to
    one stored in a computers memory used in
    inference engines, image processing, neural
    computing, and speech recognition (in other
    words, the process of classifying data into
    predetermined categories).

29
Learning in ANN
  • How a network learns
  • Learning rate
  • A parameter for learning in neural networks. It
    determines the portion of the existing
    discrepancy that must be offset
  • Momentum
  • A learning parameter in feedforward-backpropagati
    on neural networks

30
Learning in ANN
  • How a network learns
  • Backpropagation
  • The best-known learning algorithm in neural
    computing. Learning is done by comparing computed
    outputs to desired outputs of historical cases

31
Learning in ANN
  • How a network learns
  • Procedure for a learning algorithm
  • Initialize weights with random values and set
    other parameters
  • Read in the input vector and the desired output
  • Compute the actual output via the calculations,
    working forward through the layers
  • Compute the error
  • Change the weights by working backward from the
    output layer through the hidden layers

32
Developing Neural NetworkBased Systems
33
Developing Neural NetworkBased Systems
  • Data collection and preparation
  • The data used for training and testing must
    include all the attributes that are useful for
    solving the problem
  • Selection of network structure
  • Selection of a topology
  • Topology
  • The way in which neurons are organized in a
    neural network

34
Developing Neural NetworkBased Systems
  • Data collection and preparation
  • The data used for training and testing must
    include all the attributes that are useful for
    solving the problem
  • Selection of network structure
  • Selection of a topology
  • Determination of
  • Input nodes
  • Output nodes
  • Number of hidden layers
  • Number of hidden nodes

35
Developing Neural NetworkBased Systems
36
Developing Neural NetworkBased Systems
  • Learning algorithm selection
  • Identify a set of connection weights that best
    cover the training data and have the best
    predictive accuracy
  • Network training
  • An iterative process that starts from a random
    set of weights and gradually enhances the fitness
    of the network model and the known data set
  • The iteration continues until the error sum is
    converged to below a preset acceptable level

37
Developing Neural NetworkBased Systems
  • Testing
  • Black-box testing
  • Comparing test results to actual results
  • The test plan should include routine cases as
    well as potentially problematic situations
  • If the testing reveals large deviations, the
    training set must be reexamined, and the training
    process may have to be repeated

38
Developing Neural NetworkBased Systems
  • Implementation of an ANN
  • Implementation often requires interfaces with
    other computer-based information systems and user
    training
  • Ongoing monitoring and feedback to the developers
    are recommended for system improvements and
    long-term success
  • It is important to gain the confidence of users
    and management early in the deployment to ensure
    that the system is accepted and used properly

39
Developing Neural NetworkBased Systems
40
A Sample Neural Network Project
41
Other Neural Network Paradigms
  • Hopfield networks
  • A single large layer of neurons with total
    interconnectivityeach neuron is connected to
    every other neuron
  • The output of each neuron may depend on its
    previous values
  • One use of Hopfield networks Solving constrained
    optimization problems, such as the classic
    traveling salesman problem (TSP)

42
Other Neural Network Paradigms
  • Self-organizing networks
  • Kohonens self-organizing network learn in an
    unsupervised mode
  • Kohonens algorithm forms feature maps, where
    neighborhoods of neurons are constructed
  • These neighborhoods are organized such that
    topologically close neurons are sensitive to
    similar inputs into the model
  • Self-organizing maps, or self organizing feature
    maps, can sometimes be used to develop some early
    insight into the data

43
Applications of ANN
  • ANN are suitable for problems whose inputs are
    both categorical and numeric, and where the
    relationships between inputs and outputs are not
    linear or the input data are not normally
    distributed
About PowerShow.com