COMP9444: Neural Networks - PowerPoint PPT Presentation


PPT – COMP9444: Neural Networks PowerPoint presentation | free to download - id: b954f-ZDc1Z


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation

COMP9444: Neural Networks


... from the ratio, i.e. from our mind and we just apply our innate concepts to ... philosopher Descartes claims dualism between mind and body mind-body problem. ... – PowerPoint PPT presentation

Number of Views:63
Avg rating:3.0/5.0
Slides: 41
Provided by: ach35


Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: COMP9444: Neural Networks

COMP9444 Neural Networks
  • Neuroanatomy Divisions of the Nervous System

Neural Networks
  • Natural neural networks are complex arrangements
    of a (usually large) number of neurons. A neuron
    is the basic unit of neural networks, which can
    be said to perform computation. Artificial neural
    networks are (usually very simplified) models of
    natural neural networks.

Why neural networks
  • Reverse engineering approach to artificial
    intelligence.Understanding the human
    brain.Understanding cognition.
  • Good learning capabilities,function holistically,
    graceful degradation, fault tolerant, compute
    continuous valued non-linear functions.

Theories about human intelligence
  • Rationalism (Plato) is the idea that our
    understanding comes from the ratio, i.e. from our
    mind and we just apply our innate concepts to our
    sense experiences.
  • Plato spoke of remembering the innate' concepts
    of the world when we perceive objects in order to
    classify them. We do not always remember these
    concepts, but need to learn to remember and to
    apply them.

Theories about human intelligence
  • Empiricism (Aristotle) is the idea that our
    understanding comes from experiences, i.e.that we
    shape our concepts through our sense experiences
    with the world around us.
  • The German philosopher Kant (18th century)
    presented a foundational analysis in his"Critique
    of Pure Reason", which essentially rejects both
    views and shows that both aspects, innateness and
    experience,must interplay with each other

Theories about human intelligence
  • 17th century French philosopher Descartes claims
    dualism between mind and body ? mind-body
  • Late 19th century Sigmund Freud founds the field
    of psychology.
  • Early 20th century Psychology develops
    behavioural theories about man and animal.
  • Late 20th century First physiologically-based
    theories of how intelligence can be produced by
    the human brain.

Artificial neural networks and AI's History
  • 1943 McCulloch and Pitts develop basic models of
  • 1962 Rosenblatt's learning Perceptron.
  • 1969 Minsky and Papert published their book
    "Perceptrons"on the limitations of neural models.
  • 1970s Knowledge-based systems.
  • 1980s AI becomes an industry. Renewed
    enthusiasm.First commercial successes through
    expert systems were achieved.

Artificial neural networks and AI's History
  • Mid to late 1980s, people feel again much more
    difficulties in getting expert systems work, than
    they expected.
  • Around 1986 Renaissance of neural networks -
  • 1990s More consolidated approaches to problems
    and techniques.More realistic expectations (i.e.
    reduced expectations and much better computing
    resources available).

History of AI
  • Major problems in the past in AI were
  • the lack of AI researcher's understanding of the
    problems they tackled.the combinatorial explosion
    in search spaces.the lack of knowledge in the
    machine.the difficulty of formalising everyday
    knowledge as well as expert knowledge.

Neuroanatomy Divisions of the Nervous System
  • The Nervous System is divided into
  • The Central Nervous System
  • The Peripheral Nervous System.

The Central Nervous System
  • The central nervous system is divided into two
    major parts
  • the brain
  • and the spinal cord.

The Human Brain
  • In the average adult human, the brain weighs 1.3
    to 1.4kg (about 3 pounds).
  • The brain contains about 100 billion nerve cells
  • and trillons of "support cells" called glia.

The Spinal Cord
  • The spinal cord is about 43 cm long in adult
    women and 45 cm long in adult men and weighs
    about 35-40 gm.
  • The vertebral column, the collection of bones
    (back bone) that houses the spinal cord, is about
    70 cm long.
  • So the spinal cord is much shorter than the
    vertebral column.

The Peripheral Nervous System
  • The peripheral nervous system is divided into
    three major parts
  • the somatic nervous system and the
  • autonomic nervous system
  • the enteric nervous system.

Divisions of the Nervous System
Human Brain
Cerebral Cortex
  • The word "cortex" comes from the Latin word for
    "bark" (of a tree). This is because the cortex is
    a sheet of tissue that makes up the outer layer
    of the brain. The thickness of the cerebral
    cortex varies from 2 to 6mm. The right and left
    sides of the cerebral cortex are connected by a
    thick band of nerve fibers called the "corpus
  • Functions Thought, Voluntary movement, Language,
    Reasoning, Perception

  • The word "cerebellum" comes from the Latin word
    for "little brain.
  • Functions Movement, Balance, Posture

Brain stem
  • The brain stem is a general term for the area of
    the brain between the thalamus and spinal cord.
    Structures within the brain stem include the
    medulla, pons, tectum, reticular formation and
  • Functions breathing, heart rate, blood pressure,
    and others.

  • The hypothalamus is composed of several different
    areas and is located at the base of the brain. It
    is only the size of a pea (about 1/300 of the
    total brain weight).
  • Functions Body Temperature, Emotions, Hunger,
    Thirst, Circadian Rhythms

  • The thalamus receives sensory information and
    relays this information to the cerebral cortex.
    The cerebral cortex also sends information to the
    thalamus which then transmits this information to
    other areas of the brain and spinal cord.
  • Functions Sensory Integration, Motor Integration

Limbic System
  • The limbic system (or the limbic areas) is a
    group of structures that includes the amygdala,
    the hippocampus, mammillary bodies and cingulate
    gyrus. These areas are important for controlling
    the emotional response to a given situation. The
    hippocampus is also important for memory.
  • Functions Emotional Behavior

  • The midbrain includes structures such as the
    superior and inferior colliculi and red nucleus.
    There are several other areas also in the
  • Functions Vision, Audition, Eye Movement, Body

Animal Brains
  • Chimpanzee 2) Cat 3) Spiny Anteater 4)
  • 5) Capybara 6) Squirrel Monkey 7) Manatee
  • 8) Least Weasel 9) Beaver

  • The body is made up of billions of cells. Cells
    of the nervous system, called neurons, are
    specialized to carry "messages" through an
    electrochemical process. The human brain has
    about 100 billion neurons.
  • Neurons (nerve cells) come in many different
    shapes and sizes. Some of the smallest neurons
    have cell bodies that are only 4 microns wide,
    while some of the biggest neurons have cell
    bodies that are 100 microns wide.

Neurons and body cells
  • Neurons are similar to other cells in the body in
    some ways such as
  • 1.Neurons are surrounded by a cell membrane.
  • 2.Neurons have a nucleus that contains genes.
  • 3.Neurons carry out basic cellular processes like
    protein synthesis and energy production.

Neurons versus body cells
  • 1.Neurons have specialized extensions called
    dendrites and axons. Dendrites bring information
    to the cell body and axons take information away
    from the cell body.
  • 2.Neurons communicate with each other through an
    electrochemical process.
  • 3.Neurons contain some specialized structures
    (for example, synapses) and chemicals (for
    example, neurotransmitters).

  • A neuron typically has many dendrites and one
    axon. The dendrites branch and terminate in the
    vicinity of the cell body. In contrast, axons can
    extend to distant targets, more than a meter away
    in some instances.
  • Dendrites are rarely more than about a millimeter
    long and often much shorter.
  • Neurons communicate through specialized junctions
    called synapses.

  • The electrical potential travels along an axon to
    another synapse connected to another neuron.Axons
    are either myelinated (layers of tissue around
    the axon) or unmyelinated. Axon diameters range
    between about 0.2?m and 1 mm.

Axons - signal propagation
  • Generally the signal travelling speed grows with
    the diameter of the axon.The signal travelling
    speed for unmyelinated axons is 20m/s for a 1mm
    axon in the squid.Compared to that is the speed
    in myelinated axons, which can reach up to 100m/s
    in a 20 ?m axon. Speed may vary due to branching
    and changing diameters etc.In a 3.5mm long
    branched axonal tree it was found that the signal
    delay ranges between 3ms and 6ms.

Different Structures of a Neuron
Dendrites and Synapses
The Synapse
  • In most synapses the direct cause of change in
    potential is not electrical but chemical The
    electrical pulse reaches the endbulb and causes
    the release of transmitter molecules from little
    packets (vesicles) through the synaptic membrane.
  • Transmitter than diffuses through the synaptic
    cleft to the other side.When the transmitter
    reaches the post-synaptic membrane, it causes the
    change in polarisation of the membrane.The change
    in potential can be excitatory (moving the
    potential towards the threshold)or inhibitory
    moving the potential away from the threshold.

A schematic Synapse
Presynaptic Axon
Postsynaptic Spine
Active Zone
Synaptic Vesicle
Synaptic Cleft
Postsynaptic Density
An Asymmetric Synapse
From an electron microscope
3-D Image of a Dendrite (rat, hippocampus)
(No Transcript)
Action Potential
(No Transcript)