Introduction to Artificial Intelligence: Applications in Computational Biology - PowerPoint PPT Presentation

Loading...

PPT – Introduction to Artificial Intelligence: Applications in Computational Biology PowerPoint presentation | free to download - id: 14102b-YTczZ



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Introduction to Artificial Intelligence: Applications in Computational Biology

Description:

'Artificial intelligence (AI) is the study of how to make ... Mundane tasks. Perception. Vision. Speech. Natural Language. Understanding. Generation. Translation ... – PowerPoint PPT presentation

Number of Views:215
Avg rating:3.0/5.0
Slides: 77
Provided by: brid157
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Introduction to Artificial Intelligence: Applications in Computational Biology


1
Introduction to Artificial Intelligence
Applications in Computational Biology
  • Susan M. Bridges bridges_at_cs.msstate.edu

2
Outline
  • What is AI?
  • Search
  • Expert systems
  • Uncertainty
  • Machine learning
  • Data mining

3
Intelligent Systems and Computational Biology
  • First applications (DNA) in which great progress
    was made were digital
  • Signal processing algorithms
  • Text processing techniques
  • Many of the most interesting and difficult
    problems to be tackled are analog
  • Protein structure
  • Gene expression
  • Metabolic networks

4
Definitions of AI (What is AI?)
  • Rich, E. and K. Knight . 1991. Artificial
    Intelligence. New York McGraw-Hill.
  • Artificial intelligence (AI) is the study of
    how to make computers do things which at the
    moment, people do better.

5
Another definition of AI
  • Winston, Patrick Henry. 1984. Artificial
    Intelligence. 1984. Addison-Wesley, Reading,
    MA.
  • Artificial Intelligence is the study of ideas
    that enable computers to be intelligent.
    Intelligence includes ability to reason,
    ability to acquire and apply knowledge, ability
    to perceive and manipulate things in the physical
    world, and others.

6
Why Study AI?
  • Understand human human intelligence
  • Develop intelligent machines
  • Robotics
  • Programs with intelligent properties

7
Acting Rationally Turing Test Approach
Interrogator
8
AI Tasks
  • Formal tasks
  • Games
  • Mathematics
  • Geometry
  • Logic
  • Integral calculus
  • Expert tasks
  • Engineering
  • Scientific analysis
  • Medical diagnosis
  • Financial analysis
  • Mundane tasks
  • Perception
  • Vision
  • Speech
  • Natural Language
  • Understanding
  • Generation
  • Translation
  • Common sense reasoning
  • Robot control

9
Intelligent Agents
  • Agent
  • Perceives its environment using sensors
  • Acts on environment using effectors
  • Rational agent
  • An agent that does the right thing
  • Basis for action
  • A measure of degree of success.
  • Knowledge of what has been perceived so far.
  • The actions that the agent can perform
  • Autonomous Agent
  • Learns from experience
  • Makes independent decisions

10
Major Topics
  • Search
  • Knowledge Representation
  • Machine Learning

11
Problem-solving agent
  • A type of goal-based agent
  • Find sequence of actions that lead to a desirable
    state
  • Intelligent agents should make a set of changes
    in the state of the environment that maximizes
    the performance measure
  • Life is simpler if we can set a goal and aim to
    satisfy it.

12
Components of a problem
  • Initial state
  • Set of possible actions
  • actions can be described as operators
  • an operator describes an action by specifying the
    state that can be reached by carrying out an
    action in a particular state
  • actions can be described in terms of a successor
    function S. Given a particular state x, S(x)
    returns the set of states reachable from x by any
    single action.

Operator a
State x
State y
13
State Space
  • The set of all states reachable from the initial
    state by any sequence of actions
  • A path in the state space is a sequence of
    actions leading from one state to another
  • The agent can apply a goal test to any single
    state to determine if it is a goal state.
  • If one path is preferable to another, then we may
    need to compute path cost (g).

14
3
5
4
1
2
6
1
8
8
4
7
3
2
7
6
5
Initial State
Goal State
States Goal Test Operators Path Cost
15
Problem Find route from Louisville to West Point
West Point
Pheba
Mathiston
Mayhew
Maben
Columbus
Starkville
Sturgis
Artesia
Ackerman
Crawford
Brooksville
Louisville
16
Louisville
A. The initial state
Louisville
B. After expanding Louisville
Ackerman
Starkville
Brooksville
Louisville
C. After expanding Ackerman
Ackerman
Starkville
Brooksville
Maben Sturgis Louisville
17
Some terms
  • New states are generated from old states by
    operators.
  • This is called expanding the state.
  • The choice of which state to expand first is
    called the search strategy
  • Result is called a search tree
  • The set of nodes waiting to be expanded is called
    the fringe or frontier

18
Search Strategies
  • Requirements for a good search strategy
  • causes motion
  • is systematic
  • State space can usually be represented as a tree
    or a graph
  • Two important parameters of a tree
  • branching factor (b)
  • depth (d)

19
Two Types of Searches
  • Uninformed or blind search
  • systematically generate states
  • test states to see if they are goal states
  • Informed or heuristic search
  • use knowledge about the problem domain
  • explore search space more efficiently
  • may sacrifice accuracy for speed

20
Breadth-first search
  • All nodes at each depth d are expanded before any
    nodes at depth d1

21
Depth-first search
  • Always expands one of the nodes at the deepest
    level of the tree
  • Parameter m is the maximum depth

22
What is a heuristic? (rule of thumb)
  • A heuristic is a formalized rule for choosing
    those branches in a state space that are most
    likely to lead to an acceptable solution (Luger
    and Stubblefield, 1998).
  • Used two ways
  • some problems do not have exact solutions, so we
    just do the best we can (medical diagnosis)
  • there may be an exact solution, but it may be
    very expensive to find

23
Hill Climbing
  • Use an heuristic function (or objective or
    evaluation function) to decide which direction to
    move in the search space.
  • Always move toward the state that appears to be
    best (basing all decisions on local information).
  • Assume that we want to maximize the value of the
    function.
  • Can also be used for minimization (called
    gradient descent)

24
Steepest Ascent Hill Climbing Using Manhattan
Distance Heuristic
Goal
1 2 3 7 4 6 8 5
h
h
h
25
A Search
  • Minimizing the total path cost
  • Combines uniform-cost search and greedy search.
  • Evaluation function
  • f(n) g(n) h(n)
  • g(n) cost of path from start to node n
  • h(n) estimate of cost of path from n to goal
  • f(n) estimated cost of the cheapest solution
    through n

26
Goal Minimum length path. Is h(n) an admissible
heuristic? f(n) g(n) h(n)
A(22)
d 0
3
5
10
d 1
B (18) C (21)
D (8)
4
6
12
7
8
11
d 2
E(12) F(7) G (9) H(6)
I (13) J(14)
3 11 4 2 7
1 5 12 3 4
d 3
K (18) L ( 3) M(2) N(9)
O(5) P(2) Q(10) R(12) S
(18) T(0)
U (0)
3 4 14
6 5
d 4
Numbers in parentheses are h(n) Numbers on edges
are operator costs
27
Multiple Sequence Alignment
  • DNA and protein sequences
  • Alignment of multiple sequences created by
    inserting gaps to shift characters to matching
    positions
  • ATCG -ATCG-
  • TGA --T-GA
  • GAT GAT---
  • Optimal alignment maximizes the number of
    matching positions

28
Multiple Sequence Alignment As State-Space
Search (Eric Hansen, Rong Zhou)
Space Complexity O (LN) Time Complexity
O (2NLN)
Where L is the average length of sequences and
N is the number of sequences
start
ATCG -ATCG-
TGA --T-GA GAT GAT---

goal
29
An Illustration of Anytime A
f g 2?h
Nodes pruned by Anytime A
Goal
expanded node
Total number of nodes stored 8
stored but not expanded node
30
Genetic Algorithms
  • Search procedure based on a simple model of
    evolution
  • Uses a random process to explore search space
  • Has been applied in many domains

31
Terminology
  • Begin with a population of individuals. Each
    individual represents a solution to the problem
    we are trying to solve.
  • A data structure describes the genetic structure
    of the individual. (Assume for initial
    discussion that this is a string of 0s and 1s).
  • In genetics, the strings are called chromosomes
    and the bits are called genes.
  • The string associated with each individual is its
    genotype
  • Selection is based on fitness of individuals

32
The Genetic Algorithm
  • Each evolving population of individuals is called
    a generation.
  • Given a population of individuals corresponding
    to one generation, the algorithm simulates
    natural selection and reproduction in order to
    obtain the next generation.

33
Three basic operations
  • Reproduction
  • Individuals from one generation are selected for
    the next generation
  • Crossover
  • Genetic material from one individual is exchanged
    with genetic material from another individual
  • Mutation
  • Genetic material is altered

34
General GA Procedure
Selection, crossover, and mutation operations
Initial population
Evaluate fitness
Parent candidate pool
Father and Mother
Select parents
Crossover and mutate
Offspring
Evaluate fitness and replace
no
yes
Next generation population
Converge?
35
Example of General GA Procedure
Selection, crossover, and mutation operations
Generation n
1 1 0 1
1 0 1 1
0 1 0 0
1 0 0 1
Reproduction
Crossover
Mutation
Generation n 1
36
Two keys to the success of a GA
  • Data structures for
  • Genes
  • Chromosomes
  • Population
  • Fitness evaluation function

37
Knowledge Representation
  • Semantic networks
  • Frame based systems
  • Rule based expert systems
  • Ontologies
  • Neural networks

38
Anything
AbstractObjects
Events
Representational Objects
Places
Processes
Sets
Numbers
Physical Objects
Intervals
Sentences
Measurements
Moments
Things
Stuff
Categories
Animals
Agents
Times
Weights
Humans
39
Expert Systems
  • Rule based systems
  • Garnered a great deal of attention in the 1980s
  • Most famous examples are in medical domains
  • Stimulated interest in logic programming
  • Encode knowledge of people as sets of rules
  • Still widely used
  • Knowledge acquisition bottleneck

40
Representing Uncertainty
  • Fuzzy logic
  • Bayesian reasoning

41
Uncertainty versus Vagueness
  • Certaintydegree of belief
  • there is a 50 probability of rain today
  • I am 30 sure the patient is suffering from
    pneumonia
  • Vaguenessthe degree to which an item belongs to
    a category
  • the man is tall
  • move the wheel slightly to the left
  • the patients lungs are highly congested

42
Fuzzy Sets Represent Vagueness
  • Lotfi Zadeh popularized the idea in the 60s
  • Popular concept in Eastern philosophy
  • Reasoning with fuzzy sets is called fuzzy logic
  • Fuzzy logic is also called
  • approximate reasoning
  • continuous logic

43
Fuzzy Set Definitions
  • Set membership can be expressed using a
    characteristic (or descrimination) function
  • Classic (or crisp) sets
  • If objects x are chosen from some universe X
  • Fuzzy sets - an element can be a partial member
    of a set (grade of membership)

44
Examples of Fuzzy Concepts from Natural Language
  • John is tall
  • The weather is rainy
  • Turn the volume up a little
  • Dr. Bridges tests are long
  • Add water until the dough is the right
    consistency
  • There was very little change in the cost
  • The water bill was somewhat high

45
Representing Fuzzy Sets
  • Enumeration of membership values of all elements
    with non-zero membership
  • TALL .125/5.5, .5/6, .875/6.5, 1/7, 1/7.5,
    1/8
  • Represent membership with a function

46
Functional Representations Fuzzy Set Tall
Membership
1
?Tall
0
Height in feet
4 5 6
7
47
Linguistic (or Fuzzy)Variable
  • Usually corresponds to a noun
  • The values of a linguistic variable are fuzzy
    sets (which correspond to adjectives)
  • Examples
  • Linguistic variable Fuzzy sets
  • Height short medium tall
  • Weight light average heavy
  • Temperature cold cool typical warm hot
  • Speed slow medium fast

48
Linguistic Variable Temperature
Cold Normal Hot
1
0
30 40 50 60 70 80 90
100
49
Some Fuzzy Set Operations
  • Set union A ? B
  • ?A ? B(x)??max(?A(x),??B(x)) for all x ??X
  • alternate syntax (join operator)
  • ?A ? B(x)???A(x)????B(x)) for all x ??X
  • Set intersection A ??B
  • ?A??B(x)???min(?A(x),??B(x)) for all x ??X
  • alternate syntax (meet operator)
  • ?A ? B(x)???A(x) ???B(x)) for all x ??X

50
Fuzzy Reasoning
  • A fuzzy proposition is a statement that asserts a
    value for a linguistic (or fuzzy) variable
  • Example Joes height is medium
  • Linguistic variable (noun) Joes height
  • Fuzzy set (adjective) medium
  • The fuzzy set medium is a value of the
    linguistic variable Joes height
  • A fuzzy rule relates two or more fuzzy
    propositions
  • Fuzzy inference techniques are used to draw
    conclusions using fuzzy rules

51
Example Fuzzy Rule
  • If speed is normal
  • then braking.force is medium
  • Speed
  • Normal (0/0, .1/20, .8/40, 1/60, .1/80, 0/100)
  • braking.force
  • Medium (0/0, .5/1, 1/2, 1/3, .2/4, 0/5)

52
J. Dickerson, D. Bedeant, Z.Cox, W. Qi, D.
Ashlock, and E. Wurtele, Atlantic Symposium
on Computational Biology and Genome Information
Systems Technology (CBGIST 2000, 26-30.
53
Bayesian Reasoning
  • Bayesian networks Represent knowledge as a
    network of random variables
  • Many names and many variations
  • Belief networks
  • Probabilistic networks
  • Causal networks
  • Knowledge Maps
  • Influence Diagram (extension)
  • Decision Network (extension)

54
Belief Network
P(B) 0.001
P(E) 0.002
Burglary
Earthquake
B E P(AB,E) T T 0.95 T
F 0.94 F T 0.29 F
F 0.01
Alarm
A P(JA) T 0.90 F 0.05
A P(MA) T 0.70 F 0.01
JohnCalls
MaryCalls
55
(No Transcript)
56
Classification of Learning Systems
  • Supervised learning
  • Give the system a set of examples and an answer
    for each example.
  • Train the system until it can give the correct
    response to these examples (or most of them).
  • Unsupervised learning
  • Give the system a set of examples and let it
    discover interesting patterns in the examples.
  • Reinforcement learning
  • Learn from rewards and penalties

57
Feature Vectors
  • Simple representation used by most learning
    systems.
  • Represents each example as a vector or numbers
  • Quantities
  • Nominal data
  • Ordinal data

58
Neural Networks
  • Computational models loosely based on the
    structure of the brain
  • Characteristics of the brain
  • Large number of simple processing units (neurons)
  • Highly connected
  • No central control
  • Neurons are slow devices compared to digital
    computers
  • Can perform complex tasks in a short period of
    time
  • Neurons are failure-prone devices
  • Handles fuzzy situations very well.
  • Information accessed on the basis of content
  • Learns from experience

59
Neural Networks
  • Based on model of nervous system
  • Large numbers of simple processing units
  • Units are highly connected and connections are
    weighted.
  • Highly parallel distributed control
  • Emphasis on learning internal representations
    automatically

60
Neural Network Concepts
  • Cell or unit or neuron or node
  • Autonomous processing unit that models a neuron
  • Purpose
  • Receives information from other cells
  • Performs simple processing
  • Sends results on to one or more cells
  • Layers
  • A collection of cells that perform a common
    function
  • Types
  • Input layer
  • Hidden layer
  • Output layer

61
Layers of Neurons
I1
H1
O1
I2
H2
I3
Input Layer
Hidden Layer
Output Layer
62
Properties
  • In general, there is no interconnection between
    cells in the same layer
  • Connections are one or two way communications
    links between two cells
  • Weights are the strength of the connections. A
    weight wij is a real number than indicates the
    influence that cell ui has on cell uj

63
More about weights
  • Positive weights indicate reinforcement
  • Negative weights indicate inhibition
  • Weight of 0 indicates no influence or connection
  • Weights may be initialized to one of these
  • 0
  • predefined values
  • random values
  • Weights are altered by experience

64
Multilayer Feed-Forward Networks
  • Networks that are connected acyclic graphs
  • Backpropagation
  • Most popular training method for feed forward
    layered networks.
  • Invented in 1969 by Bryson and Ho
  • Ignored until 80s
  • Supervised learning technique

65
Back Propagation
  • Initialize the network with random weights
  • Show it an input instance
  • Compute the output
  • Determine how much the output differs from the
    goal.
  • Feed small adjustments to the weights back
    through the network based on the error.

66
General Algorithm for Training Network
Initialize NN for epoch 1 to MAXEPOCHS for
each input-output pair in the training
set present an example and compute
error adjust weights to reduce the
error compute mean-square-error for training
set for each input-output pair in the test set
compute error compute mean-square-error for
test set
67
Generalization
Training set
correct
Test set
Training time ( of epochs)
68
Computational Biology Applications
  • Protein classification
  • Sequence analysis
  • DNA fragment assembly
  • Prediction of transmembrane regions
  • Phylogenetic classification of ribosome sequences
  • And many more

69
Self-Organizing Maps
  • Also called Kohonen maps
  • Used for unsupervised learning
  • Widely applied for comparison of gene expression
    data

70
Principle of Self-Organizing Maps
71
Self-Organizing Map from Yeast Gene Expression
Data (German Florez)
72
Knowledge Discovery
  • Definition Non-trivial extraction of implicit,
    previously unknown, and potentially useful
    information from data.
  • Applications in biology
  • Text mining
  • Association rule mining

73
KDD Process
Interpretation/ Evaluation
Knowledge
Patterns
--- --- --- --- --- --- --- ---
Transformed Data
Preprocessed Data
Target Data
Data
74
Iterative Clustering Procedure ( Wan, Bridges,
J.Boyle, A.Boyle)
75
Positional Weight Matrix Representation

76
Clustering Results
About PowerShow.com