Evolutionary Computations, Genetic Rule-based Systems, and Evolutionary Games for Real-word and Military Applications

1
Evolutionary Computations, Genetic Rule-based
Systems, and Evolutionary Games for Real-word and
Military Applications

• Jae C. Oh, Ph.D.
• EECS
• Syracuse University

2
Overview of Presentation

• Evolutionary Computations (ECs)
• Some verities
• I.e., GA, ES, EP, GP, etc.
• The Main Idea of ECs
• Genetic Learning Classifier Systems
• Speaker Identification
• Other possible applications
• Multi-agent Systems and Evolutionary Game theory,
  Emergent Behavior
• Applications

3
I should go to the game
4
Brief History

• Evolutionary Programming
• Fogel in 1960s
• Individuals are encoded to be finite state
  machines
• Intelligent Behavior
• Evolutionary Strategies
• Rechenberg, Schwefel in 1960s
• Real-valued parameter optimization
• Genetic Algorithms
• Holland in 1960s
• Adaptive Systems
• Crossover Operators

5
Present Status

• Wide variety of evolutionary algorithms
• No one seriously tries to distinguish them except
  for some special cases
• Mostly for theoretical reasons
• We will call all Evolutionary Algorithms
• And I will call them Genetic Algorithms or
  Evolutionary Algorithms, interchangeably.

6
Evolution is a search process
From the Tree of the Life Website,University of
Arizona
Orangutan
Human
Gorilla
Chimpanzee
7
Evolution is a parallel search
8
What are Evolutionary Algorithms?

• Find solutions for a problem using the idea of
  evolution
• Randomized search and optimization algorithms
  guided by the principle of Darwins natural
  selection Survival of fittest.
• Evolve potential solutions

• Search!!

9
Evolutionary Algorithms?

• Search Algorithms?
• Learning Algorithms?
• Function Optimization Algorithms?

They are fundamentally the same!!
10
Search
11
Search
12
Search
13
Notion of Search Space

• Real world problem
• Search space
• Abstraction -gt State Space
• Exploring the state space for given problem ?
  Search Algorithms

Dome
My friend
Search Space
14
Genetic Algorithm in search space
The one
15
GA in Pseudo code

• 0 START Create random population of n
  chromosomes
• 1 FITNESS Evaluate fitness f(x) of each
  chromosome in the population
• NEW POPULATION
• SELECTION Based on f(x)
• RECOMBINATION Cross-over chromosomes
• MUTATION Mutate chromosomes
• ACCEPTATION Reject or accept new one
• REPLACE Replace old with new population
  the new generation
• 4 TEST Test problem criteria
• 5 LOOP Continue step 1 4 until criteria
  is satisfied

16
Learning Algorithms

• Finding (through search) a suitable program,
  algorithm, function for a given problem

Learning Algorithm
Training Data (Experience)
Program
17
Learning Algorithms (function Optimizations)
Set of Hypothesis
The One??
18
Learning Algorithms (Digression)

• How do we know the found hypothesis, program,
  function, etc. are the one we are looking for?
• We dont know for sure
• Is there any mathematical way of telling how good
  hypothesis is?
• I.e., h(x) f(x) ?
• Computational Learning Theory can tell us this
• Valiant (1984)

19
Genetic Learning Classifier Systems

• General Learning Systems Russell Norvig

Performance Standard
Environment
Critic
Sensors
Feedback
changes
Learning Element
Performance Element
Knowledge
Learning Goals
Problem Generator
Actuators
20
Genetic Learning Classifier Systems

• Use population of rules
• Rules if ltconditiongtn then ltactiongtm

Environment
21
Rule-based Learning Classifier Systems

• Two approaches
• The Michigan Approach Riolo, Holland
• The Pitt Approach Smith
• Rules are represented with 0, 1,
• 0 absence of info/knowledge
• 1 existence of info/knowledge
• dont care condition.
• Rules are randomly generated
• Evolved using GA

22
Example (Wolf or Grandmother?)
23
Application to Speaker Identification
Communications (tend to be short and bursty)
24
Application to Speaker Identification

• Text-independent or Text-dependent?
• Closed-set or Open-set?
• The system must be robust to noise
• Traditional statistical methods works well for
  the closed-set problem
• Neural-net
• But the open-set problem is challenging
• Need the ability to introduce new speakers
  dynamically

Use of Rule-based Genetic Classifier System With
the Audio Group at AFRL (Grieco)
25
An Overview of Speaker Identification
Training Phase
26
GA Rule-based Speaker ID System
Temporal loop
Codebooks -gt Actions
codebook
Effector
ltcodebook1, codebook Ngt -gt ltLt. Ohgt
ltcodebook1, codebook Ngt -gt ltSmithgt
Genetic Algorithm (Learning)
27
Other Applications of GA Rule-based Systems

• Missile Evasion Problem Shultz, Grenfenstte,
  1995
• UAV or Autonomous Robots NRL, AFRL
• Simple rules.
• Learning to follow
• Learning to make teams
• Flocks around
• Whats different? Emergent Behavior, Interactions
  among agents
• Handwritten Character recognition Oh 1993
• Learning Classifier Web page http//www.cs.bath.a
  c.uk/amb/LCSWEB/

28
Multi-agent Systems and Game Theory

• Agents Selfishly Maximizing Utility
• Do the right thing but information is limited
• Goals of individual agents can conflict.
• Utility of an agent can depends on what others
  are doing too
• The action(s) that maximizes the Expected Value
  of the Performance Measures ( Utility) given the
  percept sequence to date.
• Being rational doesnt mean the smartest!

29
Multi-Agent Systems Research

• the study, construction, and application of
  multi-agent systems, involve several interacting,
  intelligent agents pursue some set of goals or
  perform some set of tasks
• Goal- and Task-Oriented Coordination
  Cooperation, Collaboration, and Competition

Micromotives and Macrobehavior Thomas
Schelling W.W. Norton and Co. Inc
30
Interactions among agents

• Things to consider
• Are all agents under the same administration
  roof?
• Are agents homogeneous or heterogeneous?
• Are agents rational? sacrificing? Cooperating?
  Collaborating?
• Hierarchical? Distributed?

31
Areas of interests

• Military Applications
• Unmanned-Robots
• Elimination of Mines (Coordination,
  Collaboration)
• Search and Rescue
• UAVs
• Autonomous Sensor Arrays (Sensor dusts)
• War games (Collaborating with 5th Generation war
  game group. (Gemelli, Bello, Wright)
• Etc.
• Industry
• High-performance Distributed Computing
• Replica Management
• Resource sharing and allocationGnutella, GRID
  computing
• Computer on-line games
• E-commerce, etc.

32
Multi-agent systems and other sciences
Social Sciences, Biology, etc.
Computer Scientists
33
We need a formalism

• Economic Game Theory
• Von Neumann and Morganstern Theory of Games and
  Economic Behavior
• My interest
• Promoting cooperation among rational agents
• Prisoners Dilemma Game (PD)

34
Examples of General PD

• Building a bridge participate or not
• Two competing companies setting the price for a
  product high price or low price
• Nuclear Arms race (Mutually Assured Destruction)
  build more bombs or not
• Server replication
• Coalition formation among rational agents
• War game
• Search and rescue by multiple parties
• Many many others

35
Classic Prisoners Dilemma Game

• Two accomplices in bank robberies caught by the
  Police Interrogated separately The police are
  bargaining
• Choice of a prisoner Confess (Defect) or Do not
  confess (Cooperate) to the other prisoner
• Dominant Strategy is to defect

Prisoner One
Prisoner Two
36
Payoff Matrix for 2-Person PD
C D
C R/R (3/3) S/T (0/5)
D T/S (5/0) P/P (1/1)
T gt R gt P gt S and 2R gt T S, Typically, T
5, R 3, P 1, and S 0
37
Tragedy of the Commons

• Commons is Resource Shared
• Agents are selfishly rational

38
In Repeated PD Games

• Defect is not an obvious choice for winning
  strategy
• Promoting cooperation is nevertheless difficult.
  But very important!!
• When there are many many agents
• Central coordination is too expensive
• Let them survive by interacting locally
• And globally exhibit desired emergent behavior

39
Rational Agents (No clustering incentive)
40
Rational Agents (Clustering incentive)
41
Some notable results

• Given number of agents and resource amount, is
  cooperation beneficial to me?
• Lower- upper-bound on amount of resources found,
  mathematically Oh 2000.
• Evolutionary Game can evolve strategies that can
  promote cooperation and survive. Riolo,
  Oliphant, Axelrod, etc

42
Conferences and Workshops

• Congress on Evolutionary Computations
• GECCO
• Problem Solving in Nature
• Hybrid Intelligent Systems
• Frontiers in Evolutionary Algorithms
• Applications of Game Theory and Artificial
  Intelligence Techniques on Distributed Computing
  and Internet-wide computing with PDCS
• Games and Emergent Behavior in Distributed
  Computing Environments with PPSN

43
Conclusions

• Historically, Evolutionary Computation came a
  long way.
• From single system learning/search/optimization
  algorithms
• To multi-agent systems
• The application areas are numerous.
• The time is for distributed systems (not only for
  computers, but robots, sensors, people, soldiers,
  etc.)
• Evolutionary game theory and Classifier System
  are very promising.