CSINFO 372: Explorations in Artificial Intelligence

1
CS-INFO 372Explorations in Artificial
Intelligence

• Prof. Carla P. Gomes
• gomes_at_cs.cornell.edu
• Introduction
• http//www.cs.cornell.edu/courses/cs372/2008sp

2
Overview of this Lecture

• Course Administration
• What is Artificial Intelligence?
• Course Themes, Goals, and Syllabus

3
Course Administration
4
INFO372 Explorations in Artificial
Intelligence Course Administration
Lectures Tuesday and Thursday - 1010 -
1125 Location Phillips Hall, room
307 Lecturer Prof. Gomes Office 5133 Upson
Hall Phone [phone] Email gomes_at_cs.cornell.edu
Administrative Assistant Beth Howard
(bhoward_at_cs.cornell.edu)     5136 Upson Hall,
255-4188 TAs Robert Xiao rkx2_at_cornell.edu
Yunsong Guo ltguoys_at_cs.cornell.edugt Web Site
http//www.cs.cornell.edu/courses/cs372/2008sp
5
Office Hours

• TAs
• Robert Xiao rkx2_at_cornell.edu TBA
• Yunsong Guo guoys_at_cs.cornell.edu TBA
• Prof. Gomes
• Office 5133 Upson Hall
• If you need to meet with me at a different time
  please
• schedule an appointment by email.

Wednesdays 1200 100 p.m.
6
Grades
Midterm (30) Homework                     (25
) Participation                   (5) Final     
                          (40)
Homework is very important. It is the best way
for you to learn the material. You are encouraged
to discuss the problems with your classmates, but
all work handed in should be original, written by
you in your own words. No late homework will be
accepted
7
Textbook
Artificial Intelligence A Modern Approach
(AIMA) (Second Edition) by Stuart Russell and
Peter Norvig

Artificial Intelligence A New Synthesis By
Nils Nilsson
Principles of Constraint Programming By
Krzysztof Apt
Linear Programming by Vasek Chvatal
8
Overview of this Lecture

• Course Administration
• What is Artificial Intelligence?
• Course Themes, Goals, and Syllabus

9
What is Intelligence?Historical Perspective of
AIState-of-the-art and Challenges
What is Artificial Intelligence (AI)?
10
What is AI?

• Ambitious goals
• understand intelligent behavior
• build intelligent agents

11
What is Intelligence?

• Intelligence
• the capacity to learn and solve problems
• (Webster dictionary)
• the ability to act rationally
• Artificial Intelligence
• build and understand intelligent entities
• synergy between
• philosophy, psychology, and cognitive science
• computer science and engineering
• mathematics and physics

12
AI Leverages from Different Disciplines

• Philosophy
• e.g., foundational issues in logic, methods of
  reasoning,
• mind as physical system, foundations of
  learning,
• language, rationality
• Computer science and engineering
• e.g., complexity theory, algorithms, logic and
  inference,
• programming languages, and system building
  (hardware
• and software).
• Mathematics and physics
• e.g., statistical modeling, continuous
  mathematics, Markov
• models, statistical physics, and complex systems.
• and others, e.g., cognitive science,
  neuroscience, economics, psychology, linguistics,
  statistics

13
AIHistorical Perspective

• Obtaining an understanding of the human mind is
  one of the
• final frontiers of modern science.
• Founders
• George Boole (1779-1848), Gottlob Frege
  (1848-1925), and Alfred Tarski (1902-1983)
• formalizing the laws of human thought
• Alan Turing (1912-1954) , John von Neumann
  (1903-1957), Claude Shannon (1916-2001)
• thinking as computation
• John McCarthy (1927- ), Marvin Minsky (1927 - ) ,
  Herbert Simon (1916-2001), and Allen Newell
  (1927-1992)
• the start of the field of AI (1959)

14
In 1936, Alan Turing, a British mathematician,
showed that there exists a relatively simple
universal computing device that can perform any
computational process. Computers use such a
universal model.
Turing Machine (abstraction)
Turing also showed the limits of computation
some problems cannot be computed even with the
most powerful computer and even with unlimited
amount of time e.g., Halting problem.
15
Halting Problem

• The halting problem is a decision problem which
  can be stated as follows
• Given a description of a program and a finite
  input, decide whether the program finishes
  running or will run forever, given that input.
• Alan Turing proved in 1936 that a general
  algorithm to solve the halting problem for all
  possible program-input pairs cannot exist. We say
  that the halting problem is undecidable.

16
Acting humanly Turing Test
Alan Turing

• Turing (1950) "Computing machinery and
  intelligence"
• "Can machines think?" ? "Can machines behave
  intelligently?"
• Operational test for intelligent behavior the
  Imitation Game
• Predicted that by 2000, a machine might have a
  30 chance of fooling a lay person for 5 minutes
• Anticipated all major arguments against AI in
  following 50 years
• Suggested major components of AI knowledge,
  reasoning, natural language understanding,
  learning
  

AI system passes if interrogator cannot tell
which one is the machine
17
Some Famous Imitation Games

• 1960s ELIZA Joseph Weizenbaum
• Rogerian psychotherapist
• 1990s ALICE
• Loebner prize
• win 100,000 if you pass the test

18
ELIZA impersonating a Rogerian psychotherapist
1960s ELIZA Joseph Weizenbaum
http//www-ai.ijs.si/eliza/eliza.html
Eliza a friend you could never have before
Eliza Hello. I am ELIZA. How can I help you?
You Well I feel sad Eliza Do you often feel
sad? You not very often, but it's becoming more
common Eliza Please go on.
19
Chat Bot ALICE(AI Foundation)

• Human hi my name is CarlaALICE How are you
  doing, Carla.Human what is your name?ALICE My
  name is ALICE.Human what is your full
  name?ALICE ALICE Artificial Linguistic
  Internet Computer Entity.Human when were you
  born?ALICE I was connected to the net on
  November 23, 1995.
• Human Have you passed the Loebner Prize?ALICE
  The Loebner Prize is an annual Turing Test
  sponsored by New York philanthropist Hugh
  Loebner. Dr. Wallace and ALICE won it in 2000.
  The ALICE program was ranked "most human
  computer" by the panel of judges.
• Human who is Dr. Wallace?ALICE He is a famous
  computer scientist, the author of ALICE, and a
  graduate of Carnegie Mellon.
• Human Where does Dr. Wallace work?ALICE he
  works from his home in San Francisco.

20
Acting humanly Turing Test
Turing test ? identified key research areas in AI

• Natural Language Processing to communicate with
  the machine
• Knowledge Representation to store and
  manipulate information
• Automated reasoning to use the stored
  information to answer questions and draw new
  conclusions
• Machine Learning to adapt to new circumstances
  and to detect and extrapolate patterns.

but does a machine need to act humanly to be
considered intelligent?
21
Other perspectives on Intelligence

• Thinking humanly cognitive modeling
• Requires scientific theories of internal
  activities of the brain How to validate?
• 1) Cognitive Science (top-down) ? Predicting
  and testing behavior of human subjects
• computer models experimental techniques
  from psychology
• 2) Cognitive Neuroscience (bottom-up) ?
  Direct identification from neurological data
• Thinking rationally "laws of thought
• Logic ? Making the right inferences! Several
  Greek schools developed various forms of logic
  notation and rules of derivation for thoughts
• Aristotle what are correct arguments/thought
  processes? (characterization of right
  thinking)
• Socrates is a man
• All men are mortal
• --------------------------
• Therefore Socrates is mortal
• More contemporary logicians (e.g. Boole, Frege,
  Tarski) ? Direct line through mathematics and
  philosophy to modern AI
• Acting rationally rational agent
• Rational behavior doing the right thing that
  which is expected to maximize goal achievement,
  given the available information Doesn't
  necessarily involve thinking e.g., blinking
  reflex but thinking should be in the service
  of rational action

Always doing the right thing ? sometimes not
feasible in complex environments ? Computational
demands can be too high!
22
Different Approaches

• I Building exact models of human cognition
• view from psychology and cognitive science
• II Developing methods to match or exceed human
• performance in certain domains, possibly by
• very different means ? e.g., Deep Blue

23
Man vs. Machiens The Hardware

• The brain
• a neuron, or nerve cell, is the basic information
  processing unit (1011 )
• many more synapses (1014) connect the neurons
• cycle time 10(-3) seconds (1 millisecond)
• How complex can we make computers?
• 108 or more transistors per CPU
• supercomputer hundreds of CPUs, 1010 bits of
  RAM
• cycle times order of 10(-9) seconds (1
  nanosecond)

24
Computer vs. Brain
25
(No Transcript)
26

• Conclusion
• In near future we can have computers with as many
  processing elements as our brain, but
• far fewer interconnections (wires or
  synapses)
• much faster updates.
• Fundamentally different hardware may require
  fundamentally different algorithms!
• Very much an open question.

27
What is AI?
Human-like Intelligence
Ideal Intelligent/ Rationally
Thought/ Reasoning
Behavior/ Actions
28
What's involved in Intelligence?

• A) Ability to interact with the real world
• to perceive, understand, and act
• speech recognition and understanding
• image understanding (computer vision)
• B) Reasoning and Planning
• modelling the external world
• problem solving, planning, and decision making
• ability to deal with unexpected problems,
  uncertainties
• C) Learning and Adaptation
• We are continuously learning and adapting.
• We want systems that adapt to us!

29
A few examples
State-of-the-art Reasoning and Planning in AI
30
1997 Deep Blue beats the World Chess Champion
vs.
I could feel human-level intelligence across the
room -Gary Kasparov, World Chess
Champion (human)
31
Deep Blue vs. Kasparov
Game 1 5/3/97 Kasparov wins   Game 2
5/4/97Deep Blue wins     Game 3
5/6/97Draw      Game 4 5/7/97Draw     
Game 5 5/10/97 Draw      Game 6
5/11/97Deep Blue wins     
I felt a new kind of Intelligence ( across the
board from him) Kasparov 1997 The value of IBMs
stock Increased by 18 Billion!
One of the most famous modern computers, Deep
Blue, which defeated Gary Kasparov at chess.
32
How Intelligent is Deep Blue?

• Saying Deep Blue doesn't really think about chess
  is like saying an airplane doesn't really fly
  because it doesn't flap its wings.
• - Drew McDermott

33
On Game 2

• (Game 2 - Deep Blue took an early lead.
• Kasparov resigned, but it turned out he could
• have forced a draw by perpetual check.)
• This was real chess. This was a game any human
• grandmaster would have been proud of.
• Joel Benjamin grandmaster, member Deep Blue team

34
Kasparov on Deep Blue

• 1996 Kasparov Beats Deep Blue
• I could feel --- I could smell --- a new kind
• of intelligence across the table.
• 1997 Deep Blue Beats Kasparov
• Deep Blue hasn't proven anything.

35
Game Tree Search

• How to search a game tree was independently
  invented by Shannon (1950) and Turing (1951).
• Technique called MiniMax search.
• Evaluation function combines material position.

36
Game Tree Search
37
History of Search Innovations

• Shannon, Turing Minimax search 1950
• Kotok/McCarthy Alpha-beta pruning 1966
• MacHack Transposition tables 1967
• Chess 3.0 Iterative-deepening 1975
• Belle Special hardware 1978
• Cray Blitz Parallel search 1983
• Hitech Parallel evaluation 1985
• Deep Blue All of the above 1997

38
Transposition Tables

• Introduced by Greenblat's Mac Hack (1966)
• Basic idea caching
• once a board is evaluated, save it in a hash
  table (data structure that associates keys with
  values), avoid re-evaluating.
• called transposition tables, because different
  orderings (transpositions) of the same set of
  moves can lead to the same board.
• Form of root learning (memorization)
• Dont repeat blunders ? cant beat the computer
  twice in a row using same moves

Deep Blue --- huge transposition tables
(100,000,000), must be carefully managed.
39
Positions with Smart Pruning

• Search Depth Positions
• 2 60
• 4 2,000
• 6 60,000
• 8 2,000,000
• 10 (lt1 second DB) 60,000,000
• 12 2,000,000,000
• 14 (5 minutes DB) 60,000,000,000
• 16 2,000,000,000,000

How many lines of play does a grand master
consider?
Around 5 to 7
40
Special-Purpose and Parallel Hardware

• Belle (Thompson 1978)
• Cray Blitz (1993)
• Hitech (1985)
• Deep Blue (1987-1996)
• Parallel evaluation allows more complicated
  evaluation functions
• Hardest part coordinating parallel search
• Deep Blue never quite plays the same game,
  because of noise in its hardware!

41
Deep Blue

• Hardware
• 32 general processors
• 220 VSLI chess chips
• Overall 200,000,000 positions per second
• 5 minutes depth 14
• Selective extensions - search deeper at unstable
  positions
• down to depth 25 !

42
(No Transcript)
43
Tactics into Strategy

• As Deep Blue goes deeper and deeper into a
  position, it displays elements of strategic
  understanding. Somewhere out there mere tactics
  translate into strategy. This is the closest
  thing I've ever seen to computer intelligence.
  It's a very weird form of intelligence, but you
  can feel it. It feels like thinking.
• Frederick Friedel (grandmaster), Newsday, May 9,
  1997

44
1996 - EQP Robbins Algebras are all boolean
A mathematical conjecture (Robbins conjecture)
unsolved for decades
The Robbins problem was to determine whether one
particular set of rules is powerful enough to
capture all of the laws of Boolean algebra. One
way to state the Robbins problem in mathematical
terms is Can the equation not(not(P))P be
derived from the following three equations? 1
P or Q Q or P, 2 (P or Q) or R P or (Q or
R), 3 not(not(P or Q) or not(P or not(Q)))
P.
An Argonne lab program has come up with a major
mathematical proof that would have been called
creative if a human had thought of it.
New
York Times, December, 1996
http//www-unix.mcs.anl.gov/mccune/papers/robbins
/
45
1999 Remote Agent takes Deep Space 1 on a
galactic ride
For two days in May, 1999, an AI Program called
Remote Agent autonomously ran Deep Space 1 (some
60,000,000 miles from earth)
46
Remote Agent1999 Winner of NASA's Software of
the Year Award
It's one small step in the history of space
flight. But it was one giant leap for
computer-kind, with a state of the art artificial
intelligence system being given primary command
of a spacecraft. Known as Remote Agent, the
software operated NASA's Deep Space 1 spacecraft
and its futuristic ion engine during two
experiments that started on Monday, May 17,
1999. For two days Remote Agent ran on the
on-board computer of Deep Space 1, more than
60,000,000 miles (96,500,000 kilometers) from
Earth. The tests were a step toward robotic
explorers of the 21st century that are less
costly, more capable and more independent from
ground control.  
http//ic.arc.nasa.gov/projects/remote-agent/index
.html
47
2000 SCIFINANCE synthesizes programs for
financial modeling

• Develop pricing models for complex derivative
  structures
• Involves the solution of a set of PDEs (partial
  differential equations)
• Integration of object-oriented design, symbolic
  algebra, and plan-based scheduling

48
Proverb 1999 Solving Crossword Puzzles as
Probabilistic Constraint Satisfaction

• Proverb solves
• crossword puzzles
• better than most humans

Michael Littman et a. 99
49
Robocup _at_ Cornell199
http//www.mae.cornell.edu/raff/MultiAgentSystems/
MultiAgentSystems.htm
50
2003 Robocup Italy
51
2005 Autonomous ControlDARPA GRAND CHALLENGE
October 9, 2005 Stanley and the Stanford
RacingTeam were awarded 2 million dollars for
being the first team to complete the 132 mile
DARPA Grand Challenge course (Mojave Desert).
Stanley finished in just under 6 hours 54
minutes and averaged over 19 miles per hours on
the course.
52
(No Transcript)
53
DARPA - Urban Challenge (2007)

• The Urban Challenge features autonomous ground
  vehicles maneuvering in a mock city environment,
  executing simulated military supply missions
  while merging into moving traffic, navigating
  traffic circles, negotiating busy intersections,
  and avoiding obstacles.

54
(No Transcript)
55
(No Transcript)
56
Many Other Applications

• Financial planning
• Marketing
• E-business
• Telecommunications
• Manufacturing
• Operations Management
• Production Planning
• Transportation Planning
• System Design
• Health Care

57
Course Themes, Goals, and Syllabus
58
Goals of INFO 372

• Focus of Info 372 Problem Solving
• Introduce the students to a range of
  computational modeling
• approaches and solution strategies using examples
  from AI and
• Information Science.
• Formalisms
• Logical representations
• Constraint-based languages,
• Mathematical programming
• Multi-agent formalisms (including adversarial
  games)
• Solution strategies
• Logical inference
• General complete backtrack search
• Local search
• Dynamic Programming

59
Goals of INFO 372

• Special models
• Satisfiability (SAT) Maximum SAT Horn
• Constraint Satisfaction Binary Constraint
  Satisfaction
• Mixed Integer Programming, Linear Programming
  and
• Network Flow Models

Themes Expressiveness and efficiency tradeoffs
of the various representation formalisms
?Students learn about the tradeoffs in
modeling choices. Concrete examples to move
from one representation modeling formalism to
another formalism
60
Summary

• Discussed Artificial Intelligence and
  characteristics of intelligent systems.
• Gave series of example systems, involving e.g.
• game playing, automated reasoning, and
  planning.
• Computers are getting smarter !!!

Suggested Reading Chapter 1 Russell Norvig
61

• The END