CS 4700: Foundations of Artificial Intelligence

1
CS 4700Foundations of Artificial Intelligence

• Carla P. Gomes
• gomes_at_cs.cornell.edu
• http//www.cs.cornell.edu/Courses/cs4700/2008fa/Mo
  dule
• Introduction
• (Reading RN Chapter 1)

2
Overview of this Lecture

• Course Administration
• What is Artificial Intelligence?
• Course Themes, Goals, and Syllabus

3
Course Administration
4

CS 4700Foundations of Artificial Intelligence
Lectures Monday, Wednesday, Friday 1115 1205
Location Phillips Hall, room 101 Lecturer
Prof. Gomes Office 5133 Upson Hall Phone 255
9189 Email gomes_at_cs.cornell.edu Administrative
Assistant Kelly Duby Kelly Duby
 ltkduby_at_cs.cornell.edugt      4105 Upson Hall,
255-0980 Web Site http//www.cs.cornell.edu/Cou
rses/cs4700/2008fa/
5
CS 4700Foundations of Artificial Intelligence
Head Teaching Assistants   Yunsong Guo guoys
_at_cs.cornell.edu Anton Morozov  amoroz
_at_cs.cornell.edu   Teaching Assistants   Clayton
Chang cc843 _at_cornell.edu Sean Sullivan sps27
_at_cornell.edu   Web Site http//www.cs.cornell.ed
u/Courses/cs4700/2008fa/
6
Office Hours

• Prof. Gomes
• Office 5133 Upson Hall
• I prefer to meet during my scheduled office
  hours, however,
• if you need to meet with me at a different time
  please
• schedule an appointment by email.
• TAs - TBA

Fridays 115p.m 215 p.m. (starting next week)
7
Grades
Midterm (15) Homework                     (45
) Participation                   (5) Final     
                          (35)
8
Homework

• Homework is very important. It is the best way
  for you to learn the
• material. You are encouraged to discuss the
  problems with your
• classmates, but all work handed in should be
  original, written by you in
• your own words.
• Assignments turned in late will drop 5 points for
  each period of 24
• hours for which the assignment is late. In
  addition, no assignments
• will be accepted after the solutions have been
  made available. No late
• homework will be accepted

9
Mailing List

• cs4700ta-l_at_lists.cs.cornell.edu.
• Contact us by using this mailing list.  The list
  is set to mail all
• the TA's and Prof. Gomes -- you will get the best
  response
• time by using this facility, and all the TA's
  will know the
• question you asked and the answers you receive.

10
CS 4701Practicum in Artificial Intelligence
(Optional)

• CS4701 Project  (Optional)
• CS4700 is a co-requisite for CS473. 
• There will be an organizational meeting in
  Hollister Hall room 110 on Tuesday, September 2nd
  at  335pm.
•  
• The main assignment for CS4701 is a course
  project. Students will work in groups (probably
  pairs).  A project proposal is required.  A
  separate project handout with project
  suggestions, details, and due dates regarding the
  project proposal,  and final project write-up
  will be made available from the course home page.
•  
• Grading CS4701
• 20 Project proposal
• 80 Final code, write-up, and presentation

11
Textbook
Artificial Intelligence A Modern Approach
(AIMA) (Second Edition) by Stuart Russell and
Peter Norvig

Artificial Intelligence A New Synthesis By
Nils Nilsson
12
Lecture notes and reading material
http//www.cs.cornell.edu/Courses/cs4700/2008fa/
Optional reading material
13
Welcome to this class!

• We will work together throughout this semester.
• Questions and suggestions are welcome anytime.
• E.g., if you find anything incorrect or unclear,
  send an email or talk to me.
• Any questions?

14
Overview of this Lecture

• Course Administration
• What is Artificial Intelligence?
• Course Themes, Goals, and Syllabus

15
AI Goals

• Ambitious goals
• understand intelligent behavior
• build intelligent agents

16
What is Intelligence?

• Intelligence
• the capacity to learn and solve problems
• (Webster dictionary)
• the ability to act rationally

17
What is AI?
Views of AI fall into four different
perspectives Thinking versus Acting Human
versus Rational

Human-like Intelligence
Ideal Intelligent/ Rationally
2.Thinking humanly 3.Thinking Rationally
1.Acting Humanly 4.Acting Rationally
Thought/ Reasoning
Behavior/ Actions
18
Different AI Perspectives
2. Systems that think like humans
3. Systems that think rationally

• Human Thinking
• Human Acting

• Rational Thinking
• Rational Acting

1. Systems that act like humans
4. Systems that act rationally
19
1. Acting Humanly
Human-like Intelligence
Ideal Intelligent/ Rationally
2. Thinking humanly 3. Thinking Rationally
1. Acting Humanly ?Turing Test 4. Acting Rationally
Thought/ Reasoning
Behavior/ Actions
20
In 1936, Alan Turing, a British mathematician,
showed that there exists a relatively simple
universal computing device that can perform any
computational process. Computers use such a
universal model.
Turing Machine (abstraction)
Turing also showed the limits of computation
some problems cannot be computed even with the
most powerful computer and even with unlimited
amount of time e.g., Halting problem.
21
Acting humanly Turing Test
Alan Turing

• Turing (1950) "Computing machinery and
  intelligence"
• "Can machines think? Instead, "Can machines
  behave intelligently?"
• Operational test for intelligent behavior the
  Imitation Game
• Predicted that by 2000, a machine might have a
  30 chance of fooling a lay person for 5 minutes
• Anticipated all major arguments against AI in
  following 50 years
• Suggested major components of AI

AI system passes if interrogator cannot tell
which one is the machine
(interaction via written questions)
22
Turing Test

• Three rooms contain a person, a computer, and an
  interrogator.
• The interrogator can communicate with the other
  two by teleprinter.
• The interrogator tries to determine which is the
  person and which is the machine.
• The machine tries to fool the interrogator into
  believing that it is the person.
• If the machine succeeds, then we conclude that
  the machine can think.

23
Acting humanly Turing Test
Turing test ? identified key research areas in AI

• Natural Language Processing to communicate with
  the machine
• Knowledge Representation to store and
  manipulate information
• Automated reasoning to use the stored
  information to answer questions and draw new
  conclusions
• Machine Learning to adapt to new circumstances
  and to detect and extrapolate patterns.

but does a machine need to act humanly to be
considered intelligent?
24
Some Famous Imitation Games

• 1960s ELIZA Joseph Weizenbaum
• Rogerian psychotherapist
• 1990s ALICE
• Loebner prize
• win 100,000 if you pass the test

25
ELIZA impersonating a Rogerian psychotherapist
1960s ELIZA Joseph Weizenbaum
http//www-ai.ijs.si/eliza/eliza.html
Eliza a friend you could never have before
Eliza Hello. I am ELIZA. How can I help you?
You Well I feel sad Eliza Do you often feel
sad? You not very often, but it's becoming more
common Eliza Please go on.
26
Chat Bot ALICE(AI Foundation)

• Human hi my name is CarlaALICE How are you
  doing, Carla.Human what is your name?ALICE My
  name is ALICE.Human what is your full
  name?ALICE ALICE Artificial Linguistic
  Internet Computer Entity.Human when were you
  born?ALICE I was connected to the net on
  November 23, 1995.
• Human Have you passed the Loebner Prize?ALICE
  The Loebner Prize is an annual Turing Test
  sponsored by New York philanthropist Hugh
  Loebner. Dr. Wallace and ALICE won it in 2000.
  The ALICE program was ranked "most human
  computer" by the panel of judges.
• Human who is Dr. Wallace?ALICE He is a famous
  computer scientist, the author of ALICE, and a
  graduate of Carnegie Mellon.
• Human Where does Dr. Wallace work?ALICE he
  works from his home in San Francisco.

27
The Loebner contest

• A modern version of the Turing Test, held
  annually, with a 100,000 cash prize.
• Hugh Loebner was once director of UMBCs Academic
  Computing Services (née UCS)
• http//www.loebner.net/Prizef/loebner-prize.html
• Restricted topic (removed in 1995) and limited
  time.
• Participants include a set of humans and a set of
  computers and a set of judges.
• Scoring
• Rank from least human to most human.
• Highest median rank wins 2000.
• If better than a human, win 100,000. (Nobody
  yet)

28
2. Thinking Humanly
Human-like Intelligence
Ideal Intelligent/ Rationally
2. Thinking humanly ? Cognitive Modeling Thinking Rationally
Acting Humanly ?Turing Test Acting Rationally
Thought/ Reasoning
Behavior/ Actions
29
Thinking humanly modeling cognitive processes

• Requires scientific theories of internal
  activities of the brain
• 1) Cognitive Science (top-down) computer models
  experimental techniques from psychology
• ? Predicting and testing behavior of human
  subjects
• 2) Cognitive Neuroscience (bottom-up)
• ? Direct identification from neurological data

Both approaches are now distinct from AI
1960s "cognitive revolution" information-processi
ng psychology
30
3. Thinking Rationally
Human-like Intelligence
Ideal Intelligent/ Rationally
Thinking humanly ? Cognitive Modeling 3. Thinking Rationally ?Laws of Thought
Acting Humanly ?Turing Test Acting Rationally
Thought/ Reasoning
Behavior/ Actions
31
Thinking rationally formalizing the "laws of
thought

• Logic ? Making the right inferences! Several
  Greek schools developed various forms of logic
  notation and rules of derivation for thoughts
• Aristotle what are correct arguments/thought
  processes? (characterization of right
  thinking)
• Socrates is a man
• All men are mortal
• --------------------------
• Therefore Socrates is mortal

• More contemporary logicians (e.g. Boole, Frege,
  Tarski) ?
• Direct line through mathematics and philosophy to
  modern AI

• Limitations
• Not all intelligent behavior is mediated by
  logical deliberation
• What is the purpose of thinking? What thoughts
  should I have?
  

32
4. Acting Rationally
Human-like Intelligence
Ideal Intelligent/ Rationally
Thinking humanly ? Cognitive Modeling 3. Thinking Rationally ?Laws of Thought
Acting Humanly ?Turing Test 4. Acting Rationally
Thought/ Reasoning
Behavior/ Actions
33
Acting rationally rational agent

• Rational behavior doing the right thing
• The right thing that which is expected to
  maximize goal achievement, given the available
  information
  
• Doesn't necessarily involve thinking e.g.,
  blinking reflex but thinking should be in the
  service of rational action
  

34
Rational agents

• An agent is an entity that perceives and acts
• This course is about designing rational agents
• Abstractly, an agent is a function from percept
  histories to actions
  
• f P ? A
• For any given class of environments and tasks, we
  seek the agent (or class of agents) with the best
  performance
  
• Caveat computational limitations make perfect
  rationality unachievable
• ? design best program for given machine resources

35
Building Intelligent Machines

• I Building exact models of human cognition
• view from psychology and cognitive science
• II Developing methods to match or exceed human
• performance in certain domains, possibly by
• very different means ? e.g., Deep Blue

36
Methodology of AI

• Theoretical aspects
• Mathematical formalizations, properties,
  algorithms
• Engineering aspects
• The act of building (useful) machines
• Empirical science
• Experiments

37
What's involved in Intelligence?

• A) Ability to interact with the real world
• to perceive, understand, and act
• speech recognition and understanding
• image understanding (computer vision)
• B) Reasoning and Planning
• modelling the external world
• problem solving, planning, and decision making
• ability to deal with unexpected problems,
  uncertainties
• C) Learning and Adaptation
• We are continuously learning and adapting.
• We want systems that adapt to us!

38

• Historic Perspective

39
AI Leverages from different disciplines

• Philosophy
• e.g., foundational issues in logic, methods of
  reasoning,
• mind as physical system, foundations of
  learning,
• language, rationality
• Computer science and engineering
• e.g., complexity theory, algorithms, logic and
  inference,
• programming languages, and system building
  (hardware
• and software).
• Mathematics and physics
• e.g., probability theory, statistical modeling,
  continuous mathematics,
• Markov models, statistical physics, and complex
  systems.
• and others, e.g., cognitive science,
  neuroscience, economics, psychology, linguistics,

40
AI More direct Influence

• Obtaining an understanding of the human mind is
  one of the
• final frontiers of modern science.
• George Boole, Gottlob Frege, and Alfred Tarski
• formalizing the laws of human thought
• Alan Turing, John von Neumann, and Claude Shannon
• thinking as computation
• Direct Founders
• John McCarthy, Marvin Minsky, Herbert Simon, and
  Allen Newell
• the start of the field of AI (1959)

41
History of AIMilestonesThe gestation of AI
1943-1956

• 1943 McCulloch and Pitts
• McCulloch and Pittss model of artificial neurons
• Minskys 40-neuron network
• 1950 Turings Computing machinery and
  intelligence
• 1950s Early AI programs, including Samuels
  checkers program, Newell and Simons Logic
  theorist
• 1956 Dartmouth meeting Birth of Artificial
  Intelligence
• A 2-month Dartmouth workshop of 10 attendees
  the name of AI
• Newell and Simons Logic Theorist
• Do you think AI is a god name?

42
Perceptrons Early neural nets
More about Neural Nets later in the course
43
History of AILook, Ma, no hands
!(1952-1969)Early enthusiasm, great
expectations

• 1957 Herb Simon
• It is not my aim to surprise or shock you but
  the simplest way I can summarize is to say that
  there are now in the world machines that think,
  that learn and that create. Moreover their
  ability to do these things is going to increase
  rapidly until in the visible future the range
  of problems that they can handle will be
  coextensive with the range to which human mind
  has been applied.
• 1958 John McCarthys LISP
• 1965 J.A. Robinson invents the resolution
  principle, basis for automated theorem
• Intelligent reasoning in Microworlds (such as
  Blocks world)

44
The Blocks world
45
History of AIA dose of reality (1966-1978)

• 1965 Weizenbaums ELIZA
• Difficulties in automated translation ( try
  http//babelfish.yahoo.com/)
• Syntax is not enough
• the spirit is willing but the flesh is weak
• the vodka is good but the meat is rotten
• Limitations of Perceptrons discovered
• ? can only represent linearly separable
  functions
• Neural network research almost disappears
• ???NP-Completeness (Cook 72)
• Intractability of the problems attempted by AI,
• Worst- case result.

46
History of AIKnowledge based systems (1969-79)

• Intelligence requires knowledge - Knowledge based
  systems as opposed to weak methods
  (general-purpose search methods)
• ? Expert Systems,
• E.g.
• Mycin diagnose blood infections
• R1 configuring computer systems

47
History of AIAI becomes industry (1980-88)

• Expert systems
• Lisp-machines
• Return of Neural Nets

? End of 80s limitations of expert systems
became clear, even though they have been quite
successful in certain domains.
48
History of AI2000-AI is Alive and Kicking

• Current work on intelligent agents
• Emphasis on integration of reasoning (search and
  inference as well as probabilistic reasoning),
  knowledge representation, and learning
  techniques
• AI as a science Combining theoretical and
  empirical analysis
• ?Mathematical sophistication of AI techniques

AAAI08
Key challenge building flexible and scalable AI
systems in the Open World .
A better understanding of the problems and
their complexity properties, combined with
increased mathematical sophistication, has led
to workable research agendas and robust methods
RN.
49
A few recent examples
AI Achievements
50
1996 - EQP Robbins Algebras are all boolean
A mathematical conjecture (Robbins conjecture)
unsolved for 60 years!
The Robbins problem was to determine whether one
particular set of rules is powerful enough to
capture all of the laws of Boolean algebra. One
way to state the Robbins problem in mathematical
terms is Can the equation not(not(P))P be
derived from the following three equations? 1
P or Q Q or P, 2 (P or Q) or R P or (Q or
R), 3 not(not(P or Q) or not(P or not(Q)))
P.
First creative mathematical proof by computer
unlike brute-force based proofs such as the
4-color theorem.
An Argonne lab program has come up with a major
mathematical proof that would have been called
creative if a human had thought of it.
New
York Times, December, 1996
http//www-unix.mcs.anl.gov/mccune/papers/robbins
/
51
Microsoft Office97 Answer Wizard

• Diagnosis reasoning using Bayesian Models
• Restricted NLP

52
1997 Deep Blue beats the World Chess Champion
vs.
I could feel human-level intelligence across the
room -Gary Kasparov, World Chess
Champion (human)
53
Deep Blue vs. Kasparov
IBM Stock price skyrocketed on the day Deep Blue
beat Kasparov
Game 1 5/3/97 Kasparov wins   Game 2
5/4/97Deep Blue wins     Game 3
5/6/97Draw      Game 4 5/7/97Draw     
Game 5 5/10/97 Draw      Game 6
5/11/97Deep Blue wins     
I felt a new kind of Intelligence ( across the
board from him) Kasparov 1997 The value of IBMs
stock Increased by 18 Billion!
One of the most famous modern computers, Deep
Blue, which defeated Gary Kasparov at chess.
54
1999 Remote Agent takes Deep Space 1 on a
galactic ride
For two days in May, 1999, an AI Program called
Remote Agent autonomously ran Deep Space 1 (some
60,000,000 miles from earth)
55
Remote Agent1999 Winner of NASA's Software of
the Year Award
It's one small step in the history of space
flight. But it was one giant leap for
computer-kind, with a state of the art artificial
intelligence system being given primary command
of a spacecraft. Known as Remote Agent, the
software operated NASA's Deep Space 1 spacecraft
and its futuristic ion engine during two
experiments that started on Monday, May 17,
1999. For two days Remote Agent ran on the
on-board computer of Deep Space 1, more than
60,000,000 miles (96,500,000 kilometers) from
Earth. The tests were a step toward robotic
explorers of the 21st century that are less
costly, more capable and more independent from
ground control.  
http//ic.arc.nasa.gov/projects/remote-agent/index
.html
56
Proverb 1999 Solving Crossword Puzzles as
Probabilistic Constraint Satisfaction

• Proverb solves
• crossword puzzles
• better than most humans

Michael Littman et a. 99
57
2000 SCIFINANCE synthesizes programs for
financial modeling

• Develop pricing models for complex derivative
  structures
• Involves the solution of a set of PDEs (partial
  differential equations)
• Integration of object-oriented design, symbolic
  algebra, and plan-based scheduling

58
Robocup _at_ Cornell1999
http//www.mae.cornell.edu/raff/MultiAgentSystems/
MultiAgentSystems.htm
Raff Dandrea
59
From Robocup to Warehouse Automation
First user of system
Raff DAndrea
60
Machine learning successes
Source R. Greiner
61
Machine learning successes
Source R. Greiner
62
Machine learning successes
Source R. Greiner
63
2005 Autonomous ControlDARPA GRAND CHALLENGE
October 9, 2005 Stanley and the Stanford
RacingTeam were awarded 2 million dollars for
being the first team to complete the 132 mile
DARPA Grand Challenge course (Mojave Desert).
Stanley finished in just under 6 hours 54
minutes and averaged over 19 miles per hours on
the course.

64
(No Transcript)
65
A algorithm
66
2007 Darpa Urban ChallengeWinner CMU Tartan
Racing's Boss

• http//www.tartanracing.org/blog/index.html26

67
The DARPA Urban Challenge is being held at the
former George Air Force Base. The old base
buildings are abandoned now and the Marines use
the area to train for urban missions.
68
Where can you learn more about AI?
69
Main annual AI conference AAAI Association
for Advancement of AI
70
Association for Advancement of Artificial
Intelligence(AAAI)AI Topics
http//www.aaai.org/AITopics/pmwiki/pmwiki.php/AIT
opics/HomePage
71
Goals for this course
72
Setting expectations for this course

• Are we going to build real systems and robots?

NO!!!
Goal Introduce the theoretical and
computational techniques that serve as a
foundation for the study of artificial
intelligence (AI).
73
Syllabus

• Structure of intelligent agents and environments.
  
• Problem solving by search principles of search,
  uninformed (blind) search, informed
  (heuristic) search, and local search.
• Constraint satisfaction problems definition,
  search and inference, and study of structure.
• Adversarial search games, optimal strategies,
  imperfect, real-time decisions.
• Logical agents propositional and first order
  logic, knowledge bases and inference.
• Uncertainty and probabilistic reasoning
  probability concepts, Bayesian networks, 
  probabilistic reasoning over time, and decision
  making
• Learning inductive learning, concept formation,
  decision tree learning, statistical approaches,
  neural networks, reinforcement learning

74
Notes

• The syllabus is quite ambitious some of the
  topics may only be covered
• briefly, depending on time.
• Detailed reading information (chapters and
  sections of RN) will be
• provided in the lectures notes and homework
  assignments.
• This is not a machine learning course we will
  only cover some
• introductory material learning topics ? if you
  are looking for a machine
• learning course, here is a specialized machine
  learning course offered this
• fall
• CS4782 - Probabilistic Graphical Models.

75
Summary

• Artificial Intelligence and characteristics of
  intelligent systems.
• Brief history of AI
• Examples of AI achievements
• Computers are getting smarter !!!

Reading Chapter 1 Russell Norvig
76

• The End ?!