Heuristics and Prolog for Artificial Intelligence Applications

1
Heuristics and Prolog for Artificial Intelligence
Applications

• A course of 22 lectures
• Lecturer
• Professor Bruce Batchelor, DSc

2
About the lecturer - ask some questions

• Allowed answers
• YES
• NO
• NO ANSWER - rude or inappropriate question
• Time allowed
• 5 minutes

3
Rules for calculating the answer

• Rule for calculating the answers
• Find the 3rd word in the question
• Find the second letter of that word
• If this letter is in range A-K answer is YES
• If this letter is in range L-Z answer is NO

4
Ask me some questions about myself

• Rule for calculating the answers
• Find the 3rd word in the question
• Find the second letter of that word
• If this letter is in range A-K answer is YES
• If this letter is in range L-Z answer is NO

5
Ask me some questions about myself

• Rule for calculating the answers
• Find the 3rd word in the question
• Find the second letter of that word
• If this letter is in range A-K answer is YES
• If this letter is in range L-Z answer is NO

6
Ask me some questions about myself

• Rule for calculating the answers
• Find the 3rd word in the question
• Find the second letter of that word
• If this letter is in range A-K answer is YES
• If this letter is in range L-Z answer is NO

7
Ask me some questions about myself

• Rule for calculating the answers
• Find the 3rd word in the question
• Find the second letter of that word
• If this letter is in range A-K answer is YES
• If this letter is in range L-Z answer is NO

8
Ask me some questions about myself

• Rule for calculating the answers
• Find the 3rd word in the question
• Find the second letter of that word
• If this letter is in range A-K answer is YES
• If this letter is in range L-Z answer is NO
• If there are fewer than three letters in the
  third word, apply the above rules to the fourth
  word instead.

9
Result

• Rule for calculating the answers
• You know nothing at all about me and

10
Result

• Rule for calculating the answers
• You know nothing at all about me and any
  naughtiness is your mind, not mine!

11
IGNORE EVERYTHING THAT YOU THINK YOU HAVE LEARNED
ABOUT ME SO FAR
12
WARNINGRepeating any of the erroneous
information heard so far about Prof. Bruce
Batcheloris slander and actionable in law.
13
WARNINGRepeating any of the erroneous
information heard so far about Prof. Bruce
Batcheloris slander and actionable in law.The
truth will berevealed soon!
14
What is the point?

• The lecturer gave answers according to
  predefined rules.
• The answers are meaningless
• The course of the conversation was determined by
  the questioners, not the lecturer, and could not
  be predicted in advance.
• The audience could not identify whether or not
  the lecturer was telling the truth.
• This is roughly analogous to the Turing Test

15
Turing Test

• More about this later

16
Heuristics Prolog for Artificial Intelligence
Applications

• Lecturer Professor Bruce Batchelor,
• DSc, CEng, FIEE, FSPIE, FSME.
• Note Prof. Batchelor works part time
• Office South 2.18(B)
• Tel [phone] (Internal 74390)
• Email bruce.batchelor_at_cs.cf.ac.uk
• URL http//bruce.cs.cf.ac.uk/bruce/index.html

17
How the Course is Taught

• Lectures, 22
• Two lectures/week
• Tutorials
• Five one-hour sessions
• Laboratory (Prolog programming)
• Two one-hour sessions
• Teaching assistants
• Mr. Philip Smart

18
Heuristics for a Happy Class . and Lecturer

• No food or drink to be consumed.
• No mobile phones.
• No cuddling.
• Late arrivals may be refused entry to lectures
• Keep talking to an absolute minimum.
• Please feel free to ask questions as necessary
  during lectures.

19
Course Assessment

• Course-work
• Test 1 10
• Test 2 15
• Examination
• Weight 75
• 2 hours duration
• Answer 3 questions
• Total of 5 questions

20
Invitation to come and talk

• Please come to my office to discuss any aspect of
  this course - do not wait to be invited again!
• You are always welcome to come to my office, if
  you have any sort of personal problem. All
  conversations are in confidence.
• However, you should e-mail first, as I work part
  time.

21
What is Intelligence?

• Intelligence is what we use when we dont know
  what to do.

22
Intelligence is Difficultto Define

• Go to the Library and find at least 3 definitions
  of Intelligence. (Try the Oxford English
  Dictionary and Encyclopaedia Brittanica)
• So why are you insulted if somebody says that you
  are not very intelligent?
• People can recognise intelligence in other people
  and animals

23
Features of Intelligence

• Most people agree that intelligence requires an
  ability to
• Perform complex tasks
• Recognise complex patterns
• Solve unseen problems
• Learn from experience
• Learn from instruction
• Use Natural Language English, Welsh, Dutch, etc.
• Be aware of self (consciousness)
• Use tools

24
Some Questions

• Many people assert that intelligence is a unique
  property of living things (i.e. man and animals)
• Are plants and micro-organisms intelligent?
• Can a system be intelligent if it is not based on
  organic (i.e. carbon chain) chemistry?
• Can an intelligent system (e.g. man) design
  another system that is more intelligent than
  itself?

25
Intelligent Animals

• Great Apes are able to
• communicate with people
• use a computer (Macintosh!)
• use tools
• Other intelligent animals
• Monkeys and other primates
• Dolphins, whales, horses, dogs, octopus
• Rats
• Ants exhibit complex behaviour not as an
  individual but as a complete colony.
• Flat worms

26
Central Nervous System is Extremely Complicated

• Central Nervous System is very complex has about
  1011 neurons in a young adult, roughly the same
  in Autonomic Nervous System)
• Roughly 30,000 dendritic connections to each
  neuron

27
von Neumanns Conjecture

• " the simplest decription of the brain is the
  brain itself"

28
Are the Following Intelligent?

• Electrons, quarks, atoms
• Molecules
• DNA molecules
• Viruses
• Bacteria
• Clouds
• Plants

29
How We Judge Intelligence

• System response time affects our judgement of
  intelligence?
• Can a system be said to be intelligent if it
  takes a thousand years to respond to a "query"?
• Can a system ever be judged to be intelligent if
  it cannot communicate with the outside world?
  (Are autistic people intelligent?)
• Can a system be intelligent if it is not based
  on organic (i.e. carbon chain) chemistry?

30
Something to Think About

• We all acknowledge that intelligence is found in
  human beings and some of the higher animals.
• Can a system be intelligent if it is not based on
  organic (i.e. carbon chain) chemistry?

31
What is Artificial Intelligence (AI)?

• Artificial Intelligence is the study of those
  topics discussed in books with Artificial
  Intelligence in the title.
• AI is the study of heuristics, rather than
  algorithms.

32
Heuristics Algorithms

• Heuristic A rule of thumb, which usually works
  but may not do so in all circumstances. Example
  getting to university in time for a 9.00 lecture.
• Algorithm A prescription for solving a given
  problem, over a defined range of input
  conditions. Example solving a quadratic
  equation, or a set of N linear equations
  involving N variables.

33
Optimality Sufficiency

• In many real life problems, there may be no
  possibility of our ever finding an optimal
  solution, or even proving that a provisional
  solution is optimal.
• It may be more appropriate to seek and accept a
  sufficient solution (Heuristic search) to a given
  problem, rather than an optimal solution
  (algorithmic search).

34
Impossibility A study of tasks that neither
people nor computers can do - and never will!
35
A Big Mistake!

• Given enough time and money, science /
  mathematics can solve any problem.

36
Things to Ponder

• It may be that the simplest description of the
  human mind is the human mind itself.
• John von Neumann.
• Write an examination question suitable for a
  course on Artificial Intelligence and answer it.
• Question in an examination on Artificial
  Intelligence
• (University of Wales Cardiff)

37
Types of Impossibility

• Limitations of the human mind
• Limitations of present-day technology (dynamic)
• Limits due to the finite age / size of the
  Universe
• Problems fundamental to Nature
• Fractals
• Heisenbergs Uncertainty Principle (not
  discussed)
• Quantum Computing (not discussed)
• Fundamental limits
• Gödels Theorem
• Paradoxes

38
Limitations of the Human Mind

• Naming of colours. Based on learning, not on
  absolute standards.
• Face recognition. Cannot be passed on to another
  person by explanation.
• Object recognition. People cannot properly
  explain how they recognise objects.

39
What colour is this rectangle?
40
Is this called yellow?
41
People define the limits of a colour, such as
yellow

• Everybody has a different idea of what is
  yellow.
• We learn what yellow is from our parents
  other people.
• Disease, drugs, language and job all affect our
  naming of colours.

42
How do we build a machinethat can recognise
colours?

• Answer By building a machine that can learn from
  human beings.
• Note While one person teaches the machine, other
  people inevitably disagree about the names of
  colours near the limits of what most people call
  yellow.

43
Machine Recognition of Colours
44
Colour Recognition
45
Face Recognition - easy to do, impossible to
explain
Select the names associated with the four faces
opposite Karl Philip Eric William Fred Stua
rt Peter Gordon David Stephen Mark Paul
46
Face Recognition - easy to do, impossible to
explain
Answer David Fred Stephen Karl Probabili
ty of guessing correctly at random is less than
10-4 (1 in 10000)
47
How do we recognise images?

• We cannot reliably explain how we see things.
  Introspection does not work!
• We cannot design a machine to recognise even
  simple objects / patterns simply by telling it
  how to do so.
• However, we can build machines that learn. We
  teach the machine by example, in the same way
  that we teach a child.

48
How not to recognise printed characters (e.g.
numeral 2)

• Represent the characters by an array of black and
  white squares (pixels).
• A chess board with 100100 pixels produces
  210000 (over 103000) patterns.
• You have never seen all of them, nor will you
  ever do so. (Universe is too young.)

49
Recognising printed characters by look-up table
50
We cannot use a look-up table to recognise
printed characters

• The number of patterns that can be drawn on even
  a low-resolution grid (e.g. 100100 pixels) is
  far too large allow us to recognise printed
  letters by using a look up table.
• Remember The Universe has only 1081 particles
  and is 4.1x1017 seconds old.

51
Moores Law(Gordon Moore, Co-founder of Intel)
52
Autism in Computers

• Computers need sensors and actuators to perform
  useful tasks in the physical world.
• Todays sensors and actuators are very clumsy
  compared to their human counterparts.

53
Weather forecasting

• Precise weather forecasting is impossible because
  we do not have enough information about the
  atmosphere.
• To do so would require pressure, temperature,
  humidity and wind-speed sensors no more than 10m
  apart, over the whole surface of the world. (We
  also need to sample every 10m in the vertical
  direction, up to 50Km)
• We would also need a model of ocean currents
  every field, tree, rock, building and elephant!

54
More Difficulties -computers cannot yet model

• The machines used in the National Lottery.
• The performance of horses in the Grand National
• The behaviour of a colony of ants.
• Even a simple natural evolutionary milieau.
• Bacterial growth in a human organ.
• Human behaviour.
• Criminal tendencies
• Stock market movements.
• Popularity ratings of politicians, pop stars, etc.

55
Universe is TooYoung / Small

• To decipher certain coded messages.
• To recognise visual patterns using obvious
  methods.
• To decide whether life is predestined.
• Consensus view of the Universe by scientists
• Age 1.3x1010 years (4.1x1017 seconds)
• Size 1081 particles.

56
Unpredictable Deterministic Systems
57
Binary Feed-back Shift Register - Prolog Program

• go - bfbsr(1,0,0, 0,0,0, 0,0,0, 0,0).
  Starting state
• bfbsr(A,B,C,DE) -
• exor(A,C,F), Exclusive OR
• append(E,F,G), Feedback
• write(A), Output one bit
• !, Improves efficiency
• bfbsr(B,C,DG). Repeat indefinitely
• exor(1,0,1). exor(0,1,1). Defining exor/3
• exor(0,0,0). exor(1,1,0). Defining exor/3

58
Unpredictable Deterministic System - Sample Output

• 11010100001100001001111001011100111001011110111001
  00101011
• 10110000101011100100001011101001001010011011000111
  10111011
• 00101010111100000010011000010111110010010001110110
  10110101
• 10001100011101111011010100101100001100111001111110
  11110000
• 10100110010001111110101100001000111001010110111000
  01101011
• 00111000111110110110001011011101001101010011110000
  11100110
• 01101111111110100000001001000001011010001001100101
  01111110
• 00010000110010100111110001110001101101101110110110
  10101101
• 100000110111000111010110110100011011001011101111X
  
• Can you predict the next value for X?

59
Unpredictable Deterministic System - Sample Output

• 11010100001100001001111001011100111001011110111001
  00101011
• 10110000101011100100001011101001001010011011000111
  10111011
• 00101010111100000010011000010111110010010001110110
  10110101
• 10001100011101111011010100101100001100111001111110
  11110000
• 10100110010001111110101100001000111001010110111000
  01101011
• 00111000111110110110001011011101001101010011110000
  11100110
• 01101111111110100000001001000001011010001001100101
  01111110
• 00010000110010100111110001110001101101101110110110
  10101101
• 100000110111000111010110110100011011001011101111X
  
• It is relatively easy if you use intelligence but
  it is very difficult if you rely on brute force
  methods.

60
Pseudo-random Number Generators

• The Mersenne Twister algorithm (current
  champion) has a period of 219937-1 (about 106000)
• Reminder about the Universe
• Age 4.1x1017 seconds
• Size 1081 particles (visible Universe)
• Agesize
• Could we really discover the periodicity of this
  algorithm in any computer that we could
  conceivably build within this Universe?

61
Universe is Too Young / Small

• For a random process to create works of William
  Shakespeare.(This is the famous monkey at a
  keyboard conundrum.)
• To write down all of the English sentences that a
  person could understand
• To solve the Travelling Salesman Problem

62
Monkey Typing at Random

• There are 27 characters (A,B,,Z,space). For
  simplicity, ignore numerals and punctuation
  symbols.
• There are 27N sequences of N characters.
• The average time to type a book with 250000
  characters (40000 words) is equivalent to typing
  0.5x2250000 (about 1080000) sequences.

63
Number of English Sentences

• There are far more sentences possible than we can
  ever write down.
• Here is a sentence that you have never seen, or
  heard before today
• On Wednesday 29th January 2007, at Cardiff
  University in the capital city of Wales, Prof.
  Bruce Godfrey Batchelor gave a lecture describing
  tasks that cannot be performed by a computer nor
  a human being, while his wife Eleanor Gray
  Batchelor was at home, looking after their
  grand-daughter, Victoria Elizabeth Paynter."

64
Number of English Sentences

• Consider only sentences of the following very
  simple form
• sentence - np , vp, np.
• np- qualifier, al, noun. Noun phrase
• al - adjective. Adjective list
• al - adjective, al. Adjective list
• vp - adverb, verb. Verb phrase
• Assume there are 100 words of each type
• noun, adjective, adverb, verb, qualifiers.

65
Counting English Sentences

• Examples
• a rich young man sometimes drives a powerful red
  car
• the poor old woman often sees the big green bus
• a spotty green frog sometimes crosses this busy
  main road
• all short pink dreams wearily travel the last
  painful mile.
• There are over 1020 different sentences of this
  very simple form. (There are many more sentence
  forms in English. Each sentence form has many
  more variants than this.)
• You have never seen all sentences of even this
  simple type. (The Universe is too young.)

66
Travelling Salesman Problem
67
How Computation Time Rises

• It is impossible to solve the Travelling
  Sales-man Problem by any faster method than
  exhaustive search (i.e. trying all possible
  routes)
• Computation time rises in proportion to N!
• N! N.(N-1)(N-2)3.2.1.
• N is the number of cities to be visited.
• Some values for N!
• 10! 3.6x106 100! 9.3x10157
• 1000! 4.0x102567 1750! 2.1x104917

68
Packing 2D Shapes - Examples
Rectangles (eg glass)
Polygons (eg carpet)
Odd shaped material (eg leather for shoes)
Random Shapes (eg chocolates in a tray)
69
Packing Random Shapes

• Packing 2-dimensional shapes requires an
  exhaustive search - we have to try all possible
  combinations of position (X,Y axes) and
  orientation, for each shape (3 degrees of
  freedom). (Applications in leather industry)
• Packing 3-dimensional shapes reuuires 6 degrees
  of freedom. (Applications packing baggage in
  aircraft hold, building dry stone wallls)
• Packing occurs in higher dimensions
  (Applications planning a diet, planning
  television commercials)

70
Packing Spheres - a digression

• It is obvious that hexagonal packing is
  optimal but we cannot yet prove this
  mathematically.

71
Fractals - Impossible to Measure

• The length of a fractal curve (e.g. the coast
  line of Great Britain) cannot be measured without
  first defining the length of the measuring rod.

72
Unpredictability of Fractals (Example Hilbert
Curve)
73
Inability to Count Fractals(Example Mandelbrot
Set)

• Consider the the following recursive
  relationship
• Zn Zn-12 C
• where Z0 0 and C is a complex number.
• C ? MS ? Zn 0.
• MS is infinitely large

74
Mandelbrot Set

• If we magnify the edge of the MS, we obtain a
  picture much like the original, whatever the
  magnification.
• MS is said to be be self-similar.

75
Alan M. Turing, (1912 - 1954)

• Major contributor to the code-breaking work at
  Bletchley Park (Enigma), during World War II.
• Major contributor to the early development of
  computers.
• Foresaw Artificial Intelligence devised the
  Turing Test.

76
Turing Machine

• A Turing Machine is a mathematical model of a
  digital computer running a program.
• Alan Turing hypothesised that a TM can perform
  any arithmetic / logical task that we may
  specify. TM can perform tasks such as solving
  equations.
• There is one TM that can simulate all other TMs.
  (Called the Universal Turing Machine, UTM)

77
Turing Machine (TM)Abstract model of computer
78
Turing Halting Problem

• Alan Turing was able to show that a Turing
  Machine is unable to predict how long another TM
  will take to halt. (Whether it is finite or
  infinite time)
• A computer program cannot predict how long
  another program will take to execute, except by
  running it.

79
Recent Successes

• Fermats Last Theorem
• Four Colour Problem
• Squaring the Circle
• These were once considered impossible.

80
Our present lack of knowledge

• Computers are too slow and too weak to model the
  nervous system of even a simple animal, let alone
  a human being.
• Bumble bees cannot fly.
• We can model protein folding but with
  considerable difficulty.
• We cannot model flames in a fire. (i.e produce
  realistic images for the cinema.)
• We cannot explain how a handkerchief falls to the
  ground.

81
Self-referential Sentences, Recursion and Viral
Ideas

• I, me, this screen etc. are examples of
  self-reference
• Recursion is widely used in mathematics AI
• E.g defining ancestor, calculating change, route
  planning
• Viral ideas (memes)
• Copy me
• Copy everything on this screen
• Do the same for somebody else.
• Chain letters
• Missionary outreach

82
Fundamental ProblemsParadoxes of Recursion

• All Cretans are liars. (Epimenides, The Cretan)
• All sentences written in blue are false.
• This sentence cannot be translated into Welsh.
• Murphys Law was not invented by Murphy but by
  somebody else with the same name.

83
Recursion

• Many ideas are expressed recursively. Examples
• Ancestor / descendent
• Factorial N! Nx(N-1)!
• Calculating change in a shop
• Packing shopping in a bag
• Route finding
• Education (Teachers gradually refine ideas)
• Many computer programs use recursion.
• Used properly, recursion is a very powerful tool
  but we cannot allow infinite recursion in
  practice.

84
Kurt Gödel (1906-1978)

• Major work On Formally Undecidable
  Propositions in Principia Mathematica and Related
  Systems, 1931.
• Widely considered to be one of the most
  important results of human thinking ever
  achieved.
• He also predicted time travel!

85
Gödels Incompleteness Theorem
86
Gödels Incompleteness Theorem (Kurt Gödel, 1931)

• In any consistent axiomatic system, there exist
  theorems that are impossible to prove or
  disprove.
• There is no constructive procedure that will
  prove or disprove that a set of axioms is
  consistent.
• GIT is proved - it is not a theory.

87
SummaryHuman Mind

• Cannot properly analyse itself.
• It may not be possible for one human mind to ever
  understand another human mind fully.
• Human beings define many concepts yet cannot
  properly explain them to another person - they
  can teach by example.

88
SummaryTechnological Advances

• Many things that are impossible today will become
  possible in a few years.
• Moores Law Computer Power (no. of transistors /
  mm2 on a silicon chip) doubles every 2 years.
  (Recently revised to 18 months)
• Computers limited by lack of sensors (We always
  will be because they are very expensive to
  install.)
• We are approaching fundamental limits imposed by
  the finite size of the atom.

89
SummaryCombinatorial Explosion

• Many systems consist of parts that can be
  interchanged at will (e.g. nouns in a sentence,
  individual dishes on a menu, accessories in a
  car)
• Number of possible combinations depends on many
  individual choices
• N1xN2xN3xxMm-1xNm
• Such a product can be very large - much larger
  than we can ever compute in this Universe.

90
Summary Gödels Incompleteness Theorem

• The proof of Gödels Incompleteness Theorem is
  reminiscent of the Epimenides Paradox
• I am lying.
• For a detailed discussion of GIT, see D. R.
  Hofstadter, "Gödel, Escher, Bach An Eternal
  Golden Braid", Penguin Books, Harmondsworth,
  England, 1979, ISBN 0 140055797.
• An informal proof of GIT is available later.

91
Final Thoughts

• For men it is impossible but not for God to
  God everything is possible. (Mark 1027)
• This is apparently in direct contradiction of
  Gödels Incompleteness Theorem. However, if we
  allow God infinite time, GIT is irrelevant. The
  traditional Jewish-Christian-Moslem view of God
  is that He has existed and will exist for
  eternity. GIT says nothing about this!
• We can never know all of the truth.
• The existence of an omniscient / omnipotent God
  cannot ever be disproved.

92
Levels of Infinity

• Georg Cantor (1845 - 1918) showed that there are
  many infinities - some are larger than others.
• On which level of infinity does God reside?

93
Informal Proof of Gödels Incompleteness
Theorem/1

• Someone introduces Gödel to a machine that is
  supposed to be a Universal Truth Machine (UTM),
  capable of correctly answering any question at
  all.
• 2. Gödel asks for the program and the circuit
  design of the UTM. The program may be
  complicated, but it can only be finitely long.
  Call the program P(UTM) for Program of the
  Universal Truth Machine.

94
Informal Proof of Gödels Incompleteness Theorem
/ 2

• 3. Gödel writes out the following sentence "The
  machine constructed on the basis of the program
  P(UTM) will never say that this sentence is
  true." Call this sentence G for Gödel. Note that
  G is equivalent to "UTM will never say G is
  true.
• 4. Now Gödel asks UTM whether G is true or not.

95
Informal Proof of Gödels Incompleteness Theorem
/ 3

• 5. (a) If UTM says G is true, then "UTM will
  never say G is true" is false.
• (b) If "UTM will never say G is true" is false,
  then G is false (since G "UTM will never say G
  is true).
• So if UTM says G is true, then G is in fact
  false, and UTM has made a false statement. So UTM
  will never say that G is true, since UTM makes
  only true statements.

96
Informal Proof of Gödels Incompleteness Theorem
/ 4

• 6. We have established that UTM will never say G
  is true. So "UTM will never say G is true" is in
  fact a true statement. So G is true (since G
  "UTM will never say G is true).
• 7. "I know a truth that UTM can never utter,"
  Gödel says. "I know that G is true. UTM is not
  truly universal."