chenyuicst'pku'edu'cn

1
????

• ??
• ??????????????
• ??????????

2
??????

• ??????
• chenyu_at_icst.pku.edu.cn
• Tel[phone]
• ?????
• wanghongyan2003_at_163.com
• ????http//www.icst.pku.edu.cn/course/mlearning/i
  ndex.htm

3
Ch1 Introduction

• What is machine learning (ML)?
• Design a learning system an example
• ML applications
• Miscellaneous issues

4
Ch1 Introduction

• What is machine learning (ML)?
• Design a learning system an example
• ML applications
• Miscellaneous issues

5
A Brief History of Machine Learning

• ML as a scientific discipline was born in
  mid-seventies of last century.
• The first ML workshop was held in 1980 at CMU,
  with some two dozen participants and photocopied
  preprints.
• The first ML publication Machine Learning
  started in 1986.

6
Some Early Seminal Works

• Perceptron model proposed by Rosenblatt (1958),
  so-called connectionist approach, a seminal
  work in neural work.
• A system that learns to play checkers (Samuel,
  1959 1963)
• META-DENTRAL program, which generates rules that
  explains mass spectroscopy data used by expert
  system DENTRAL (Buchanan, 1978), an example of
  symbolic learning.

7
What is Machine Learning?

• The central problem it studies
• How can we build computer systems that
  automatically improve with experience, and what
  are the laws that govern all learning processes?
• We state a learning problem as a machine learns
  with respect to (w.r.t.) a particular task T,
  performance metric P, and type of experience E.

8
What is Machine Learning (2)

• More precisely, a computer program is said to
  learn from experience E, w.r.t. to some class of
  tasks T and performance measure P, if its
  performance at tasks in T, as measured by P,
  improves with experience E.

9
Alternative Views

• Machine learning as an attempt to automate parts
  of the scientific method (Wikipedia)
• Scientific method refers to a body of techniques
  for investigating phenomena, acquiring new
  knowledge, or correcting and integrating previous
  knowledge.
• Machine learning as computational approaches to
  learning

10
Example of Learning Problem

• Handwriting Recognition
• Task T recognizing and classifying handwritten
  words within images
• Performance measure P percent of words correctly
  classified
• Training experience E a database of handwritten
  words with given classification

11
Place within Computer Science

• Think about a niche within the space of all
  software applications where ML plays a special
  role
• Software applications that we cant program by
  hand (too complicated)
• Self customizing programs

12
Relation with other Disciplines

• Human and animal learning (Psychology,
  Neuroscience )
• Biology, economics, control theory (adaptiveness,
  optimization)

Computer Science
Machine Learning
Statistics
13
Ch1 Introduction

• What is machine learning (ML)?
• Design a learning system an example
• ML applications
• Miscellaneous issues

14
Design a Learning System

• Consider the example of learning how to play
  checkers
• T playing checkers
• P the percent of games it wins in the world
  tournament
• E?

15
starting position of a checkers game, from
Wikipedia
16
a checkers board state, from http//www.5025488.n
et/bbs/thread-49430-1-1.html
17
Choose the Training Experience

• Type of feedback provided by training examples
  (to improve P)
• Direct individual checkers board states plus the
  correct move for each state
• Indirect move sequences plus final outcome for
  each game
• Need to assign each move a credit/punish for the
  final outcome

18
Choose the Training Experience

• Type of feedback provided by training examples
  (to improve P)
• Direct individual checkers board states plus the
  correct move for each state
• Indirect move sequences plus final outcome for
  each game
• Need to assign each move a credit/punish for the
  final outcome

Easy for learning!
19
Choose the Training Experience (2)

• How much the learner can control training
  examples?
• Completely rely on a teacher to select board
  states and provide correct move for each state,
• have complete control over board states and final
  game outcome (indirect feedback), as in the case
  of playing against itself, or
• propose confusing board states to a teacher and
  ask for correct move.

20
Choose the Training Experience (3)

• How well the training examples resemble to the
  cases in which the final performance P is
  measured?
• Theoretical assumption vs. reality
• Related topics
• Concept drifting
• Incremental learning
• Transfer learning

21
Chose the Training Experience (3)

• How well the training examples resemble to the
  cases in which the final performance P is
  measured?
• Theoretical assumption vs. reality
• Related topics
• Concept drifting
• Incremental learning
• Transfer learning (current research hotspot!)

22
Update Summary

• A checkers learning problem
• T playing checkers
• P the percent of games it wins in the world
  tournament
• E games played against itself

23
Remaining Issues

• What knowledge to be learned?
• How to represent this knowledge?
• What algorithm used to learn the knowledge
  (learning mechanism)?

24
Remaining Issues

• What knowledge to be learned?
• How to represent this knowledge?
• What algorithm used to learn the knowledge
  (learning mechanism)?

25
Choose the Target Function

• Think of a checker learning program as an
  optimization problem At every board state the
  program chooses the best move among all the legal
  moves.
• Reformulate what to be learned as a function
  ChooseMove B ? M, or a better representation, V
  B ? R

real number set
26
How to Define Target Function V?

• If b is a final board state, then it is simple
• If b is won, V(b)100 (or other big number)
• If b is lost, V(b)-100
• If b is draw, V(b)0

27
How to Define Target Function V ?(2)

• Otherwise, it is tough! We might define
  V(b)V(b), where b is the best final state
  that can be achieved starting from b and playing
  optimally until end of the program.
• However, such definition is not operational!

28
Remaining Issues

• What knowledge to be learned?
• How to represent this knowledge?
• What algorithm used to learn the knowledge
  (learning mechanism)?

29
Choose a Representation of V

• A tradeoff between the expressiveness of V and
  demand for training data
• Let us consider a simple representation ? of V a
  linear combination of following board states
• x1 black pieces on the board
• x2 red pieces
• x3 black kings
• x4 red kings
• x5 black pieces threatened by red (i.e. which
  can be captured on reds next move)
• x6 red pieces threatened by black (i.e. which
  can be captured on blacks next move)

30
A Simple Representation of V
31
Remaining Issues

• What knowledge to be learned?
• How to represent this knowledge?
• What algorithm used to learn the knowledge
  (learning mechanism)?

32
Choose an Approximation Algorithm

• Choose a set of training examples (b, Vtrain(b))
• Estimate Vtrain(b)
• For some board state, it is obvious, e.g.
  Vtrain(b)100 if x20 (assuming the learning
  program plays black).
• Only indirect training examples are available.
  One common approach is via iteration, such as
• Vtrain(b) ??(Successor(b)).

33
Adjust the Weights

• A common approach to obtain the weights is by
  minimizing the sum of square of error

34
An Algorithm for Finding Weights

• Least mean square (LMS) weight update rule
• For each training example (b, Vtrain(b))
• Use the current weights to calculate ?(b).
• For each weight wi, update it as

35
Summary of the Whole Design Process
36
Issues in Machine Learning

• What algorithms exist for learning general target
  functions from training examples? Convergence of
  algorithms given sufficient examples? Which
  algorithms work best for which kind of target
  functions?
• How does number of examples influence accuracy of
  learned functions? How dose character of
  hypothesis space impact accuracy?
• How can prior knowledge of learner help?

37
Issues in Machine Learning (2)

• What specific functions should the learner
  attempt to learn? Can this process be automated?
• How can the learner automatically alter its
  representation to improve its ability to
  represent and learn the target function?

38
Ch1 Introduction

• What is machine learning (ML)?
• Design a learning system an example
• ML applications
• Miscellaneous issues

39
Machine Learning

• Speech Recognition

• Reinforcement learning
• Predictive modeling
• Pattern discovery
• Hidden Markov models
• Convex optimization
• Explanation-based learning
• ....

• Automated Control learning

• Extracting facts from text

40
Example Self-Learning Robot iCub

• iCub is a humanoid robot the size of a 3.5 year
  old child. It has been developing for 5-years
  under the project RobotCub, funded by European
  Commission for studying human cognition.
• RobotCub is an open source project.

41
Application Successes

• Speech recognition
• Two training stages speaker-independent and
  speaker-dependent
• Computer vision
• Face recognition, sorting letters contain
  hand-written addresses by US postal office
• Bio-surveillance
• Detecting and tracking outbreak of disease
• Robot control
• Robots drive autonomously

42
Ch1 Introduction

• What is machine learning (ML)?
• Design a learning system an example
• ML applications
• Miscellaneous issues

43
Research on ML

• Current research questions
• Long-term questions
• For the above two items, see The Discipline
  of Machine Learning by Tom Mitchell for a sample
  of questions.
• Machine learning for tough problems relevant
  novelty detection, structural learning, active
  learning.

44
Ethical Questions

• When and where to apply ML technology?
• For example, when collecting data for security or
  law enforcement, or for marketing purpose, what
  about our privacy?
• Privacy-preserving data mining. Borrow something
  from Secure Multiparty Computing (SMC)?

45
Major Conference and Journal

• International Conference on Machine Learning
  (ICML)
• Conference on Neural Information Processing
  Systems (NIPS)
• Annual Conference on Learning Theory (COLT)
• Journal of Machine Learning Research (JMLR)
• Machine Learning

46
Some Interesting Ref

• Pattern Recognition in industry, by Phiroz
  Bhagat, Elsevier, 2005.
• UCI Machine Learning Repository
• machine learning item on Wikipedia

47
HW

• Read The Discipline of Machine Learning by Tom
  Mitchell
• 1.2 (10pt, due Sept 22)
• Bonus problem pick up one challenge from Jaimes
  paper written in 1992, and write a detailed
  update progress report. (10pt)