ADABOOST '

1
????? ?????? ?????? ?????? ????????? (?????????
ADABOOST) ?????? ????? ??????.
??????????? ??????, ??????? ????? ?????, ??????
?????????? ???? ???? ???? ?????? ?????? ?????
???? ??? ???
???? ?????? ????' ???? ????? (?????????), ????'
???? ????? (?????????), ????' ???? ??????
(?????????), ??' ???? ???? (???? ????).
2
?????? ?????

• ???????? ????? ?????.
• ??? ?' ???????????.
• ????? ?? ?????? ??????? ????????? AdaBoost
  Weak-Learner , ?-Random Forest .
• ???? ?? ????? ?????.
• ?????? ???????? (??? ?????? ???????).
• ??? ?' - ?????? ????????? ????? ???????.
• ????????.
• ???? ??????.
• ?????? ????????.
• ??? ?' ??????? ?????? ??????
• ?????????? ??????.
• ??????.
• ??????? ????? ?????.

3
???????? ????? ?????

• ????????? ???????? ????? ?? ????? ????? ??? ????
  ???? ?????? ?????? ?????? ?? ????? ?????? ????
  ???? ???? ???? ????? ????? ????? ????? ???????.
• ???? ????? ??? ???? ???? ?????? ?????? ??????
  ???? ?????? ?"? ????? ????? ???????? ??? ?????
  ????? ?? ????? ?????? ???? ???? ?? ???? ???? ????
  ?? ????? ?????? ????? ???????? ????? ????? ????
  ??????? ?? ?????? ??????. ????? ?"? ??????? ?????
  ???? ????? ??? ???? ????? ????? ?????, ???? ????
  ??????? ??? ???? ????? ??? ?????? ????? ??
  ??????.
• ????? ????? ????? ????? ???? ??????? ????? (????
  ???????) ??? ???? ????? ?????? ???? ???? ?? ???
  ??? ???? ??? ?????? ???? ????? ???? ?

4
??? ?' ???????????
5
?????? ?????? ????????? - ????? ??????? (?????)

• ???? ????? ????? ?????????? ????? ??????
  ?????????? ???????? ????? ????? ???? ???????
  (?????? ?? ?????? ???????).
• ?? ????? ????? ???? ?? ????? ????? (???????
  ??????? ???????? ???? ????? ?? ?????? ???????).
  ???? ????????? ??? ????? ?? ?????? ?? ???????
  ????? ????? ?? ???
• ?????? ??????.
• ???????? ?????? ???? ???? ???? ?? ??????? (??
  ???? ???? ????? ?????) ????? ????? ???? ?????????
  ????? ??????/?????.

6
????? ?? ????? ??????
7
???????? Boosting

• ???????? Boosting ???? ???????? ????? ??????
  ????????? ????? ??? ??? ????.
• ???? ???????? ?-Boosting ???? ????? ??????? ????
  ???? ???? ???????, ???? ?"? ????? ?????????
  ????? ???, ???? ????? ??? ???? ????? ????????
  ????? ???? ???? (Weak-Learning).
• Boosting ???? ?? ??? ???? ???????? ?????? ????
  ????? ??? ???? ???????? X, ???? ??? ??? ?????? ??
  ????? ?? ????? ?? ??????? ?????. ?????????
  ??????? ?????? ?????? ??????? ?????? ??????? ????
  ???? ????.
• ????? ???? ??? ?? ??????
  .........
• ????? ???? ????? ??? ?????? ????? ????? ??.

8
Boosting (cont.)

• ??????
• ???? ???????? X ?? M ??????? , XM (x1,y1),,
  (xm,ym)
• ??????? D, ?????? D 1/M
• ????? ?????? Y ??????? Y X ? -1,1
• ???????? ????? ??? Weak Learn ?? ??????? g, l

(x1,y1),(x2,y2)(xm,ym)
Weak Learn
??????? h
errorD(h)PxD(h(x) ¹Y(x))lt1/2-g
????? ???????? ????? l
9
Boosting (cont.)
Weak Learn
(x1,y1),(x2,y2)(xm0,ylm0)
??????? 1h
??? D1
Weak Learn
(x1,y1),(x2,y2)(xm0,ym0)
??????? h2
??? D2
Weak Learn
(x1,y1),(x2,y2)(xm0,ym0)
??????? hk
??? Dk
??????? ????? hMF(h1,h2,,hk)
10
Weak Learning Pseudo Code
11
Adaboost Pseudo Code

• Input
• N examples SN (x1,y1),, (xN,yN)
• a weak base learner h h(D,S)
• Initialize
• equal example weights Dt 1/N for all n 1..N
• Iterate for t 1..T
• train base learner according to weighted example
  set (D(t),S) and obtain hypothesis ht h(D(t),S)
• compute hypothesis error et , et ?Dt I(Yt ?
  ht)) / ?Dt
• compute hypothesis weight at , at log ((1 et
  )/et
• update example weights for next iteration D(t1),
  D(t1) D(t)expat (Yt ? ht )
• Output final hypothesis as a linear combination
  of ht
• hFINAL sign ? at ht

12
Weak Learning Pseudo Code

• Input Train set S, S xi,j (i 1,2M j
  1,2N)
• Distribution
  (New weights
  (wl)
• are compute every iteration outside the weak
  learning)
• Labels Yi Y(1,...,K)
• A constant T, for instance
• 1 for t 1 to T do
• 2 Threshold(t)
• 3
• 4
• 5
• 6 End for
• 7 Output

13
???????? Adaboost ?????-???
14
Random Forest Model

• Combination of Tree predictors i.e. a large
  number of trees is generated to create a Random
  Forest.
• Each Tree created depends on the value of a
  Random vector .
• is generated independently using an
  identical distribution for all trees

What is a Random Vector ?

• Represents a Random Sample chosen from Training
  Data using i.i.d
• Data Samples are usually selected with
  replacement

15
???? ???? ????????? Random Forest

• ???? ????? ????? ???? ???? ?????? (??"? ??????
  ???? ???????)

• ???? ?? ????? ?"? ????? ?????? ??????? ???????
  ???????. ???' ???? ?? ??? ?????? ?? ?????? ???
  ????.

lt 0
1
lt 3-
1-
1

• ???? ??? ?? ???? ???? ???? ??????.
• ?? ????? ??? ??? ????? ????? ???? ????? ?????
  ?????, ?????? ??? ?? 10 ????? ?-1, 10- ????? ?-1
  ?- 1- ????? ?- 1- .
• ?? ????? ??? ?????? ???? ????? ???? ??? ??
  ??????, ???? ??? ?? ???? 1, ?????? ???? ?????.

16
Random Forest Pseudo Code

• Initially select the number of trees to be
  generated e.g. K.
• At Step k (1 lt k lt K)
• A Vector is generated, represents
  Samples (data selected for creating Tree)
• Construct Tree h(x, )
• Using any Decision Tree Algorithm
• Each Tree casts 1 vote for the most popular class
  at X
• The class at X is predicted by selecting the
  Class with maximum Votes

17
????? ?????

• 1

18
????? ?????

• ??? ????? ????? ?? ??? ????? ????? ????? ????????
  ?????? ?? ????? ?? 2 ?????????? Adaboost
  ?-Random Forest. ??????? ?????? ????????? ????
  ??????? ????? ???, ??????? ???? ????? ?????
  ?????? Boosting ???????) ????? ????? ??? ??
  (Adaboosts .
• ?????? ?? ???????? Random Forest ????? ??? ????
  ???????? Adaboost ???? ???????? ??? ?????? ????
  ?????? ?????? (????? ????? ?????? ????) ??? ?????
  ??????.

19
????

• ????? ???? ?? ????? ???? ????? ????? ??? ?????
  ?"? Drucker (1999) ??? ????? ????????? ??? ?????
  C4.5 ????? ???. Drucker ????? ?? ????? ??????
  ??? ?? ??? ????? ???? (????? ?? ??? ???? ?????? ?
  SVMs ?????).
• ???? ????? ?? ?? ????? ??? ?? ???? ??????
  ???? ???? ????? ?????? ?? ???? ???? ?? ???
  ??????.
• ?????? ???? ?????? ???? ????? satimage ?? ????
  ??? "?? ????? C4.5" ????? ???.

20
(No Transcript)
21
Adaboost with Random Forest as Weak Learner

• ???????? Adaboost ??? ????? ?"? Schapire ??
  ?????, ??? ????? ????? ?- Weak Learner "??????"
  ????? ??????? ???? ????? ??? ????? ??? ???? ?
  Random Forest, ??? ???? ???????? ?????, ????
  ????? ??????? h ???? ?????? ?????? ??? ?????
  ????????? ?????? ??????? ????? ?? ????? ??????
  ????? ?????? ????? ?????? ????.
• ????????? ???? ???? ???? ?????? ?? ?????? ?????,
  ?????? ???? ???? ????? ??? ?????.
• ????????? ???? ???? ???? ?????? ???????? (??????
  ???????) ?? ????????.

22
Adaboost with Random Forest as Weak Learner
Pseudo Code

• Input
• N examples SN (x1,y1),, (xN,yN)
• a weak base learner h h(D,S)
• Initialize
• equal example weights Dt 1/N for all n 1..N
• Iterate for t 1..T
• 1. weak learner ht Learn( S dt ) call Random
  Forest
• Initially select the number K of trees to be
  generated
• For k 1 to K
• A Vector is generated.
• Construct Tree h(x, ) Using
  any Decision Tree Algorithm.
• Each Tree casts 1 vote for the most popular
  class at X , The class at X is predicted by
  selecting the Class with maximum Votes
• Return a hypothesis ht (Training set)
• end For
• 2. compute hypothesis error et
• 3. compute hypothesis weight at
• 4. update example weights for next iteration
  D(t1)
• Output final hypothesis as a linear combination
  of ht

23
?????? ???????? 1 (satimage)
24
?????? ???????? 2 (satimage)
25
??? ?' ?????? ????????? ????? ???????
26
??????? ????? ?????? ???????(???? ????? ?????
??????? ???????? -????' ???? ??????)

• ???? ?????? ??????
• ????? ????? ?? ????.
• ?????? ??????? ??????? ???? ????.
• ?????.
• ???? ????? ??????
• ????? ????? ?? ????? ??????? ??? ????? ??? ?????
  ????? ????????.
• ????? ????
• ??????? ??? ??? ???? ??????
• ?????? ?????, ?????? ?????
• ????? ???? congestion ????? ??????? (rush hour)
• ????? ??????? ???????? ?????? ??????? ???????
• ?????? ------- ????? ??????
• ????? ------- ???? ??????, ????? ??????, ?????
  (?????) ??????
• ????? ------- ?????????, ????? ??????
• ?????? ------ ?????? ?????, ????? ???
• ?????? ------ ????? ??????
• ??????? ??????? ?????? ???????
• ???? 2 ??????? ???? ???? ??????.

27
??? ????? (?-? ??????)

• ???? ?? ????? ?????? ?????? ????? ???? ?? ????
  ?????? ????? ????? ?????? ?????.
• ????? ????? ?? ??????? ??????? ???? ??? ?????
  ??????? ?????? ???? ????? ????? ????.
• ????? ????? ??????? ???? 78,000 ?????? ???
  ????
• ??? ???? ???? 2010, ??? ?- 50 ?????? ????
  ???????.

28
????? ?????? ????? ????

• ????? ?????? ?? 4 ???? ????? ?"? ????? ?-?.
• ??? ?????? (???? ????????) ???? ?"? ???? ???? ??
  ???? ?? ¼ ??? (???? ?????? ???? ??? ????).
• ?????? ?????? ???? ???? ????? ?????.

????? ????? ????? ????
29
???? ??????

• ???? ????? ???? ?? ??????? ????? ???? ????
  ??????? ??
• ¼ ??? TIME LINK NDET VOLU OCCU SPEE
• 0000 10 1 292
  5 40
• ????
• TIME ??? ??????.
• LINK - ???? ????? ?? ????? ??????.
• NDET - ???? ?????? ????? ?? ????? ??????.
• VOLU - ???? ?????? ???? ?????? ????? ?"? ?????
  ????.
• OCCU - ??? ?? ?????.
• SPEE - ?????? ??????.

30
????? ???????? ?????? ?????

• ????? ????????? ?????? ????? ???? ??????? ?????
  ????, ??"? ????? ???????? ????? ?? ????? ????,
  ????? ??? ?? ??? ?????? ???? ???? ???????
  ???????? ???? ???? ????? ?? ????? ?????.
• ?????? ????? ????? ?? ???????, ??"? ?? ???? ?????
  ?? ?????? ?????? ????? ?????? ????? ??? ???????
  ??????? ?? ???? ?? ??-????.
• (????? ????? ??? ?????? ????? "????" ??????
  ????? ?? ???? ????? ????? ?????)

Demo
31
??? ?' ??????? ?????? ??????
32
?????????? ??????

• ????? ????? (Preprocessing) ?? ????? ??????
  ???? ????? ?????? (train file) ??????? (test
  file) ???? ??????? ???????.
• ????? ?? ????? ?????? ?"? ????????? (?????? ???
  ??? ?????? ??????).
• ????? ?? ??????? ?????? ?"? ?????? ?? ???????.

??????? ????? ????? (??????? ?????)
????? ??? ????? ?????
????? ?????? ??????? ??????
????? ????? ????? ?"? ???????? ??????
????? ???? ????? ???? ???
33
??????

• ??????????? ?????? ?"? ?????? ?? ???????? ????
  Adaboost ???
• ????? ????? ???Random Forest ???? ?? ???? ????
  ???? ????
• ???? ???? ??? ??? ?? ??????,C4.5 ?????? ????
  ???????
• ??????? ??????? ?? ???????? ???????? ????? (?????
  ?? ???? ??????
• ?? ????? 100 ?? ????).
• ?????? ????????? ??? ???? ??????? ???? ?????
  ????? ????
• ????? ?????? ???? ??????. ????? ????? ???? ???
  ???? ?????? ??
• ??? ????? ?? ????? ?????? ??????? ?????. ?????
  ???? ?????? ?????
• ????? ???? ??? (??? ???? ?????? ???? ??????) ????
  ????? ?????
• ????? ?? ???? ?????? ??"? ?? ?????? ?? ??????
  ??????. ??????
• ?????? ?????? ????? ????? ??-?? ????? ??????
  ?????? ??????
• ?????? ????? ?? ?????.

34
??????? ????? ????

• ????? ????? ????? ???????? (????????? ADABOOST )
  ??????? ????? ??????? ????? ??? ???? ?????? ????
  ?????? ??????, ?? ???? ?????? Boosting ??????
  (???? ??????).
• ???? ????? ???????? ?? ??? (?-?) ?"? ?????
  ???????? ?????? ????? ???? VISSIM, MITSIM ????.
• ????? ????????? ?????? ????? ????? ????? ??????
  ??????? ?????? ???? ?????? ?? ????? ?-? ?? ???
  ????? ?????? ?? ????? ?????? ????.
• ????? ?? ?????? ?????? ????????? ?? ????? ??????
  ?? ?-? ??????? ????? ????????? ????? ??????.
• ??? ?? ?????? ????? ???? ?????? ??????? ??????
  ????? ????????.