Christopher%20M.%20Bishop%20(2006): - PowerPoint PPT Presentation

View by Category
About This Presentation
Title:

Christopher%20M.%20Bishop%20(2006):

Description:

1 Christopher M. Bishop (2006): Pattern Recognition and Machine Learning, Springer, pp.37-57 – PowerPoint PPT presentation

Number of Views:143
Avg rating:3.0/5.0
Slides: 32
Provided by: show164
Learn more at: http://bin.t.u-tokyo.ac.jp
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Christopher%20M.%20Bishop%20(2006):


1
????????????1???(??)
  • Christopher M. Bishop (2006)
  • Pattern Recognition and Machine Learning,
    Springer, pp.37-57

2
????????
  • 1.5 ????
  • 1.5.1 ????????
  • 1.5.2 ????????
  • 1.5.3 ???????
  • 1.5.4 ?????
  • 1.5.5 ??????????
  • 1.6 ????
  • 1.6.1 ??????????????

???
????
????
1????
3
????
  • ??????????????,??????????????????????
  • ????????????????????????????????????????
  • ??????x?????????t
  • ???x??????t??????????
  • ex)????t?????
  • ?????t???????
  • ??????t?????
  • ??????p(x,t)?????????????????????
  • p(x,t)??????????????????

4
????????(1)
  • ????????????X??????????????
  • ??????x??????????
  • ????t????????C1(t0), ???????C2(t1)
  • ?????????????p(x, t)??????????,
  • ???????????????????
  • ??????????????????????,
  • ??????????????????????????????
  • ???????????????????
  • ?????(decision)??????,??????????????
  • ????????????????????????????????

5
????????(2)
  • ??????X???x????????,???2????????????????????

(1.77)
??????/????(???????(????)??????) x???X????????(
?????????????) Ck????x???X????????(?????????)
6
????????(1)
  • (1)??????????????????????
  • ???????x?????????????1??????????????
  • ??????????????????1???????????(decision
    region)???????Rk????Rk???????????Ck??????.
  • ??????????????(?????decision boundary)????????(de
    cision surface)?????
  • ?????????????,?????????????????????.

7
????????(2)
  • ???
  • ????????C1???????????C2???????????????????.???????
    ??

(1.78)
?????????x??? x0???
8
????????
  • (2)????????????????????
  • ???????????????,????????????????????????????
  • ???????(loss function),?????(cost function)???

(1.80)
?
??
?
??
9
???????
  • ????????????????????
  • ????????????????????,????????(???)????????????????
    ???????
  • ????????????

10
??????????(1)
  • ????????????????????
  • ????????x??????????y(x)?????.???,??L(t,
    y(x))??????.
  • ?????

(1.86)
  • ????????????????????????????,

(1.87)
11
??????????(2)
  • ???EL??????y(x)?????
  • ????????

(1.88)
(1.89)
????????? ????(regression function) ?????????????
???????????????????
12
????
  • ????x????.??????????????????????,?????????????????
  • ?????x????????????????
  • ?????????????????????,?????????????

ex) ???,????????????????? ???,????????????????????
?????
??????????? ????????
13
???
????h()????????
(1.92)
?????
ex). ??????1?????????????
4
4
x
y
x,y
??p(x)
p(y)
p(x,y)
???h(x)
h(y)
h(x,y)
14
(????)?????? (entropy)
  • ???????????????????????, ????????????????(1.92)?
    ??p(x)? ???????????????,????????????

(1.93)
?????????????? ???????(entropy)?
?????????,??????????????? ???????????????????????
??????? ????????????????? ???????????????????????
? ??????????????????????????????
15
??????????(2)
  • 8????????a, b, c, d, e, f, g, h
  • ???????1/8, 1/8 , 1/8 , 1/8 , 1/8 , 1/8 , 1/8 ,
    1/8
  • ???????????

???
  • 8????????a, b, c, d, e, f, g, h
  • ???????1/2, 1/4 , 1/8 , 1/16 , 1/64 , 1/64 ,
    1/64 , 1/64
  • ???????????

???
16
??????????(3)
  • ?????????????????????????????????
  • ???????????????

N???????????????????????????. ?????????????????N??
??????, ?????N-1??????,N?????????????????N!???? i?
?????ni!????????????,N???????????????
???(multiplicity)
(1.94)
17
?????
???(multiplicity)
(1.94)
ex). N8, ????4????
18
????????????
  • ???????????????????????

(1.95)
  • ???,ni/N?????????N?8????????,???????????????

(1.96)
19
????????????
  • ???????????????????????
  • ???,pi?????i??????????????
  • ??????????,
  • ?????????????
  • ?????(?????microstate)
  • ?ni/N???????????????
  • ?????(?????macrostate)
  • ???W?????????(weight)??????

(1.97)
20
??????????????
  • ????????X???xi????????,p(Xxi)pi???.???????X?????
    ???

(1.98)
  • ????????????????????
  • ??,????????????

21
???????????????
  • ????????????????
  • H??????????????????????

(1.99)
  • ????,M?xi?????????
  • ?????????(????)

22
??????????
  • ????x???p(x)?????
  • ??,x????????????
  • p(x)????????????,????????

(1.101)
  • ???xi???????
  • i????????????x??xi????????????????,xi??????????p(x
    i)????
  • ????,????????????(1.101)??

(1.102)
23
????????
(1.102)
  • ???(1.102)????????????????,??0????????

(1.103)
  • ????????????????????
  • ????????????????ln ??????
  • ??????0???????????
  • ????????????????????????????????????

24
?????????????????
  • ????????????
  • ????????????????????

(1.105)
(1.106)
(1.107)
25
?????????????????
  • ??????,?????????0????

(1.108)
  • ????

(1.109)
  • ?????.??????????

(1.110)
  • ????????????????????????????????

26
??????????????
  • ???????????????????????????????????
  • ?????p(x)???,???????q(x)??????????
  • q(x)????x???????????????????????
  • ????p(x)?????q(x)????x????????????
    ??(additional)??????????????

(1.113)
  • ?????p(x)?q(x)???????????(relative entropy)
  • ???????????????????????(Kullback-Leibler
    divergence)???????KL??????????? ??????

27
???
  • ???,KL????????? ????,?????????????p(x)q(x)?
    ??,????????????
  • ??,???(convex function)????????
  • ??f(x)????????????????,??????????????????

(1.114)
28
???????????????????
  • ???????????(1.114)?????f(x)???????xi????,
  • ??????????.???S?1???.
  • (1.115)???????????????????.
  • ?i??xi?????????x??????????????
  • ????.?????????????????????????????(1.113)?????????
    ?,
  • ?????

(1.115)
(1.116)
(1.118)
29
KL?????????????(1)
  • ??????????(???????????????)??????????,????????????
    ???????????????
  • ??????????????,??????????,?????????????????????2??
    ??????????????????????????????
  • ?????????p(x)?????????,????????????
  • ??????????????????????q(x?)(???????????)?????????
    ?????
  • ?????1?????p(x)?q(x?)????????????????????????????
    ?????????????
  • ???,p(x)?????????????????????

30
KL?????????????(2)
  • p(x)????????????????xnn1,..,N??????
  • p(x)???????????????????????,
  • ???.??????????????,???????????????????????????????
    ???.
  • ???,KL?????????????????????????

(1.119)
31
?????
  • 2??????x?y?????p(x, y)????
  • ???????????????????????????? p(x, y)p(x)p(y)???
  • ??????????,???????????????,??????????????KL???????
    ??????????,
  • ?????x,y????????(mutual information)?????
  • ??????y???????????x?????????????????????.
  • ????????p(x)?x?????,p(xy)???????y????????????????
    ?.?????,???y??????????,x??????????????????????

(1.120)
About PowerShow.com