Polynomial Time Probabilistic Learning of a Subclass of Linear Languages with Queries

1
Polynomial Time Probabilistic Learning of a
Subclass of Linear Languageswith Queries

• Yasuhiro TAJIMA, Yoshiyuki KOTANI
• Tokyo Univ. of Agri. Tech.

2
This talk

• Probabilistic learning algorithm of a subclass of
  linear languages with membership queries
• learning via queries special examples ?
  Probabilistic learning
• Use translation algorithms
• representative sample ? random examples
• equivalence query ? random examples

3
Motivations

• A simple deterministic grammar (SDG) has
• at most one rule for every
  pair of
• ? learning algorithm for SDG from
• membership queries
• representative sample
• ? for linear languages

(Tajima et al. 2004)
CFLs
Linear
SDLs
Regular
4
Linear grammar
A context-free grammar is a linear grammar if
every rule is of the form
Any linear grammar can be written in RL-linear
s.t. if every rule is of the form
and
5
Strict-deterministic linear grammar
An RL-linear is a Strict-det linear if, for any
pair of rules
or
for some a,B,c,D
has only left linear rules ( or right linear
rules)
Ex)
6
Deterministic linear grammar

• A linear grammar is deterministic linear (DL)
• if every rule is of the form

or
and
Theorem(de la Higuera, Oncina 2002) DL
identifiable in the limit from polynomial time
and data
7
MAT learning (Angluin1987)
learner hypothesis
teacher target language
membership query
yes or no
hypothesis
equivalence query
counter example
8
PAC learning (Valiant 1984)

• PAC Probabilistic Approximate Correct

probability distribution
learning algorithm
hypothesis
target concept
example
is PAC
9
Equivalence query ? PAC learning algorithm
(Angluin1987)
If a hypothesis is consistent with
examples
If there is a consistent hypothesis ? PAC
learnable


consistent with examples
10
Probabilistic learning with queries
Example oracle
Learning algorithm
hypothesis
Membership query
Yes or No
target language
11
Representative sample for a Strict-det

• Strict-det

representative sample (RS)
for some
All rules are used to generate Q
12
Example
then
is a representative sample (RS)
13
Rule occurring probability
a target grammar a probability
distribution on for an
example error parameter
confidential parameter the size of
target grammars rules For every rule
, define
14
is a rule occurring probability s.t. appears
in the derivation of an example

• is an probability that
• and
• is used in the derivation

15
Let Suppose The set of m-examples contains a
set of RS with the probability Proof Any
rule doesnt appear in derivations of
m-examples occurs
RS
16
We can conclude that

• Equivalence query can be replaced by
• random examples
• Representative sample can be replaced by
• random examples

17
membership oracle
example oracle
probabilistic learning algorithm with queries
n-random examples
m-random examples
consistency check
query
response
nega
equivalence query
posi
learning algorithm
membership query
representative sample
18
Learning algorithm via queries and RS

• while (finish 0) begin
• make nonterminals from
• make rules and hypothesis
• if (equivalence query for responds yes)
• output , finish 1
• else
• update by the counterexample
• end

19
Making nonterminals
then
a nonterminal an equivalence class contains
(u,v,w)
20
Making rules
Make all rules as follows except for not
consistent with query results
Select a hypothesis randomly
21
Exact learning of strict-det

• Strict-det is polynomial time exact learnable via
• membership queries, and
• a representative samples (RS)
• c.f. Angluin(1980) for regular sets

The learning algorithm overview
a set of Strict-det (not bounded by a
polynomial)
Chose one randomly, Equivalence query
SD
SD
SD
RS
Possible rules
SD
SD
SD
SD
The correct hypothesis
Witnesses delete incorrect rule
22
Conclusions

• Strict-det linear language can be probabilistic
  learnable with queries in polynomial time
• Future works
• Identification from polynomial time and data
  (teachability)
• RS ? Correction queries

23
(No Transcript)
24
Theorem

• Strict-det linear languages are
• polynomial time probabilistic learnable with
  membership queries

25
Simple Deterministic Languages

• Context-free grammar(CFG)
• in 2-standard Greibach normal form is
• Simple Deterministic Grammar (SDG) iff
• is unique for every and
• Simple Deterministic Language (SDL) is the
  generated language by a SDG

26
Representative sample for an SDG

• SDG

representative sample (RS)
for some
All rules are used to generate Q
27
Example
then
is a representative sample (RS)
28
PAC learning
(Valiant1984)
Target language
Hypothesis language
on
Probability distribution
A PAC learning algorithm outputs such that
where
29
Query learning of SDLs
(Tajima2000)

• SDLs are polynomial time learnable via membership
  queries and a representative sample

the teacher
the learner
membership query
yes / no
representative sample
at the beginning
representative sample a special finite subset
of
30
Learning model
the learner
the teacher
membership query
yes / no
representative sample
at the beginning
representative sample a special finite subset
of