Title: Special Topics in Computer Science Advanced Topics in Information Retrieval Lecture 11: Natural Language Processing and IR. Semantics and Semantically-rich representations
1Special Topics in Computer Science Advanced
Topics in Information Retrieval Lecture 11
Natural Language Processing and IR.
Semanticsand Semantically-rich representations
- Alexander Gelbukh
- www.Gelbukh.com
2Previous Lecture Conclusions
- Syntax structure is one of intermediaterepresenta
tions of a text for its processing - Helps text understanding
- Thus reasoning, question answering, ...
- Directly helps POS tagging
- Resolves lexical ambiguity of part of speech
- But not WSD-type ambiguities
- A big science in itself, with 50 (2000?) years of
history
3Previous Lecture Research topics
- Faster algorithms
- E.g. parallel
- Handling linguistic phenomena not handled
bycurrent approaches - Ambiguity resolution!
- Statistical methods
- A lot can be done
4Contents
- Semantic representations
- Semantic networks
- Conceptual graphs
- Simpler representations
- Head-Modifier pairs
- Tasks beyond IR
- Question Answering
- Summarization
- Information Extraction
- Cross-language IR
5Syntactic representation
- A sequence of syntactic trees.
6Semantic analysis
Semanticanalysis
7Semantic representation
- Complex structure of whole text
8Semantic representation
- Expresses the (direct) meaning of the text
- Not what is implied
- Free of the means of communications
- Morphological cases (transformed to semantic
links) - Word order, passive/active
- Sentences and paragraphs
- Pronouns (resolved)
- Free of means of expressing
- Synonyms (reduced to a common ID)
- Lexical functions
9Lexical Functions
- The same meaning expressed by different words
- The choice of the word is a function of other
words - Few standard meanings
- Example Magn much, very
- Strong wind, tea, desire
- Thick soup
- High temperature, potential, sea highly
expensive - Hard work hardcore porno
- Deep understanding, knowledge, appreciation
10...Lexical Functions
- give
- pay attention
- provide help
- adjudge a prize
- yield the word
- confer a degree
- deliver a lection
- get
- attract attention
- obtain help
- receive a degree
- attend a lection
11...Syntagmatic lexical functions
- In semantic representation, are transformed to
the function name - Magn wind, tea, desire
- Magn soup
- Magn temperature, potential, sea MAGN expensive
- Magn work Magn porno
- Magn understanding, knowledge, appreciation
- In different languages, different words are
used... - Russian dense soup Spanish loaded tea, lend
attention - ...but the same function names.
12Example Translation
13...Paradigmatic lexical functions
- Used for synonymic rephrasing
- Need to reduce the meaning to a standard form
- Example Syn, hyponyms, hypernyms
- W ? Syn (W)
- complex apparatus ? complex mechanism
- Example Conv31, Conv24, ...
- A V B C ? C Conv31(V) B A
- John sold the book to Mary for 5
- Mary bough the book from John for 5
- The book costed Mary 5
14Semantic network
- Representation of the text as a directed graph
- Nodes are situations and entities
- Edges are participation of an entity in a
situation - Also situation in a situationbegin reading a
book, John died yesterday - Situation can be expressed with a nounProfessor
delivered a lection to studentsProfessor
lectured to studentsLecture on history,
memorial to heroes - A node can participate in many situations!
- No division into sentences
15Situations
- Situations with different participants are
different situations - John reads a book and Mary reads a newspaper. He
aks her whether the newspaper is interesting. - Here two different situations of reading!
- But the same entities John, Mary, newspaper,
participating in different situations - Tense and number is described as situations
- John reads a book
- Now (reading (John, book) quantity (book, one)
16Semantic valencies
- A situation can have few participants (up to 5)
- Their meaning is usually very general
- They are usually naturally ordered
- Who (agent)
- What (patient, object)
- To whom (receiver)
- With what (instrument, ...)
- John sold the book to Mary for 5
- So, in the network the outgoing arcs of a node
are numbered
17Semantic representation
- Complex structure of whole text
Now
Give
2
1
ATTENTION
GOVERNMENT
Now
1
IMPORTANT
2
Now
COUNTRY
1
2
2
Possess
SCIENCE
Quantity
1
1
WE
18Reasoning and common-sense info
- One can reason on the network
- If John sold a book, he does not have it
- For this, additional knowledge is needed!
- A huge amount of knowledge to reason
- A 9-year-old child knows some 10,000,000 simple
facts - Probably some of them can be inferred, but not
(yet) automatically - There were attempts to compile such knowledge
manually - There is a hope to compile it automatically...
19Semantic representation
- ... and common-sense knowledge
20Computer representation
- Logical predicates
- Arcs are arguments
- In AI, allows reasoning
- In IR, can allow comparison even without reasoning
21Conceptual Graphs
- A CG is a bipartite graph.
- Concept nodes represent entities, attributes, or
events (actions). - Relation nodes denote the kinds of relationships
between the concept nodes. - John?(agnt)?love?(ptnt)?Mary
22(No Transcript)
23Use in IR
- Restrict the search to specific situations
- Where John loves Mary, but not vice versa
- or
- Soften the comparison
- Approximate search
- Look for John loves Mary, get someone loves Mary
24Obtaining from text
- Algebraic formulation of flow diagrams
- AlgebraicJJ formulationNN ofIN flowNN
diagramsNNS - np, n, formulation, sg, adj,
algebraic, of, np, n, diagram, pl,
n_pos, np, n, flow, sg - algebraically?(manr)?formulate?(ptn)?flow-dia
gram
25Steps of comparison
- Determine the common elements (overlap) between
the two graphs. - Based on the CG theory
- Compatible common generalizations
- Measure their similarity.
- The similarity must be proportional to the size
of their overlap.
26An overlap
- Given two conceptual graphs G1 and G2, the set of
their common generalizations O g1, g2,...,gn
is an overlap if - If all common generalizations gi are compatible.
- If the set O is maximal.
27An example of overlap
28Similarity measure
- Conceptual similarity indicates the amount of
information contained in common concepts of G1
and G2. - Do they mention similar concepts?
- Relational similarity indicates how similar the
contexts of the common concepts in both graphs
are. - Do they mention similar things about the common
concepts?
29Conceptual similarity
- Analogous to the Dice coefficient.
- Considers different weights for the different
kinds of concepts. - Considers the level of generalization of the
common concepts (of the overlap).
30Relational Similarity
- Analogous to the Dice coefficient.
- Considers just the neighbors of the common
concepts. - Considers different weights for the different
kinds of conceptual relations.
31Similarity Measure
- Combines the conceptual and relational
similarities. - Multiplicative combination a similarity roughly
proportional to each of the two components. - Relational similarity has secondary importance
even if no common relations exits, the pieces of
knowledge are still similar to some degree.
32Flexibility of the comparison
- Configurable by the user.
- Use different concept hierarchies.
- Designate the importance for the different kind
of concepts. - Manipulate the importance of the conceptual and
relational similarities.
33Example of the flexibility
Gore criticezes Bush vs. Bush criticizes Gore
34An Experiment
- Use the collection CACM-3204 (articles of
computer science). - We built the conceptual graphs from the document
titles. - Query Description of a fast procedure for
solving a systemof linear equations.
35The results
- Focus on the structural similarity, basically on
the one caused by the entities and attributes. - (a0.3,b0.7, WeWa10,Wv1)
- One of the best matches
- Description of a fast algorithm for copying list
structures.
36The results (2)
- Focus on the structural similarity, basically on
the one caused by the entities and actions. - (a0.3,b0.7, WeWv10,Wa1)
- One of the best matches
- Solution of an overdetermined system of equations
in the L1 norm.
37Advantages of CGs
- Well-known strategies for text comparison (Dice
coefficient) with new characteristics derived
from the CGs structure. - The similarity is a combination of two sources of
similarity the conceptual similarity and the
relational similarity. - Appropriate to compare small pieces of knowledge
(other methods based on topical statistics do not
work). - Two interesting characteristics uses domain
knowledge and allows a direct influence of the
user. - Analyze the similarity between two CGs from
different points of view. - Selects the best interpretation in accordance
with the user interests.
38Simpler representations
- Head-Modifier pairs
- John sold Mary an interesting book for a very low
price - John sold, sold Mary, sold book, sold for
priceinteresting book, low price - A paper in CICLing-2004
- Restrict your semantic representation to only two
words - Shallow syntax
- Semantics improves this representation
- Standard form Mary bought ? John sold, etc.
39Tasks beyond IR Question Answering
- User information need
- An answer to a question
- Not a bunch of docs
- Who won Nobel Peace Prize in 1992? (35500 docs)
40...QA
- Answer Rigoberta Menchú Tum
- Logical methods
- Understand the text
- Reason on it
- Construct the answer
- Generate the text expressing it
- Statistical methods (no or little semantics)
- Look what word is repeated in the docs
- Perhaps try to understand something around it
41...Better QA
- What is the info is not in a single document?
- Who is the queen of Spain?
- King of Spain is Juan Carlos
- Wife of Juan Carlos is Sofía
- (Wife of a king is a queen)
- Logical reasoning may prove useful
- In practice, the degree of understanding is not
yet enough - We are working to improve it
42Tasks beyond IR Passage Extraction
- If the answer is long a story
- What do you know on wars between England and
France? - Or if we cannot detect the simple answer
- Then find short pieces of the text where the
answer is - Can be done even with keywords
- Find passages with many keywords
- (Kang et al. 2004) Choose passages with greatest
vector similarity. Too short few keywords, too
long normalized - Awful quality ?
- Reasoning can help
43Tasks beyond IR Summarization
- And what if the answer is not in a short passage
- Summarize say the same (without unimportant
details) but in fewer words - Now statistical methods
- Reasoning can help
44Tasks beyond IR Information Extraction
- Question answering on a massive basis
- Fill a database with the answers
- Example what company bought what company and
when? - A database of three columns
- Now (statistical) patterns
- Reasoning can help
45Cross-lingual IR
- Question in one language, answer in another
language - Or question and summary of the answer in
English, over a database in Chinese - Is a kind of translation, but simpler
- Thus can be done more reliably
- A transformation into semantic network can
greatly help
46Research topics
- Recognition of the semantic structure
- Convert text to conceptual graphs
- All kinds of disambiguation
- Shallow semantic representations
- Application of semantic representations to
specifictasks - Similarity measures on semantic representations
- Reasoning and IR
47Conclusions
- Semantic representation gives meaning
- Language-specific constructions used only in
theprocess of communication are removed - Network of entities / situations and predicates
- Allows for translation and logical reasoning
- Can improve IR
- Compare the query with the doc by meaning, not
words - Search for a specific situation
- Search for an approximate situation
- QA, summarization, IE
- Cross-lingual IR
48Thank you! Till June 15? 6 pm Thesis
presentation? Oral test?