Relevance Feedback and Thesauri

1
Relevance Feedback and Thesauri

• How to Make Queries Better

2
Overview

• Ranked Retrieval
• Relevance Feedback
• Thesaurus

3
Typical Web Retrieval Process
Link Following
Need
KeywordQuery
More Like this
4
Ranked Retrieval

• How can we present the best item to the user
  first

5
What are we trying to do in IR

• Find the Document which is most similar to the
  query
• Ranking Interpretation
• show the most similar document first
• then the next most similar document
• and so on

6
Reprise

• Vector Model of IR
• Bag of Words Model of Text for IR

7
Bag of Words Model of Text

• Ignore the order of words in the document
• Just record whether a word appears in a document
• OR
• Just count the number of occurrences of each word

8
Similarity Measures

• Sum of Products
• Similarity(query, document)
• ?(query-termidocument-termI)
• Cosine Formula
• Various Others
• See Kowalski Chapter 7

9
Similarity as Ranking

• Use the Similarity Measure to rank the documents

10
Issues

• Most Web Searches are of Length One
• Mentioning the queried item many times does not
  necessarily make the document the most relevant
• Essentially returns the whole web as the result
  needs modification to work in practice

11
Relevance Feedback

• More Like this done properly

12
Observation

• The user is probably in the best position to
  judge the relevance of a document
• Likewise the user is probably in the best
  position to judge which returned (highly ranked)
  documents are irrelevant

13
Retrieval Process
No More Like This
Need
Analytic Query
More Like this
14
Relevance Feedback in Nutshell

• Perform an initial retrieval
• Ask the user to indicate which documents are
  relevant/irrelevant
• Add all terms from relevant documents
• Remove all terms from irrelevant documents
• requery

15
Variants

• Using Ranking and Weighting
• Pseudo relevance feedback
• use terms from all (highly ranked) retrieved
  documents
• very helpful if very few documents retrieved
• perpetuates errors/misunderstandings from
  original query

16
Exercise

• What are advantages of positive feedback ?
• What are advantages of negative feedback ?
• Whis is best ?

17
Relevance Feedback Conclusion

• Consistently proven an effective way to improve
  retrieval
• Biggest problem is getting users to engage in the
  interaction, especially if no highly relevant
  documents are in the initially retrieved set

18
Thesauri
19
Improving Recall and/or Precision

• If you get too few documents
• Use more general terms in the query
• Use automobile instead of drop head coupe
• Use an alternative term which is more common
• Use car rather than automobile
• If you get too many (overall)
• Use a more specific term
• Use hot hatch rather than car

20
Thesaurus example
Automobile
Car
Hot Hatch
Drop head coupe
21
Conclusions

• Ranked Retrieval
• similarity matching
• Relevance Feedback
• positive and negative feedback
• Thesauri