Title: Advantages of Query Biased Summaries in Information Retrieval by A. Tombros and M. Sanderson
1Advantages of Query Biased Summaries in
Information Retrievalby A. Tombros and M.
Sanderson
- Presenters
- Omer Erdil Albayrak
- Bilge Koroglu
2Outline
- Brief Introduction to Typical Information
Retrieval System - Why query biased summaries needed?
- How a query biased summary generated?
- Experimental Environment
- Experimental Results
- Conclusion and Future Works
3Introduction
- Typical IR System
- input information need
- output ranked document list
Figure 1. Screenshot from an IR System
4Introduction (cont...)
- Deciding whether the retrieved document is worth
to further investigation - Examining the static summaries
- Checking the whole content
- Weak content indicators static summaries
- Time consuming job refering full text nearly all
times - Aiming to minimise reaching whole content
- Generating query biased summaries
5Generating Query Biased Summaries
- Query-specific summaries for each retrieved
document - Classical approach to summarization
- Extracting sentences
- Assigning scores to sentences
- Selection of best-scoring sentences
- Some modification on score assignment
- Extra importance to titles and subtitles
- More weights to sentences with clusters of terms
- Additional points to sentences includes query
terms
6Experimental Environment
- TREC test collection Wall Street Journal news
- 50 queries of which relevant documents are known
- 2 groups of 10 postgraduate students to find
relevant docs for 50 queries - One group with static summaries
- Another is to use query biased summaries
- 50 retrieved docs per each query
- 5 queries per student
- 5 minutes allocated per query
- Identical computers in hardware/software aspects
7Experimental Environment (cont...)
Figure 2. The subjects performing on query biased
static summaries in the experiments
8Experimental Results
- Recall the ratio of total number of relevant
documents for query to the number of retrieved
relevant documents - Precision the ratio of total number of retrieved
documents for query to the number of retrieved
relevant documents
Figure 5. Precision values of 2 groups
Figure 4. Recall values of 2 groups
9Experimental Results (cont...)
- Speed of evaluators judgments of relevancy
- 2.62 on 20 documents corresponds 13 increase of
average number of examined documents
Figure 6. The number of doc percentage for 2
group
10Experimental Results (cont...)
- The need for checking full text
- Average number of full text reference per query
- query biased summary group 0.3
- static summary group 4.74
Figure 7. Average number of references to the
full text of documents per query
11Conclusion and Future Works
- Effective method employing query biased
summaries in IR Systems - Easily identifiable more relevant documents
- Decreasing the need to check whole content of
document - Applicable to web search engines
- Expensive to retrieve the documenst from slow,
not reliable, and remote servers - Requiring to manage an index file
- More experiments on different summarization
methods with different datasets
12