SmartSearch: A Voice Sensing Personalized Mobile Web Search Application - PowerPoint PPT Presentation


PPT – SmartSearch: A Voice Sensing Personalized Mobile Web Search Application PowerPoint presentation | free to download - id: 3fcdb7-YmQ3Y


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation

SmartSearch: A Voice Sensing Personalized Mobile Web Search Application


SmartSearch: A Voice Sensing Personalized Mobile Web Search Application Team: Abilash Bhanoori, Amit Chaube, Sumit Shrivastava Date-11/24/2011 Mobile Computing Fall 2010 – PowerPoint PPT presentation

Number of Views:133
Avg rating:3.0/5.0
Slides: 51
Provided by: Own2293
Learn more at:


Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: SmartSearch: A Voice Sensing Personalized Mobile Web Search Application

SmartSearch A Voice Sensing Personalized Mobile
Web Search Application
  • Team
  • Abilash Bhanoori, Amit Chaube, Sumit Shrivastava

  • SmartSearch is a convenient and efficient way to
    search the web using mobile phones.
  • Supports two input modes, i.e. voice and keypad
  • Maintains User Profile.
  • Its a meta search engine which uses results from
    Google, Yahoo, and Bing.
  • Personalizes the results using the clickthrough
    history, and users preference profile.
  • Concept Video http//

  • How to efficiently convert Speech to Text.
  • How to classify users?
  • How to create and maintain user profile?
  • How to get the meaning of what the user is trying
    to search?

Significance to Mobile Computing
  • Mobile phones based Web Search Application.
  • Usage convenience provided by voice enabled
  • Context Awareness provided by personalization of
  • Extracting and Maintaining Session becomes easy
    as mobile devices are personal to the users.

Personalized Web Search with Location Preferences
  • Authors
  • Kenneth Wai-Ting Leung, Dik Lun Lee, Wang-Chien

Presented By Amit Kumar Chaube
  • The Problem
  • Proposed Solution in brief
  • Ontologies
  • Concept and Entropy
  • Query and User Clustering
  • Personalized Ranking Functions
  • Important Formulae
  • Evaluation and Analysis
  • Conclusions
  • Criticism

Personalized Web Search with Location Preferences
(ICDE 2010)
The Problem
  • As the amount of Web information grows rapidly,
    search engines must be able to retrieve
    information according to the user's preference.
  • The interaction between users and mobile devices
    are constrained by the small form factors of the
    mobile devices.
  • Different classes of users and queries having
    different emphases on content and location

Personalized Web Search with Location Preferences
(ICDE 2010)
Proposed Solution
Personalized Web Search with Location Preferences
(ICDE 2010)
Proposed Solution (Cont)
  • Reranking
  • When a user submits a query, the search results
    are obtained from the backend search engines
    (e.g.,Google, MSNSearch, and Yahoo). The search
    results are combined and reranked according to
    the user's profile trained from the user's
    previous search activities.
  • Profile Updating
  • After the search results are obtained from the
    backend search engines, the content and location
    concepts (i.e. important terms and phrases) and
    their relationships are mined online from the
    search results and stored, respectively, as
    content ontology and location ontology.
  • When the user clicks on a search result, the
    clicked result together with its associated
    content and location concepts are stored in the
    user's clickthrough data. The content and
    location ontologies, along with the clickthrough
    data, are then employed in RSVM training to
    obtain a content weight vector and a location
    weight vector for reranking the search results
    for the user.

Personalized Web Search with Location Preferences
(ICDE 2010)
  • Authors propose an ontology-based, multi-facet
    (OMF) framework, in which concepts can be
    classified into different types, such as content
    concepts, location concepts, name entities, dates
  • A content concept, like a keyword or key-phrase
    in a Web page, defines the content of the page,
    whereas a location concept refers to a physical
    location related to the page.

Personalized Web Search with Location Preferences
(ICDE 2010)
Concept and Entropy
  • For a given query issued by a particular user, if
    the personalization based on content concepts is
    more effective than based on location concepts,
    more weight should be put on content-based
    preference and vice versa.
  • Content and location entropies are used for
    measuring the diversity of content and location
    information from the search results of a query.
  • The click content and location entropies are used
    to determine how much a user is interested in the
    content and location information associated with
    a query.
  • A query result set with high content/location
    entropy indicates that it has a high degree of
  • If the click content/location entropies are low,
    the personalization effectiveness would be high
    because the user has a focus on certain precise
    topic in the search results.

Personalized Web Search with Location Preferences
(ICDE 2010)
Concept and Entropy (Cont)
Personalized Web Search with Location Preferences
(ICDE 2010)
Query and User Clustering
  • By using K-Means clustering the test queries are
    classified into four classes
  • Explicit Queries Queries with low degree of
    ambiguity, i.e., they have small total content
    and location entropies.
  • Content Queries Location specific Queries.
  • Location Queries Content centric Queries.
  • Ambiguous Queries Queries with high degree of
    ambiguity, i.e., they have large total content
    and location entropies.
  • Users are divided into four user classes as
  • Very Focused Users with low content and location
    entropies, i.e. they have very clear topic
    focuses in the search results, and only click on
    a few topics.
  • Focused Users with higher content and location
    entropies and hence less focused than the Very
    Focused class.
  • Diversified Users with even higher content and
    location entropies and hence more diversified
    topical interests than the first two user
  • Very Diversified Users with high content and
    location entropies they click on many topics.
    These users can be considered novice search
    engine users.

Personalized Web Search with Location Preferences
(ICDE 2010)
User Preferences Extraction
  • Joachims Method
  • A user would scan the search result list from top
    to bottom.
  • If a user skips a document dj at rank j but
    clicks on document di at rank i where j lt i,
    he/she must have read dj 's web snippet and
    decided to skip it. Thus the user prefers di to
    document dj
  • SpyNB Method
  • Users would only click on documents that are of
    interest to them. Thus, it is reasonable to treat
    the clicked documents as positive samples.
  • Predict from the unlabeled set reliable negative
    documents which are irrelevant to the user. To do
    this, the spy technique incorporates a novel
    voting procedure into Naive Bayes classier.

Personalized Web Search with Location Preferences
(ICDE 2010)
Personalized Ranking Functions
  • A set of content concepts and a set of location
    concepts are extracted from the search result as
    the document features.
  • Since each document can be represented by a
    feature vector, it can be treated as a point in
    the feature space.
  • The feature vectors are extracted by taking into
    account the concepts existing in a documents and
    other related concepts in the ontology of the
  • The similarity and parent-child relationships of
    the concepts in the extracted concept ontologies
    are also incorporated in the training based on
    the following four different types of
  • (1) Similarity, (2) Ancestor, (3) Descendant, and
    (4) Sibling

Personalized Web Search with Location Preferences
(ICDE 2010)
Personalized Ranking Functions (Cont)
Personalized Web Search with Location Preferences
(ICDE 2010)
Important Formulae
  • To measure the interestingness of a particular
    keyword/phrase ci with respect to the query q
  • Where sf(ci) is the snippet frequency of the
    keyword/phrase ci (i.e. the number of
    web-snippets containing ci), n is the number of
    web-snippets returned and ci is the number of
    terms in the keyword/phrase ci.
  • To compute the content and location entropies of
    a query q (i.e. HC(q) and HL(q))
  • Equations to estimate the personalization
    effectiveness using the extracted content and
    location concepts with respect to the user u.

  • Using a metasearch engine which comprises Google,
    MSNSearch and Yahoo as the backend search engines
    to ensure a broad topical coverage of the search
  • Joachims(Content) method performs the best on
    content queries. It boosts the top 1, 10, and 20
    precisions of content queries from 0.4583,
    0.3563, and 0.3125 to 0.7519, 0.5874, and 0.4176
    (64, 65, and 34 in percentage gain), comparing
    to the baseline method.
  • Joachims(Location) method performs the best on
    location queries, boosting the top 1, 10, and 20
    precisions of location queries from 0.5208,
    0.4063, and 0.3563 to 0.6989, 0.4269, and 0.3583
    (34, 5, and 0.5 in percentage gain).
  • Using SpyNB for preference extraction performs
    better than using Joachims' method in all classes
    of queries, because SpyNB generates more accurate
    preferences comparing to Joachims' method.

Personalized Web Search with Location Preferences
(ICDE 2010)
  • In this paper, an Ontology-Based, Multi-Facet
    (OMF) personalization framework has been proposed
    for automatically extracting and learning a
    user's content and location preferences based on
    the user's clickthrough.
  • The notion of content and location entropies are
    introduced to measure the diversity of content
    and location information associated with a query
    and and click content and location entropies to
    capture the breadth of the user's interests in
    these two types of information.
  • Experimental results confirmed that OMF can
    provide more accurate personalized results
    comparing to the existing methods.

Personalized Web Search with Location Preferences
(ICDE 2010)
  • Memory and space issues are completely ignored.
  • Testing of the solution is required by large
    number of random users (ignorant to the
    implementation) to validate the efficiency of the

Personalized Web Search with Location Preferences
(ICDE 2010)
Survey of The Speech Recognition Techniques for
Mobile Devices
  • Author
  • Dmitry Zaykovskiy

Presented By Abilash Bhanoori
  • Introduction
  • Basic Methodology of ASR Systems
  • Mobile ASR Dilemma
  • ASR architectures
  • Conclusion
  • Criticism

  • This paper presents an overview of different
    approaches for providing automatic speech
    recognition technology to mobile users.
  • Why do we need Automatic Speech Recognition (ASR)
    in mobile devices?
  • i) The basic problem of handheld gadgets is
    their miniature
  • size.
  • ii) Typing on such tiny keyboards or pointing
    with the stylus is
  • very uncomfortable error prone.
  • E.g. PDA are often used when a person is
    really on the move.
  • Operating in such conditions is very

Basics of Automatic Speech Recognition
  • Goal Finding most probable sequence of words W
    (w1,w2,w3..) belonging to a fixed vocabulary
    given some set of acoustic observations O
  • Calculating Best estimation for the word sequence
    (Using Bayes Theorem)
  • In order to generate an output the speech
    recognizer has to basically
  • perform the following operations
  • i) Extracting acoustic observations
    (features) out of the spoken utterance.
  • ii) Estimating P(W)- probability of
    individual word sequence to happen,
  • regardless of acoustic observations.
  • iii) Estimating P(O/W) the likelihood that
    the particular set of features
  • originates from certain sequence of
  • iv) Find word sequence that delivers maximum
    of above equation.
  • SPECOM2006

The Mobile ASR Dilemma
  • The implementation of effective efficient
    mobile ASR systems is challenged by many border
    conditions . In contrast to the generic ASR, the
    mobile recognition system has to encounter the
    following aspects
  • i) Limited available storage volume.
  • ii) Tiny cache of 8-32KB and small slow
    memory from
  • 1MB up to 32 MB.
  • iii) Low processor clock frequency.
  • iv) Cheap Microphones.
  • v) Highly challenging acoustic environments.

System Configurations for Mobile Speech
  • ASR systems can be structurally decomposed into
    two parts namely
  • i) Acoustic front-end where the feature
    extraction takes place.
  • ii) Acoustic back-end where Viterbi search
    is implemented based on the
  • acoustic language models.
  • Based upon the location of the front-end
    back-end mobile ASR systems can be classified
    into three principal system structures
  • i) Client Based architecture or embedded ASR,
    where both front-end
  • backend are implemented on the
  • ii) Server Based or network speech recognition
    (NSR), where speech
  • is transmitted over communication
    channel and the recognition is
  • performed on the remote server.
  • iii) Client-Server ASR or DSR , where the
    features are calculated on the
  • terminal, while the classification is
    done on the server side.

Embedded Speech Recognition
  • Entire process of speech recognition is performed
    on the terminal device.
  • Figure 1 Embedded Speech
    Recognition Architecture
  • Embedded ASR is often the architecture of the
    choice for PDAs for the following reasons
  • i) PDAs are driven well under well
    established operating systems.
  • ii) PDAs have well known processor
    architectures, e.g. Intel XScale.
  • iii) PDAs dont always have a wireless
    communication link available ,hence
  • remote connection is rather unwelcome for

  • Advantage
  • No communication between the server and the
    client is needed. Hence ASR system is always
    ready for use doesnt depend on quality of data
  • Disadvantage
  • Embedded ASR systems has very limited system
    resources on the mobile devices.
  • For effective implementation of embedded ASR two
    important characteristics have to be considered
  • i) Memory Usage.
  • ii) Execution Speed.

Network Speech Recognition (NSR)
  • Figure 2 ASR features extracted from the
    transcoded speech (Initial Version)
  • Advantages
  • i) One basic advantage over Embedded ASR is
    that in NSR all complications caused by the
    resource limitations of the mobile devices can be
    avoided shifting both ASR front-end and back-end
    from the terminal to the remote server.

  • ii) Unlike the embedded ASR, the NSR
    architecture can not only augment not only PDAs
    but also thin terminals ,e.g. cellular phones ,
    with a very large vocabulary ASR.
  • iii) NSR can provide access to the recognizers
    based on the different grammars or even different
  • Disadvantage
  • Performance degradation of the recognizer
    caused by using low bit rate codecs, which become
    more severe in presence of data transmission
    errors and background noise.

Improved Version of NSR Architecture
  • Distortion introduced from source coding in
    previous version can be diluted to certain extent
    if the recognizer is trained on the respectively
    corrupted speech. However, grouping of different
    source coding schemes in addition to the
    different channel noise levels spans too large
    number of possible acoustic models.
  • In the improved version recognition is performed
    based on the features derived from the parametric
    representation of the encoded speech with out the
    actual speech reconstruction as shown in the
    figure below.
  • Figure 3 ASR features derived from
    speech codec parameters

Distributed Speech Recognition (DSR)
  • DSR represents the client-server architecture,
    where one part of the ASR system , primary
    features ,resides on the client, while the
    computation of temporal derivatives and the ASR
    search are performed on the remote server as
    shown in figure below.
  • Figure 4 Client-Server based
    ASR system

Advantages of DSR over NSR
  • Even though both DSR NSR make use of the server
    based back-end, there are substantial
    differences in these two schemes favoring DSR.
    Following are advantages of DSR over NSR
  • i) Speech codecs unlike the feature
    extraction algorithms are optimized to deliver
    the best perceptual quality and not for providing
  • ii) DSR doesnt need high quality speech,
    but rather some set of characteristic parameters.
    Thus it requires lower data rate.
  • iii) In DSR unlike NSR we are not
    constrained to the error mitigation algorithm of
    the speech codecs, hence better handling methods
    in terms of WER can be developed.

  • Medium Recognition tasks having 1000-2000 words,
    which represent good coverage of the certain
    application domain will be successfully running
    on the terminal devices like PDAs or in car
    embedded systems.
  • DSR can be used to implement for high data bit
    rate networks. Because of its superior
    performance in presence of the transmission
    errors and surrounding noise it is expected that
    NSR will be totally supplanted by the DSR
    architecture in the near future.

  • This paper gives brief introduction of different
    architectures but fails to analyze them in
  • Only Trivial advantages or very basic advantages
    have been mentioned of ASR systems.

Context-Aware Ranking in Web Search
  • Authors

Presented By Sumit Shrivastava
ACM SIGIR conference on Research and development
in information retrieval 2010
  • Mobile phones screen is very small and cannot
    accommodate much record.
  • What if the records re-ranked so that desired
    records get higher ranking.
  • How can we take advantage of different types of
    contexts in ranking?
  • How can we integrate context information into a
    ranking model?
  • Current search engines dont consider context in

  • Maintains log of previous search queries,
    returned result and clicked results.
  • Assumes inspection of returned results
    sequentially from top to bottom.
  • User is looking for car. He searches BMW.
  • His second query is jaguar.
  • Jaguar could be a bird or could be a car model.

  • Responsible for promoting or demoting documents
    according to the context of the current query.
  • Studies say a search result is likely to be
    viewed by a user if it is
  • a) among the top two search results.
  • b) ranked above the lowest clicked result or
  • c) ranked one position below the lowest clicked

  • Unrelated query.
  • Reformulating.
  • Specializing.
  • Generalizing.
  • Associated with previous query in session.
  • There can not be any kind of prediction for
    Unrelated queries.

  • A user may reformulate her previous query into a
    new one because the search results for the
    previous one do not or only partially fulfill
    her information need.
  • Example
  • 1) first query house on rent.
  • 2) next query home on rent.

  • When a user issues a specializing query, she
    likely wants to see results that are more
    specific about her interests.
  • Example
  • a) The user first asked time life music and
    clicked on the homepage of the store.
  • b) The user further asked time life Christian
    CDs and clicked on the 4th and 5th results.

  • When a query (especially an ambiguous one) is
    generally associated with its context, the
    context may help to narrow down the users search
  • Example
  • a. First query Xbox 360.
  • b. Second query Fifa 2010.
  • If both queries are related. Second query is
    meant for fifa 2010 game on Xbox.

  • A user may ask a query more general than the
    previous one
  • In such a situation, the user may like to see
    some information not covered by the first query.
  • Example
  • a) A user first asked query free online Tetris
    game and clicked on the 1st and 2nd search
  • b) The user then asked query Tetris game and
    clicked on the 3rd and 4th results.

  • Data were collected from major search engines.
  • Session boundary was 30 minutes idle time.
  • Experiment was done on query pairs qn and qn-1.
  • Relations were labeled manually.

  • We considered only consecutive pair of queries.
  • More than one principals can be applied at a
  • There could be some more useful contexts like
    positions of documents returned by queries or
    shared terms between current and previous query.

  • Learning to rank approach to build model. rankSVM
    model is used.
  • In training it takes ordered pair of document
    with respect to query under context.
  • A feature in this model is a function of query,
    document, and context.

  • MCP is mean click position.
  • Less MCP better performance.
  • Compared with baseline method.

  • we studied the problem of using context
    information in ranking documents in Web search.
  • Conducted an empirical study on real search logs
    and developed four principles for context-aware
  • adopted a learning-to-rank approach and
    incorporated our principles to ranking models.
  • The experimental results verified the
    effectiveness of our approach.

  • Thank You!