Exploring ClientSide Instrumentation for Personalized Search Intent Inference: Preliminary Experimen

1
Exploring Client-Side Instrumentation for
Personalized Search Intent InferencePreliminary
Experiments

• Qi Guo and Eugene Agichtein
• Intelligent Information Access Lab
• Mathematics Computer Science

2
Outline

• Introduction
• CSIP Client-side Intent Predictor
• Evaluation of CSIP

3
Outline

• Introduction
• CSIP Client-side Intent Predictor
• Evaluation of CSIP

4
Introduction

• Recovering user intent is an important yet
  difficult problem
• Traditional methods typically model a single
  intent for the same query
• Navigational/Informational/Transactional
• However user goals can vary a great deal

5
An example, query obama

• Informational People may search to know more
  about Barak Obama
• Navigational visit his official website
• Transactional perhaps the user goal is to donate
  money online to support Mr. Obamas campaign

6
Introduction (cont.)

• Incorrect to classify the query into a single
  intent
• What really necessary is to classify user goals
  for each query instance

7
Goals

• Infer Personalized intent for each query instance
  using Client-side instrumentation
• Therefore, provide tailored user experience
• Focus
• mouse movements
• Query intent classification into
  navigational/informational/transactional

8
Outline

• Introduction
• CSIP Client-side Intent Predictor
• Evaluation of CSIP

9
Outline

• Introduction
• CSIP Client-side Intent Predictor
• Evaluation of CSIP

10
CSIP Client-side Intent Predictor

• Capture as much information as possible
• Model implicit user feedback based on the
  real-time interactions
• Keep light-weight and scalable

11
CSIP Client-side instrumentation

• Implementation within the LibX Toolbar
  (http//www.libx.org)
• JavaScript code to track real-time interactions
  (eg. mouse movements)
• Installed on public-use shared machines in Emory
  University Libraries
• All participated users opted in, and no directly
  identifiable information was stored

12
Our approach Learning to recover intent

• Represent full client-side interactions as
  feature vectors
• Apply standard machine learning classification
  methods

13
CSIP System Overview
Figure1. Overview of CSIP
14
CSIP Query text

• Traditional feature for inferring user intent
• Query length

15
CSIP Other User/Server-Side Clickthrough
Features

• Click distribution
• Average deliberation time
• Similarity between a clicked search result URL
  and the query

16
CSIP Real-Time Interaction/Client-side
instrumentation

• Focus on the mouse movements
• 1. CS Client Simple
• 2. CF Client Full

17
CS Client Simple
Horizontal range

• First representation
• Trajectory length
• Horizontal range
• Vertical range

Trajectory length
Vertical range
18
CF Client Full

• Second representation
• 5 segments
• initial, early, middle, late, and end
• Each segment
• speed, acceleration, rotation, slope, etc.

1
2
3
4
5
19
Outline

• Introduction
• CSIP Client-side Intent Predictor
• Evaluation of CSIP

20
Outline

• Introduction
• CSIP Client-side Intent Predictor
• Evaluation of CSIP

21
Experimental Setup

• Dataset
• Gathered from mid-January 2008 until mid-March
  2008 from the public-used machines in Emory
  University libraries.
• Consist of 1500 initial query instances/search
  sessions
• Randomly sample 300 initial query instances
• Behavioral pattern for follow-up queries might be
  different

22
Creating Truth Labels

• Difficulty no identifiable user information
• How to recover the Truth?
• Use our best guess based on clues
• Query terms
• Next URL (eg. clicked result)
• How user behaves before click/exit

23
Navigational query facebook
24
Informational query spanish wine
25
Transactional query integrator
26
Intent Statistics in Labeled Sample
27
Task 1 Classify a query instance into
Navigational / Informational / Transactional
CSIP gt CF gtgt CS gt S
28
Task 2 same, but not distinguish
betweenTransactional and Navigational queries
All improved. Still, CSIP gt CF gtgt CS gt S
29
Most Important CSIP features
30
Error Analysis

• CSIP can help identify
• Relatively rare navigational queries (re-finding
  queries or queries for obscure websites)
• Informational queries that resemble navigational
  queries (coincides with a name of a website)

31
Summary

• Presented CSIP, a practical lightweight
  client-side instrumentation for web search
• Demonstrated the feasibility by the experiments
  with real user interactions
• Conducted preliminary result analysis exploring
  the benefits of client-side vs. server-side
  instrumentation

32
Future Work

• Incorporate user history modeling
• Develop tailored machine-learning algorithms
• Apply our methods to other tasks such as
  predicting user satisfaction or query performance

33
Thank you!

• More information
• http//ir.mathcs.emory.edu/intent/

34
Related Work

• The origins of user modeling research can be
  traced to library and information science
  research of the 1980s.
• Previous research on user behavior modeling for
  web search focused on aggregated behavior of
  users to improve web search

35
Related Work (cont.)

• Previous studies have primarily focused on
  indicators such as clickthrough to disambiguate
  queries and recover intent and model user goals.
• Recently, eye tracking has started to emerge as a
  useful technology for understanding some of the
  mechanisms behind user behavior
• Correlation between eye movements and mouse
  movements