HSE304: Improving Relevance and the Search Experience Using the API

1
HSE304 Improving Relevance and the Search
ExperienceUsing the API

• Erik Mau
• Inetium
• emau_at_inetium.com

2
Objectives

• Understand how to impact relevance in MOSS
• Understand how to leverage the API to enhance the
  search user interface

3
Improving Relevance
4
Improving Relevance Topics

• How does MOSS determine relevance?
• How can farm admins impact relevance?
• How can site admins impact relevance?
• MOSS Relevance-related API
• Demo

5
Relevance Defined

• Relevance is a term used to describe how
  pertinent, connected, or applicable something is
  to a given matter. A thing is relevant if it
  serves as a mean to a given purpose.
• Wikipedia http//en.wikipedia.org/wiki/Relevance

6
How is Relevance Determined in MOSS?

• Anchor Text
• lta hrefBenefits.aspxgtBenefits Informationlt/agt

7
How is Relevance Determined in MOSS?

• Anchor Text
• Metadata and Document Properties

8
How is Relevance Determined in MOSS?

• Anchor Text
• Metadata and Document Properties
• Metadata Extraction

9
How is Relevance Determined in MOSS?

• Anchor Text
• Metadata and Document Properties
• Metadata Extraction
• URL Length and URL Matching
• http//portal/
• http//portal/sites/MarketingTeam/Shared20Docs/Do
  c.docx

10
How is Relevance Determined in MOSS?

• Anchor Text
• Metadata and Document Properties
• Metadata Extraction
• URL Length and URL Matching
• File Type Biasing
• Web Pages
• PPT / PPTX
• DOC / DOCX
• XML
• XLS / XLSX

11
How is Relevance Determined in MOSS?

• Anchor Text
• Metadata and Document Properties
• Metadata Extraction
• URL Length and URL Matching
• File Type Biasing
• Click Distance
• Based on Authoritative Pages

12
How is Relevance Determined in MOSS?

• Anchor Text
• Metadata and Document Properties
• Metadata Extraction
• URL Length and URL Matching
• File Type Biasing
• Click Distance
• Text Analysis
• Matching terms, term frequency

13
How can Admins Impact Relevance OOTB?

• Administrators can
• Include / Exclude Crawled Properties

14
How can Admins Impact Relevance OOTB?

• Administrators can
• Include / Exclude Crawled Properties
• Define Managed Properties

15
How can Admins Impact Relevance OOTB?

• Administrators can
• Include / Exclude Crawled Properties
• Define Managed Properties
• Define and Configure Scopes

16
How can Admins Impact Relevance OOTB?

• Administrators can
• Include / Exclude Crawled Properties
• Define Managed Properties
• Define and Configure Scopes
• Control Authoritative Sites and Pages

17
How can Admins Impact Relevance OOTB?

• Administrators can
• Include / Exclude Crawled Properties
• Define Managed Properties
• Define and Configure Scopes
• Control Authoritative Sites and Pages
• Define Keywords / Best Bets

18
How can Admins Impact Relevance OOTB?

• Administrators can
• Include / Exclude Crawled Properties
• Define Managed Properties
• Define and Configure Scopes
• Control Authoritative Sites and Pages
• Define Keywords / Best Bets
• Manage the Thesaurus
• Expansions
• Replacements

19
How can Site Admins Impact Relevance OOTB?

• SharePoint Site Admins can
• Extend Search Centers with tabs

20
How can Site Admins Impact Relevance OOTB?

• SharePoint Site Admins can
• Extend Search Centers with tabs
• Use Scopes

21
How can Site Admins Impact Relevance OOTB?

• SharePoint Site Admins can
• Extend Search Centers with tabs
• Use Scopes
• Use Managed Properties
• Advanced Search
• Search Results XSLT

22
Impacting Relevance The API

• Developers can
• Adjust Managed Property Weightings
• Impact the relevance of content with metadata
  that applies to your organization.
• Set the weight of a ManagedProperty to a float gt
  0

Schema ssp new Schema(SearchContext.GetContext
(ServerContext.Default)) ManagedPropertyCollectio
n properties ssp.AllManagedProperties ManagedPr
operty prop propertiesMyManagedProperty pro
p.Weight weight // float value from 0 to
float.MaxValue prop.Update()
23
Impacting Relevance The API

• Developers can
• Adjust Managed Property Weightings
• Adjust Managed Property Length Normalization
• Impact the relevance based on the amount of text
  contained in the property (i.e. Chapter vs Book)
• Set the length normalization property on a
  ManagedProperty to a value between 0 and 1.

Schema ssp new Schema(SearchContext.GetContext
(ServerContext.Default)) ManagedPropertyCollectio
n properties ssp.AllManagedProperties ManagedPr
operty prop propertiesMyManagedProperty pro
p.LengthNormalization len // float value from
0 to 1 prop.Update()
24
Impacting Relevance The API

• Developers can
• Adjust Managed Property Weightings
• Adjust Managed Property Length Normalization
• Remove a Crawled Property from the Index
• Remove unnecessary / irrelevant metadata from the
  content index.

Schema ssp new Schema(SearchContext.GetContext(S
erverContext.Default)) IEnumerable properties
ssp.QueryCrawledProperties(name, 1000,
Guid.NewGuid(), string.Empty, true) CrawledProper
ty prop (from CrawledProperty p in properties
where p.Name name select p).First() prop.IsMap
pedToContents isMapped // boolean prop.Update()

25
Impacting Relevance The API

• Developers can
• Adjust Managed Property Weightings
• Adjust Managed Property Length Normalization
• Modify Ranking Parameters
• Provides developers with the ability to impact
  file type ranking, term frequency, click
  distance, etc.

Ranking rank new Ranking(SearchContext.GetContex
t(ServerContext.Default)) RankingParameter rp
rank.RankingParametersrankingParamName rp.Value
value // float value rank.StartRankingUpdate(R
ankingUpdateType.FullUpdate)
26
Ranking Parameters
27
Relevance API Summary

• Acquiring Search Context
• SearchContext ctx SearchContext.GetContext(Serve
  rContext.Default)
• Schema ssp new Schema(ctx)
• Working with Managed Properties
• ManagedPropertyCollection properties
  ssp.AllManagedProperties
• Working with Ranking Parameters
• SearchContext ctx SearchContext.GetContext(Serve
  rContext.Default)
• Ranking rank new Ranking(ctx)
• RankingParameter rp rank.RankingParametersfile
  typepriordoc

28
Relevance Demos

• OOTB Relevance Configuration Options
• Extending the Platform for Search Administrators
• Trimming Content from the Indexer

29
Improving the Search Experience
30
Improving Search Experience Topics

• OOTB Capabilities
• Community-based Solutions
• MOSS Search API
• Demo

31
Out-of-the-box Capabilities

• Web Parts
• Keyword Entry
• Paging
• Summary
• Statistics
• Keywords / Best Bets
• High Confidence
• Core Results

32
Community-based Solutions

• Wildcard Search
• Search as You Type
• Faceted Search
• SPAdvancedSearch

http//www.codeplex.com/FacetedSearch
http//www.codeplex.com/SPAdvancedSearch
33
Core Search API

• KeywordQuery
• Allows developers to easily leverage search
  capabilities
• Uses Keyword syntax (i.e. AuthorErik
  SharePoint)
• FullTextSqlQuery
• Allows developers to fully customize the search
• Uses Enterprise Search SQL Syntax
• Web Services
• Allows developers to use search in applications
  outside of SharePoint
• Can use Keyword or FullText

34
Query Classes
35
KeywordQuery

• Microsoft.Office.Server.Search.Query.KeywordQuery
• Constructed with an SPSite (or ServerContext)
• Properties of importance
• SelectProperties (columns SELECT clause)
• QueryText (keywords WHERE clause)
• SortList (columns ORDER BY clause)
• StartRow
• RowLimit
• ResultTypes

36
KeywordQuery

• Returns a ResultTableCollection
• Contains ResultTables (ResultTable implements
  IDataReader)
• Query Syntax

37
KeywordQuery Example

• KeywordQuery query new KeywordQuery(SPContext.C
  urrent.Site)
• query.RowLimit 10
• query.ResultTypes ResultType.RelevantResults
• query.SelectProperties.Add("Title") // select
• query.QueryText finch" // where
• query.SortList.Add("Rank", SortDirection.Descendi
  ng) // order by
• ResultTableCollection results query.Execute()
• ResultTable relevantResults resultsResultType.
  RelevantResults

38
FullTextSqlQuery

• Microsoft.Office.Server.Search.Query.FullTextSqlQu
  ery
• Inherits from Query base class (just like
  KeywordQuery)
• Constructed with an SPSite (or ServerContext)
• Properties of importance
• QueryText (SQL Syntax)
• SELECT FROM SCOPE() WHERE ORDER BY
• StartRow
• RowLimit
• ResultTypes

39
FullTextSqlQuery Example

• FullTextSqlQuery query new FullTextSqlQuery(SP
  Context.Current.Site)
• query.RowLimit 10
• query.QueryText SELECT Rank, Title, Path,
  Habitat
• FROM SCOPE()
• WHERE FREETEXT(defaultproperties, finch)
• ORDER BY Rank DESC"
• query.ResultTypes ResultType.RelevantResults
• ResultTableCollection results query.Execute()
• ResultTable relevantResults resultsResultType.
  RelevantResults

40
Web Service

• http//server/_vti_bin/Search.asmx
• Methods of Importance
• Query(string queryPacketXml) Returns Xml String
• QueryEx(string queryPacketXml) Returns DataSet
• GetSearchMetadata() Returns DataSet of Scopes
  and Managed Properties
• Tips
• Query Packet Schema http//msdn.microsoft.com/en-
  us/library/ms563775.aspx
• Can use Keyword or MSSQLFT syntax

41
Search User Interface Demos

• Wildcard Search Results Web Part
• Related Searches Web Part
• Advanced Search Web Part

42
Wildcard Search

• Search UI Demo

43
Related Searches

• Search UI Demo

44
Advanced Search

• Search UI Demo

45
Recap

• Understand how to impact relevance using OOTB
  features
• Carefully test relevance changes
• Roll your own search user interface using
  KeywordQuery, FullTextQuery, or other aspects of
  the SharePoint API
• Be creative!

46
Resources

• Search Admin Toolkit http//www.codeplex.com/SPSe
  archAdminToolkit
• SPAdvancedSearch http//www.codeplex.com/SPAdvanc
  edSearch
• SPSearchBench Query Testing http//www.codeplex.
  com/SPSearchBench
• Faceted Search http//www.codeplex.com/FacetedSea
  rch
• SQL Search Syntax http//msdn.microsoft.com/en-us
  /library/ms519321.aspx
• Evaluating and Customizing Relevance
  http//msdn.microsoft.com/en-us/library/bb499682.a
  spx

47
Your Feedback is Important

• Please fill out a session evaluation form and
  either put them in the basket near the exit or
  drop them off at the conference registration
  desk.
• Thank you!