SunGard Higher Education

1
SunGard Higher Education

• The Case for a Search Channel in Luminis
• Dr Francisco Queirós Pinto

2
Agenda

• Context of this Presentation
• The Case for a Search Channel in Luminis
• Search, Technologies Targets
• Information Indexing, Search Retrieval
• Educational Data Sources
• Existing Search Features in Luminis Portal
• Meta-Search/Federated Search
• Luminis Platform and its Portal
• Architecture, Common and Educational Features
• A Meta-Search Channel
• Meta-Search Channel Prototype
• Externally Internally Integration Approaches
• Summary
• Questions Answers

3
Context of this Presentation
4
The Case for a Search Channel in Luminis

• The Case
• Many existing and prospective customers have
  expectations for a Search Channel/Portlet
  available in Luminis. However, when asked for
  details about the functionality they would like
  to see on this search tool, the response is very
  vague.
• Some Requirements
• At the one extreme, some customers are happy to
  see as the potential search target, Luminis
  dynamically generated Web resources, based on
  information located internally. Some other
  customers go further and expect the search target
  to be any kind of resources of interest for their
  users (e.g. Intranet/Extranet, Library,
  Business). At the other extreme, some customers
  would like to have a kind of Google engine for
  any kind of resources located anywhere on the
  Internet.

5
The Case for a Search Channel in Luminis

• Background Information
• Finding a tool that meets all of these
  requirements is virtually impossible as it
  involves many search tools and complex
  technologies. The truth is that no Search Holly
  Grail exists and only a combination of different
  search solutions, wrapped by a Meta-Search
  interface, possibly complemented by Single
  Sign-On, could provide a satisfactory response
  for all kind of customer expectations.
• Possible Solution
• In this presentation I intend to talk how a
  search tool materialised as a Meta Search
  Channel/Portlet, could provide a trade-off for
  all customers expectations with any kind of
  search requirements.

6
Search
7
Search

• Well Know Paradigm
• Seeking Relevant Information
• Doesnt Live in Isolation, it Depends On
• Technologies
• Indexing
• Enable Efficient and Effective Search
• Search
• Seek Relevant Information
• Retrieval
• Obtain the Information Resources Found (or their
  Surrogates)
• Targets
• Information Space Where Relevant Resources are
  Stored
• Complements Browse Paradigm
• When Information Space is Too Large

8
Search Technologies
9
Indexing

• Data Structure
• Enable Fast Access to Large Amounts of
  Information
• Indexing of the Targets Information Space is
  Required For
• Efficient and Effective Search Retrieval
• Equivalent to the Indexing that exists in
  Databases
• RDBMS, ORDBMS, .., LDAP
• Basically, What Web Spiders/Crawlers/Robots Do
• Google Co.
• Access New Horizons
• The Deep Web Robot goes the Further it Offers
• The Hidden/Invisible/Deep Web
• Your Hard-Drive (e.g. Google Desktop)
• Applications/Systems (e.g. Intranet)
• Research Resources (e.g. PubMed, Google Scholar)

10
Search Retrieval Protocols

• Z39.50 (ISO 23950) SRU/SRW
• Library Environments
• Integrated Library Systems
• Interlibrary Loan Systems
• Personal Bibliographic Reference Systems
• OpenSearch
• HTTP/XML/RSS
• A9/Amazon
• Recognised by Major Browsers
• Search the Web
• HTTP
• Google Co.

11
Query Languages

• Query Languages
• Z39.50
• ASN.1 Type-1 Reverse Polish Notation (RPN) Query
  Syntax
• SRW/SRU
• Contextual Query Language (CQL)
• OpenSearch
• XQuery, XML Query Language
• Web/HTTP
• Everything seen as a Keyword
• Google Co.

12
Search Targets
13
Data Sources

• Any Information Available Online
• Structured Information
• Databases
• Well Defined Schema
• Semi-Structured Information
• Parts of the Web
• Requires Metadata with Well Known Semantics
• XML/DC
• Unstructured Information
• All the Rest
• HTML Pages, Text Files, Images, Video, Etc.
• Brute-Force Applied for Indexing
• (Very) Intelligent Algorithms for Searching
• Provides Access to Resources (or their Surrogates)

14
Educational Data Sources

• Learning, Teaching, Research Administration
• Library Resources
• Licensed Resources
• eJournals, Catalogues, Databases, References, Etc
• British Library Library of Congress
• Many Others
• Intranet
• Institutional Applications/Systems
• WebMail, VLE, Digital Repository (e.g. DSpace,
  Fedora, ePrints), CMS, Etc
• Content Spread over several RDBMS(s), LDAP
  Directorie(s) and File System(s)
• Only Some Parts are Useful, Accessible and
  Publically Available
• Internet
• Mainly the Web
• Again, only Some Parts are Useful
• And Accessible

15
Educational Data Sources

• Learning, Teaching, Research Administration
• Library Resources
• Typically, Structured Information
• High Precision Search Retrieval
• Intranet
• Typically Mixed Content From Structured to
  Unstructured Information
• Mostly Accessible via Browsing
• Search Retrieval Possible
• Indexing Tools might be Employed (e.g. Google
  Box)
• Privacy Considerations
• Internet
• Searchable via Google Co.
• Poor Precision, but

16
Educational Data Sources

• Luminis
• Basically, the Same as for the Intranet
• Data Sources
• RDBMS
• Bulking Content
• LDAP Directory
• High-Level Content (e.g. Users, Roles, Groups)
• File System
• External Content (e.g. Uploaded Files Images)
• Indexing
• RDBMS Indexing
• LDAP Indexes (e.g. Users IDs - uid)
• Search Retrieval
• Some via de Portal
• 3rd Party Tools (e.g. Google, Lucene, Verity
  Ultraseek)

17
Educational Data Sources

• Existing Search Features in Luminis Portal
• Potential Search Targets
• Multiple Data Sources
• Data in the LDAP Database
• Objects in the Directory
• Files in the File System
• Search Technologies
• Internal
• UI
• External
• Command Line (e.g. cptool, configman, os, apps)
• Web Services (e.g. TA SOAP API)

18
Meta-Search/Federated Search

• Search Retrieval Over Several Targets in
  Parallel
• Application Level Using Existing Technologies
• User Submits a Query
• Application Translates and Launches the Query
• Over One or More Targets in Parallel
• Different Query per Target, Using their Native
  Idiom
• Protocol and Query Language
• Collects the Results
• Ideally Removes any possible Repetitions (a.k.a.
  de-dup)
• Presents them Showing (or not) the Target(s)
  where the Resource was Found
• Homogeneous Data, Format and UI for Heterogeneous
  Targets
• Wider Coverage
• Possibly at the Cost of Precision

19
Luminis Platform and its Portal
20
Luminis Platform Architecture
Web Server (powered w/ a servlet engine for
Java applications)
Luminis Platform
Luminis Portal (uPortal)


Calendar Server (optional)
Messaging Server (optional might be external)
Directory Server
Message Broker Server
21
Common Features

• Single Point of Entry
• Coherent UI
• Rich Presentation Features
• Layouts, Tabs, Columns Channels
• Personalisation
• Aggregation
• Integration
• Good IAM Citizen
• AuthN
• Single Sign-On (SSO)
• AuthZ

22
Educational Features

• Targeted Content for Selected Audiences
• Role-Based Layouts
• Targeted Channels
• Targeted Announcements
• Collaboration and Communication Applications
• Group Studio
• Course Studio
• Enterprise Applications
• Messaging, Calendar and Directory Servers
• Integration with Institutional ERP System and
  Applications/Systems
• Message Broker/Queue Server
• Data-Level Integration via LDIS
• Presentation-Level Integration (a.k.a. SSO) via
  GCF

23
A Meta-Search Channel

• Considerations for Integration
• Meta-Search against Several Targets
• Externally Integrated
• Presentation-Level via an Existing Channel Type
• Inline Frame
• Takes Advantage of Existing Meta-Search
  Applications
• Data NOT Available for the Portal Framework
• Extremely Simple
• Internally Integrated
• Programming-Level
• Technologies Required Depend on the Targets
  Involved
• Data Available for the Portal Framework
• Significant Complexity
• Might Require SSO for Protected Targets

24
A Meta-Search Channel Architecture
Luminis Portal
Luminis Platform
Meta-Search Channel
Channel 1
Channel n
25
Meta-Search Channel Prototype
26
Externally Integrated (Presentation-Level)

• A Very Simple Meta-Search Web Application
• Functionality
• Java Server Pages (JSP)
• Presentation Transformation
• Not Required, External Application
• Development Environment
• Not Required, Configuration-Level
• Deployment
• Not Required, Configuration-Level
• Publishing
• My Search Channel (Inline Frame Channel Type)
• Subscribing
• Search Tab (My Search Channel)
• Rendering
• Search Tab

27
Externally Integrated (Presentation-Level)

• Target 1 Luminis File System File System
  Shared Files

28
Externally Integrated (Presentation-Level)

• Target 1 Luminis File System File System
  Resource

29
Externally Integrated (Presentation-Level)

• Target 1 Luminis File System Lucene Index
  Phase

30
Externally Integrated (Presentation-Level)

• Target 1 Luminis File System LuceneWeb
  Search Query

31
Externally Integrated (Presentation-Level)

• Target 1 Luminis File System LuceneWeb
  Search Results

32
Externally Integrated (Presentation-Level)

• Target 1 Luminis File System LuceneWeb
  Resource

33
Externally Integrated (Presentation-Level)

• Target 2 Google Scholar

34
Externally Integrated (Presentation-Level)

• Target 3 Google

35
Externally Integrated (Presentation-Level)

• A Very Simple Meta-Search Web Application Code

36
Externally Integrated (Presentation-Level)

• A Very Simple Meta-Search Web Application
  Search Query

37
Externally Integrated (Presentation-Level)

• A Very Simple Meta-Search Web Application
  Search Result

38
Externally Integrated (Presentation-Level)

• A Very Simple Meta-Search Web Application
  Search Result

39
Externally Integrated (Presentation-Level)

• A Very Simple Meta-Search Web Application
  Search Result

40
Externally Integrated (Presentation-Level)

• Publishing the Channel

41
Externally Integrated (Presentation-Level)

• Subscribing the Channel

42
Externally Integrated (Presentation-Level)

• Showing the Result of the Same Search Query via
  the Portal

43
Externally Integrated (Presentation-Level)

• Showing the Result of the Same Search Query via
  the Portal

44
Externally Integrated (Presentation-Level)

• Showing the Result of the Same Search Query via
  the Portal

45
Externally Integrated (Presentation-Level)

• Showing the Result of the Same Search Query via
  the Portal

46
Internally Integrated (Programming Level)

• A Very Simple Meta-Search Web Application
• Functionality
• Java
• Presentation Transformation
• XML Stylesheet Language Transformation (XSL)
• Development Environment
• Ant Build Tool
• Deployment
• Channel Archives (CARs)
• Publishing
• Search Channel (Custom Channel Type)
• Subscribing
• All Users Sample Tab (Search Channel)
• Rendering
• All Users Sample Tab

47
Internally Integrated (Programming Level)

• Channel Development Java Code

48
Internally Integrated (Programming Level)

• Channel Development Stylesheet Transformation

49
Internally Integrated (Programming Level)

• Development Environment - Ant

50
Internally Integrated (Programming Level)

• Development Environment Ant Env

51
Internally Integrated (Programming Level)

• Development Environment Ant List

52
Internally Integrated (Programming Level)

• Development Environment Ant List

53
Internally Integrated (Programming Level)

• Development Environment Deployment via CARs
  Ant Car

54
Internally Integrated (Programming Level)

• Development Environment Deployment via CARs
  Ant Car

55
Internally Integrated (Programming Level)

• Development Environment Deployment via CARs
  Car File

56
Internally Integrated (Programming Level)

• Development Environment Deployment via CARs
  Car File Contents

57
Internally Integrated (Programming Level)

• Development Environment Re-Start Web Server

58
Internally Integrated (Programming Level)

• Publishing Meta-Search Channel

59
Internally Integrated (Programming Level)

• Publishing Meta-Search Channel

60
Internally Integrated (Programming Level)

• Subscribing Meta-Search Channel

61
Internally Integrated (Programming Level)

• Subscribing Meta-Search Channel

62
Internally Integrated (Programming Level)

• Rendering Meta-Search Channel in the Portal

63
Summary
64
Summary

• Meta-Search Channel
• Offers the most Flexible Answer for All Customer
  Requirements
• Can Take Advantage of Existing Investments on
  Search Technologies, Applications and Licenses
• Targets Can be Added on a Needed Basis
• Channel can be Internally or Externally
  Integrated
• Parallel Search
• Coherent UI
• Target Location and Other Details can be Hidden
• Meta-Search Channel
• Could Initially be Externally Integrated
• Short-Term
• Gradually become Internally Integrated
• Mid-, Long-Term

65
Questions Answers

• ?

66
Thank You!Francisco PintoProfessional
ServicesSunGard Higher Educationfrancisco.pinto_at_
sungardhe.com