Abe Lederman, President and CTO

1
Federated Search The Technology For Making
Global Discovery a Reality

• Abe Lederman, President and CTO
  
• Deep Web Technologies
• 2008 STIP Working Meeting, April 23, 2008

2
Who We Are

• Founded in 2002
• Headquartered in Santa Fe, New Mexico
• 20 Employees
• Over 2 million in RD through DOE SBIR program

DWT is focused on providing state-of-the-art
federated search products and solutions which
search, retrieve, aggregate and analyze content
from web-based databases.
3
Abe Lederman-Background

• Earned B.S. and M.S. Computer Science degrees,
  MIT
• Began work in information retrieval in 1988,
  Verity
• Consulted to LANL, 1994-2000
• Developed some of the first web-based
  applications that searched text-based content,
  1994
• Pioneered Deep Web searching in 1999, OSTI

4
Some of Deep Webs Customers

• DOE Office of Scientific and Technical
  Information
• Department of Defense
• National Agricultural Library
• Scitopia.org
• The Intel Corporate Library
• Stanford University

5
DWT-OSTI Partnership1999 - Present

6
What is Federated Search?

• Federated Search is an application or service
  that allows a user to submit a search in parallel
  to multiple, distributed information sources and
  retrieve aggregated, ranked and de-duped results.

7
In Other WordsOne Search, Many Sources
Subscription Sources
Journal Articles
Technical Reports
Conference Proceedings
Library Resources
Public Web Sources
8
Benefits of Federated Search

• One-stop access to multiple information sources
• Users dont need to know where/how to search
• Saves researcher time and money
• Improves utilization of information sources
• Consolidated, ranked and de-duped results
• Important results are not missed

9
Speeding up the Diffusion of New Scientific
Knowledge
Global Discovery
2
10
Knowledge Diffusion in Action
Mathematicians Scientific Discovery
Math Community

• Biology Databases
• Research Papers
• Correspondence
• Conferences

Biology Researchers Scientific Discovery
Biology Community

• Physics Databases
• Research Papers
• Correspondence
• Conferences

Physics Scientific Discovery
Physics Community
11
Challenges of Global Discovery

• Identify, configure and maintain access to
  reliable information sources
• Automatically select sources to search
• Perform hundreds or thousands of searches in
  parallel
• Rank and organize results retrieved

Goal Search 1000 sources by February 2009
12
Nuclear fission
13
(No Transcript)
14
(No Transcript)
15
(No Transcript)
16
Nuclear fission
17
(No Transcript)
18
(No Transcript)
19
Using Federated Search at your DOE Lab
Internal Sources
Workstation
Public sources including Science.gov
worldwidescience.org
Subscription Sources
20
Polar bears
21
(No Transcript)
22
Turning Vision into Reality
Abe Lederman abe_at_deepwebtech.com www.deepwebtech.c
om
Thank you!
11