ICONS Access Algorithms and Data Structures Underlying a Distributed Knowledge Base

1
ICONS Access Algorithms and Data Structures
Underlying a Distributed Knowledge Base 

• Witold Litwin Gérard Levy
• CERIA
• University Paris 9 Dauphine

2
ICONS data management needs
3
(No Transcript)
4
The ICONS reference architecture
D18
5
ICONS Storage System Needs

• Fast Access Performance !!!!!
• Crucial factor to the industrial failure of
  previous ambitious prototypes
• LDL, ECRC
• Dynamic Data
• Size unknown in advance or rapidly growing
• Large Volume Data
• Gbytes, Tbytes
• Scalability
• No performance deterioration
• When data size grows
• When there are more ICONS apps (clients)
• High Availability
• Knowledge Data are a precious resource
• ICONS Apps may need them 24/7

6
ICONS Storage SystemResponse ICONS-SDDS Storage

• Based on SDDS-2000 V.1 system
• Permanent temporary data in large and scalable
  volumes
• In the form of records with keys
• Ground atoms with OIDs, Rules, Relational tuples,
  SBQL Datalog stack elements, XML, RDF pages with
  URL as the key, semantic index records.
• Provides distributed RAM for processing
• Multicomputer storage
• Rapid access to data
• Record access time currently 200-300 faster than
  to a disk record
• 25-35 ?s on 1 Gbs Ethernet 1.8 GHz PCs
• Client / server architecture
• SDDS client handles the application on the app.
  node
• SDDS server stores data at its node

7
SDDS-2000 global architecture
Applications
Applications
Applications
etc
UDP TCP
8
ICONS Storage System(SDDS-2000)

• Horizontal scalability transparent for the
  application
• Each overloaded server splits and half or its
  content move to a new server
• Record insert or search times do not deteriorate
• Automatic storage and CPU load sharing
• Under the usual random access pattern assumption
• Disk backup of RAM data
• Using algebraic digital signatures
• Scalable high-availability
• Based on erasure correcting codes
• Protection against a multiple server failure
  scales with the file
• Transparently for the ICONS apps.

9
ICONS Storage System(SDDS-2000)

• At least two scalable distributed data
  partitioning schemes
• Hash (LH)
• Range (RP)
• Dedicated management of relational data
• SD-SQL Server prototype
• Available in 3-6 months

10
Technical Description

• Rather complex
• Presented for ICONS in
• D 18
• Previous deliverables
• Based also on several published articles and
  research reports
• Referred to or recalled in D 18
• Available in extenso at http//ceria.dauphine.fr/
• Also with SDDS-2000 V1 working proto version
• Available for ICONS community at CERIA site above

11
Conclusion

• ICONS should dispose of powerful data storage
  capabilities
• Very fast access
• Much faster than to a disk storage
• 5 h of searches or inserts reduce to 1 min
• 5 min of searches or inserts reduce to 1 sec
• Scalability to very large volumes
• Transparent load sharing
• High-availability

12
Conclusion

• All these capabilities are under the development
• About as initially planned
• The performance they should provide is among the
  keys to ICONS success
• Remember the unfortunate experience of the others

13
THANK YOU FOR YOUR ATTENTION