701 - Building with the Data Tools project

1
701 - Building with the Data Tools project

• John Graham
• Der Ping Chou

2
Data Tools Platform

• DTP
• New top-level project at Eclipse
• Managed by Sybase, IBM, and Actuate
• Additional contributions
• Versant
• ObjectNation GmbH
• Pervasive Software
• Broad community support

3
Who We Are

• John Graham
• DTP PMC
• Sybase

• Der Ping Chou
• WTP Committer
• DTP
• IBM

4
Outline

• Overview of DTP
• Model Base project detail
• Case study Demonstration of extending the base
  models
• Questions

5
Purpose of DTP

• Provide a comprehensive set of
• extensible data frameworks
• exemplary tools
• for data-centric applications

6
Why DTP?

• Data is fundamental to software applications
• Known set of requirements based on years of
  experience
• Unify the approach for Eclipse
• Enable higher-level data applications

7
Scope of DTP

• PMC worked very hard on this section
• The Scope section was very difficult
• Data is used everywhere
• How to draw the line between what naturally
  belongs in DTP and what does not?
• Settled on a test based on problem domain

8
Scope Test

• Data-centric applications
• Associated with a data source
• Having a problem domain no more specific than
  just data within the Eclipse ecosystem of
  projects
• Data-dependent applications
• Built on data-centric frameworks and tools
• Have a more specific problem domain in the
  Eclipse ecosystem of projects

9
Example In Scope

• SQL Editor
• Associated with a data source
• SQL statement itself
• Database system using the statement
• Problem domain
• Generic (relational) data operations
• Not naturally in scope for other, existing
  Eclipse projects

10
Example Out of Scope

• Java source code editor
• Uses data (source code)
• Problem domain is Java tooling
• Eclipse has the Java Development Tools (JDT)
  project
• So, not in scope for DTP, because the more
  specific JDT problem domain includes it

11
DTP Structure

• As a top-level project at Eclipse
• DTP project
• (Sub)project
• Component
• DTP is managed by the PMC
• A project is managed by a project lead
• A component has a technical lead
• All volunteers, all open source

12
Current Data Tooling

• Web Tools Platform (WTP) project has an rdb
  component
• Business Intelligence and Reporting Tools Project
  (BIRT)
• Eclipse plug-ins, both commercial and open source

13
Consolidation in DTP

• rdb moving components to DTP
• BIRT providing Open Data Access (ODA) to DTP
• Sybase open sourcing connectivity and SQL tooling
  for DTP
• Donations from ObjectNation GmbH
• Other areas under discussion

14
Possible Future Components

• In support of Object/Relational Mapping (ORM)
• Versant, Oracle have projects at Eclipse in this
  area
• Working with them, and others, to insure that DTP
  meets the communitys data requirements
• XML/Relational Mapping
• Administration
• Extract-Transform-Load (ETL)

15
First DTP Release

• Concentrating on relational structures
• Milestones planned for
• 2 in 5Q4
• 2 in 6Q1
• Release in late 6Q1
• Project plan draft posted on DTP site

16
Milestone Components
17
DTP Architecture
18
DTP Projects

• Model Base
• Connectivity
• SQL Development Tools

19
Model Base Project

• Provides foundational domain models for DTP and
  DTP consumers, based on Eclipse Modeling
  Framework (EMF) and using a model-driven
  development with UML

20
Model Base Benefits

• Support for change management
• Broad editing support visual and otherwise
• Extensible, database agnostic models
• Compliance with latest standards, such as SQL
• Supporting JDBC and other connection standards

21
Model Base Components

• Driver Definition model
• SQL model (99/03)
• Database Definition model
• SQL Query model

22
Connectivity Project

• Provides foundational driver and connection
  support for data-driven application build in
  Eclipse.

23
Connectivity Benefits

• One place to define drivers
• One place to define (configure) connections
• Unified view of data sources
• Unified view of data

24
Connectivity Components

• Driver Management
• Connection Profiles
• Data Source Explorer
• JDBC Connection Support
• Open Data Access

25
Driver Management

• Allows for driver management in Eclipse
• Template-based
• Generic
• Extensible

26
Connection Profiles

• Abstraction of connection
• Configuration details
• Generic, not just for databases
• Extensible for
• Types
• Content

27
Data Source Explorer

• Exemplary code to demonstrating how to
• Populate the view from the SQL Model.
• Use the Navigator View framework to display
  database in Database Explorer (WTP)

28
Data Source Explorer cont..

• Use the DDL Generator to generate DDL
• Extract from and load data into a relational
  table

29
Data Source Explorer cont..

• Filter support on database elements schema,
  table, view

30
Data Source Explorer cont..

• Connect, disconnect, execute/launch SQL Scrapbook
  and work offline

31
Data Source Explorer cont..

• Sample, open/update, extract from and load data
  into a relational table

32
JDBC Connection Support

• Exemplary code that demonstrates
• Supporting a generic JDBC catalog loader
• Extending the wizard framework to support
  multiple drivers for a single db vendor
• Specialization of the catalog loading mechanism

33
Open Data Access (ODA)

• Previously in BIRT project
• Provides abstraction over data, like Connection
  Profile does for connections
• Define data drivers for data types
• Meta-data access support

34
SQL Dev Tools

• Provides a set of frameworks and exemplary tools
  for using SQL.

35
SQL Dev Tools Benefits

• How many text editors do you have to write?
• How many SQL editors does Eclipse need?
• How about visualization of SQL?
• Provide a common SQL Query Model framework
• Extension of SQL Model
• Detailed EMF model representation of SQL query
• Covers DML Select, Insert, Update, Delete
• Based on SQL- 2003 with vendor-specific
  extensions
• UML Model -gt EMF Utilities -gt Generated Code
• Extensible SQL source generation
• Support a SQL DML parser
• Based on LPG parser generator
• LPG is also used to create the Eclipse Java
  parser
• Targets SQL Query Model directly as parse result
• Uses Eclipse extension-point mechanism to support
  multiple SQL dialects

36
SQL Query Model/ParserArchitecture
SQL Parser
SQL
Client
37
SQL Dev Tools Components

• Frameworks for
• Routines
• SQL Query parser
• SQL Execution Plan

38
SQL Dev Tools Components

• Exemplary tools
• SQL Editor
• Visual SQL Editor
• Results View
• Script History

39
Routines

• A routine is
• Stored procedure
• Function
• Generic, vendor-neutral
• Creation, editing, saving
• Execution
• Debugging

40
SQL Query Parser

• Complies with SQL 03
• Based on LPG parser generator
• LPG is also used to create the Eclipse Java
  parser
• Targets SQL Query Model directly as parse result
• Uses Eclipse extension-point mechanism to support
  multiple SQL dialects

41
SQL Execution Plan

• Execution Plans important for
• Understanding database performance
• Tuning queries
• Framework enables
• Generic access to execution plans
• Ability to visualize in text and (later, perhaps)
  graphical means

42
SQL Editor

• Eclipse text editor for SQL
• Syntax colorization
• Code assist
• Templates
• Syntax checks and feedback
• Dialect specialization through underlying SQL
  Query model

43
Visual SQL Editor

• Graphical representation of SQL statements
• Ability to construct statements without worrying
  (as much) about SQL syntax
• As with SQL Editor, dialect specialization
  supported
• Smaller brother of Visual Query Builder notion
  floated earlier in DTP

44
Results View

• Display results of SQL execution
• Uses a Connection Profile instance
• Uses a SQL statement
• Sorting, filtering of view
• Export of results

45
Script History

• Stores queries executed during an Eclipse session
• Allows for running queries multiple times
• Thinking about filters based on Connection
  Profile instances
• Thinking about persistence across platform
  instances

46
In Depth

• The Model Base Project

47
Model Base Themes

• Using UML to model SQL Model and Database
  Definition Model, SQL Query Model to support EMF
  for DTP
• Published Model
• http//www.eclipse.org/webtools/wst/components/rd
  b/WebPublishedSQLModel/SQLModel.htm
• http//www.eclipse.org/webtools/wst/components/rd
  b/WebPublishedDBDefinitionModel/DBDefinition.htm

48
Model Base Themes - 1

• Tightly integrated and loosely coupled
• Support the latest SQL Standard

49
Model Base Themes - 2

• Exemplary application built on top of the model
  to exhibit generic implementation vs. Vendor
  specific extension

50
Model Base Themes - 3

• Minimum support vs. rich database element support
• Future model base upgrade

51
Case Study
Supporting richer database features in DTP model
framework
52
Case Study

• Adding new database element support to SQL Model
• Change the SQLModel (Rose model) file to add your
  database element
• Use the Eclipse framework to generate a new EMF
  model
• Generate code to create a new Database Definition
  Model
• Generate db vendor document using the provided
  utility
• Provide databaseDefinition and uri_mapping
  extension points a database definition plugin
• Extend database catalog loader
• ltextension point"org.eclipse.datatools.connecti
  vity.core.catalog"gt
• Adding class file to handle newly added database
  elements
• Extend database DDL Generator
• ltextension point"org.eclipse.datatools.connectiv
  ity.core.ddlGeneration"gt
• Adding database UI plug-in to support Vendor
  specific JDBC connection
• ltextension point "org.eclipse.datatools.connect
  ivity.connection.ui.connectionUIProvider"gt

53
Contacting DTP

• Web site www.eclipse.org/datatools
• Newsgroup news//news.eclipse.org/eclipse.dtp
• Mailing lists www.eclipse.org/mail/main.htmldata
  tools
• Wiki http//eclipsewiki.editme.com/DataTools
• Conferences
• EclipseWorld
• EclipseCon

54
Questions?