GLORP Generic Lightweight ObjectRelational Persistence Demonstration

1
GLORPGeneric Lightweight Object-Relational
PersistenceDemonstration

• Alan Knight
• knight_at_acm.org

2
About this Demo

• Smalltalk object-relational tool
• Very boring to demo
• Lots of slides
• Implementation-focused

3
Outline

• What is GLORP
• Goals
• Concepts
• Features
• Nifty Implementation Stuff
• Plans
• Questions

4
Goals

• Illustrate O/R issues
• Simplicity
• Rich Feature Set
• Free (LGPL)

5
History

• TOPLink-influenced
• Started in March 2000 (still very young)
• Camp Smalltalk in San Diego
• Attempt to kick-start open source Smalltalk
  development
• Project sponsored by The Object People

6
Supported Platforms

• Few platform dependencies
• Smalltalk Dialects
• Developed in VisualWorks, EXDI drivers
• Some work in VisualAge, Squeak, Dolphin
• Operating systems any
• Databases
• Oracle
• Postgresql

7
Concepts and Architecture

• Non-intrusive
• Object model (no foreign keys in objects,
  object-level queries, normal ST objects)
• Database model
• Development process
• Metadata-driven
• Queries in terms of objects

8
Concepts and Architecture 2

• Rich mappings
• classes to multipletables
• multiple objects in a row
• complex keys
• Transactional
• no explicit writes
• defer and re-order writes

9
Terminology

• Single broker (Session)
• Descriptors and Mappings
• Unit of Work (object transaction)

10
Implementation Themes

• Full and separate models of object and relational
  worlds
• Real objects everywhere, with identity
• Particularly clever bits
• block queries and expressions
• row unification for writes
• rollback/undo

11
Features

• Caching and Identity Preservation
• Based on primary key of the first table

12
Mappings

• Direct
• OneToOne
• normal or reversed foreign keys
• OneToMany
• no back-reference required
• ManyToMany
• intermediate link table
• EmbeddedValue

13
Mapping Example

• table self tableNamed 'PERSON'.
• aDescriptor table table.
• aDescriptor addMapping (
• DirectMapping from id to (table fieldNamed
  'ID')).
• aDescriptor addMapping (OneToOneMapping new
• attributeName address
• referenceClass Address
• mappingCriteria (PrimaryKeyExpression
• from (table fieldNamed 'ADDRESS_ID')
• to ((self tableNamed 'GR_ADDRESS')
• fieldNamed 'ID'))).

14
Writes

• Transactional
• beginUnitOfWork
• modify objects
• commitUnitOfWork
• Nestable (real soon now)
• All writes deferred to the end
• re-order writes for integrity constraints,
  deadlock avoidance, optimizations

15
Mechanics of Writes

• Two-phase
• create rows in memory from objects
• write rows to database
• RowMap
• set of dictionaries (one per table)
• keys are the domain objectcs
• values are rows

16
Writes

• Find all the objects in the unit of work
• For each mapping in each object, apply that
  mapping
• Direct Mapping write the value into the row
• Reference Mapping impose an equality constraint

17
Write Example
a
Order id14 itembook
b
Customer id 8 nameAlan
CUST
ID NAME
ORDER
c
Order id39 itemgadget
ID ITEM CUST_ID
letters indicate object identity
18
- write (random) b, a, c - write b - write as
name, orders - write as id
CUST a -gt ROW ID(constraint) NAMEAlan
ORDER b -gtRow ID14 ITEMbook
CUST_ID(constraint) c -gtRow
CUST_ID(constraint)
19
- write (random) b, a, c - write b - write as
name, orders - write as id - write c
CUST a -gt ROW ID8 NAMEAlan
ORDER b -gtRow ID14 ITEMbook
CUST_ID8 c -gtRow ID39 ITEMgadget
CUST_ID8
20
What Just Happened?

• Writing a reference mapping imposes an equality
  constraint
• Writing a value into any of the constrained
  columns sets the value for all of them
• All rows are computed before any are written
• Orders dont need to know the Customer
• Mapping code becomes very, very simple

21
Recall the Mapping

• OneToOneMapping new
• attributeName address
• referenceClass Address
• mappingCriteria (PrimaryKeyExpression
• from (table fieldNamed 'ADDRESS_ID')
• to ((self tableNamed 'GR_ADDRESS')
• fieldNamed 'ID'))).

22
Expressions

• PrimaryKeyExpression
• Expresses one or more field equalities
• This foreign key must equal that primary key
• General expressions
• Parse tree representation
• Usually built from blocks
• cust cust address city Minneapolis.

23
Block Queries

• session readManyOf Customer
• where cust cust lastOrder price gt 1000.
• Queries expressed at the object level
• Subset of blocks
• Pass in a dNU handler as block argument
• Result is effectively a parse tree
• Generate SQL

24
Joins

• Traversing an object relationship implies a join
• PrimaryKeyExpression tells us the foreign key
  correspondence
• AND clauses onto the core expression

25
Advanced Queries

• Queries can also be constructed on-the-fly
• name isEmpty ifFalse
• expression expression _and (
• cust(cust name name)(cust name The ,
  name) asGlorpExpressionOn base).
• Queries can have data-level components
• cust(cust getFieldNamed ROW_TYPE) 3.
• All GLORP SQL is generated through the expression
  mechanism. Anything it can do can be done at the
  user level.

26
Performance

• Low-level
• static SQL
• parameterized queries ()
• parameterized queries with bound parameters
• stored procedures
• Real Soon Now
• () currently implemented

27
Performance (contd)

• High Level
• Lazy Reads ()
• Minimal writes
• Read with joins (single object) ()
• Read with joins (multiple objects) ()
• Partial Reads
• () currently implemented
• () can be specified, implementation real soon
  now

28
Explicit Broker

• Queries addressed to the session
• Single broker rather than per-class factory
• Does not use processes/threads to determine
  transaction
• Very simple model

29
Development Techniques

• Semi-XP
• No customer
• Developers set priorities
• Completely distributed team with occasional
  pairing
• Way too much forethought and cleverness
• But
• Relentless testing
• Merciless refactoring
• Simplest thing that could possibly work

30
Metrics

• 143K of code, 112 K of tests
• 48 classes, 43 test classes
• 5KLOC code, 3.7KLOC tests

31
Future Possibilities

• Parallel transactions (shared read-only cache)
• Transparent transactional state
• More sophisticated queries (outer joins,
  distinct, data queries, anySatisfy, database
  functions)

32
Stale Cache Entries

• Weak references for objects not in use
• Mark stale and referesh when next touched
• Read barrier? code generation, wrap everything
  in proxies
• Flush at end of transaction?
• Flush on optimistic locking failure?
• Handling depends on application

33
Things That Need Work Now

• Inheritance
• Documentation!!!!!!!!!!!
• Broader range of types
• Dialect Portability
• Locking and concurrency

34
Status

• Still very bleeding edge
• Very strong core engine
• Simple (as these things go) and easy to enhance
• Needs some real-world use, but close to ready

35
Acknowledgements

• The Object People/WebGain
• Bruce Badger, Anthony Lander, John-Reed Maffeo,
  Mark Schwenk, Tom Robinson, and David Siegel.
• Ralph Johnson for starting and organizing Camp
  Smalltalk

36
More Information

• glorp.sourceforge.net
• source code, forums (need to sign up)
• www.glorp.org
• Camp Smalltalk Wiki (wiki.cs.uiuc.edu)
• Ask Me (knight_at_acm.org)

37
Additional Slides
38
Object Level Rollback

• If a transaction fails in the database, or the
  user doesnt want to commit, objects must be
  restored.
• Generalized Undo mechanism
• Find all the objects were interested in
• only persistent objects count
• only objects that have been registered count
• transitive closure from registered objects

39
Undo

• For each registered object, make a shallow copy.
• On commit, do nothing
• On rollback, copy the state from the shallow copy
  back into the original
• Thats it.
• Yes, it actually works.

40
Undo Limitations

• Works with originals
• very convenient for developers
• only one unit of work at a time
• servers require a separate session per user
• no sharing of objects for read OR

41
Alternative Mechanisms

• Register objects and get copies
• Mixing originals and copies is very bad.
• Easy to make mistakes, especially with nesting
• Transparent delegation of state
• heavy code generation
• tools require rewrite
• still some loopholes (can pass one transaction an
  object that doesnt exist in that context)

42
Tracings

• Can execute any query with a tracing
• Specifies the graph of objects to be read at the
  same time
• Also expressable as blocks
• person person address
• emp emp manager manager manager

43
aStreamgtgtcollect

• An interesting idiom
• tries to reconcile encapsulation breakage by
  passing back the original vs. cost of passing
  back a copy
• Java idiom of passing back an iterator
• Useful?