Building Robust Heterogeneous XML Pipelines

1
Building Robust Heterogeneous XML Pipelines

• - From Pipelines to XML Processes

2
Background

• Pre-99 led HP Information Commerce Research
• Developed e-payment and e-contract systems
• Characterized by XML messages bound to procedures
• But system economics were not viable because...
• X in XML stands for EXTENSIBLE but means
  CHANGE...
• 1999 established and led HP Lab's XML Research
  Program to develop a generic XML processing
  infrastructure.
• 2002 Founded 1060 Research.
• Talk could have been titled Applying a REST
  Abstraction to XML Processes

3
XML Procedural Mismatch

• Parsing, Serialization,Transformation
• Problems
• DOM, SAX, JDOM... Which object model to choose?
• Thread-safety a serious issue
• Debugging, Exception Handling...
• All are surmountable (with expertise) but still
  the fundamental issueprocedural processing of
  XML produces brittle systems.
• This is analogous to the history of RDBMS
• Early Relational Databases were written as
  procedures brittle/costly/inflexible. SQL
  offered a declarative abstraction that was
  in-step with the relational table model.

4
REST

• REpresentation State Transfer, Roy T. Fielding's
  PhD thesis formalism of the foundations of
  HTTP/WWW.
• Stateless protocol with verbs GET, POST, PUT...
• What's important about REST?
• it's very simple.
• it works the most successful information system
  ever.
• Important System Properties
• URI address space is scale invariant scales
  vertically
• Transparent proxying/caching scales
  horizontally
• Tolerant to version mismatch in client and
  server.
• Robust and adaptive to change.
• XML technology set is implicitly REST-centric

5
NetKernel REST Abstraction

• Software components are services addressed by
  URI.
• Request verbs SOURCE, SINK, DELETE, EXISTS, NEW
• A service's URI is published in an internal URI
  Address Space.
• URI Address Spaces are encapsulated in modules
  which can be layered into Virtual Private Address
  Spaces.
• Software components may issue URI requests
  against the Virtual Private Address Space
• The Web doesn't stop at the edge of the
  Web-server...

6
URI Addressable Services

• Introducing Active URI's..
• activeserviceuriarg1_at_arg1uriarg2_at_arg2uri...
• argX is a URI, can be another active URI!
• URI is a functional programme, lazy evaluation
  etc.
• Make the XML technologies services...
• Example XSLT transform is a software component
  accessed as a URI addressable serviceactivexslt
  operand_at_file/my.xmloperator_at_file/mytransform.x
  sl
• Why is this valuable? XML technology-set assumes
  an implicit URI infrastructure xslinclude,
  document() etc etc...
• Active URI works with all XML techs but not
  limited to XML
• Active URI is abstracted by high-level language
  runtimes...

7
A simple XQuery pipeline

• Pipeline Process
• from Shakespeare's King Lear extract Act1
• from result extract all speeches
• from result extract only speeches by 'Gloucester'
• from result extract speeches containing the word
  'France'
• style result to XHTML
• Illustrative of general XML problems.

8
1. Using Active URIs to Sequence Pipelines

• XQuery pipeline using doc() resource
  references.
• Illustrative only this is a bad design-pattern
  since each XQuery is hard-wired to the next.
• Decouple using DPML - a declarative high-level
  pipeline assembly language.
• One or more DPML ltinstrgt is compiled down to an
  active URI.
• Active URI undergoes lazy evaluation which
  invokes each pipeline stage.
• A service (URI interface) may receive multiple
  arguments so pipelines may be decomposed into
  sub-assemblies.

9
2. Declarative or ProceduralPipeline Sequencing

• Creating pipelines by sequencing services lends
  itself to description by declarative languages.
• But declarative is not always sufficient
  conditions, iterations...
• Treat language Runtimes as services that execute
  programs declarative or procedural it depends
  on the application.

10
3. Robustness - Errors / Testing

• XML technology set is diverse, but...
• No common error handling model.
• Pipeline Exception Handling runtime errors
• Pipeline Debugging development errors
• Pipeline unit testing
• Probe the pipeline
• Voltmeter / Ammeter (S. McGrath)
• Oscilloscope / Logic Analyzers
• Hook in test pipelines

11
4. XML Object Model Heterogeneity

• XML object model choice is subjective.
• Frequently dictated by legacy code/design
  decisions.
• Object model should only concern the service
  implementer
• The pipeline sequencer need not know anything
  about the object model.
• Support all object models equally.
• If necessary for performance, tune pipelines to a
  single object model
• Asynchronous pipeline optimizations no time!!

12
Summary

• REST-based software development
• Active URI decouples a process from each service.
• Unlimited reconfigurability.
• Dynamic object model translation completes the
  decoupling
• Any process is just simple service composition.
• Each service abstracts complexity
• low-level APIs
• other services (important!)
• Language Runtimes are services for scheduling
  services.
• General error handling can be applied to all
  services irrespective of implementing technology.
• XML processing is adaptable which changes the
  economics of XML systems.
• Its fun