Software Engineering Challenges Web Services and SOAP

1
Software Engineering ChallengesWeb Services and
SOAP

• Richard Taylor Eric Dashofy
• UC Irvine
• http//www.isr.uci.edu/classes/ics123s02/

2
Traditional Software Systems

• are developed by a single organization
• undergo a phased development process
• with multiple phases of review, inspection,
  testing
• have a synchronized release schedule
• have a proprietary design and proprietary
  component interfaces
• have a monolithic code base
• go through a painful evolution

3
Our Vision

• The future of software development requires
  integratingnetwork services that are very far
  away and owned by strangers

Rohit Khare 16 May 2002 OReilly Emerging
Technology ConfUC Irvine KnowNow, Inc.
4
Three Elements of Our Vision

• The abstract unit of software is a network
  service
• Prior levels of abstractions were subroutines,
  libraries, processes, objects,
• In this view, the only way to understand software
  is to watch its message traffic
• Latency is an absolute limit of system
  architecture
• Just as real architects grapple with gravity,
  so with the speed of light
• London will always be 30ms from New York,
  regardless of Moores Law
• Decentralization means crossing agency boundaries
• The essential complication above and beyond
  distributed (parallel) computing is extending a
  system to achieve consensus between several
  separate agencies
• Every computing and communication device is owned
  by someone.

5
How Our Vision is Emerging Today

• The technological shift from a component model to
  a service model of software is already underway
• SOAP and Web Services technologies make it easy
  to write new protocols
• The Internet is inexorably shifting to
  higher-latency, decentralized connectivity
• Ad-hoc, peer-to-peer networking, mainly wireless,
  is being deployed at every scale from one meter
  (handhelds) to billions of meters (space probes)
• Enterprise Application Integration (EAI) is only
  now reaching outside the firewall to business
  partners
• The next revolution in productivity requires
  reengineering processes end-to-end

6
So Whats This Talk About?

• There is a missing link between the near-future
  predicted by todays Web Services hype and
  realizing the true potential of our vision
• It is a new architectural style for software
  integration we call Application-Layer
  Internetworking (ALIN)
• ALIN applies networking concepts to software
  engineering
• The primary way that developers will use ALIN is
  an unheralded implication of SOAP 1.2 Routing
• so this talk is about a new style and a new
  device

7
Whats EAI?

1. Enterprise Application Integration is the
   unrestricted sharing of data and business
   processes throughout the networked applications
   or data sources in an organization
2. The Linking of Data, Business Processes,
   Applications that result in a common virtual
   system

8
Whats EAI?

• Enterprise Application Integration is real-time
  integration of the information systems within an
  organization that support the different steps of
  a business process
• Consider the following typical business process
• An order might be captured by
• A CSR using an order entry system
• A consumer using a web application
• A business partner system using a web service
  interface
• Regardless of the channel, the order must be
  financially and operational processed
• The financial system processes the billing
• The operational system processes the
  manufacturing, scheduling, provisioning,
  clearance, etc. depending upon industry.

9
Without EAI
AR
Internal
Credit
AP
External
Logistics

• Numerous Point-to-Point Interfaces
• Manual Hand-Offs
• Web Applications Increase the Number of
  Enterprise Integration Points
• Inconsistent Business Processes
• Inconsistent Qualities of Service

ECommerce
External
Inventory
10
Without EAI

• Numerous Point-to-Point Interfaces
• Manual Hand-Offs
• Web Applications Increase the Number of
  Enterprise Integration Points
• Inconsistent Business Processes
• Inconsistent Qualities of Service

11
With EAI
EAI Infrastructure
AR
Business Process Automation
Internal
Credit
Portal
AP
Data Transfor-mation
Logistics
External
ECommerce
Web Service
Message Routing
Inventory
External
Message Storage
Adapter
12
With EAI

• Reduced Number of Common, Consistent, Known
  Interfaces
• Consistently Automated Business Processes
• Increased Quality of Service (e.g. Order can
  Still be Captured even when Financial and
  Operational systems are Unavailable)
• Framework for Integration of Web Services

13
Components in an EAI Solution

• Adapters
• Also known as Connectors, Translators,
  Wrappers
• Provide seamless connectivity to the undrelying
  application or data
• Convert information and events into data that can
  be utilized by the infrastructure
• Two way translation between the proprietary lingo
  and the common lingo
• Many adapters available off the shelf from EAI
  Software vendors such as Tibco, or Packaged
  Software vendors such as SAP.

EAI Infrastructure
Business Process Automation
Data Transformation
Message Routing
Message Storage
14
Components in an EAI Solution

• Message Broker
• An intelligent broker that directs the flow of
  messages between sources and consumers of
  information
• Message-Oriented Middleware (MOM)
• A set of products that connect applications
  running on different systems by sending and
  receiving application data as messages
• Message Routing
• Set of tools that route messages between sources
  and consumers based on pre-defined business rules
• Message Storage
• Also identified as Message Warehousing
• Central repository for temporary storage of
  transactions
• Message Queuing
• asynchronous communications style and provides a
  loosely coupled exchange across multiple
  operating systems
• Publish Subscribe

EAI Infrastructure
Business Process Automation
Data Transformation
Message Routing
Message Storage
15
Components in an EAI Solution

• Data Transformers
• Mapping of data elements to achieve common
  denominator
• Transformation to and from Formats Used by
  Different Systems
• GUI Configuration
• Handles simple or complex data structures

EAI Infrastructure
Business Process Automation
Data Transformation
Message Routing
Message Storage
16
Components in an EAI Solution

• Process Modeling
• Often GUI based
• Describing the flow of information in the context
  of business processes
• Using the input/output of processes as the
  Integration points between diverse business
  processes
• Process Brokering
• Execution of discrete steps within a business
  process
• Ability to recover from failed steps
• Workflow engine
• Process Management
• Monitor business processes
• Correlate metrics to specific business process
  steps

EAI Infrastructure
Business Process Automation
Data Transformation
Message Routing
Message Storage
17
EAI Power Tools

• Many vendors have off the shelf connector
  technology that allows connectivity and
  integration of existing software packages and
  data stores.
• GUI Transformation tools allow for visual
  creation and maintenance and versioning of data
  transformations.
• Many MOMs (Message Oriented Middleware) already
  support automatic SOAP transformations
• Many software packages are committed to SOAP as a
  standard which would make the EAI piece
  transparent

18
EAI The Next Level Business Webs
While EAI focuses on connecting data and
applications internally, Business Webs focus on
connecting applications and business flows across
organizational boundries
19
Current Condition
Transportation

• Paperwork intensive
• Human Intervention
• Inherent inefficiencies

20
The Challenge
The Future
Distribution Channels
1000s
100s
10s
1
100s
1000s
10,000s
1
10s
Suppliers
21
Relationships Are Dynamic.
Each eBusiness relationship is unique. Every
eBusiness initiative integrates disparate
systems. Every change requires a skilled
programmer.
Time Cost
1
5
20
100
1,000
of unique eBusiness relationships
22
What are Web Services?

• A Web service is a collection of functions that
  are packaged as a single entity and published to
  the network for use by other programs.
• Web services are building blocks for creating
  open distributed systems, and allow companies and
  individuals to quickly and cheaply make their
  digital assets available worldwide.
• A Web service can aggregate other Web services to
  provide a higher-level set of features

23
What are Web Services?

• Based on UDDI (Universal Description, Discovery
  and Integration) and XML
• Platform-independent, open framework for
  describing services, discovering businesses, and
  integrating business services.

24
Web Services Strengths

• Interoperability
• Based on open standards, utilizes existing
  infrastructure
• Ubiquity
• Communicates through XML/HTTP Any system that
  supports these standards, supports Web Services
• Low barrier to entry
• Concepts easy to understand, easy to implement
• Toolkits allow COM, J2EE, and CORBA components to
  be exposed as Web Services
• Industry Support

25
Web Services Layers
Publish, Find, Use Services UDDI
Service Interactions SOAP
Universal Data Format XML
Communications Layer HTTP
26
SOAP- Definition

• Simple Object Access Protocol
• W3C standard Not proprietary to any one software
  vendor
• Like CORBA and COM, designed as an
  interoperability protocol that allows objects to
  talk to each other
• SOAP defines a simple mechanism for expressing
  application semantics by providing a modular
  packaging model and encoding mechanisms for
  encoding data within modules

27
SOAP- Structure

• Objects can be functions, programs, or systems
• Three parts to SOAP
• Envelope Defines a framework for describing what
  is in a message and how to process it,
• Encoding rules Expressing instances of
  application-defined datatypes,
• RPC Representation Convention for representing
  remote procedure calls and responses.

28
SOAP Example

• Example 1 Sample SOAP Message embedded in an
  HTTP Request
• POST /StockQuote HTTP/1.1
• Host www.stockquoteserver.com
• Content-Type text/xml charset"utf-8"
• Content-Length nnnn
• SOAPAction "http//example.org/2001/06/quotes"
• ltenvEnvelope xmlnsenv"http//www.w3.org/2001/06
  /soap-envelope" gt
• ltenvBodygt
• ltmGetLastTradePrice
• envencodingStyle"http//www.w3.org/2001/
  06/soap-encoding"
• xmlnsm"http//example.org/2001/06/quotes
  "gt
• ltsymbolgtDISlt/symbolgt
• lt/mGetLastTradePricegt
• lt/envBodygt
• lt/envEnvelopegt

29
Example, continued

• Example 2 shows the SOAP message sent by the
  StockQuote service in the corresponding HTTP
  response to the request from Example 1.
• HTTP/1.1 200 OK
• Content-Type text/xml charset"utf-8"
• Content-Length nnnn
• ltenvEnvelope xmlnsenv"http//www.w3.org/2001/06
  /soap-envelope" gt
• ltenvBodygt
• ltmGetLastTradePriceResponse
• envencodingStyle"http//www.w3.org/2001/
  06/soap-encoding"
• xmlnsm"http//example.org/2001/06/quotes
  "gt
• ltPricegt34.5lt/Pricegt
• lt/mGetLastTradePriceResponsegt
• lt/envBodygt
• lt/envEnvelopegt

30
SOAP- Advantage

• Once applications are wrapped with SOAP they
  are in effect a web service
• A web service can be invoked directly through an
  HTTP call
• A web service can be discovered utilizing a
  directory service

31
Whats underneath SOAP?

• Soap is the abstraction layer not the
  integration tool
• EAI techniques are used to connect between the
  SOAP wrapper and legacy systems
• Once EAI objects are wrapped in SOAP they can
  easily communicate with each other thus creating
  a cross enterprise integration architecture

32
Web Services- Layers
Publish, Find, Use Services UDDI
Service Interactions SOAP
Universal Data Format XML
Communications Layer HTTP
33
UDDI

• Universal Description, Discovery and Integration
• Platform-independent, open framework for
  describing services, discovering businesses, and
  integrating business services.
• A layer above SOAP, it is the glue that connects
  all the SOAP objects into a Business Web
• DNS like model implementation of the directory

34
UDDI
UDDI Business Registry
Service Type Registrations
BusinessRegistrations
35
Registry Data

• Businesses register public informationabout
  themselves
• Standards bodies, Programmers, Businesses
  register information about their Service Types

36

• Business name
• Text description
• List of multi-language text strings
• Contact info
• Names, phone numbers, fax numbers, web sites
• Known identifiers
• List of identifiers that a business may be known
  by - DUNS, Thomas, other

37

• Business categories
• 3 standard taxonomies in V1
• Industry NAICS (Industry codes - US Govt.)
• Product/Services UN/SPSC (ECMA)
• Location Geographical taxonomy
• Implemented as name-value pairs to allow any
  valid taxonomy identifier to be attached to the
  business white page

38

• New set of information businesses use to describe
  how to do e-commerce with them
• Nested model
• Business processes
• Service descriptions
• Binding information
• Programming/platform/implementation agnostic
• Services can also be categorized

39
Service Type Registration

• Pointer to the namespace where service type is
  described
• What programmers read to understand how to use
  the service
• Identifier for who published the service
• Identifier for the service type registration
• called a tModelKey
• Used as a signature by web sites that implement
  those services

40
Business Registration

• XML document
• Created by end-user company (or on their behalf)
• Can have multiple service listings
• Can have multiple taxonomy listings

businessEntity
businessKey name URL description contacts business
Services identifierBag categoryBag
businessService
businessService
serviceKey tModelKey Name Description BindingTempl
ates
Key Name Description BindingTemplates
41
(No Transcript)
42
(No Transcript)
43
(No Transcript)
44
Talk Overview

• Whats really new about Web Services
• What a SOAP Router is
• What a SOAP Router can do for you
• How an SOAP Router works
• ALIN in theory
• ALIN in practice

45
Web Services are NOT New

• There is nothing new under the sun in distributed
  (or parallel , or grid, ) computing
• These are all efforts to lash many systems
  together to act as one
• SOAP today is a distributed computing technology
• Sprinkling angle-bracket pixie dust over plain
  ol client/server RPC
• Sure, it works across the public Internet, but
  only by treating it as a very slow LAN
• Trade rags already lament the gap at this
  boundary real security, reliability, and
  performance problems reveal the fallacy of
  assuming TCP/IP just works

46
Web Services ARE new

• Decentralization does pose brand-new challenges
• Latency
• Services are no longer mere milliseconds away.
• We need middleware for seconds up to weeks
• Agency
• Services can be owned by other organizations
• We need middleware for translating semantics
• Because SOAP messages are self-contained, they
  can be stored, analyzed, and replayed at will

47
SOAP is Simpler Than You Think

• Starts w/point-to-point, synchronous invocations
• just as Web page servers are accessed today
• Main virtue is that it is not DCOM, IIOP, or RMI
• Encapsulates the messy details of cross-platform,
  cross-language binding using standard marshalling
  formats
• Getting started requires very little
  re-education.
• On the other hand, straight RPC message exchange
  patterns (MEPs) dont inherently reduce the tight
  coupling between client and server semantics.

48
SOAP is Cooler Than You Think

• The vastly underestimated power of SOAP is that
  its model is genuinely asynchronous
• Today, we use it as a form of XML RPC
• Underneath, thats merely a pair 1-way messages
• Furthermore, SOAP Actors are not hosts
• An actor is a much more general concept than a
  fixed service provider machine and port
• One actor may be a proxy for many others
• Ideally, actors represent actual business models

49
What is SOAP Routing?

• Today, the canonical Web Services diagram
  directly connects SOAP endpoints
• There will also be a complementary technology
  that sits in the middle
• Store-and-forward reliable messaging
• Filtering, transformation, analysis of flows
• Intelligent active proxy chaining of actors

50
What Good is SOAP Routing? 1/2

• Internet-scale Middleware
• Just as distributed objects needed
  message-oriented-middleware (MOM)
• decentralized services will need mediators
• Wrappers SOAP brokers to legacy systems
• Monitors SOAP transaction correlation audit
• Queues decouples SOAP provider from requestor
• Pub/Sub decouples SOAP actors entirely
• The difference is coping with firewalls and other
  kinds of agency boundaries

51
What Good is SOAP Routing? 2/2

• Add new ilities without modifying either end.
• Security Router can authenticate and authorize
• Reliability Router can store-and-forward msgs
• Availability Router can redirect to live servers
• Scalability Router can redistribute messages
• Interoperability Router can transform msgs
• Interactivity Router can update msgs in-flight
• Routing lets us reason about a network of
  services -- not just pipelines, but graphs

52
How SOAP Routing Works 1/2

• Consider a simple Bank ATM service
• In the old world, you would have fine-grained
  account objects and setUserName() functions
• As Web Services, you have more flexible
  XML-document services like getStatement()
• A client-side ATM program can easily call it
  directly today by composing a SOAP message and
  POSTing it to the Bank server

53
How SOAP Routing Works 2/2

• First, it provides simple middleware
• Recall how CGI scripts directly invoked scripts.
• Later, Application Servers emerged to
  encapsulate the effort of storing state c
• By passing the message through a SOAP router
• It can enforce the Banks security policy
• It can store a copy of the message for audit logs
• It can redirect requests across a server farm
• This is exactly what Web proxies do today

54
How SOAP Routing Helps 1/2

• Second, it affords flexibility of connections
• What if you wanted to change banks?
• The router can filter the captured message stream
  so that messages from the new bank look like
  the old bank
• What if you wanted to change currencies?
• This is equivalent to routing through another
  actor
• The router can transform messages with plugin
  scripts today, XML schemas tomorrow
• What if you wanted to know if cash is running
  out?
• The router can analyze messages for trends

55
How SOAP Routing Helps 2/2

• Third, it allows 3rd 4th-party extensions
• What if you wanted to check on your bonds, too?
• The router can multicast the balance inquiry to
  the Bank service and a Broker service
• What if you wanted to buy more bonds?
• The router can monitor the joint transaction of a
  money transfer to the broker and the purchase of
  the bonds
• What if you wanted your limit kids access?
• The router can delegate access control rights

56
ALIN in theory 1/3

• ALIN is NOT merely Actor intermediation (piping)
• Actor chains today are like UUCP !path
  addressing its source-routing only
• Its the ability to infer semantic chains,
  decouple political control, etc.
• SOAP Routers directly address the twin novel
  challenges of decentralized service integration
• Latency
• by decomposing every communication into a
  one-way, asynchronous message, routers force us
  to confront extreme latency, rather than hiding
  it
• Agency
• by introducing a trusted 3rd-party between the
  original endpoints, routers allow us to represent
  the interests of many other 4th-party agencies

57
ALIN in theory 2/3

• The Internet itself is a result of addressing
  high latency and diverse agency at Layer 3
• IP Routers store-and-forward packets across
  autonomous systems, each containing many LANs
• IP-format packets are a generic interchange for
  LANs
• Consequently, Internet-scale services must
• Scale Across Time Interfaces must be stable for
  decades
• Scale Across Space Protocols must handle gt100ms
  delays
• Scale Across Organizations Namespaces must
  peer
• This is merely an application-layer router for
  TPs

58
ALIN in theory 3/3

• This architectural style builds on REST C2
• Beyond merely transferring representation, now we
  can mediate them
• Can connect any graph, not just a strict lattice
• It generalizes other MOM styles
• Queues, sync/async, 1-1, N-N, transactional
• Pub/Sub, address-based and content-based
• The most contentious aspect of ALIN for MOM folks
  is best-effort it is still difficult to sell
  TCP over IP
• If your bottom turtle is already transactional,
  it rules out scalability and perf
• The very definition of agency boundary is that
  transactions cant span them!

59
ALIN in practice

• Lots of ink is being spilled on missing
  infrastructure for Web Services
• Major software platform vendors are only
  beginning to market RPC-style Web Services
• Existing middleware products are having
  difficulty outside the firewall
• JMS, for example, is single-vendor,
  single-language
• High-performance EAI tools often assume multicast
  IP
• Early Internet-scale messaging startups include
  KnowNow, Bang, Kenamea, SonicXQ, Grand Central,
  and others
• Microsoft has outlined its WS-Routing proposal
• IBM has Business Rule Beans and MQ Event Broker
  in WS5.0

60
Distributed vs. Decentralized

• We already have lots of techniques for
  decomposing systems to run on many distributed
  processors acting as one
• The genuinely novel potential of Web Services is
  decentralization
• Marketplaces, rather than central brokers
• Routing is a new technique for integrating
  services provided by 3rd and 4th parties

61
A Parting Thought

• The dirty little secret is that the 7-Layer ISO
  Network model is insufficient to model
  decentralized computing.
• Its really all about integration at Layer 8 9

Political
Economic
You Are Here
Application
Presentation
Session
Transport
Network
Link
Physical