Introduction to Mercator

1
Introduction to Mercator

• ManTech CALS Project
• In-house Training
• (Day One - Mercator Introduction)
• August 4, 1999

2
Course Agenda

• Day One
• (some examples will be given as we go through the
  presentation)
• General Background
• Mercator Overview Introduction
• Define Data Using Type Editor
• Build Debug Maps Using Map Editor
• Error Handling Data Auditing
• Options for Map Execution
• Web and Database Integration Examples

3
Course Agenda

• Day Two
• EDI EDI Concepts
• EDI Type Tree Concepts
• ANSI X12
• UN/EDIFACT
• Map Inbound/Outbound EDI data
• Future Training
• Trading Partner PC
• Advanced Web Database Integration Examples

4
Overview

• Who is TSI Software International?
• TSI is a leading player in the enterprise
  application integration (EAI) market
• TSI spun off of DNB in the mid-late 80s
• TSI went public in July of 1997
• TSIs flagship product is Mercator
• EDI (both ANSI X12 and EDIFACT)

5
ManTech Introduction to Mercator

• Brief Intro Background
• What is Mercator?
• Applications
• EDI
• Application Integration
• Any-to-Any
• How do I use Mercator?

6
Information is Interchanged in many formats...
X12
Oracle
EDIFACT
Legacy Proprietary
XML
STEP
PDML
7
Information comes from many sources and has many
destinations...
Flat Files
FTP
ODBC Database
SMTP
VANs
EDIFACT
X12
8
Software System Overview
Supports 19 major platforms
Oracle Plug-in Adapter
10 Operating System Specific
Any... Plug-in Adapter
FTP Plug-in Adapter
SMTP Plug-in Adapter
SAP Plug-in Adapter
9
Mercator High ROI

• Addresses most costly aspect of integration
• Data transformation
• Eliminates code generation and maintenance
• Accelerates ROI as complexity increases
• Insulates the task of integration from changes in
  technology

10
Mercator Component Applications

• Authoring System
• Type Editor, Map Editor, and Windows Execution
  Engine
• Type Tree Collections
• ANSI X12 EDIFACT
• Execution Engines
• Enable Map Execution on 19 major platforms
• Enable command line execution of maps
• Database Editor
• Full ODBC connectivity, auto type tree generation

11
Other TSIComponent Applications

• Trading Partner PC
• Integrated EDI product
• Handles communications and task/event scheduling
• FTP Adapter for EDI via the Internet
• Kits for EDI partner rollout, for example Rip
  Read, etc...

12
Getting Started

• 1. Overview
• 2. Defining Data using the Type Editor
• 3. Specifying Rules in the Map Editor
• 4. More Type Editor Map Editor Concepts
• 5. Error Handling Data Audit / Debugging
  Using the Trace File
• 6. Map Execution Options
• 7. Brief Demo of Database Editor
• 8. Where to go to learn more...

13
Methodology for Using Mercator
Look at Your Data(know your data!)
You must know your data!!!!
(at both the Source and Destination)
Define Your Data
Type Editor
Specify Mapping Rules
Map Editor
ExecutionEngine
Execute Map Examine Results
14
Fundamentals

• Characteristics of Source and Target Data
• Data Types
• Text, Number, Date, Integer, etc.
• Attributes
• Max / Min Size, Fixed Length, Occurrence
  Indicators, etc.
• Delimiters
• In-fixed commas, Post-fixed asterisks, etc.

15
Mercator Framework for Defining Data

• Source Data
• Has characteristics as described previous slide
• Desired Format and Location for Target Data
• Has characteristics as described previous slide
• Reusable Building Blocks
• Intermediate link between Source and Target
• Called Type Trees, but first...

16
Source / DestinationType Tree Hierarchy
Step 3
Step 2
Step 1
17
Mercator Authoring System

• Type Editor
• Define data
• Both Source and Target
• Input validation criteria
• Conditional checks, restriction lists, etc.
• Map Editor
• Define transformation of data
• Target format and location
• Mapping mapping rules

18
Mercator Type Trees

• Type Trees
• Compositional
• Contains this set of components
• Layout
• Set of components are in this sequence
• Input Validation Criteria
• Component rules filtering unwanted input data
• Apply to both Source and Target Data
• Reusable

19
Type Trees
Types (classes)
Root
File In
File Out
Record
Field 1
Record is comprised of Fields
Field 2
Record is comprised of Fields
Record is comprised of Fields
Field ...n
20
A Simple Mail List ExampleConvert address file
into mailing labels

• Use the Type Tree editor to create
• type tree for the input and output files
• component rules for input data
• Use the Map Editor to define
• input and output cards
• mapping rules
• Run this Mercator map via the Web

21
Look at Your Data(know your data!!!)

• Sample Portion of Source Data File Format
• Adams,James,P,ABC Co.,29 Frankford
  Rd,Bloomington,IL,60525,708,3525555
• Miller,Maria,B,Conrad Corp,1234 Smith St,Buffalo
  Grove,CA,60089,708,3334567
• Smith,Fred,A,Sand Inc.,Beach Street,Pismo
  Beach,FL,33321,407,8123456
• Veldin,Beth,M,Any Co.,697 Berry Road,Highland
  Park,IL,60012,708,4445987
• Jones,Mary,,McCormick Enterprises,777 Lincoln
  Drive,Maryland,CA,19444,415,6666666
• Martin,Mary,,Hooks and Hangers,123 Neverland
  Ave,Sky,TX,44444,302,6616000
• Each Record Consists ofLast, First, Middle
  Initial, Company, Street, City, State, Zip, Phone
• - Note that Middle Initial is Optional
• - Note that data is comma delimited

22
Mercator Type Tree - Items
23
Item Properties
24
Item Properties - Item Format
25
Additional Item Properties

• Initiator
• specify what comes before the item
• Terminator
• specify what comes after the item
• Release
• indicates that the characters following the
  release should be interpreted as data, and not
  a delimiter

26
Recall the Type Tree HierarchyWere finished
defining the Data Items
Step 3
Step 2
Step 1
27
Mercator Type Tree - Records
28
Creating the Contact Record Layout
29
Type Tree - Group Formats
30
Group Format Properties
31
Record-level Component Layout
32
Input Validation

• Restriction Lists
• Useful when an item is limited to a set of static
  literal values

33
Input Validation

• Component Rules
• Test a condition to accept or reject the
  input data

34
FYI There is an additional type of rule.

• Mapping Rules (will be covered later today...)
• Are used to assign output values (either from
  source data or derived) to the output type tree
  group or item
• Unlike component rules, these rules evaluate to
  another value (component rules evaluate to either
  true or false)

35
Recall the Type Tree HierarchyWere finished
defining the Record
Step 3
Step 2
Step 1
36
File-level Component Layout
37
Recall the Type Tree HierarchyWere finished
defining the File Layout
Step 3
Step 2
Step 1
38
Repeat Process for Output Object

• Identify your desired layout for the output data
• Full Name
• Company
• Street Address
• City, State ZIP
• Define your output data in Mercator
• Create type tree
• Items
• Record
• File

Know your data!!!
39
Analyze Tree

• After creating a type tree, it should be analyzed
  by using the Type Tree Analyzer.
• Checks definitions for internal consistency.
• Note The Analyzer only checks your data
  definitions for logical consistency.
• It does not compare your definitions to your
  actual data.
• The resulting Analyzer messages indicate whether
  your type tree definitions are okaynot whether
  or not they match your data.

40
Mercator Maps Map Editor

• Specify Input Card(s)
• Type Tree for Source Object
• Location of Source Object
• Specify Output Card(s)
• Type Tree for Target Object
• Location of Source Target Object
• Specify Maps Mapping Rules
• From Input Card(s) to Output Card
• Reusable

41
Specify Input Card

• Specify the Type Tree and Object
• Specify the Data Source
• File, VAN, FTP, Email, Database, Application,
  etc
• Filename, DB Query, Command, or etc.

42
Specify Output Card

• Specify the Type Tree and Object
• Specify the Data Destination
• File, VAN, FTP, Email, Database, Application,
  etc.
• Filename, SQL Statement, Command, or etc.
• Write Options create new, append, update, etc.

43
Develop Execution Map
44
Functions Available to Mapping Rules

• Roughly 100 Functions
• Most can be used bothin Component Rules aswell
  as Mapping Rules
• Recall that
• Component Rules evaluate to True or False
• Mapping Rules evaluate to some Value
• A Mercator function is an expression that tells
  Mercator to generate an output by performing a
  certain operation on one or more inputs.

45
Specify Mapping RulesExecution Map
46
Building the Map

• This analyzes the map for any errors that were
  made when the map was created.
• This includes checking for missing map rules,
  invalid rules, invalid card definitions,
  verifying map references, and verifying the
  arguments of Mercator functions.
• The Build command also looks for circular map
  referencesi.e., maps that reference one another.

47
Running the Map

• Maps can be Executed both from the Map Editor,
  and from the command line (using the map
  execution engine)

• The Execution Engine window displays the progress
  of execution
• In general, you will not know exactly how many
  objects exist in your data.

48
Viewing the Results of Map
49
Maps Occur at Two Levels

• Top Level Execution Maps
• Main map that links input card(s) to source data
  file, and output card(s) to target data file
• Functional Maps
• Nested map(s) that contain a collection of
  mapping rules that apply to the set of items
  comprising each record related to the files for
  the top level execution map
• Called from another map via a map rule
• Maps a piece of data one at a time

50
When to use Functional Maps

• When an output group can occur more than once
• Output has a range, e.g. (015) or (s), etc.
• Some unknown number of objects will be created in
  the output
• The number of output objects is based on some
  number of of occurences in the input data
• The output type is different than the input type

51
Basic Mercator Mapping Transformation Framework
Start
Data Source(s)
output file object
For_input file object
Top Level Map
Target Data
output record
For_each record(s)
Map Rule at Component triggers the functional map
to iterate through the records
note that functional map is not directly
associated with any input or output file
52
Specify Input Card

• Specify the Type Tree and Object
• Specify the Data Source
• File, VAN, FTP, Email, Database, Application, etc

53
Specify Output Card

• Specify the Type Tree and Object
• Specify the Data Destination
• File, VAN, FTP, Email, Database, Application,
  etc.
• Filename, SQL Statement, Command, or etc.

54
Specify Mapping RulesExecution Map
55
Specify Mapping RulesFunctional Map
56
Running the Maps

• Maps can be Executed both from the Map Editor,
  and from the command line (using the map
  execution engine)

• The Execution Engine window displays the progress
  of execution
• In general, you will not know exactly how many
  objects exist in your data.

57
Viewing the Results of Map Execution
58
Debugging Using the Trace File

• The trace file is a text file that records
  Mercator's progress during the execution of a
  map. You can choose to trace the input data, the
  output data or both. You use the trace file for
  debugging purposes.

59
Using Audit Logs

• Execution Log option
• The execution log includes the command line to
  use to run the map from a command line, the
  number of input and output data objects, and
  information about data files and work files.
• Data Log option
• The data log option includes information about
  the data - size, type, name.

60
More Type Map Editor Concepts Component
Attributes

• Any component in any component list (in Type
  Editor) can be assigned any of the three
  component attributes
• I - Identifier
• Specifies the components used to determine
  existence of a data object
• R - Restart
• Specifies an error recovery point.
• S - Sized
• Specifies that the value of the given component
  represents the size of the following component.

61
More Type Map Editor Concepts Component
Identifier Attribute

• Tells Mercator which components can be used to
  identify the type a data object belongs to
• All the components, up to and including the
  component with the identifier attribute, are used
  to identify the object
• Therefore, as soon as a field with this attribute
  is valid, Mercator knows the record exists and
  the type of the record

62
More Type Map Editor Concepts Partitioning...

• Partitioning
• A type whose data objects are mutually exclusive
  from the data objects of other types at the same
  level.
• (used mainly for input trees)
• A partitioned type has a diamond shaped icon.
  When you expand it, its subtypes, called
  partitions, appearjust as they do when you
  expand it in the type tree.
• A partition is referenced by the symbol
  "ltgt"which brings to mind the diamond-shaped icon
  of the partitioned type.

63
More Type Map Editor Concepts Partitioning

• Common Partitioning Methods
• Items by Restrictions
• Use mutually exclusive restriction lists for item
  subtypes
• Groups by Identifier and Component Rule
• Component Identifier ( test) component
  rule
• Order from most common to least common for
  performance
• Items Groups by Initiators
• Specify the initiator, segment abbreviation in EDI

64
Error Detection, Recovery Audit

• Error Detection
• Performs validation checks - type matching,
  existence,
• Error Recovery
• Exception handling - checks for recovery points
  (recall the Restart component attribute)
• Data Audit
• User-configurable, detail info about input and/or
  output data

65
Map Execution Options

• Mercator can be run from
• A command prompt,
• Windows Command Execution Engine,
• Run function in a map rule, or
• Execution Engine API
• Options can be specified on command line

66
Map Execution Web Example

• http//ec_testing/servlets/SimpleServlet
• Run Mercator map from Java servlet (or CGI, etc.)
• Read resultant output (or populate DB, etc.)

67
Mercator Maps Other Sources Destinations

• Database Systems DBLOOKUP Calls
• Executes SQL statement against a database
• Application Calls
• Applications can be integrated to Mercator using
  adapters, which are programs that utilize
  pre-built DLL functions shipped with Mercator
• DDE Calls
• allows you to interface to other Windows
  applications, such as Trading Partner PC, Excel,
  etc.

68
Database Editor Demo...

• Specify a Database and SQL Statement
• Can automatically generate type trees from
• a query,
• database table, or
• stored procedure

69
Database Editor Demo
70
Where to go to learn more...

• CALS Drive
• Documentation
• H\1-EC-EDI-1999\Mercator_Documentation\Library.pd
  f
• Intro. Class Materials
• H\1-EC-EDI-1999\Mercator_Documentation\Class_Mate
  rials\Intro
• EDI Class Materials
• H\1-EC-EDI-1999\Mercator_Documentation\Class_Mate
  rials\EDI

71
Where to go to learn more

• TSI Tech Support
• http//www.tsisoft.com/desktop/support.html
• ftp//ftp.tsisoft.com
• ftp//ftp.tsisoft.com/Technical_Support/Mercator/F
  requently_Asked_Questions/