XML Mgmt Update

1
XML Mgmt Update

• Dr. Glenda Hayes
• MITRE/DII-COE SHADE

2
Agenda

• Market-Driven Data Strategy
• Electronic Marketplace For XML
• DII-COE XML Registry
• XML Coordination Guidance

3
Data Management Challenges
Expect Heterogeneity!

• No Single Standard can be imposed!
• Various DoD communities will adopt multiple
  standards
• Government (message, database, symbology),
  Commercial, International, de facto/legacy etc.

4
XML Management Challenges One Language, Many
Vocabularies
ltlat_deggt30Nlt/lat_deggt
ltlatitude unitsdegrees hemispherenorthgt30lt/l
atitudegt
ltlatitudegt lthemispheregtNlt/hemispheregt
ltdegreesgt30lt/degreesgt lt/latitudegt

• These 3 XML fragments are
• Equally valid ways to express the same data in
  XML
• Well-formed per W3C Specification
• Different terms and XML structures

NOT INTEROPERABLE!
5
Management OptionsContrasting Styles
What will work for Defense?
TIGHT

• Top-down, Command
• versus
• Market

SPECTRUM OF CONTROL
Recommended Approach Market with Some Controls
LOOSE
6
Principles from President Bush

• Citizen-centered
• Results-oriented
• Market-based

Mar-15-01
7
Key Management Mechanism
Developers and Warriors Asking for Help!!
What data is available?
Which data is better?
Who has it?
How do I get it?

• Market Visibility

8
Market Players

• Developers
• Re-use available data components and/or Register
  new ones they have created
• Community Data Managers
• Use Market visibility services for configuration
  control (e.g., current version distro, version
  change notification etc.)
• Defense Acquisition Policy Makers
• Use Market metrics for acquisition oversight
  (e.g., reflecting Program participation, specific
  data component re-use etc.)

9
Market Features

• Low barriers to entering the process
• Electronic Marketplace as Visibility mechanism
• Components as unit of exchange (commodities)
• Communities of Interest (COIs) as arenas for
  trade
• Program Engineers as primary traders

Must engage hands on Developers who build
important Data Resources!
10
Electronic Market DISAs Data Emporium
http//diides.ncr.disa.mil/shade

• One Stop Publish Subscribe for Defense
  Metadata
• Data Component Registration
• COI Creation Management
• New Version Jan 01
• Public Access via WWW plus
• Password protected and classified instances

Purpose visibility and re-use, not
standardization through mandate!
11
Marketplace Rules

• Data Component Registration
• Consult Emporium before creating new components
  and reuse existing data where practical
• Indicate planned use of components by formally
  subscribing to them
• Register additional components or recommended
  mods
• Communities of Interest (COIs) Formation
• Created as required when someone will agree to
  manage
• Requirements for new COIs staffed with
• Existing COI Managers
• Senior Service/Agency engineers
• Flag Level Review Board

12
Market Organization

• Configuration Control features
• Distributed across DISA Engineering Staff and
  Hands On Managers from multiple COIs
• Capitalizes on existing configuration control
  bodies
• COI Managers govern data in multiple physical
  forms
• Coordination Venues
• Individual COI boards
• COI Managers Forum
• Data Technical Working Group (Any voice heard)
• Ad hoc contacts with other Government, Allied,
  and Commercial activities

Configuration Control Organizations
13
Electronic Market Place XML Registry
http//diides.ncr.disa.mil/xmlreg

• IOC May 99
• Public Access via WWW
• (FOUO) Password protected
• Improved Version Jan 01
• 10K elements and schema
• SIPRNET version early CY01
• Handles
• XML Registration
• Namespace Creation Mgt
• CIO EB Expanding Scope
• XML Registration Policy and Plan

Purpose is management visibility and re-use, not
standardization through mandate!
14
XML Market Place Rules

• XML Registration Requirement
• Developers using XML for public interfaces
  should
• Consult XML Registry before creating new XML
  components and reuse existing XML where practical
• Indicate planned use of Registered XML by
  formally subscribing to it
• Submit (where required)
• additional XML components (with amplifying
  information) or
• recommended modifications to existing components
• Creation of XML Namespaces
• Formed as required when someone will agree to
  manage
• Requirements for new Namespaces staffed with
• Existing Namespace Managers
• Senior Service/Agency engineers
• Flag Level Review Board
• CIO EB kept apprised

15
Namespaces/Managers

• COE/DISA
• Ground Operations/Army
• General Military Intelligence/DIA
• Aerospace Operations/USAF
• Messages/DISA
• Tracks Reports/Navy
• Combat Support/DISA
• Geospatial Imagery/NIMA
• METOC/Navy
• Personnel/DIMHRS
• Finance and Accounting/DFAS
• TBD/DISA
• Enterprise/DISA

• Other Proposed
• Logistics
• MASINT
• NBC
• Transport
• Training
• Trade/Export

16
DII-COE XML Registry Update

• Features
• Structure/Schematic
• Submission Process
• Submission Assistance
• 3 case studies
• Namespace Population Status

17
DII-COE XML Registry Features

• Search/Browse Filters
• Across or within namespaces
• By Information Resource Type
• By Substring in Name, Definition, Comment
• By Submitter
• By Status
• By Version
• Subscription
• On-Line Submission Pkg Verification Submission
• On-Line Registry Administrative Features

18
DII-COE XML Registry Resources and Relationships
Information Resources and Relationships
XML schema ATO confirmation
19
DII-COE XML Registry Information Resource Types

• Manifest
• an XML document IAW http//diides.ncr.disa.mil/xml
  reg/DTD/registry.dtd
• Schema
• e.g., Bookshop.dtd
• XML Element (atomic and container)
• Atomic Title
• Container Book
• XML Attribute
• e.g., Genre, in_stock
• Domain Document
• an XML document IAW http//diides.ncr.disa.mil/xml
  reg/DTD/registry_domain_values.dtd
• e.g., con_genre.xml, con_yesno.xml
• Stylesheet
• e.g., Bookshop.xsl
• XML Example
• e.g., Bookshop.xml, BookshopStyled.xml
• Amplifying Document
• e.g., bookshop.gif

20
DII-COE XML Registry XML Component Submission
Process

• 1) User submits package (wo/write cap)
• 2) Verifier emails to DISA
• 3) DISA screens for embarrassing material
• 4) DISA submits package (w/write cap)

DII-COE XML Registry
1
BookshopSubmission.zip
Registry Submission Package Verifier
Registry DB
4
3
Submission Script
2
email
http//diides.ncr.disa.mil/xmlreg/XMLUserSubmitFor
m.cfm
21
DII-COE XML Registry Submission Package Example

• Package Contains
• 1 manifest (XML)
• 0 docs
• 0 domain docs (XML)
• 0 schemas

Submission.zip
Manifest.xml
bookshop.gif
con_genre.xml
con_yesno.xml
Bookshop.xml
BookshopStyled.xml
Bookshop.dtd
Bookshop.xsl
22
DII-COE XML Registry Leveraging Existing
Metadata Assets
Exports from Modeling Tools MITRE Prototype
Submission Assistance Tool
xml
Domain Values
Submission Manifest
Erwin
Domain-Child Info
Winzip
SAT1
Domain Values
Rational
Domain Values
Child Info
Domain Values
Domain Values
SAT2
Parent Info
Registry Verifier
Parent-Child Assoc
COE XML Registry
XML Schema (DTD, Schema)
Registry Ingestor
SAT Submission Assistance Tool
23
Leveraging Existing Metadata Assets Case 1 - USMTF

• USMTF CCB
• Maintains Data Dictionary in relational database
• Algorithm to transform USMTF-to-xml-mtf
• Stylesheet to transform xml-mtf-to-USMTF
• Challenges
• Mismatch in Data Types with DII-COE Registry
• Lack of Name for Groups, Segments
• SHADE Strategy
• Enter FUDs (atomic elements)
• Enter Domains, link to FUDs, store xml files of
  domains
• Enter groups, sets, segments, messages
• Registration Status
• Messages Namespace
• 6165 XML Elements Registered (2/28/2000)

24
USMTF xml-mtf
USMTF
MSGID/TACREP/CTF 124// MAROP/011800Z/1/US/SUB/CLW
ASHINGTON/NAMESEAROVER /LM4040N01100E// OPSUP/AC
TTYPASW// AIROP/020200Z/6/US/FTR/F15/TN401/LM41
30N01000E/CRS180/SPD600KPH /ALT12000FT// OPSUP/
ACTTYPDCA//
USMTF United States Message Transmission
Format MIL-STD 6040
25
xml-mtf Elements Registered
26
Leveraging Existing Metadata Assets Case 2 Data
Models

• MIDB
• Erwin Physical Model
• 7 addl domain documents
• DIA-supplied Long Names
• Challenges
• Relational v. Hierarchical Models
• Erx errors
• Multiple techniques for storing domains
• SHADE Strategy
• Harvest metadata from erx
• Register Columns Attributes as atomic XML
  Elements
• Register Tables Entities as container XML
  Elements
• Register Enumerated Domains as Domain Documents
• Develop Viewer
• Register Better Definitions for Domain Documents
• Register Logical-to-Physical Mappings
• Register Parent-Child Relationships as contain
  relationships
• Register algorithmic constraints using XML Schema
  Definition

• Registration Status
• GMI Namespace
• 2000 XML Elements
• 228 Domain Docs
• 1 XML Mapping Doc

27
MIDB Elements Registered
28
MIDB Domain Values Registered
29
Leveraging Existing Metadata Assets Case 3 XML
Schema Definition

• Convert DTDs to XML Schema Definition
• Tool support is available
• Augment elements/attributes with
• Annotation
• Data Type
• Constraints
• Patterns
• Enumerated Values
• SHADE Strategy Construct Submission Manifest
  Shell from .xsd
• No progress to report at this time

ltminInclusive value "1"/gt ltmaxInclusive value
"5"/gt
ltmaxLength value "5"/gt
ltpattern value "a-hj-np-zA-HJ-NP-Z1,3"/
gt
30
DII-COE XML Registry Namespace Population Progress

• Messages
• 6165 XML elements
• Domain docs
• Container (complex) elements
• Aerospace Operations
• 1 DTD, 8 XML elements
• Ground Operations
• Technical coordination required
• Tracks and Reports
• 452 XML elements
• 1 schema
• 8 domain docs

• Intelligence
• 2000 XML elements
• 228 domain docs
• 1 mapping doc
• MDITDS request
• Geospatial Imagery
• Baseline GML and IML
• Finance Accounting
• Leveraging existing metadata
• METOC
• Leveraging existing metadata

31
XML Coordination and Guidance
32
XML Management at-a-glance
Supports
Operates, Maintains
Participate in
Governs
Hosts
Consults Submits to/Downloads from XML
Registry
Participates in
Participates in
33
XML Convergence Process

• Namespaces managers chartered as required
• Overlap among Namespaces is inevitable
• Namespaces act as Buckets for in-use XML
• Managers run Working Groups (collaborative
  venues)
• Entry points for XML Registry submissions
  Developer requirements
• Status mechanism (developmental, operational,
  deprecated) provides ability to express COI
  preference
• Enterprise Namespace holds DoD common XML
• Identifies and Registers Common XML from COI
  Namespaces
• Governed by Namespace Managers Forum
• Registry provides XML Market visibility
• Includes system and developer usage information

34
DII-COE XML Guidance XML Data Compliance
Integration and Runtime Specification DII-COE
Compliance Level for XML
8
Reconciled Differences with Standards
Used Production Components from COI
7
Submitted Mapping of Logical Model to Physical
Schema
6
5
Submitted XML Components to Registry
35
XML CoordinationDII-COE XML Forum

• Sponsor (DII-COE DATATWG)
• Defense Information Infrastructure (DII)
• Common Operating Environment (COE),
• Data Access Services Technical Working Group
  (DATATWG),
• Semi-Structured Data and Metadata Subpanel
  (SSD-MD)
• Objectives
• Develop specifications and/or DTDs
• Select metadata standards and tools
• Create DTD repository / distribution mechanism /
  versioning management
• Provide guidance for tag terminology
• Develop enhanced XML editors for coded XML docs
• Develop application interpreters for XML
• Reference implementations

36
XML CoordinationDII-COE XML Forum (contd)

• Meetings
• Chartered by DII-COE Architectural Oversight
  Group (AOG) Nov 1998
• Distribution List 400 Names, Attendance 70
• Bi-monthly meetings, 11 meetings since Jan 1999
• Topics
• XML Activities Survey
• XML Vendor Briefings and Demos
• DII-COE XML Requirements
• DII-COE XML Registry Debut
• XML Technical Discussions
• XML Policy Coordination
• SOAP
• XML Schema

Next Meeting 27 April
37
XML Coordination XML Namespace Managers Forum

• Draft Forum Objectives
• Propose, review, and implement DoD XML policy. 
• Develop and promote best practices in XML. 
• Seek opportunities for convergence. 
• Oversee the operation of the Registry. 
• Determine what metrics to use, analyze and make
  recommendations as inputs to DoD and other
  policies (IRTS, JTA) 
• Define, review, update the procedures for the
  Namespace Mgrs Forum. 
• Review proposals for additional namespaces to
  make recommendation to DII-COE AOG. 
• Participate, Respect, and influence international
  and coalitions metadata standards.        

Next Meeting 10 May
38
DoD XML Registration Policy Draft Implementation
Plan Coordination

• CRCB directs Chief Engineer to propose DII-COE
  XML process for DoD
• Chief Engineer briefs DoD CIO EB 29 Aug
• CIO EB asks for
• Draft registration Policy memo
• Draft implementation plan
• Resource assessment
• Staff through ASD C3I, USD ATL, etc.
• Info Brief to CIO EB for status May

39
Contact Information

• Mr. Pete Pasek - pasekp_at_ncr.disa.mil
• Mr. Jim Pipher - pipherj_at_ncr.disa.mil
• Mr. Stan Davis - davis2s_at_ncr.disa.mil
• Ms. Toni Weir - weirt_at_ncr.disa.mil
• Ms. Alesia Jones-Harewood harewooa_at_ncr.disa.mil
• Ms. Ellen Minderman (FGM) - minderma_at_fgm.com
• Dr. Glenda Hayes (MITRE) - ghayes_at_mitre.org
• Mr. Chuck Heazel (MITRE) - cmheazel_at_mitre.org

SHADE Data Emporium http//diides.ncr.disa.mil/sha
de
40
Backup
41
DII-COE Guidance

• CIO EB DoD CIO Executive Board
• AOG Architectural Oversight Group
• CRCB Configuration Review Control Board
• TWG Technical Working Group

Establishes/Advises on DOD Policy
Selects or Mods Tech Mgt Options
Establishes TWG Priorities
Namespace Mgrs Forum
SSD-MD (XML) Subpanel
42
Registration is Easy!

• Developer identifies or creates XML not currently
  registered.
• Developer creates Registry package for new XML
  and proposed mods.
• Developer submits Registry package via on-line
  capability, specifying an existing namespace.
• http//diides.ncr.disa.mil/xmlreg/XMLUserSubmitFor
  m.cfm
• Registry Ops reviews submittal for propriety and
  registers package as Developmental.
• Receiving Namespace Mgr reviews submittal and
  accepts or rejects as properly belonging to
  another namespace.
• If rejected, Namespace Mgrs Forum determines
  whether another namespace will accept it.
• Upon developers' request, entries rejected by all
  namespaces are retained in TBD namespace.
• Namespace Mgrs may review accepted XML to change
  status from Developmental to Operational.

43
Viewing the Intermediate ResultsEnumerated Domain
44
MIDB BaselineSubmission Package
45
Registered XML Status

• Developmental Registered
• Operational Employed by Community
• Deprecated Legacy Operational version (still
  in use)
• Retired No longer in use

46
Analyzing the Market

• Registry support for Market Analysis
• ID of duplicate terms
• XML Component page visits, downloads and
  subscriptions
• ID of users
• COE Compliance

Lat Long
XML Registry
ENT
?
Lat Long
Lat Long
Lat Long
Lat Long
Lat Long
AOP
GOP
TAR
GEO
GMI
MET
PER
FIN
Exploit Observed Commonality!