Mark W. Skall, Division Chief - PowerPoint PPT Presentation


Title: Mark W. Skall, Division Chief


1
Program Review
Mark W. Skall, Division Chief Candy Leatherman,
Secretary email skall_at_nist.gov Tel. No.
301-975-3262 Div. Web Site http//www.itl.nist.g
ov/div897/
2
Division 897 Funding - FY00
Projected Balance 257,062 ( 637,570
transferred to 2001 OA Reserve)
OA - 1,615K
ITS - 585K
STRS - 5,207K
Employees 35 Full Time 3 Part Time 10
Intermittent 5 Guest Researchers
3
SDCTs GOAL
  • To improve quality of software in industry
    through the development of
  • Technology
  • Measurements
  • Standards
  • Philosophy
  • Concentrate on key areas at forefront of
    technology
  • Get involved early and partner with industry
  • Fill industry void
  • Transfer technology and move on
  • Specific Strategy
  • Conformance Tests
  • Reference Implementations
  • Diagnostic Tests
  • Research to reduce costs of testing
  • Reference Data
  • Standards developed jointly with industry

4
Excerpts from Assessment Panel Report
6/26/00Division Review
The planning and documentation methods used by
SDCT could serve as a model for other ITL
divisions. The process has used clearly-defined
criteria to select state-of-the-art programs with
clearly defined priorities and goals,
well-identified roles, and measurable
contributions to national and international
standards organizations. Examples include the
work on conformance tests for XML, Distributed
Interactive Learning Systems, and for DASE . .
.Directions can change quickly within the
software industry. The division carefully
monitors these changes and acts appropriately. .
. .Industry has provided many testimonials
concerning the great value of the divisions
standards work, including The OASIS-NIST XML
Conformance test suite is critical for our
industry, Norbert Mikula, CTO, OASIS NIST made
strong contributions (X3D) and resolved our
knottiest problems, Don Brutzman, Board of
Directors, Web3D consortium, and The high
quality test suite and certification program is
an invaluable resource for ATA, Robert Peel,
Director of Airworthiness and Standards
5
Excerpts from Assessment Panel Report
6/26/00Laboratory Level Review
New projects are started based on the importance
of the work to U.S. industry, and the work viewed
by the panel generally had specific, focused
goals. An example of this approach is the work
on XML done in SDCT. This project is notable
because it addresses a major standard on which
industry was making little progress, and it also
effectively leveraged skills and approaches
developed in a now terminated project on VRML. .
.Another example of the improved planning process
is the existence of clear termination criteria,
which have been useful in sunsetting projects.
For example, the Real-Time Java project and the
work on RBAC satisfied their completion criteria
and therefore ended this year.
6
  • Standards and Conformance Testing Group
  • Software Quality Group
  • Interoperability Group

7

Group Leader Lynne Rosenthal Supy Proj. Leader
Lisa Carnahan Clare Lucey, Secretary
8

Mary Brady Lisa Carnahan Laurent Ciarletta
(GR) Anthony Cincotta (S) Chris Dabrowski Alden
Dima Leonard Gallagher Leonard Gebase Neil Gima
(S) Alan Goldfine
Martha Gray Michael Kass Clare Lucey Thomas
Logue Carmelo Montanez-Rivera Thomas
Rhodes Richard Rivello Jacqeline Schneider John
Tebbutt Marie-Noelle Terasse (GR) Mark Zimmerman
9
Division 897 Standards and Conformance
Testing Group
10
SQG
Software Quality Group
  • Develop methods to automate software testing
  • Develop software diagnostic tools
  • Develop reference data
  • Formal methods

Group Leader John Barkley Debbie Blackstone,
Secretary
11
SQG
Software Quality Group
Tamer Ahmed (GR) Paul Ammann (F) Debbie
Blackstone David Brinkley (F) Paul Black Neva
Carlson John Cherniavsky Anthony Cincotta
Keith Gallagher (F) Roger Gima (S) Michael
Koo Mary Laamanen James Lyle William
Majurski Douglas White
12
Division 897 Software Quality Group
13
IG
Interoperability Group
  • Ensure Federal agency requirements are input into
    voluntary standards committees - work through
    Federal CIO Council
  • Help Federal agencies and industry achieve
    interoperability through application of Division
    products
  • Provide technical support to voluntary standards
    committees
  • Support NIST paperless office effort utilizing
    digital signatures

Acting Group Leader Lisa Carnahan Vacant,
Secretary
14
IG
Interoperability Group
Daniel Allen Bruce Bargmeyer (RA) Frederick
Boland Gary Fisher Larry Fitzwater (GR)
Elizabeth Fong Kathryn Harvill Roy Morgan Judith
Newton Gertrude Sherwood
15
Division 897 Interoperability Group
16
Strategy
17
Development of Conformance Tests
  • Emphasis on newer technologies
  • Tests developed in parallel with standards
  • Tests developed before implementations
  • Need buy-in from industry
  • Need high impact
  • Need technology transfer

18
Development of Conformance Tests (contd)
  • Legacy activities
  • CGM
  • SQL
  • POSIX
  • Ada and other programming language
  • VRML
  • Current activities
  • XML and companion standards (DOM, XSL, etc.)
    Digital TV
  • X3D
  • Learning Technologies (IMS)

19
XML Conformance Testing
Project Team Mary Brady Carmelo Montanez Ricky
Rivello Mark Zimmerman

20
Electronic Commerce
B2C (business-to-consumer) Interactive Televisio
n (DASE, SMPTE)
B2B (business-to-business) Manufacturing (ICM)
Distributed Learning (IMS)
XML Technologies
Internet / World Wide Web
21
Why NIST?
  • Electronic Commerce Growth
  • Growth is exponential, already outpacing last
    years predictions
  • Changing the way businesses do business
  • Revolutionizing the way we consume and play
  • Advances are necessary in
  • Tools for describing / sharing information
  • XML technologies
  • Applying IT to solve vertical market demands
  • NIST provides
  • XML Technologies
  • Conformance Testing
  • NIST developed tests, available on-line, for use
    in testing applications
  • NIST technical leadership, neutral third-party
  • Unbiased Feedback of Specification Errors
  • Vertical Markets
  • Applied IT research

22
HTML or XML?
  • ltPurchasergt
  • ltNamegt Mary Brady lt/Namegt
  • ltAddress StreetNN Rm 572 CityGaithersburg,
    StateMD Zip20899lt/Addressgt
  • lt/Purchasergt
  • ltItemgt
  • ltPart_numbergt1234lt/Part_numbergt
  • ltItemDescriptiongtTreeslt/ItemDescriptiongt
  • ltCostgt50.00lt/Costgt
  • ltQuantitygt20lt/Quantitygt
  • ltTotalCostgt1000.00lt/TotalCostgt
  • ltInstructionsgtBe sure to water the root ball day
    of deliverylt/Instructionsgt
  • lt/Itemgt
  • ltShipTogt
  • ltNamegtSusan Carscaddenlt/Namegt
  • ltAddress StreetNIST, Bldg 301 CityGaithersburg
    StateMD Zip20899lt/Addressgt
  • lt/ShipTogt

Figure 2 XML Purchase Order
23
XML Technologies
Electronic Commerce, Distance Learning, Health
Care Manufacturing, Interactive Televison
Programming Languages
Transformations Formatting Objects
Signatures
Stylesheets
Query Language
DOM
Domain Specific
Registries Repositories
Namespaces
XML.ORG BIZTalk
E-Business Learning Objects XML/EDI
Boards
Medical
Education
Procurement
Auto Parts
...
...
Ratings
Libraries
Control
...
Information Description
XML Syntax, Information Set, Schema, Linking,
Fragment
24
XML Conformance Testing
  • OASIS XML Conformance
  • Sun, IBM, Fuji Xerox, DataChannel, MicroStar, W3C
    members

25
DOM Conformance Testing
  • W3C, OASIS
  • Sun, IBM, Microsoft, Netscape, Oracle

26
FY00 Progress -Tests
  • Completed XML Test Suite (release 1)
  • Completed XML Test Suite (release 2)
  • Incorporated new XML tests and fixes
  • Completed DOM-Ecmascript Test Suite (release 1)
  • Completed DOM-Java Test Suite (release 1)
  • Develop XSLT Test Suite
  • partnership with Lotus Corp. and OASIS

27
FY00 Progress -Committee Leadership
  • Chair OASIS Conformance and XML Testing
    Committees
  • Provide leadership for XSLT and Schema Testing
    committees
  • Provide leadership for UN/CEFACT ebXML initiative
    (conformance and reg/rep)
  • Provide conformance guidance
  • Develop conformance clause
  • Harmonize OASIS and ebXML efforts

28
FY01 Plans
  • Complete XSLT test suite
  • Develop test suites for XML Schema and other XML
    Technologies
  • Develop XML-based automated test tools to improve
    test development
  • Continue to chair OASIS testing
    committees

29
FY00 Resources
  • 3.5 FTE
  • 700K (STRS)
  • FY01 Resources
  • 5 FTE
  • Travel required to collaborate in standards and
    test development efforts
  • Equipment 10K

30
XML Conformance Testing
  • Customers
  • IT industry e.g., IBM, Sun, Microsoft, Oracle,
    DataChannel, Documentum
  • Vertical industries e.g., Education (IMS),
    Financial (e.g., Dun and Bradstreet), Travel
    (e.g., Sabre)
  • Impact
  • Conformance test suites have been used to improve
    the quality of XML/DOM processors (many of which
    are embedded in other XML software solutions).
    These are used in many vertical markets by
    millions of customers.
  • Feedback to standards developers improve the
    specifications

31
Interactive TV
Project Team Alan Goldfine John Barkley Doug
White Len Gebase
32
Interactive TV
  • TV and the Internet are each recognized as a
    technology that seems to change everything (BW,
    Oct 4 99)
  • Interactive TV is the convergence of TV and the
    Internet
  • Profound impact on Electronic Commerce expected
  • WebTV 1 million subscribers, 350 hrs/wk of
    programming
  • AOLTV projected 2 million subscribers in 2
    years
  • ITL partnering to develop the standards and
    tests required for the success of Interactive TV

33
Interactive TV
Traditionally separate media are converging
  • Wireless phones do email
  • Internet does broadcast radio and TV
  • Broadcast TV references links to Web pages

34
Interactive TV
Currently, for the viewer
35
Interactive TV
Demonstration
Current production
36
Interactive TV
Demonstration
37
Interactive TV
Advanced Television Systems Committee (ATSC)
  • Members include ABC, CBS, IBM, Intel, Lucent,
    Microsoft, NBC, Warner Brothers
  • DASE (DTV Application Programming Environment)
    Standard for a platform independent, high-level
    abstraction for integrating DTV and the Internet
  • DASE specifies programming environment for DTV
    receivers
  • Downloadable DASE applications portable across
    different TV set tops

38
Interactive TV
NISTs Role in ATSC DASE
  • Joint project with the High Performance Systems
    and Services Division (895)
  • Chair DASE Conformance Working Group where key
    players include ABC, Gateway, Microsoft, Sun
  • Partner with Unisoft to develop conformance
    tests
  • Reference implementation for programming
    environment

39
Interactive TV
Society of Motion Picture and Television
Engineers (SMPTE)
  • Members include ABC, CBS, IBM, Intel,
    Microsoft, NBC, Warner Brothers
  • DDE (Declarative Data Essence) Standard for
    Internet content and bindings to analog and
    digital streams (formerly ATVEF)
  • Declarative Content HTML, CSS, ECMAScript, DOM,
    Triggers (for syncing the two mediums)

40
Interactive TV
NISTs Role in SMPTE DDE
  • Contribute text to specification
  • Partner with Unisoft to develop conformance
    tests
  • Develop conformance tests for triggers and
    stream binding to integrate with existing
    Unisoft DDE element tests
  • Develop prototype receiver testbed
  • Participate in Applications Data Essence (ADE)
    Study Group

41
FY00 Progress
  • Chair DASE conformance task group
  • Conformance section of the DASE Standard
    completed
  • Review of DASE conformance test assertions from
    Unisoft completed
  • DDE Standard completed, adoption underway
  • DDE Bindings Standard underway
  • Test assertions for DDE triggers underway
  • Testbed for DDE receivers underway
  • ADE Study draft completed

42
FY01 Plans
  • Continue to chair DASE conformance group
  • DASE Standard adopted
  • Continue DASE conformance test development with
    Unisoft
  • DDE and DDE bindings Standards adopted
  • Develop DDE conformance tests for triggers,
    integrate with Unisoft tests
  • Develop test assertions for DDE bindings
  • Complete DDE Receiver testbed
  • Begin DDE-2 Standard development

43
FY00 Resources
  • 2 FTE
  • 400K (STRS, ATP-100K)
  • FY01 Resources
  • 4.5 FTE
  • Equipment 20K
  • Travel required for standards meetings and
    testing collaborations

44
Interactive TV
  • Customers
  • IT, e.g., IBM, Microsoft
  • broadcast TV, e.g., NBC, DirecTV
  • entertainment, e.g., Disney, Universal
  • consumer electronics, e.g., RCA, Intel
  • Impact
  • New Medium TV/Internet convergence
  • Interactive applications portable across
    receivers from different manufacturers
  • ATSC and SMPTE invited NIST participation
  • NIST chairs DASE Conformance Group

45
X3D Conformance Testing
Project Leader Mike Kass
46
Objectives
  • Continue the NIST/Web3D partnership begun in 1996
  • ITL created the VRML Test Suite (VTS) and Viper
    parser
  • Create an X3D Test Suite ( XTS )
  • Modify VRML test requirements and test
    descriptions
  • Translate existing VRML tests to X3D format
  • Chair the Interoperability/Conformance WG
  • Interface with developers and specification
    writers to resolve ambiguities in specification
  • Provide feedback to developers on browser
    conformance

47
FY00 Progress
  • Completed translation of VTS to X3D Test Suite
    (XTS)
  • Developed test harness to enable developers to
    contribute tests
  • Interest from DraW and Sun for using the test
    suite and making the results public

48
FY01 Plans
  • Low level support - assisting community in using
    the VRML and X3D Test Suites
  • No new development planned. Maintenance as
    needed

49
FY00 Resources
  • 0.75 FTE
  • 150K (STRS)
  • FY01 Resources
  • 0.2 FTE (minimal)
  • No Travel is expected

50
X3D Conformance Testing
  • Customers
  • X3D Consortium including Sun, Sony DraW
    Computing, Shout, Blaxxun
  • Impact
  • Conformance test suites are being used to improve
    the quality of X3D browsers
  • DraW Computing has said that they would make
    their test results public

51
Learning Technologies
Project team Martha Gray Tom Rhodes Jacki
Schneider Lisa Carnahan
52
Learning Technologies
  • Learning technology is
  • Learning anytime, anyplace
  • Interactive, distributive learning systems
  • In collaboration with industry, NIST/ITL is
  • defining requirements and specifications
  • developing prototypes and reference
    implementations
  • developing testing techniques and tools

53
Why Now?
  • The Gartner Group estimates that companies in
    the United States spent 58 billion on training
    in 1998, and of that, 9.4 billion went to
    IT-related training. They expect this figure to
    grow to 10.5 billion this year. Gilbane
    Report, July/Aug 1999
  • Just as electronic commerce exploded over the
    past two years, the stage is now set for
    e-learning to become the next big wave in
    Internet-based applications Cisco Systems CEO
    and President John Chambers
  • Worldwide, the education and training market is
    over a trillion (US) dollars. IEEE

54
Projects
  • IMS Global Learning Consortium, Inc.
  • Chair conformance team
  • advise members on writing IMS specifications
  • ensure testability of specifications
  • advise on how to test and set up testing programs
  • Develop Meta-data repository reference
    implementation
  • IEEE and ISO
  • Provide conformance expertise
  • Harmonization of efforts among specifications
  • DoDs Advanced Distributive Learning (ADL)
  • Executive Order 13111 on Federal Training -
    NIST/ITL co-leaders with ADL and DOL.

55
FY00 Progress
  • Continue collaboration with IMS, IEEE LT
    committee and NIST/ATP
  • Chair IMS conformance testing activities
  • Developed IMS conformance test policies and
    procedures
  • Served as Technical Lead of SBIR solicitation,
    assessed and recommend proposals
  • Develop repository of learning objects (this work
    coordinates with Reg/rep subproject)

56
FY01 Plans
  • Continue to chair IMS conformance committee
    providing expertise to emerging IMS
    specifications
  • Develop a conformance test plan for applicable
    IMS specifications

57
FY00 Resources
  • 1.5 FTE
  • 300K (STRS, ATP)
  • FY01 Resources
  • 1 FTE
  • Minimal travel required to attend standards
    meetings and interoperability demos

58
Learning Technologies
  • Customers
  • IMS, DoL, DoD, SimonSchuster, Sun, IBM, Apple,
    Microsoft, UNC, Calif State U
  • Impact
  • Use of standard metadata for global learning will
    improve discovery, evaluation, and use/reuse of
    learning objects
  • NISTs conformance expertise contributed to
    clearer, less ambiguous, testable IMS
    specifications
  • Conformance test plans and procedures will enable
    IMS to determine compliant products

59
ATA CGM Testing Program
60
FY00 Progress
  • Completed final release of the ATA CGM Test Suite
  • Provided guidance to ATA Recognized Test
    Laboratory on using the test suite and assessing
    clients
  • Chair Control Board - resolve questions of
    interpretation of the test suite and standard.
  • Initiated discussions on ATA's eCommerce
    activities - explore a possible XML testing
    program

61
FY01 Plans
  • Continue to support ATA's CGM Test Program
  • No new CGM development planned
  • Maintenance as needed
  • Provide guidance on the use of NIST's XML test
    suite and its applicability and use in ATA

62
FY00 Resources
  • 0.25 FTE
  • 75K (STRS)
  • FY01 Resources
  • 0.25 FTE
  • Travel required to support ATA test program and
    potential for an ATA XML test program

63
CGM
  • Customers
  • Air Transport Association Airlines and their
    suppliers (e.g., United, Northwest, Airbus, FedX,
    Boeing, Rolls-Royce)
  • CGM Developers (e.g., Autotrol, Intergraphics)
  • Impact
  • NIST Conformance test suite used by ATA CGM Test
    Program to validate CGM interpreters
  • Quality, reliable CGM products used for
    representing graphics in airframe maintenance
    manuals

64
(No Transcript)
65
Technology Transfer from NIST to ATA
Developed CGM Interpreter Test Suite Developed
Test Program Framework Developed test procedures
Formed and chair Control Board Developed Test
Lab criteria and evaluation criteria Assessed
Test Lab for ATA recognition Serve as technical
adviser Completed transfer
66
Testing Services
Status of Software Operational Testing Services
67
Testing Services (contd)
Testing Products or Services - Offered by Other
Organizations -
Developed with NIST Assistance
68
Strategy
  • Conformance Tests
  • Reference Implementations
  • Standards developed jointly with industry
  • Diagnostic Tests
  • Research to reduce costs of testing
  • Reference Data

69
(No Transcript)
70
AromaPervasive Computing Project
Project Team Alden Dima Bill Majurski Mary
Laamanen
71
AromaPervasive Computing Project
  • Emerging trend toward
  • numerous, easily accessible, often inconspicuous
    computing devices and sensors mobile or embedded
    in the environment connected to ubiquitous
    network structure
  • Industry is asking for
  • Standards for service discovery, APIs, e-commerce
    services
  • Testing and diagnostic tools to improve quality,
    interoperability
  • Reference implementations
  • Aroma multi-division effort to
  • Explore standardization and measurement issues
  • Attract industry interest and partnerships

72
Aroma Prototype (Aroma adaptor Aroma projector)
  • Goals - To discover technical, standards and
    measurement issues related to
  • Connecting portable wireless devices to
    traditional networks
  • Service discovery, self-configuration and dynamic
    resource sharing
  • Mobile code and data
  • Software infrastructure needed to create and
    manage pervasive services and applications

73
FY00 Progress
  • Present papers at NIST PC 2000, JavaOne, and
    International Pervasive Computing Workshop
  • Co-chaired NIST Pervasive Computing 2000
    conference
  • Developed Aroma Adapter-Projector with Div. 892
  • Measured performance of Aroma adapter
  • Developed conceptual model based on the OSI
    Reference Model
  • Exploring low-end pervasive computing
  • Embedded and 8,16-bit processors
  • Collaboration with US Naval Academy

74
FY01 Plans
  • Co-chair NIST Pervasive Computing 2001 conference
  • Explore use of ADL in pervasive computing model
  • Implement a virtual private smart space (VPSS)
    test bed
  • Extend Aroma projector to include digital cameras
    and VPSS technology
  • Continue performance measurements
  • Develop diagnostic and testing tools

75
FY00 Resources
  • 1.50 FTE
  • 300K (STRS, OA-DISA)
  • FY01 Resources
  • 3 FTE
  • Travel required to conferences to present work
    and collaborate with colleagues
  • Equipment 15K

76
Pervasive Computing
  • Customers
  • IT industry the usual suspects
  • Consumers e.g., GE, Sunbeam, Phillips, Northeast
    Utilities, Xerox
  • Universities USNA, UMD, UC Berkeley
  • Impact
  • Solicited industry needs for standards and
    measurements
  • PC 2000 conference launched awareness of NIST as
    key player
  • Papers presented at conference stimulated
    discussions on pc and standards
  • Ref imp demonstrated feasibility of PC
  • Enable widespread adoption and integration with
    software tools to create, manage, measure and
    test pervasive services and applications

77
XML Registry/Repository(Reg/Rep)
Project Team Lisa Carnahan Len Gallagher Mike
Kass
78
XML Registry/RepositoryReference Implementation
  • Built to OASIS specification
  • Being used by both OASIS and UN/CEFACT (ebXML)
  • Built using IMS Meta-data specification
  • Apply lessons learned to EC repositories

79
FY00 Progress
  • Revised the OASIS Registry and Repository
    Specification
  • Implement a reference implementation
  • Participate in ebXML Reg/Rep team
  • Ensure testability of specification
  • ensure compatibility between ebXML and OASIS
    specifications

80
FY01 Plans
  • Update reference implementation
  • Develop conformance test suite/data for OASIS
    specification
  • Coalesce the ebXML and OASIS specifications

81
FY00 Resources
  • 1.5 FTE
  • 325K (STRS)
  • FY01 Resources
  • 2 FTE
  • Travel required to attend standards meetings

82
XML Registry/Repository
  • Customers
  • EC/Enterprise Integrators Data Channel,
    Documentum, Oracle, Microsoft, Sun, IBM
  • Consumers Vertical market partnerships and
    supply-chains (e.g., automotive, insurance,
    travel, manufacturing)
  • Impact
  • Repositories allow efficient use of XML and
    eliminate duplication
  • Enable SMEs to participate in EC marketplaces and
    supply chains
  • Reduce cost of doing business through timeliness
    and increased automation

83
Internet Commerce for Manufacturing
(ICM)(ITL,EEEL, MEL, MEP)
Project team Tom Rhodes Liz Fong Kathy Harvill
84
FY00 Progress
  • Assumed project leadership (at request of EEEL)
  • Defined standards-based framework for ICM testbed
    (reference implementation)
  • Conducted workshops and presented papers/demos
  • NAMT, SIMA, and ATP reviews
  • Industry conferences IPC, NEMI, ITEM2000
  • Collaborate with NEMI, IPC, Georgia Tech, UMBC
  • Evaluating industry specifications (e.g.,
    RosettaNet, OAG, C-Net, eCo FW) for B2B
    interactions
  • Evaluating applicability of agent and web
    technologies

85
FY01 Plans
  • Continue industry collaborations
  • Demonstrate distributed ICM testbed operations
  • (with GaTech, NEMI, IPC partners)
  • Test and validate ICM information model in ICM
    testbed
  • Continue evaluation of applicable standards and
    technologies and their incorporation in ICM
    testbed

86
FY00 Resources
  • 1.5 FTE
  • 350K (STRS-EEEL, ATP)
  • FY01 Resources
  • 2 FTE (other FTE from EEEL, MEL, MEP)
  • Travel required to collaborate, participate in
    standards meetings, and present work

87
ICM
  • Customers
  • Consumers PC assembly supply chain participants
  • Consortia NEMI, IPC, Rosetta Net
  • Impact
  • Reduce the cost of circuit-board manufacturing by
    enabling the rapid exchange of build information
    and change notices for all supply-chain
    participants
  • Demonstrates the use of XML in a vertical industry

88
Strategy
  • Conformance Tests
  • Reference Implementations
  • Standards developed jointly
  • with industry
  • Diagnostic Tests
  • Research to reduce costs of testing
  • Reference Data

89
Forward-Looking Standards
Role-Based Access Control (RBAC) Virtual Reality
Modeling Language (VRML)/X3D XML and companion
standards (DOM, CSS, etc) Java ADL IMS DASE/SM
PTE OASIS Registry/Repository GIS
90
STANDARDS COMMITTEES AND RELATED ACTIVITIES
FEDERAL GOVERNMENT CIO Council Interoperability
Committee Standards Working Group
91
Real-Time Java
Project team Lisa Carnahan Alden Dima
92
Real-Time Java
  • Assistance asked by industry providers and
    government users
  • Collaborate with Sun, HP, IBM, New Monics, FTL
  • Hosted continuing workshops for real-time Java
    requirements
  • Addresses real-time issues in both Java embedded
    and Java server application environments
  • NIST coalesced industry to produce one set of
    requirements leading to one spec
  • Open consensus forum for requirements
  • BOTH camps (Sun and HP, Microsoft, etc.) have
    used NIST-led requirements document to produce
    specs

93
EE Times
  • Interestingly, both the splinter group and
    Sun are using as a serious basis for their
    efforts a requirements document for real-time
    (Java) extensions prepared by a separate group
    working under the auspices of the National
    Institute of Standards and Technology (NIST).
    That is raising hopes that both standards might
    ultimately come together.

94
Real-time Java Requirements Group
  • Requirements for Real-time Extensions to the
    Java Platform (final release -September, 1999)
  • NISTs Requirements Group effort dissolved after
    successful completion of its goal.

The J Consortium
The J Consortium uses real-time requirements
developed by the Requirements Group for Real-Time
Extensions for the Java Platform, an effort
sponsored by the National Institute of Standards
and Technologies (NIST).
Press Release -J Consortium Unveils First
Public Real-Time Specification For The Java
Platform
CUPERTINO, Calif., September 24, 1999
95
FY00 Progress
  • Real-Time Java Requirements document completed
  • Worked with both spec development groups to help
    coalesce industry
  • Member of Sun RT Experts Group - contributed to
    specification
  • Worked with J Consortium to help develop
    specification
  • Project sunset

96
FY00 Resources
  • 1.25 FTE
  • 250K (STRS)
  • FY01 Resources
  • 0 (Project sunset)

97
Real-Time Java
  • Customers
  • Sun, IBM, HP, NewMonics and users of real-time
    Java
  • Impact
  • Coalesced industry to produce one set of
    requirements
  • 2 spec efforts compatible
  • New book on Real-time Java mentions NIST
  • Presented to and endorsed by VCAT

98
ADL (Architecture Description Language)
Project Team Chris Dabrowski John Tebbutt
99
Architecture Description Language
  • Focus on high-level structure of the overall
    application (i.e., systems architecture)
  • Provide features for modeling software systems
    architecture
  • E.g., Structure, components, interfaces,
    interconnections
  • Support automated analysis and verification of
    design
  • Enable testing during design phase
  • Saves over testing at code phase

100
Architecture Description Language (ADL)
Software Architecture ..an abstract system
specification consisting primarily of functional
components described in terms of their behaviors
and interfaces and component-component
interconnections Hayes-Roth, 1994
ADL A language that provides features for
modeling a software systems conceptual
architecture Medvidovic 1996
101
NISTs Unique Role
Standards Role Neutral Expert Applied Research
Contribute to ADL body-of-knowledge
  • Standards Collaborations
  • Propose additional functionality to SAE (first
    industry standards effort)
  • Uniquely positioned to bring academic and
    industry communities together.
  • Apply developing AADL standard (based on
    Honeywells MetaH) to Unmanned Ground Vehicle
    build additional functionality into AADL standard
  • Explore ADL relationships with IEEE and Open
    Group efforts
  • Applied Research Contribute to ADL
    body-of-knowledge
  • Research application of formal approaches to
    ADLs
  • Enhance ADL features for verifying conformance of
    application systems architectures to reference
    architectures (e.g. RCS)
  • Publish successes and lessons-learned for
    industry awareness
  • Provide feedback to appropriate ADL tool
    developers

102
FY00 Progress
  • Participated in development of SAEs Avionics ADL
    (AADL) standard
  • Explore ADL standardization efforts in IEEE and
    Open Group
  • Established partnerships with DISA, US Army,
    ITL/Security Division
  • Identify domain areas for applying ADL to explore
    standardization and testing issues
  • E.g., government smartcard, unmanned ground
    vehicle, pervasive computing

103
FY01 Plans
  • Provide support for the formalization of AADL to
    ensure its testability
  • Develop prototype of US Army robotics
    architecture
  • collaboration with MEL
  • dependent on DISA funds
  • Develop prototype of Smartcard Architecture w.
    Div 893
  • Demonstrate and document the benefits and
    functionality of ADLs
  • Produce a comparative study of different ADLs to
    support further development and standardization
    of ADLs

104
FY00 Resources
  • 1 FTE
  • 200K (STRS)
  • FY01 Resources
  • 2 FTE
  • Equipment and Software 10K
  • Travel required to standards meetings and to
    participate in conferences and industry meetings

105
Architecture Description Language
  • Customers
  • Consumers Industries that rely on
    component-based systems (e.g., manufacturing,
    avionics, real-time)
  • ADL tool vendors
  • Impact
  • Moved ADL from University research to industry
    standard
  • Provide uniform descriptions of the structure and
    function of software.
  • Increases software quality by reducing errors
    through testing in design phase.
  • Increase the producibility, quality, and
    reusability of component-based software systems.

106
GIS
Project Leader Chris Dabrowski
107
FY00 Progress
  • Convener of ISO/TC211 Framework and Reference
    Model Working Group
  • Initiated and lead effort to use UML to formalize
    and integrate GIS family of standards
  • Shared lessons learned with OMG
  • Presented UML/GIS work at OMG meetings
  • Provided recommendations for the evolution of the
    UML language

108
FY01 Plans
  • Complete integration of UML in GIS standards
  • Sunset project

109
FY01 Resources
  • 0.25 FTE
  • Limited Travel anticipated to transition ISO GIS
    work and contribute experiences to OMG

110
Geographic Info Systems
  • Customers
  • Consumers Industries with GIS requirements
    (e.g., GPS industry)
  • Developers of GIS information systems
  • ISO TC 211, the OpenGIS consortium, Federal
    Geographic Data Committee (FGDC), Object
    Management Group (OMG)
  • Impact
  • enable industry growth and preclude barriers to
    business activity
  • provide a basis for exchange of information in
    heterogeneous environments distributed over space
    and time
  • Less ambiguities in the UML specification
  • Increased use of UML in standards (normative)

111
CIO Council Standards Working Group
Technology Transfer Act/OMB Circular A-119 direct
federal agencies to use voluntary consensus
standards in lieu of government unique
standards NIST directed to coordinate federal
technical standards activities with private
sector technical standards activities NIST/ITL
initiated effort to create Standards Working
Group within Federal CIO Council Standards
Working Group approved late last year by the CIO
Council Interoperability Committee Standards
Working Group to be focal point of discussion,
definition, and coordination of federal
requirements for voluntary IT standards, and will
assess the results of voluntary IT standards in
meeting requirements Standards Working Group
chaired by a member of this Division, and will be
supported by this Division in its efforts to
provide a unified voice to the voluntary IT
standards community for Federal IT standards
technical requirements
112
FY00 Progress
  • Prototype of web based support software for
    collection, evaluation, and status reporting on
    Federal Government technical requirements for
    voluntary standards developed and demonstrated.
  • Electronic Documents conference held featuring
    speakers from private industry, Federal
    Government, and voluntary standards
    organizations.
  • Meetings of the Standards Working Group held on
    an as needed basis.
  • Standards requirements forwarded to appropriate
    voluntary standards committees as the
    requirements are approved.

113
FY01 Plans
  • Continue development of web based support
    software for Standards Working Group.
  • Make software available for possible use by other
    Working Groups within the EIEIT structure.
  • Meetings of the Standards Working Group held on
    an as needed basis.
  • Standards requirements forwarded to appropriate
    voluntary standards committees as the
    requirements are approved.
  • Consider holding second Electronic Documents
    conference.

114
FY00 Resources
  • 1.5 FTE
  • 325K (STRS, OA-CIO Council)
  • FY01 Resources
  • 1.5 FTE Combination of time from Group Leader
    and others in Interoperability Group

115
CIO Council Standards Working Group
  • Customers
  • Other Federal Agencies
  • Impact
  • Federal agencies input requirements to
    standards groups and utilize voluntary consensus
    standards
  • Eliminates duplicative standards

116
Strategy
  • Conformance Tests
  • Reference Implementations
  • Standards developed jointly with industry
  • Diagnostic Tests
  • Research to reduce costs of testing
  • Reference Data

117
Software Diagnostic Tools
  • Program slicer tool - Project Leader Jim Lyle
  • Computes lines of code which could affect given
    variables
  • Reduces debugging time by directing effort to
    relevant parts of computation
  • Currently only for C programs
  • Extending slicer to Java and Fortran
  • Commercialized by private company as Y2K solution
  • VM View Toolkit - Project Leader Alden Dima
  • Diagnostic tool embedded at runtime into the JVM
  • Exposes internal VM conditions in the form of an
    execution trace
  • Downloaded by Sun, HP, Rockville Collins, Apogle,
    Insignia, Wind River, Ada Core, Johns Hopkins

118
Year 2000 Problems - Project Leader Gary Fisher
Developed source code scanner
Participate in Interagency Year 2000 Task Force
119
FY00 Progress
  • Y2K web site completed
  • Embedded system study completed
  • Report sent to Koskinen
  • Problems highlighted and impacts reported
  • Produced new emphasis on embedded systems
  • Project sunset

120
FY00 Resources
  • 0.25 FTE
  • 50K (STRS)
  • FY01 Resources
  • 0 (Project sunset)

121
Year 2000
  • Customers
  • All businesses that use computers
  • Consumers who use computers
  • Impact
  • Y2K rollover went without incident

122
Strategy
  • Conformance Tests
  • Reference Implementations
  • Standards developed jointly with industry
  • Diagnostic Tests
  • Research to reduce costs of testing
  • Reference Data

123
Automatic Generation of Spec Based Tests Using
Formal Methods
Project Team Paul Black Bill Majurski Jim
Lyle Micheal Koo
124
Industry Need
  • Software testing is enormous cost to industry
  • 30 - 50 of budget
  • Development may be 1000s of staff-years
  • Testing cost may be millions
  • Current methods mostly hand generated tests
  • Inefficient
  • Not rigorous enough
  • Software testing key to reducing security
    incidents
  • More than 75 of the incidentsare the direct
    result of bugs. and more real testing - not
    'beta testing' in the marketplace - will yield
    programs that are less vulnerable to attack.
  • (Business Week, February 28 "Locking Out the
    Hackers")

125
(No Transcript)
126
Current Test Development Methodology
Requirements
English Language
review inspect
Specification
English Language
Human Programmers
review inspect
Source Code
Software Tests
Programming Language
127
Advanced Test Development Methodology
Requirements
Mathematical Logic
formal proof
Specification
Mathematical Logic
Test Generator
informal proof
Source Code
Software Tests
Programming Language
128
Accomplishments
  • Method for automatically generating tests from
    formal specifications
  • Associated coverage metric
  • Publications and presentations
  • ICFEM (98), WSEST(99), HASE (99), DASC (99)
  • GMU, UMBC, College Park, Brisbane, NRL
  • Feasibility collaborations leading to
    commercialization
  • Argus, a secure computer operating system
  • Ford Motor Research Lab powertrain control
    systems
  • Northrop Grumman fighter radar computers

129
FY00 Progress
  • Feasibility tests on models from Ford and Argus
  • Initial formalization of BLAST (Basic Local
    Alignment Search Tool identifies similar DNA
    sequences) Algorithm completed in collaboration
    with CSTL
  • Hosted Workshop on Testing Software Using Formal
    Methods
  • Hosted Tutorial on Formal Methods
  • 3 papers published
  • 2 papers submitted and accepted
  • 4 papers submitted
  • 9 talks (6 invited)

130
FY01 Plans
  • Commercialize ATG Method with Ford and software
    company (tbd)
  • Continue feasibility studies with Argus
  • Complete BLAST formalization

131
FY00 Resources
  • 2.5 FTE
  • 500K (STRS-Competence)
  • FY01 Resources
  • 2.5 FTE
  • Equipment 5K
  • Travel required for validation collaborations and
    presentations

132
Automatic Generation of Tests
  • Customers
  • Software developers
  • Software end users, e.g., web surfers
  • Security administrators
  • Researchers in software testing methodologies
  • Impact
  • Reduce the cost of software testing (50 of
    development costs)
  • Improve software quality
  • Reduce security incidents (75 due to bugs)
  • Ford and Argus interested in commercialization
    of ATG methodology
  • Strongly endorsed by VCAT

133
Software Testing by Statistical
Methods(Competence Project)
  • Objective develop new methods for software
    testing based on stochastic processes and
    metrics, in order to
  • Improve the quality of software development
  • Develop reliability estimates that software
    correctly adheres to its specification
  • Joint project
  • Software Diagnostics and Conformance Testing
    Division (897)
  • Statistical Engineering division (898)
  • Collaboration with other NIST Labs (MEL, EEEL,
    CSTL)

Project Team Len Gallagher Lynne Rosenthal
134
(No Transcript)
135
Specification
Conformance Tests
136
FY00 Progress
  • Published paper on Coverage Model of Mutation
    Testing (with Paul Black)
  • Produced paper on applying statistical methods to
    component based testing
  • Produce paper on Test Input Needed to Improve
    Reliability
  • Analysis of effort to apply statistical method
    (partitioning) to new test development

137
FY01 Plans
  • Continue effort to apply statistical method
    (partitioning) to new test development
  • Apply statistical method to pervasive computing
    applications

138
FY00 Resources
  • 1 FTE
  • 200K (STRS-Competence, plus 100K given
    to ITL/SED)
  • FY01 Resources
  • 1 FTE
  • 200K competence funding (plus 100K given to
    ITL/SED)

139
Software Testing by Statistical Methods
  • Customers
  • Conformance Test Developers
  • Ourselves
  • Users of conformance tests
  • Impact
  • Improve our knowledge and skills in designing and
    developing quality software.
  • Quantitative measures associated with conformance
    testing.

140
Quantum Information
Project Team Paul Black Neva Carlson
141
Quantum Information
  • Goal to develop information representations and
    computing machines that take advantage of quantum
    effects
  • Joint effort of physicists and computer scientists

142
FY00 Progress
  • Invited to develop joint proposal for
    Nanotechnology Initiative with Physics
  • Invited by NIST Physicists to present Basic
    Concepts in Theory of Computation
  • Developed Quantum Information Demo
  • Developed Quantum Information Bibliography
  • Attended weekly Physics Quantum Information
    seminar

143
FY01 Plans
  • Develop products for Nanotechnology Quantum
    Information project
  • Organize Workshop on Computer Science in Quantum
    Information
  • Research questions in testing Quantum Algorithms

144
FY00 Resources
  • .5 FTE
  • 100K (STRS Physics Lab?)
  • FY01 Resources
  • 1.5 FTE (depends on Nanotechnology Initiative or
    from Physics)
  • Equipment 0
  • Travel anticipated

145
Quantum Information
  • Customers
  • Researchers in computational theory, and
    algorithm design and analysis
  • Developers of large computer models, e.g.,
    global weather
  • Cryptanalists
  • Impact
  • Computing machines of unprecedented
    capability and small size
  • Solutions to intractable computational
    problems, e.g., factoring, searching
  • ITL staff invited to present at Physics seminars
    and participate in Physics Lab efforts

146
Strategy
  • Conformance Tests
  • Reference Implementations
  • Standards developed jointly with industry
  • Diagnostic Tests
  • Research to reduce costs of testing
  • Reference Data

147
Error, Fault, and Failure (EFF)Data Collection
and Analysis
Project Team Dolores Wallace Michael Koo Tony
Cincotta Jason Rager
148
Error, Fault, and Failure (EFF)Data Collection
and Analysis
  • Repositories of reference data on software
    failures
  • Tools to create and manage repositories
  • Tools to analyze software failure data
  • Research in taxonomies to characterize software
    failures

149
FY00 Progress
  • Repository systems
  • Completed public repository with predefined
    queries
  • Completed internal repository with dynamic
    queries
  • Data 1000 data points added
  • Paper on failures in medical devices finalized
  • Papers report on NASA data submitted to NASA
  • Project was sunset

150
FY00 Resources
  • 1.5 FTE
  • 300K (STRS, ATP - 160K)
  • FY01 Resources
  • 0 (Project sunset)

151
Error, Fault, and Failures
  • Customers
  • Software Developers
  • Researchers in Software Development
    Methodologies
  • Impact
  • Identifies the stages in the software life cycle
    whose practices can be improved
  • Endorsed by the Software Division of the
    American Society for Quality
  • Project methodologies and tools used in
    graduate programs at Johns Hopkins and the
    University of Louisville
  • EFF Tool used by RST Corporation

152
Other Agency Projects
  • NIJ - Software and Tool Verification
  • Project Team Gary Fisher, Len Gebase, Jim Lyle,
    Jackie Schneider, Mark Zimmerman
  • VA - Health Care Information Systems
  • Project Leader Bill Majurski
  • GSA - EC Standards Roadmap
  • Project Leader Mary Brady
  • EPA - Metadata Descriptions and Registries
  • Project Leader Judy Newton

153
NIJNational Software Reference Library Beta
Testing
154
NIJNational Software Reference Library Beta
Testing
155
The NSRL BIG Picture
Requirements Funding and Feedback
SW Manufacturers
NIJ
MOU
Contact
Software
Law Enforcement Agencies
NIST
NSRL
Admin.
Research and Production
Use
NSRL DB
File Signatures
156
  • National Software Reference Library
  • GOAL Reference Data Set of file signatures
    (hash) of COTS file
  • Used during examination of digital evidence to
    identify pertinent files and eliminate others
  • BOTTOM LINE One computer may have 5,000-20,000
    files. NSRL data set can eliminate 40-95 of
    those files and save hundreds of staff-hours.
  • Beta Testing Project
  • GOAL Verification of computer forensic tools and
    their use in analyzing digital evidence
  • Used to ensure tool usage yields objective,
    repeatable, reproducible, results that hold up in
    court.


157
FY00 Progress
  • Developed NSRL data dictionary and database
    structure for hash codes
  • Established contacts with Microsoft, Adobe, et al
    to supply software
  • CBD announcement, Press release
  • Identified hash algorithms
  • Creating Reference Data Set (i.e, file
    signatures)
  • Developed draft test plan for beta testing
  • Develop procedures and tests for imaging tools

158
FY01 Plans
  • Establish the reference data set and distribution
    mechanism
  • (SQL database with web interface)
  • Develop test metrics for efficacy of hash codes
    used in the reference data set
  • Test the NSRL web site and make it available to
    approved users
  • Continue to develop tests and procedures for
    forensic tools
  • Issue test reports and test suites (publish on
    web site)

159
FY00 Resources
  • 3.5 FTE
  • 700K (OA-NIJ)
  • FY01 Resources
  • 3.5 FTE
  • Travel required for training courses and
    presentations

160
NIJ
  • Customers
  • Law Enforcement Agencies (e.g., FBI, Treasury,
    DCFL, state and local agencies)
  • Software companies (e.g., Adobe, Microsoft)
  • Impact
  • Improve the efficiency of forensic examination of
    suspect computer files/date (NSRL data set can
    eliminate 40-95 of suspect files and save
    hundreds of staff-hours).
  • Provide testing procedures for forensic software
    tools to ensure the analysis process and results
    holds up in court

161
Healthcare Information Systems
Project Team Bill Majurski Tony Cincotta Roger
Gima John Barkley
162
Objectives
  • Improve the quality of Healthcare Information
    Systems by
  • Designing distributed models and architectures
    for Healthcare Information Systems
  • Developing reference implementations
  • Technology transfer of RBAC
  • Participating in the development of standards,
    e.g., OMGs RAD, G-8 Healthcare Data Card Project
  • Support from Veterans Affairs and NIST ATP

163
NIST Role
  • Consulting Architect to VA (major healthcare
    provider 170 hospitals)
  • Assess new technologies
  • Developed design for RPC (Remote Procedure Call)
    Broker
  • Designed and implemented Enterprise Single
    Sign-On (ESSO)
  • Designing and implementing Inter-Organizational
    Role Based Access Control (IORBAC) authorization
    mechanism

164
FY00 Progress
  • Enterprise Single Sign-On (ESSO) beginning
    national trials
  • Inter-Organizational RBAC (IORBAC) V1 Reference
    Implementation completed (to be used for
    requirements development)
  • OMG Resource Access Decision Facility (RAD)
    proposed for IORBAC
  • NIST designs become permanent part of VA HIS
    Architecture
  • Now serve as permanent consultant to VA
    Architecture Design Group
  • Invited to speak at two VA HIS Conferences

165
FY01 Plans
  • Implement IORBAC Reference Implementation V2
    which includes RAD
  • Technology transfer of ESSO to VA developers
  • Review G8 Healthcare Smartcard Specification and
    provide advice on conformance testing

166
FY00 Resources
  • 2.5 FTE
  • 500K (OA-VA, ATP-200K)
  • FY01 Resources
  • Personnel 3 FTE
  • Equipment 20K
  • Travel required to participate in Architecture
    Design meetings and deployment of implementations

167
Health Care Information Systems
  • Customers
  • HIS developers and users
  • Standards bodies developing standards in HIS
  • Developers of authentication and access control
    mechanisms for protecting the privacy of patient
    information
  • Impact
  • Higher quality HIS and improved patient privacy
  • NIST designs permanent part of VA HIS
    Architecture
  • Remote Procedure Call (RPC) Broker in use in all
    VA Hospitals
  • ESSO selected for national trails
  • RAD adopted by OMG, implemented by 2AB, DASCOM

168
GSA - EC Standards Roadmap
169
FY00 Progress
  • Developed initial report on scope of effort,
    standards of interest, and design of web-based
    roadmap
  • Develop initial prototype of roadmap
  • Coordinate activities with appropriate GSA and
    CIO groups

170
FY01 Plans
  • Complete interactive standards roadmap
  • Pursue with MEP, deployment of roadmap for SMEs

171
FY00 Resources
  • 0.5 FTE
  • 75K (OA-GSA)
  • FY01 Resources
  • 1 FTE
  • GSA Funding required

172
EPA - Metadata Registries
173
FY00 Progress
  • Advance the standards effort in the ISO/NCITS
    arena by continuing to serve as a member of
    ISO/IEC SC 32 WG 2 and Chair of NCITS L8 SIG
  • Prepare new proposed standards and technical
    reports for Metadata Registries (MDR).
  • Coordinate and encourage implementations of
    metadata registries based on those standards by
    hosting the Metadata Registry Implementers
    Coalition (MDRIC).
  • Advance progress on the EDEN project by COTR
    activities.
  • Publicize activities through presentations and
    publications in national and international fora.

174
FY01 Plans
  • Continue the work shown above under FY00
    Progress.
  • Serve as Editor of Part 5 of ISO/IEC 11179 which
    is up for 5-year review.
  • Serve as Editor of the Technical Report
    Procedures for Achieving Metadata Registry
    Content Consistency, which is to be balloted for
    CD.

175
FY00 Resources
  • 1 FTE
  • 200K (OA-EPA)
  • FY01 Resources
  • 1 FTE
  • Travel required to participate in ISO meetings

176
Industry Testimonials
  • The OASIS-NIST XML Conformance Test Suite is
    critical for our industry.
  • Norbert Mikula, CTO, OASIS
  • N
View by Category
About This Presentation
Title:

Mark W. Skall, Division Chief

Description:

Program Review Software Diagnostics And Conformance Testing Mark W. Skall, Division Chief Candy Leatherman, Secretary email: skall_at_nist.gov Tel. No.: 301-975-3262 – PowerPoint PPT presentation

Number of Views:57
Avg rating:3.0/5.0
Slides: 164
Provided by: CandyLea2
Learn more at: http://www.itl.nist.gov
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Mark W. Skall, Division Chief


1
Program Review
Mark W. Skall, Division Chief Candy Leatherman,
Secretary email skall_at_nist.gov Tel. No.
301-975-3262 Div. Web Site http//www.itl.nist.g
ov/div897/
2
Division 897 Funding - FY00
Projected Balance 257,062 ( 637,570
transferred to 2001 OA Reserve)
OA - 1,615K
ITS - 585K
STRS - 5,207K
Employees 35 Full Time 3 Part Time 10
Intermittent 5 Guest Researchers
3
SDCTs GOAL
  • To improve quality of software in industry
    through the development of
  • Technology
  • Measurements
  • Standards
  • Philosophy
  • Concentrate on key areas at forefront of
    technology
  • Get involved early and partner with industry
  • Fill industry void
  • Transfer technology and move on
  • Specific Strategy
  • Conformance Tests
  • Reference Implementations
  • Diagnostic Tests
  • Research to reduce costs of testing
  • Reference Data
  • Standards developed jointly with industry

4
Excerpts from Assessment Panel Report
6/26/00Division Review
The planning and documentation methods used by
SDCT could serve as a model for other ITL
divisions. The process has used clearly-defined
criteria to select state-of-the-art programs with
clearly defined priorities and goals,
well-identified roles, and measurable
contributions to national and international
standards organizations. Examples include the
work on conformance tests for XML, Distributed
Interactive Learning Systems, and for DASE . .
.Directions can change quickly within the
software industry. The division carefully
monitors these changes and acts appropriately. .
. .Industry has provided many testimonials
concerning the great value of the divisions
standards work, including The OASIS-NIST XML
Conformance test suite is critical for our
industry, Norbert Mikula, CTO, OASIS NIST made
strong contributions (X3D) and resolved our
knottiest problems, Don Brutzman, Board of
Directors, Web3D consortium, and The high
quality test suite and certification program is
an invaluable resource for ATA, Robert Peel,
Director of Airworthiness and Standards
5
Excerpts from Assessment Panel Report
6/26/00Laboratory Level Review
New projects are started based on the importance
of the work to U.S. industry, and the work viewed
by the panel generally had specific, focused
goals. An example of this approach is the work
on XML done in SDCT. This project is notable
because it addresses a major standard on which
industry was making little progress, and it also
effectively leveraged skills and approaches
developed in a now terminated project on VRML. .
.Another example of the improved planning process
is the existence of clear termination criteria,
which have been useful in sunsetting projects.
For example, the Real-Time Java project and the
work on RBAC satisfied their completion criteria
and therefore ended this year.
6
  • Standards and Conformance Testing Group
  • Software Quality Group
  • Interoperability Group

7

Group Leader Lynne Rosenthal Supy Proj. Leader
Lisa Carnahan Clare Lucey, Secretary
8

Mary Brady Lisa Carnahan Laurent Ciarletta
(GR) Anthony Cincotta (S) Chris Dabrowski Alden
Dima Leonard Gallagher Leonard Gebase Neil Gima
(S) Alan Goldfine
Martha Gray Michael Kass Clare Lucey Thomas
Logue Carmelo Montanez-Rivera Thomas
Rhodes Richard Rivello Jacqeline Schneider John
Tebbutt Marie-Noelle Terasse (GR) Mark Zimmerman
9
Division 897 Standards and Conformance
Testing Group
10
SQG
Software Quality Group
  • Develop methods to automate software testing
  • Develop software diagnostic tools
  • Develop reference data
  • Formal methods

Group Leader John Barkley Debbie Blackstone,
Secretary
11
SQG
Software Quality Group
Tamer Ahmed (GR) Paul Ammann (F) Debbie
Blackstone David Brinkley (F) Paul Black Neva
Carlson John Cherniavsky Anthony Cincotta
Keith Gallagher (F) Roger Gima (S) Michael
Koo Mary Laamanen James Lyle William
Majurski Douglas White
12
Division 897 Software Quality Group
13
IG
Interoperability Group
  • Ensure Federal agency requirements are input into
    voluntary standards committees - work through
    Federal CIO Council
  • Help Federal agencies and industry achieve
    interoperability through application of Division
    products
  • Provide technical support to voluntary standards
    committees
  • Support NIST paperless office effort utilizing
    digital signatures

Acting Group Leader Lisa Carnahan Vacant,
Secretary
14
IG
Interoperability Group
Daniel Allen Bruce Bargmeyer (RA) Frederick
Boland Gary Fisher Larry Fitzwater (GR)
Elizabeth Fong Kathryn Harvill Roy Morgan Judith
Newton Gertrude Sherwood
15
Division 897 Interoperability Group
16
Strategy
17
Development of Conformance Tests
  • Emphasis on newer technologies
  • Tests developed in parallel with standards
  • Tests developed before implementations
  • Need buy-in from industry
  • Need high impact
  • Need technology transfer

18
Development of Conformance Tests (contd)
  • Legacy activities
  • CGM
  • SQL
  • POSIX
  • Ada and other programming language
  • VRML
  • Current activities
  • XML and companion standards (DOM, XSL, etc.)
    Digital TV
  • X3D
  • Learning Technologies (IMS)

19
XML Conformance Testing
Project Team Mary Brady Carmelo Montanez Ricky
Rivello Mark Zimmerman

20
Electronic Commerce
B2C (business-to-consumer) Interactive Televisio
n (DASE, SMPTE)
B2B (business-to-business) Manufacturing (ICM)
Distributed Learning (IMS)
XML Technologies
Internet / World Wide Web
21
Why NIST?
  • Electronic Commerce Growth
  • Growth is exponential, already outpacing last
    years predictions
  • Changing the way businesses do business
  • Revolutionizing the way we consume and play
  • Advances are necessary in
  • Tools for describing / sharing information
  • XML technologies
  • Applying IT to solve vertical market demands
  • NIST provides
  • XML Technologies
  • Conformance Testing
  • NIST developed tests, available on-line, for use
    in testing applications
  • NIST technical leadership, neutral third-party
  • Unbiased Feedback of Specification Errors
  • Vertical Markets
  • Applied IT research

22
HTML or XML?
  • ltPurchasergt
  • ltNamegt Mary Brady lt/Namegt
  • ltAddress StreetNN Rm 572 CityGaithersburg,
    StateMD Zip20899lt/Addressgt
  • lt/Purchasergt
  • ltItemgt
  • ltPart_numbergt1234lt/Part_numbergt
  • ltItemDescriptiongtTreeslt/ItemDescriptiongt
  • ltCostgt50.00lt/Costgt
  • ltQuantitygt20lt/Quantitygt
  • ltTotalCostgt1000.00lt/TotalCostgt
  • ltInstructionsgtBe sure to water the root ball day
    of deliverylt/Instructionsgt
  • lt/Itemgt
  • ltShipTogt
  • ltNamegtSusan Carscaddenlt/Namegt
  • ltAddress StreetNIST, Bldg 301 CityGaithersburg
    StateMD Zip20899lt/Addressgt
  • lt/ShipTogt

Figure 2 XML Purchase Order
23
XML Technologies
Electronic Commerce, Distance Learning, Health
Care Manufacturing, Interactive Televison
Programming Languages
Transformations Formatting Objects
Signatures
Stylesheets
Query Language
DOM
Domain Specific
Registries Repositories
Namespaces
XML.ORG BIZTalk
E-Business Learning Objects XML/EDI
Boards
Medical
Education
Procurement
Auto Parts
...
...
Ratings
Libraries
Control
...
Information Description
XML Syntax, Information Set, Schema, Linking,
Fragment
24
XML Conformance Testing
  • OASIS XML Conformance
  • Sun, IBM, Fuji Xerox, DataChannel, MicroStar, W3C
    members

25
DOM Conformance Testing
  • W3C, OASIS
  • Sun, IBM, Microsoft, Netscape, Oracle

26
FY00 Progress -Tests
  • Completed XML Test Suite (release 1)
  • Completed XML Test Suite (release 2)
  • Incorporated new XML tests and fixes
  • Completed DOM-Ecmascript Test Suite (release 1)
  • Completed DOM-Java Test Suite (release 1)
  • Develop XSLT Test Suite
  • partnership with Lotus Corp. and OASIS

27
FY00 Progress -Committee Leadership
  • Chair OASIS Conformance and XML Testing
    Committees
  • Provide leadership for XSLT and Schema Testing
    committees
  • Provide leadership for UN/CEFACT ebXML initiative
    (conformance and reg/rep)
  • Provide conformance guidance
  • Develop conformance clause
  • Harmonize OASIS and ebXML efforts

28
FY01 Plans
  • Complete XSLT test suite
  • Develop test suites for XML Schema and other XML
    Technologies
  • Develop XML-based automated test tools to improve
    test development
  • Continue to chair OASIS testing
    committees

29
FY00 Resources
  • 3.5 FTE
  • 700K (STRS)
  • FY01 Resources
  • 5 FTE
  • Travel required to collaborate in standards and
    test development efforts
  • Equipment 10K

30
XML Conformance Testing
  • Customers
  • IT industry e.g., IBM, Sun, Microsoft, Oracle,
    DataChannel, Documentum
  • Vertical industries e.g., Education (IMS),
    Financial (e.g., Dun and Bradstreet), Travel
    (e.g., Sabre)
  • Impact
  • Conformance test suites have been used to improve
    the quality of XML/DOM processors (many of which
    are embedded in other XML software solutions).
    These are used in many vertical markets by
    millions of customers.
  • Feedback to standards developers improve the
    specifications

31
Interactive TV
Project Team Alan Goldfine John Barkley Doug
White Len Gebase
32
Interactive TV
  • TV and the Internet are each recognized as a
    technology that seems to change everything (BW,
    Oct 4 99)
  • Interactive TV is the convergence of TV and the
    Internet
  • Profound impact on Electronic Commerce expected
  • WebTV 1 million subscribers, 350 hrs/wk of
    programming
  • AOLTV projected 2 million subscribers in 2
    years
  • ITL partnering to develop the standards and
    tests required for the success of Interactive TV

33
Interactive TV
Traditionally separate media are converging
  • Wireless phones do email
  • Internet does broadcast radio and TV
  • Broadcast TV references links to Web pages

34
Interactive TV
Currently, for the viewer
35
Interactive TV
Demonstration
Current production
36
Interactive TV
Demonstration
37
Interactive TV
Advanced Television Systems Committee (ATSC)
  • Members include ABC, CBS, IBM, Intel, Lucent,
    Microsoft, NBC, Warner Brothers
  • DASE (DTV Application Programming Environment)
    Standard for a platform independent, high-level
    abstraction for integrating DTV and the Internet
  • DASE specifies programming environment for DTV
    receivers
  • Downloadable DASE applications portable across
    different TV set tops

38
Interactive TV
NISTs Role in ATSC DASE
  • Joint project with the High Performance Systems
    and Services Division (895)
  • Chair DASE Conformance Working Group where key
    players include ABC, Gateway, Microsoft, Sun
  • Partner with Unisoft to develop conformance
    tests
  • Reference implementation for programming
    environment

39
Interactive TV
Society of Motion Picture and Television
Engineers (SMPTE)
  • Members include ABC, CBS, IBM, Intel,
    Microsoft, NBC, Warner Brothers
  • DDE (Declarative Data Essence) Standard for
    Internet content and bindings to analog and
    digital streams (formerly ATVEF)
  • Declarative Content HTML, CSS, ECMAScript, DOM,
    Triggers (for syncing the two mediums)

40
Interactive TV
NISTs Role in SMPTE DDE
  • Contribute text to specification
  • Partner with Unisoft to develop conformance
    tests
  • Develop conformance tests for triggers and
    stream binding to integrate with existing
    Unisoft DDE element tests
  • Develop prototype receiver testbed
  • Participate in Applications Data Essence (ADE)
    Study Group

41
FY00 Progress
  • Chair DASE conformance task group
  • Conformance section of the DASE Standard
    completed
  • Review of DASE conformance test assertions from
    Unisoft completed
  • DDE Standard completed, adoption underway
  • DDE Bindings Standard underway
  • Test assertions for DDE triggers underway
  • Testbed for DDE receivers underway
  • ADE Study draft completed

42
FY01 Plans
  • Continue to chair DASE conformance group
  • DASE Standard adopted
  • Continue DASE conformance test development with
    Unisoft
  • DDE and DDE bindings Standards adopted
  • Develop DDE conformance tests for triggers,
    integrate with Unisoft tests
  • Develop test assertions for DDE bindings
  • Complete DDE Receiver testbed
  • Begin DDE-2 Standard development

43
FY00 Resources
  • 2 FTE
  • 400K (STRS, ATP-100K)
  • FY01 Resources
  • 4.5 FTE
  • Equipment 20K
  • Travel required for standards meetings and
    testing collaborations

44
Interactive TV
  • Customers
  • IT, e.g., IBM, Microsoft
  • broadcast TV, e.g., NBC, DirecTV
  • entertainment, e.g., Disney, Universal
  • consumer electronics, e.g., RCA, Intel
  • Impact
  • New Medium TV/Internet convergence
  • Interactive applications portable across
    receivers from different manufacturers
  • ATSC and SMPTE invited NIST participation
  • NIST chairs DASE Conformance Group

45
X3D Conformance Testing
Project Leader Mike Kass
46
Objectives
  • Continue the NIST/Web3D partnership begun in 1996
  • ITL created the VRML Test Suite (VTS) and Viper
    parser
  • Create an X3D Test Suite ( XTS )
  • Modify VRML test requirements and test
    descriptions
  • Translate existing VRML tests to X3D format
  • Chair the Interoperability/Conformance WG
  • Interface with developers and specification
    writers to resolve ambiguities in specification
  • Provide feedback to developers on browser
    conformance

47
FY00 Progress
  • Completed translation of VTS to X3D Test Suite
    (XTS)
  • Developed test harness to enable developers to
    contribute tests
  • Interest from DraW and Sun for using the test
    suite and making the results public

48
FY01 Plans
  • Low level support - assisting community in using
    the VRML and X3D Test Suites
  • No new development planned. Maintenance as
    needed

49
FY00 Resources
  • 0.75 FTE
  • 150K (STRS)
  • FY01 Resources
  • 0.2 FTE (minimal)
  • No Travel is expected

50
X3D Conformance Testing
  • Customers
  • X3D Consortium including Sun, Sony DraW
    Computing, Shout, Blaxxun
  • Impact
  • Conformance test suites are being used to improve
    the quality of X3D browsers
  • DraW Computing has said that they would make
    their test results public

51
Learning Technologies
Project team Martha Gray Tom Rhodes Jacki
Schneider Lisa Carnahan
52
Learning Technologies
  • Learning technology is
  • Learning anytime, anyplace
  • Interactive, distributive learning systems
  • In collaboration with industry, NIST/ITL is
  • defining requirements and specifications
  • developing prototypes and reference
    implementations
  • developing testing techniques and tools

53
Why Now?
  • The Gartner Group estimates that companies in
    the United States spent 58 billion on training
    in 1998, and of that, 9.4 billion went to
    IT-related training. They expect this figure to
    grow to 10.5 billion this year. Gilbane
    Report, July/Aug 1999
  • Just as electronic commerce exploded over the
    past two years, the stage is now set for
    e-learning to become the next big wave in
    Internet-based applications Cisco Systems CEO
    and President John Chambers
  • Worldwide, the education and training market is
    over a trillion (US) dollars. IEEE

54
Projects
  • IMS Global Learning Consortium, Inc.
  • Chair conformance team
  • advise members on writing IMS specifications
  • ensure testability of specifications
  • advise on how to test and set up testing programs
  • Develop Meta-data repository reference
    implementation
  • IEEE and ISO
  • Provide conformance expertise
  • Harmonization of efforts among specifications
  • DoDs Advanced Distributive Learning (ADL)
  • Executive Order 13111 on Federal Training -
    NIST/ITL co-leaders with ADL and DOL.

55
FY00 Progress
  • Continue collaboration with IMS, IEEE LT
    committee and NIST/ATP
  • Chair IMS conformance testing activities
  • Developed IMS conformance test policies and
    procedures
  • Served as Technical Lead of SBIR solicitation,
    assessed and recommend proposals
  • Develop repository of learning objects (this work
    coordinates with Reg/rep subproject)

56
FY01 Plans
  • Continue to chair IMS conformance committee
    providing expertise to emerging IMS
    specifications
  • Develop a conformance test plan for applicable
    IMS specifications

57
FY00 Resources
  • 1.5 FTE
  • 300K (STRS, ATP)
  • FY01 Resources
  • 1 FTE
  • Minimal travel required to attend standards
    meetings and interoperability demos

58
Learning Technologies
  • Customers
  • IMS, DoL, DoD, SimonSchuster, Sun, IBM, Apple,
    Microsoft, UNC, Calif State U
  • Impact
  • Use of standard metadata for global learning will
    improve discovery, evaluation, and use/reuse of
    learning objects
  • NISTs conformance expertise contributed to
    clearer, less ambiguous, testable IMS
    specifications
  • Conformance test plans and procedures will enable
    IMS to determine compliant products

59
ATA CGM Testing Program
60
FY00 Progress
  • Completed final release of the ATA CGM Test Suite
  • Provided guidance to ATA Recognized Test
    Laboratory on using the test suite and assessing
    clients
  • Chair Control Board - resolve questions of
    interpretation of the test suite and standard.
  • Initiated discussions on ATA's eCommerce
    activities - explore a possible XML testing
    program

61
FY01 Plans
  • Continue to support ATA's CGM Test Program
  • No new CGM development planned
  • Maintenance as needed
  • Provide guidance on the use of NIST's XML test
    suite and its applicability and use in ATA

62
FY00 Resources
  • 0.25 FTE
  • 75K (STRS)
  • FY01 Resources
  • 0.25 FTE
  • Travel required to support ATA test program and
    potential for an ATA XML test program

63
CGM
  • Customers
  • Air Transport Association Airlines and their
    suppliers (e.g., United, Northwest, Airbus, FedX,
    Boeing, Rolls-Royce)
  • CGM Developers (e.g., Autotrol, Intergraphics)
  • Impact
  • NIST Conformance test suite used by ATA CGM Test
    Program to validate CGM interpreters
  • Quality, reliable CGM products used for
    representing graphics in airframe maintenance
    manuals

64
(No Transcript)
65
Technology Transfer from NIST to ATA
Developed CGM Interpreter Test Suite Developed
Test Program Framework Developed test procedures
Formed and chair Control Board Developed Test
Lab criteria and evaluation criteria Assessed
Test Lab for ATA recognition Serve as technical
adviser Completed transfer
66
Testing Services
Status of Software Operational Testing Services
67
Testing Services (contd)
Testing Products or Services - Offered by Other
Organizations -
Developed with NIST Assistance
68
Strategy
  • Conformance Tests
  • Reference Implementations
  • Standards developed jointly with industry
  • Diagnostic Tests
  • Research to reduce costs of testing
  • Reference Data

69
(No Transcript)
70
AromaPervasive Computing Project
Project Team Alden Dima Bill Majurski Mary
Laamanen
71
AromaPervasive Computing Project
  • Emerging trend toward
  • numerous, easily accessible, often inconspicuous
    computing devices and sensors mobile or embedded
    in the environment connected to ubiquitous
    network structure
  • Industry is asking for
  • Standards for service discovery, APIs, e-commerce
    services
  • Testing and diagnostic tools to improve quality,
    interoperability
  • Reference implementations
  • Aroma multi-division effort to
  • Explore standardization and measurement issues
  • Attract industry interest and partnerships

72
Aroma Prototype (Aroma adaptor Aroma projector)
  • Goals - To discover technical, standards and
    measurement issues related to
  • Connecting portable wireless devices to
    traditional networks
  • Service discovery, self-configuration and dynamic
    resource sharing
  • Mobile code and data
  • Software infrastructure needed to create and
    manage pervasive services and applications

73
FY00 Progress
  • Present papers at NIST PC 2000, JavaOne, and
    International Pervasive Computing Workshop
  • Co-chaired NIST Pervasive Computing 2000
    conference
  • Developed Aroma Adapter-Projector with Div. 892
  • Measured performance of Aroma adapter
  • Developed conceptual model based on the OSI
    Reference Model
  • Exploring low-end pervasive computing
  • Embedded and 8,16-bit processors
  • Collaboration with US Naval Academy

74
FY01 Plans
  • Co-chair NIST Pervasive Computing 2001 conference
  • Explore use of ADL in pervasive computing model
  • Implement a virtual private smart space (VPSS)
    test bed
  • Extend Aroma projector to include digital cameras
    and VPSS technology
  • Continue performance measurements
  • Develop diagnostic and testing tools

75
FY00 Resources
  • 1.50 FTE
  • 300K (STRS, OA-DISA)
  • FY01 Resources
  • 3 FTE
  • Travel required to conferences to present work
    and collaborate with colleagues
  • Equipment 15K

76
Pervasive Computing
  • Customers
  • IT industry the usual suspects
  • Consumers e.g., GE, Sunbeam, Phillips, Northeast
    Utilities, Xerox
  • Universities USNA, UMD, UC Berkeley
  • Impact
  • Solicited industry needs for standards and
    measurements
  • PC 2000 conference launched awareness of NIST as
    key player
  • Papers presented at conference stimulated
    discussions on pc and standards
  • Ref imp demonstrated feasibility of PC
  • Enable widespread adoption and integration with
    software tools to create, manage, measure and
    test pervasive services and applications

77
XML Registry/Repository(Reg/Rep)
Project Team Lisa Carnahan Len Gallagher Mike
Kass
78
XML Registry/RepositoryReference Implementation
  • Built to OASIS specification
  • Being used by both OASIS and UN/CEFACT (ebXML)
  • Built using IMS Meta-data specification
  • Apply lessons learned to EC repositories

79
FY00 Progress
  • Revised the OASIS Registry and Repository
    Specification
  • Implement a reference implementation
  • Participate in ebXML Reg/Rep team
  • Ensure testability of specification
  • ensure compatibility between ebXML and OASIS
    specifications

80
FY01 Plans
  • Update reference implementation
  • Develop conformance test suite/data for OASIS
    specification
  • Coalesce the ebXML and OASIS specifications

81
FY00 Resources
  • 1.5 FTE
  • 325K (STRS)
  • FY01 Resources
  • 2 FTE
  • Travel required to attend standards meetings

82
XML Registry/Repository
  • Customers
  • EC/Enterprise Integrators Data Channel,
    Documentum, Oracle, Microsoft, Sun, IBM
  • Consumers Vertical market partnerships and
    supply-chains (e.g., automotive, insurance,
    travel, manufacturing)
  • Impact
  • Repositories allow efficient use of XML and
    eliminate duplication
  • Enable SMEs to participate in EC marketplaces and
    supply chains
  • Reduce cost of doing business through timeliness
    and increased automation

83
Internet Commerce for Manufacturing
(ICM)(ITL,EEEL, MEL, MEP)
Project team Tom Rhodes Liz Fong Kathy Harvill
84
FY00 Progress
  • Assumed project leadership (at request of EEEL)
  • Defined standards-based framework for ICM testbed
    (reference implementation)
  • Conducted workshops and presented papers/demos
  • NAMT, SIMA, and ATP reviews
  • Industry conferences IPC, NEMI, ITEM2000
  • Collaborate with NEMI, IPC, Georgia Tech, UMBC
  • Evaluating industry specifications (e.g.,
    RosettaNet, OAG, C-Net, eCo FW) for B2B
    interactions
  • Evaluating applicability of agent and web
    technologies

85
FY01 Plans
  • Continue industry collaborations
  • Demonstrate distributed ICM testbed operations
  • (with GaTech, NEMI, IPC partners)
  • Test and validate ICM information model in ICM
    testbed
  • Continue evaluation of applicable standards and
    technologies and their incorporation in ICM
    testbed

86
FY00 Resources
  • 1.5 FTE
  • 350K (STRS-EEEL, ATP)
  • FY01 Resources
  • 2 FTE (other FTE from EEEL, MEL, MEP)
  • Travel required to collaborate, participate in
    standards meetings, and present work

87
ICM
  • Customers
  • Consumers PC assembly supply chain participants
  • Consortia NEMI, IPC, Rosetta Net
  • Impact
  • Reduce the cost of circuit-board manufacturing by
    enabling the rapid exchange of build information
    and change notices for all supply-chain
    participants
  • Demonstrates the use of XML in a vertical industry

88
Strategy
  • Conformance Tests
  • Reference Implementations
  • Standards developed jointly
  • with industry
  • Diagnostic Tests
  • Research to reduce costs of testing
  • Reference Data

89
Forward-Looking Standards
Role-Based Access Control (RBAC) Virtual Reality
Modeling Language (VRML)/X3D XML and companion
standards (DOM, CSS, etc) Java ADL IMS DASE/SM
PTE OASIS Registry/Repository GIS
90
STANDARDS COMMITTEES AND RELATED ACTIVITIES
FEDERAL GOVERNMENT CIO Council Interoperability
Committee Standards Working Group
91
Real-Time Java
Project team Lisa Carnahan Alden Dima
92
Real-Time Java
  • Assistance asked by industry providers and
    government users
  • Collaborate with Sun, HP, IBM, New Monics, FTL
  • Hosted continuing workshops for real-time Java
    requirements
  • Addresses real-time issues in both Java embedded
    and Java server application environments
  • NIST coalesced industry to produce one set of
    requirements leading to one spec
  • Open consensus forum for requirements
  • BOTH camps (Sun and HP, Microsoft, etc.) have
    used NIST-led requirements document to produce
    specs

93
EE Times
  • Interestingly, both the splinter group and
    Sun are using as a serious basis for their
    efforts a requirements document for real-time
    (Java) extensions prepared by a separate group
    working under the auspices of the National
    Institute of Standards and Technology (NIST).
    That is raising hopes that both standards might
    ultimately come together.

94
Real-time Java Requirements Group
  • Requirements for Real-time Extensions to the
    Java Platform (final release -September, 1999)
  • NISTs Requirements Group effort dissolved after
    successful completion of its goal.

The J Consortium
The J Consortium uses real-time requirements
developed by the Requirements Group for Real-Time
Extensions for the Java Platform, an effort
sponsored by the National Institute of Standards
and Technologies (NIST).
Press Release -J Consortium Unveils First
Public Real-Time Specification For The Java
Platform
CUPERTINO, Calif., September 24, 1999
95
FY00 Progress
  • Real-Time Java Requirements document completed
  • Worked with both spec development groups to help
    coalesce industry
  • Member of Sun RT Experts Group - contributed to
    specification
  • Worked with J Consortium to help develop
    specification
  • Project sunset

96
FY00 Resources
  • 1.25 FTE
  • 250K (STRS)
  • FY01 Resources
  • 0 (Project sunset)

97
Real-Time Java
  • Customers
  • Sun, IBM, HP, NewMonics and users of real-time
    Java
  • Impact
  • Coalesced industry to produce one set of
    requirements
  • 2 spec efforts compatible
  • New book on Real-time Java mentions NIST
  • Presented to and endorsed by VCAT

98
ADL (Architecture Description Language)
Project Team Chris Dabrowski John Tebbutt
99
Architecture Description Language
  • Focus on high-level structure of the overall
    application (i.e., systems architecture)
  • Provide features for modeling software systems
    architecture
  • E.g., Structure, components, interfaces,
    interconnections
  • Support automated analysis and verification of
    design
  • Enable testing during design phase
  • Saves over testing at code phase

100
Architecture Description Language (ADL)
Software Architecture ..an abstract system
specification consisting primarily of functional
components described in terms of their behaviors
and interfaces and component-component
interconnections Hayes-Roth, 1994
ADL A language that provides features for
modeling a software systems conceptual
architecture Medvidovic 1996
101
NISTs Unique Role
Standards Role Neutral Expert Applied Research
Contribute to ADL body-of-knowledge
  • Standards Collaborations
  • Propose additional functionality to SAE (first
    industry standards effort)
  • Uniquely positioned to bring academic and
    industry communities together.
  • Apply developing AADL standard (based on
    Honeywells MetaH) to Unmanned Ground Vehicle
    build additional functionality into AADL standard
  • Explore ADL relationships with IEEE and Open
    Group efforts
  • Applied Research Contribute to ADL
    body-of-knowledge
  • Research application of formal approaches to
    ADLs
  • Enhance ADL features for verifying conformance of
    application systems architectures to reference
    architectures (e.g. RCS)
  • Publish successes and lessons-learned for
    industry awareness
  • Provide feedback to appropriate ADL tool
    developers

102
FY00 Progress
  • Participated in development of SAEs Avionics ADL
    (AADL) standard
  • Explore ADL standardization efforts in IEEE and
    Open Group
  • Established partnerships with DISA, US Army,
    ITL/Security Division
  • Identify domain areas for applying ADL to explore
    standardization and testing issues
  • E.g., government smartcard, unmanned ground
    vehicle, pervasive computing

103
FY01 Plans
  • Provide support for the formalization of AADL to
    ensure its testability
  • Develop prototype of US Army robotics
    architecture
  • collaboration with MEL
  • dependent on DISA funds
  • Develop prototype of Smartcard Architecture w.
    Div 893
  • Demonstrate and document the benefits and
    functionality of ADLs
  • Produce a comparative study of different ADLs to
    support further development and standardization
    of ADLs

104
FY00 Resources
  • 1 FTE
  • 200K (STRS)
  • FY01 Resources
  • 2 FTE
  • Equipment and Software 10K
  • Travel required to standards meetings and to
    participate in conferences and industry meetings

105
Architecture Description Language
  • Customers
  • Consumers Industries that rely on
    component-based systems (e.g., manufacturing,
    avionics, real-time)
  • ADL tool vendors
  • Impact
  • Moved ADL from University research to industry
    standard
  • Provide uniform descriptions of the structure and
    function of software.
  • Increases software quality by reducing errors
    through testing in design phase.
  • Increase the producibility, quality, and
    reusability of component-based software systems.

106
GIS
Project Leader Chris Dabrowski
107
FY00 Progress
  • Convener of ISO/TC211 Framework and Reference
    Model Working Group
  • Initiated and lead effort to use UML to formalize
    and integrate GIS family of standards
  • Shared lessons learned with OMG
  • Presented UML/GIS work at OMG meetings
  • Provided recommendations for the evolution of the
    UML language

108
FY01 Plans
  • Complete integration of UML in GIS standards
  • Sunset project

109
FY01 Resources
  • 0.25 FTE
  • Limited Travel anticipated to transition ISO GIS
    work and contribute experiences to OMG

110
Geographic Info Systems
  • Customers
  • Consumers Industries with GIS requirements
    (e.g., GPS industry)
  • Developers of GIS information systems
  • ISO TC 211, the OpenGIS consortium, Federal
    Geographic Data Committee (FGDC), Object
    Management Group (OMG)
  • Impact
  • enable industry growth and preclude barriers to
    business activity
  • provide a basis for exchange of information in
    heterogeneous environments distributed over space
    and time
  • Less ambiguities in the UML specification
  • Increased use of UML in standards (normative)

111
CIO Council Standards Working Group
Technology Transfer Act/OMB Circular A-119 direct
federal agencies to use voluntary consensus
standards in lieu of government unique
standards NIST directed to coordinate federal
technical standards activities with private
sector technical standards activities NIST/ITL
initiated effort to create Standards Working
Group within Federal CIO Council Standards
Working Group approved late last year by the CIO
Council Interoperability Committee Standards
Working Group to be focal point of discussion,
definition, and coordination of federal
requirements for voluntary IT standards, and will
assess the results of voluntary IT standards in
meeting requirements Standards Working Group
chaired by a member of this Division, and will be
supported by this Division in its efforts to
provide a unified voice to the voluntary IT
standards community for Federal IT standards
technical requirements
112
FY00 Progress
  • Prototype of web based support software for
    collection, evaluation, and status reporting on
    Federal Government technical requirements for
    voluntary standards developed and demonstrated.
  • Electronic Documents conference held featuring
    speakers from private industry, Federal
    Government, and voluntary standards
    organizations.
  • Meetings of the Standards Working Group held on
    an as needed basis.
  • Standards requirements forwarded to appropriate
    voluntary standards committees as the
    requirements are approved.

113
FY01 Plans
  • Continue development of web based support
    software for Standards Working Group.
  • Make software available for possible use by other
    Working Groups within the EIEIT structure.
  • Meetings of the Standards Working Group held on
    an as needed basis.
  • Standards requirements forwarded to appropriate
    voluntary standards committees as the
    requirements are approved.
  • Consider holding second Electronic Documents
    conference.

114
FY00 Resources
  • 1.5 FTE
  • 325K (STRS, OA-CIO Council)
  • FY01 Resources
  • 1.5 FTE Combination of time from Group Leader
    and others in Interoperability Group

115
CIO Council Standards Working Group
  • Customers
  • Other Federal Agencies
  • Impact
  • Federal agencies input requirements to
    standards groups and utilize voluntary consensus
    standards
  • Eliminates duplicative standards

116
Strategy
  • Conformance Tests
  • Reference Implementations
  • Standards developed jointly with industry
  • Diagnostic Tests
  • Research to reduce costs of testing
  • Reference Data

117
Software Diagnostic Tools
  • Program slicer tool - Project Leader Jim Lyle
  • Computes lines of code which could affect given
    variables
  • Reduces debugging time by directing effort to
    relevant parts of computation
  • Currently only for C programs
  • Extending slicer to Java and Fortran
  • Commercialized by private company as Y2K solution
  • VM View Toolkit - Project Leader Alden Dima
  • Diagnostic tool embedded at runtime into the JVM
  • Exposes internal VM conditions in the form of an
    execution trace
  • Downloaded by Sun, HP, Rockville Collins, Apogle,
    Insignia, Wind River, Ada Core, Johns Hopkins

118
Year 2000 Problems - Project Leader Gary Fisher
Developed source code scanner
Participate in Interagency Year 2000 Task Force
119
FY00 Progress
  • Y2K web site completed
  • Embedded system study completed
  • Report sent to Koskinen
  • Problems highlighted and impacts reported
  • Produced new emphasis on embedded systems
  • Project sunset

120
FY00 Resources
  • 0.25 FTE
  • 50K (STRS)
  • FY01 Resources
  • 0 (Project sunset)

121
Year 2000
  • Customers
  • All businesses that use computers
  • Consumers who use computers
  • Impact
  • Y2K rollover went without incident

122
Strategy
  • Conformance Tests
  • Reference Implementations
  • Standards developed jointly with industry
  • Diagnostic Tests
  • Research to reduce costs of testing
  • Reference Data

123
Automatic Generation of Spec Based Tests Using
Formal Methods
Project Team Paul Black Bill Majurski Jim
Lyle Micheal Koo
124
Industry Need
  • Software testing is enormous cost to industry
  • 30 - 50 of budget
  • Development may be 1000s of staff-years
  • Testing cost may be millions
  • Current methods mostly hand generated tests
  • Inefficient
  • Not rigorous enough
  • Software testing key to reducing security
    incidents
  • More than 75 of the incidentsare the direct
    result of bugs. and more real testing - not
    'beta testing' in the marketplace - will yield
    programs that are less vulnerable to attack.
  • (Business Week, February 28 "Locking Out the
    Hackers")

125
(No Transcript)
126
Current Test Development Methodology
Requirements
English Language
review inspect
Specification
English Language
Human Programmers
review inspect
Source Code
Software Tests
Programming Language
127
Advanced Test Development Methodology
Requirements
Mathematical Logic
formal proof
Specification
Mathematical Logic
Test Generator
informal proof
Source Code
Software Tests
Programming Language
128
Accomplishments
  • Method for automatically generating tests from
    formal specifications
  • Associated coverage metric
  • Publications and presentations
  • ICFEM (98), WSEST(99), HASE (99), DASC (99)
  • GMU, UMBC, College Park, Brisbane, NRL
  • Feasibility collaborations leading to
    commercialization
  • Argus, a secure computer operating system
  • Ford Motor Research Lab powertrain control
    systems
  • Northrop Grumman fighter radar computers

129
FY00 Progress
  • Feasibility tests on models from Ford and Argus
  • Initial formalization of BLAST (Basic Local
    Alignment Search Tool identifies similar DNA
    sequences) Algorithm completed in collaboration
    with CSTL
  • Hosted Workshop on Testing Software Using Formal
    Methods
  • Hosted Tutorial on Formal Methods
  • 3 papers published
  • 2 papers submitted and accepted
  • 4 papers submitted
  • 9 talks (6 invited)

130
FY01 Plans
  • Commercialize ATG Method with Ford and software
    company (tbd)
  • Continue feasibility studies with Argus
  • Complete BLAST formalization

131
FY00 Resources
  • 2.5 FTE
  • 500K (STRS-Competence)
  • FY01 Resources
  • 2.5 FTE
  • Equipment 5K
  • Travel required for validation collaborations and
    presentations

132
Automatic Generation of Tests
  • Customers
  • Software developers
  • Software end users, e.g., web surfers
  • Security administrators
  • Researchers in software testing methodologies
  • Impact
  • Reduce the cost of software testing (50 of
    development costs)
  • Improve software quality
  • Reduce security incidents (75 due to bugs)
  • Ford and Argus interested in commercialization
    of ATG methodology
  • Strongly endorsed by VCAT

133
Software Testing by Statistical
Methods(Competence Project)
  • Objective develop new methods for software
    testing based on stochastic processes and
    metrics, in order to
  • Improve the quality of software development
  • Develop reliability estimates that software
    correctly adheres to its specification
  • Joint project
  • Software Diagnostics and Conformance Testing
    Division (897)
  • Statistical Engineering division (898)
  • Collaboration with other NIST Labs (MEL, EEEL,
    CSTL)

Project Team Len Gallagher Lynne Rosenthal
134
(No Transcript)
135
Specification
Conformance Tests
136
FY00 Progress
  • Published paper on Coverage Model of Mutation
    Testing (with Paul Black)
  • Produced paper on applying statistical methods to
    component based testing
  • Produce paper on Test Input Needed to Improve
    Reliability
  • Analysis of effort to apply statistical method
    (partitioning) to new test development

137
FY01 Plans
  • Continue effort to apply statistical method
    (partitioning) to new test development
  • Apply statistical method to pervasive computing
    applications

138
FY00 Resources
  • 1 FTE
  • 200K (STRS-Competence, plus 100K given
    to ITL/SED)
  • FY01 Resources
  • 1 FTE
  • 200K competence funding (plus 100K given to
    ITL/SED)

139
Software Testing by Statistical Methods
  • Customers
  • Conformance Test Developers
  • Ourselves
  • Users of conformance tests
  • Impact
  • Improve our knowledge and skills in designing and
    developing quality software.
  • Quantitative measures associated with conformance
    testing.

140
Quantum Information
Project Team Paul Black Neva Carlson
141
Quantum Information
  • Goal to develop information representations and
    computing machines that take advantage of quantum
    effects
  • Joint effort of physicists and computer scientists

142
FY00 Progress
  • Invited to develop joint proposal for
    Nanotechnology Initiative with Physics
  • Invited by NIST Physicists to present Basic
    Concepts in Theory of Computation
  • Developed Quantum Information Demo
  • Developed Quantum Information Bibliography
  • Attended weekly Physics Quantum Information
    seminar

143
FY01 Plans
  • Develop products for Nanotechnology Quantum
    Information project
  • Organize Workshop on Computer Science in Quantum
    Information
  • Research questions in testing Quantum Algorithms

144
FY00 Resources
  • .5 FTE
  • 100K (STRS Physics Lab?)
  • FY01 Resources
  • 1.5 FTE (depends on Nanotechnology Initiative or
    from Physics)
  • Equipment 0
  • Travel anticipated

145
Quantum Information
  • Customers
  • Researchers in computational theory, and
    algorithm design and analysis
  • Developers of large computer models, e.g.,
    global weather
  • Cryptanalists
  • Impact
  • Computing machines of unprecedented
    capability and small size
  • Solutions to intractable computational
    problems, e.g., factoring, searching
  • ITL staff invited to present at Physics seminars
    and participate in Physics Lab efforts

146
Strategy
  • Conformance Tests
  • Reference Implementations
  • Standards developed jointly with industry
  • Diagnostic Tests
  • Research to reduce costs of testing
  • Reference Data

147
Error, Fault, and Failure (EFF)Data Collection
and Analysis
Project Team Dolores Wallace Michael Koo Tony
Cincotta Jason Rager
148
Error, Fault, and Failure (EFF)Data Collection
and Analysis
  • Repositories of reference data on software
    failures
  • Tools to create and manage repositories
  • Tools to analyze software failure data
  • Research in taxonomies to characterize software
    failures

149
FY00 Progress
  • Repository systems
  • Completed public repository with predefined
    queries
  • Completed internal repository with dynamic
    queries
  • Data 1000 data points added
  • Paper on failures in medical devices finalized
  • Papers report on NASA data submitted to NASA
  • Project was sunset

150
FY00 Resources
  • 1.5 FTE
  • 300K (STRS, ATP - 160K)
  • FY01 Resources
  • 0 (Project sunset)

151
Error, Fault, and Failures
  • Customers
  • Software Developers
  • Researchers in Software Development
    Methodologies
  • Impact
  • Identifies the stages in the software life cycle
    whose practices can be improved
  • Endorsed by the Software Division of the
    American Society for Quality
  • Project methodologies and tools used in
    graduate programs at Johns Hopkins and the
    University of Louisville
  • EFF Tool used by RST Corporation

152
Other Agency Projects
  • NIJ - Software and Tool Verification
  • Project Team Gary Fisher, Len Gebase, Jim Lyle,
    Jackie Schneider, Mark Zimmerman
  • VA - Health Care Information Systems
  • Project Leader Bill Majurski
  • GSA - EC Standards Roadmap
  • Project Leader Mary Brady
  • EPA - Metadata Descriptions and Registries
  • Project Leader Judy Newton

153
NIJNational Software Reference Library Beta
Testing
154
NIJNational Software Reference Library Beta
Testing
155
The NSRL BIG Picture
Requirements Funding and Feedback
SW Manufacturers
NIJ
MOU
Contact
Software
Law Enforcement Agencies
NIST
NSRL
Admin.
Research and Production
Use
NSRL DB
File Signatures
156
  • National Software Reference Library
  • GOAL Reference Data Set of file signatures
    (hash) of COTS file
  • Used during examination of digital evidence to
    identify pertinent files and eliminate others
  • BOTTOM LINE One computer may have 5,000-20,000
    files. NSRL data set can eliminate 40-95 of
    those files and save hundreds of staff-hours.
  • Beta Testing Project
  • GOAL Verification of computer forensic tools and
    their use in analyzing digital evidence
  • Used to ensure tool usage yields objective,
    repeatable, reproducible, results that hold up in
    court.


157
FY00 Progress
  • Developed NSRL data dictionary and database
    structure for hash codes
  • Established contacts with Microsoft, Adobe, et al
    to supply software
  • CBD announcement, Press release
  • Identified hash algorithms
  • Creating Reference Data Set (i.e, file
    signatures)
  • Developed draft test plan for beta testing
  • Develop procedures and tests for imaging tools

158
FY01 Plans
  • Establish the reference data set and distribution
    mechanism
  • (SQL database with web interface)
  • Develop test metrics for efficacy of hash codes
    used in the reference data set
  • Test the NSRL web site and make it available to
    approved users
  • Continue to develop tests and procedures for
    forensic tools
  • Issue test reports and test suites (publish on
    web site)

159
FY00 Resources
  • 3.5 FTE
  • 700K (OA-NIJ)
  • FY01 Resources
  • 3.5 FTE
  • Travel required for training courses and
    presentations

160
NIJ
  • Customers
  • Law Enforcement Agencies (e.g., FBI, Treasury,
    DCFL, state and local agencies)
  • Software companies (e.g., Adobe, Microsoft)
  • Impact
  • Improve the efficiency of forensic examination of
    suspect computer files/date (NSRL data set can
    eliminate 40-95 of suspect files and save
    hundreds of staff-hours).
  • Provide testing procedures for forensic software
    tools to ensure the analysis process and results
    holds up in court

161
Healthcare Information Systems
Project Team Bill Majurski Tony Cincotta Roger
Gima John Barkley
162
Objectives
  • Improve the quality of Healthcare Information
    Systems by
  • Designing distributed models and architectures
    for Healthcare Information Systems
  • Developing reference implementations
  • Technology transfer of RBAC
  • Participating in the development of standards,
    e.g., OMGs RAD, G-8 Healthcare Data Card Project
  • Support from Veterans Affairs and NIST ATP

163
NIST Role
  • Consulting Architect to VA (major healthcare
    provider 170 hospitals)
  • Assess new technologies
  • Developed design for RPC (Remote Procedure Call)
    Broker
  • Designed and implemented Enterprise Single
    Sign-On (ESSO)
  • Designing and implementing Inter-Organizational
    Role Based Access Control (IORBAC) authorization
    mechanism

164
FY00 Progress
  • Enterprise Single Sign-On (ESSO) beginning
    national trials
  • Inter-Organizational RBAC (IORBAC) V1 Reference
    Implementation completed (to be used for
    requirements development)
  • OMG Resource Access Decision Facility (RAD)
    proposed for IORBAC
  • NIST designs become permanent part of VA HIS
    Architecture
  • Now serve as permanent consultant to VA
    Architecture Design Group
  • Invited to speak at two VA HIS Conferences

165
FY01 Plans
  • Implement IORBAC Reference Implementation V2
    which includes RAD
  • Technology transfer of ESSO to VA developers
  • Review G8 Healthcare Smartcard Specification and
    provide advice on conformance testing

166
FY00 Resources
  • 2.5 FTE
  • 500K (OA-VA, ATP-200K)
  • FY01 Resources
  • Personnel 3 FTE
  • Equipment 20K
  • Travel required to participate in Architecture
    Design meetings and deployment of implementations

167
Health Care Information Systems
  • Customers
  • HIS developers and users
  • Standards bodies developing standards in HIS
  • Developers of authentication and access control
    mechanisms for protecting the privacy of patient
    information
  • Impact
  • Higher quality HIS and improved patient privacy
  • NIST designs permanent part of VA HIS
    Architecture
  • Remote Procedure Call (RPC) Broker in use in all
    VA Hospitals
  • ESSO selected for national trails
  • RAD adopted by OMG, implemented by 2AB, DASCOM

168
GSA - EC Standards Roadmap
169
FY00 Progress
  • Developed initial report on scope of effort,
    standards of interest, and design of web-based
    roadmap
  • Develop initial prototype of roadmap
  • Coordinate activities with appropriate GSA and
    CIO groups

170
FY01 Plans
  • Complete interactive standards roadmap
  • Pursue with MEP, deployment of roadmap for SMEs

171
FY00 Resources
  • 0.5 FTE
  • 75K (OA-GSA)
  • FY01 Resources
  • 1 FTE
  • GSA Funding required

172
EPA - Metadata Registries
173
FY00 Progress
  • Advance the standards effort in the ISO/NCITS
    arena by continuing to serve as a member of
    ISO/IEC SC 32 WG 2 and Chair of NCITS L8 SIG
  • Prepare new proposed standards and technical
    reports for Metadata Registries (MDR).
  • Coordinate and encourage implementations of
    metadata registries based on those standards by
    hosting the Metadata Registry Implementers
    Coalition (MDRIC).
  • Advance progress on the EDEN project by COTR
    activities.
  • Publicize activities through presentations and
    publications in national and international fora.

174
FY01 Plans
  • Continue the work shown above under FY00
    Progress.
  • Serve as Editor of Part 5 of ISO/IEC 11179 which
    is up for 5-year review.
  • Serve as Editor of the Technical Report
    Procedures for Achieving Metadata Registry
    Content Consistency, which is to be balloted for
    CD.

175
FY00 Resources
  • 1 FTE
  • 200K (OA-EPA)
  • FY01 Resources
  • 1 FTE
  • Travel required to participate in ISO meetings

176
Industry Testimonials
  • The OASIS-NIST XML Conformance Test Suite is
    critical for our industry.
  • Norbert Mikula, CTO, OASIS
  • N
About PowerShow.com