Knowledge Base Editor Life Cycle Objective Architectural Review Board

1
Knowledge Base Editor Life Cycle
Objective Architectural Review Board

• Team 21
• Raffi Tikidjian
• Donna Djordjevich
• Sean Cahill
• Mustansir Golawala
• David Wong
• Nagesh Arkalgud

2
Operational Concept Description Donna Djordjevich

• Shared Vision
• System Capability
• Expected Benefits
• Benefits Chain
• Success Critical Stakeholders
• System Boundary and Environment
• Desired Capabilities and Goals
• Operational Concepts
• Organizational and Operational Implications

3
System Capability

• Jet Propulsion Laboratorys Integrated Systems
  Health Management department needs a graphical
  user interface component to manage knowledge
  bases for one of their internal products.
• CURRENT Files are written in a LISP-like format
  and are dependant on an expert tool smith
• PROPOSED Standardize the file format in XML
• BENEFIT Improved management mechanisms for
  viewing and modifying the files
• There are no competitors to this tool that
  fulfill all
• the principle requirements.

4
Expected Benefits

• The proposed knowledge base management system
  will enable researchers within Jet Propulsion
  Laboratorys Integrated Systems Health Management
  department to
• Improve the development and maintenance of their
  knowledge bases.
• Reduce the number of errors in their knowledge
  bases.
• Eliminate the need for an expert tool-smith
  bottleneck to improve workflow.

5
Benefits Chain
6
Success Critical Stakeholders
7
System Boundary
8
Desired Capabilities and Goals
9
Operational Concepts
Current Process
Proposed Process
10
Organizational and Operational Implications

• Within the current procedure
• Domain experts are dependant on tool smiths
• Cumbersome knowledge base representation language
• Development infrastructure is not designed for
  knowledge base development
• Process is difficult and ineffective for both the
  tool smith and domain expert to manage
• Within the proposed procedure
• All users of this system will be empowered to
  create their own knowledge base systems
• Agility of development
• Input is through a simple, configurable language
• Output is stored in an extensible and easily
  transformable format
• Custom toolset specifically designed to maximize
  the productivity of knowledge base development
  and maintenance
• Streamlined process through removing the
  bottleneck between the tool smiths and domain
  experts.

11
Prototype Mustansir Golawala

• Prototype Rationale
• Types of Risks
• Bounding the solution
• Building a Shared Vision
• Revised Vision
• Storyboard
• Risk Mitigation Future Prototyping
• What wont be prototyped

12
Prototype Rationale

• A hybrid of two prototyping methodologies
• Value-Based Prototyping
• Didnt want to prototype Everything (value
  neutral)
• Buy information in a risk-based manner
• Importance of the requirement being prototyped
• Likelihood of an unfavorable scenario associated
  with the requirement
• What is the expected value from prototyping? (Net
  value-gained)
• Staged Prototyping
• Inception Phase
• Rapid non-functional prototypes
• Bounding the scope of the solution
• Elaboration Phase
• Detailed functional prototypes
• Risk mitigation

13
Types of Risks

• Technology
• Parser
• Effort (Schedule)
• Parser
• General Functionality
• Code completion
• XML output and specifications
• Key feature relating to system interface
  requirements
• Very large size of loss, low probability of loss
• Levels of service
• Rule optimizations
• Syntax error detection
• Shared vision
• UI
• Clients/Users and developers all on the same page
• Build domain knowledge
• Tacit understanding of deliverable

14
Bounding the Solution Building a Shared Vision
Initial prototype
15
Bounding the Solution Revised Vision

• Knowledge Base
• Export
• Import

Outline View
Auto-Completion
Knowledge Base Management
16
Bounding the Solution Storyboard
XML SCHEMA
Export XML
Import XML
Source XML File
Target XML File
Newly Created KB
Parent .NET Application
17
Risk Mitigation Future Prototyping

• Sample knowledge base annotation language
• Tightly coupled to many of the requirements of
  the project. The size of loss is potentially
  high.
• Auto completion
• Syntax checking
• Parser functionality
• Part of the end to end solution completeness
• Development Strategy Part system boundary
  definition and part risk mitigation
• Already have a draft (boundary)
• Detailed prototype in Elaboration phase (risk
  mitigation)
• Specified through Context Free Grammar
• Use as test case during construction

18
Risk Mitigation Future Prototyping

• Parser
• Dependant on sample knowledge base language
  prototype
• Two risk types are associated to it
• Technology
• Schedule
• A functional prototype is required to lower risks
• GUI Interface
• Complex run-time behavior not effectively
  storyboarded (IKIWISI)
• Two major features
• Auto-completion
• Outline View
• Semi-Functional prototype using Clients required
  technology (.NET) and development tools (Visual
  Studio.NET)

19
Risk Mitigation Future Prototyping

• Schedule permitting (currently optional)
• Mid-risk, Low-priority features
• Syntax checking
• Context sensitive auto-completion
• Sample XML schema
• Sample Context Free Grammar

20
What wont be prototyped

• Eliminated through the use of Value-Based
  Prototyping
• Schedule constraint
• Value of prototyping was not enough to justify
  the effort
• XML output and specifications
• Considered low risk and medium size of loss
• Significant amount of domain knowledge
• .NET provided libraries
• Levels of Service
• These are not deemed to be high risk
• Schedule constraint
• Rule Optimizations Input Error Detection
• Lower priority requirements
• Will only be implemented if schedule permits
• Input error detection is a function of parsing

21
Requirements Sean Cahill

• Project Requirements
• Capability Requirements
• System Interface Requirements
• Level of Service Requirements
• Evolutionary Requirements

22
Project Requirements

• General
• Zero Budget
• No Budget has been set aside for the development
  of this tool
• 12 hour work-weeks
• The schedule should be planned according to 12
  hour work weeks for the development team.
• Risk-Driven Iterative Development Cycle
• The Development Process must be a Risk Driven
  Iterative Spiral model with Schedule as an
  Independent Variable (SAIV).
• Use of LeanMBASE Guidelines
• Tool Language
• .NET User Control
• The system must be developed as a .NET user
  control that can be embedded or plugged into
  .NET applications at development time.
• C Development Language
• The system must be developed in the C
  development language.

23
Capability Requirements

• Knowledge Base Management
• The system must allow the user to Add, Edit or
  Delete any object within the Knowledge Base
  (Attribute, Rule or Function).
• Parsing
• The system must contain a custom-developed parser
  for the Knowledge Base text entered into the
  tool.
• Input Error Detection
• The system shall keep track of the context of the
  user input, and detect syntax errors, and inform
  the user when such syntax errors occur.
• Auto-Completion
• The system shall keep track of the context of the
  user input, and provide helpful prompts to the
  user to complete Object names appropriate for
  that context (eg. A List of Attributes).

24
System Interface Requirements

• Knowledge Base Outline View
• The system shall be able to generate and display
  a knowledge base outline view based upon the
  rules and Attributes entered into the Knowledge
  Base.
• Knowledge Base Importable/Exportable to XML
• The system must be able to import a Knowledge
  base from a user specified XML file, provided it
  meets the format specified in the XML schema.
• The system must be able to export the active
  Knowledge Base to a user specified XML file.
• Deployed Integrated Development Environment
• This system should interface with an internally
  developed IDE at JPL.
• Rule Inference Engine
• The XML output from this system will be used by a
  rule inference engine developed by JPL.

25
Level of Service Requirements

• Usability
• Lag Free
• Scalability
• Knowledge base size
• Extensibility
• Embedding in the IDE
• Reliability
• Should not corrupt knowledge base via the
  import/export process
• Correct and complete XML files

26
Evolutionary Requirements

• XML Schema
• Along with having a default XML schema, the
  system should be able to allow the maintainer to
  modify the schema.
• The XML import/export module must conform to the
  XML schema specified by the maintainer.
• Context Free Grammar (CFG)
• The system should use a modifiable CFG that is
  provided by the maintainer to parse and perform
  syntax checking.
• Consider future capability enhancements
• The systems internal components, shall be
  developed in a modularized Object oriented
  manner, so that individual components can be
  easily upgraded to encompass enhanced
  functionality.

27
Architecture David Wong

• Risk-based Design
• Top-level Logical Architecture
• COTS Assessment

28
Risk-based Design

• Simple system
• Stand-alone application
• GUI based
• Agility vs. Discipline
• Plan-driven risks
• P-Speed Need for rapid results due to SAIV
• P-Emerge Emergent requirements due to
  unprecedentedness
• Agile risks
• A-YAGNI Overuse of simple design
• A-Churn Personnel turnover for 577B
• Our approach
• Use agile method
• Encapsulate parts where anticipate more
  architecture
• Apply design pattern or abstraction on them

29
Top-level Logical Architecture
30
Top-level Logical Architecture Simple Design

• GUI
• Presentation Layer
• Maps to OC-1 Knowledge Base Management
• Maps to OC-2 Knowledge Base Outline View
• Maps to OC-7 Auto-Completion of Code
• Parser
• Business Logic Layer
• Maps to OC-5 Knowledge Base Grammar
  Specification
• Maps to OC-6 Knowledge Base Syntax Checking
• Maps to OC-7 Auto-Completion of Code

31
Top-level Logical Architecture Abstraction

• Broker Persister
• Data Access Layer
• Maps to OC-1 Knowledge Base Management
• Maps to OC-3 Knowledge Base Importing and
  Exporting as XML
• Maps to OC-4 Knowledge Base XML Schema
  Specification

32
Top-level Logical Architecture Factory Pattern

• Presentation Layout
• Presentation Layer
• Maps to OC-2 Knowledge Base Outline View

33
COTS Assessment

• Areas to Apply COTS
• Business Logic Layer (i.e. parser)
• Presentation Layer (i.e. fancy GUI components)
• COTS Choices
• Bison Parser Generator
• Microsoft .NET Framework (e.g. .NET TreeView)
• COTS Assessment
• Bison generates parser in C
• Developers and maintainers knowledge on Bison /
  YACC

34
Life Cycle Plan Nagesh Arkalgud

• Life Cycle Strategy
• Elaboration Phase Details
• Key Stakeholder Responsibilities

35
Life Cycle Strategy

• Custom development project
• Schedule As Independent Variable (SAIV)
• 2 12 week semester courses (577a and 577b)
• LeanMBASE
• Iterative, Risk driven, Win-Win Spiral process
• 3 anchor points
• Life Cycle Objectives (LCO)
• Life Cycle Architecture (LCA)
• Initial Operational Capability (IOC)
• Project Life Cycle Status
• LCO Package Coming up
• LCO ARB
• LCO Draft Completed
• LCO Core Completed
• Early OCD Completed

36
Elaboration Phase MS Project Plan
37
Elaboration Phase Key Focus Areas

• Requirements satisfiable by the architecture
• Precise descriptions of each architectural
  component, i.e. interface(s), behavior, data,
  qualities
• COTS choices finalized
• Feasibility of architecture documented in the FRD
• Deliverables, budgets and schedules achievable
  based on the architecture
• Detailed development plans for Construction Phase
  outlined
• Assurance of consistency and traceability within
  document artifacts

38
Key Stakeholder Responsibilities
39
Feasibility Rationale Raffi Tikidjian

• Business Case Analysis
• Cost Analysis
• Benefit Analysis
• Return on Investment (ROI)
• Risks
• Risk Management Plan
• Top Risk Items
• COTS Analysis Results

40
Business Case Analysis - Cost

• Cost Analysis
• Development Costs
• CSCI 577A Team 21 6 members 24 weeks 12
  hrs/week 1728 hrs BUT Zero Team Cost
• Clients 2 clients 24 weeks 1hr/week 48
  hrs
• Transition Costs
• No dedicated resources (equipment, facilities,
  COTS licenses)
• Client/Maintainer part of development team no
  transition required
• Possible one-time purchase cost of COTS GUI
  components
• Operational Costs
• MSDN Visual Studio Already owned by department
• No dedicated IT Administrators or resources
• Maintenance Costs Zero Maintenance Budget
• Only cost is time invested for development
• Avg. Employee Burdened Cost 300K/year
• 48 hrs 1.25 4K

41
Business Case Analysis - Benefit

• Difficult to predict knowledge base tool usage
• Assume 1 unit KB contained 10 rules
• Similar to estimating (Size of Lines of Code)
  SLOC
• Current Process
• Domain Expert 0.5 hr/unit
• Tool-smith 1 hr/unit
• Improved Process
• Domain Expert 1 hr/unit
• Tool-smith 0 hr/unit
• Gain
• Can measure improvement linearly
• 1 unit saves 0.5 hr or 0.075K

42
Business Case Analysis Benefits Continued

• Many benefits are not measurable easily by ROI
  (non-monetary)
• HIGHEST PRIORITIES TO BUSINESS CASE
• Rule Inference Engine Marketability
• Improved relay time of knowledge base development
  process
• Ability to create knowledge bases graphically vs.
  learning proprietary development language
• Knowledge base editor can be applied to other
  application domains
• Improved quality and correctness of knowledge base

43
Business Case Analysis - ROI

• ROI first year / development is -1
• Development Cost and No Benefit
• ROI after first year 1 (Diagram below shows
  benefit NOT ROI)
• No maintenance costs

500 units estimated 1st yr usage
40 units break-even
44
Risk Management

• Weekly Risk Assessment
• Involves Developers, IVV and Clients
• Identifies Risk Mitigation Plan
• Determines Risk Exposure
• Risk-driven planning
• DART Tool

45
Top Risk Items

• Dynamic Language Specification
• Exceeding Planned Project Effort
• Unknown Knowledge Base Grammar
• Emerging Requirements
• Lack of AI Domain Knowledge
• Evolving LeanMBASE Guidelines
• Possible IKIWISI of UI

Risk Exposure Matrix
1
2
3
7
4
5
6
Size of Loss
Probability of Loss
46
COTS Analysis Results

• COTS choices primarily restricted due to client
  requirements
• Microsoft .NET Framework
• Client does not want any COTS products used
• Exception COTS products can be used ONLY for GUI
  components
• GUI COTS products to be identified would be small
  components (i.e. fancy toolbar package)
• Will be identified with further prototyping
  during the elaboration phase

47
Conclusion

• Thanks for attending!