Automating Human Performance Measurement using XSLT in SimulationBased Exercises Unclassified

1
Automating Human Performance Measurement using
XSLT in Simulation-Based Exercises Unclassified
2
Problem Description

• Automated Human Performance Measurement (HPM) to
  reduce the need for instructor
• Add learning to simulation-based exercises
• Standards-Based HPM Approach
• Employ Mature Technologies in Approach
• Applicable Across Many Simulation Domains

3
The Approach

• HPM implemented with an Event-Condition-Action
  (ECA) rule engine
• XML Stylesheet Language Transformations as the
  language for ECA rules

4
The Performance Measurement Pipeline
5
Scenario

• Goal to describe and demonstrate the technique
  in a simulation-based exercise
• Terminal Learning Objective to drive a tank
• Learning Objective to traverse a ditch
• Derived from Mission Training Plan (MTP)
  Operate Vehicle under Adverse Conditions

6
Driver Training Example
7
Scenario Highlights

• Provide Initial Instruction
• Driver Performs Task Upon Cue
• Task Performance is Determined by
• Vehicle Position
• Vehicle Velocity
• Time Duration
• Provide Evaluative Feedback

8
Development Approach

• Training storyboards were created
• Based on MOS-specific task analysis
• Key HPM events were determined
• UML models were constructed to relate HPM events
  to DIS PDU information
• PDU filtering software was configured
• XSLT Event detection software was implemented

9
Events Captured

• Entity (Vehicle) crossed Phase Line
• Entity orientation out of alignment
• Entity velocity exceeds threshold
• Entity velocity below threshold
• Task timeframe exceeded
• Entity damaged

10
Scenario

• Simulation Events
• Entity Crosses Phase Line A
• Entity Crosses Phase Line B With Proper
  Orientation
• Entity Crosses Phase Line C

Phase Line C

• Actions
• Provide Initial Instruction
• In this exercise, you will learn
• Provide Evaluative Feedback
• Good Job. Now orient the vehicle
• Provide Evaluative Feedback
• Good Job. Now slowly move the vehicle forward
• Perform Final Evaluation
• If vehicle is not damaged, then GO else NOGO

Phase Line B
Phase Line A
Start Position
11
HPM System Components

• Man-In-The-Loop vehicle simulator
• IEEE 1278 DIS
• Data Recording Tool
• Wireshark Network Protocol Analyzer
• Simulation Event Detector
• XSLT Processor
• XSL produces XML-RPC method calls
• Event Processor
• XML-RPC Server
• JavaScript Interpreter

12
Try This at Home

• tshark -f "filter" -T pdml \
• xsltproc stylesheet \
• curl --header \
• "Content-Type text/xml" \
• --data _at_- http//www.foo.com/RPC2/doit.php

• Wireshark
• Network Protocol Analyzer
• http//www.wireshark.org/
• Xsltproc
• Applies XSLT Stylesheets to XML Documents
• http//xmlsoft.org/XSLT/
• Curl
• URL File Transfer Library
• http//curl.haxx.se/

13
Lessons Learned

• XSLT can be used as an ECA interpreter
• Tools are needed to assist instructional
  designers in relating PDU data to HPM events.
• XML Conversion and Processing is Time Consuming
• Performance is Adequate for Simple Scenarios
• Performance for large scale exercises is unknown
• User-defined functions can improve XSLT
  performance
• Additional filtering of PDU streams should be
  performed with other tools than XSL to improve
  performance