The Man-Machine Integration Design - PowerPoint PPT Presentation


PPT – The Man-Machine Integration Design PowerPoint presentation | free to download - id: 648bc9-N2VjO


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation

The Man-Machine Integration Design


The Man-Machine Integration Design & Analysis System (MIDAS): Recent Improvements Sandra G. Hart Brian F. Gore Peter A. Jarvis NASA Ames Research Center – PowerPoint PPT presentation

Number of Views:46
Avg rating:3.0/5.0
Slides: 44
Provided by: Sandy106


Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: The Man-Machine Integration Design

The Man-Machine Integration Design Analysis
System (MIDAS) Recent Improvements
  • Sandra G. Hart
  • Brian F. Gore
  • Peter A. Jarvis
  • NASA Ames Research Center
  • Moffett Field, CA 94035
  • 604 6072
  • 10/19/04

  • Human Performance Modeling
  • MIDAS Phase 1 Initial design
  • Early applications
  • MIDAS Phase 2 Move from Lisp to C
  • Recent applications
  • MIDAS Phase 3 PC Port/Integrate Apex

Human Performance Models Components
Psychological Models
Sensory Models
Task Network
Anthropometric Models
Performance WL

Biodynamic Models
Performance Time
Model Architecture, Library, Tools
Team/Org Models
Performance SA
Vehicle Models
Performance Errors
Equipment Models
Environment Models
FoV/Reach Envelope
Procedural Models
Human Performance Models Architectures
Anthropometric/Biodynamic Physical
characteristics of human body static dynamic
population characteristics limitations RAMSIS,
Psychological theories, mathematical models,
descriptive functions
  • Task network Top-down, based on sequences of
    human/system tasks (derived from task analysis

MicroSaint WinCrew Crewcut IPME IMPRINT
Cognitive Bottom-up, combine theory-based
models of memory, decision making, perception,
attention, movement, etc
Vision Computational representation of the way
the human visual system processes an image to
predict performance given image characteristics
ORACLE NASA Standard Visual Observer NASA Text Visibility Optimetrics Visual Perf Model Georgia Tech Vis Model
Human Performance Models can
  • Generate hardware, software, training
    requirements for tasks that will involve human
  • Depict operators performing tasks in prototype
    workspaces and/or in remote or risky
  • Perform tradeoff analyses among alternative
    designs and candidate procedures, saving time and
  • Identify general human/system vulnerabilities to
    estimate overall system performance and
  • Provide dynamic, animated examples for training
    and developers
  • Generate realistic schedules and procedures

Phase 1
A comprehensive suite of computational tools - -
3D rapid prototyping, models of perception,
cognition, response, real- and fast-time
simulation, performance analysis, visualization -
- for designing and analyzing human/machine
systems was developed primarily in Lisp on a
fleet of SGIs
Data Analysis
Run Time Visualization
  • Pioneered the development of an engineering
    design environment with integrated tools for
    rapid prototyping, visualization, simulation and
  • Advanced the capabilities and use of
    computational representations of human
    performance in design including a state of the
    art anthropometric model (Jack)
  • Flexible enough to support a range of potential
    users and target applications
  • But.
  • Component models written in Lisp, Fortran, C, C
  • Required a suite of SGI machines
  • Modeled a single operator
  • Time based rather than event based scheduler
    established optimal inter-leaving of task
  • No emergent behaviors

Richmond, CA Police 911 Dispatch
  • Goal Upgrade the facilities and procedures used
    in the 911 dispatch facility
  • Accomplished
  • Modeled control console and dispatch activities
    in MIDAS
  • Evaluated prototype graphical decision aid

US Army Air Warrior
  • Goal Establish baseline performance measures for
    crews flying Longbow Apache with and without MOPP
  • Accomplished
  • Modeled copilot/gunner with Jack (95th male ltgt
    5th female)
  • Rendered cockpit using CAD files from
  • Simulated performance of more than 400 activities
  • Measured reach, FoV, workload, timelines

Short Haul/Civil Tiltrotor
  • Goal Evaluate crew performance/workload issues
    for steep (9º), noise abatement approaches into a
  • Accomplished
  • Constructed MIDAS models of normal and aborted
  • Contrasted impact of manual vs automated
    nacelle control modes

NASA Shuttle Upgrade
  • Goal Support development of an advanced orbiter
    cockpit with an improved display/control design
  • Accomplished
  • Created virtual rendition of current shuttle
  • Conducted simulation of first 8 min of nominal
  • Provided quantitative measures of workload/SA,

Phase 2
  • Decreased model development from months to weeks
  • Increased run-time efficiency from 50x RT to near
  • Multiple operators
  • Modeled external vision, audition, situation
  • Conditional behaviors emerging from interaction
    of top-down goals and environmentally driven
  • Option of non-proprietary head hands model
  • But
  • The interface still user un-friendly
  • SGI platform
  • Cognitive models no longer state of the art
  • Performance moderating functions not integrated

Overview of Architecture
User Interface
  • The interactive graphical user interface is used
    to create models, specify and run simulations,
    and view data. It is organized into a
    hierarchical series of screens or editors that
    are navigated with tabs
  • Different views of the simulation are offered
    Structure, Geometry, Outline, Animation,
    Real-time/post-hoc data

Vehicle Models
  • A modeled vehicle represents the combination of a
    guidance/ dynamics model and a visual
  • The guidance/dynamics model moves the vehicle
    along a prescribed route. MIDAS provides two
  • NoE helicopter model
  • Simple point mass model (used to model arbitrary
    vehicles in a generic way)
  • The visual representation is CAD geometry chosen
    from the MIDAS library or developed by the

Environment Model
  • Tools are provided to model the environment
    outside the crew station (e.g., terrain, weather,
  • Terrain is modeled as a single object
  • Features are simple objects that have no inherent
    behavior and do not move
  • One weather condition may be applied to the
    environment by specifying lighting/visibility
    (these are used by the visual perception model)

Crew Station/Equipment Models
  • The crew station is a collection of equipment
    with which operators interact
  • Crew station models may be given a graphical
    representation for animation
  • Multiple crew stations per vehicle and multiple
    operators per crew station possible

Anthropometric Models
  • Anthropometric models provide an animated, 3D
    graphical representation of one or more modeled
    human operators for visualization
  • Jack (developed at U Penn/distributed by UGS)
    full-body figure realistic movements
  • Head and Hands model government-developed
    representation adequate for many purposes for
    users without a Jack license

Vision Models
  • Visual attention modeled as single cone,
    varying from 3-15º based on task type.
  • External vision
  • Peripheral 160 degrees
  • Foveal 2.5 degrees
  • Perceivability f(visibility, size, distance,
    local contrast ratio)
  • Perception level f(dwell time, perceivability)
  • Levels of Perception
  • Detection
  • Recognition
  • Identification
  • Internal vision
  • Symbolic (check read)
  • Digital (exact value)
  • Text (character string)

Auditory Model
  • Only within crew station
  • External sounds are represented only if channeled
    through equipment
  • Two Stages of Processing
  • Detection
  • Comprehension
  • Content
  • Verbal strings
  • Signals
  • All or none processing (Interruptions disrupt
    entire message)

Symbolic Operator Models
  • Significant advance over earlier version, which
    required specification of all activities at
    primitive task level
  • High-level scripting language, Operator Procedure
    Language (OPL) serves as front-end to a reactive
    planning system (RAPS)
  • User-supplied procedures are instructions for
    accomplishing tasks
  • Manages knowledge and beliefs, integrates human
    actions with scenario events


Memory Model
  • Simpler model than in MIDAS 1.0
  • Distinction between long-term/short-term memory
    was lost
  • Memories are represented as a database of
    assertions or beliefs that are symbolic
    expressions describing the property of objects
  • Memory can be examined by powerful tools in a
    querying language built into OPL

Attention Model
  • Based on Wickens Multiple Resource Model.
  • Acts as a mediator that maintains an account of
    attention resources in six different channels
  • Necessary attention resources must be available
    before primitive tasks are initiated
  • Task onset may be delayed if insufficient

Output Behavior Models
  • If required resources are available an activity
    that corresponds to a primitive procedure is
  • Physical actions and their effects on equipment
    or environmental objects are modeled, regulated
    by a motor control process
  • 60 primitive tasks are available in a Procedure
    Library with pre-defined load values easy to
    add more

Visual Auditory Cognitive Spatial Cognitive Verbal Manual Vocal
Estimate Time 0 0 0 2.0 0 0
Visual monitor 5.4 0 6.8 0 0 0
Type (1 hand) 5.9 0 0 5.3 7.0 0
Say message 0 0 0 5.3 1.0 4.5
Move object 5.0 0 1.0 0 2.6 0
Simulation System
  • Engine/executive (uses discrete-event, fixed-time
    increment approach for advancing the simulation)
  • Data collection mechanisms for generating runtime
    data that is graphically displayed which the
    simulation runs and is saved for post-run
  • Event generation mechanism provides a way for
    timed events to occur on schedule or with
    stochastic variance
  • Provisioning system allows users to change the
    simulation and re- run without re-loading/ re-star

Workload Situation Awareness
  • Workload calculations based on McCracken
    Aldrich (1988)
  • Load levels for Visual, Auditory, Cognitive, and
    Psychomotor dimensions are defined for task
    primitives on a scale of 1-7
  • Momentary load based on aggregation
  • SA calculations based on
  • Ratio of operators relevant knowledge/required
  • Distinguishes actual SA from perceived SA
  • Situational elements can be objects in the crew
    station or the environment that define a
    situation or are in the operators memory and
    are operationally relevant.
  • WL and SA values offer a powerful way to
    simulate realistic errors

Validation Search Rescue Mission
Comparison of Models to Simulator Data
ACT-R/PM U of Illinois Rice University
D-OMAR BBN Technologies
Air MIDAS San Jose State University
IMPRINT/ACT-R MAAD Carnegie Mellon
A - SA U of Illinois
Goodness-of-fit of individual model outputs to
empirical data
Preliminary timeline, SA, attention, wkld,
analysis,task execution times error
Nominal Approach Landing Simulation
  • PF scanning for TFX, runway
  • PNF monitoring PFD, Nav
  • PF/PNF monitoring radio
  • Flaps 30º/set confirm
  • PF requests before landing checklist
  • PNF checks/responds hear down
  • PF confirms visually/verbally
  • PNF checks/responds flaps 30
  • PF confirms visually/verbally
  • PNF checks/responds speed brakes set
  • PF confirms visually/verbally
  • PNF declares checklist complete
  • PF sets/declares DA at 650
  • PNF visually confirms DA set
  • Note passing FAF
  • Confirms final descent initiated

Traffic Call During Approach
  • Final approach checklist is complete
  • ATC call with traffic advisory
  • Both pilots scan for traffic I dont see it
  • Neither pilot notices as the decision altitude is
  • After the fact, the First Officer notices
    Were past FAF and not descending
  • Crew must decide whether to continue with the
    approach or abort

Life Sciences Glove Box
Virtual Glovebox
  • Challenges
  • Astronauts must follow detailed instructions
    within strict time constraints failure to do so
    introduces risk of science mission failure
  • Role of Computational Modeling
  • Predict interactive influences of microgravity
    (posture, bracing, precise movements, placing,
    moving, stowing) to develop/evaluate procedures
  • Watching an animated dry run enables efficient
    communication among scientists, implementers,
    astronauts more effective training

Life Sciences Glovebox Payload Development Unit
received at Ames from the National Aerospace
Development Agency of Japan (NASDA)
Onboard KC-135
MIDAS rendering
Life Sciences Glove Box Simulation
  • Goal Predict astronauts performance of complex
    experiments designed to answer questions about
    living organisms adaptation to the space
  • Objectives Evaluate feasibility of following
    proposed procedures within time/performance
    constraints ID factors that will increase risk
    of mission failure e.g., waiting too long to
    photograph slides interruptions task requires
    (unavailable) resource(s)
  • The Task
  • Turn on experimental equipment (monitor,
    microscope, camera)
  • Measure cell density/viability for each of 6
  • Invert sample vial
  • Place aliquot of sample on slide
  • Place drop of viability stain in sample
  • Record time on sample record
  • Place cover slip on slide
  • Observe on microscope
  • Take photographs within specific time window
  • Dispose of trash, return vials to containers,
    turn equipment off

Cell Staining/Photographing Experiment
Phase 3
MIDAS v3.0 Features
  • Runs on high-end PC
  • Simple model of microgravity influence on
  • Physics model of microgravity impact on objects
  • Simple within-task fatigue model implemented
  • Fatigue state model (U Penn/Astronaut Scheduling
    Assistant) selected
  • Notion of task duration - - how long a task
    should take as well as how long it did take
  • Grasping, moving, manipulating objects in
  • Apex will become the heart of the Task Manager
    and enable multi-tasking, task prioritization,
    shedding, deferral, resumption
  • Task primitive definitions include failure modes
    (time/quality) that enable the occurrence of
    emergent behaviors
  • Mission success/performance measures computed
    vulnerability to error, slipped schedules
    performance degradation

MIDAS v3.0 Structure
Task Manager Plans Monitors Remembers Senses Actua
Task Network
List of Tasks/Procedures
Mission Environment
Physical Simulation Perceives Attends Moves/Acts C
Workstation Geometry
Fit/Reach/Vis envelope
Dynamic models
Dynamic Animation
Task executions
Library Primitive tasks Human model
Task state
Operator state
Mission success
Cognitive simulation Behavior modifiers Situation
Awareness Error, Workload Timeliness
Operator Characteristics
Performance measures
Typical Outputs
PC Version Early Simulation
  • MIDAS 3.0 now operates on a PC platform and will
    soon incorporate significantly enhanced cognitive
    model (Apex)
  • MIDAS 3.0 gives users the ability to model the
    functional and physical aspects of a variety of
    operators, systems, and environments.
  • It brings these models together in an
    interactive, event-filled simulation for
    quantitative and visual analysis
  • The interplay between top-down and bottom-up
    processes and a suite of performance modifying
    functions enables the emergence of un-forseen,
    un-scripted behaviors
  • The government has done what it set out to do - -
    spur development of human performance modeling
    tools integrated into a design environment
  • Our goal is to continue to add functionality with
    each new application