The Human Factors Practitioner as Change Agent: Patriot after the Fratricides of the Second Gulf War

1
The Human Factors Practitioner as Change
AgentPatriot after the Fratricides of the
Second Gulf War

• John K. Hawley, PhD
• U.S. Army Research Laboratory
• Human Research and Engineering Directorate
• Ft. Bliss Field Element

2
Patriot in the Second Gulf War

• During the combat operations phase of the second
  Gulf War (Operation Iraqi Freedom)?March-April
  2003, Army Patriot units were involved in two
  fratricide incidents involving coalition
  aircraft
• British Tornado misclassified as an
  anti-radiation missile (ARM)
• U.S. Navy F/A-18 misclassified as a tactical
  ballistic missile (TBM)

Two of 11 U.S. Patriot shots were fratricides
(18)
3
Patriot Battalion
PATRIOT FIRE UNIT X 5
HQ HQ BTRY
AMG
ICC
AMG
CRG


1 PATRIOT BATTALION
AMG
CRG
AMG
CRG
GMT
AMG
CRG
GMT
GMT
GMT
SRPT
BME
GMT
4
Patriot Fire Unit
Electric Power Plant III (EPP III) (1)
Radar Set (RS) (1)
Antenna Mast Group (AMG) (1)
Engagement Control Station (ECS) (1)
Small Repair Parts Trailer (SRPT) (1)
PAC-3 Launching Station (LS) (2)
Large Repair Parts Trailer (LRPT) (1)
Heavy Expanded Mobility Tactical Truck (HEMTT)
(7)
PAC-2 Launching Station (LS) (4)
Battery Maintenance Center (BMC) (1)
PAC-3 Missile
5
Why is Patriot important?

• Existing systemin the Armys inventory since the
  early 1980s
• Operational concept and operating environment has
  evolved significantly and become more complex
• Spectrum of air threats (ABTs, RWA, TBMs, ARMs,
  CMs, UAVs, etc.)
• Static, point defense concept to maneuver force
  protection
• As Patriot evolved it acquired features and
  characteristics more typical of systems the rest
  of the Army will employ than those in the current
  inventory
• Joint command and control (C2) (Army-Air
  Force-Navyand sometimes multinational)
• Network-centricand dependent
• Complexan early system of systems
  configuration many interacting parts
• Knowledge-intensive
• Technical and tactical complexity flows through
  directly to operators
• Relevant data point for future Army systems

6
Presentation Outline

• The Patriot Vigilance project 2004
• Follow-on work 2005-2008
• Phase 2 What does right look like?
• Command and control initiatives Putting
  operators back into the Patriot control loop
• Training initiatives
• Where do we stand now?
• Summary Patriot Vigilance evolution and
  transitions
• Observations and lessons
• And theres always dirty laundry

7
Howd we get started?The Patriot Vigilance
project

• MG Vanes question in the Spring of 2004
• How do you establish vigilance at the proper
  time? 23 59 of boredom followed by 1 of
  panic?
• MG Vane expressed a concern about
• Lack of vigilance on the part of Patriot
  operators
• Resulting lack of situation awareness (SA)
• Lack of cognizance of whats being presented to
  them
• Resulting absolute trust in automation
• MG Vane was interested in
• Understanding how we got to the fratricide
  incidents
• Exploring interventions that could be implemented
  now
• Tipping point for MG Vanes concern was Patriot
  fratricides during the second Gulf War
• These events are still reverberating and remain
  an issue

8
Assessment How We Got There
Reinforced by training practices Rote
drills vs. high-level judgment
Contributed to by lack of experience on the
part of personnel in key crew positions
Little explicit regard for the human performance
consequences of design decisions
Fratricide Incidents
Vigilance Problems and Inadequate SA lack of
comprehension of tactical situation
Automation Misuse Automation bias by operators
Undisciplined Automation
Reinforced by organizational culture
React quickly, engage early, and trust the
system w/o question
Unacknowledged system fallibilities
9
Recommendations

• Reexamine air defense battle command automation
  concepts to emphasize effective human supervisory
  control (HSC)
• Look into ways to mitigate situation awareness
  problems resulting from undisciplined automation
  of Patriot control functions
• Develop more effective command and control teams
• Reexamine the level of expertise required to
  employ a system such as Patriot on the modern
  battlefield.
• Explore ways to develop the necessary expertise

10
What is Effective HSC?

• Operators and not the automated system are the
  ultimate decision makers in air defense firing
  decisions
• Decisions to shoot or not to shoot made by crews
  having
• Technical potential for adequate SA
• Expertise to understand the significance of the
  information available to them (sensemaking
  ability)
• Uncritical acquiescence to the automated systems
  recommendations is not effective HSC

11
Independent Confirmation

• The Patriot system should migrate to more of a
  man-in-the-loop philosophy versus a fully
  automated philosophyproviding operator awareness
  and control of decision processes
• Patriot training and simulations should be
  upgraded to support this man-in-the-loop protocol
  including the ability to train in confusing and
  complex scenarios that contain unbriefed
  surprises

Source Report of the Defense Science Board Task
Force on Patriot System Performance, November
2004.
12
Phase 2 What does right look like?

• Question posed by Patriot Capability Manager
• Resulted in two focused technical reports
• Design (broadly defined) to support effective HSC
• Training to support effective HSC
• Reports oriented to an air defense user audience,
  but peer-reviewed by some the best in our
  business
• Served as technical foundation for later
  change-related initiatives
• Provided essential support for our on-going
  missionary work Building and maintaining a
  consensus for change

13
Command and Control InitiativesPutting
Operators Back into the Control Loop

• Human systems integration (HSI) evaluator during
  an operational test of a new Patriot software
  drop (Post-Deployment Build 6, or PDB-6)
• Fire Coordination Center (FCC) concept
  evaluationloosely based on Navy Combat
  Information Center operations
• Incremental SA essential for more reliable
  engagement decision making?
• Engagement timeline considerations?
• The new equipment training (NET) and unit
  train-up period for the operational test
• Green/Red pre-test training status evaluation
• FCC concept evaluation compromised by unvalidated
  operational procedures and inadequate training
• Test-based training conclusion
• The level of expertise required to employ
  Patriot properly with the software upgrade
  exceeds the current training standard
• A blueprint for training change
• Performance-oriented training strategies focused
  on deliberate practice

14
Training Initiatives

• Two training-related reports Theory Lite and
  How To
• The Reconfigurable Tactical Operations Simulator
  (RTOS) Operational Demonstration (April 2007)
• Modified instructional strategies
• Part-task, less-than-full fidelity simulator (the
  RTOS)
• Trial instructional modules focused on documented
  training shortcomings
• Application in a line Patriot unit (5-52 ADA) In
  the full glare of publicity
• Results
• RTOS judged to have utility to support unit
  collective training
• Training strategy focusing on deliberate practice
  and other improved instructional techniques was
  effective for the trial modules used
• Overall training package was well received by
  participants
• The OpDemo
• Maintained the momentum for training reform
  coming out of the PDB-6 operational test
• Provided a partial model for aspects of that
  reform

15
Where do we stand now?

• FCC promising, but not fully validated or
  completely accepted
• Operational Demonstration II in the offing
  (January 2008)
• Expand upon Demonstration I
• Additional trial modules
• Improved instructional procedures
• Upgraded RTOS
• Transfer of learning issues RTOS to full-task,
  high-fidelity simulation environment
• Support requirements for effective use by
  receiving units Its more than just the training
  equipment!
• Maintain momentum to change the way the Army
  trains in air defense units
• Mental models and knowledge structures versus
  simple habit transfer (rote drills)

16
Patriot Vigilance Evolution and Transitions
2004-2008
MG Vanes Request for HF support
HREDs fratricide incident assessment
Recommendations - Design - Training
TCMs request What does right look like?
OIF Patriot fratricides
ARL-TR-3468 The Human Side of Automation
DSB concurrence
Design report (ARL-TR-3742)
Training report (ARL-TR-3765)
AMD PCC and Top Gun participation
11th ADA Brigade OPD seminars
Conceptual training framework for
AMD (ARL-SR-149)
AEC/HREDs training conclusion
PDB-6 operational test
ITEA article Training and testing
RTOS OpDemo I In 5-52 ADA (ARL-SR-156)
Ft. Bliss Center for Enhanced Performance
Patriot PDB-6.5 operational test New human
performance testing methods
AAMDIS performance verification test at MCAS Yuma
OPD seminars for AMD students at CGSC and SAMS
RTOS OpDemo II in 11th ADA Brigade
RTOS or RTOS-like part-task trainer deployed to
operational Patriot units
Prototype air battle operations training program
for Patriot units
Legend 2003/2004 2005 2006 2007 2008
17
Observations and Lessons

• We became a catalyst for changeeven though
  thats not what we started out to do
• Could not stop with the fratricide assessment and
  formulation of potential solutions
• Change is a different game from that most of us
  are trained or our organizations are equipped to
  play
• It is one thing to talk about bulls. It is
  another thing entirely to be in the bull ring
• Change in large organizations does not come
  easily
• We had to be opportunistic and identify and work
  through champions
• The Army has few formal mechanisms for effecting
  non-equipment-related change
• Many change initiatives become personality-driven
• The wheel is always in turnseeming gains often
  are fleeting and transitory
• Change efforts have a finite life

18
The Change Agents Dilemma
Politics and Spin
Field Reality
Technical Reality
19
And Theres Always Dirty Laundry

• Patriot Capability Managers musing
• Weve been doing MANPRINT on Patriot for more
  than 20 years. How did we miss all of the things
  you wrote about and were now seeing? Certainly
  all of these things are not new.
• These things are
• Eroding potential for effective human supervisory
  control
• Need for increased levels of operator expertise
• Armys post-incident inquiry, the Defense Science
  Board review, and HREDs Patriot Vigilance
  assessment all picked right up on these issues
• We looked at Patriot MANPRINT-related documents
  going back more than 20 years and attempted to
  answer the Patriot CMs question
• but thats another story, for another
  presentation, and for another time