SIP PETII User Requirements, Accomplishments and Future Vision

1
SIPPET-II User Requirements, Accomplishments and
Future Vision
Dr. Stanley Ahalt FAPOC, SIP The Ohio
Supercomputer Center
PET Technical Overview 12-14 May 2009
2
Presentation Outline

• Introduction (Slides 1 12)
• SIP Team
• Key Users
• Overview of FA
• Recent Technical Accomplishments (Slides 13 -
  58)
• Desktop to HPC Access
• Sharing and Mining SIP Data
• Computationally Intensive SIP Algorithms
• Code Transitions
• Future Vision (Slides 59 - 70)
• Challenges
• Opportunities
• PET-II Lessons Learned

3
SIP Team

• Dr. Stanley Ahalt, FAPOC
• Ohio Supercomputer Center, Ohio
• Dr. Alan Chalker, FAPOC assistant
• Ohio Supercomputer Center, Ohio
• Dr. Juan Carlos Chaves, ARL on-site
• Army Research Lab - Adelphi, Maryland
• Dr. Bracy Elton, AFRL on-site
• Wright Patterson AFB, Ohio
• Dr. Jose Unpingco, SSC-Pacific on-site
• SPAWAR - San Diego, California

• At-Institution Technical Experts
• Mr. Vijay Gadepally (PhD Cand.)
• Dr. Judy Gardiner (Staff)
• Mr. Brian Guilfoos (Staff)
• Ms. Laura Humphrey(PhD Cand.)
• Mr. Sid Samsi (Staff)
• Mr. Ben Smith (PhD Cand.)

Image from 2007 HPCMP Annual Report Success story
of RF absorption of the human head
4
Key Interactions

• Users
• Mr. Kelley Bennett (ARL)
• Dr. Yaroslav Chushak (AFRL)
• Dr. Fernando Escobar (SSC-Pacific)
• Lt. Col Scott Fawaz (CAStLE)
• Mr. Tom Kendell (ARL)
• Mr. Kevin Magde (AFRL)
• Mr. Tom Majumder (AFRL)
• Mr. Benji Maruyama (AFRL)
• Dr. Srinivasan Rajaraman (BHSAI)
• DoD Leaders
• Mr. Jeff Graham (AFRL)
• Dr. Aram Kevorkian (SSC-Pacific)
• Dr. Rich Linderman (AFRL)
• Dr. Lynn Parnell (SSC-Pacific)
• Dr. Bob Pritchard (SSC-Pacific)
• Dr. Terry Wilson (AFRL/RY)

• Locations
• AFRL/RX RY _at_ WPAFB, OH
• AFRL/RY _at_ Rome, NY
• ARL/ALC _at_ Adelphi, MD
• SSC-Pacific _at_ San Diego, CA
• And LOTS of others too many to list everyone
  here

Carbon Foam Visualizations
5
SIP UAP

• Dr. Terry Wilson, SIP CTA Lead
• AFRL, WPAFB, OH
• Dr. Keith Bromley
• SSC-Pacific, San Diego, CA
• Dr. Richard Linderman
• AFRL, Rome, NY
• Mr. Gary Stolovy
• ARL, Adelphi, MD
• Dr. Mike Bryant
• AFRL, WPAFB, OH

Example Digital Terrain Scenario
6
Overview of SIP FA

• Using HPC to extract, condense, and deliver
  timely innovative and accurate signal/image
  intelligence products e.g., target localizations,
  identifications, imagery, to enhance warfighter
  situational understanding
• SIP impact on DoD and warfighter results in
• Actionable intelligence
• Pervasive, all-weather, all-hour surveillance
• Real-time battlespace awareness
• Reduced casualties
• In SIP (and some other FAs) a key value-add of
  HPC/PET is in refining codes, as well as in
  REDUCING the time-to-solution and rapidly
  ADAPTING to continually CHANGING threats

7
What does SIP Cover?

• From the SIAM Activity Group on Imaging Science
• Sensors-to-Images The formation and construction
  of images (visible light, radar, computed
  tomography, ultrasound, seismic, molecular, etc.)
  from measured data, e.g., photon counts.
• Images-to-Images The transformation of "raw"
  images to "processed" images which are more
  useful or informative for specific applications,
  as in image restoration and compression.
• Images-to-Interpretations The semantic and
  structural annotation of images, for instance
  finding specific patterns, locating instances
  from generic object classes and recognizing
  activity and context.
• Note, acquisition, processing and interpretation
  are deeply interconnected for example, effective
  image restoration depends on a good model for
  image formation, and efficient image
  representation is crucial for image
  interpretation.
• The same applies to any type of signals, not just
  images!

8
SIP Team Support Capabilities

• Our SIP Team features complementary strengths and
  expertise
• Different capabilities meld into an incredibly
  effective team
• Can attack problems with a tiger team approach

9
Leveraging OSC

• Access to OSC HPC resources
• Often can serve as early pre-access to HPCMP
  systems for new users
• Can help avoid security policy paradox loops
• Provides testing of emerging architectures before
  available to general HPCMP community
• Access to OSC staff
• Experience with a wide variety of computational
  challenges
• Pipeline to a large, diverse user community.
• connections to a broad range of ongoing federal
  programs

10
User Requirements Overview

• The URs fall into 2 main areas
• Non-traditional HPC usage (desktop to HPC,
  high-level languages, remote viz)
• Data intensive (SIP data generation, sharing,
  mining, management)
• Issues/Concerns of UAP
• Working with the very large data sets SIP codes
  generate (see later MPSCP discussion)
• Facilitating transitions from desktops to HPCs /
  high level languages

11
Priority User Requirements for SIP

• Run SIP applications from desktops on HPC
  (UR-SIP-05-01)
• Users are requiring easy access to HPCs as
  problems increase in size, but they are more
  familiar and comfortable with desktop
  environments and GUI driven applications
• Sharing and mining SIP data (UR-SIP-05-08)
• Allow distributed users to share and mine
  independently created and stored data
• Computationally intensive SIP algorithms on HPC
  (UR-SIP-05-02)
• Many widely used serial versions of SIP
  algorithms do not have HPC counterparts readily
  available, and need to be ported to HPC
  platforms.
• Simplify uniprocessor SIP code transitions to
  multiprocessor (multicore) HPC (UR-SIP-05-05)
• Non-conventional high-level prototyping languages
  have become more and more important for improving
  SIP researchers productivity, yet arent
  necessarily available on HPCs or in parallel
  versions.

12
Other User Community Needs

• Data challenges and the use of high level / high
  productivity languages for HPC are issues that
  are becoming increasingly prominent at DOE / NSF
  / NIH
• Developing a Coherent Cyberinfrastructure
  from Local Campus to National Facilities
  Challenges and Strategies
• Interactive and virtualized HPC are the next big
  things moving into production environments

13
Recent PETII Technical Accomplishments

• Desktop to HPC Access
• SSH Toolbox
• VISION/HPC
• Star-P
• Sharing and Mining of SIP Data
• GOTCHA Radar
• MPSCP
• Metadata Tools
• Computationally Intensive SIP Algorithms
• STAP Codes
• Predator Analysis
• CHSSI Collaboration
• Code Transitions
• JVNCW
• SQUIDs
• RF Radiation Effects on the Warfighter

Tank Column Raw Image
Tank Column CBIC Image
14
1.a. SSH Toolbox

• Application
• The ultimate goal of this effort was to provide
  an easy to use and install solution for
  submitting computational requests from the users
  desktop to be executed on HPCMP resources and
  providing information back to the desktop from a
  variety of front end applications, including
  MATLAB, Python, Octave, and Java.
• SIP Team Members
• Dr. John Nehrbass
• Mr. Sid Samsi
• Mr. Brian Guilfoos

• Users Impacted
• Dr. Brent Foy, Dr. Brian Rigling (Wright State
  University)
• Dr. Mike Minardi, Mr. Tom Majumder,
• Mr. Steven Scarborough,1st Lt Curtis Casteel,
  Dr. LeRoy Gorham (AFRL/RYAS)
• Dr. Ron Dilsavor (SET Corporation)
• Dr. Catherine Deardorf (AFRL/RYAT)
• Dr. Randy Moses, Dr. Lee Potter (OSU)
• Mr. Howard Nichols, Mr. Greg Owirka, Mr. Thomas
  Kragh (BAE)

Desktop to HPC Access
Image Matching Demo
15
1.a. SSH Toolbox

• Technical Effort
• Created SSH toolbox for MATLAB, Octave, Python,
  and Java
• Developed install wizards, documentation, and
  tutorials for SSH toolboxes.
• Provided APIs to allow for direct implementation
  in user code
• Interfaced with the SIP High Productivity Tool

Desktop to HPC Access
Queue Status Application
16
1.b. VISION / HPC

• Application
• VISION/HPC is a Python-based, drag-and-drop
  visual-programming environment that reduces
  sophisticated programming tasks to dropping and
  connecting icons in a GUI flowchart. It has been
  developed, documented, and demonstrated to be a
  productivity-enhancing visual computing framework
  for parallel computing that allows users to draw
  flowcharts on a locally running GUI and compute
  those flowcharts on a remote back-end (or offline
  in a local sandbox).
• SIP Team Members
• Dr. Jose Unpingco
• Michel Sanner, Guillaume Vareille,
• Sargis Dallakyan (Scripps RI)
• Fernando Perez, Benjamin Ragan-Kelley
• (UC, Berkeley)
• Brian Granger (Cal Poly)
• Users Impacted
• Dr. Bob Pritchard (SSC-Pacific)
• Mr. Ken LeSueur (RTTC)

Desktop to HPC Access
VISION / HPC Screenshot
17
1.b. VISION / HPC

• Technical Effort
• Extend VISION to utilize IPython as underlying
  framework for parallel computation.
• Build comprehensive one click Windows installer
• 50 of effort was from open-source volunteers
• Results
• VISION/HPC makes HPCs easier to use for
  non-specialist Windows users .
• As a Windows-based open source Python package, it
  installs with one mouse click and encapsulates
  over 45 Python modules including numpy (numerical
  arrays and linear algebra), SciPy (statistics,
  interpolation, etc.), Matplotlib (scientific
  visualization), PIL (Python Imaging Library) and
  IPython (interactive parallel computing).
• Tutorial screen cast videos are embedded in the
  documentation

Desktop to HPC Access
18
1.b. VISION / HPC
Desktop to HPC Access
19
Fast web-based targeted training
Desktop to HPC Access
20
1.b. Unpingco Letter of Commendation
Desktop to HPC Access
21
1.c. Star-P

• Application
• Interactive parallel computing platform from
  Interactive Supercomputing, Inc. (ISC)
• Extends existing desktop simulation tools for
  simple, user-friendly parallel computing to a
  spectrum of computing architectures SMP
  servers, multicore servers, and distributed
  clusters
• SIP Team Members
• Dr. Bracy Elton (OSC)
• Mr. Sid Samsi (OSC)
• Mr. Ben Smith (OSC)
• Dr. Niraj Srivastava (ISC)
• Users Impacted
• Mr. Kevin Magde (AFRL)
• Dr. Srinivasan Rajaraman (BHSAI)

Desktop to HPC Access
Star-P System Diagram
22
1.c. Star-P

• Technical Effort
• Address Star-P becoming permitted on HPCMP
  systems
• Examine how to use Star-P in batch environments,
  especially those in use on DSRC systems
• Look at scalability of radio frequency (RF)
  tomography algorithms on large HPCMP DSRC systems
• Results
• Star-P available on ARL DSRC MJM system in
  various modes
• Interactive Star-P client on Windows or Linux
  desktop Star-P server in batch reservation or
  regular batch job
• Pure Batch Star-P client server in same batch
  job (useful for parameter studies)

Desktop to HPC Access
RF Tomography Visualization
23
1.c. Star-P

• Papers
• UGC 2009 Oral Paper
• Bracy H. Elton, Siddarth Samsi, Harrison Ben
  Smith, Laura Humphrey, Brian Guilfoos, Stanley
  Ahalt, Alan Chalker, Kevin M. Magde, Niraj K.
  Srivastava, Aquil H. Abdullah, Patrick Boyle,
  Using Star-P on DoD High Performance Computing
  Systems
• UGC 2009 Poster Paper
• Bracy H. Elton, Siddharth Samsi, Harrison Ben
  Smith, Stanley Ahalt, Alan Chalker, Kevin M.
  Magde, Niraj Srivastava, Aquil H. Abdullah,
  Patrick Boyle, A Scalability Study on DSRC HPC
  Systems of Radio Frequency Tomography Code
  Written in Star-P/MATLAB
• SC09 Paper (submitted)
• Bracy H. Elton, Siddarth Samsi, Harrison Ben
  Smith, Laura Humphrey, Brian Guilfoos, Stanley
  Ahalt, Alan Chalker, Kevin M. Magde, Niraj K.
  Srivastava, Aquil H. Abdullah, Patrick Boyle,
  Practical High Performance Computing A Case
  Study

Desktop to HPC Access
24
1. Other Desktop to HPC Access

• OSSIM VSIPL Comparison
• Application Many SIP users utilize the Vector
  Signal Image Processing Library (VSIPL) for
  image processing tasks. A similar product is
  available, Open Source Image Map (OSSIM), which
  several intelligence and defense agencies have
  developed and currently use.
• Result OSSIM not sufficiently mature to be
  suitable for wide-scale deployment by the HPCMP
  but VSIPL is
• Users Impacted Dr. Keith Bromley (SSC-Pacific),
  Dr. Richard Linderman (AFRL/RI), Young,
  Robertson, Sim, Gill, Mirelli, Zong, Fischer, Vu,
  Liss, Wellman, Filipov, Chan, Saini, Weber
  (ARL/SEDD)
• Distributed Interactive HPC Testbed (DIHT)
• Application Provide DoD scientists/engineers
  interactive HPC distributed capabilities over
  wide geographic area
• Results Transferred and optimized parallel
  MATLAB technologies. Cray Henry presented _at_ HPEC
  2004
• Users Impacted AFRL/RI SIP community

Desktop to HPC Access
25
1. Other Desktop to HPC Access

• Biomolecular Network Modeling
• Application MATLAB code used to model the
  Biomolecular Network of Glutathione Synthesis in
  a Cell-Free Transcription/Translation System
• Results 64x speedup with MatlabMPI
• Users Impacted HPCMPO Biotechnology HPC Software
  Applications Institute (BHSAI), Dr. Brent Foy
  (Wright State University), Dr. John Frazier and
  Dr. Yaroslav Chushak (Air Force Research
  Laboratory)

Desktop to HPC Access
Biomolecular Gene Model
26
2.a. GOTCHA Radar

• Application
• Project involves real-time persistent localized
  high resolution synthetic aperture radar (SAR)
  with spotlighting capability, forensics, tracking
  other features that generate significant
  amounts of data (terabytes to petabytes)
• PET Team Members
• Dr. Bracy Elton (SIP), Dr. Rhonda Vickery (ET)
• Users Impacted

Sharring / Mining SIP Data
SAR Image of Ohio Stadium
27
2.a. GOTCHA Radar
Gotcha DHPI Logical Diagram For Real-Time SAR
Sharring / Mining SIP Data
CD change detection SAR synthetic aperture
radar GMTI ground motion target indication
28
2.a. GOTCHA Radar
Gotcha DHPI HPC System Diagram For Real-Time SAR
Altix ICE 8200 Cluster 2048 cores Nehelam-EP 1.5
GB/core Memory 22.9 Tflop/s - 256 nodes 4
Compute/1 I/O Racks 4X IB Connected to Lustre
2 x10GbE
1 x10GbE
2 x10GbE
4 Login Nodes
Login
2 Ingest Nodes
Ingest
Batch
1 Optional Batch Node
Meta-data Servers
Admin
1 Admin Node
CS-Admin
Sharring / Mining SIP Data
MDS
Cold Spare Admin Node
Storage Node-to-IRU Connections
2
IS220
30
MDS
2x10GbE
24-port GigE Switch
Altix 450 32 cores Itanium-MV 4 GB/core
Memory 0.2 Tflop/s 1 node IB Connected to
Lustre
102 TB Raw - 87 TB Usable Capacity
29
2.a. GOTCHA Radar

• Technical Effort
• Trained 24 Gotcha Radar HPC users
• Multiple orbits (0.5 TB) of 2006 Gotcha Radar
  Data Collection loaded onto AFRL DSRC Falcon
  Hawk systems
• Developed strategies for developing real-time
  codes in AFRL DSRC batch environment
• Facilitated AFRL DSRC persistent data storage
  (workspace) allocation and scrubber exceptions
• Consulted on Video SAR parallel algorithm
  implementation performance analysis
• Facilitated successful demonstrations of Gotcha
  Radar at AFRL Sensors Scientific Advisory Board
  in October 2008
• Coauthored papers presented at various
  conferences, e.g., Tri-Service, AFRL Technical
  Forum
• Consulted on future HPC needs of Gotcha Radar
  Exploitation Program

Sharring / Mining SIP Data
30
2.a. GOTCHA Radar

• Technical Effort (cont.)
• Collaborated to prepare winning(!) HPCMP DHPI
  proposal for a real-time HPC system for Gotcha
  Radar program
• Gotcha DHPI coming to AFRL DSRC Summer 2009
• Worked with TI-09 AFRL Integration Team to Ensure
  proper HW SW configurations for flexibility
  success
• Worked with Gotcha Radar Team to
• Prepare SW for Gotcha DHPI system
• Facilitated Gotcha Video SAR development on ARL
  DSRC MJM system (most like Gotcha DHPI system
  architecture)
• Evaluate site for Gotcha DHPI infrastructure
• Boeing RapidLink Ground Station SAR system
  compatibility
• Ground Station proximity to HPC system
• Consulted on network connectivity how WorldWind
  DataTable can access Gotcha DHPI products
• Developed began implementing plan for uploading
  5 TB of 2008 Layered Sensing Data Collection
  Gotcha Radar data to AFRL DSRC

Sharring / Mining SIP Data
31
2.a. Elton Letter of Commendation
Sharring / Mining SIP Data
32
2.b. MPSCP

• Application
• There has been a persistent need for fast
  large-scale file transfer for SIP users in
  particular, since these users typically work with
  multi-terabyte data that is remotely collected
  and requires data transfer to an HPC for
  processing. Multiple Path Secure Copy (MPSCP)
  from DOEs Sandia National Laboratory accelerates
  file transfer by using multiple TCP streams and
  SSH-authentication
• SIP Team Members
• Mr. Brian Guilfoos
• Ms. Laura Humphrey
• Dr. Jose Unpingco
• Users Impacted
• Dr. Frank Ryan (SSC-Pacific)
• Dr. Kiranmai Naidu (ex AFRL/RY)
• Lt. Col Scott Fawaz, Center for Aircraft
  Structural Life Extension (CAStLE)

Sharring / Mining SIP Data
33
2.b. MPSCP

• Technical Effort
• Incorporated stream encryption and real-time
  diagnostics for performance modeling into code
• Developed documentation on usage and
  administration
• Worked with Baseline Configuration Team and Vern
  Staats regarding development and usage at the
  centers
• Did development and testing on OSC HPC systems
• User Feedback
• Our research group uses mpscp exclusively for 
  transferring TB of data from USAFA to/from 
  AFRL/NAVO/ERDC.  I would estimate a minimum of 5
  TB per  month over the past few years.  I would
  not be able to  execute my Challenge Project,
  C2G, without mpscp.  I know  Scott Morton and
  Keith Bergeron who also have a Challenge  Project
  use mpscp extensively. Regarding on-site
  support, every time we have a problem,  you fix
  it.  Doesn't get any better than that.  By the 
  way, the problems you fix have been due to
  hardware  changes on the two machines, not a bug
  with mpscp. - Lt. Col Scott Fawaz, Center for
  Aircraft Structural Life Extension (CAStLE)

Sharring / Mining SIP Data
34
2.b. MPSCP
Sharring / Mining SIP Data
35
2.c. SIP Metadata Tools

• Application
• The process of transferring large data sets to
  HPCs for analysis is often a significant hurdle.
  The focus for this effort is squarely on the
  development, deployment, and documentation of a
  web based meta-data generation, browsing, and
  transfer tool.
• SIP Team Members
• Mr. Brian Guilfoos
• Mr. Sid Samsi
• User Partners
• Dr. Terry Wilson (AFRL)
• Dr. Rich Linderman (AFRL)
• Mr. Jeff Graham (AFRL DSRC)

Sharring / Mining SIP Data
Metadata Tool Screenshot
36
2.c. SIP Metadata Tools

• Technical Effort
• Continued focus on effort as a direct result of
  positive comments from Linderman, Wilson, and
  Graham at UGC 2007
• Enable MPSCP through the existing web interface
• Re-write the web application code (specifically,
  the Tomcat servlet) so it is more flexible and
  robust.
• Add additional features including the ability to
  transfer files to multiple hosts in one
  transaction, the ability to upload files from the
  users desktop to the HPC and the ability to
  download files from the HPC to the users
  desktop.
• Create a partner application that allows a user
  to specify the encoding of the metadata used by
  the path/filename, which then will automatically
  generate an RDF file for a database that has no
  RDF metadata file.

Sharring / Mining SIP Data
37
2. Other Sharing and Mining SIP Data

• Grid Computing for DoD (GridFTP KX.509)
• Application A key objective of the DoD HPCMP
  Metacomputing Working Group (MCWG) has been to
  establish a secure and robust bridge from the DoD
  HPCMP's Kerberos authenticated computational
  infrastructure to PKI-based Grids to fully
  leverage matured grid capabilities and services
  that are being continually advanced by the
  academic and DOE communities using NSF's
  Extensible Terascale Facility (ETF) or the
  TeraGrid and other grids.
• Results First successful demonstration of job
  submittals from HPCMPs Kerberos authenticated
  computational infrastructure to a PKI-based Grid
• Users Impacted Dr. Aram Kevorkian (SSC-Pacific)

Sharring / Mining SIP Data
38
3.a. STAP Applications

• Application
• Space-Time Adaptive Processing (STAP) for
  Heterogeneous Clutter Scenario involves
  improvements to airborne radar performance for
  the detection of embedded targets in the clutter.
  These improvements include clutter suppression
  for the detection of low-velocity targets,
  enhancement in the detection of small targets
  embedded in the clutter, and the efficient
  detection in a combined clutter and hostile
  jamming environment.
• SIP Team Members
• Dr. Juan Carlos Chaves
• Users Impacted
• Dr. Muralidhar Rangaswamy
• Dr. Freeman Lin
• Capt. Patrice Wolfe (AFRL/RYHE)

SIP Algorithms
39
3.a. STAP Applications

• Results
• Ported code to MatlabMPI
• Considerable speedup obtained (ranged 35x to
  infinite)
• Completed STAP simulation for required parameters
• Several months of computation at ARL DSRC
• Without the use of HPC this would not have been
  possible

SIP Algorithms
Image from HPCMP 2006 Annual Report Success Story
40
3.b. Predator Analysis

• Application
• Radar cross section analysis of a Predator
• SIP Team Members
• Dr. Juan Carlos Chaves
• Dr. John Nehrbass
• Users Impacted
• Dr. Ed Zelnio (AFRL/RYA)
• Dr. Ron Dilsavor (AFRL/RYAS)

FMS
SIP
CEA
ET
Predator Analysis
ASCDSRC
ARLDSRC
SIP Algorithms
AFIT OR Dr. J.O. Miller Combat Modeling
3D RCS Visualization Tool
AFIT EN Capt Peter Muend MS Thesis XPatch analysis
SIP-04-002 Signal MiningFusion andValidation
FMS-04-002 Multiple Constructive Model Runs
ASC/FBMr. Tyle Kanazawa AFRL/SNASDr. John
Malas XPatch analysis
AFRL/SNA Dr. Zelnio, Dr. Dilsavor Time
CriticalXpatchTime Domain Processing
Multiple Cross-CTA Visualization Efforts
41
3.b. Predator Analysis

• Technical Effort
• Perl script prototype
• Xpatch experience
• DEMACO/SAIC contact and support
• ARL Classified and unclassified access
• Fluid support for users that overlapped FMS
  support. Thus Classified processing was
  monitored and adjusted every day

SIP Algorithms

• Result
• The HPCMPO and PET involvement are allowing a
  much higher fidelity and more timely RF signature
  prediction for the Predator MQ-9. Without the
  support of the HPCMPO compute power, it would
  take us months, if not years, to provide such a
  detailed signature prediction to the Predator
  Program Office. These results help the Predator
  Program Office ensure that the Predator MQ-9 will
  be properly employed in operational support of
  the warfighter. - Richard Graeff, ASC/HPMT

Image from 2005 HPCMP Annual Report Success Story
42
3.c. CHSSI Collaborations

• Application
• Extensive alpha and beta testing activities in
  support of SIP CHSSI Projects and Portfolios
• SIP Team Members
• Dr. Juan Carlos Chaves
• Dr. John Nehrbass

SIP Algorithms
Screen shot of web-enabled codes from HIE
Portfolio
Hyperspectral data cube
43
3.c. CHSSI Collaborations

• Technical Effort
• SIP-8, Infrared Search and Track for Missile
  Surveillance (IRST)
• User Impacted Cottel (SPAWAR)
• Hyperspectral Image Exploitation (HIE) Portfolio
• User Impacted Linderman (AFRL-IF)
• SIP-7 Task 2 (VSIPL) Efficient, Maintainable,
  Portable and Scalable HPC Codes for Image Fusion
  and Signal/Image Processing
• User Impacted Linderman (AFRL-IF)
• HIE-3 Automatic Target Detection in
  Hyperspectral Imagery using Principal Components
  Analysis
• User Impacted Stolovy (ARL- ALC)
• EM General Framework for 21st Century Integrated
  Military Platforms
• User Impacted Rockway (SPAWAR San Diego)
• Integrated Parallel Framework for Network Centric
  Warfare Simulations (PAWARS)
• User Impacted Perlman (CEN)

SIP Algorithms
44
3. Other SIP Algorithms

• Non-destructive Missile Evaluation
• Application A cone-beam X-Ray computed
  tomography (CB-CT) system used for
  non-destructive evaluation
• Results Developed and implemented statistical
  based image reconstruction algorithms
• Users Impacted Mr. Hayden Martin, Mr. Scott
  McLain (NSWC)
• Image Segmentation and Analysis
• Application Focused ion beam microscope /
  optical microscope images of special metal alloys
  and carbon foams
• Results Processed images for grain size,
  morphology and defect characterization in an
  effort to build 3-D models for input to
  structural analysis codes
• Users Impacted Dr. Jeff Simmons, Dr. Benji
  Maruyama (AFRL/RX)
• Search Radar for Thin Wire Detection
• Application A radar analysis code that could be
  used to detect IEDs in the direction of travel of
  a vehicle
• Results Provided guidance on parallelizing the
  code
• Users Impacted Dr. Steven Bishop and Dr. Jay
  Marble (CERDEC), Dr. Matthew Ferrara (AFRL)

SIP Algorithms
45
3. Other SIP Algorithms

• Urban Acoustic Array Processing
• Application A challenge project to simulate
  geo-seismic and geo-acoustic events in a urban
  environment for source detection, localization,
  identification, and perimeter defense
• Results Provided suggestions on how to process
  signals in the acoustic propagation domain
• Users Impacted Challenge Project with Dr.
  Stephen Ketcham (ERDC), Dr. Harley Cudney (ERDC)
• Atmospheric Laser Optics Testbed Facility Support
• Application The Atmospheric Laser Optics Testbed
  Facility at ARL ALC (A_LOT) supports the study of
  flow patterns and microclimate along the optical
  path affecting free-space laser performance.
• Results Helped users perform sophisticated data
  analysis visualization
• Users Impacted Mr. Arnold Tunick, (Computational
  and Information Sciences Directorate at ARL ALC)

SIP Algorithms
46
3. Other SIP Algorithms

• Feature Selection for Object Classification in
  Thermal Images
• Application Target classification recognition
  exploiting infrared thermal images.
• Results Improved speeds of classification codes.
• Users Impacted Lt. Col. William L. Fehlman II,
  College of William Mary, Dr. Stephen Landowne,
  United States Military Academy

SIP Algorithms
Infrared Thermal Images
47
3. Chaves Commendation

• Lieutenant Colonel William Fehlman, provided the
  following quote
• Dr. Juan Carlos Chaves (Signal and Image
  Processing Ohio Supercomputer Center team)
  provided unparalleled assistance in porting and
  optimizing my Feature Selection for Object
  Classification MATLAB code for use on the HPCMP
  HPC resources. His assistance allowed me to
  reduce my computation time by 80 to yield an
  increased research productivity in computing
  performance measures for over 260,000 feature
  vectors generated from thermal images of various
  targets.

SIP Algorithms
48
4.a. Joint Virtual Network Centric Warfare

• Application
• The Joint Virtual Network Centric Warfare Project
  visualizes and models communication channels and
  corresponding propagation on a global scale.
• SIP Team Members
• Dr. Jose Unpingco
• Users Impacted
• Dr. Robert Pritchard (SSC-Pacific)

Code Transitions
JVNC Visualization
49
4.a. Joint Virtual Network Centric Warfare

• Technical Effort
• Ported existing line-of-sight code from x86
  architecture to an IBM/Linux system at
  SSC-Pacific
• Worked on visualization, documentation and
  demonstrations
• Working on embedding python-based open source
  modules (e.g. VISION) for virtual component
  prototyping and integration with existing Java
  applets.
• Results
• Code has been ported successfully
• Was able to separate the visualization component

Code Transitions
50
4.a. Joint Virtual Network Centric Warfare
Code Transitions
51
4.b. SQUIDs Highly Sensitive Superconductor
Sensors

• Application
• Optimization/parallelization of codes to
  simulate Superconducting Quantum Interference
  devices (SQUIDs) (a superconducting circuit based
  on Josephson junctions). SQUIDs are the worlds
  most sensitive detectors of magnetic signals
  (sensitivity fT) for the detection and
  characterization of signals so small as to be
  virtually immeasurable by any other known sensor
  technology.
• SIP Team Members
• Dr. Juan Carlos Chaves
• Users Impacted
• Dr. Patrick Longhini, Dr. Fernando Escobar, Dr.
  Anna Leese, Mr. Kenneth Simonsen and Mr. Kevin
  Lam (SSC-Pacific)

Code Transitions
SQUID a tiny loop of superconducting material
interrupted by narrow gaps / Josephson junctions
52
4.b. SQUIDs Highly Sensitive Superconductor
Sensors

• Technical Effort
• Provided extensive support in transition of users
  and code to HPCMP HPC resources
• Assisted with profiling/analysis and porting of
  code
• LSF and HPC consultancy / support
• Extensive parallelization/vectorization through
  SIP-KY8-001 PET project
• Results
• Simulations running with 10000 SQUIDs are now
  possible at OSC and ARL DSRC
• Potential of new physics insight thanks to 100
  fold increase in of SQUIDs that now can be
  simulated
• Details to be showcased at UGC 2009

Code Transitions
Concealed weapon detection accomplished by a
superconducting hot spot antenna-coupled
microbolometer record net-equivalent-temperature
-difference sensitivity of 0.13 K (image produced
by NIST at Boulder, CO)
53
4.c. RF Radiation Effects on the Warfighter

• Application
• The understanding of the biophysical and
  biological impact of electromagnetic fields on
  humans many DoD personnel work in close
  proximity to intense and possible pervasive EM
  fields.
• SIP Team Members
• Dr. Juan Carlos Chaves
• Users Impacted
• Jason Payne at Frequency Radiation Branch at the
  Air Force Research Laboratory, Human
  Effectiveness Directorate, Directed Energy
  Bioeffects Division, (AFRL/RHDR).

Code Transitions
Image from 2007 HPCMP Annual Report Success story
of RF absorption of the human head
54
4.c. RF Radiation Effects on the Warfighter

• Methodology
• Finite Difference Time Domain (FDTD) code plus 3
  mm resolution model of electrical properties of a
  human being
• Results
• Extremely quick turnaround for required
  optimization
• Code optimized by a factor of 30
• Code optimization techniques transferred to
  AFRL/RHDR potentially benefiting many more of
  Brooks Visible Man AFB codes
• Vectorization of these EM for-loops resulted in
  30X speedup

Code Transitions
EM Code Speedup Graph

• From Jason Payner Initial runs indicate that
  implementing this process may decrease our
  simulation run time by up to a factor of 30.
  This type of performance enhancement will greatly
  increase the quality and efficiency of work that
  our modeling team can output.

55
4. Other Code Transitions

• MIRAGE Scene Generation
• Application Address the slight misalignment (be
  it rotational, translational, azimuthal or a
  combination of any of these) or magnification
  error between the MIRAGE emitter the sensor FPA.
• Results Reduced runtime from 3.5 days to under
  an hour
• Users Impacted Mr. Corey Slick (RTTC)
• Soil ATR Support Codes
• Application Ultra-wideband synthetic aperture
  radar (UWB SAR) technology - detect and classify
  targets concealed by foliage and subsurface
  targets
• Results Ported and optimized code on HPCMP
  resources
• Users Impacted Sensors and Electron Devices
  Directorate (SEDD) at ARL ALC (POC Mr. Mosharraf
  Qaadri)

Code Transitions
MIRAGE Generated Scene
56
4. Other Code Transitions

• General Utility Algorithms
• Application Various SIP relevant algorithms
  including Woodbury Algorithm, Interpolation via
  Triangulation, Random Number Generator, Markov
  Chain Monte Carlo, Content Based Image
  Compression, Support Vector Machine, K-Means,
  Multidimensional FFT
• Results Ported algorithms to parallel MATLAB
• Users Impacted Dr. Aram Kevorkian
  (SSC-Pacific), Dr. Ed Zelnio, Ms. Kiran Naidu,
  Dr. Mike Minardi, Mr. Tom Majumder, Dr. Ron
  Dilsavor, Terry Wilson (AFRL/RY), Murali
  Rangaswamy (ARL)

Code Transitions
Support Vector Machine
Interpolated Points (Triangulation)
57
UGC 2009 Participation

• VISION/HPC
• Oral Presentation
• Dr. Jose Unpingco
• Python
• Tutorial
• Dr. Jose Unpingco
• SQUIDs
• Oral Presentation
• Dr. Juan Carlos Chaves
• RDF Meta-data
• Oral Presentation
• Mr. Sid Samsi
• MPSCP
• Poster Presentation
• Brian Guilfoos
• Star-P
• Oral and Poster Presentation
• Dr. Bracy Elton

58
Future Vision Challenges

• Addressing user requirements on a spectrum of
  machines
• From many-core desktops to highly parallel
  machines gtgt UGC08 multicore programming tutorial
• User requirements collection
• Impact metrics
• Relevance to warfighter
• High Level Languages software issues / advantages
• HPCMP policies (batch environment, deployment,
  availability)
• O/S support / licensing for commercial products
  such as MATLAB and Star-P
• Provides quick tool development platforms
• Allocation of PET (SIP) resources
• Balancing focused expertise vs. general support
• Expanding and intermittent user base
• Providing mechanisms for non-traditional support
  and interactions
• Eclectic users need once-ware vs. developing
  general workflow solutions
• Software security issues at centers
• Getting large amounts of data into DSRC systems
  in light of USB device restrictions

59
Tying it All Together
Asymmetric threats
Biological modeling
HLL
Pervasive surveillance
RADAR
Rapid prototyping
60
Tying it All Together
Technology identified, enhanced, and delivered
tech transfer collaborations that leads to real
solutions.
61
Tying it All Together

• Our research group uses mpscp exclusively for 
  transferring TB of data from USAFA to/from 
  AFRL/NAVO/ERDC.  I would estimate a minimum of 5
  TB per  month over the past few years.  I would
  not be able to  execute my Challenge Project,
  C2G, without mpscp.  I know  Scott Morton and
  Keith Bergeron who also have a Challenge  Project
  use mpscp extensively. Regarding on-site
  support, every time we have a problem,  you fix
  it.  Doesn't get any better than that.  By the 
  way, the problems you fix have been due to
  hardware  changes on the two machines, not a bug
  with mpscp. - Lt. Col Scott Fawaz, Center for
  Aircraft Structural Life Extension (CAStLE)

62
Tying it All Together
This is really about leadership, credibility, and
relationships. This is technology that is
conceptualized, consolidated, and deilvered.
This is technology transfer.
63
Tying it All Together
64
Tying it All Together
pervasive surveillance
Detection and identification
Not just tools, but real systems. SIP team has
had a real impact on fusion products that provide
in-theater benefits to the warfighter. Its
software, and systems and relevance to the
warfighter.
65
Tying it All Together
66
Tying it All Together
Human safety
Radiation effects
And its not just about the technology, or the
labels (stove pipes) its the people.
67
Tying it All Together

• Lieutenant Colonel William Fehlman, provided the
  following quote
• Dr. Juan Carlos Chaves (Signal and Image
  Processing Ohio Supercomputer Center team)
  provided unparalleled assistance in porting and
  optimizing my Feature Selection for Object
  Classification MATLAB code for use on the HPCMP
  HPC resources. His assistance allowed me to
  reduce my computation time by 80 to yield an
  increased research productivity in computing
  performance measures for over 260,000 feature
  vectors generated from thermal images of various
  targets.

68
Lessons Learned

• Proven, productive, and proactive technical
  service is vital
• The onsites are a core pool of proven, dedicated
  HPC professionals easily accessible to DoD users
• They need to be backed by a deep pool of
  at-instituion highly skilled senior experts and
  junior support staff
• There needs to be a focus on proactive discovery
  of user needs in addition to provide highly
  responsive reactionary support
• Efficient Management is important
• Dedicated technical leaders yield efficiency in
  management and technical communication and
  control
• Performance management is a critical and
  necessary precursor to improvement
• FA stovepipes are hard to overcome, but with
  appropriate cross-integration of areas, users
  receive the benefits of collaboration with users
  in others areas, and overall management
  efficiency and cross-team communication are
  improved

69
Lessons Learned

• We need to provide
• Lean, professional management
• Dedicated, passionate technical expertise
• Immediate reach-back into a large pool of
  outstanding technical talent at varying levels.
• We need to operationalize
• Daily (highly involved) management with a
  continual awareness of emerging HPCMP user needs
• A capability for rapid deployment of expertise
  needed to address both established and emerging
  needs, and sometimes emergency needs
• An engaged team of technical experts
• Results
• Users receive consistent, highly technical,
  peer-to-peer support from proven technical
  experts with deep academic/multi-agency/cross-disc
  ipline roots