Title: Using Visualization to Understand the Behavior of Computer Systems
1Using Visualization to Understandthe Behavior of
Computer Systems
- Robert P. Bosch Jr.
- Stanford University
- May 3, 2001
2Motivation
- Explosion in complexity of computer systems
- Development of rich data collection tools
- Complete Machine Simulation SimOS
- Software Monitoring Performance Co-Pilot (PCP)
- Firmware Instrumentation FlashPoint
- Hardware Monitoring DCPI
- Challenge how do we fully exploit the large,
detailed data sets these tools can generate?
3Data Analysis Challenge
- How do we typically handle this data?
- Visual inspection of huge log files
- Summarize through statistics and aggregation
- Focus on restricted data subsets
- Alternative approach data visualization
- Display large amounts of data at once
- Enable interactive exploration of entire data set
- Overview, zoom and filter, details-on-demand
- Use human perception to discover patterns,
trends, and interesting information
4Computer Systems VisualizationExisting Work
- Pedagogical examples
- Processors DLXView, Pentium Pro Tutorial
- Memory Cache Visualization Tool (CVT)
- Parallel systems performance
- AIMS, Pablo, Paradyn, ParaGraph, PARvis,
StormWatch, VAMPIR - Other examples
- Network performance, file systems, etc.
5Computer Systems VisualizationExisting Work
- Demonstrate potential of visualization
- Limitations
- Focused on particular system components
- Integrated with specific data collection tools
- Limited to a fixed set of visual representations
- Conclusion rich, flexible data sources require
an equally powerful and flexible visualization
system
6Outline
- Motivation
- The Rivet visualization environment
- Architecture
- Implementation
- Focused visualization systems
- SUIF Explorer
- Thor
- PipeCleaner
- Visible Computer
- Ad hoc analysis and visualization
- Contributions
7Rivet Architecture Goals
- Learn once, apply to wide range of problems
- Decouple visualization and data collection
- Interactive exploration of large data sets
- Couple visualization and analysis
- Rapid prototyping of visualizations
- Extensibility
- Allow users to add new components
8Rivet Architecture Approach
- Identify visualization building blocks
- Mix and match to create visualizations
- Users can add new components as needed
- Three basic object types
- Data management tuples, tables, and transforms
- Visual representation primitives and metaphors
- Mapping from data to visual encodings
9Data Model
- Simplified relational model
- Tuple collection of data fields
- Table set of tuples with common data format
- Familiar, homogeneous model
- Load data by parsing text files
- Directly save/load tables in binary format
Tuple
Table
10Data Transforms
- Transforms enable users to operate on data
- Can be composed to form data networks
- Active data changes are propagated
- Rivet includes a set of standard transforms
- Filter, sort, group, merge, join, aggregate, etc.
- Users may design and incorporate their own
- Example clustering algorithms
GroupBy
Transform
11Visual Objects and Data Mapping
- Primitives draw individual tuples
- Metaphors draw entire tables
- Draws table attributes axes, labels, etc.
- For each tuple compute bounding box, select
primitive - Encodings map tuple contents to visual properties
- Metaphors use spatial encodings
- Map tuple fields to bounding box parameters
- Primitives use attribute encodings
- Map tuple fields to retinal properties color,
fill pattern, size - Encodings encapsulate data?visual mapping
- Metaphors and primitives are data independent
12The Encoding Process Example
13Coordination and Interaction
- Implicit coordination sharing of objects
- Examples
- Primitives share attribute encodings brushing
- Metaphors share primitives common appearance
- Metaphors share spatial encodings common axes
- Listener mechanism
- Explicit coordination events and bindings
- Rivet objects can raise events
- User can bind actions to these events
- Example details-on-demand
14Rivet Implementation
- Goal balance performance and flexibility
- Interactive visualizations of large data sets
- Rapid prototyping of visualizations
- C and OpenGL for performance
- Also provides platform independence
- Scripting language interfaces for flexibility
- Simplified Wrapper and Interface Generator (SWIG)
- Create visualizations by writing scripts
- Interpreter is not in the main loop
- Listener mechanism for high-frequency events
- Event bindings for low-frequency user interaction
15Sample Rivet Script
16Outline
- Motivation
- The Rivet visualization environment
- Architecture
- Implementation
- Focused visualization systems
- SUIF Explorer Interactive user-directed
parallelization - Thor Detailed memory profiling on FLASH
- PipeCleaner Superscalar processor pipelines
- Visible Computer System and cluster monitoring
- Ad hoc analysis and visualization
- Contributions
17SUIF ExplorerInteractive Parallelization
18SUIF Explorer Background
- Goal enable sequential codes to run fast on MPs
- Compiler parallelizes loops when possible
- Otherwise
- Compiler presents loop analysis results to user
- User applies application knowledge to assist
compiler - Data source SUIF dynamic analyzers
- Performance metrics for all loops in program
- Coverage, granularity, loop level
- Visualization
- Displays data in context of program source code
- Focuses on loops most deserving of user attention
19SUIF Explorer Discussion
- Benefits
- Combines data with source, instead of loop IDs
- Allows user to filter uninteresting lines of code
- Facilitates comparisons between runs
- Limitations
- Loosely coupled with rest of SUIF Explorer
- Short loops less prominent than long loops
- Add sortable table of results as a linked view
20Thor Detailed Memory Profiling
21Thor Background
- Memory getting slower relative to CPU
- High remote access latencies on NUMA systems
- Memory system is often performance bottleneck
- Data source FlashPoint protocol on FLASH
- Collects all cache and TLB misses in firmware
- Classified as local/remote, read/write
- Attributed to CPU, procedure, data structure
- Visualization
- Stacked bar charts showing miss counts
- Collection of UI controls for configuring the
display
22Thor Discussion
- Both post-mortem and real-time analysis
- Live connection to FLASH via socket
- Useful for multi-phase applications
- Simple, familiar visualization
- Interactive filtering, sorting, aggregation
- Drill down from overview to data of interest
23PipeCleaner Superscalar Processors
24PipeCleaner Background
- High peak performance
- Complex implementation techniques
- Speculation, multiple functional units,
out-of-order execution - Intended to be transparent to the programmer
- True for correctness, not necessarily for
performance - Data sources MXS and MMIX simulators
- Detailed superscalar processor pipeline models
- Visualization three linked views
- Overview occupancy strip charts
- Detail animated pipeline display
- Context program source code
25PipeCleaner Discussion
- Applications of PipeCleaner
- Program development
- Compiler optimizations
- Hardware design
- Education
- Simulator development
- Possible extensions
- Other computer pipelines graphics hardware
- Physical pipelines assembly lines, etc.
26Visible ComputerSystem and Cluster Monitoring
27Visible Computer Background
- Real-time analysis of system behavior
- Observe behavior of system in its entirety
- Explore interesting phenomena in detail
- Data sources PCP, SimOS
- Comprehensive data sources
- Collect low-level hardware events
- Cache misses, disk requests, CPU utilization,
etc. - Classify using high-level structures
- Process name, user/group, CPU mode, etc.
28Visible Computer Visualization
- Organizes data using nested physical hierarchy
- Provides overviews at each level of detail
- Active icons representing system components
- User defines data ranges of interest for each
component - Icons activate when components are in range
- Allows users to remove the cover, show next
level - Displays detailed charts on demand
- Data filtered/colored using high-level classifiers
29Visible Computer Discussion
- Unified interface for computer systems data
- Focus-plus-context view of system
- Physical layout, virtual classification
- Provides an overview of system behavior
- Draws attention to potential problem areas
- Suggests targets for further exploration
30Outline
- Motivation
- The Rivet visualization environment
- Architecture
- Implementation
- Focused visualization systems
- SUIF Explorer
- Thor
- PipeCleaner
- Visible Computer
- Ad hoc analysis and visualization
- Contributions
31Ad hoc Analysis and Visualization
- SimOS Complete machine simulator
- Full non-intrusive access to HW, OS, SW state
- Flexible data collection mechanism
- Deterministic execution
- Combine with Rivet
- Flexible data import mechanism
- Rapid prototyping of visualizations
- Result powerful analysis framework
32Simulation Visualization Cycle
Change HW model or SW
Configure simulated machine and software
Change annotations
Perform simulation
Change visualization
Visualize results
33Case Study Argus
- Parallel, multithreaded graphics library
- NURBS rendering application
- Implemented using fork shared memory region
- Performance limitations on SGI Origin
- Linear speedup up to 26 processors
- Rapid performance falloff beyond 26 CPUs
- Imported into SimOS environment
- Same performance characteristics observed
- Expectation memory system is the problem
34Memory Visualization
35Memory Visualization
1. Histograms of memory stall time vs.
physical/virtual address
36Memory Visualization
2. Memory stall time incurred by each line of
source code
37Memory Visualization
3. Local/remote stall time and idle time for each
process
38Memory Visualization
Very little memory stall time in process view
39Visualization of Process Data
40Process Scheduling Kernel Locks
41Process Scheduling Kernel Locks
42Results With Processes Pinned
43Results Using sproc
- Minimal kernel time
- No idle time at all
- Completes nearly twice as fast as original
version - 95 parallel efficiency
44Argus Conclusion
- Five simulation visualization iterations
- Initial suspicions were totally incorrect
- Flexibility of Rivet SimOS enabled us to follow
leads - Visualizations led us to the bottleneck
- Single scheduling event out of over 50,000 events
45Contributions
- Rivet computer systems visualization environment
- Rapid prototyping support
- Quick rough cut visualization of data
- Incremental development of sophisticated data
displays - Modular architecture
- Mix and match basic building blocks to create
visualizations - Enables coordination through object sharing
- Data transforms as first-class citizens in viz
environment - Unifies the analysis and visualization process
- Collection of computer systems visualizations
- Emphasize interactive data exploration
- Use relatively conventional visual
representations - Provide extensive support for filtering, sorting,
brushing
46Acknowledgments
- Orals committee Joel Ferziger, Mendel Rosenblum,
Pat Hanrahan, Mark Horowitz, Monica Lam - Mendel
- Visualization collaborators
- PipeCleaner Donald Knuth
- Visible Computer John Gerth
- Argus Gordon Stoll
- Thor Jeff Gibson
- SUIF Explorer Shih-Wei Liao
47Acknowledgments
- Groupmates Rivet, SimOS, FLASH, Graphics
- Studio 354
- Steve Herrod and John Heinlein
- The foosball table
- Chris Stolte and Diane Tang
- Staff
- John, Charlie, Thoi, Kevin
- Ada, Heather, Chris,
- Funding agencies ONR, D?ARPA, ASCI
- Robert Bosch Corporation
48Acknowledgments
- Rains 7A Steve, Hoa, Marco
- Friends
- Family
- Ming