Advanced Media-oriented Systems Research at Georgia Tech: A Retrospective

1
Advanced Media-oriented Systems Research at
Georgia Tech A Retrospective

• September 1999 August 2005

Umakishore Ramachandran, Karsten Schwan, Phillip
Hutto, Matthew Wolf College of Computing, Georgia
Tech
A presentation for the NSF CISE/CNS Pervasive
Computing Infrastructure Experience
Workshop Urbana, Illinois ? July 27, 2005
2
Outline

• Research vision
• Evolution of goals
• Infrastructure capture, access, interpretation
• Theme experimental or production facility?
• Lessons learned the devil in the details
• Outcomes integrating big and small
• Conclusions

3
Advanced Media-oriented Systems Research
Ubiquitous Capture, Access, and Interpretation

• Faculty involved with RI-related projects
• Kishore Ramachandran, Mustaque Ahamad, Karsten
  Schwan, Richard Fujimoto, Ken Mackenzie, Sudha
  Yalamanchili, Irfan Essa, Jim Rehg, Gregory
  Abowd, Yannis Smaragdakis, Santosh Pande, Ramesh
  Jain,Calton Pu, Ling Liu, Thad Starner...
• Federal funding
• NSF RI, NSF ITR, DARPA, DOE
• State funding
• Yamacraw, GT Broadband Institute
• Industry funding (equipment and personnel)
• HP, Intel, Microsoft, IBM

4
(No Transcript)
5
Original Research Statement

• Using two large-scale research applicationsa
  distributed education repository, and perceptual
  computational spaces for multimedia-based
  collaborationas drivers, we propose to carry out
  extensive systems research and integration to
  support ubiquitous access, capture, and
  interpretation of a variety of multimedia
  systems.
• -- Grant proposal executive summary 99

ubiquitous access, capture, and interpretation
perceptual computational spaces
extensive systems research and integration
distributed education repository
multimedia streams
6
What We Anticipated

• Demand for rich access of complex data/media
  streams
• Ubiquitous computational resources (sensors to
  servers)
• Ubiquitous connectivity
• high-speed Internet, wireless, Bluetooth
• Emerging application class
• multiple media streams and data sources
• real-time coordination
• fusion, correlation, sampling, feature
  extraction
• Aim end-to-end analysis of required
  infrastructure
• driven by experience with real-world
  applications

7
Confirmation of Vision

• Recent terrorist attacks
• Emerging sensor network applications
• Rich home media networks
• Increasingly sophisticated cell phone apps

8
Confirmation of Vision
9
Evolution of Research Goals

• Change in driver applications
• Education repository -gt Aware Home, Event Web, TV
  Watcher, Surveillance,
• Change in personnel
• Atkeson, Chervenak -gt many participants
• Changes in research interests
• e.g. Increasing importance of sensor networks
• Changes based on research experiences
• e.g. Increasing importance of middleware,
  plumbing
• Changes due to emerging technologies
• Grid, sensors, virtualization

• Change is
• Good!

10
Remaining True to Original Intent

• Many infrastructure components
• DFuse, MediaBroker, Energy Aware Traffic Shaping
  and Transcoding, Stream Scheduling, Agile Store,
  Differential Data Protection
• involving many different driver applications
• Aware Home, Smart Spaces, High Performance
  Computing Program Steering, Event Web, TV
  Watcher, Vehicle-to-Vehicle Networks
• and many application domain collaborators
• Abowd, Essa, Fujimoto, Jain, Rehg, Starner

• On Course!

11
Infrastructure

• Compute servers
• Clusters
• Networking enhancements
• Storage
• Display facilities
• Conferencing facilities
• Sensor technologies

12
Infrastructure Overview
Compute Servers
Systems Studio
Sensor Lab
13
Infrastructure Systems Studio

• Multi-use experiential smart space
• Video wall with touchpad control
• Immersadesk with SGI display engine
• Sophisticated control center/interconnect
• Projectors, whiteboard
• Various cameras
• Various microphones speakers
• Conference table
• Various workstations, servers
• Sensor lab elements (location tracking, motes,
  temp sensors, robot, marquee sign, mobile
  devices, RFID)

14
Infrastructure Experiential Conference Room

• Smaller, more up-to-date version of Systems
  Studio
• Dual-use for production meetings as well as
  research targeted to augmented conferencing
• Experiential meeting room - Jain

15
Theme Research/Commodity Tension

• We believe a natural tension arises over the
  proper use of facilities funded by long-lived,
  equipment-based grants seeking to involve diverse
  personnel (such as RI grants), particularly for
  smart spaces and pervasive computing
  infrastructure
• Various forces conspire to treat experimental
  research facilities as commodity or
  production facilities
• The resulting tension has far-reaching
  consequences

16
Theme Research/Commodity Tension

• Research facilities
• must be flexible
• undergo frequent transformation, reconfiguration
• include new equipment that must be explored
• host experimental software
• in short, research facilities are often broken
• Production facilities
• must be accessible, friendly, easy to use,
  available, reliable
• in short, production facilities must just work

17
Theme Research/Commodity Tension

• To encourage wide use by various researchers,
  research facilities must be friendly, easy to
  use, available that is, they must have qualities
  of production facilities
• In addition, smart research spaces need users
  to evaluate effectiveness
• Finally, smart spaces, when successful
  implemented, encourage production use (by their
  very usefulness!)
• But production use of research facilities
  hinders research usage catch-22

18
Lessons Learned

1. Plan ahead, get buy in, put it in writing
2. Experimental wireless research is difficult
3. Use professional services where appropriate
4. Knowledge transfer is difficult
5. Avoid the urge to buy cool stuff
6. Plan on upgrading long-lived facilities

• D'Oh!

19
Plan Ahead

• Plan ahead, get buy in, get it in writing
• System Studio originally imagined as dedicated
  research facility
• Almost immediate pressure for public use (e.g.
  general use workstations, regular meetings, etc.)
• Some hazy memories about original agreements as
  administration personnel changed over the years

20
Wireless Research is Difficult

• Research wireless infrastructure increasingly
  conflicts with production wireless
• Research nets viewed as rogues interfering with
  production wireless
• Constraints on what we could do with campus
  wireless
• E.g. detection of local machines by wireless
  clients for cyber foraging application
• Solutions?
• Wireless isolation of research facility?
• Negotiate more flexibility with campus wireless?
• Special off-sight wireless lab?

21
Use Professional Services

• Use professional services where appropriate, even
  if researchers/students can do the work
• Case in point audio infrastructure in
  conferencing facilities
• E.g. Echo cancellation algorithms
• Sometimes doing it yourself has benefits
  often just a distraction

22
Knowledge Transfer is Difficult

• Knowledge transfer of detailed equipment usage
  information over time is difficult
• We often worked on certain equipment and then
  moved on
• Later, other researchers wanted to use the same
  equipment and had to go through a time consuming
  spin up on how to use equipment
• In the worst case, students that previously
  worked with equipment were gone, with much
  knowledge lost
• Solutions
• Documentation, build simplifying utilities
• Research scientists effectively maintained
  institutional knowledge about equipment

23
Avoid the Urge to Buy Cool Stuff

• Avoid the urge to buy interesting new equipment
  without a clearly defined use avoid the Well
  figure it out later syndrome
• Carefully weigh cost/benefit of all proposed
  equipment acquisitions
• Equipment should either be easy to use out of the
  box or have a clearly defined, high-priority role
  in ongoing research (or be made serviceable by
  professionals)
• Without, equipment will be under-utilized
• We experienced this with some switching
  infrastructure in the Systems Studio and with the
  Immersadesk to some extent

24
Plan on Upgrading

• Plan on upgrading facilities that will be in use
  for more than three years
• We staged equipment purchases over the lifetime
  of the grant which reduced the need for upgrade
• Still we needed to refresh some of the earliest
  purchases for continued use
• We upgraded (and enhanced) Systems Studio
  infrastructure during the 5th grant year
• We also upgraded early cluster purchases
• SGI display engine reached its end-of-life during
  the grant and was not replaced

• New Lamps
• for Old

25
Synergistic Research OutcomesIntegrating Big
and Small

• Application-driven research has significantly
  enhanced interaction among systems and
  application-domain researchers
• Collaborations most vigorous when funded by
  related grants
• Collaboration often facilitated by sharing grad
  students
• Synergy of equipment/researchers helped clarify
  integration of big and small in service of
  pervasive applications with computationally
  intensive demands (e.g. video analysis)

26
Research Nuggets

• MediaBroker
• Clearing house for sensors and actuators in a
  pervasive computing environment
• DFuse
• Data fusion architecture for futuristic sensor
  networks
• Streamline
• Scheduling heuristic for stream-based
  applications on the grid

27
Research Nuggets (contd.)

• Dynamic energy aware transcoding of media streams
• Changing application needs
• Changing resource availability
• Dynamic differential data protection
• Streaming in public networks to user devices

28
Summary

• Reviewed six year history (9/99-8/05) of advanced
  media-oriented systems research at Georgia Tech
• Discussed original goals, evolution and
  adaptation
• Summarized facilities
• Highlighted fundamental tension between research
  and production use of such facilities
• Offered lessons learned
• Characterized synergistic research outcomes

29
Questions?

• Applause!!!
• ?

30
Systems Studio
31
Systems Lab
32
DFuse ACM SenSys 2003

• An architecture for Infrastructure Adaptation
• A sample scenario for an aware environment
• Field trip for a class!
• Deployed power-constrained sensors
• Dynamic wireless network consisting of the
  students PDAs
• In-network stream filtering and aggregation

33
DFuse Fundamentals

• Fusion Module Deploys task graph on sensor
  network
• Comprehensive API for fusion and migration
• Low-overhead
• Placement Module Employs a self-stabilizing
  algorithm to place fusion points in the network
• Energy and application aware cost functions
• Localized decisions

34
DFuse EvaluationApplication Timeline showing
Network Traffic
35
Streamline

• Ubiquitous and dynamic nature of streaming
  applications require distributed HPC resources,
  possibly spanning administrative domains
• Goal Develop middleware-infrastructure that
  provides access to ambient HPC resources for
  performing compute-intensive tasks of streaming
  applications
• Grid Computing
• Provide access to ambient HPC resources
• Focused on scientific and engineering
  applications
• Some recent efforts for Interactive and Streaming
  applications (Interactive Grid HP , GATES
  OSU)
• Existing infrastructure primarily for
  batch-oriented applications

36
Streaming Grid
37
Streamline Scheduling
38
Streamline Scheduling Heuristic
39
Streamline Results

• Platform
• Scheduling heuristic, Streamline, using Globus
  Toolkit
• Baseline grid scheduler for streaming apps using
  Condor
• Approximation algorithm using Simulated Annealing
  for comparison as an Optimal (although infeasible
  in practice)
• Results
• Streamline outperforms baseline by an order of
  magnitude for both compute and communication
  bound kernels, particularly under non-uniform
  load conditions
• Streamline performs close to Simulated Annealing
  with very low scheduling overhead (by a factor of
  1000)
• "Streaming Grid A Proposal for Grid Middleware
  Services Supporting Streaming Applications
  Bikash Agarwalla  - 2nd place winner of 2005 IBM
  North America Grid Scholars Challenge
• Streamline A Scheduling Heuristic for Streaming
  Applications on the Grid - Bikash Agarwalla,
  Nova Ahmed, David Hilley, Umakishore Ramachandran
  - Under Review

40
Energy-Aware Transcoding

• Wireless multimedia
• faster processors, high-speed wireless links
• energy is the constraining resource!
• But energy is different
• can be finite (battery), non-replenishable, out
  of energy, app over
• can be associated with costs (server cluster,
  heat dissipation, cooling)
• tightly coupled with other resources
  (utility-cost model)
• add CPU, network, disk, memory resources
  increased utility
• add energy increased cost (reduced mission time,
  increased expenses, increased heat dissemination,
  cooling)

41
Application Level Media Transcoding

• Transcoders bring data from one form into
  another reduce size, reduce color, compress, ...
• Some are mandatory, some are optional

42
Transcoder Characteristics

• Transcoder characteristics
• Relationship input data size output data size
  (rd)
• Relationship input data size transcoder
  run-time (rr)
• Use rd,rr, Kd(n), and Kr(n) to predict potential
  energy savings
• Compare transcoders for a given frame

43
Transcoder Evaluation
44
Energy Aware Transcoding Discussion

• Transcoder characteristics can be obtained
  offline and online
• Power model obtained online, future devices may
  have capabilities for determining power model
  online or will have it part of the architecture
• Experiments
• run-time predictions vary about 5-7 from
  measured numbers
• data size predictions vary less than 1
• energy predictions vary about 5-8.5
• Overhead O(tp)
• number of transcoders t (low, e.g., 1-10)
• number of parameters p (typically set to Uij(min)
  to minimize energy)
• Online transcoder evaluation
• add new transcoders
• data content changes
• update transcoder characteristics with measured
  results

45
Dynamic Differential Data Protection D3P
46
PBIO/D3P Implementation

• High-performance binary wire formats
• Reflection, component-wise access
• PBIO objects ? capabilities
• crypto, object-specific rights, payloads

publisher (type 640x480 image)
47
Active Video Streams
48
D3P Performance Win