Visual Lexicons: The Quest for Data Driven Decision Making

1
Visual Lexicons The Quest for Data-
Driven Decision Making
(Here, let me show you what I mean . . . )

• Charles H. House
• Director
• Research Collaboratory
• Intel Corporation

MLMI 2nd Annual Conference
2
Visual Lexicons are one topic Effective
Communication is the goal

• I applaud researchers at this conference for your
  bold investigations of meetings
• I will argue for Visual Lexicon inclusion in such
  meetings, using dramatically better
  Computer-aided tools
• A more fundamental point is that meetings are
  undergoing radical character change
• Understanding more about the character of
  meetings (today) and the pressures for change
  could prove enormously fruitful

3
Communications means many things

• so pervasive that it is seldom taught or
  studied in a holistic fashion
• Communications schools are usually Journalism
  or Creative Writing (the literary side), or
  Advertising or Film-making (the graphic arts
  side), or else MEDIA-heavy
• Political Science and Sociology programs usually
  emphasize the Leadership and Crowd Behavior side
• Communications technologies usually include
  Signal Processing, Network Queuing, Switching
  Systems, Topology, or Bit Rate / Bandwidth /
  Frequency Spectrum studies -- these are EE and CS
  depts

4
Enhanced Communications

• Effective Communication is the basis of most
  ORGANIZED ACTIVITY, and is ESPECIALLY CRITICAL TO
  MODERN BUSINESS
• There has been but modest study of the specific
  character of effective Business communications
  methodologies from the holistic point of view
• Small focused laboratories, seeking to answer the
  right questions, have produced great results

• Digital Communications Technologies radically
  impact our ABILITY and METHODOLOGIES to
  communicate
• Both some GREAT OPPORTUNITIES and some
  SIGNIFICANT CHALLENGES await solution

5
This research requires a multi-disciplinary
approach

• Just as XeroxPARC tried to understand the flow
  of paper in an office, and used cultural
  anthropologists and linguists to develop the
  Office Document metaphor,
• and THEN they used Computer Scientists to create
  better solutions
• we believe that the various threads of
  Communications, from Personal abilities and
  perceptions, to Institutional opportunities, to
  Societal technological solutions -- all merit
  serious inter-acting contextual study

6
How do we learn about the future? 99

• By ALREADY HAVING LIVED IT
• according to Bill Buxton of Alias Research
  you cant imagine the future while living in the
  past
• according to Laurence Wilkinson (GBN) If
  you think things are REALLY CHANGING
  . Write new rules . Invest in
  learning new rules Invest in companies who use
  the new rules
• according to Chuck House (HP) I never
  thought of it as insubordination

7
Structured meetings at Intel

• 4 million hrs of Classroom instruction / yr
• 3 million hrs of scheduled meeting rms / yr
• 5.7 million Audio Bridge Conferences held / yr
• 56,000 hrs Effective Meetings taught / yr

We have sizable operations in 243 centers in 22
states, 49 nations, and 5 continents 88,000
Blue Badge employees, 145,000 Green Badge
contractor employees, and 270,000 vendor/client
inter-actors (Firewall perimeter?) EFFECTIVE
COMMUNICATION is VITAL
8
Meeting Effectiveness presumes . . .
ALIGNMENT?
LEADER

• Correct
• Goals
• Issues
• Assignments
• Momentum

Other things Horn WICKED, ILL-STRUCTURED
PROBLEMS
9
Communities in Centralized vs Distributed
Development
Our Lab
Issues A. Does each Lab 1. know its role? 2.
agree to its role? B. The Program is 1.
Designed where? 2. Managed where? C.
Communication is 1. Structured 2. Ad hoc
10
Purpose of collaboration

• Similar to all other communications
• To share information with others
• To co-solve problems
• To build trust, camaraderie
• Different than many communications
• Problem-solving is paramount
• TEAM SOLUTION is most effective
• Implicit inhibitors are the most insidious
• NASCENT SKILLS
• Team decision-making
• Data-driven decision making
• Virtual teams have many unnatural inhibitors

11
Many of you might know the Edward Tufte
books Envisioning Information Horns work is far
more useful for most business applications
12
Many, nay MOST, employees cite DATA OVERLOAD as
their 1st or 2nd biggest problem
13
Wicked, ill-structured Problems Abound for Teams
SELDOM CITED, however
14
A TRULY EFFECTIVE Colab capability will deal
strongly with this class of issues
These are the true hobblers of effectiveness, not
speed of Spreadsheet transfer
15
Intel Research Collaboratory

• New idea at Intel more experimentalists than
  true researchers
• Born of frustration w company IT backbone
• Loose federation of individuals
• Created a Virtuality Index, identified some key
  problems, and created a Concept Car for the
  company with a Flash Demo
• Obtained some (modest) funding
• Underway a few months now

16
Virtuality Index What we found

• Trend toward being distributed
• Across locations, time zone, business units,
  cultural diversity and different ways of working
  GROWING
• 75 of Intel folk work on Multi-site teams
  weekly
• Multi-teaming 2/3 of workforce on 3 projects
  simultaneously
• Overall a 2/3 virtual organization and trending
• Constant adaptation
• 20 are NOT co-located with their supervisor
• Good corporate culture holding up
• highest rated value GREAT PLACE TO WORK
• lowest rated values communication timeliness
  across distance
• Some problems with
• Different software tools
• Different cultures

BIGGEST PROBLEM is that no COLLABORATION TOOLS
EXIST that deal with MY REALITY
17
A Big Thought Problem Statement

• No team can be presumed to be co-located
• Multiple team membership must be presumed
• Data-Driven Decision Making must be enhanced
• Turbulent environment, esp. re Investments and re
  Politically correct must be presumed
• Infrastructure gap must be presumed
• 
  

Effective Virtual Collaboration capability is the
single biggest Innovation and Productivity
enhancer for 21st century workforces ? BETTER
THAN BEING THERE
18
Better than Being There?

• Collaboration Tools (especially ROI analyses) are
  often defended on two bases
• Saving of Travel
• Saving of Travel Time

• Seldom defended on basis that its TRULY BETTER
• 30 years experimentation with Stanford ? BETTER
  GRADES (by a WHOLE LOT) Gibbons, JSB, House
• 2 years experimentation at Dialogic ? BETTER
  DECISIONS (by a WHOLE LOT) House
• 4 years study at Intel ? MORE EFFECTIVE
  ORGANIZATION (by a WHOLE LOT) Bless / Wynn

19
Factors in Distance Learning
30 year experiment with 15,000 students ½
million student hours
Stanford Instructional Television Network
This bothered the Dean
This irked the professors
This angered the on-campus students
This worried the administration
Faculty revolution overthrew this experiment 4
years ago the administration killed the golden
goose with class size and class
support issues
20
What does the Stanford example tell us?
Virtual Meeting structures

• Semi-synchronous interaction is distracting
• Question/Answer environment crucial
• Videotape allowed replay archived event
  

• Demands zero latency or a queuing mechanism
• Interaction with others produced better outcome
• A Replay of a Staff Meeting has enormous
  benefit

Theres a message here for managers who want to
build teams And companies who want to
facilitate better communications
21
The HP Halo Collaboration Studio
Video Presence
An immersive collaboration environment
that provides the customer with a
connection experience with people in remote
locations that is better than being
there. It is a simple, walk-in ready
experience that dramatically changes
how people work w/others in distant
locations. It
literally improves how people work -
more frequent involvement - greater
effectiveness - multiple places at one
day - higher productivity - reduced
travel - better time mgmt personal
efficiency - lower travel costs
The growing HVEN network enables expanded
communications with a range of external
contacts and, in turn, increases
the value of the solution.
Not announced yet, but see Thomas
Friedman, The World is Flat
Undetectable LATENCY, incredible
REALISM, can do six simultaneous conversations
We are jointly exploring with HP re
converged products for Halo
22
Video Presence
Our hope is to do research with Users of
these stations
First room is 600,000 (gulp) Volume
discounts are available
23
Now, imagine one staff team in a meeting
where the meeting is distributed
The meeting is (A) status reports 11 vs. 9 B
occasionally happen in parallel fewer, but more
likely, since 5 of 11 A reports are via phone
C, the of possible conversational
relationships during a full meeting goes down
dramatically with almost half the group on the
phone D is the number of parallel visual/body
language cues possible at one time
A. Number of primal 1-on-1 conversations
(n-1) 11
B. Number of easy 1-on-1 conversations in
background 3 or 4
C D. Number of 1-on-1 communications able to be
conducted nlocal(nlocal-1) 42
Better with dual-remote steerable cameras and
HDTV
24
What happens for the remote attendee?
During 90 of the meeting, listening to
presentations

• Irritation, since you typically can only hear
  audio but dont have access to handouts or to
  projected slides
• Frustration, since the audio is terrible for most
  presenters
• Rage, since you cannot interrupt due to
  Half-duplex lines or High Latency

The result RESENTMENT, ANGER, then INATTENTION
During 10 of the meeting, when presenting

• Frustration, since you cannot see the
  body-language of the listeners and tune your
  presentation to meet the audience need
• Irritation, when you hear murmuring in the
  background, without a clue who is saying what,
  but clear that sidebar conversations are going on
• Rage, if you have to request something like
  support or funding, since you cant make an
  impact coming from the remote site

The result DISEMPOWERMENT
During the meeting, a tremendous feeling of
ALONE-NESS, of being a SPECTATOR in a
PARTICIPATIVE SPORT
25
CHANGING THE DYNAMICS OF A NASDAQ 100 COMPANY
STAFF with WebeX, Full Duplex Confercg
Phones
Percentage Change (Yr 2/Yr1) for Folk based
in ?
Remote
HQ
Questioning speaker while remote
130
148
26
PITAC1 Vision -- IT Transforming our Society

• Transforming the Way We Communicate
• Transforming the Way We Deal w Info
• Transforming the Way We Learn

And we know about Cell Phones, IM, and eLearning
1 Presidential Info Technology Advisory
Committee, Feb 99
27
What are the important Research issues? 1

• Software -- Nation (and the world) needs FAR more
  usable, reliable, powerful SW
• Scalable Information Infrastructure -- Learn how
  to build and use large, complex, highly-reliable
  and secure systems
• High-End computing -- drive research by trying to
  attain sustained petaflops on Real Applications
  by 2010
• Socio-economic impact IT research on
  socio-economic and policy issues. Accelerate and
  expand education in IT at all levels (incl
  President of USA)

1 Presidential Info Technology Advisory
Committee, Feb 99
28
PITAC got it right mostly, except . . .

• Collaboration to labor with the enemy was
  understated, altho the group had earlier coined
  the word Collaboratory
• Rich data acquisition, coupled with significant
  mathematics both at the node and topologically,
  is the key to much system understanding
• We lack a visual lexicon, or lingua franca, to
  represent this rich data
• PITAC research is unfunded

29
What might be done?
Virtual Meeting structures
To answer that question, we need to consider four
areas

• The technology of creating presence
• The technology of creating archives
• The act of creating context
• The skills and training of participants/leaders

• Audio and video systems, network bandwidth, room
  arrangement, shared whiteboards can all be
  invoked to increasing degree. FOR ASYNC, trust,
  participation, acknowledgment, affirmation,
  regularity are all key
• Audio and video-taped recordings, archived
  handouts and slide presentations, and meeting
  notes. FOR ASYNC, ARCHIVAL FILES, esp. with
  auto-indexed audio/video tracks and text word
  structures is imperative
• The meeting moderator or support aide appends
  related material from group input or other
  sources, including URLs and archive files. KEY
  to INCLUDE mathematical manipulators, dynamic
  graphics, et al
• Multi-year research and experience with Virtual
  Teams has resulted in books, seminars, and
  training courses teaching the rules of Small
  Group and Team Communication with special Virtual
  attention

Even BOB KAHN said that HPs new Project HALO CBD
NONE TEACH VISUAL LEXICONS todays CALCULUS
30
Three simple wishes

• If only . . . the Computer Science community
  could work more closely with the Social Science
  community
• If only . . . the AI / IA community could work
  more closely with the CSCW community
• If only . . . users could be more rational

31
CS and Social Science

• Internet II Sociotech Conference 9/99 UMA
• Bring one each from CS / SS (100 Univ)
• Who knew whom?
• CS folk could design and build something REALLY
  useful
• But it wouldnt generate new PhD theses
• And it would have to WORK
• SS cannot even define, let alone design, a really
  interesting system
• So they study IM, and Democracy via e-Mail
• And they get excited about yesterdays Video
  Conferencing technologies

Oh, let me tell you now
Ask me later about Project ARGUS
32
When DID scientists build anything?

• Radar / sonar / oscilloscopes in WWII
• Transistors at Bell Labs
• ARPAnet
• Berkeley UNIX
• XEROX Star

33
Best product of the year

• TCP/IP (Kahn, Cerf)
• In 1982 . . . BPY

In 1997, at ACM 1, Cerf Internet happened so
fast, if you think 35 years is fast
34
For IEEE 100th Anniversary

• I preceded the Dean of EE at MIT, each to give a
  pithy ten minute talk on Information Transfer
• He held a hand-scribbled set of notes
• I did as well, for the first 2½ minutes,
• followed by a 2½ minute overhead set of graphs
• followed by a 2½ minute set of 35mm pictures
• followed by a 2½ minute videotape
• He wasnt very happy, walking on stage
• Lets do a simple simulation of that right now

35
Youre a geographer/sociologist, really a 20th
century urbanologist

• You want to describe to your students the
  shifting tide of American cities away from the
  East Coast
• You describe the decay of the inner city, and
  the rise of the new Western city
• You focus on the difference in cities designed
  for horse and buggy vs. those built around the
  auto
• You build a word-image of Detroit as mecca, and
  then bitterness, for automotive industry jobs

And the students yawn, and look at the ceiling .
. .
Worse, some are IMing suddenly one points at
you, and seven students break into hysterical
laughter
36
Armed with some graphing skills, though, you show
them some PowerPoint / Excel graphs of 50 years
at Americas largest cities
Some, fabled in American lore, are pretty
constant in size
This is an exceptional Geographer /
Sociologist Most are not adept with Logarithmic
calculations (nor are they much good with Kalman
filters, Bayesian networks, or Markov chains)
Baltimore was 2nd, Boston 3rd in 1850
St. Louis, Gateway to the West was Americas
4th largest city in 1900 only 49th a century
later
37
And THEN, armed with a cool 4-D plotting
package, you show them a multi-variate dynamic
graph (which this really isnt . . . )of the
population of seventeen cities in 1950
38
And THEN, you overlay a multi-variate dynamic
graph (which this really isnt . . . ) of the
population of the same seventeen cities in 2000
39
We communicate lots of thingsWe collaborate
about HARD PROBLEMS
Edward Tuftes famous books ? Envisioning
Information, or this one, Visual Display of
Quantitative Data e.g. Comparative Colon Cancers
by Gender, Race, Locale, Employment
Applicable to Epidemiology Climatology
Mineral / Gas exploration Crime patterns
Infrastructure weaknesses
Difficulties are manifest Graphical
interpretive literacy Statistical data
selection Application packages missing
Sensor nets deployment Semantic net
understanding
40
Computer Generated Graphic images
The Visual Display of Quantitative
InformationEdward R. Tufte, 1983, Graphics
Press, Cheshire, CN
All types of cancer, white females, Age-adjusted
rate by county, 1950-1969
41
Heres another Graphic image
The Visual Display of Quantitative
InformationEdward R. Tufte, 1983, Graphics
Press, Cheshire, CN
Trachea, bronchus, and lung cancer white
females age-adjusted rate by county, 1950-1969
42
All types of cancer white females age-adjusted
rate by county, 1950-1969
These plots were shown on Page 17 in the First
Edition Printings 1-4
All types of cancer white males age-adjusted
rate by county, 1950-1969
43
All types of cancer white females age-adjusted
rate by county, 1950-1969
These plots were shown on Page 17 in the First
Edition from printing 5
All types of cancer white males age-adjusted
rate by county, 1950-1969
44
This data had been compiled for multiple
diseases With multi-variate discrimination ( gt 17
variables) And it has taken AMA 25 years to
BEGIN to learn how to use it
1500 more likely to die of emphysema if you
lived in a high mountain valley in 1940s-1960s.
AMA denied it
Trachea, bronchus, and lung cancer white
females age-adjusted rate by county, 1950-69
My assertion ? Images are compelling
We are NOT practiced in reading them
These plots are shown on Page 18 in the First
Edition from Printing 5 on
Trachea, bronchus, and lung cancer white males
age-adjusted rate by county, 1950-1969
45
Recent Advances in Vision
Finding, gathering, scaling, stitching together
similar images for automatic free-form panoramas
Images from M. Brown and D. G. Lowe. Recognising
Panoramas. In Proceedings of the 9th
International Conference on Computer Vision
(ICCV2003)
back
46
Video Security Application Landscape
26 million surveillance cameras have been
installed worldwide, 11 m US IBM hires 3000
consultants for customers to incorporate digital
video security into existing IT operations The
remote digital surveillance camera market is
growing 40-50/year
Back
47
Application Landscape Bioinformatics
THE RACE TO COMPUTERISE BIOLOGY DECEMBER 12TH
2002, Economist Bioinformatics In life-sciences
establishments around the world, the laboratory
rat is giving way to the computer mouse--as
computing joins forces with biology to create a
bioinformatics market that is expected to be
worth nearly 40 billion within three years
2500 more likely to die of a sudden mid-life
heart attack if you have both markers. AMA
denied it for 62 years
Models can be learned via structure learning.
My Brother, 2 years ago Low Body Fat, Mod
Cholesterol 1 hour Workout sessions 4
days/week Perfect EKGs
Dead fr Sudden Cardiac Arrest 95 blockage,
Coronary Artery 70 in other three
Homocysteine Markers Simple Blood Test Two
common Mutant Genes HE HAD BOTH THIS HAS A
SIMPLE FIX
48
Video is often a turn-off
Great Video can be compelling
Stored, malleable Video is awesome
ALL e-Learning Video Systems show the Teachers
face to the Students never the other way round
The power of video is evolving Talking Heads
are never compelling
N-way multicast provides great video imaging
with low-bandwidth ip v.6
Archiving compelling video enriches collaboration
How many of you have seen the dual skiers on
World Cup or Olympics TV?
http//www.dartfish.com
49
If a Picture 1000 words, whats a video stream
worth?
"If a picture is worth a thousand words then
Dartfish is worth a thousand pictures! I believe
that The Dartfish Motion Analysis Software
Program has completely redefined the future of
all athletic training environments.
It decreased the time required for our skiers to
internalize their skill levels and move rapidly
forward toward enhancing them. The time spent
with their coaches has become significantly more
productive and efficient. . . . simply placing
athletes in front of a television for 'video'
could be compared to coaching in the stone age.
Tony Nunnikhoven Steamboat Springs Winter
Sports Club, Alpine Program Dir.
85 of Americas 103 medal winners in the 2004
Summer Olympics trained with Dartfish SW
50
The point?

• Computer-augmented data EXISTS in copious
  quantities in field after field. It is often
  graphic, visual, dynamic. It is usually NOT
  understood.
• People, even well-intentioned smart educated
  leaders, dont avail of data very well for a wide
  variety of reasons politics, ignorance, apathy
• Systems could approach this problem 3 ways
• Designing answers in graphical/visual formats
• Providing intuitive training for erstwhile
  users
• Automating answers, removing people from data
• If people are to be included in the equation,
  graphical interpretive dataset presentation
  visual lexicons, if you will are fundamental

51
In conclusion

• Computer Science COULD consort with Social
  Scientists to build some very nice Test Beds
  this community, studying more effective Meetings,
  is a GREAT START
• AI / IA community has a LOT TO OFFER to the CSCW
  world itd be great if it works
• People may not become more rational, BUT they
  just might learn to use better data
• MORE OF THIS IS UP TO YOU THAN YOU MIGHT
  APPRECIATE. THE SOCIAL SCIENTISTS WILL NOT ASK
  YOU FOR IT