Situating Technology:

1
Situating Technology

• Creating and evaluating mobile technologies
  in-situ

Jennifer Mankoff
2
Scribe4Me
3
Scribe4Me

• Mobile system
• Provides text transcriptions of the last 30
  seconds of audio, upon user request

4
Scribe4Me Design
1) A participant needs sound info clicks
what happened? button.
5
Scribe4Me Design
2) The past 30 seconds of audio is sent to
the desktop transcription interface.
A person transcribes the audio and
sends it to the participant.
1) A participant needs sound info clicks
what happened? button.
6
Scribe4Me Design
2) The past 30 seconds of audio is sent to
the desktop transcription interface.
A person transcribes the audio and
sends it to the participant.
1) A participant needs sound info clicks
what happened? button.
3) The participant receives the
transcription as a text message.
7
Scribe4MeUses

• Interviews design exploration

8
Scribe4MeUses

• Interviews design exploration
• Situated deployment

9
Scribe4MeUses

• Interviews design exploration
• Situated deployment
• Overlap

10
Scribe4MeUses

• Interviews design exploration
• Situated deployment
• Overlap
• Situated use is CRITICAL

11
Methods we used

• Surveys
• Brainstorming
• Focus group
• Scenario-based
• Workshop
• Diary studies
• Probes
• Bodystorming

Proboscis 05
12
But where do these methods fail?

• Case study of diary studies
• Interviews with 28 developers
• Literature survey

13
Diary Studies

• Participants report on events
• Feedback or elicitation
• Form completion (feedback)
• Participant controlled capture (elicitation)

14
Diary Studies (CHI05)

• Improved with new recording devices
• but recordings of new media affect the process
  and outcome
• Studies show effects of different media

15
Method Findings

• Poor recallgt Need for rapid, situated
  annotationgt Need time-stamping
• Feedback to participants
• Out-of-sequence referencesgt Need automated
  organization

16
Interviews

• Nine mobile developers
• All with deployment experience
• Variety of methods used
• Logging
• Experience Sampling
• Diary studies

17
Interviews Results

• Successes
• Event-contingent sampling
• Logging
• Frustrations
• Adoption and retention
• Sparsity
• Unreliable feedback from users
• Prototype robustness

18
Challenges (IJHCI 2007)

• Ambiguity and error
• how would recognition errors impact Scribe4Me?

19
Challenges

• Ambiguity and error
• Sparse data
• Only used twice a day
• Two weeks barely got qualitative data

20
Challenges

• Ambiguity and error
• Sparse data
• Critical mass
• It needs to work everywhere
• Some apps need enough users too

21
Challenges

• Ambiguity and error
• Sparse data
• Critical mass
• Unobtrusiveness
• Interruptiveness
• Changing existing patterns (pagers)

22
Challenges

• Ambiguity and error
• Sparse data
• Critical mass
• Unobtrusiveness
• Rapid iteration in real environments
• WOz
• Remote data gathering

23
Momento

• Iteratively designed, usable system for remote
  evaluation of Ubicomp
• Lowers threshold for deployment
• Makes wizarding usable
• Integrates needfinding and later stage evaluation

24
Studies Types Supported

• Diary studies
• ESM studies
• Probes
• Rapid prototype iteration
• Wizard-of-Oz
• Automated

25
Application Areas

• Iterative design
• Mobile games and services
• Context sensitive displays and reminders
• Recognition-based services
• Human sensors
• Situated studies
• Time use
• Cellphone use
• Social network studies

26
Lowers Threshold for Deployment

• No client install required
• No coding required
• Robust over days/weeks

27
Makes Wizarding Usable

• Easy setup for wizarding

28
Makes Wizarding Usable

• Easy setup for wizarding
• Peripheral display for wizards

29
Makes Wizarding Usable

• Easy setup for wizarding
• Peripheral display for wizards
• Context-sensitive, real-time notifications
• Take action
• Watch user

30
Integrated Support

• Allows novel combinations (e.g., using
  participants as sensors via ESM)
• One system for prototyping and evaluation
• Consistent experience for wizards
• Consistent experience for users

31
What is an experiment?

• Participants use clients
• Experimenters configure, observe, analyze

32
Architecture
33
Architecture
34
Clients

• Fixed
• Context toolkit (Dey) widgets
• Two-way communication
• Mobile
• Standard messaging
• Configurable application
• Email

35
Mobile Client
Standard mobile messaging
Momento mobile
36
Mobile Client

• Text file configuration of...
• Study description
• Continuous capture
• Audio, photo, Bluetooth/GPS
• Buffered or sent automatically
• Buttons
• Manual entry of audio, photo, sketch, video,
  comment
• Send buffered data
• Always on or triggered
• Color

37
Desktop Interface

• Study design
• Participants
• ID
• Phone info
• Groups
• Places
• BT ID
• GPS info
• Rules

38
Rules
o originator of message g group
39
Desktop Interface

• Monitor incoming events

40
Desktop Interface
Respond to events
41
Server

• Storage for studies
• Similar to versioning systems (e.g., CVS)
• Networking
• Gateway to mobile clients
• Context Toolkit discovery
• Web interface
• Participant or experimenter can review
• captured data

42
Data Flow
clients
clients
43
Iterating a Field Study
Momento

• Goal Support lightweight communication between
  groups of friends

44
Iterating a Field Study
Momento

• Add some participants
• Diary what they are doing via text messages
• Review data

45
Iterating a Field Study
Momento

• Add some participants
• Create and populate groups
• Configure rule to forward text messages to other
  group members

46
Iterating a field study

1. Add some participants
2. Create and populate groups
3. Configure rule to forward text messages to other
   group members

47
Iterating a field study

1. Add some participants
2. Create and populate groups
3. Configure a rule to forward picture messages to
   other group members

48
Iterating a field study

1. Add some participants
2. Create and populate groups
3. Configure a rule to forward picture messages to
   other group members
4. WoZ voice commands

49
Iterating a field study

1. Add some participants
2. Create and populate groups
3. Configure a rule to forward picture messages to
   nearby group members

50
Related Work
Prototyping
Report methods
ContextESM (Intille 03)
Topiary (Li 04)
rapid iteration
rapid iteration
use infrastructure
use infrastructure
multiple devices
multiple devices
self-report data
self-report data
other work
other work
SUEDE (Klemmer 00)
Me Tool (Intel 06)
DART (Dow 05)
iESP (Intel 03)
Doorman (Makela 01)
Mobile probes (Hulkko, 04)
Crossweaver (Sinha 03)
Active Capture (Chang 05)
51
Validation

• Usable/lowers floor
• Developing Scribe4Me 3 configuration actions
• Testing Scribe4Me 3 configuration actions
• Successfully supports evaluation
• 4 studies
• 2 weeks to 3 months in length
• Varying levels of involvement by our team

52
Studies
Informal learning
Scribe4Me
12 users 7 days
6 users 1-2 weeks
ESM study
Photo Sketch
24 users 1 day
14 users 2 months
53
Studies
Informal learning
Scribe4Me
ESM study
Photo Sketch
Tacit experience Unobtrusiveness Critical mass
54
Photosketch

• Augment learning activity for children
• Workshop
• Find something with moving parts.
• Photograph it.
• Show how the parts move by drawing arrows on the
  photo.

55
Photosketch

• Groups of 3-4
• Experimenters, teachers mentoring
• Post hoc review
• Facilitated curriculum design

56
Supporting Experimentation

• Sparse data
• Feedback
• ESM study, digital youth
• Critical mass
• Control coordination interface
• ESM study, digital youth, photosketch

• Unobtrusiveness
• Everyday devices, self report
• Digital youth, ESM study
• Rapid iteration in real environments
• Configurable Apps
• Scribe4Me, Photosketch

57
Conclusions

• Need better understandingto push forward
  iterative design, and therefore to push forward
  the field.
• Important at all stages of design
• Situated data is important
• Momento supports all this and more

58
Other work

• Understanding situational impairment
• How do different kinds of movement affect PDA
  use?
• Energy efficiency sustainability
• What UI-level choices could reduce energy use?
• Automation for accessibility
• Auto sensing capabilities
• General techniques for adaptation
• Example magnetic click dust

59
Thanks!

• jmankoff_at_cs.cmu.edu
• Work done in collaboration with Scott Carter (PhD
  Work) and other students and faculty
• Supported by NSF

60
Momento Support for Situated Ubicomp
Experimentation
Other possible applications
social    proximity       jabberwocky (paulos
04)   location       urban tapestry (lane
05)   activity
resource queries    transit services   tour
guide (abowd 97)
61
Momento Support for Situated Ubicomp
Experimentation
Other possible applications
social    proximity       jabberwocky (paulos
04)   location       urban tapestry (lane
05)   activity
capture and access    field data        health
(alivetech 05)    reminder apps
gaming    cysmn (blast theory 01)  
location,proximity-based
resource queries    transit services   tour
guide (abowd 97)
62
Participant view

• SMS/MMS
• OR
• Custom application

63
Validation 3 studies

• Pure diary study
• conducted by others
• two weeks long

64
Validation 3 studies

• Pure diary study
• it worked
• 1-click cameras won
• immediate review great

65
Validation 3 studies

• Pure diary study
• Early prototype
• Lasted a semester
• Ourselves
• ESM sensing

66
Validation 3 studies

• Pure diary study
• Early prototype
• Application very flawed
• Bluetooth sensors hard to control
• New devices/late adoptors big problems

67
Validation 3 studies

• Pure diary study
• Early prototype
• Scribe4Me
• Lasted 2 weeks
• Run by Tara
• Heavy use of wizarding

68
Validation 3 studies

• Pure diary study
• Early prototype
• Scribe4Me
• Huge success
• Network delays a problem --gt leverage WiFi more