Advanced Multimedia and Presence Services using Classical and P2P SIP

1
Advanced Multimedia and Presence Services using
Classical and P2P SIP

• Henning Schulzrinne
• (with Kundan Singh, Ron Shacham, Xiaotao Wu,
  Jonathan Lennox and others)
• Department of Computer Science
• Columbia University
• hgs_at_cs.columbia.edu
• May 23, 2005

2
Overview

• Quick overview of SIP
• Ring-and-hope ? presence-mediated communications
• Uses for presence
• Old Im on-line
• Location-based services
• Presence-derived call handling
• Presence-derived trust
• Presence and privacy
• Service and session mobility
• Application sharing

3
Philosophy transition
One computer, many users
One computer, one user
mainframe era
Many computers, one user
ubiquitous computing
anywhere, any time any media
right place (device), right time, right media
4
Evolution of VoIP
how can I make it stop ringing?
does it do call transfer?
long-distance calling, ca. 1930
going beyond the black phone
amazing the phone rings
catching up with the digital PBX
1996-2000
2000-2003
2004-
5
Collaboration in transition
inter-organization multiple technology
generations diverse end points
intra-organization small number of systems
(meeting rooms)
standards-based solutions
proprietary (single-vendor) systems
6
SIP Overview
7
Internet services the missing entry
Service/delivery synchronous asynchronous
push instant messaging presence event notification session setup media-on-demand messaging
pull data retrieval file download remote procedure call peer-to-peer file sharing
8
Filling in the protocol gap
Service/delivery synchronous asynchronous
push SIP RTSP, RTP SMTP
pull HTTP ftp SunRPC, Corba, SOAP (not yet standardized)
9
SIP as service enabler

• Rendezvous protocol
• lets users find each other by only knowing a
  permanent identifier
• Mobility enabler
• personal mobility
• one person, multiple terminals
• terminal mobility
• one terminal, multiple IP addresses
• session mobility
• one user, multiple terminals in sequence or in
  parallel
• service mobility
• services move with user

10
What is SIP?

• Session Initiation Protocol ? protocol that
  establishes, manages (multimedia) sessions
• also used for IM, presence event notification
• uses SDP to describe multimedia sessions
• Developed at Columbia U. (with others)
• Standardized by
• IETF (RFC 3261-3265 et al)
• 3GPP (for 3G wireless)
• PacketCable
• About 100 companies produce SIP products
• Microsofts Windows Messenger (4.7) includes SIP

11
Philosophy

• Session establishment event notification
• Any session type, from audio to circuit emulation
• Provides application-layer anycast service
• Provides terminal and session mobility
• Based on HTTP in syntax, but different in
  protocol operation
• Peer-to-peer system, with optional support by
  proxies
• even stateful proxies only keep transaction
  state, not call (session, dialogue) state
• transaction single request retransmissions
• proxies can be completely stateless

12
Basic SIP message flow
13
SIP trapezoid
destination proxy (identified by SIP URI domain)
outbound proxy
1st request
SIP trapezoid
2nd, 3rd, request
a_at_foo.com 128.59.16.1
registrar
voice traffic RTP
14
SIP message format
SDP
15
PSTN vs. Internet Telephony
PSTN
Signaling Media
Signaling Media
China
Internet telephony
Signaling
Signaling
Media
Australia
Belgian customer, currently visiting US
16
SIP addressing

• Users identified by SIP or tel URIs
• sipalice_at_example.com
• tel URIs describe E.164 number, not dialed
  digits (RFC 2806bis)
• tel URIs ? SIP URIs by outbound proxy
• A person can have any number of SIP URIs
• The same SIP URI can reach many different phones,
  in different networks
• sequential parallel forking
• SIP URIs can be created dynamically
• GRUUs
• conferences
• device identifiers (sipfoo_at_128.59.16.15)
• Registration binds SIP URIs (e.g., device
  addresses) to SIP address-of-record (AOR)

tel110
sipsos_at_domain
domain ? 128.59.16.17 via NAPTR SRV
17
3G Architecture (Registration)
mobility management
signaling
serving
interrogating
interrogating
CSCF
proxy
home IM domain
registration signaling (SIP)_
visited IM domain
18
SIP is PBX/Centrex ready
boss/admin features
call waiting/multiple calls RFC 3261
hold RFC 3264
transfer RFC 3515/Replaces
conference RFC 3261/callee caps
message waiting message summary package
call forward RFC 3261
call park RFC 3515/Replaces
call pickup Replaces
do not disturb RFC 3261
call coverage RFC 3261
simultaneous ringing RFC 3261
basic shared lines dialog/reg. package
barge-in Join
Take Replaces
Shared-line privacy dialog package
divert to admin RFC 3261
intercom URI convention
auto attendant RFC 3261/2833
attendant console dialog package
night service RFC 3261
centrex-style features
attendant features
from Rohan Mahys VON Fall 2003 talk
19
A constellation of SIP RFCs
Non-adjacent (3327) Symmetric resp.
(3581) Service route (3608) User agent caps
(3840) Caller prefs (3841)
Request routing
Resource mgt. (3312) Reliable prov. (3262) INFO
(2976) UPDATE (3311) Reason (3326)
SIP (3261) DNS for SIP (3263) Events (3265) REFER
(3515)
ISUP (3204) sipfrag (3240)
Mostly PSTN
Core
Content types
Digest AKA (3310) Privacy (3323) P-Asserted
(3325) Agreement (3329) Media auth. (3313) AES
(3853)
DHCP (3361) DHCPv6 (3319)
Configuration
Security privacy
20
An eco system, not just a protocol
configures
XCAP (config)
SIMPLE policy RPID .
XCON (conferencing)
initiates
carries
SIP
RTSP
SDP
carries
controls
provide addresses
STUN TURN
RTP
21
SIP a bi-cultural protocol

• multimedia
• IM and presence
• location-based service
• user-created services
• decentralized operation
• everyone equally suspect

• overlap dialing
• DTMF carriage
• key systems
• notion of lines
• per-minute billing
• early media
• ISUP BICC interoperation
• trusted service providers

22
Context-aware communication

• context the interrelated conditions in which
  something exists or occurs
• anything known about the participants in the
  (potential) communication relationship
• both at caller and callee

time CPL
capabilities caller preferences
location location-based call routing location events
activity/availability presence
sensor data (mood, bio) privacy issues similar to location data
23
GEOPRIV and SIMPLE architectures
rule maker
DHCP
XCAP (rules)
target
location server
location recipient
notification interface
publication interface
GEOPRIV
SUBSCRIBE
presentity
presence agent
watcher
SIP presence
PUBLISH
NOTIFY
caller
callee
SIP call
INVITE
INVITE
24
The role of presence

• Guess-and-ring
• high probability of failure
• telephone tag
• inappropriate time (call during meeting)
• inappropriate media (audio in public place)
• current solutions
• voice mail ? tedious, doesnt scale, hard to
  search and catalogue, no indication of when call
  might be returned
• automated call back ? rarely used, too inflexible
• ? most successful calls are now scheduled by email

• Presence-based
• facilitates unscheduled communications
• provide recipient-specific information
• only contact in real-time if destination is
  willing and able
• appropriately use synchronous vs. asynchronous
  communication
• guide media use (text vs. audio)
• predict availability in the near future (timed
  presence)

Prediction almost all (professional)
communication will be presence-initiated or
pre-scheduled
25
Basic presence

• Role of presence
• initially can I send an instant message and
  expect a response?
• now should I use voice or IM? is my call going
  to interrupt a meeting? is the callee awake?
• Yahoo, MSN, Skype presence services
• on-line off-line
• useful in modem days but many people are
  (technically) on-line 24x7
• thus, need to provide more context
• simple status (not at my desk)
• entered manually ? rarely correct
• does not provide enough context for directing
  interactive communications

26
Presence data architecture
presence sources
PUBLISH
raw presence document
privacy filtering
create view (compose)
depends on watcher
XCAP
XCAP
select best source resolve contradictions
composition policy
privacy policy
(not defined yet)
draft-ietf-simple-presence-data-model
27
Presence data architecture
candidate presence document
raw presence document
post-processing composition (merging)
watcher filter
SUBSCRIBE
remove data not of interest
difference to previous notification
final presence document
watcher
NOTIFY
28
Presence data model
calendar
cell
manual
person (presentity) (views)
alice_at_example.com audio, video, text
r42_at_example.com video
services
devices
29
Rich presence

• More information
• automatically derived from
• sensors physical presence, movement
• electronic activity calendars
• Rich information
• multiple contacts per presentity
• device (cell, PDA, phone, )
• service (audio)
• activities, current and planned
• surroundings (noise, privacy, vehicle, )
• contact information
• composing (typing, recording audio/video IM, )

30
RPID rich presence
ltpersongt lttuplegt ltdevicegt
ltactivitiesgt
ltclassgt
ltmoodgt
ltplace-isgt
ltplace-typegt
ltprivacygt
ltrelationshipgt
ltservice-classgt
ltspheregt
ltstatus-icongt
lttime-offsetgt
ltuser-inputgt
31
RPID rich presence

• Provide watchers with better information about
  the what, where, how of presentities
• facilitate appropriate communications
• wait until end of meeting
• use text messaging instead of phone call
• make quick call before flight takes off
• designed to be derivable from calendar
  information
• or provided by sensors in the environment
• allow filtering by sphere the parts of our
  life
• dont show recreation details to colleagues

32
CIPID Contact Information

• More long-term identification of contacts
• Elements
• card contact Information
• home page
• icon to represent user
• map pointer to map for user
• sound presentity is available

33
The role of presence for call routing
PUBLISH

• Two modes
• watcher uses presence information to select
  suitable contacts
• advisory caller may not adhere to suggestions
  and still call when youre in a meeting
• user call routing policy informed by presence
• likely less flexible machine intelligence
• if activities indicate meeting, route to tuple
  indicating assistant
• try most-recently-active contact first (seq.
  forking)

PA
NOTIFY
translate RPID
LESS
CPL
INVITE
34
Presence and privacy

• All presence data, particularly location, is
  highly sensitive
• Basic location object (PIDF-LO) describes
• distribution (binary)
• retention duration
• Policy rules for more detailed access control
• who can subscribe to my presence
• who can see what when

lttuple id"sg89ae"gt ltstatusgt ltgpgeoprivgt
ltgplocation-infogt ltgmllocationgt
ltgmlPoint gmlid"point1 srsName"ep
sg4326"gt ltgmlcoordinatesgt374630N
1222510W lt/gmlcoordinatesgt
lt/gmlPointgt lt/gmllocationgt
lt/gplocation-infogt ltgpusage-rulesgt
ltgpretransmission-allowedgtno lt/gpretransmissi
on-allowedgt ltgpretention-expirygt2003-06-2
3T045729Z lt/gpretention-expirygt
lt/gpusage-rulesgt lt/gpgeoprivgt lt/statusgt
lttimestampgt2003-06-22T205729Zlt/timestampgt lt/tupl
egt
35
Privacy policy relationships
common policy
geopriv-specific
presence-specific
future
RPID
CIPID
36
Privacy rules

• Conditions
• identity, sphere
• time of day
• current location
• identity as lturigt or ltdomaingt ltexceptgt
• Actions
• watcher confirmation
• Transformations
• include information
• reduced accuracy

• User gets maximum of permissions across all
  matching rules
• privacy-safe composition removal of a rule can
  only reduce privileges
• Extendable to new presence data
• rich presence
• biological sensors
• mood sensors

37
Example rules document
ltrule id1gt
ltidentitygtltidgtuser_at_example.comlt/idgtlt/identitygt
ltconditionsgt
ltsub-handlinggtallowlt/sub-handlinggt
ltactionsgt
ltprovide-servicesgt ltservice-uri-schemegtsiplt/ser
vice-uri-schemegt ltservice-uri-schemegtmailtolt/se
rvice-uri-schemegt lt/provide-servicesgt ltprovide-per
songttruelt/provide-persongt ltprovide-activitiesgttrue
lt/provide-activitiesgt ltprovide-user-inputgtbarelt/pr
ovide-user-inputgt
ltrulesetgt
lttransformationsgt
38
Creating and manipulating rules

• Uploaded in whole or part via XCAP
• XML not user-visible
• Web or application UI, similar to mail filtering
• Can also be location-dependent
• if at home, colleagues dont get presence
  information
• Possibly implementation-defined privacy levels

39
Location-based services

• Finding services based on location
• physical services (stores, restaurants, ATMs, )
• electronic services (media I/O, printer, display,
  )
• not covered here
• Using location to improve (network) services
• communication
• incoming communications changes based on where I
  am
• configuration
• devices in room adapt to their current users
• awareness
• others are (selectively) made aware of my
  location
• security
• proximity grants temporary access to local
  resources

40
Location-based SIP services

• Location-aware inbound routing
• do not forward call if time at callee location is
  11 pm, 8 am
• only forward time-for-lunch if destination is on
  campus
• do not ring phone if Im in a theater
• outbound call routing
• contact nearest emergency call center
• send delivery_at_pizza.com to nearest branch
• location-based events
• subscribe to locations, not people
• Alice has entered the meeting room
• subscriber may be device in room ? our lab stereo
  changes CDs for each person that enters the room

41
Location detection
42
DHCP for locations

• modified dhcpd (ISC) to generate location
  information
• use MAC address backtracing to get location
  information

8020abd5d
DHCP server
CDP SNMP 8020abd5d ? 458/17
DHCP answer staDC locRm815 lat38.89868
long77.03723
458/17 ? Rm. 815 458/18 ? Rm. 816
43
Location-based service language
NOTIFY
true
false
action
alert
IM
alert
incoming
proximity
message
outgoing
log
conditions
occupancy
actions
events
notify
call
message
time
transfer
subscription
join
44
Program location-based services
45
(No Transcript)
46
Tracking
47
Internet2 WG PIC trial
SUBSCRIBE to my location
NOTIFY myself and others locations
PUBLISH presence status
48
Example user-adaptive device configuration
all devices that are in the building RFC 3082?
802.11 signal strength ? location
SLP
device controller
HTTP
PA
tftp
REGISTER To 815cepsr Contact alice_at_cs
SUBSCRIBE to each room

1. discover room URI
2. REGISTER as contact for room URI

SIP
room 815
SUBSCRIBE to configuration for users currently in
rooms
49
Session mobility

• Walk into office, switch from cell phone to desk
  phone
• call transfer problem ? SIP REFER
• related problem split session across end devices
• e.g., wall display desk phone PC for
  collaborative application
• assume devices (or stand-ins) are SIP-enabled
• third-party call control

50
How to find services?

• Two complementary developments
• smaller devices carried on user instead of
  stationary devices
• devices that can be time-shared
• large plasma displays
• projector
• hi-res cameras
• echo-canceling speaker systems
• wide-area network access
• Need to discover services in local environment
• SLP (Service Location Protocol) allows querying
  for services
• find all color displays with at least XGA
  resolution
• slp//example.com/SrvRqst?public?typeprinter
• SLP in multicast mode
• SLP in DA mode
• Need to discover services before getting to
  environment
• is there a camera in the meeting room?
• SLP extension find remote DA via DNS SRV

51
Session mobility
Local Devices
Transcoder
Internet
SLP DA
SLP SA
SLP UA
SIP SM
SIP UA
SIP UA
Correspondent Node (CN)
SLP SIP RTP
SIP SM
SIP UA
SLP UA
Mobile Node (MN)
52
Presence for spam prevention

• VoIP spam (spit) and IM spam (spim) likely to
  be more annoying than email spam
• Subscription to another person is indication of
  mutual trust
• other person allows me to see their status ?
  trusts me
• Thus, use watcher list (who is watching me) as
  trust vector

53
Conferencing issues
54
Open issues for conferencing standardization

• Multi-device systems and session mobility
• Conference floor control ? BFCP
• simple RPC-like protocol
• Centralized conference control (XCON WG)
• still struggling with control model
• set get variables (SNMP)
• RPC-like model (? SOAP, XML-RPC, NETCONF)
• send descriptions?
• forms (XForms?)?

55
XCON System
56
Open issues application sharing

• Current T.120
• doesnt integrate well with other conference
  control mechanisms
• hard to make work across platforms (fonts)
• ill-defined security mechanisms
• Current web-based sharing
• hard to integrate with other media, control and
  record
• generally only works for Windows
• mostly limited to shared PowerPoint
• Current vnc
• whole-screen sharing only
• can be coerced into conferencing, but doesnt
  integrate well with control protocols

57
IETF effort standardized application sharing

• Remote access application sharing
• Four components
• window drawing ops ? PNG
• keyboard input
• mouse input
• window operations (raise, lower, move)
• Uses RTP as transport
• synchronization with continuous media
• but typically, TCP
• allow multicast ? large group sessions

58
Conclusion

• Basic multimedia communication solutions
  available
• scalable, secure, good quality
• Media delivery ? controlled interaction
• Binary presence ? rich presence
• Presence as enabler of loosely-coupled advanced
  services
• location-based services
• serve as service script inputs
• trust indicator for spit/spim prevention
• Application sharing as missing link
• Harder to maintain focus in the IETF
• exhaustion, moving on to other things