SPACE SPlit Application architeCturE Enabling predictable SW integration

1
SPACESPlit Application architeCturEEnabling
predictable SW integration
Bas Engel
Royal Philips / Consumer Electronics Division
CELF Embedded Linux Conference 2007
April 17, 2007
2
Introduction
3
SW in TV
Software size evolution
Software partitioning

• Total SW size follows Moores law
• Philips part decreasing (not just relative)
• 3rd party involvement increasing

4
Some rationales on 3rd party SW

• Cost, however
• balance NRE and royalties vs. own development
  cost
• TTM
• No time to build up competence in own team
• Off the shelf solution available
• Critical competence available at supplier side

Somebody else makes a business out of supplying
common features
5
Traditional approach integrate
process
Philips
3rd party
MIPS (Linux 2.x)
6
Integration nightmare

• Integration has its advantages
• Optimized resource usage
• Better viewing experience due to integral
  architecture approach
• Integration pitfall
• High effort due to increased complexity
• Longer TTM as there is no little change
• System validation requires global big bang
  approach
• No independent lifecycle, no distributedintegrati
  on cycle

7
The challenge
Create a Win-Win atmosphere with 3rd parties

• We have to create an environment in which 3rd
  parties can integrate their software
• Without deep system knowledge
• We have to enable predictable integration
• Preventing spaghetti SW
• We must cater for sharing critical resources
• AVG Platform
• General purpose infrastructure

8
Design objectives

• Fast and predictable integration of system
  extensions
• Avoid an extensive (re)validation cycle
• Enable convincing module test opportunities
• Enable PC based testing
• Orthogonal 3rd party SW integration
• Leverage standard Linux infrastructure
• Multi client usage of platform resources
• Manage independently deployable building blocks
• Limited building block correlation
• Cater for extensions without the need to know all
  the details
• Enhanced execution architecture
• Independent application lifecycle
• Enable true Multi window architecture
• For Video and Graphics
• Preserving application orthogonality

9
The mental model SW bolt-ons
Feat4
Feat1
Feat2
Feat3
process
TV Middleware
TV Platform
MIPS (Linux 2.x)
10
SPACE
11
Introducing SPACE
Application
TVMiddleware
FeatN
Feat1
Platform API
TV platform
ApplicationManager
Linux 2.x

• Have orthogonal applications
• The resources in the system are explicitly and
  centrally managed
• The client applications are system context
  unaware
• The lifecycle, focus and visual layout of the
  client applications is centrally managed

12
Application Manager

• Application lifecycle management
• Starting, stopping applications based on remote
  control keys
• Knows the system requirements of all
  applications
• Focus handling
• Determines the active window, reacts to user
  request
• Manages the application requirements to the AV
  resources
• Layout management
• Determines how the applications are presented

13
Managing resources

• There are implicitly managed resources by the
  kernel
• TCP, flash, USB
• Memory allocation
• There are explicitly managed resources
• AV platform resources
• Memory and CPU resources
• The explicitly managed resources
• Have a statically defined execution behavior
• Can by design limit the parallelism in the system
• Require an explicitly resource controlled system

14
Resource Controlled System

• Application Manager (amApp)
• Knows the resource dependencies of any client
  application
• Statically by design
• The platform application (plfApp)
• Has a static model of resource sharing
• By design resources are multi/single client
• Every application
• Must be resource aware
• Meaning the resource is or is not reserved for
  them

15
Multi Client Management
tvApp
otherApp
SetFrequency (tuner)
SetFrequency (tuner)
?
?
plfApp
amApp
ResourceOwner (tvApp)
Linux 2.x
16
Resource Groups

• There are different resource groups offered by
  the platform
• Front-end, demux, decode, viewing mode, source
  selection
• To allow multiple applications to access
  different parts of the system
• Every resource group has a static set of
  interfaces
• The Application Manager is the main Resource
  controller
• It designates resource groups to clients
• By informing the platform application whom to
  grant access

17
The platform API

• The Philips platform API
• Leverages as much as possible the current APIs
• Enables the resource sharing model
• Is supplier independent
• AV Interface definition
• Analog interfaces Philips Analog TV API
• Digital interfaces NXPs Digital TV API
• AV Interface instance
• Simple header files and libraries
• Tool to generate proxy/stub code
• Graphics interface definition
• DirectFB

?
?
?
18
General Connection Management Concept

• Connection Management is split up in
• Destination setup
• Output FullScreen, PIP
• Source setup
• Input HDMI, Tuner
• Connection Management is distributed
• Application Manager is responsible for
• Destination setup
• Client application lifecycle
• Client applications are responsible for
• Source setup
• Client applications are destination unaware
• Application Manager is source unaware

19
Example
3 SetSource(tuner) 6 Program Freq, PID,
2 eventDestination(tvApp)
5 eventOnSourceSelected()
tvApp
4 SelectTuner
plfApp
amApp
Linux 2.x
1 SetDestinationFullScreen(tvApp)
20
Border Windows and Video Windows
Border
Video

• Border windows are linked to video windows
• Every application controlling video creates a
  border window
• Application manager controls visibility of such a
  couple
• Visible border windows are equal to the visible
  video windows
• There can be multiple border windows for 1 video
  window
• The border window determines the position of the
  video window
• DirectFB support this via the concept of Input
  Only Windows
• Geometry (positioning) is still available to
  allow focus management
• No client buffer is required, no scaling is done

21
Graphics Connection Management
tvApp
App2
amApp
Select
App1
configure
Draw
DirectFB infrastructure
source
destination
22
Window Management responsibilities
amApp
focus
Layout
start/kill
Size
tvApp
App1
App2
tvApp
create
draw
App1
App2
23
Traditional DirectFB Window manager

• Pluggable WM API
• Move, Resize, Raise, Lower, Hide, Show Windows
• Process Input, Handle Focus and Grabbing,
  Dispatch Events
• Full Cursor implementation including Show, Hide,
  Reshape, Move
• Composition of the screen or parts, e.g.
  triggered by
• Flip() on a window surface
• Move, Raise, Show etc.
• 2D composition
• Background, Windows, Cursor
• Default WM module
• Just follow application requests
• Basic focus handling

App
App
DirectFB API
WM Module
SHM
24
SaWMan

• Shared Application Window Manager
• New DirectFB window manager
• On its own like default window manager
  implementation
• Default is maintained to preserve backwards
  compatibility
• Required was a more controlled behavior
• Add/Remove windows, can modify initial
  configuration
• Configuration requests (move, resize, opacity),
  modify or reject
• Filter input events before being processed by
  SaWMan
• Completely overrule window layout, enable focus
  borders etc.
• Application Manager can hook into most of the API
  flow

App
App
App
Application Manager
DirectFB API
SaWMan API
SHM RPC to AM
SaWMan Module
25
Window Management
Focus mgt
Z-order
Layers
Clients
Child abducted San Jose
Child abducted San Jose
Message
high
Application
med
Border
low
Video
HW
26
DirectFB layer mapping on HW

• If we need fast rendering (gt10FPS) and no HW
  acceleration available
• We must use direct surface drawing
• To achieve gt25 FPS without SW scaling overhead
• DirectFB must allow Layers to be mixed across HW
  Surfaces
• Some Layers can have SW scaling, others may not

Legend
Graphics
Versatile
Z-order
Layers
Surface
Z-order
Layers
Surface
Performance
Video
Message
GFX1
Message
GFX1
high
high
Application
GFX1
Application
GFX1
med
med
GFX1
GFX0
Border
Border
low
low
Video
Video0
Video
Video0
HW
HW
Option 1
Option 2
27
Window Association

• Windows can be associated to another
• Meaning identical position and size
• Focus only to associated group
• Focus only to top window, not associated window
• Example use-case teletext menu
• Teletext and menu of teletext separate windows

CreateAssociative
Draw
Client
Draw
Flip
window
time
28
Pixel Alignment

• plfApp can crop video
• Eg to remove DNM border artifacts
• plfApp can scale video within border window
• Eg to set the scaling to 43 or 169
• Any client application can indicate pixel
  alignment
• To assure that the graphics is pixel accurately
  positioned on top of the video
• DirectFB caters for the alignment
• plfApp configures the scale/crop factors
• Other application is pan/zoom
• Any application can use the SW scaling

Crop Scale
Crop value
VideoAligned
Crop Scale
29
Road to success
30
Planned extensions to DirectFB

• SawMan
• Shared Application Window Manager
• Enhanced SW up/down scaling for windows
• With dynamic layer reconfiguration and color key
  conversion
• FusionDale FusionIPC
• Applied Fusion for Event handling and IPC
• Memory usage optimization
• Shared Memory Heap
• Window Association
• Link windows together for size and location
• Window mapping on HW layers
• Including layer specific configurations Pixel
  alignment
• Extended Key handling
• Sending keys to windows and return key not used
• Audio Nodes
• General ID for audio resources
• Multi processor SPACE
• Enable multiple control processors to host client
  applications

Published
To be published
Plan
31
Main conclusion

• The integration nightmare can become manageable
• Using an open standard based multi application
  approach
• DirectFB, Linux
• With a clear orthogonal view
• No dependencies between applications
• Adding the necessary components for success
• SPACE, SaWMan
• SPACE
• Is the architectural concept for multi
  application systems
• Coping with the resource constraints in CE
  devices
• SaWMan
• Enables application lifecycle management, focus
  handling,and window management
• Is fully available via the DirectFB mainline

32
Demo SPACE
33
(No Transcript)