A Software Infrastructure for Applications and Networked Services on Embedded Hardware

1
A Software Infrastructure for Applications and
Networked Services on Embedded Hardware

• Leon Sodhi
• Ls59_at_kent.ac.uk

Supervisor Fred Barnes (F.R.M.Barnes_at_kent.ac.uk)
2
Overview

• What is it
• Similar systems
• Motivation
• Technical details
• Concluding thoughts
• Future work

3
What is it

• A simple Operating System designed for speed that
  provides networking services on embedded hardware
• Embedded hardware meaning a general use device
  e.g. an old PC, Personal digital assistant (PDA)
  etc
• Not based on Linux or Windows but does borrow
  some ideas
• No context switching
• No user mode, kernel mode
• No memory protection

4
Similar systems

• Real time OS
• Designed only for one specific task that may run
  for many years, you wouldnt generally build the
  kernel on the device
• Standard desktop OS - Linux, Windows etc
• Provides too many unwanted features
• Less of a problem on Linux but certainly an issue
  on Windows that is only getting worse!
• A stripped down Linux kernel could be used but
  because so much would need to be removed it would
  take longer than writing something simple from
  scratch
• Research OS and others
• Too many to consider and not in wide use in
  industry or the home

5
Motivation 1

• Modern Operating Systems include services in the
  kernel that cant be disabled and hinder system
  performance
• Two of the main culprits
• User mode and memory management if my
  application is well written and has been around
  for many years do I really want each memory
  access and system call checked?
• Scheduling algorithms - maybe I want all or most
  of my processing power devoted to one application

6
Do we really need this speed?

• Make the computer more responsive
• Multimedia, server software and other
  applications can be quite power hungry
• Examples of these
• Servers providing services on the internet some
  companies have even installed hardware that
  processes parts of the network stack in order to
  save CPU cycles!
• Computer games
• Viewing high definition video even with hardware
  acceleration requires a large amount of CPU power
• Personally discovered a significant increase in
  speed just by raising the video player process
  priority (without dedicated hardware)
• Its far from perfect even with dedicated hardware

7
Do we really need this speed?
8
Yes!
Note This is when you also have dedicated
hardware

• Taken from part of the NVIDIA web site
  http//www.nzone.com/object/nzone_pvhd_build.html

9
Motivation 2

• Why network services?
• Replace routers in homes with an old PC or
  install on embedded hardware
• The routers capabilities can be relatively easily
  expanded
• Reasonable goal to achieve given the short amount
  of time and limited man power
• Simplicity Allows the system to act as an
  educational tool for students to learn about
  Operating Systems

10
Technical details - Overview
11
System initialisation

• Grub boots the kernel in protected mode
• Kernel loads a new GDT (Global descriptor table)
  from the one Grub gave us allowing access to all
  memory available
• GDT is basically a list of memory descriptions
  specifying protection levels and other attributes
  e.g. call gates
• Setup the stack
• Initialise the 8259 (interrupt controller) so
  that CPU exceptions dont overlap with hardware
  interrupts
• Setup exception handlers
• Initialise the keyboard, timer
• Enable interrupts
• Initialise memory management - setting up the
  free lists with memory addresses, maximum sizes,
  maximum block sizes etc
• Initialise the floppy drive
• Detect the network cards present
• Setup telnet and service hooks
• Demo

12
PCI hardware detection

• Find the 32bit PCI BIOS in memory by looking for
  _32_ and verify there is an implementation
  present
• Find each network card based on its device ID and
  vendor ID using the find PCI Device PCI BIOS
  function
• Most (if not all) PCI devices have a device ID
  and vendor ID that identifies the card, can be
  very useful in finding out what an unknown device
  is in Windows device manager
• You can lookup these IDs at http//www.pcidatabas
  e.com
• Determine each devices I/O address and interrupt
  request line by querying memory in the PCI
  configuration space
• This configuration space is a 256KB block of
  memory assigned to store information about each
  device
• Difficulties obtaining information
• The PCI spec isn't free so it was quite a
  challenge just to find information on how it
  works
• Had a few problems discovering which parts of
  memory I needed to read for the I/O address and
  eventually ended up referring to the Linux source
  code

13
Memory management

• Only one stack FIXME
• Single system wide heap for kernel, services and
  applications FIXME
• How it works
• Heap implementation based in part on dlmalloc
  (Doug Leas Malloc used in Linux)
• Memory manager is given an address range and
  divides this up into 3 free lists are used for
  small, medium and large blocks
• What is a good size for these blocks? needs
  profiling!

14

• Allocating
• Select a list based on the block size requested
• Align the size to a 4 byte boundary for more
  efficient memory access
• Find the first free block which is big enough
  algorithm needs improvement! needs to use some
  sort of tree (maybe binary tree? Or buddy memory
  allocation)

• Free
• Determine the list based on the memory address
• Merge any free blocks on either side and add back
  to just after the beginning of the free list

15
File access

• Quite basic
• Only a floppy disk driver is currently present
  but does provide read and write functionality
• File system interface provides mount, fOpen,
  fRead, readDirListing and uMount no fClose
  required!
• fRead reads in the entire file to a static or
  generally malloced buffer nice and simple but
  has problems with huge files
• readDirListing only supports reading in files on
  the root of the disk
• FAT12 only partially implemented no write
• Why is it called FAT12? very fiddly!
• Inefficient access to the disk
• reads in the entire FAT when you mount the disk
• reads in the entire file as explained above
• Slight optimisation in reading the directory
  listing
• reads in each sector and determines whether we
  have reached the end
• Didnt actually make a huge amount of difference

16
Networking

• Very basic at the moment,
• Network card driver at the bottom for the NE2000
  NIC
• Network stack consists of structures that filter
  packets through the system
• Sends ARP and ICMP replies but other than that
  pretty much only understands a few packet headers
• No fancy TCP implementation that supports time
  outs etc
• No checksum implemented for UDP (value of 0 used)
• Only semi-interesting part is the checksum
  calculated for IP packets but this was code taken
  from another open source operating system
• IP and MAC addresses specified in the code but
  this could fairly easily be changed to come from
  a file on system boot or a telnet option could be
  created to set the addresses
• ARP bug need to request and then request again
  FIXME

17
Telnet framework

• Telnet framework (includes a Windows client)
• Used to remotely manage applications and services
  installed on the machine
• Not actually telnet but a similar text interface
• Required a custom made client that unfortunately
  only runs on Windows
• Can be enabled or disabled per Ethernet adapter
  Can save CPU cycles if the interface isn't needed
  and can prevent people changing settings on the
  machine over the internet
• Applications or services can register text based
  options that are sent to the client when it
  connects
• The client can then select from the list, enter
  some data e.g. an IP address to add to the filter
  list, this will then get sent back to the service
  or app that registered the option
• Makes it very easy for services and apps to be
  managed remotely, all they have to do is provide
  a call back that will then get sent the clients
  data

18
Net hooks

• Allows applications or services to easily filter
  incoming or outgoing network traffic
• Similar to the telnet framework, a call back is
  registered which then receives network data
• The call back can return either true to allow the
  packet to continue or false to drop it

19
Applications

• Applications
• Only one can be run at once
• Takes complete control of the machine while its
  executing other than registered hooks and other
  code added by the user
• Has to be position independent code (untested) or
  linked at a fixed start address
• An example
• include ltAPI.hgt
• void main( PAPILIST pTheAPI )
• pTheAPI-gtSTDIO-gtprint( "Hello world\n" )
• Demo

20
3rd party services

• Services
• Main sets up global service resources and
  register hooks and then exits leaving resources
  in place sort of like (TSRs) terminate and
  stay resident in DOS. Code is only executed when
  an event occurs
• Has a slightly different main function that
  includes a service ID
• Currently up to 4 services can be registered each
  reserving 1MB of memory
• A bit fiddly to setup without Position
  independent code (untested) a service slot
  address has to be obtained from the OS using the
  svcaddr command. Service is then linked at this
  address. This means each service is tied to a
  slot!
• Service is then installed with the installsvc
  command which prints the service ID
• This service ID can be used with the removesvc
  command to uninstall it
• Demo

21
Binary loader

• Uses the COFF executable format for both services
  and applications
• Has two functions executeApp and executeSvc
• Both gets the executables entry point, copies the
  code to the appropriate address or service slot
  and then executes main
• executeApp differs in that it can preserve the
  current applications address space
• This is useful if an application wants to execute
  another application
• Current application address space is copied to a
  malloced region
• The new application overwrites the current apps
  memory space
• When the new app exits the old apps address space
  is restored and the malloced region freed
• This functionality is used for the command line
  tool so that it is restored when the users
  application code exits

22
API

• Passed in to the application or service main
  function (other parameters with services)
• void main( PAPILIST pTheAPI )
• struct APILIST
• PAPISTDIO STDIO
• PAPISTRING STRING
• PAPISERVICES SERVICES
• PAPITELNET TELNET
• PAPIFILE FILE
• PAPINET NET
• PAPINETHOOKS NETHOOKS
• PAPIMEMORY MEMORY
• PAPICONSOLE CONSOLE
• PAPIPROCESS PROCESS
• PAPILIST

• FIXME
• Changing the API currently involves recompiling
  each application
• Either need to give the application a version
  structure that has several APILIST structures or
  find another way to do it.

23
Concluding thoughts

• Almost everything implemented that was planned
• Gateway service was planned but dropped due to
  time constraints
• Enough functionality has been created so that
  many real world services and applications could
  be implemented although they may have certain
  functionality omitted e.g. the ability to save to
  disk
• Shouldnt be too difficult to port the code to
  another type of CPU useful for the many embedded
  devices that dont use x86
• Significant knowledge has been gained by the
  author through the large amount of research
  carried out into how the hardware functions and
  hopefully this can be passed on to other students
• No benchmarking was performed, in particular it
  would have been nice to benchmark
• Malloc implementation Profile the heap usage
  and especially fragmentation as the service and
  application are used
• Compare network traffic throughput to Linux on
  continually slower computers particularly as more
  IP address filter rules are applied

24
Future work

• Huge amounts!
• Implement missing features e.g. writing to the
  file system, API versioning system or separate
  implementation, more than one heap etc
• Fix bugs
• Optimise existing code
• Improve code readability
• Add more GUI, drivers, other optional services,
  complete the To-Do's scattered across the code
• The list goes on
• Particular work of interest includes
• Multiple threads using hardware Cell processor
• Application compatibility Implement POSIX,
  Windows (see http//www.ReactOS.org)

25
Cell processor

• We want multitasking support but without the
  overhead of context switching
• Do away with memory protection etc on processors
  that do user related work (SPE)
• Operating system only executes on the PPU
  processor

26
Questions?