Processes, Threads and Virtualization

1
Processes, Threads and Virtualization

• Chapter 3.1-3.2
• The role of processes in distributed systems

2
Concurrency Transparency

• Traditional operating systems use the process
  concept to provide concurrency transparency to
  executing processes.
• Process isolation virtual processor
• Multithreading provides concurrency with less
  overhead (so better performance)
• Also less transparency application must provide
  memory protection for threads.

3
Overhead Due to Process Switching
Save CPU context Modify data in MMU registers
Save CPU context Modify data in MMU
registers Invalidate TLB entries

• Figure 3-1. Context switching as the result of
  IPC.

4
Large Applications

• Early operating systems (e.g., UNIX)
• Supported large apps by supporting the
  development of several cooperating programs
  (multiple processes) via fork( ) system call
• Rely on IPC mechanisms to exchange info
• Pipes, message queues, shared memory
• Overhead numerous context switches
• Multithreading versus multiple processes
  communication through shared memory with little
  or no intervention by kernel

5
Threads

• Kernel-level
• Support multiprocessing
• Independently schedulable by OS
• Can continue to run if one thread blocks on a
  system call.
• User-level
• Less overhead than k-level faster execution
• Light weight processes (LWP)
• Example in Suns Solaris OS
• Scheduler activations
• Research based

6
Hybrid Threads Lightweight Processes (LWP)

• LWP is similar to a kernel-level thread
• It runs in the context of a regular process
• The process can have several LWPs created by the
  kernel in response to a system call.
• The user-level thread package creates user-level
  threads and assigns them to LWPs.

7
Thread Implementation

• Figure 3-2. Combining kernel-level lightweight
  processes and user-level threads.

8
Hybrid threads LWP

• The operating system schedules an LWP
• The process (through the thread library) decides
  which user-level thread to run
• If a thread blocks, the LWP can select another
  runnable thread to execute
• User level functions can also be used to
  synchronize user-level threads

9

• Advantages
• Most thread operations (create, destroy,
  synchronize) are done at the user level
• Blocking system calls need not block the whole
  process
• Applications only deal with user-level threads
• LWPs can be scheduled in parallel on the separate
  processing elements of a multiprocessor.

10
Scheduler Activations

• Another approach to combining benefits of u-level
  and k-level threads
• When a thread blocks on a system call, the kernel
  executes an upcall to a thread scheduler in user
  space which selects another runnable thread
• Violates the principles of layered software

11
Threads in Distributed Systems

• Threads gain much of their power by sharing an
  address space
• No shared address space in distributed systems
• Individual processes e.g., a client or a server,
  can be multithreaded to improve performance

12
Multithreaded Clients

• Main advantage hide network latency
• Addresses delays in downloading documents from
  web servers in a WAN
• Hide latency by starting several threads
• One to download text (display as it arrives)
• Others to download photographs, figures, etc.
• All threads execute simple blocking system calls
  easy to program this model
• Browser displays results as they arrive.

13
Multithreaded Clients

• Even better if servers are replicated, the
  multiple threads may be sent to separate sites.
• Result data can be downloaded in several
  parallel streams, improving performance even
  more.
• Designate a thread in the client to handle and
  display each incoming data stream.

14
Multithreaded Servers

• Improve performance, provide better structuring
• Consider what a file server does
• Wait for a request
• Execute request (may require blocking I/O)
• Send reply to client
• Several models for programming the server
• Single threaded
• Multi-threaded
• Finite-state machine

15
Threads in Distributed Systems - Servers

• A single-threaded server processes one request at
  a time
• Creating a new server process for each new
  request creates performance problems.
• Creating a new server thread is much more
  efficient.
• Processing is overlapped without the overhead of
  context switches.

16
Multithreaded Servers

• Figure 3-3. A multithreaded server organized in a
  dispatcher/worker model.

17
Finite-state machine

• The file server is single threaded but doesnt
  block for I/O operations
• Instead, save state of current request, switch to
  a new task client request or disk reply.
• Outline of operation
• Get request, process until blocking I/O is needed
• Record state of current request, start I/O, get
  next task
• If task completed I/O, resume process waiting
  on that I/O using saved state.

18
3.2 Virtualization

• Multiprogrammed operating systems provide the
  illusion of simultaneous execution through
  resource virtualization
• Use software to make it look like concurrent
  processes are executing simultaneously
• Virtual machine technology creates separate
  virtual machines, capable of supporting multiple
  instances of different operating systems.

19
Benefits

• Hardware changes faster than software
• Suppose you want to run an existing application
  and the OS that supports it on a new computer
  the VMM layer makes it possible to do so.
• Software is more easily ported to other machines
• Compromised systems (internal failure or external
  attack) are isolated.

20
Interfaces Offered by Computer Systems

• Unprivileged machine instructions available to
  any program
• Privileged instructions hardware interface for
  the OS/other privileged software
• System calls OS interface to the operating
  system for applications
• API An OS interface through function calls

21
Two Ways to Virtualize
Process Virtual Machine program is compiled to
intermediate code, executed by a runtime system
Virtual Machine Monitor software layer mimics
the instruction set supports an OS and its
applications
22
Processes in a Distributed System

• Chapter 3.3, 3.4, 3.5
• Clients, Servers, and Code Migration

23
Client Server Interaction
Fat client each remote app has two parts one
on the client, one on the server. Communication
is app. specific
Thin client the client is basically a terminal
and does little more than provide a GUI interface
to remote services.
24
Client Side Software

• Manage user interface
• Parts of the processing and data (maybe)
• Support for distribution transparency
• Access transparency Client side stubs hide
  communication and hardware details.
• Location, migration, and relocation transparency
  rely on naming systems, among other techniques
• Failure transparency (e.g., client middleware can
  make multiple attempts to connect to a server)

25
Client-Side Software for Replication Transparency

• Figure 3-10. Transparent replication of a server
  using a client-side solution.

Here, the client application is shielded from
replication issues by client-side software that
takes a single request and turns it into multiple
requests takes multiple responses and turn them
into a single response.
26
Servers

• Processes that implement a service for a
  collection of clients
• Passive servers wait until a request arrives
• Iterative servers handles one request at a time,
  returns response to client
• Concurrent servers act as a central receiving
  point
• Multithreaded servers versus forking a new process

27
Contacting the Server

• Client requests are sent to an end point, or
  port, at the server machine.
• How are port numbers located?
• Global e.g 21 for FTP requests and 80 for HTTP
• Or, contact a daemon on a server machine
• For services that dont need to run continuously,
  superservers can listen to several ports, create
  servers as needed.

28
Stateful versus Stateless

• Some servers keep no information about clients
  (Stateless)
• Example a web server which honors HTTP requests
  doesnt need to remember which clients have
  contacted it.
• Stateful servers retain information about clients
  and their current state, e.g., updating file X.
• Loss of state may lead to permanent loss of
  information.

29
Server Clusters

• A server cluster is a collection of machines,
  connected through a network, where each machine
  runs one or more services.
• Often clustered on a LAN
• Three tiered structure
• Client requests are routed to one of the servers
  through a front-end switch

30
Server Clusters (1)

• Figure 3-12. The general organization of a
  three-tiered server cluster.

31
Three tiered server cluster

• Tier 1 the switch
• Tier 2 the servers
• Some server clusters may need special
  compute-intensive machines in this tier to
  process data
• Tier 3 data-processing servers, e.g. file
  servers and database servers
• For other applications, the major part of the
  workload may be here

32
Server Clusters

• In some clusters, all server machines run the
  same services
• In others, different machines provide different
  services
• May benefit from load balancing
• One proposed use for virtual machines

33
3.5 - Code Migration Overview

• So far, focus has been on DS (Distributed
  Systems) that communicate by passing data.
• Why not pass code instead?
• Load balancing
• Reduce communication overhead
• Parallelism e.g., mobile agents for web searches
• Code migration v process migration
• Process migration yay require moving the entire
  process state can the overhead be justified?
• Early DSs focused on process migration tried
  to provide it transparently

34
Client-Server Examples

• Example 1 (Send Client code to Server)
• Server manages a huge database. If a client
  application needs to perform many database
  operations, it may be better to ship part of the
  client application to the server and send only
  the results across the network.
• Example 2 (Send Server code to Client)
• In many interactive DB applications, clients need
  to fill in forms that are subsequently translated
  into a series of DB operation where validation
  at server side is required.

35
Examples

• Mobile agents independent code modules that can
  migrate from node to node in a network and
  interact with local hosts e.g. to conduct a
  search at several sites in parallel
• Dynamic configuration of DS Instead of
  pre-installing client-side software to support
  remote server access, download it dynamically
  from the server when it is needed.

36
Code Migration

• Figure 3-17. The principle of dynamically
  configuring a client to communicate to a server.
  The client first fetches the necessary software,
  and then invokes the server.

37
A Model for Code Migration (1) as described in
Fuggetta et. al. 1998

• Three components of a process
• Code segment the executable instructions
• Resource segment references to external
  resources (files, printers, other processes,
  etc.)
• Execution segment contains the current state
• Private data, stack, program counter, other
  registers, etc.

38
A Model for Code Migration (2)

• Weak mobility transfer the code segment and
  possibly some initialization data.
• Process can only migrate before it begins to run,
  or perhaps at a few intermediate points.
• Requirements portable code
• Example Java applets
• Strong mobility transfer code segment and
  execution segment.
• Processes can migrate after they have already
  started to execute
• Much more difficult

39
A Model for Code Migration (3)

• Sender-initiated initiated at the home of the
  migrating code
• e.g., upload code to a compute server launch a
  mobile agent, send code to a DB
• Receiver-initiated host machine downloads code
  to be executed locally
• e.g., applets, download client code, etc.
• If used for load balancing, sender-initiated
  migration lets busy sites send work elsewhere
  receiver initiated lets idle machines volunteer
  to assume excess work.

40
Security in Code Migration

• Code executing remotely may have access to remote
  hosts resources, so it should be trusted.
• For example, code uploaded to a server might be
  able to corrupt its disk
• Question should migrated code execute in the
  context of an existing process or as a separate
  process created at the target machine?
• Java applets execute in the context of the target
  machines browser
• Efficiency (no need to create new address space)
  versus potential for mistakes or security
  violations

41
Cloning v Process Migration

• Cloned processes can be created by a fork
  instruction (as in UNIX) and executed at a remote
  site
• Clones are exact copies of their parents
• Migration by cloning improves distribution
  transparency because it is based on a familiar
  programming model

42
Models for Code Migration

• Figure 3-18. Alternatives for code migration.

43
Resource Migration

• Resources are bound to processes
• By identifier resource reference that identifies
  a particular object e.g. a URL, an IP address,
  local port numbers.
• By value reference to a resource that can be
  replaced by another resource with the same
  value, for example, a standard library.
• By type reference to a resource by a type e.g.,
  a printer or a monitor
• Code migration cannot change (weaken) the way
  processes are bound to resources.

44
Resource Migration

• How resources are bound to machines
• Unattached easy to move my own files
• Fastened harder/more expensive to move a large
  DB or a Web site
• Fixed cant be moved local devices
• Global references meaningful across the system
• Rather than move fastened or fixed resources, try
  to establish a global reference

45
Migration and Local Resources

• Figure 3-19. Actions to be taken with respect to
  the references to local resources when migrating
  code to another machine.

46
Migration in Heterogeneous Systems

• Different computers, different operating systems
  migrated code is not compatible
• Can be addressed by providing process virtual
  machines
• Directly interpret the migrated code at the host
  site (as with scripting languages)
• Interpret intermediate code generated by a
  compiler (as with Java)

47
Migrating Virtual Machines

• A virtual machine encapsulates an entire
  computing environment.
• If properly implemented, the VM provides strong
  mobility since local resources may be part of the
  migrated environment
• Freeze an environment (temporarily stop
  executing processes) move entire state to
  another machine
• e.g. In a server cluster, migrated environments
  support maintenance activities such as replacing
  a machine.

48
Migration in Heterogeneous Systems

• Example real-time (live) migration of a
  virtualized operating system with all its running
  services among machines in a server cluster on a
  local area network.
• Presented in the paper Live Migration of Virtual
  Machines, Christopher Clark, et. al.
• Problems
• Migrating the memory image (page tables,
  in-memory pages, etc.)
• Migrating bindings to local resources

49
Memory Migration in Heterogeneous Systems

• Three possible approaches
• Pre-copy push memory pages to the new machine
  and resending the ones that are later modified
  during the migration process.
• Stop-and-copy pause the current virtual machine
  migrate memory, and start the new virtual
  machine.
• Let the new virtual machine pull in new pages as
  needed, using demand paging
• Clark et.al use a combination of pre-copy and
  stop-and-copy claim downtimes of 200ms or less.

50
Migration in Heterogeneous Systems - Example

• Migrating local resource bindings is simplified
  in this example because we assume all machines
  are located on the same LAN.
• Announce new address to clients
• If data storage is located in a third tier,
  migration of file bindings is trivial.