COSC 3407: Operating Systems

1
COSC 3407 Operating Systems

• Lecture 4 Concurrency Threads and Dispatching

2
This lecture

• Goal Abstraction that each thread has illusion
  of its own CPU.
• Reality Hardware-shared CPU, interrupts.
• How does this work?

3
Threads or Lightweight Process

• Thread a sequential execution stream within a
  process (concurrency)
• Provides the illusion that each activity (or
  thread) is running on its own CPU, entirely
  sequentially.
• Process still contains a single Address Space
• No protection between threads
• Address space all the state needed to run a
  program (literally, all the addresses that can be
  touched by a program)
• Provides the illusion that a program is running
  on its own machine (protection).

4
Why separate the concept of a thread from that of
a process?

• Discuss the thread part of a process
  (concurrency), separately from the address
  space part of a process (protection).
• Heavyweight Process ? Process with one thread
• Many situations where you want multiple threads
  per address space.
• Question Why would you want this?
• Multithreading a single program made up of a
  number of different concurrent activities
• Sometimes called multitasking, as in Ada

5
Examples of multithreaded programs

• Embedded systems
• elevators, planes, medical systems, wristwatches,
  etc.
• Single program, concurrent operations.
• Most modern OS kernels
• internally concurrent because have to deal with
  concurrent requests by multiple users.
• But no protection needed within kernel.
• Database Server
• provides access to shared data by potentially
  many concurrent users.
• Also has background utility processing that must
  get done.

6
Examples of multithreaded programs (cont)

• Network servers
• user applications that get multiple requests
  concurrently off the network.
• Again, single program, multiple concurrent
  operations
• file servers, Web server, and airline reservation
  systems
• Parallel programming (More than one physical CPU)
• split program into multiple threads to make it
  run faster.
• This is called multiprocessing.
• Some multiprocessors are actually uniprogrammed
• Multiple threads in one address space but one
  program at a time

7
Threads (Lightweight Processes)

• Basic unit of CPU utilization
• (What?! you say)
• Own
• TCB ? Thread Control Block
• Thread id
• program counter
• register set
• Execution stack
• Shares
• code section
• data section (global variables, heap)
• OS resources (open files and signals)

Process
Program Counter (Threads)
A B C
text segment
A B C
data segment
Multithreaded Program
8
Thread State

• Execution stack where parameters, temporary
  variables, and return PC are kept, while called
  procedures are executing
• (for example, where are As variables kept, while
  B, C are executing?)

A(int tmp) if temp lt 2 B()
printf(tmp) B() C() C()
A(2) A(1)
Execution stack
9
Thread State

• Threads encapsulate concurrency
• Active component of a process
• Address spaces encapsulate protection
• Keeps a buggy program from trashing everything
  else on the system.
• Passive component of a process

10
Thread Design Space

• Real operating systems have either
• One or many address spaces
• One or many threads per address space
• Did Windows 95/98/ME have real memory protection?
• No Users could overwrite process tables/System
  DLLs

address spaces threads/address space One many
One MS/DOS, early Machintosh Traditional UNIX
many Embedded systems, JavaOS, Pilot(PC) Mach, OS/2, Windows NT to XP, Solaris, Linux, HP-UX, OS X..
11
Example Implementation Java OS

• Many threads, one Address Space
• Why another OS?
• Recommended Minimum memory sizes
• UNIX X Windows 32MB
• Windows 98 16-32MB
• Windows NT 32-64MB
• Windows 2000/XP 64-128MB
• What if want a cheap network point-of-sale
  computer?
• Say need 1000 terminals
• Want lt 8MB
• What language to write this OS in?
• C/C/ASM? Not terribly high-level. Hard to
  debug.
• Java/Lisp? Not quite sufficient need direct
  access to HW/memory management

12
User Threads

• User threads are supported above the kernel
• Implemented by thread library
• The library provides support for thread creation,
  scheduling, and management with no support from
  the kernel.
• Drawbacks
• If kernel is single threaded, then
• any user-level thread performing a blocking
  system call will cause the entire process to
  block
• Examples of user-thread libraries
• POSIX Pthreads
• Mach C-threads
• Solaris 2 UI-threads

13
Kernel Threads

• Supported by the Kernel (operating system)
• Thread creation, scheduling, and management are
  done by the kernel in kernel space.
• Kernel threads are slower to create and manage
  than are user threads.
• Advantage
• If a thread performs a blocking call, the kernel
  can schedule another thread in the application
  for execution.
• In a multiprocessor environment, the kernel can
  schedule threads on different processors.
• Examples
• - Windows 95/98/NT/2000/XP
• - Solaris 2
• - Tru64 (formerly Digital UNIX)

14
Multithreading Models

• Many-to-One
• Many User-Level Threads Mapped to Single Kernel
  Thread.
• Thread management is done in the user space.
• Entire process will block if a thread makes a
  blocking system call.
• Since only one thread can access the kernel at a
  time, multiple threads are unable to run in
  parallel on multiprocessors
• Used on Systems That Do Not Support Kernel
  Threads
• Green Threads a thread library in Solaris 2

15
Multithreading Models

• One-to-one Model
• Each User-Level Thread Maps to Kernel Thread.
• Provides more concurrency
• multiple threads can run in parallel on
  multiprocessors
• when one thread makes a blocking system call,
  another thread can be executed.
• Drawback
• creating a user thread requires creating the
  corresponding kernel thread.
• Most systems restrict the number of threads
  supported by the system due to the overhead in
  creating kernel threads.
• Examples
• - Windows 95/98/NT/2000/XP
• - OS/2

16
Multithreading Models (contd)

• Many-to-many Model
• Multiplexes many user-level threads to a smaller
  or equal number of kernel threads.
• The number of kernel threads may be specific for
  either a particular application or a particular
  machine.
• Developers can create as many user threads as
  necessary
• the corresponding kernel threads can run in
  parallel on a multiprocessor.
• When a thread performs a blocking system call,
  the kernel can schedule another thread for
  execution.

17
Multithreading Models (contd)

• Two-level Model (A variation of Many-to-many
  Model)
• Multiplexes many user-level threads to a smaller
  or equal number of kernel threads.
• Addition also allows a user-level thread to be
  bound to a kernel thread
• Examples
• IRIX, HP-UX, Tru64 Unix, Solaris ver. 1 8
• Solaris 9 onwards supports one-to-one model

18
Threads

• Imagine we had the following C program to compute
  and list the digits of Pi into a file and to
  print out the class list to another file
• This program would unfortunately never print out
  the class list since ComputePi() will never
  finish.
• With threads we could have the two activities go
  on in parallel

main() ComputePi(/tmp/pi.text)
PrintClassList(/tmp/clist.text)
main() CreateThread(ComputePi(/tmp/pi.tex
t)) CreateThread(PrintClassList(/tmp/clis
t.text))
19
Threads

• If we stopped this program and examined it with a
  debugger we would see two stacks that could be
  listed and two sets of CPU registers to examine.
• Note we cant simply let each stack grow
  backwards towards the heap anymore! Have to worry
  about stacks overrunning each other.
• Two key concepts
• Thread control block (per-thread state)
• Dispatching loop

code
Global data
heap
Stack 1
Stack 2
20
Thread Control Block (TCB)

• This is where all information relevant to the
  thread is kept.
• There is one per active thread. The contents
  include
• Execution state CPU registers, program counter,
  pointer to stack
• Scheduling information state, priority, CPU time
  used
• Accounting info owner, groups,
• Various pointers (for implementing scheduling
  queues)
• Etc. (add stuff as you find a need)
• In Nachos Thread is a class that includes the
  TCB
• OS keeps an array (or linked list or ) of TCBs
  in its own protected memory.
• We describe one approach to this next.

21
The Lifecycle of a Thread

• Threads all go through the lifecycle shown below
• They are created and admitted to the system
• They remain in the system, sharing resources with
  all other threads in the system. Of course, only
  one thread can be actually running on the CPU
  at any time (assuming a single-CPU system).
• They leave the system.
• We will first describe how threads are managed
  when they are in the system.
• We will then discuss how thread creation and
  termination are handled.

22
Thread queues

• An active thread (i.e., one that is in the
  ready, waiting, or running state) is represented
  in the system by its TCB.
• The TCBs are organized into queues based on their
  state.
• For threads waiting for an event (I/O, other
  thread completion, etc.) they can be queued for
  each such event.

23
Dispatching Loop (scheduler.java)

• The main loop of the scheduler is responsible for
  choosing a thread to run and ensuring that thread
  switching is done correctly.
• Conceptually, it looks like this

Loop Run thread Choose new
thread to run Save state of old thread
from CPU into its TCB Load state of new
thread from its TCB into CPU
24
Running a thread

• Consider first portion Run thread
• How do I run a thread?
• Load its state (registers, PC, stack pointer)
  into the CPU,
• Load environment (virtual memory space, etc)
• jump to the PC value.
• How does dispatcher get control back? Two ways
• Internal events thread hands control back
  voluntarily
• External events thread gets preempted

25
Internal events

• Thread blocks on I/O (e.g., disk I/O, or waiting
  on keyboard input)
• The act of requesting I/O implicitly yields the
  CPU
• Thread blocks waiting on a signal from another
  thread
• Thread asks to wait and thus yields the CPU
• Yield (give CPU to someone else whos ready to
  run)
• Thread volunteers to give up CPU

26
Yield thread switch example
Stack for yielding thread

• How do we run a new thread?

switch
ComputePi() while (TRUE)
ComputeNextDigit()
yield()
run_new_thread()
newThread PickNewThread()
switch(curThread, newThread)
ThreadHouseKeeping() /covered next lecture /
Run_new_thread
Trap to OS
Kernel_yield
yield
computePi
27
Context Switch Saving/restoring state

• What do you need to save/restore when the
  dispatcher switches to a new thread?
• Anything next thread may trash PC, registers,
  change execution stack
• Want to treat each thread in isolation.

28
Two threads looping and calling yield()

• To take an example from the Nachos code
• What if two threads loop, each calling Yield?
• Yield calls Switch to switch to the next thread.
• But once you start running the next thread, you
  are on a different execution stack.
• Thus, Switch is called in one threads context,
  but returns in the others!

switch
switch
A() B() B()LOOPyield
yield
yield
B (loop)
B (loop)
A
A
Thread T
Thread S
Thread T switching to thread S
29
Context Switching

• switch routine (shown here in C syntax, but would
  be implemented in assembler)

switch(int tCur, int tNew)
/ Save registers of running thread to TCB
/ TCBtCur.regs.r7
CPU.r7
TCBtCur.regs.r0 CPU.r0
TCBtCur.regs.sp
CPU.sp
TCBtCur.regs.retpc CPU.retpc / return addr
/ / Load state of
new thread into CPU /
CPU.r7 TCBtNew.regs.r7
CPU.r0
TCBtNew.regs.r0
CPU.sp TCBNnew.regs.sp
/ Henceforth running on new threads stack
/ CPU.retpc
TCBtNew.regs.retpc
return / Return
from switch returns to CPU.retpc /
30
Context Switching

• Note, the retpc is where the return from switch
  should jump to.
• In practice this is really implemented as a
  jump to that location.
• There is a real implementation of Switch in
  Nachos in switch.s of course, its magical! (but
  you should be able to follow it)
• What if you make a mistake in implementing
  switch?
• For instance, suppose you forget to save and
  restore register 4?
• Get intermittent failures depending on exactly
  when context switch occurred, and whether new
  thread was using register 4.
• Potentially, system will give wrong result,
  without any warning (if program didnt notice
  that register 4 got trashed).
• Can you devise an exhaustive test to guarantee
  that switch works? No!

31
What happens when thread blocks on I/O?
Blocked on I/O thread switch example

• Note We assume that all threads go through
  run_new_thread() in order to be removed from and
  reinstated upon the CPU.
• Blocked on thread signal example is similar /
  thread join.
• What if thread never did any I/O, never waited,
  and didnt yield control?
• Dispatcher has to gain control back somehow.

Run_new_thread
Trap to OS
Kernel_read
read
copyFile
32
External events

• Interrupts type character, disk I/O request
  finishes wakes up dispatcher, so it can choose
  another thread to run
• Timer like an alarm clock that goes off every n
  milliseconds
• Interrupts are a special kind of hardware-invoked
  context switch.
• No separate step to choose what to run next
  always run the interrupt handler immediately.

33
Example Network Interrupt
? add r1,r2,r3 subi r4,r1,4 slli
r4,r4,2
lw r2,0(r4) lw r3,4(r4) add r2,r2,r3 sw 8(
r4),r2 ?
34
Use of Timer Interrupt to Return Control

• Timer interrupt routine
• I/O interrupt same as timer interrupt except
  that DoHouseKeeping() is replaced by ServiceIO().

TimerInterrupt()
DoPeriodicHouseKeeping()
run_new_thread()
Stack of preempted thread
Run_new_thread
timerInterrupt
HW timer Interrupt
someRoutine
35
Choosing a thread to run

• Dispatcher keeps a list of ready (runnable)
  threads how does it choose among them?
• Zero ready threads dispatcher just loops.
• Alternative is for OS to create an idle thread
  that simply loops or does nothing. Then theres
  always at least one runnable thread.
• Exactly one ready thread easy.
• More than one ready thread
• LIFO (last in, first out) put ready threads on
  front of the list, and dispatcher takes threads
  from front. Results in starvation.
• Pick one at random Starvation can occur.
• FIFO (first in, first out) put ready threads on
  back of list, pull them off from the front (this
  is what Nachos does). Fair.
• Priority queue keep ready list sorted by
  priority field of TCB. This allows important
  threads to always get the CPU whenever they need
  it.

36
Summary

• Processes have two parts
• Threads (Concurrency)
• Address Spaces (Protection)
• Concurrency accomplished by multiplexing CPU
  Time
• Unloading current thread (PC, registers)
• Loading new thread (PC, registers)
• Such context switching may be voluntary (yield(),
  I/O operations) or involuntary (timer, other
  interrupts)
• Protection accomplished restricting access
• Memory mapping isolates processes from each other
• Dual-mode for isolating I/O, other resources
• Book talks about processes
• When this concerns concurrency, really talking
  about thread portion of a process
• When this concerns protection, talking about
  address space portion of a process

37
Summary

• Threads have two key concepts
• Thread Control Block (TCB)
• Dispatching loop
• Dispatcher selects a runnable thread to be run
  next according to one of the following
  algorithms
• LIFO
• Random
• FIFO
• Priority queue