CMSC 412 Operating Systems Fall 2002

1
CMSC 412Operating SystemsFall 2002

• Liviu Iftode
• iftode_at_cs.umd.edu

2
Course overview

• Goals

• Understand how an operating system works as a
  mediator between the computer architecture and
  user programs
• Learn how OS concepts are implemented in a real
  operating system
• Introduce to systems programming
• Learn about performance evaluation
• Learn about current trends in OS research

3
OS Learning
project
OS Implementation
OS Concepts
(lectures, textbooks)
(source code, project doc)
recitations
homeworks
Real OS
OS Programming
(man pages)
(Unix/Linux textbooks)
4
Course Timeline


Concept B
Concept C
Concept A
Lecture
Tu
Th
Tu
Th
Tu
Th
Apply B
Apply A
Apply C
Lab
W
W
W
HW A Due
HW BC Due
Homework
W
W
Project AB Due
Project
W
5
Suggested Approach

• Read the assigned chapter from the textbook
  before the lecture to understand the basic idea
  and become familiar with the terminology
• Attend the recitation
• Start homeworks and project right away, systems
  programming is not easy !
• Ask questions during lecture, recitation.
• Use the mailing list/newsgroup for discussions,
  do not be afraid to answer a question posted by
  your colleague even if you are not sure. This is
  a way to validate your understanding of the
  material. Do not forget, questions and
  discussions are not graded !

6
Course Outline

• Processes and process management
• Threads and thread programming
• Synchronization and Deadlock
• Memory management and Virtual Memory
• CPU Scheduling
• File systems and I/O Management
• Networking and Distributed Systems
• Security

7
Course requirements

• Prerequisites

• computer architecture
• good programming skills (C!!, C, Java)

Expected work

• substantial readings (textbooks and papers)
• challenging project extended over the entire
  semester
• homeworks (require programming)
• midterm and final exams

8
Work Evaluation

• midterm exam 25
• homework 25
• project 25
• final exam 25

9
Homeworks

• Goals

• Deepen the understanding of OS concepts
• Develop systems programming skills virtual
  memory, threads, synchronization, sockets
• Learn to design, implement, debug and evaluate
  the performance of an OS-bound program

Structure

• 4-5 homeworks
• Both theoretical and C-programming problems

10
Project

• Goals

• learn to design, implement and evaluate basic OS
  mechanisms and policies

Structure

• individual project
• multiple phases
• project report for each phase

11
Textbooks

• Stallings Operating Systems. Internals and
  Design Principles, 4th Edition,
  Prentice-Hall, 2001.
• Silberschatz, Galvin and Gagne, Operating System
  Concepts, 6th Edition, John Wiley Sons,
  2001.
• Papers will be made available on the course
  homepage

12
Logistics

• TAs Chunyuan Liao Iulian
  Neamtiu (to be confirmed)
• Course homepage
• http//www.cs.umd.edu/iftode/cs412/cs412sylla
  bus.htm
• Preliminary schedule and course notes are already
  available for the entire semester but they may be
  updated before each class.
• Homeworks every other week.
• A project phase every other week.
• A mailing list/newsgroup will be announced
  shortly.

13
What is an operating system
application (user)
operating system
hardware

• a software layer between the hardware and the
  application programs/users which provides a
  virtual machine interface easy and safe
• a resource manager that allows programs/users to
  share the hardware resources fair and efficient

14
How does an OS work
application (user)
system calls
upcalls
hardware independent
OS
commands
interrupts
hardware dependent
hardware

• receives requests from the application system
  calls
• satisfies the requests may issue commands to
  hardware
• handles hardware interrupts may upcall the
  application
• OS complexity synchronous calls asynchronous
  events

15
Files
A file is a storage abstraction
application/user copy file1 file2
naming, protection, operations on files
operating system files, directories
operations on disk blocks...
hardware disk

• traditional approach OS does disk block
  allocation and caching (buffer cache) , disk
  operation scheduling and replacement in the
  buffer cache
• new approaches application-controlled cache
  replacement, log-based allocation (makes writes
  fast)

16
Traditional file system
application read/write files
translate file to disk blocks
OS
...
...buffer cache
maintains
controls disk accesses read/write blocks
hardware
17
Mechanism vs Policy
application (user)
operating system mechanismpolicy
hardware

• mechanism data structures and operations that
  implement the abstraction (e.g. the buffer cache)
  
• policy the procedure that guides the selection
  of a certain course of action from among
  alternatives (e.g. the replacement policy for the
  buffer cache)
• traditional OS is rigid mechanism together with
  policy

18
Mechanism-policy split

• traditional OS cannot provide the best policy in
  all cases
• new OS approaches separate mechanisms from
  policies

• OS provides the mechanism some policy
• applications contribute to the policy

• flexibilityefficiency require new OS structures
  and/or new OS interfaces

19
Application-controlled caching
application read/write files
replacement policy
translate file to disk blocks
OS
...
...buffer cache
maintains
controls disk accesses read/write blocks
hardware
20
Processes
A process is a processor abstraction
user run application
create, kill processes, inter-process comm
operating system processes
context switch
hardware processor

• traditional approach OS switches processes on
  the processor (process scheduling), provides
  inter-process communication and handles
  exceptions
• new approaches application-controlled
  scheduling, reservation-based scheduling, agile
  applications

21
Traditional approach
active
processes
IPC
OS
processor

• OS mediates inter-process communication (IPC)
• OS schedules processes on the processor
• application provides hints priorities

22
Hierarchical scheduling
schedulers
processes
OS
processor

• OS schedules schedulers which schedule dependent
  processes

23
Virtual memory
Virtual memory is a memory abstraction
application address space
virtual addresses
operating system virtual memory
physical addresses
hardware physical memory

• traditional approach OS provides a sufficiently
  large virtual address space for each running
  application, does memory allocation and
  replacement and may ensure protection
• new approaches external memory management, huge
  (64-bit) address space, global memory

24
VM mechanism and policy
virtual address spaces
p1
p2
processes
v-to-p memory mappings
physical memory

• processes can run being partially loaded in
  memory
• illusion of more memory than physically
  available swapping
• processes can share physical memory if permitted
• replacement policy can be exposed to the
  application

25
Communication
Message passing is a communication abstraction
application sockets
naming, messages
operating system TCP/IP protocols
network packets
hardware network interface

• traditional approach OS provides naming schemes,
  reliable transport of messages, packet routing to
  destination
• new approaches zero-copy protocols, active
  messages, memory-mapped communication

26
Traditional OS structure

• monolithic/layered systems

• one/N layers all executed in kernel-mode
• good performance but rigid

user process
user
system calls
memory system
OS kernel
file system
hardware
27
Micro-kernel OS
client process
file server
memory server
user mode
IPC
micro-kernel
hardware

• client-server model, IPC between clients and
  servers
• the micro-kernel provides protected communication
• OS functions implemented as user-level servers
• flexible but efficiency is the problem
• easy to extend for distributed systems

28
Extensible OS kernel
process
process
user mode
my memory service
default memory service
extensible kernel
hardware

• user processes can load customized OS services
  into the kernel
• good performance but protection and scalability
  become problems

29
Virtual Machines

• old concept which is heavily revived today

• the real hardware in cloned in several
  identical virtual machines
• OS functionality built on top of the virtual
  machine

user
allocate resource
OS on virtual machine
exokernel
hardware
30
Computer System

• Processor performs data processing
• Main memory stores both data and programs,
  typically volatile
• Disks secondary memory devices which provide
  persistent storage
• Network interfaces inter-machine communication
• Buses intra-machine communication

• memory bus (processor-memory)
• I/O bus (disks, network interfaces, other I/O
  devices, memory-bus)

31
Basic computer structure
Memory
CPU
memory bus
I/O bus
disk
Net interface
32
Memory caches

• motivated by the mismatch between processor and
  memory speed
• closer to the processor than the main memory
• smaller and faster than the main memory
• act as attraction memory contains the value
  of main memory locations which were recently
  accessed (temporal locality)
• transfer between caches and main memory is
  performed in units called cache blocks/lines
• caches contain also the value of memory locations
  which are close to locations which were recently
  accessed (spatial locality)

33
Cache design issues
cpu
word transfer
cache
block transfer
main memory

• cache size and cache block size
• mapping physical/virtual caches, associativity
• replacement algorithm LRU
• write policy write through/write back

34
Memory Hierarchy
cpu
word transfer

• decrease cost per bit
• decrease frequency of access
• increase capacity
• increase access time
• increase size of transfer unit

cache
block transfer
main memory
page transfer
disks
35
Data transfer on the bus
CPU
Memory
cache
memory bus
I/O bus
disk
Net interface

• cache-memory cache misses, write-through/write-b
  ack
• memory-disk swapping, paging, file accesses
• memory-Network Interface packet send/receive
• I/O devices to the processor interrupts

36
Direct Memory Access (DMA)

• bulk data transfer between memory and an I/O
  device (disk, network interface) initiated by the
  processor

• address of the I/O device
• starting location in memory
• number of bytes
• direction of transfer (read/write from/to memory)

• processor interrupted when the operation
  completes
• bus arbitration between cache-memory and DMA
  transfers
• memory cache must be consistent with DMA

37
Multiprocessors
CPU
CPU
Memory
cache
cache
memory bus
I/O bus
disk
Net interface

• simple scheme more than one processor on the
  same bus
• memory is shared among processors-- cache
  consistency
• bus contention increases -- does not scale
• alternative (non-bus) system interconnect --
  expensive
• single-image operating systems

38
Multicomputers
CPU
CPU
Memory
Memory
cache
cache
memory bus
memory bus
I/O bus
I/O bus
network
disk
Net interface
disk
Net interface

• network of computers share-nothing -- cheap
• communication through message-passing difficult
  to program
• challenge build efficient shared memory
  abstraction in software
• each system runs its own operating system