Cluster Boot Issues

1
Cluster Boot Issues

• Shawn Marriott
• COSC 3P93

2
What is a Cluster?

• A cluster is a collection of individual
  autonomous nodes working in concert to create the
  illusion of a single system.

3
The trouble with nodes

• If you must consider the state, and
  configuration of each node in a cluster the
  illusion of the single system is lost. As the
  size of a cluster grows the management of
  individual nodes becomes tedious, and error prone.

4
A Paradox

• How does a node get the information it needs
  to boot, if it needs to boot to get the
  information it needs ?

5
Booting a computer

• Part of the startup procedure for most
  computers is to initialize a bootstrap, which in
  turn starts the operating system. Where to find
  the bootstrap is generally configure on a per
  machine basis in the bios. Common places that are
  searched for the bootstrap are the hard drive(s),
  optical drives, solid state devices, and
  floppies.

6
How do you boot a cluster?

• Or at least convince the nodes to boot.

7
A Sub Ideal Solution

• Manually configure every node in the cluster

8
An ideal solution

• Unpack the computer.
• Plug in the computer.
• Turn on the computer.
• Done.

9
How could this be done?

• If the nodes in our cluster are computers
  connected to a network. Then when a computer
  starts up have it use the network to broadcast
  information about itself to a server, and have
  the server provide configuration information and
  a boot strap.

10
In the beginning.

• There was the Reverse Address Resolution
  Protocol (RARP). RFC 903, 1984.

11
Next try

• Bootstrap protocol (BOOTP). RFC 951, 1985

12
Sample BootP config

• node01htetherha080009030166ip15.19.8.2sm2
  55.255.248.0gw15.19.8.1bf/bootloader
• node02htetherha080009030176ip15.19.8.3sm2
  55.255.248.0gw15.19.8.1bf/bootloader
• node03htetherha080009030186ip15.19.8.4sm2
  55.255.248.0gw15.19.8.1bf/bootloader
• node04htetherha080009030196ip15.19.8.5sm2
  55.255.248.0gw15.19.8.1bf/bootloader

13
Another go around

• Dynamic Host Configuration Protocol (DHCP).
• RFC 1531, 1993.

14
Sample DHCP Configuration

•  
• subnet 192.168.0.0 netmask 255.255.255.0
• range 192.168.0.10 192.168.0.49
• option subnet-mask 255.255.255.0
• option broadcast-address 192.168.0.255
• option routers 192.168.0.1
• filename "pxelinux.0"
• next-server 192.168.0.100

15
One more try?

• In 1999 Intel releases the Wired for
  Management Framework (WMF)
• The framework was mostly ignored, but two
  interesting parts of the specification endure.
• Preboot Execution Environment (PXE).
• Boot Integrity Services (BIS).

16
What is PXE?

• PXE is a client designed to work with DHCP and
  TFTP to retrieve boot strap information, it also
  defines APIs to allow a loaded boot strap to
  query a local host for additional information

17
What is BIS?

• BIS enables a PXE client to examine a
  digitally signed boot image. This provides a
  mechanisms to verify the integrity of a supplied
  boot strap image.

18
How is PXE implemented?

• If you want to use a network to provide
  configuration information for a node, then you
  better have a network card (NIC).
• Since you must have a network card, it seems
  reasonable to put the PXE client on the network
  card, and have the bios treat network cards as a
  bootable device.

19
Intels vision
20
What is a Network Bootstrap Program(NBP)?

• NBP is a binary executable file, specific to a
  give CPUs architecture.
• They are small, usually less than 512KB in size.
• What an NBP does is up to whoever creates it. The
  PXE specification does not go into any detail on
  NBP.

21
PXE Work flow with an NLB
22
What is involved in setting up a PXE environment?

• A Node with a PXE client set as the boot device.
• A DHCP server
• TFTP server

23
Sample DHCP Configuration

•  
• subnet 192.168.0.0 netmask 255.255.255.0
• range 192.168.0.10 192.168.0.49
• option subnet-mask 255.255.255.0
• option broadcast-address 192.168.0.255
• option routers 192.168.0.1

24
Sample proxy DHCP config

• subnet 192.168.0.0 255.255.0.0
• vendor pxe
• bootstrapserver 192.168.0.100
  TFTP server ip address.
• Type, SystemArch, MajorVers
• pxebootfile 1 2 1 window.one 1
  0
• pxebootfile 2 2 1 linux.one 2
  3
• pxebootfile 1 2 1 hello.one 3
  4
• client 6 10005a8ad14d
• pxebootfile 1 2 1 aix.one
  5 6
• pxebootfile 2 2 1 window.one
  6 7

25
What PXE does right

• It is a standard feature on modern NICS, and
  mother boards with integrated NICS
• A PXE client knows the architecture of the node
  it is running on, and can request an appropriate
  NBP
• It builds upon existing technologies (DHCP, TFTP)
• It has extensions to authenticate the boot
  server, and verify the integrity of a downloaded
  NBP.

26
The problems with PXE

• PXE has limited knowledge of its host.
• PXE relies on DHCP, which is inefficient on large
  networks
• PXE relies on TFTP, which is impractical for
  large files, or many concurrent file transfers
• If you need different NBPs for different nodes
  you must uniquely identify each node(By MAC
  address) and group them accordingly in DHCP or
  Use a DHCP proxy server and separate the
  management of network address and images

27
Questions and Discussion
28
References Thomas L. Sterling , 1998. Beowulf
Cluster Computing with Linux. Cambridge,
Massachusetts The MIT Press Intel Corporation,
1998. Boot Integrity Services Application
Programming. Version 1.0. ftp//download.intel.com
/design/archives/wfm/downloads/bisspec.pdf Intel
Corporation, 1999. Preboot Execution Environment
(PXE) Specification. Version 2.1. http//download.
intel.com/design/archives/wfm/downloads/pxespec.pd
f http//www.ietf.org/rfc/rfc903.txt http//www.i
etf.org/rfc/rfc951.txt http//www.ietf.org/rfc/rfc
1531.txt http//www.beowulf.org http//en.wikipedi
a.org/wiki/Preboot_Execution_Environment http//pu
blib.boulder.ibm.com/infocenter/pseries/v5r3/index
.jsp