Diskless, Low-Form-Factor OpenBSD Systems

In a we built a tiny OpenBSD system out of a Soekris miniature PC, a bootstrap workstation, and a Compact Flash (CF) card. While this combination works nicely for many purposes, once you have Soekrii scattered all around your network, managing the CF cards can be annoying. Replacing the CF cards with a diskless boot system eases management problems. One modest server can manage many small diskless devices, and it's possible to do much of the system administration on the server instead of on the devices. This time, we'll build an OpenBSD diskless device; netbooting FreeBSD and NetBSD will have their own future articles.

Your server must be a NFS server capable of running dhcpd(8), rarpd(8), and tftpd(8). The server operating system is irrelevant; any BSD, Linux, or even a commercial UNIX will work. I chose to use a FreeBSD server, simply because I had spare capacity on a server-grade system. (My OpenBSD bootstrap station is a Celeron 433 with a 10GB hard drive. The original desktop user complained that it was unusable, but the problems disappeared when we removed the original operating system. It's perfectly adequate for any free UNIX, but not exactly what you want in a server.)

Experimenting with an NFS server, rarpd, and tftpd probably won't affect your environment, but doing unfamiliar things to the DHCP server most certainly can, so the main corporate LAN might not be a wise choice for your test bed. For my initial experiments in diskless booting, I installed a second network card in my diskless server and ran a crossover cable between that NIC and the Soekris box. Running a private DHCP server on that interface allowed me to experiment freely without corrupting the corporate network. While developers asking, "Why did my machine boot something other than Windows?" would be amusing, it would cause too many meetings.

Soekris Preparation

To configure your Soekris box to run properly diskless, you need the MAC address of its Ethernet port. Find this by booting the Soekris with a CF card and running ifconfig(8), or by using a packet sniffer such as tcpdump or Ethereal. When the Soekris boots, it will attempt to grab its IP address via DHCP; you can always watch the DHCP logs.

All of the rest of our work takes place on the server. (That's kind of the point, you know.)

Bootstrapping

In the first stage of a diskless boot (as described in diskless(8)), a PROM or stage-1 bootstrap fetches a boot program from the diskless boot server. Where can you find such a bootstrap for OpenBSD/i386? Google reveals a variety of sources, but I chose to go with GRUB for OpenBSD. . (This site also has GRUB versions modified to netboot OpenBSD on an Intel "fxp" network card, if you're interested in playing with diskless operations on a standard PC.) Grab the pxegrub.sis-tty-19200 file and put it somewhere safe.

The Soekris will use both dhcpd(8) and rarpd(8) to do its initial configuration. rarpd will provide the basic IP address information, while dhcpd will provide boot-time configuration information. Let's start by configuring rarpd.

rarpd(8) handles reverse ARP requests. In a normal ARP request, a machine has an IP address and requests the matching Ethernet MAC address. In reverse ARP, a machine has a MAC address and requests the corresponding IP address. The Soekris knows its own Ethernet MAC address and wants an IP address.

Different rarpd(8) implementations have different features and requirements; cursory checks for FreeBSD, OpenBSD, and Red Hat Linux show different behaviors and different command-line flags. For example, the FreeBSD rarpd I used expected to be able to provide a boot loader to the client, and by default ignored requests for which it had no boot loader. I don't want to use rarpd(8) to provide the loader information, so I had to use the -s command-line switch to tell the program to provide answers only for those MAC addresses for which it had a mapping. Red Hat Linux also defaults to checking for a bootable image, but uses a different command-line switch to just provide service. OpenBSD's rarpd(8) simply provides the RARP mapping service without trying to provide a bootable image.

I won't discuss differences in every program we require under every operating system; the point is, check your server's man pages!

All common rarpd(8) implementations use /etc/ethers to map MAC addresses to hostnames. My /etc/ethers contains the single line:

00:00:24:c1:35:18       soekris-diskless.blackhelicopters.org

The hostname soekris-diskless.blackhelicopters.org must be available in either DNS or /etc/hosts, so that the RARP server can get the information.

DHCP Configuration

Next, the diskless machine will attempt to fetch its boot loader information from the DHCP server. Here, I'm using isc-dhcpd3.

group soekris {
        filename "pxegrub.sis-tty-19200";

        host soekris-diskless {
                hardware ethernet 00:00:24:c1:35:18 ; 
                fixed-address 192.168.1.88 ;
        }
}

While we don't need to group all of our diskless clients together, it will make further expansion simpler. The filename keyword means "you should go grab this file from the TFTP server." All of our Soekrii need to know about it, so it goes under the group heading.

Each diskless client also needs a separate entry for its MAC address and its IP address.

TFTP Configuration

Once the diskless Soekris has figured out its IP address and where to get its boot loader, it will attempt to grab that boot loader via TFTP. All modern, free, UNIX-like operating systems include a TFTP implementation, usually run out of inetd(8). Again, the manual pages for all of the tftpd(8) servers I checked have slightly different options. On my FreeBSD server, I decided to use /var/tftpboot as the TFTP root directory, symlinked /tftpboot to it, and then set up /etc/inetd.conf like this:

tftp  dgram  udp  wait  root  /usr/libexec/tftpd   tftpd -lns /var/tftpboot

The -l flag tells tftpd(8) to log all requests, which is very helpful when debugging. The -s chroots tftpd(8), and the -n tells tftpd to be as quiet as possible.

Now, copy pxegrub.sis-tty-19200 into the tftpboot directory. The DHCP server will tell the Soekris to grab and run this file. pxegrub needs a configuration file: grub.conf. Here's a working example.

default=0
timeout=5
serial --unit=0 --speed=19200
terminal serial
title soekris
kernel //bsd 
root (nd)
boot

While you shouldn't have to change any of this for a Soekris, you might wish to edit the kernel line. This example makes pxegrub look for a file called bsd on the tftpd server and attempt to load it as the kernel. Switch between kernels by changing this entry.

If you're not using a Soekris and wish to change some of the settings (for example, to use the monitor and keyboard instead of a serial console), check the for guidance. Note that the default GRUB configuration file is called menu.lst, not grub.conf; while pxegrub has changed the configuration file name, the settings within the file are all identical to standard GRUB.

Final Setup

The kernel will boot the system and should recognize its basic hardware. At the point that it needs to mount a root file system, the system will automatically broadcast a request for its boot parameters across the local network. Your NFS server needs to respond to this request via bootparamd(8). bootparamd's entire reason for being is to answer broadcast configuration requests. These configurations are stored in /etc/bootparams.

diskless-soekris.blackhelicopters.org root=192.168.1.1:/var/obsd/diskless-soekris/root

The first entry on a line is the fully qualified host name of the diskless machine. In this example, our second entry is the path to the NFS-exported root directory for the diskless machine. You can also export swap and dump device information, but we only need the root directory to start.

bootparamd(8) requires rpcbind(8). On FreeBSD, I run both of these daemons with the -s flag, which causes them to drop privileges as soon as possible. (One mistake I made initially was to have rpcbind attach only to the IP address of the server machine, which precludes it from answering broadcast requests. The lesson is, don't be too secure too quickly when you're trying to make this work!)

Exports? That's right, we need NFS! Fortunately, this is very easy to set up for a single client. The root of our file system is in /var/obsd/diskless-soekris/root and we only need to export it to a single host. Here's an /etc/exports for our FreeBSD server.

/var/obsd/diskless-soekris/root -maproot=root 192.168.1.88

We'll also have to start the NFS server daemons at boot. By the time we're done, the FreeBSD server has the following set in /etc/rc.conf:

inetd_enable="YES"
nfs_server_enable="YES"
nfsd_enable="YES"
mountd_enable="YES"
rpcbind_enable="YES"
rpcbind_flags="-s"
bootparamd_enable="YES"
bootparamd_flags="-s"
rarpd_enable="YES"
rarpd_flags="-s fxp1"

Put the proper interface name in the rarpd_flags field for your server.

Filesystem Setup

You then need an OpenBSD file system under your exported root. That's actually quite easy to do. You could just mount the root directory from your OpenBSD bootstrap station and copy the entire system with tar -xvpf, but that will give you everything on your bootstrap station. (The -p flag is important; it preserves ownership information.) Another way to get a small image is to use the file system prepared for a Compact Flash system; you'll have only the minimal files required, but can easily add more. Copy over the /etc/rc.* files from your bootstrap station anyway, as OpenBSD's startup system has specific features for diskless operations. You will also have to do some debugging of the startup process, however, and create some directories under /var if you choose this option. Finally, you could simply extract some OpenBSD distribution tarballs in that directory and install vital /etc files from your bootstrap station.

The DISKLESS Kernel

OpenBSD provides a kernel configuration specifically for i386 diskless operations. It's called DISKLESS. Remember, the OpenBSD folks are primarily interested in supporting the GENERIC kernel; you'll have minimal support if you use a non-GENERIC kernel. However, in my experience, the DISKLESS kernel configuration is quite stable and reliable.

As long as you're on a custom kernel anyway, you can make your life a little easier by equalizing the serial console speeds. The Soekris uses a serial console speed of 19200, while OpenBSD defaults to 9600. If you connect to the serial console at 19200, the Soekris boot messages will be legible, but the OpenBSD console will show only garbage. If you connect at 9600, you'll get the OpenBSD messages, but the Soekris information will be illegible. If you add a few kernel options to make OpenBSD speak at 19200, everyone will just magically get along.

option          COMCONSOLE
option          CONSPEED=19200
option          CONSOLE=com0

With this addition, our Soekris box is trivial to manage. When the time comes to perform a system upgrade, you can move aside the current root file system and replace it with the new one. This also makes reverting a bad upgrade trivial. The Soekris can mount the root file system as read-only, and you can still easily edit files on the server. Soekris boxes make excellent firewalls or other types of network devices; with diskless operations, they can also become small network servers, with very little increase in administrative overhead.

lives in a haunted house in Detroit, Michigan with his wife Liz, assorted rodents, and a multitude of fish. He's the author of and , and is currently preparing a book about NetBSD.