Raspberry PI: network boot explained
The network booting process has three phases:
- get an IP address via DHCP
- load kernel and configuration files using TFTP
- mount root partition by NFS
This example-tutorial uses a Raspberry PI as a boot client and a Debian server (maybe another Raspberry PI running Raspian). The tools needed on the server are dnsmasq and nfs-kernel-server. The required packets can be loaded in this way:
# apt-get install dnsmasq nfs-kernel-server
Any Raspberry PI can be net-booted
...but it needs an SD card. Only Raspberry PI 3 supports network booting without any card (although I have not tested it yet).
The SD card needs to contain one FAT partition and inside it one file: bootcode.bin. The latest version of bootcode.bin can be dowloaded from here. It is a 50K file so any size of SD Card is okay. Even a partially damaged SD card or a fake SD card can be used for this purpose. This file will be read at boot time and never modified so the card should not be further damaged.
Further information on the boot modes can be retrieved from the boot modes page on the raspberry pi foundation documentation site.
Phase 1: DHCP
The goal of DHCP is to provide our Raspberry PI with an IP address and inform it that the server is available for the next steps of the network boot
The Raspberry PI MAC address is required to provide it with its specific IP address.
I suggest to use wireshark (clearly on the server) to get this information: a view on the network packets flowing on the net can teach a lot about how the boot process really works.
Start wireshark, put the SD Card in the Raspberry PI, connect the Raspberry PI to the network, and power it on.
The screenshot shows the DHCP request. The MAC address of my Raspberry PI is: b8:27:eb:15:e4:ef
The next step is to configure a DHCP server to provide the Raspberry PI with its own IP address.
In this test I'll use DNSmasq.
This is the configuration to assign the IP address 192.168.1.100 to the MAC b8:27:eb:15:e4:ef on the interface eth0.
interface=enx0050b6175c07 bind-interfaces dhcp-range=192.168.1.100,192.168.1.100,255.255.255.0,1h dhcp-host=b8:27:eb:15:e4:ef,192.168.1.100,1h pxe-service=0,"Raspberry Pi Boot" log-dhcp
The option pxe-service notifies the Raspberry pi that this server supports the net boot.
Restart dnsmasq:
sudo systemctl restart dnsmasq.service
restart wireshark and powercycle the Raspberry PI.
If everything is working properly now the Raspberry PI receives a DHCP offer and tries to continue with the following step (and it fails as we have not configured TFTP, yet).
Here is the screenshot:
Packet #39 is the DHCP reply whih assigns the address to the Raspberry PI, then packet #42 tries to load the file "6c15e4ef/start.elf" via TFTP...
We are ready for the second phase.
Phase 2: TFTP
TFTP means trivial file transfer protocol. Nothing really complex can be used at boot time, the code to manage TFTP at boot time needs to fit in the BIOS ROM or, in our case, in the 50K bootcode.bin file.
DNSmasq is able to provide TFTP too.
Let us enable TFTP: it is possible just by add two lines in /etc/dnsmasq.conf:
enable-tftp tftp-root=/tftpboot
The tftp-root parameter defines the root directory for tftp: any file inside this directory (and in its subdirectories) will be available for downloading via TFTP. Feel free to put it where it is more suitable in your file system.
The files needed to boot a Raspberry PI can be stored in the TFTP root directory or in directories named upon the Raspberry PI serial number/MAC address. In the former case the files will be shared between all the raspberry pis using the net-boot service on the same server, the latter choice permits to define specific boot files for each client.
From the wireshark trace above it is possible to read that the specific directory for my RPI is "6c15e4ef".
The files needed to boot a raspberry PI are those available in the boot partition of a standard disk image.
It is possible to write an image on an SD card and then copy all the contents of the first partition (FAT). The following commands show how to take the files directly from a disk image, in the example from 2017-11-29-raspbian-stretch-lite.img (the latest image can be downloaded for here).
sudo losetup -f --show -P 2017-11-29-raspbian-stretch-lite.img sudo mount -o ro /dev/loop0p1 /mnt sudo mkdir -p /tftpboot/6c15e4ef sudo cp -a /mnt/* /tftpboot/6c15e4ef sudo umount /mnt sudo losetup -d /dev/loop0
The first command (losetup) creates a loopback to mount a partition of a disk image. The second command mounts the first partition in read-ony mode as /mnt. Then mkdir creates the directory in the tftp root, cp copies all the tree from mnt to the newly created directory. The last two commands undo the effects of the first and second.
The cmdline to start the kernel need to be modified in order to mount the root partition via nfs. Here the /tftpboot/6c15e4ef/cmdline file has the following contents:
dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=/dev/nfs nfsroot=192.168.1.1:/srv/nfs/raspianroot,nfsvers=3 rw ip=dhcp rootwait elevator=deadline
The IP address and path of the directory exported as root partition need to be changed to suit your environment.
Restart dnsmasq,
sudo systemctl restart dnsmasq.service
and power cycle the raspberry PI client.
If you have a monitor connected to the Raspberry PI now if everything is working properly it boots and.... fails. It should terminate (after a while) with Kernel Panic complaining that it is unable to mount the root partition. It is correct as the third phase is still missing.
Phase 3: NFS
Network File System service must be configured to provide the root partition to our Raspberry PI.
Edit /etc/exports, the line to support the Raspberry PI client should sound as:
/srv/nfs/raspianroot 192.168.1.100/255.255.255.255(rw,sync,insecure,no_subtree_check,no_root_squash)
as ususal IP addresses and root directory are specific to this example and should be modified as required.
Create the directory and copy in it all the contents of the second partition of the raspian image.
sudo mkdir -p /srv/nfs/raspianroot sudo losetup -f --show -P 2017-11-29-raspbian-stretch-lite.img sudo mount -o ro /dev/loop0p2 /mnt sudo cp -a /mnt/* /srv/nfs/raspianroot sudo umount /mnt sudo losetup -d /dev/loop0
Edit the file /etc/fstab in the exported root and delete all the lines beginning with PARTUUID (otherwise it tries to mount the partitions from the SD card). At the end the file /srv/nfs/raspianroot/etc/fstab can have only one line:
proc /proc proc defaults 0 0
restart the nfs server:
sudo exportfs -a sudo systemctl restart nfs-kernel-server.service
Now power cycle the Raspberry PI and the boot process should be complete up to the login prompt.
Mount /boot via NFS
This is an optional extra feature. It is possible to mount via NSF the boot partition (the one whose file have been loaded via TFTP at boot time).
This enhancement permits to the client to manage and update its own kernel and configuration as if it where a standard Raspberry PI using its own SD card.
In order to reach this goal one line must be added to /etc/exports:
/tftpboot/6c15e4ef 192.168.1.100/255.255.255.255(rw,sync,insecure,no_subtree_check,no_root_squash)
and a line in the client's fstab /srv/nfs/raspianroot/etc/fstab:
192.168.1.1:/tftpboot/6c15e4ef /boot nfs4 defaults 0 0
When /etc/exports is updated, exportfs is needed to inform the NFS server of the new configuration:
sudo exportfs -a
Enable SSH on a headless netbooted RPI
Just add an empty file in the /boot directory in the nfs root. For this example:
sudo touch /srv/nfs/raspianroot/boot/ssh
Final Remarks
If something is not working properly, the main debug tools are wireshark and tail -f /var/log/syslog
In fact the analysis of the network traffic and the log file on the server can show what is going wrong.
Still to be tested
- RPI 3 booting without any SD card