Installing Redports Slaves with FreeBSD 9.x Root on ZFS using GPT
Contents
1. Starting off
Go through the procedure in clusteradm/NYINetinstall up to but not including "do the install" in the "machines which will boot from hard drive" section.
2. Creating a bootable ZFS Filesystem
- pxeboot the FreeBSD Install image
- Choose LiveCD when asked
- Mount Root filesystem read-writeable to prevent further problems
mount -rw /
- Create GPT Disk
Fixit# gpart create -s gpt da0
Note:
- If the disk already contains a partitioning scheme, and this scheme would first need to be destroyed using:
Fixit# gpart destroy da0
- Create the boot, swap and zfs partitions
Fixit# gpart add -b 40 -s 88 -t freebsd-boot da0 Fixit# gpart add -s 4G -t freebsd-swap -l swap0 da0 Fixit# gpart add -t freebsd-zfs -l disk0 da0
This creates 3 partitions. The first partition will contains the gptzfsboot loader which will be able to load /boot/loader or /boot/zfsloader (9.0-CURRENT) from a ZFS partition. The second partition is a 4 GB swap partition. The third partition is dedicated to the ZFS zpool and uses the remaining space on the disk.
Note:
While a ZFS Swap Volume can be used instead of the freebsd-swap partition, crash dumps can't be saved to the ZFS Swap Volume.
- Install the Protected MBR (pmbr) and gptzfsboot loader
Fixit# gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0
This may fail with an "operation not permitted" error message, since the kernel likes to protect critical parts of the disk. If this happens for you, run:Fixit# sysctl kern.geom.debugflags=0x10
- Load ZFS kernel module
Fixit# kldload opensolaris.ko Fixit# kldload zfs.ko
- Create ZFS Pool zroot
Note:
- zroot is the name of the ZFS Pool, it could be anything (i.e. tank, data, ...)
Fixit# zpool create zroot /dev/gpt/disk0
![/!\](/moin_static1911/modern/img/alert.png "/!\")
We will work around it if you get the error "cannot mount '/zroot': failed to create mountpoint". Fixit# zpool set bootfs=zroot zroot
- Confirm that you have successfully created it via
Fixit# zpool status
For the following "create" steps, ignore the error "filesystem successfully created, but not mounted" Create ZFS filesystem hierarchy.
The fletcher4 algorithm should be more robust than the fletcher2 algorithm.
Fixit# zfs set checksum=fletcher4 zroot
Fixit# zfs create -o compression=on -o exec=on -o setuid=off zroot/tmp
Fixit# zfs create zroot/usr Fixit# zfs create zroot/usr/home
Fixit# zfs create -o compression=lzjb -o setuid=off zroot/usr/ports Fixit# zfs create -o compression=off -o exec=off -o setuid=off zroot/usr/ports/distfiles Fixit# zfs create -o compression=off -o exec=off -o setuid=off zroot/usr/ports/packages
Note:
If you use nullfs or nfs to mount /usr/ports to different locations/servers, you will also need to nullfs/nfs mount /usr/ports/distfiles and/or /usr/ports/packages.
Fixit# zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/usr/src
Fixit# zfs create zroot/var Fixit# zfs create -o exec=off -o setuid=off zroot/var/db Fixit# zfs create -o compression=lzjb -o exec=on -o setuid=off zroot/var/db/pkg Fixit# zfs create -o exec=off -o setuid=off zroot/var/empty Fixit# zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/var/log Fixit# zfs create -o compression=lzjb -o exec=on -o setuid=off zroot/var/tmp
Note:
- Compression may be set to on, off, lzjb, gzip, gzip-N (where N is an integer from 1 (fastest) to 9 (best compresion ratio. gzip is equivalent to gzip-6).
Compression will cause some latency when accessing files on the ZFS filesystems. Use compression on ZFS filesystems which will not be accessed that often.
- Mount filesystems to /mnt
Fixit# zfs set mountpoint=/mnt zroot Fixit# zfs set mountpoint=/mnt/tmp zroot/tmp Fixit# zfs set mountpoint=/mnt/usr zroot/usr Fixit# zfs set mountpoint=/mnt/var zroot/var Fixit# zfs mount -a
- Fine-tune the contents of the filesystems:
Fixit# chmod 1777 /mnt/tmp Fixit# chmod 1777 /mnt/var/tmp Fixit# ln -s /mnt/usr/home /mnt/home
3. Installing FreeBSD to the ZFS filesystem
- Install FreeBSD to zroot using bsdinstall
- When asked about Partitioning choose "shell"
- Just
Fixit# exit
to skip the Partitioning step. - Continue with the install up to the "last chance" step.
4. Finish install
- Exit to the shell. You should see
This shell is operating in a chroot in the new system.
- Enter
Fixit# exit
to leave the chroot. - Edit /mnt/etc/rc.conf
Fixit# echo 'zfs_enable="YES"' >> /mnt/etc/rc.conf
- Create /mnt/boot/loader.conf
Fixit# echo 'zfs_load="YES"' > /mnt/boot/loader.conf Fixit# echo 'vfs.root.mountfrom="zfs:zroot"' >> /mnt/boot/loader.conf Fixit# echo 'console="comconsole,vidconsole"' >> /mnt/boot/loader.conf
- Create /mnt/etc/fstab
Fixit# cat << EOF > /mnt/etc/fstab # Device Mountpoint FStype Options Dump Pass# /dev/gpt/swap0 none swap sw 0 0 EOF
- Unmount all zfs filesystems
Fixit# zfs unmount -a
- Further fine-tune the contents of the filesystems:
Fixit# zfs set readonly=on zroot/var/empty
If /tmp is not r/w, make it so (e.g. via an mdconfig.)
Copy the metadata to the desired destination.
You will not be able to ZFS boot without this! Fixit# cd /tmp Fixit# zpool export -f zroot Fixit# zpool import -o cachefile=/tmp/zpool.cache zroot
- Change mount points for zroot pool
Fixit# zfs set mountpoint=legacy zroot Fixit# zfs set mountpoint=/tmp zroot/tmp Fixit# zfs set mountpoint=/usr zroot/usr Fixit# zfs set mountpoint=/var zroot/var
- Exit Fixit mode, and then bsdinstall.
5. Finishing Up
Go through the the remaining steps in the procedure in clusteradm/NYINetinstall after the "do the install" step in the "machines which will boot from hard drive" section. When you reboot the first time, the system should boot using the ZFS root.