Installing FreeBSD Root on ZFS using a FreeBSD MBR Slice

This guide is a work in progress and hasn't been fully tested

Fails to boot 

BTX loader 1.00  BTX version is 1.02
Consoles: internal video/keyboard
BIOS drive C: is disk0
BIOS 638kB/2094720kB available memory

FreeBSD/i386 bootstrap loader, Revision 1.1
(root@, Sat Jul 18 10:58:41 UTC 2009

can't load 'kernel'

Type '?' for a list of commands, 'help' for more detailed help.

Contents

  1. Creating a bootable ZFS Filesystem
  2. Installing FreeBSD to the ZFS filesystem
  3. Finish install

1. Creating a bootable ZFS Filesystem

  1. Boot FreeBSD install DVD or USB Memstick
  2. Choose Fixit option in sysinstall

  3. Create MBR disk

    If you are installing on an New drive, you will need to create the MBR geometry first: 

     Fixit# gpart create -s mbr ad0

  4. Create Slice using gpart

    This example already has two slices defined on the drive. 

     Fixit# gpart show ad0
 =>       63  625142385  ad0  MBR  (289G)
          63  209712447    1  !7  (100G)
   209712510     417690    2  !136  (204M)
   210130200  415012248       - free -  (198G)
    Create a third slice then create a BSD partition table on the new slice. 
     Fixit# gpart add -t freebsd ad0
 ad0s3 added
    And examine the result: 
     Fixit# gpart show ad0
 =>       63  625142385  ad0  MBR  (289G)
          63  209712447    1  !7  (100G)
   209712510     417690    2  !136  (204M)
   210130200  415012248    3  freebsd  (198G)
    Sizes and offsets are specified in sectors (1 sector is typically 512 bytes) but can also be specified in MB or GB by adding an 'M' or 'G' suffix.

  5. Set partition active (Optional)

    Some BIOSes might require setting a Slice active. Here we set partition 1 (the Windows Partiton) as the active partition. 

     Fixit# gpart set -a active -i 1 ad0
  6. Load ZFS kernel module 
     Fixit# kldload /mnt2/boot/kernel/opensolaris.ko
 Fixit# kldload /mnt2/boot/kernel/zfs.ko
  7. Create ZFS Pool zroot 
     Fixit# mkdir /boot/zfs
 Fixit# zpool create zroot /dev/ad0s3
 Fixit# zpool set bootfs=zroot zroot
    NOTE: zroot is the name of the ZFS Pool, it could be anything (i.e. tank, data, ...)
  8. Create ZFS Swap volume 
     Fixit# zfs create -V 2gb zroot/swap
 Fixit# zfs set org.freebsd:swap=on zroot/swap
 Fixit# zfs set checksum=off zroot/swap
  9. Install boot Manager 
     Fixit# gpart bootcode -b /mnt2/boot/boot0 ad0
 ad0 has bootcode
    It's ok if you get an "unknown class" message here, but it you get "failed to install" or the like, you'll need to try again after making sure that nothing is mounted from /dev/ad0.

  10. Install ZFS boot
    - Export zroot, before installing boot code 

     Fixit# zpool export zroot
    - Install the boot1 stage: 
     Fixit# dd if=/mnt2/boot/zfsboot of=/dev/ad0s3 count=1
    This may fail with an "operation not permitted" error message, since the kernel likes to protect critical parts of the disk. If this happens for you, run: 
     Fixit# sysctl kern.geom.debugflags=0x10
    - Install the boot2 zfs stage into the convienient hole in the ZFS filesystem on-disk format which is located just after the ZFS metadata (this is the seek=1024). 
     Fixit# dd if=/mnt2/boot/zfsboot of=/dev/ad0s3 skip=1 seek=1024
    - Import zroot to continue the install 
     Fixit# zpool import zroot

2. Installing FreeBSD to the ZFS filesystem

  1. Create ZFS filesystem hierarchy

    The fletcher4 algorithm should be more robust than the fletcher2 algorithm. 

     Fixit# zfs set checksum=fletcher4                                      zroot
     Fixit# zfs create -o compression=on    -o exec=on      -o setuid=off   zroot/tmp
 Fixit# chmod 1777 /zroot/tmp
     Fixit# zfs create                                                      zroot/usr
 Fixit# zfs create                                                      zroot/usr/home
 Fixit# cd /zroot ; ln -s /usr/home home
     Fixit# zfs create -o compression=lzjb                  -o setuid=off   zroot/usr/ports
 Fixit# zfs create -o compression=off   -o exec=off     -o setuid=off   zroot/usr/ports/distfiles
 Fixit# zfs create -o compression=off   -o exec=off     -o setuid=off   zroot/usr/ports/packages

    Note:

    If you use nullfs or nfs to mount /usr/ports to different locations/servers, you will also need to nullfs/nfs mount /usr/ports/distfiles and/or /usr/ports/packages. 

     Fixit# zfs create -o compression=lzjb  -o exec=off     -o setuid=off   zroot/usr/src
     Fixit# zfs create                                                      zroot/var
 Fixit# zfs create -o compression=lzjb  -o exec=off     -o setuid=off   zroot/var/crash
 Fixit# zfs create                      -o exec=off     -o setuid=off   zroot/var/db
 Fixit# zfs create -o compression=lzjb  -o exec=on      -o setuid=off   zroot/var/db/pkg
 Fixit# zfs create                      -o exec=off     -o setuid=off   zroot/var/empty
 Fixit# zfs create -o compression=lzjb  -o exec=off     -o setuid=off   zroot/var/log
 Fixit# zfs create -o compression=gzip  -o exec=off     -o setuid=off   zroot/var/mail
 Fixit# zfs create                      -o exec=off     -o setuid=off   zroot/var/run
 Fixit# zfs create -o compression=lzjb  -o exec=on      -o setuid=off   zroot/var/tmp
 Fixit# chmod 1777 /zroot/var/tmp

    Note:

    1. Compression may be set to on, off, lzjb, gzip, gzip-N (where N is an integer from 1 (fastest) to 9 (best compression ratio. gzip is equivalent to gzip-6).

    2. Compression will cause some latency when accessing files on the ZFS filesystems. Use compression on ZFS filesystems which will not be accessed that often.

  2. Install FreeBSD to zroot 
     Fixit# cd /dist/8.0-*
 Fixit# export DESTDIR=/zroot
 Fixit# for dir in base catpages dict doc games info lib32 manpages ports; \
          do (cd $dir ; ./install.sh) ; done
 Fixit# cd src ; ./install.sh all
 Fixit# cd ../kernels ; ./install.sh generic
 Fixit# cd /zroot/boot ; cp -Rlp GENERIC/* /zroot/boot/kernel/
  3. Make /var/empty readonly 
     Fixit# zfs set readonly=on zroot/var/empty
  4. chroot into /zroot 
     Fixit# chroot /zroot
  5. Create /etc/rc.conf 
     Fixit# echo 'zfs_enable="YES"' > /etc/rc.conf
 Fixit# echo 'hostname="beastie.mydomain.local"' >> /etc/rc.conf
 Fixit# echo 'ifconfig_re0="DHCP"' >> /etc/rc.conf

    Note:

    • Replace re0 with the name of the Network interface for the new system
  6. Create /boot/loader.conf 
     Fixit# echo 'zfs_load="YES"' > /boot/loader.conf
 Fixit# echo 'vfs.root.mountfrom="zfs:zroot"' >> /boot/loader.conf
  7. Change root's password 
     Fixit# passwd
  8. Set the local time zone 
     Fixit# tzsetup
  9. Create /etc/mail/aliases.db 
     Fixit# cd /etc/mail
 Fixit# make aliases
  10. Exit from the /zroot 
     Fixit# umount /dev
 Fixit# exit
  11. Install zpool.cache to the ZFS filesystem 
     Fixit# cp /boot/zfs/zpool.cache /zroot/boot/zfs/zpool.cache

3. Finish install

  1. Create /etc/fstab 
     Fixit# cat << EOF > /zroot/etc/fstab
 # Device                       Mountpoint              FStype  Options         Dump    Pass#
 EOF
  2. export LD_LIBRARY_PATH 
     Fixit# export LD_LIBRARY_PATH=/mnt2/lib
  3. Unmount all zfs filesystems 
     Fixit# zfs unmount -a
  4. Change mount points for zroot pool 
     Fixit# zfs set mountpoint=legacy zroot
 Fixit# zfs set mountpoint=/tmp zroot/tmp
 Fixit# zfs set mountpoint=/usr zroot/usr
 Fixit# zfs set mountpoint=/var zroot/var
  5. Exit Fixit mode, and then sysinstall. Remove the FreeBSD install DVD/Memstick and the system will boot using the ZFS root.

CategoryInactiveProject CategoryStale

RootOnZFS/ZFSBootSlice (last edited 2018-07-28T04:17:45+0000 by ScotHetzel)