• NewZfsBootEnvMgmtLibAndTool

A New ZFS Boot Environment Management Library & Tool For FreeBSD

A Summer of Code 2017 Project

• Student: KyleKneitinger (kneitinger@)

• Mentor: AllanJude (allanjude@)

Project description

The purpose of this project is to build a convenient framework for managing ZFS boot environments. Currently, boot environments are managed manually or with scripts such as beadm, but both approaches are shell based. The approach that this project will take is to first create a boot environment library, libbe to offer simplified boot environment interaction in C to userspace applications (for example, TrueOS’ SysAdm), and then write an application that provides users with a command, be, that retains all of the features of beadm, and adds some functionality. Among the new features are the ability to recursively create boot environments containing child datasets, the ability to activate an environment for the next boot only, and the ability to attach/detach a boot environment to a jail.

Approach to solving the problem

This library will use the functions in the libzfs* libraries to incrementally add heatures to both libbe and be concurrently. For instance, once the list features are completed in libbe, they will be leveraged to complete the be list subcommand, ensuring that implementation is correct before moving onto a new feature. When relevent, behavior will be tested against beadm, with the goal that once complete, beadm can be aliased or replaced to be without any breakage or incompatibility.

Deliverables

The final deliverable is be, a command with a superset of beadm features (illustrated below in the usage), and the libbe library, that provides the features to the command and userspace ecosystem.

$ be -h
usage: be command args ...
where 'command' is one of the following:

    activate [-t] <bootenv>
    create [-r] [-e nonActiveBe | -e bootenv@snapshot] <bootenv>
    create beName@snapshot
    destroy [-F] <bootenv|bootenv@snapshot> [bootenv|bootenv@snapshot]*
    list [-a] [-D] [-H] [-s]
    rename <bootenv> <bootenv>
    mount <bootenv> [mountpoint]
    unmount|umount [-f] <bootenv>
    jail <jailid|jailname> <bootenv>
    unjail <jailid|jailname> <bootenv>

Each bootenv is a name of a dataset in (or to be created in) $POOLNAME/bootenv/

Milestones

• End of community bonding period:

  • Preliminary man page created

• June 15th [Still in school]:

  • Makefiles for library and user command

  • Headers created for library, with all user-facing function prototypes present and documented.

• June 30th [Still in school for half]:

  • Command's subcommand/argument parser completed

    • Command builds, displays usage, and for all subcommand/arg combinations (based on presented usage and man page): exits with code 64 (EX_USAGE) if syntax is incorrect and displays message describing why, or if syntax is valid, exits 0.

  • Begin work on library's list functionality.

• July 15th:

  • list, activate, rename, destroy functionality completed in both library and command (list remains)

  • Begin mount and unmount if time allows

• July 31st:

  • mount, unmount functionality completed in both library and command

  • Ideally create functionality completed, yet, since this is likely the most in-depth part of the project, leaving a buffer of time into the next phase if necessary.

• August 4th:

  • create functionality completed if not already. (At this point, command can completely replace `beadm)

• August 15th:

  • jail functionality complete

End of SoC time frame edit: See "Remaining Work" for fine grained list of tasks that aren't reflected by this schedule

The Code

https://svnweb.freebsd.org/socsvn/soc2017/kneitinger/

Remaining

Before being polished enough to consider submitting a patch, some work remains. The majority of this work is related to cleanliness, removing redundancies, and polishing error reporting, however a few technical points remain:

• Incorporate Solaris-to-FreeBSD nvlist_t compatibility definitions.

  • Currently, be_get_props() (the main function behind the be list subcommand) is commented out since libbe uses Solaris nvlist_t because of libzfs. Without the compatibility definitions, and program using libbe would have to include a lot of Solaris code (which is only available in src). When this is complete be_get_props() will return a FreeBSD nvlist_t.

• Deep Boot Environment creation:

  • Handle (or prevent?) scenario where a user adds a child that is at least two levels down from /, and then later adds the parent of that child. For instance, if Alice added /foo/bar/baz to a boot environment, then /foo and /foo/bar are created as canmount=off,mountpoint=none datasets. libbe does not currently allow Alice to then add /foo/bar to the boot environment.

  • If specified by function parameter, be_add_child() should copy existing data (if path already exists outside of boot env). Currently this only happens when the specified path is itself a ZFS dataset.

• Ensure that bypass-on-error is consistently implemented.

  • The user can decide to let succesive modification calls to silently bypass if the internal error state is set by a previous function. Inspired by the Cairo vector graphics library's error handling, this allows the user of libbe to set up many operations and only have to check for error after the last call, without any worries that calls will operate with the assumption about the side effects of previous calls. This should not be implemented for informative, non-modifying, functions, only for functions with side effects, yet coverage is spotty.

• If deep boot environments are used, there should be a check (and warning?) that zfsbe_enable="YES" is present in rc.conf. A message about this should probably be printed whenever be add (see next bullet point) is used.

• Rename be init subcommand to be add (or something along those lines)

• Documentation

  • be

    • Include add, import, export subcommands in manpage. Document that add should be an absolute path

    • Update jail, create
    • Be clear that boot environment names should not include leading path.

  • libbe

    • Ensure all (non-helper) functions have docstrings.
    • Once finalized, for each function indicate what errors it can return in both docstrings and manpage.
    • Clearly document bypass-on-error ability, and categories of functions it applies to.
    • Better format the BE_ERR_* listing in manpage.
    • Include brief sample program in manpage

• Pretty printing for be list (depends on nvlist_t compatibility defs)

• Solicit feedback on phabricator, and make any requested changes.

• be jail needs better arg parsing to be as flexible as folks might require w/ regards to ip addrs, etc.

• be jail has the code to construct jail and boot into it, but not in a fully functioning state (possibly due to devfs?). Running of /etc/rc is currently commented out and is just runs /bin/sh.

• be_destroy() needs slight modification to use zfs_iter_children() with a simple destruction callback to handle deletion of deep boot envs

• If the argument to be destroy contains an "@", delete that bootenv snapshot

• Ensure all (u)mount/snap/create/destroy/send/recv functions are deep aware with zfs_iter_children().

• The only be create flavor that needs the -r flag is the snapshot one.

• Remove unnecessary be unjail.

See: bectl(8)

CategoryZfs