• build

Improving the FreeBSD build

This page does not describe how FreeBSD is currently built, but rather how it could be done in the future.

1. Portability

Portability of the FreeBSD build pivots around using tools during the build that are either readily available on different OSes or can be easily built from the FreeBSD source tree to run on those OSes. With make(1) being at the core of the build process, it follows that portability for make(1) is key. Unfortunately, FreeBSD's make is not portable. To improve portability we therefore need to improve make's portability or switch to another make. Here we opt not to re-invent the wheel and maintain (for as much as we maintain our make) a separate make(1) utility, but rather adopt bmake(1) as the portable make. Below are the key action items involved:

1. Providing bmake(1)
2. Understanding differences between make(1) and bmake(1)
3. Port FreeBSD's source tree to bmake(1)
4. Port FreeBSD's ports tree to bmake(1)

5. Port FreeBSD's doc & www trees to bmake(1)

6. Obsolete FreeBSD's make(1)

1.1. Providing bmake(1)

1. Import bmake(1) from ftp://ftp.netbsd.org/pub/NetBSD/misc/sjg/ to contrib/bmake.

2. Build and install bmake(1) from usr.bin/bmake.
3. Provide documentation on the incompatibilities between make(1) and bmake(1).
   • The most obvious difference is the modifiers used to convert variable values to upper/lower case. The old FreeBSD make uses :L and :U, which mean something else to bmake. Bmake uses :tl and :tu, and these modifiers have been also added to FreeBSD make so that one syntax can work for both. Bmake is much more agressive about finding things via .PATH, and thus sometimes needs to be told not to look (.NOPATH), such directives are simply ignored by FreeBSD make. The other significant difference is handling of -V. FreeBSD make fully expands the value, whereas by default bmake shows the raw value (which might contain unexpanded variables). A knob has been added to bmake (.MAKE.EXPAND_VARIABLES=yes) which causes it to behave the same way as FreeBSD make.

     Thus apart from :L -> :tl etc, there is almost no visible difference.

1.2. Understanding differences between make(1) and bmake(1)

1. bmake(1) defines .MAKE.LEVEL, make(1) does not.
2. bmake(1) defines .PARSEDIR, make(1) does not -- this can be used to distinguish between the two.
3. bmake(1) issues a warning when shell output is empty for != (assign from shell).
4. Better .PATH searching of bmake(1) may find sources/objects and avoid building targets that make(1) builds.
5. Need "set -e" before a list of commands to make sure bmake(1) stops after the first failure.
6. bmake(1) handles -V differently from make(1). The equivalent of "make -V FOO" is "bmake -V '${FOO}'". The equivalent of "bmake -V FOO" is "make -X -V FOO".
7. make(1) tries BSDmakefile, makefile and Makefile. bmake(1) tries makefile and Makefile. Aside: gmake(1) tries GNUmakefile, makefile and Makefile. Note that bmake(1) can be tweaked using the .MAKE.MAKEFILE_PREFERENCE variable.

1.3. Port FreeBSD's source tree to bmake(1)

Done

1.4. Port FreeBSD's ports tree to bmake(1)

1. Add bmake(1) compatibility tweaks to the ports build infrastructure.
   • These are quite minor. The most obvious being replacing the modifiers :L with :tl and :U with :tu the only other changes affect bsd.port.mk only.
2. Identify ports that build using make(1) and try them with bmake(1).
3. Do a full package build using bmake(1) and resolve remaining issues.

1.4.1 Handle ports on older FreeBSD versions

ports/Mk/bsd.port.mk will detect when invoked by a make which is not bmake(1). This is simple, since only bmake defines .PARSEDIR.

If .PARSEDIR is not defined bsd.port.mk will run a script (bmake-sh) to find bmake and use it instead. If bmake does not exist on the system, bmake-sh will install it by making a temp copy of Mk/*.mk converted back to the old make syntax, and using that to build/install bmake.

1.5. Port FreeBSD's doc & www trees to bmake(1)

1. Add bmake(1) compatibility tweaks to the doc tree and build.
2. Add bmake(1) compatibility tweaks to the www tree and build.

1.6. Obsolete FreeBSD's make(1)

1. Install FreeBSD's make(1) as pmake(1)
2. Create make(1) symlink to bmake(1)
3. After enough settling time, remove pmake(1)

The end result is that the system will have /usr/bin/bmake and usr/bin/make. Of with make is a symlink to bmake.

To install usr.bin/make as fmake:

   cd usr.bin/make
   make obj
   make
   make install PROGNAME=fmake -DNO_MAN

2. Improve cross-builds

The next step is to improve cross building FreeBSD. Cross building relates to building for a different architecture than the one on which the build is executed, as well as relates to building on a different OS or OS version than the source tree produces. As such, building on FreeBSD/i386 7.0 and for FreeBSD/i386 8.0 is considered a cross build. Below are the key action items involved:

1. Make host- or build tools without an additional pass
2. Stage system headers using dependencies
3. Design and implement a build environment

2.1. Make host- or build tools without an additional pass

2.2. Stage system headers using dependencies

2.3. Design and implement a build environment