This article describes how to setup FreeBSD 6 system with Apache and Tomcat, with a focus on security. It's just an example, feel free to deviate.
Feedback
Feedback is welcome at daniel@quanza.net .
When you have successfully completed this guide, you will have the following:
- a FreeBSD system that is partioned into a host system and some virtual servers (jails);
the host system is running a production release of FreeBSD, with no services that listen to the network except sshd;
Apache 2.2.x in a jail, with HTTPS enabled;
Tomcat 6.0.x in a jail, running inside the native JDK 1.5;
Apache and Tomcat connected via mod_jk.
None of the jails are running sshd, they can be accessed using jexec.
Prerequisites
Basic knowledge of FreeBSD system administration is assumed. If you've never compiled and installed a FreeBSD kernel, this article may be hard to follow, since it does not provide all the necessary details. Study the FreeBSD Handbook first, especially chapter 8: 'Configuring the FreeBSD kernel'. The same goes for apache and tomcat, if you've never set up an apache server or a tomcat server this document will not be all inclusive.
Overview
Assumptions
This document will assume these things:
- Your internet facing interface is em0
- Your internal interface is em1
- Your external IP is 192.168.42.5/24
- Your internal IP is 192.168.69.1/24
Host versus jails
All services that listen on external network interfaces are contained within jails, except for sshd. Each service has it's own jail, and there is also a base jail. The jails will listen on an internal interface, this interface doesn't have to be connected to a LAN.
buildserver at 192.168.69.10 - contains a base jail which will be used to build the packages for the rest of the other jails;
webserver at 192.168.69.20 - contains the Apache httpd, listening on port 8080 for HTTP connections and on port 8443 for HTTPS connections;
appserver at 192.168.69.21 - contains the Tomcat web server, listening on port 7001 for AJP (v1.3) connections;
Software versions used
The setup described has been tested with the following software versions:
Configure a host system
If you haven't done so already, setup a FreeBSD 6 system.
Secure configuration
If you want your system to be secure, make sure your configuration process is also secure. Consider completing the configuration of the system before allowing any incoming connections. E.g. configure it while it's physically near your desk and disconnected from the Internet. This avoids that your system is compromised while being temporarily misconfigured.
Some tips for FreeBSD installation:
Use the most recent supported errata fix branch. At the time of writing, this is RELENG_6_2; see the FreeBSD Release Engineering page to determine the current one.
Perform a Custom installation.
Stick /tmp on a separate partition or make it link to /var/tmp and mount it with the noexec and nosuid flags.
Consider having a separate /usr/jails partition.
Distributions: Choose the Minimal config. Include src and ports.
- Don't configure the machine as a gateway.
Disable inetd.
- Disable NFS and anonymous FTP.
- Disable Linux compatibility.
After the installation:
Install a custom kernel (instead of GENERIC). Trim your kernel configuration to the minimum, excluding things like NFS. Test it with nextboot(8).
Install portaudit, which checks (to be) installed ports against the on-line ports vulnerabilities database via pkg_add -r portaudit.
Install and use sudo instead of working as root all the time.
- Configure a firewall on your machine.
Configure your system for mail sending only, see section 26.8 in the FreeBSD Handbook.
- Turn off password authentication in sshd_config and use ssh keys to login.
Prepare for setting up jails
Set up PF to nat and redirect the ports
We will set up the pf firewall to do simple natting and redirection, this is needed so that the jails will be able to access resources from the internet. We will also use pf to let outside hosts access the webserver
[Step 1] Set up your /etc/pf.conf to nat and redirect the ports:
Modify /etc/pf.conf so that only these lines are uncommented:
# Macros: define common values, so they can be referenced and changed easily. ext_if="em0" # The external interface int_if="em1" # The internal interface external_addr="192.168.42.5" # Your public IP address internal_net="192.168.69.0/24" # Your internal subnet # Translation: specify how addresses are to be mapped or redirected. # nat: packets going out through $ext_if with source address $internal_net will # get translated as coming from the address of $ext_if, a state is created for # such packets, and incoming packets will be redirected to the internal address. nat on $ext_if from $internal_net to any -> ($ext_if) # rdr: packets coming in on $ext_if with destination $external_addr:1234 will # be redirected to 10.1.1.1:5678. A state is created for such packets, and # outgoing packets will be translated as coming from the external address. rdr on $ext_if proto tcp from any to $external_addr/32 port 80 -> 192.168.69.11 port 8080 rdr on $ext_if proto tcp from any to $external_addr/32 port 443 -> 192.168.69.11 port 8443 # Make sure we don't block any traffic pass in all pass out all
[Step 2] Enable pf
Enable pf:
# pfctl -e -f /etc/pf.conf
Add this line to your /etc/rc.conf so that pf will start after your system reboots:
pf_enable="YES"
Install ezjail
[Step 1] Install ezjail using the ports:
# cd /usr/ports/sysutils/ezjail/ && make install
[Step 2] Create an ezjail.conf file:
# cp /usr/local/etc/ezjail.conf.sample /usr/local/etc/ezjail.conf # vi /usr/local/etc/ezjail.conf
You want to set the ezjail_ftphost to a FreeBSD ftp server near your server (ftp.nl.freebsd.org in my case) and you probably also want to set ezjail_uglyperlhack to yes, this will save you a lot of headaches with perl script you might want to run later. You may also want to adjust some other settings, if so you may want to read ezjail.conf(5). For example, all jails are by default placed under /usr/jails. Modify the ezjail_jaildir setting to change this.
[Step 3] Now have ezjail create a base jail, containing all directories and files that will be shared among the different jails.
Since this will run make world it may take a few hours.
# ezjail-admin install -p
The -p option will install ports in the base jail. For more information, please study the ezjail-admin(1) man page.
[Step 4] Make sure the jails will be started automatically when your machine starts up. Edit your /etc/rc.conf file and add the following line:
ezjail_enable="YES"
Define ezjail flavours
When you create a new jail, you can indicate of which flavour it should be. The flavour indicates which modifiable directories and files will be copied in the jail and contains a script to be run when the jail starts up for the very first time.
[Step 5] Configure the default flavour
Modify /usr/jails/flavours/default/etc/rc.conf to apply some default restrictions:
rpcbind_enable="NO" # Disable the RPC daemon cron_flags="$cron_flags -J 15" # Prevent lots of jails running cron jobs at the same time syslogd_flags="-ss" # Disable syslogd listening for incoming connections sendmail_enable="NONE" # Completely disable sendmail clear_tmp_enable="YES" # Clear /tmp at startup
Copy the /etc/resolv.conf to the default jail so that we can resolve from within the jails:
# cp /etc/resolv.conf /usr/jails/flavours/default/etc
Next, we will want to create two custom flavours:
webserver, for jails running a webserver (Apache 2.2 in our case);
appserver, for jails running an application server (Tomcat 6, in our case).
[Step 6] Create some new flavours by copying the default flavour:
# cd /usr/jails/flavours/ # cp -Rp default webserver # cp -Rp default appserver # mkdir -p webserver/pkg # mkdir -p appserver/pkg
- (i) Although it's not strictly necessary to use custom flavours, it's good practice, making it easier to resurrect a deleted jail or to instantiate a jail similar to another one.
[Step 7] Create a build jail.
We need to create a build jail first so that we can build the packages for the other jails:
# ezjail-admin create -f default build 192.168.69.10
You can now start the build jail by entering this command:
# /usr/local/etc/rc.d/ezjail.sh start build
Install the webserver jail
Now we're going to install the jail in which Apache 2.2 will run and configure it. I'll only describe the configuration which is relevant to connecting to tomcat.
Build the packages
Enter the build jail and enter these commands to build the packages needed for the webserver jail:
# mkdir -p /var/ports/packages # cd /usr/ports/www/mod_jk-apache2 # make distclean # make package-recursive
Exit the jail and copy the packages to the flavour:
# cp /usr/jails/build/var/ports/packages/* /usr/jails/flavours/webserver/pkg
Create the webserver jail
Creating the webserver will use almost the same command as creating the build jail:
# ezjail-admin create -f webserver webserver 192.168.69.20
This will create a basic jail and then add the packages in the pkg directory situated in the webserver flavour root. The webserver jail is then started by entering this command:
# /usr/local/etc/rc.d/ezjail.sh start webserver
Configuring Apache
Now we'll alter the apache configuration so that it will connect to the tomcat jail. Create /usr/local/etc/apache22/workers.properties and add these lines:
worker.list=tomcat # The name of the tomcat server worker.tomcat.port=7001 # The port of the tomcat server worker.tomcat.host=192.168.69.21 # The host of the tomcat server worker.tomcat.type=ajp13 # The type of the connection worker.tomcat.lbfactor=1 # The weight of the tomcat server
Now modify the httpd.conf and be sure to add these lines:
Listen 192.168.69.20:8080 # Listen to the right IP/Port combination # In the module section LoadModule jk_module libexec/apache22/mod_jk.so # Load the mod_jk module # mod_jk basic configuration <IfModule mod_jk.c> JkWorkersFile etc/apache22/workers.properties # Set the worker.properties file JkLogFile /var/log/jk.log # Set the jk log JkShmFile /var/log/jk-runtime-status # Set the status file JkLogLevel error # Set the log level </IfModule>
To mount your tomcat application on the root directory of a site add this line to the virtualhost:
JkMount /* tomcat
Where tomcat is the name you defined in workers.properties.
Other tips for configuring apache are:
- Remove all unneeded mods, this will make the risk of vulnerabilities in the webserver.
- Remove the default website and setup your own default virtual host.
- Use either the Location directive or tomcat for access control.
- Make apache listen to 192.168.69.20:8443 if you add SSL support.
After finishing configuration you are done with the apache2.2 configuration.
Install the tomcat jail
Now we're going to install the jail in which Tomcat 6 will run and configure it. Because the configuration of tomcat is very specific to your application I'll only show you how to bind it to the right IP.
Build the packages
Enter the build jail, deinstall all the old packages and enter these commands to build the packages needed for the webserver jail:
# cd /usr/ports/www/tomcat6 # make distclean # make package-recursive
During the build process you'll get a prompt about the ports system not being able to fetch the files it needs to build java. Follow the instructions on screen and then repeat the above commands but without the make distclean.
Exit the jail and copy the packages to the flavour:
# cp /usr/jails/build/usr/ports/packages/* /usr/jails/flavours/appserver/pkg
Create the appserver jail
Creating the appserver will use almost the same command as creating the build jail:
# ezjail-admin create -f appserver appserver 192.168.69.21
This will create a basic jail and then add the packages in the pkg directory situated in the appserver flavour root. The appserver jail is then started by entering this command:
# /usr/local/etc/rc.d/ezjail.sh start appserver
Configuring Tomcat
To make tomcat bind to the right port, find this line and change it:
<Connector port="7001" protocol="AJP/1.3" redirectPort="8443" />
Resources
Information about Jails
"The Jail Subsystem" - Chapter 4 in the FreeBSD Architecture Handbook
"Virtualization with FreeBSD Jails" - ONLamp (O'Reilly) article, discusses FreeBSD 5
Man pages: jail(8) jls(8) jexec(8) rc.conf(5)
Information about Apache httpd
Information about Tomcat
Information about integrating Apache and Tomcat
Improvements
Potential improvements for this article are:
- Add one or more diagrams. At least one that shows the host environment and the jails, with their IP addresses.
Discuss ezjail-admin install -i.