Clients who understand the power of UNIX frequently make requests that make an administrator's life difficult. They want to be able to install software or to reconfigure the Web server to enable the latest nifty Apache module. In short, they want root access, and under most UNIX systems you can't hand out root access willy−nilly to clients on a multi−user server.

Unless you're on FreeBSD. FreeBSD administrators faced this problem long ago, and solved it by improving the chroot process dramatically. In fact, they solved it so well that, when using FreeBSD, you can build an entire virtual machine on disk, and isolate that machine from the rest of your system. This is called a jail.

Think of a jail as something like a client−server environment. The main server is the host system, and each jailed system is a client. Changes made to the host can be reflected across all systems, but changes to the jail can't affect the main system, unless you allow a jail to fill up a disk drive or some such.

When in jail, clients can have root access and even install whatever nifty toys they desire without interfering with the main system. All processes that are running in the jail are restricted to the jail environment, and the kernel does not give them access to any information not in their jail. The filesystem in the jail does not know about files or filesystems outside the jail. Since no program or process in the jail knows about anything outside the jail, and cannot read or access anything outside the jail, the user is locked in. Not only can the client not break out of the jail, if the jail is hacked the intruder can't break out of the jail. This helps secure your system while meeting client needs.

On modern hardware with cheap disks and gobs of memory, a single FreeBSD system can host dozens of jailed Web servers (though you'd need to be certain that your kernel is well tweaked to allow this many Web servers to run, as discussed in Chapter 4). From a sales perspective, a jailed machine is a good intermediate step between a virtual domain on a shared server and a private colocated server.

Configuring Your Server to Use Jail

Before you begin using jails, be sure that your server is configured properly. Jails put a number of special requirements on a server, the most annoying of which is that daemons cannot bind to all available IP addresses.

Each jail is tied to a particular IP address, and is defined by that IP address. The jail must have exclusive access to that IP address; nothing else can be using it. If your main server has a daemon that binds to all available IP addresses on the system, that daemon will prevent a jail from starting. If you look at your system's sockstat(1) output, you may notice several entries where the local address resembles "*.22". This means that the daemon is listening on all IP addresses, on that port number. If you want to use a jail, you must reconfigure these daemons to only listen on a single IP address.

Check all of the following daemons before trying to start a jail.

Portmap

Of the standard FreeBSD daemons, portmap is the most problematic, preventing you from combining NFS and jails. (Since very few systems on the naked Internet use NFS, this usually isn't a problem.)

176

Syslogd

Syslogd is another story, because the system logger opens a socket so it can send log messages to other servers. To silence syslogd entirely, set syslogd_flags="ss" in /etc/rc.conf, though if you do you won't be able to log remotely. We'll discuss syslogd in detail in Chapter 15.

Named and sendmail

Other daemons, such as named and sendmail, want to attach to all available addresses. To solve this problem, you can choose to configure them to bind to only a single IP address and run them on the host system, but since you're using jails already, why not set up a "services jail" that contains these daemons? Not only is it easier, but also it allows you to provide an additional layer of security. (While named and sendmail are both quite secure today, they have a spotty history. Many older admins will feel much better if those services are jailed.)

Inetd

Inetd also attaches to all available addresses, but it is simple enough to control with the −a flag. If your jail host has an IP address of 192.168.1.222, add inetd_flags="−a 192.168.1.222" to /etc/rc.conf.

Sshd

The last problematic network service is sshd. Assume again that your jail server has the IP address of 192.168.1.222. You can tell sshd which port to listen on with the following entry in

/etc/ssh/sshd_config:

...............................................................................................

ListenAddress 192.168.1.222

...............................................................................................

Since your jail host is probably not providing any network services itself, you're better off disabling every network daemon except sshd. Ideally, your sockstat output should look something like this:

...............................................................................................

# sockstat −4
USER	COMMAND	PID	FD	PROTO	LOCAL ADDRESS	FOREIGN ADDRESS
root	sshd	248	3	tcp4	192.168.1.222:22	*:*
#

...............................................................................................

We have only one daemon listening to the network, sshd. It is listening on a particular IP address (192.168.1.222) and on a particular port. This daemon will not interfere with our jails.

Configuring Your Kernel to Use Jail

The preceding section takes care of the network part of configuring jail, but we still have some kernel configuration to do. The jail system has three special sysctls:

jail.set_hostname_allowed By default, the root user in a jail can set the host−name of that jail. Since the jail uses its hostname to communicate with the host, changing the hostname can easily confuse an administrator responsible for managing it. You can set this sysctl to 0 to disable

177

changing the hostname.

jail.socket_unixiproute_only A jail defaults to only communicating via IP. While it isn't that likely that a user might want to use, say, UNIX sockets or IPX, it's entirely possible. The jail system only supports IP, however, so if you allow use of these other protocols, you're allowing the user to "leak" out of the jail. They probably can't do anything with that access, but it's unwise to assume that you're smarter than every malicious hacker out there. Set this to 1 to be careful and restrain your users most tightly. Set it to 0 if you do choose to allow the use of any network or socket protocol.

jail.sysvipc_allowed System V IPC is a UNIX standard for allowing interprocess communication via shared memory segments. Basically, related programs can use one chunk of memory to store shared information. By default, IPC cannot be used in a jail, as the jail system does not build separate areas of memory for each jail. Enabling IPC would allow information to leak to and from the jail. Using this weakness to compromise the system would require a skilled attacker, however. You can choose to do allow System V IPC by setting this sysctl to 1. Many database programs require System V IPC.

Client Setup

Setting up a jail is straightforward, though you will need a FreeBSD source tree (see Chapter 6). For example, say you want to build a jail on the partition /jail1. (Jails can be in directories as well, but putting them on separate partitions gives you a quick−and−dirty method of controlling their size. Other admins will just keep an eye on their users, and raise their rates for disk hogs.) To begin, go to your FreeBSD source tree (generally under /usr/src). For your first jail, run this command:

...............................................................................................

# make world DESTDIR=/jail1

...............................................................................................

This command will build a complete copy of FreeBSD and install it in the directory /jail1.

For all subsequent jails, you don't have to build all the binaries; you can install the ones you built the first time by just running this command:

...............................................................................................

# make installword DESTDIR=/jail1

...............................................................................................

This will copy a complete set of FreeBSD userland programs into the jail.

Note Many people have special methods to reduce the amount of space a jail takes up, but the preceding method is the approved one. Search the FreeBSD−security mailing list archives if you're interested in other methods.

The /etc Tree

Each jail has its own /etc tree. While not everything in there is functional, it's simpler to ignore the extras than trim them out. You need to grab a copy of the /etc tree from the same source code you used to build your jail, and install it properly in the jail's directory. The commands here do exactly

178

that:

...............................................................................................

#cd /usr/src/etc

#make distribution DESTDIR=/jail1 NO_MAKEDEV_RUN=yes

...............................................................................................

Once you have the /etc directory, you'll need to create the device nodes for the jail. (Since a jail does not require all the device nodes that the full system requires, MAKEDEV has a special target for use in jails.)

...............................................................................................

#cd /jail1/dev

#sh MAKEDEV jail

...............................................................................................

Many programs expect to find a file named /kernel. Even if they don't actually do anything with this file, they're happier when the file exists. (Since you don't want people to be able to touch your actual kernel, tie this fake to a harmless point. That way hostile users can overwrite your jailed kernel all they want, but to no avail.)

...............................................................................................

#cd jail1

#ln −sf dev/null kernel

...............................................................................................

The IP Address

Now that the directory tree is established, you need to provide an IP address for the jail, since each jail has its own IP address. We'll assume that 192.168.1.223 is our jailed IP address, and use ifconfig to attach this address to our network card.

...............................................................................................

# ifconfig fxp0 alias 192.168.1.223

...............................................................................................

You can make this attachment happen automatically on boot by adding the following to /etc/rc.conf:

...............................................................................................

ifconfig_fxp0_alias0="192.168.1.223"

...............................................................................................

The Process Filesystem

Finally, every FreeBSD system requires its own process filesystem, or procfs. If you're not using jails, you really don't need to worry about procfs; it appears automatically when you boot the system, cannot be tuned, and programs fairly transparently access it when needed. It's a necessary bit of infrastructure, however. I create a script /usr/local/etc/rc.d/jail.sh and add all the procfs mount lines to this script.

179

...............................................................................................

# mount −t procfs proc /jail1/proc

...............................................................................................

Your jail is now ready.

Entering the Jail

Once you have everything configured, use jail(8) to start a jail:

...............................................................................................

# jail <path to jail> <jail hostname> <jail IP> <command>

...............................................................................................

For example, to do basic configuration of our test jail, do this:

...............................................................................................

# jail /jail1 jailhost 192.168.1.223 /bin/tcsh

...............................................................................................

You'll see a shell prompt, at which point you're in single−user mode in your jail and your jail is up and running. You could choose any shell you like in the default install—I like tcsh for interactive use, so that's my example. There are differences between your current state and FreeBSD single−user mode, however. While the jail's startup sequence has not been run, the network is configured by jail.

Some commands are unavailable in a jail. For example, try to add an alias to your network interface, and you'll get a "permission denied" error.

Play around a little, and try to break out of the jail. Try to go to a directory you know exists on the system, but is outside of your jail directory. You're root; try to access processes you know are running on the system. When you're tired of beating your head against that brick wall, explore the jailed system. Powerful UNIX tools like perl(1) and cc(1) are fully available. You could even cvsup in a jail and rebuild world, although this is not a good idea. (Remember, your kernel and userland absolutely must be in sync; a jailed userland will not crash the kernel, but it certainly won't work as expected!)

Processes

Processes in the jail cannot see the rest of the system. Our host server is running a jail, among many other things. Here's a top snapshot from within a jail running in single−user mode. You can see that the shell process is running, and the top process, but nothing else. You cannot see the processes from the main system.

...............................................................................................

last pid: 10578; load averages:

0.00,

0.00,

0.00

up 1+09:21:29

19:16:49

2 processes: 1 running, 1 sleeping

CPU states:

0.0% user,

0.0%

nice, 0.4% system,

0.0%

interrupt,

99.6%

idle

Mem: 6708K Active,

27M Inact,

23M

Wired, 36K Cache, 61M

Buf, 444M Free

Swap: 1024M

Total,

1024M

Free

PID

USERNAME PRI NICE SIZE

RES

STATE

C

TIME

WCPU

CPU

COMMAND

10574

root

20

0

1432K

1116K

pause

0

0:00

0.00%

0.00%

tcsh

10578

root

96

0

1956K

1136K

CPU1

1

0:00

0.00%

0.00%

top

180