Creating a Virtual DMZ with User-Mode Linux - Technical Notes - Ian Sharpe

Introduction

This note describes the use of User-Mode Linux to implement a virtual Demilitarized Zone.

Rationale

Many previously retired PCs are being combined with Linux to provide a firewall for small networks. Quite often these are powerful enough to provide services in addition to firewalling, but to do so would compromise their security.

User-Mode Linux is a port of the Linux Kernel to the kernel itself; that is, UML uses standard Linux system call to implement a kernel which can run as a guest on a host Linux system. To the host, the guest is just another user process; however to processes running in an instance of UML, UML is the kernel.

Running services within one or more UML processes reduces the chance that the firewall is compromised by an exploit within the service. Gaining root within a UML process does not help an attacker gain root on the firewall. By choosing a configuration similar to a conventional DMZ, it is possible to isolate the UML processes still further, to the extent that they have little or no knowledge of the system that hosts them.

Target System

As a concrete example of the technique, this note describes the construction and configuration of the following system:

Hardware - 
 266MHz PII with 128Mb RAM and 40Gb hard disk
Services 
 External: Apache Web server, Jabber IM, Internet Access
 Internal: Apache Web server, Jabber IM, Mail, News, Privoxy, CVS

Building the UML System

Overview

The aim is to mimic the conventional architecture shown in Figure 1. The network consists of five machines on the private net (atem, hyperboria, tangram, tyger and zeit) and a DMZ containing the servers blackdance, mirage and trancefer. Each of the servers in the DMZ is dual-homed, allowing distinctions to be made between traffic originating from the Internet and that generated internally. Blackdance provides externally-accessible services, trancefer accesses the Internet but only provides internal services and mirage is used for a little smoke-and-mirrors work relating to data logging. Screening routers are used to isolate the DMZ from the private net and limit access to/from the Internet to both the private net and the DMZ.

In the physical system shown in Figure 2, the firewall/UML Host is zeit. It will use ipfilter rules to provide the screening shown in Figure 1. blackdance and trancefer are UML processes, connected via TUN/TAP network devices to zeit. Physically, zeit is dual homed with one interface connected to the modem and another connected to the private net.

Host Requirements

Kernel and Applications

Since the network interfaces to the DMZ will use TUN/TAP, the host kernel must be built with that feature enabled. If SKAS mode is required, then the kernel must be patched with the skas patch and /proc/mm enabled. SKAS (Separate Kernel Address Space) provides additional security and performance, but at the time of writing it is not deemed stable.

Groups and Users

Each instance of UML is run as a specific user (with the same name as the virtual O/S). In addition, each user has a primary group of the same name. Thus the groups/users blackdance, mirage and trancefer must be created. Finally, a group uml exists for paths which all UML instances can see (i.e. users blackdance, mirage and trancefer all members of this group).

Host File Layout

The static parts of UML are rooted at /opt/uml. The filesystems for each UML instance are held in /var/uml/<hostname>.

Building and Installing the UML Program

The UML program - appropriately enough called linux - is a modified form of the Linux kernel. It is built from a patched version of the standard kerl, which adds a new architecture: 'um'. Note that this must not be built in the host kernel tree. The required steps are described in detail on the UML site, but in outline:
  1. Get source from http://user-mode-linux.sourceforge.net
  2. Unpack kernal and patch in new directory
    (Assumes kernal and patch file in current dir) 

     mkdir uml
 cd uml
 tar xzf ../linux-2.4.19.tar.gz
 cd linux-2.4.19
 bzcat ../../uml-patch-2.4.19-14.bz2 | patch -p1

  3. Build 
     make menuconfig ARCH=um
 # Configure to taste, or load this config file
 make dep ARCH=um
 make linux ARCH=um

To simplify the creation of network devices and running of UML, the command files umlstart, umlwait, umlstop and settap should be copied to /opt/uml/bin. While not essential, they will be used later in this note.

Creating the Filesystems

For simplicity, the Virtual DMZ UML processes all use the same drive configuration. Five drives are configured:

ubd0(root_fs):Root filesystem containing /bin, /dev, /etc, /lib, /mnt, /opt, /proc, /root, /sbin, /usr and the mount points for the filesystems below
ubd1(var_fs):Var filesystem - /var
ubd2(home_fs):User directories - /home
ubd3(tmp_fs):Temp space - /tmp
ubd4(swap_fs):Swap space

The primary purpose of separating the drives/directories this way is to distinguish between fixed data, temporary data and data that must be preserved. UML drives can be configured as 'Copy On Write (COW)', in which the base drive is unchanged and a parallel file updated with modifications. In this system, udb0, 1 and 2 will be COW drives. Typically the changes (if any) will be discarded for udb0 and udb2 (the systems have no 'users'). Various strategies may be adopted for the udb1 (/var) drive; for example the change file may be backed up and then merged. Given a backup of the original drive it would then be possible to recover the state of the drive at any of these 'checkpoints'. Drives ubd3 and ubd4 (/tmp and swap) are periodically rebuilt. Since they contain no long-term useful data, they are readily created and are not worth running as COW drives.

The filesystems are created as sparse files as follows 

 mkdir /usr/var/uml/<hostname>
 cd /usr/var/uml/<hostname>

 dd if=/dev/zero of=root_fs bs=1M count=5 seek=1024
 dd if=/dev/zero of=var_fs  bs=1M count=5 seek=1024
 dd if=/dev/zero of=home_fs bs=1M count=5 seek=1024
 dd if=/dev/zero of=tmp_fs  bs=1M count=5 seek=1024
 dd if=/dev/zero of=swap_fs bs=1M count=5 seek=256

 /sbin/mke2fs root_fs
 /sbin/mke2fs var_fs
 /sbin/mke2fs home_fs
 /sbin/mke2fs tmp_fs
 /sbin/mkswap swap_fs

The empty filesystems need to have a Linux installation copied onto them. This is most easily achieved by taking one of the standard UML root filesystems and copying the files. The target system uses a Debian installation, and so the file root_fs_debian2.2_small.bz2 was downloaded as the starting point. 

 mkdir uml_mnt
 bunzip2 root_fs_debian2.2_small.bz2
 mount -o loop root_fs_debian2.2_small /mnt

 mount -o loop root_fs uml_mnt
 cp -fra /mnt/bin /mnt/dev /mnt/etc /mnt/lib /mnt/mnt /mnt/opt \
         /mnt/proc /mnt/root /mnt/sbin /mnt/usr uml_mnt
 umount uml_mnt

 mount -o loop var_fs uml_mnt
 cp -fra /mnt/var uml_mnt
 umount uml_mnt

 mount -o loop home_fs uml_mnt
 cp -fra /mnt/home uml_mnt
 umount uml_mnt

 umount /mnt

Updating the Debian Linux Installation

The Debian Linux installation installed above is very minimal, and requires some of the configuration files to be modified before it can be booted. With these modifications it will be possible to boot the system and update the installation to the desired version. For the purposes of the demonstration system this will be Debian/Sarge.

/etc/fstab

The existing /etc/fstab assumes a single drive. Replace with the following: 

 # /etc/fstab: static file system information.
 #
 # 							
 /dev/ubd/0	/		ext2	defaults,errors=remount-ro	0	1
 /dev/ubd/1     /var            ext2    defaults,errors=remount-ro      0       1
 /dev/ubd/2     /home           ext2    defaults,errors=remount-ro      0       1
 /dev/ubd/3     /tmp            ext2    defaults,errors=remount-ro      0       1
 /dev/ubd/4	none		swap	sw				0	0
 proc		/proc		proc	defaults			0	0

/etc/hostname

Create /etc/hostname, which simply contains the name of the machine we are building (e.g blackdance).

/etc/network/interfaces

This is covered in more detail below; for now consider this example (for blackdance): 

 # /etc/network/interfaces -- configuration file for ifup(8), ifdown(8)

 # The loopback interface
 auto lo
 iface lo inet loopback

 auto eth0
 iface eth0 inet static
        up route add -net 10.1.0.16   netmask 255.255.255.252 gw 10.1.0.5
        up route add -net 10.1.0.12   netmask 255.255.255.252 gw 10.1.0.5
	address 10.1.0.6
	netmask 255.255.255.252
	broadcast 10.1.0.7

 auto eth1
 iface eth1 inet static
        up route add -net 10.1.1.16   netmask 255.255.255.252 gw 10.1.1.5
        up route add -net 10.1.1.12   netmask 255.255.255.252 gw 10.1.1.5
        address 10.1.1.6
        netmask 255.255.255.252
        broadcast 10.1.1.7
	gateway 10.1.1.5

/etc/resolv.conf

This needs to be set up with addresses of the DNS to be used, and the domain name of the network. For example: 

domain sharpe-practice.co.uk
search sharpe-practice.co.uk
nameserver 194.168.4.100
nameserver 194.168.8.100

/etc/apt/sources.list

Update /etc/apt/sources.list to contain the Debian Linux network sources you wish to use. For example: 

 # /etc/apt/sources.list
 # 							
 deb http://security.debian.org/ woody/updates main contrib non-free

 deb ftp://ftp.uk.debian.org/debian/ testing main non-free contrib
 deb-src ftp://ftp.uk.debian.org/debian/ testing main non-free contrib

 deb ftp://non-us.debian.org/debian-non-US testing/non-US main contrib non-free
 deb-src ftp://non-us.debian.org/debian-non-US testing/non-US main contrib non-free

 deb ftp://www.mirror.ac.uk/sites/ftp.debian.org/debian/ testing main non-free contrib
 deb-src ftp://www.mirror.ac.uk/sites/ftp.debian.org/debian/ testing main non-free contrib

/etc/inittab

To simplify shutdown, this file should be changed so that ctrl-alt-del causes a shutdown rather than a reboot. Change the ca line as follows: 

 # What to do when CTRL-ALT-DEL is pressed.
 ca:12345:ctrlaltdel:/sbin/shutdown -t1 -a -h now

In addition, the number of TTYs can be reduced (it is unlikely that all 6 will be needed in this environment): 

 1:2345:respawn:/sbin/getty 38400 tty1
 2:23:respawn:/sbin/getty 38400 tty2
 3:23:respawn:/sbin/getty 38400 tty3
 #4:23:respawn:/sbin/getty 38400 tty4
 #5:23:respawn:/sbin/getty 38400 tty5
 #6:23:respawn:/sbin/getty 38400 tty6

Note: the text 

INIT: Id "0" respawning too fast: disabled for 5 minutes
INIT: Id "1" respawning too fast: disabled for 5 minutes
INIT: Id "2" respawning too fast: disabled for 5 minutes
INIT: Id "c" respawning too fast: disabled for 5 minutes
INIT: no more processes left in this runlevel
on booting the UML is indicative of a problem with the TTY configuration.

Using apt-get to Update Debian / Linux

This requires the following steps:
  1. create an appropriate TAP device
  2. start UML
  3. log in and run apt-get

Configuration

Network

The network shown in Figure 2 is implemented by appropriately configured TAP devices on zeit. It is a simplification in as much as the interfaces to the UML instances use a netmask of 255.255.255.252. In effect therefore they are close to being point-to-point networks in the sense that no machine can snoop on another. The use of TAP devices (rather than uml_switch for example) further isolates the machines.

The command file settap is used to create the devices on zeit; the relevent commands are, for each interface: 

 # Delete and recreate the TAP device, owned by the appropriate user
 tunctl -d tap0
 tunctl -u trancefer -t tap0

 # Bring up i/f and specify route to the host
 ifconfig tap0 10.1.0.17 netmask 255.255.255.252 broadcast 10.1.0.19 up
 route add -host 10.1.0.18 dev tap0

 # Enable proxy arp for the interface
 echo 1 > /proc/sys/net/ipv4/conf/tap0/proxy_arp
 arp -Ds 10.1.0.18 eth0 pub

This example is for trancefer (10.1.0.18). With a netmask of 255.255.255.252 the broadcast address will be 10.1.0.19 and the network address will be 10.1.0.16. and there will be two available netwoork addresses. A convention of using the odd address for the host and the even address for the guest has been adopted. In this case therefore, the interface is seen as 10.1.0.17 by zeit and 10.1.0.18 by trancefer.

Each UML instance is allocated two interfaces. One (10.1.1.0) is used for traffic to/from the Internet, the other (10.1.0.0) is used for traffic from the private 192.168.1.0 network.

Firewall

The firewall is created by appropriate configuration of the netfilter capabilities of the 2.4 Linux kernel. Specifying the rules at the level of individual interfaces and ports is laborious and therefore potentially error prone. Fortunately Shorewall allows rules to be specified in terms of zones and the connections allowed between them.

Four zones are defined:
net: Internet
dmz: Demilitarized zone
svc: Untrusted servers
loc: Local networks

Shorewall allows symbolic names to be defined in the file params which can be used in the other configuration files. In this case various network addresses are defined. Note also the inclusion of the symbols defined by dhcpcd via the file /etc/dhcpc/dhcpcd-eth1.info 

 #	All columns
 . /etc/dhcpc/dhcpcd-eth1.info
 NTL_UBR=172.25.71.254
 ETH1_IP=$IPADDR
 #
 BLACKDANCE_SVC=10.1.0.6
 BLACKDANCE_DMZ=10.1.1.6
 MIRAGE_SVC=10.1.0.14
 MIRAGE_DMZ=10.1.1.14
 TRANCEFER_SVC=10.1.0.18
 TRANCEFER_DMZ=10.1.1.18
 #
 NTL_INK_GUI_1=62.252.0.4
 NTL_INK_GUI_2=62.252.0.5
 NTL_INK_GUI_3=62.252.0.6
 #
 NTL_DNS_1=194.168.4.100
 NTL_DNS_2=194.168.8.100
 #
 DDNS_SVR_1=66.37.215.47
 DDNS_SVR_2=66.37.215.48
 DDNS_SVR_3=66.37.215.49
 #LAST LINE - ADD YOUR ENTRIES ABOVE THIS ONE - DO NOT REMOVE

Zones are mapped to interfaces using the interfaces file, which is shown below. 

 #ZONE	 INTERFACE	BROADCAST	OPTIONS
 net     eth1    detect          noping,dhcp,norfc1918
 loc     eth0    detect          routestopped
 svc     tap0    detect          routestopped
 svc     tap2    detect          routestopped
 svc     tap4    detect          routestopped
 dmz     tap1    detect
 dmz     tap3    detect
 dmz     tap5    detect
 #LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE

The default action for packets moving between these zones is specified in the policy file: 

 #SOURCE  DEST		POLICY		LOG LEVEL
 loc      net           ACCEPT
 fw       net           REJECT          info
 loc      dmz           REJECT          info
 loc      svc           REJECT          info
 dmz      all           REJECT          info
 net      all           DROP            info
 all      all           REJECT          info
 #LAST LINE -- ADD YOUR ENTRIES ABOVE THIS LINE -- DO NOT REMOVE

By default, machines on the private network are allowed free access to the Internet. All other inter-zone communication is barred and will be logged by the firewall host zeit. Connection between the Internet and all other zones is stealthed (i.e. the packets are DROPped rather than REJECTed). Note that this defines egress control for all of the untrusted networks as well as ingress controls, and includes messages from the firewall (zeit) to the Internet. Later specific address/port combinations will be allowed, but other than this no communication is possible.

Since connection to the Internet is via a modem and a single IP address, some address masquerading is required. This is specified in the masq file, whose contents are listed below: 

 #INTERFACE	        SUBNET		ADDRESS
 eth1                    eth0
 eth1                    tap0
 eth1                    tap1 
 eth1                    tap2 
 eth1                    tap3
 eth1                    tap4 
 eth1                    tap5
 tap0                    192.168.1.0/24
 tap1                    192.168.1.0/24
 tap2                    192.168.1.0/24
 tap3                    192.168.1.0/24
 tap4                    192.168.1.0/24
 tap5                    192.168.1.0/24
#LAST LINE -- ADD YOUR ENTRIES ABOVE THIS LINE -- DO NOT REMOVE

With the above, any packet from eth0 or any of the tap devices will be masqueraded when sent through eth1 to the Internet as expected - this is required since only one IP address has been allocated on the Internet. More interestingly, any packet sent from the private network (192.168.1.0) to any of the tap devices will also be masqueraded. This is not required, but has the advantage that the processes running under UML never see private network addresses and cannot therefore identify potential targets (should an intruder get this far). When masquerading to the tap devices, the private net address is replaced by the address of the tap device as seen from zeit.

Specific address/port combinations can have specific rules which override the defaults in the policy file. 

 #ACTION  SOURCE         DEST            PROTO   DEST    SOURCE     ORIGINAL
 # Accept outgoing DNS connections from the firewall
 #
 ACCEPT  fw        net:$NTL_DNS_1        tcp     domain
 ACCEPT  fw        net:$NTL_DNS_1        udp     domain
 ACCEPT  fw        net:$NTL_DNS_2        tcp     domain 
 ACCEPT  fw        net:$NTL_DNS_2        udp     domain 
 #
 # Make DHCP work
 #
 ACCEPT  net:$DHCPSID  fw                udp     bootps
 ACCEPT  net:$NTL_UBR  fw                udp     bootps
 #
 # Accept SSH connections from the local network to the firewall and DMZ
 #
 ACCEPT  loc       fw                    tcp     ssh
 ACCEPT  loc       svc                   tcp     ssh
 #
 # Accept DNS from local to DMZ 
 #
 ACCEPT  loc       svc:$TRANCEFER_SVC    udp     domain
 ACCEPT  loc       svc:$TRANCEFER_SVC    tcp     domain
 #
 # DMZ DNS access to the internet
 #
 ACCEPT  dmz       net:$NTL_DNS_1        tcp     domain 
 ACCEPT  dmz       net:$NTL_DNS_1        udp     domain 
 ACCEPT  dmz       net:$NTL_DNS_2        tcp     domain
 ACCEPT  dmz       net:$NTL_DNS_2        udp     domain
 #
 # DMZ News access to the internet
 #
 ACCEPT  dmz:$TRANCEFER_DMZ net                tcp nntp
 ACCEPT  loc                svc:$TRANCEFER_SVC tcp nntp
 #
 # CVS access
 #
 ACCEPT  svc:$BLACKDANCE_SVC svc:$TRANCEFER_SVC tcp cvspserver
 ACCEPT  loc                 svc:$TRANCEFER_SVC tcp cvspserver
 #
 # Mail access
 #
 ACCEPT  dmz:$TRANCEFER_DMZ  net                    tcp     pop3
 ACCEPT  dmz:$TRANCEFER_DMZ  net                    tcp     smtp
 ACCEPT  svc:$BLACKDANCE_SVC svc:$TRANCEFER_SVC     tcp     smtp
 ACCEPT  loc                 svc:$TRANCEFER_SVC     tcp     smtp
 ACCEPT  loc                 svc:$TRANCEFER_SVC     tcp     pop3
 ACCEPT  fw                  svc:$TRANCEFER_SVC     tcp     smtp
 #
 # Limited access for NTL proxies
 #
 ACCEPT  dmz:$BLACKDANCE_DMZ net:$NTL_INK_GUI_1    tcp     webcache
 ACCEPT  dmz:$BLACKDANCE_DMZ net:$NTL_INK_GUI_2    tcp     webcache
 ACCEPT  dmz:$BLACKDANCE_DMZ net:$NTL_INK_GUI_3    tcp     webcache
 #
 # Redirect connection to the Logger
 #
 REDIRECT  svc     514               udp     514  - $MIRAGE_SVC
 REDIRECT  loc     514               udp     514  - $MIRAGE_SVC
 #
 # Local access to web server
 #
 ACCEPT    loc   svc:$BLACKDANCE_SVC tcp     www
 ACCEPT    loc   svc:$BLACKDANCE_SVC tcp     https
 #
 # Redirect connection to the DMZ web server
 #
 DNAT      net   dmz:$BLACKDANCE_DMZ tcp     www
 DNAT      net   dmz:$BLACKDANCE_DMZ tcp     https
 DNAT      loc   dmz:$BLACKDANCE_DMZ tcp     www     - $ETH1_IP
 DNAT      loc   dmz:$BLACKDANCE_DMZ tcp     https   - $ETH1_IP
 #
 # Make ping work
 #
 ACCEPT          loc       net           icmp    8
 ACCEPT          loc       dmz           icmp    8
 #
 # Kill Kazza
 #
 DROP            net       $FW           tcp     1214
 #
 # Reject anything between DMZ servers not explicitly allowed
 #
 REJECT svc:$BLACKDANCE_SVC svc:$TRANCEFER_SVC   all
 REJECT svc:$TRANCEFER_SVC  svc:$BLACKDANCE_SVC  all
 REJECT dmz:$BLACKDANCE_DMZ dmz:$TRANCEFER_DMZ   all
 REJECT dmz:$TRANCEFER_DMZ  dmz:$BLACKDANCE_DMZ  all
 #LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE

Startup and Shutdown

Describe rc scripts for starting and stopping DMZ with host.

Operation

Describe 'operational' features (backups/auto clean/software installation and upgrade)

Security

Processes

Processes executed by a UML instance operate in an environment similar to that of a normal Linux process. Normal Linux security rules apply, so any technique used to harden the system can be applied to UML. Users and privileges within the UML instance are not shared with the host. Specifically, 'root' for the UML instance is not necessarily the same as 'root' for the machine hosting it. Since the UML instance is run as an unprivileged user process, an intruder who knows an exploitable weakness within UML will at worst be able to crash the process and gain access to the host as an ordinary user.

Gaining 'root' on the UML instance will allow (of course) an intruder to do serious damage to that system, (changing data, inserting backdoors etc.). Running a Virtual DMZ as described in this note is intended to protect the firewall not the UML instances, which are effectively bastion servers and should be configured with this in mind. However, since the critical disks are all copy-on-write, it is trivial to undo damage. Attacks from the DMZ on other machines is limited by the network configuration as described below.

Networking

The most obvious change here is that services accessible to the Internet are provided by UML instances in the Virtual DMZ. Connections can only be made if the iptables configuration is set up to forward them. Misconfiguration will therefore result in a lack of service rather than exposure (assuming no services are running on the host machine), even if ports are left open on the host.

Egress control on the Virtual DMZ UML instances applies not only to connections to the Internet, but also between the DMZ servers themselves. If one of these machines is cracked, the damage that can be done either to the rest of the network or the Internet is restricted. The restrictions cannot be by the intruder even with root privileges. Attempting such accesses will fail and be recorded. It is not possible to observe or prevent this monitoring from within the UML instance.

The use of NAT between the private network and the DMZ removes another source of information from potential hackers. The DMZ UML instances see all accesses from the private network as originating from a single machine, which is just the host side of the TAP link. Attacking this address will fail since there are no real resources behind it.

Monitoring

Within an executing UML instance it is possible to use syslog-ng to log events to a remote logging host. Clearly if the UML instance is compromised then this logging can be readily terminated by an intruder, but some logging of the activity will occur prior to this. In the firewall configuration described above, the non-existent host mirage is used as the target of the logging. Behind the scenes the firewall ipfilter performs destination address translation so that the actual destination is the UML host (firewall) itself. Thus logging occurs on the host without it being explicitly specified in data that could be intercepted by users on the UML instance being logged. Attempts to attack mirage will of course fail.

The host of the UML instances is in a very good position to log network traffic and this is a natural location to place defensive packet sniffing such as snort. Such monitoring cannot be detected by the UML instances, nor can it be terminated by them.

If configured, UML can log all terminal data to/from an instance to a file. This is written to the UML host and is invisible to the UML instance. Unlike syslog, this cannot be turned off by a user within the executing UML instance.

File integrity on the UML host should also be checked. This can be readily achieved by an application like Tripwire, which maintains checksum and node data for selected files. Since the configuration of the UML host will seldom need to be changed (it isn't running any services) there should be very few false positives and the maintenance overhead of such a tool should not be significant.

Comparison

Compare simple firewall/firewall+services/Virtual DMZ.

Links

The User-mode Linux Kernel Home Page http://user-mode-linux.sourceforge.net/
User-mode Linux Community Resources http://usermodelinux.org/
Netfilter http://www.netfilter.org/
Shorewall http://shorewall.sf.net/
Snort http://www.snort.org

ToDo

  1. Tidy-up command files
  2. Jail UML processes within the host
  3. Harden host with SELinux or similar
Original at http://www.sharpe-practice.co.uk/isharpe/technote/uml_dmz.htm. $Id: uml_dmz.htm,v 1.5 2002/12/19 12:04:10 isharpe Exp $