Jan
12

Using TrueOS as a IPFW based home router

Setting up a FreeBSD / TrueOS home router with IPFW

(Updated 1/13/2015 with in-kernel NAT example!)

Over the Christmas holidays I had some spare time and was ready to take the plunge and retire an old Asus router. It had begun getting rather slow, due to the increasing number of devices connected to our network, and of course I wanted peace of mind using a FreeBSD system I could be sure was up to date with security fixes. I used PC-BSD’s server release, TrueOS 10.1, because I wanted to use ZFS with boot-environments to ensure upgrading and replacing disks would be risk-free down the road. The following details how I setup TrueOS on the new box.

 

* Disclaimer *

This is by no means a comprehensive IPFW or NATd tutorial, but hopefully it can serve as a good reference for other users trying to do similar things.

 

The Hardware:

To replace it, I had an old AMD 64 x2 Dual-Core system which hadn’t been used in a while.

CPU: AMD Athlon(tm) 64 X2 Dual Core Processor 5200+ (2719.21-MHz K8-class CPU)
real memory = 4966055936 (4736 MB)
avail memory = 3974365184 (3790 MB)
80GB IDE disk drives (2x)
Intel PWLA83911GTBLK PRO/1000 GT network adapter (2x) – (Amazon Link)

This box is easily overkill, with more than enough horsepower to handle our internet traffic, which is currently 100Mb/s down, 10MB/s up, however it was something I had on hand which means I was only out the cost of a couple network cards. Also, I expect it will be able to keep up with future speed increases, since 200Mb/s down and 20MB/s up is supposed to be coming to our area in the near future.

The Installation:

To get started, I grabbed a copy of the CD-sized installer for TrueOS from PC-BSD’s website:
http://www.pcbsd.org/en/download.html

The installation was very straight forward. I picked the defaults to use ZFS on root, with GRUB so that boot-environments work properly. The only major change was to go to advanced disk setup and attach the second 80GB disk drive as a mirror of the primary (ada0). Since these disks aren’t super new I wanted the ability to easily replace them when they begin to fail down the road.

Post Install Configuration:

With the installation finished, the first thing I did was to edit /etc/rc.conf with the following lines:

# Disable some PC-BSD / TrueOS services we don’t need to run on a router
syscache_enable=”NO”
php_fpm_enable=”NO”
appcafe_enable=”NO”

This disabled the AppCafe / Warden web-interface, since I don’t plan on running any applications or jails on this box, only using it purely as a router. Of course if you plan on running jails or doing other package work, the web-interface could be useful and you can skip these lines.

Next I started the configuration of the network interfaces:

# Network configuration
hostname=”interdictor”
ifconfig_em0=”inet 192.168.1.1 netmask 255.255.255.0″ # LAN IP / Bottom NIC
ifconfig_em1=”inet XXX.XXX.XXX.XXX netmask XXX.XXX.XXX.XXX” # WAN IP / Top NIC
defaultrouter=”XXX.XXX.XXX.XXX”

# Enable gateway mode
gateway_enable=”YES”

A couple of comments help me keep straight which ports on the back of the system are the WAN / LAN. Of course I replaced the XXX lines with the static IP / gateway from my ISP.

Next, I was going to be using this router to assign local IP’s with DHCP, so I first installed the DHCP server from PC-BSD’s package repo:

# pkg install net/isc-dhcp43-server

Next, I configured my DHCP information in /usr/local/etc/dhcpd.conf

# option definitions common to all supported networks…
option domain-name “XXXXXXXXX”;
option domain-name-servers 8.8.8.8, 8.8.4.4;
default-lease-time 86400;
max-lease-time 90000;

# Use this to enble / disable dynamic dns updates globally.
#ddns-update-style none;

# If this DHCP server is the official DHCP server for the local
# network, the authoritative directive should be uncommented.
authoritative;

# Use this to send dhcp log messages to a different log file (you also
# have to hack syslog.conf to complete the redirection).
log-facility local7;

# Home Network
subnet 192.168.1.0 netmask 255.255.255.0 {
range 192.168.1.100 192.168.1.254;
option domain-name-servers 8.8.8.8, 8.8.4.4;
option broadcast-address 192.168.1.255;
option routers 192.168.1.1;
default-lease-time 86400;
max-lease-time 90000;
}

And finally enable the service in /etc/rc.conf

# DHCPD config
dhcpd_enable=”YES”
dhdpd_ifaces=”em0″

With DHCPD now enabled, I next went and created /etc/natd.conf with the following:

interface em1
use_sockets yes
same_ports yes
dynamic yes
log_denied yes
log yes

Optionally if you run services behind NAT and need to forward ports, you can do so with syntax like the following:
# Sample redirect port
# redirect_port tcp 192.168.1.20:80 80

Where 192.168.1.20 is the local IP of your system hosting a service, and “:80 80” being the local server port, and port on your router to open.

For our last config, we now need to setup our firewall rules, which we will do by creating the file /etc/ipfw.rules with a setup something like this:

#!/bin/sh
# Flush out the list before we begin.
ipfw -q -f flush

# Set rules command prefix
cmd=”ipfw -q add”
pif=”em1″ # interface name of NIC attached to Internet
iif=”em0″ # interface name of NIC attached to LAN
server=”192.168.1.1″
inside=”192.168.1.0/24″
me=”XXX.XXX.XXX.XXX” # Public WAN address
ks=”keep-state” # Laziness
skip=”skipto 600″

# Allow everything through the local NIC
######################################
$cmd 00020 allow all from any to any via em0

######################################
# No restrictions on Loopback Interface
######################################
$cmd 00025 allow all from any to any via lo0

# NAT the inbound stuff
######################################
$cmd 00100 divert natd ip from any to any in via $pif # NAT any inbound packets

######################################
# Allow packet through if it matches existing entry in dynamic rules

######################################
$cmd 00105 check-state

######################################
# Allow all outgoing packets
######################################
$cmd 00110 $skip all from any to any out via $pif $ks

# Deny all inbound traffic from non-routable reserved address spaces
######################################
$cmd 00300 deny all from 192.168.0.0/16 to any in via $pif #RFC 1918 private IP
$cmd 00301 deny all from 172.16.0.0/12 to any in via $pif #RFC 1918 private IP
$cmd 00302 deny all from 10.0.0.0/8 to any in via $pif #RFC 1918 private IP
$cmd 00303 deny all from 127.0.0.0/8 to any in via $pif #loopback
$cmd 00304 deny all from 0.0.0.0/8 to any in via $pif #loopback
$cmd 00305 deny all from 169.254.0.0/16 to any in via $pif #DHCP auto-config
$cmd 00306 deny all from 192.0.2.0/24 to any in via $pif #reserved for doc’s
$cmd 00307 deny all from 204.152.64.0/23 to any in via $pif #Sun cluster interconnect
$cmd 00308 deny all from 224.0.0.0/3 to any in via $pif #Class D & E multicast
######################################

# Deny public pings
######################################
$cmd 00310 deny icmp from any to any in via $pif

# Log all the other troublemakers
$cmd 00550 deny log all from any to any

# Skip location for NATD
$cmd 600 divert natd ip from any to any out via $pif # skipto location for outbound stateful rules
$cmd 610 allow ip from any to any

 

Now that NATD / IPFW are setup, we can set them to run at boot with the following /etc/rc.conf entries:

# IPFW
firewall_enable=”YES”
firewall_logging=”YES”
firewall_script=”/etc/ipfw.rules”

# NATD
natd_enable=”YES”
natd_interface=”em1″
natd_flags=”-m -f /etc/natd.conf”

If you don’t want to reboot the server you can start the services with the following commands:

# service natd start
# service isc-dhcpd start
# service ipfw start

Also, the ipfw.rules file is just a shell-script, so you can always re-apply the firewall rules with “sh /etc/ipfw.rules”.

That’s it! At this point you should be able to replace your consumer router with the new system, and it’ll begin handling NAT traffic, assigning DHCP addresses, etc. Even during my network’s peak usage, I’ve yet to see the system drop below 95% idle.

 

 

Bonus content!

Some users may want to do NAT reflection for public facing services. This basically means you can have clients, such as your phone, connect to the same public address, regardless of being on the local LAN, or roaming outside the home. OpenBSD has a great explanation of this, along with the methods of solving it. I was able to easily do this with IPFW and TCP proxying. To get started, install tcpproxy:

# pkg install tcpproxy

Next, we have to create our /usr/local/etc/tcpproxy.conf file:

listen * 50080
{
  remote: 192.168.1.20 80;
  remote-resolv: ipv4;
};

In the example above, we are going to listen on port 50080 on the local router machine, and proxy this to the remote machine on the LAN: 192.168.1.20 on port 80. Next we will need to add a firewall rule to /etc/ipfw.rules causing it to redirect traffic coming from the LAN NIC to this service:

# Setup NAT reflection
$cmd 00002 fwd 127.0.0.1,50080 tcp from 192.168.1.0/24 to $me 80 in via em0

With those entries complete, you can now start the tcpproxy service, and restart IPFW to enable them.

Even More Bonus Content!

After I posted this article, I had some time to work on converting our firewall to using stateful in-kernel NAT, instead of the older userland NATd. This is how I did the conversion:

1. Disabled natd_* entries in /etc/rc.conf and changed my firewall settings to look like this:

firewall_enable=”YES”
firewall_logging=”YES”
firewall_nat_enable=”YES”
firewall_script=”/etc/ipfw.rules”

2. Change a sysctl and save it in /etc/sysctl.conf to enable more than one pass through the firewall

# Allow more than one pass through FW
net.inet.ip.fw.one_pass=0

3. Updated my /etc/ipfw.rules to look something like this:

#!/bin/sh
# Flush out the list before we begin.
ipfw -q -f flush

# Set rules command prefix
cmd=”ipfw add”
pif=”em1″     # interface name of NIC attached to Internet
iif=”em0″     # interface name of NIC attached to LAN
server=”192.168.1.1″
inside=”192.168.1.0/24″
me=”XXX.XXX.XXX.XXX”
ks=”keep-state” # Laziness
skip=”skipto 600″

# Create the NAT redirect rules
######################################
ipfw -q nat 1 config if $pif unreg_only reset \
    redirect_port tcp <lan ip>:<lan port> <remote port> \
    redirect_port udp <lan ip>:<lan port> <remote port>

######################################

# Setup all the NAT reflection stuff
######################################
$cmd 00002 fwd 127.0.0.1,50080 tcp from 192.168.1.0/24 to $me 80 in via em0
######################################

# Allow everything through the local NIC
######################################
$cmd 00020 allow all from any to any via em0
######################################

# No restrictions on Loopback Interface
######################################
$cmd 00025 allow all from any to any via lo0
######################################

# Catch spoofing from outside
######################################
$cmd 00090 deny ip from any to any not antispoof in
######################################

# NAT the inbound stuff
######################################
$cmd 0100 nat 1 ip from any to any in via $pif
######################################

# Allow packet through if it matches existing entry in dynamic rules
######################################
$cmd 00101 check-state
######################################

# Allow all outgoing packets
######################################
$cmd 00110 $skip tcp from any to any out via $pif setup $ks
$cmd 00120 $skip udp from any to any out via $pif $ks
######################################

# Deny all inbound traffic from non-routable reserved address spaces
######################################
$cmd 00300 deny all from 192.168.0.0/16 to any in via $pif   #RFC 1918 private IP
$cmd 00301 deny all from 172.16.0.0/12 to any in via $pif    #RFC 1918 private IP
$cmd 00302 deny all from 10.0.0.0/8 to any in via $pif       #RFC 1918 private IP
$cmd 00303 deny all from 127.0.0.0/8 to any in via $pif      #loopback
$cmd 00304 deny all from 0.0.0.0/8 to any in via $pif        #loopback
$cmd 00305 deny all from 169.254.0.0/16 to any in via $pif   #DHCP auto-config
$cmd 00306 deny all from 192.0.2.0/24 to any in via $pif     #reserved for doc’s
$cmd 00307 deny all from 204.152.64.0/23 to any in via $pif  #Sun cluster interconnect
$cmd 00308 deny all from 224.0.0.0/3 to any in via $pif      #Class D & E multicast
######################################

# Deny public pings
######################################
$cmd 00310 deny icmp from any to any in via $pif
######################################

# Allow specific ports IN now (for services behind NAT)
######################################
$cmd 00425 $skip tcp from any to any 80 in via $pif setup $ks
######################################

# Deny all other troublemakers
$cmd 00550 deny tcp from any to any via $pif
$cmd 00551 deny udp from any to any via $pif

# Skip location for NAT
$cmd 600 nat 1 ip from any to any out via $pif
$cmd 610 allow ip from any to any

That’s it! Now Instead of sitting at 97-98% idle, top is showing 99-100% idle, even with some moderate network traffic.

Share This Post:
  • Digg
  • Facebook
  • Twitter
  • email
  • LinkedIn
  • Slashdot

Written by Kris Moore. Posted in Guides, hardware

Trackback from your site.

Comments (5)

  • sg1efc
    January 12, 2015 at 12:23 pm |

    Cool info, Thanks Kris! 🙂

  • […] Using TrueOS as a IPFW based home router. […]

  • jdakhayman
    April 23, 2015 at 11:02 am |

    Hello Kris,
    Big fan of BSD now. Thanks for your and Allan’s work on the show. Quick question on this blog article.
    I’ve been studying this guide for a little while and I plan to use it in a router build using a PC engine APUD4. But I have few different variables than your set up and was wondering if I could get some clarification if possible. I do not have a static ip address. I will be making a connection to my ISP using pppoe, which will issue an ip from a pool and can change. I have looked at your firewall rules I see a variable that is set “me=XXX.XXX.XXX.XXX”. Now, I understand that the wan ip address goes here, but what if my wan ip address changes due to the pppoe connection dropping? What command or variable would be used instead of the hard coded ip address? Is it possible to just reference the interface or the tun0 device that ppp creates on connect?

    Also you have two other variables set for server=“192.168.1.1″ and inside=“192.168.1.0/24″ I’ve read though the rule set several time ,but I can not see them ever used like the other ones. What are they used for, if I may ask?
    Thanks in advance to any replies.

    jda

  • RCD
    May 15, 2015 at 10:28 am |

    Fantastic. Next step: Configure OpenVPN client on that server. Use that tunnel to encrypt entire entire outgoing/incoming traffic.

  • Helix
    January 31, 2016 at 2:56 am |

    Hi Kris & other readers of this post. I would like to point out, that if you are like me and use copy/paste to transfer code from this tutorial, that I went trough small hell making this tutorial work for me. And all because of small difference in the way double quotes are displayed in upper text. FreeBSD does not like ”YES” but expects “YES”. If you look close enough, you will spot the difference, which is crucial for this config files to work.

    Best regards & keep up the good work!

Leave a comment

*

Please leave these two fields as-is:

Help the Project, Donate Today!