Virtualized IPv4 home router
Introduction
I have a very limited trust in my ISP. In the country where I live, much like almost everywhere in Europe, we have pretty good Internet service with routers provided by the ISP. They give you a box, you connect it to your landline, and ta-daa the Web is all yours. Except you have zero control over said box and it’s basically an open door towards all your equipment for the ISP.
While this may sound paranoid, it’s a situation I’m not very comfortable with. Since by ISP box has a very limited administration interface, I can’t tinker with routing setting, and as such have to do chained NAT:
Having tried various all-in-one distribution like IPFire or pfSense, I always ended up wanting to throw them away for one reason or another. Sometimes it was poor KVM functionnality, sometimes it was the absurd difficulty to get something non-standard done. And I love doing non-standard things!
So, in the end, I ended up making my own virtualized router based on a CentOS 7 for my home network. I did this as a part of a global overhaul of my home network, following the arrival of my new home server. In this article, I’ll cover all the steps, from the initial setup until the moment the router in put into production.
The only prerequisite is having a running CentOS 7 KVM hypervisor with two network interfaces and Open vSwitch already setup. A similar setup with other technologies will also work, but the schematics will have to be adapted. In this document, I assume you are already familiar with virtualization and the virt-manager tool.
In this article, I only cover the IPv4 aspects. Adding IPv6 functionnaly to that setup will come later in another article.
Overview of the functionnalities
At the end of the article, the router will have the following features operational:
- DHCP server;
- DNS server (local zone + cached relay towards public servers);
- Dynamic DNS client;
- iptables firewall;
- OpenVPN server.
Since we’re installing these services as a virtual machine, we won’t be creating an NTP server now. Running an NTP server in a VM is a very bad idea, because they don’t have a physical clocksource to keep track of passing time, and are often unstable as such. VMware has a great whitepaper to explain this, but I couldn’t get my hands on it anymore when writing this.
However, I’ll probably build a stable NTP server with a Raspberry PI and a GPS/DCF77 receiver some day. Time will tell (no pun intended).
VM installation
Placement in the network
I did this as part of the overhaul of my network, and wanted to start from a clean sheet. I already have an hypervisor setup with CentOS 7, KVM and Open vSwitch. Initially, the router VM will just be an host connected to my ISP Lan through the host:
When we’re done, the setup will look like this (well, not exactly: I have multiple VLANs setup to do various things, but we’ll cover that in another post):
The host NIC is connected to the Home Lan, behind the router (eth0/eno1/brHome), while the Router VM is connected to the ISP Lan using a USB 3 NIC (ens4u1c2, Anker Aluminum USB 3.0 to Ethernet Adapter) passed through to the router VM.
VM creation & OS installation
My OS of choice for this kind of systems is CentOS 7. Although it can function with much less when not asked to do too much, it requires 1 Gig of RAM.
Also, I have two storage arrays in my host. One is an SSD, the other one is a good old RAID1 array of HDDs. As this VM is a core component of my network and won’t do much writes, I put it on the SSD. In the future, funny things like an IDS will certainly come up. Their data will go on additional partitions located on the HDD if deemed necessary.
So, we setup the VM as follows:
- 1 vCPU
- 1024 MB of RAM
- 1 vHDD of 4 GB on the SSD storage
- Spice setup to listen on all interfaces instead of just localhost (Don’t forget to do this, or you won’t be able to access the VM from another station using virt-manager !)
- The USB NIC (connected to nothing at the moment) passed through to the VM
We then proceed with a minimal installation of CentOS 7. Regarding the partition setup in the VMs:
- As I don’t except the storage to grow or change much in the future, I went with plain old partitions. 500MB for /boot, the rest for / (I might consider splitting the storage into different partitions to inscrease security in the future);
- The VM goes on the SSD: don’t create a swap partition !
No low privilege user initially. I’ll cover my view on user management in another post.
Software configuration
Now that we have a freshly installed CentOS 7 instance basically capable of doing nothing, let’s install some software ! I belieive a router should be as simple as possible and avoid using fancy desktop tools, to keep everything under total control. As such, we remove things like NetworkManager in the process.
# yum remove NetworkManager* ModemManager* firewalld* # yum upgrade -y # yum installe epel-release -y # yum install net-tools usbutils bind-utils htop dnsmasq
As announced in the introduction, for now we focus on IPv4. IPv6 will be added to the setup later. So, we change the sysctl settings to enable IPv4 forwarding and fully disable IPv6:
Added to /etc/sysctl.conf net.ipv6.conf.all.disable_ipv6 = 1 net.ipv4.ip_forward = 1
Now, we’ll configure the network interfaces. To get your interfaces list, use sysfs:
# ls /sys/class/net ens4u1c2 eth0 lo
ens4u1c2 is the USB network interface, which will be connected to the ISP Lan. I leave that one in DHCP to let my ISP box assign it an IP address:
# cat /etc/sysconfig/network-scripts/ifcfg-eth0 TYPE="Ethernet" BOOTPROTO="static" ONBOOT="yes" IPADDR="192.168.11.254" NETMASK="255.255.255.0" DEVICE="eth0" NAME="eth0"
eth0 is the interface connected to the brHome vSwitch, which is my real Home Lan. On that one, I put a fixed IP address at the end of my chosen private subnet, 192.168.11.0/24:
# cat /etc/sysconfig/network-scripts/ifcfg-ens4u1c2 TYPE="Ethernet" BOOTPROTO="dhcp" DEFROUTE="yes" PEERROUTES="yes" IPV4_FAILURE_FATAL="no" NAME="ens4u1c2" DEVICE="ens4u1c2" ONBOOT="yes"
We ensure the good old network service is enabled to take care of these network interfaces:
systemctl enable network.service
Job done ! Shut down the virtual machine, and swap the cables (ISP box on USB NIC, internal network on host machine’s NIC).
Start the virtual machine again. To regain access to the virtual machines, give yourself an IP address in the private range and connect your computer on the host computer’s NIC. From the virtual machine’s shell, you should have access to the Internet through the ISP box.
At this point, it’s a good idea to make sure IPv6 is not running on the router. This way, you’re 100% percent sure there won’t be any traffic backdoor on the router that could work around the firewall.
# ip -6 a
If this command returns nothing, you’re now the proud user of an IPv4-only machine.
DHCP & DNS
Now it’s time to get ourselves a DHCP and DNS service. Since my network is not that big and I don’t have a complex DNS zone to manage, I’m pretty happy to live with dnsmasq. While initially intended to be a caching DNS relay, it does a pretty good job being a full blown DHCPv4 & DNSv4 server in a small network.
Here’s my main dnsmasq config file, along with some comments:
# Never forward plain names (without a dot or domain part) # dnsmasq will not attempt to resolve queries such as "microsoft" with # upstream DNS servers, it will only use its own private LAN info. # However, if it receives a query for "microsoft.com" and cannot # resolve it with its private lan info, then it will query the upstream # server domain-needed # Never forward addresses in the non-routed address spaces. # If the DNS server gets a reverse query for a private IP address # (10.0.0.0/8, etc.), it will not query the upstream server. bogus-priv # As I said, this "core" router is a critical component of the network. # We want to have everything under control, including the information # sources of all services. As such, dnsmasq will only use the servers # specified in this config file, and ignore the /etc/resolv.conf file no-resolv # Since we don't care about resolv.conf, we don't poll it for changes no-poll # Using the server option, we specify the upstream servers to use. # Basically, dnsmasq will act as the DNS server of the Home Lan, # and use upstream servers for all requests outside the local # network. Here, I use Google and OpenDNS' servers. # As I said earlier, I don't trust my ISP. server=8.8.8.8 server=208.67.222.222 server=208.67.220.220 # The internal DNS zone, managed by dnsmasq, will be called "lan" local=/lan/ # We want dnsmasq to listen for DHCP & DNS queries only on the # Home (internal) lan, and not on the ISP Lan side. interface=eth0 # Similarly to /etc/resolv.conf, we don't look at /etc/hosts for DNS records # definition. no-hosts # The "local" setting is used for DNS. Similarly, we now define a DHCP # domain, to the same value. The DHCP server will push the associated # option domain=lan # We enable the DHCP server on the range 192.168.11.1 to 192.168.11.199, # with leases of 12 hours. dhcp-range=192.168.11.1,192.168.11.199,255.255.255.0,12h # dnsmasq is the one and only server in the internal zone, and should # proudly advertise so. For more info: http://www.isc.org/files/auth.html dhcp-authoritative # Set the DNS cache to 2000 entries (default is 150, which turns out to be # pretty low) cache-size=2000 # For DNS entries declarations, we'll use separate files in a dedicated # folder: conf-dir=/etc/dnsmasq.d
In /etc/dnsmasq.d, we create .conf files for the DHCP reservations and the DNS records, following this syntax:
address=/XperiaZ1.lan/192.168.11.100 ptr-record=100.11.168.192.in-addr.arpa,"XperiaZ1.lan" dhcp-host=30:a8:db:6e:ac:0b,192.168.11.100
- First line is the forward DNS record (Name to IP);
- Second line is the reverse DNS record (IP address to name). Be careful about the syntax: IP address is reversed and finishes with .in-addr.arpa;
- Third line is the DHCP reservation.
Of course, if you just want to create a DNS record without an associated DHCP reservation, the third line is not needed.
Regarding the cache size, to know the cache usage, you can use the following command:
# kill -s USR1 $(ps ax | grep dnsmasq) && cat /var/log/messages | grep dnsmasq
You will get this kind of output:
xxx dnsmasq[y]: time 1476458608 xxx dnsmasq[y]: cache size 2000, 0/69 cache insertions re-used unexpired cache entries. xxx dnsmasq[y]: queries forwarded 42, queries answered locally 21 xxx dnsmasq[y]: server 8.8.8.8#53: queries sent 8, retried or failed 0 xxx dnsmasq[y]: server 208.67.222.222#53: queries sent 42, retried or failed 1 xxx dnsmasq[y]: server 208.67.220.220#53: queries sent 8, retried or failed 0
The line in bold is of interest to us: it means that over 69 calls to the dnsmasq cache, 0 could by answered by an existing entry. After the router has been running for some time, this ratio should be close to 1. It it doesn’t rise, it means your cache is too small and you should increase it.
Once you’ve populated your configuration file, enable dnsmasq at boot and start it:
# systemctl enable dnsmasq # systemctl start dnsmasq
Confirm that the DHCP and DNS services are running:
# ss -uan State Recv-Q Send-Q Local Address:Port Peer Address:Port UNCONN 0 0 *:53 *:* DNS UNCONN 0 0 *:67 *:* DHCP UNCONN 0 0 *:68 *:* DHCP
From that point on, you may switch back to DHCP on any machine connected to the Home Lan. It will automatically receive an IP address from dnsmasq, with DNS and routing settings. Here is an example of the output from ipconfig/all on a Windows client (sorry, my machine is in French):
Carte réseau sans fil Wi-Fi :
Suffixe DNS propre à la connexion. . . : lan -> DNS domain
Description. . . . . . . . . . . . . . : TP-LINK Wireless PCI Express Adapter
Adresse physique . . . . . . . . . . . : E8-DE-27-A1-E1-2A
DHCP activé. . . . . . . . . . . . . . : Oui
Configuration automatique activée. . . : Oui
Adresse IPv4. . . . . . . . . . . . . .: 192.168.11.105(préféré)
-> IP address from the DHCP pool
Masque de sous-réseau. . . . . . . . . : 255.255.255.0
Bail obtenu. . . . . . . . . . . . . . : jeudi 13 octobre 2016 17:48:28
Bail expirant. . . . . . . . . . . . . : vendredi 14 octobre 2016 08:15:28
Passerelle par défaut. . . . . . . . . : 192.168.11.254 -> Gateway
Serveur DHCP . . . . . . . . . . . . . : 192.168.11.254
Serveurs DNS. . . . . . . . . . . . . : 192.168.11.254 -> DNS server
NetBIOS sur Tcpip. . . . . . . . . . . : Activé
However, you won’t have access to the Internet yet. For this, we have one more thing to do: configure IPtables.
The reason is simple: my ISP doesn’t let me change anything on the ISP box, meaning I can’t add a route to my internal LAN on it. Hence, the only solution is to have the router NAT the IPv4 addresses between the Home Lan and the ISP Lan.
Firewall
Here below, I propose a very basic script that configures IPtables to:
- Realise the aforementioned NAT from the Home Lan to the ISP Lan;
- Allowing routing between both networks as long as the packets are not marked as INVALID by the kernel;
- Block all direct connections to the router, except SSH, ICMP, DHCP & DNS traffic from the Home Lan.
This script could take a lot of improvement, e.g. use the kernel’s native martian packets filtering measures. But that’ll have to be covered another day. The script below is located at /opt/firewall.sh :
#!/bin/bash # Home router - Firewall script # Variables iptables="/sbin/iptables" lanHomeIf="eth0" -> Adapt depending on your setup wanIf="ens4u1c2" -> Adapt depending on your setup inputHomeLanUdpPorts="53,67,68" inputHomeLanTcpPorts="22" inputWanUdpPorts="3389" spoofIps="0.0.0.0/8 127.0.0.1/8" # Initial flush $iptables -F $iptables -X $iptables -t nat -F $iptables -t nat -X # Policies $iptables -P INPUT DROP $iptables -P OUTPUT DROP $iptables -P FORWARD DROP # Drop all spoofed IPs for ip in $spoofIps; do $iptables -A INPUT -i $wanIf -s $ip -j DROP $iptables -A INPUT -i $wanIf -s $ip -j DROP done # INPUT $iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT $iptables -A INPUT -i lo -j ACCEPT # LAN / Home side - Accept defined ranges of TCP and UDP ports $iptables -A INPUT -i $lanHomeIf -p tcp -m state --state NEW -m multiport --dports $inputHomeLanTcpPorts -j ACCEPT $iptables -A INPUT -i $lanHomeIf -p udp -m state --state NEW -m multiport --dports $inputHomeLanUdpPorts -j ACCEPT # Accept ICMP packets others than ICMP Redirect $iptables -A INPUT -p icmp -m icmp ! --icmp-type 5 -j ACCEPT # OUTPUT $iptables -A OUTPUT -m state ! --state INVALID -j ACCEPT # FORWARD $iptables -A FORWARD -m state ! --state INVALID -j ACCEPT # NAT $iptables -t nat -A POSTROUTING -o $wanIf -j MASQUERADE
To have this script executed at boot as a service, we create a systemd unit and enable it:
# chmod +x /opt/firewall.sh # cat /etc/systemd/system/firewall.service [Unit] Description=Firewall for home server [Service] Type=oneshort ExecStart=/opt/firewall.sh [Install] WantedBy=multi-user.target # systemctl enable firewall.service # systemctl start firewall.service
Dynamic DNS
Before going into the how, let’s discuss the why for a minute.
Usually, an ISP will provide you with a dynamic IPv4 address, that will change every 24-48h. While that is perfectly OK for client usage, it will prevent you from binding a domain name to your IP address to host services in your network (e.g., a VPN server to access your files or browse the web securely from an hotel).
The solution for this is known as Dynamic DNS. Basically, you let a daemon run inside your network that will detect your current public IP address and update DNS records on public DNS servers based on that info. That service can be provided either with a subscription fee or freely.
Usually, free services only provide a subdomain, e.g. ev1z.no-ip.org.
Now that we’ve covered the why, let’s go with the how. I’ve chosen to use the ChangeIP service. Once you’ve created your account and reserved your very onw subdomain, it’s time to configure the ddclient daemon to keep the DNS record up to date based on your dynamic public IPv4 address.
# yum install http://pkgs.repoforge.org/rpmforge-release/rpmforge-release-0.5.3-1.el7.rf.x86_64.rpm # yum install ddclient
I’ve based my configuration on the info found in [1]. My configuration file looks like that:
daemon=1200 # check every 20 min syslog=yes # log update msgs to syslog mail=root # mail all msgs to root mail-failure=root # mail failed update msgs to root pid=/var/run/ddclient.pid # record PID in file. ssl=yes # use ssl-support. Works with # ssl-library use=web, web=ip.changeip.com protocol=dyndns2 # default protocol server=nic.changeip.com # default server login=input your ChangeIP.com login password=input your ChangeIP.com password input your ChangeIP.com domain
Enable and start ddclient, and you’re good to go:
# systemctl enable ddclient # systemctl start ddclient
From the ChangeIP.com web interface, you should now see your public IPv4 address associated with your DNS domain name. Thanks to this, you now have a public hostname that is automatically updated with your ISP-provided IP address. Enabling access to local services from the web is now only a matter of firewall configuration. An example with OpenVPN is given below.
OpenVPN server
Last but not least, we’re going to setup an OpenVPN server to be able to access the local network from the big bad Internet. That VPN connection can also be used to encapsulate all your traffic in an encrypted tunnel, then browse the web from your home connection. Can be handy when you’re an untrusted network such as an hotel WiFi.
Let’s get down to business by installing some packages:
yum install openvpn easy-rsa
In this simple setup, we’re going to use easy-rsa. It’s a set of scripts that will generate keys (Self-signed CA, server key, client keys) for OpenVPN. In a more corporate setup, that kind of job is done with an entreprise PKI.
We copy a reference easy-rsa configuration and adapt it to make it a little bit more secure (following the advices from [2]):
# mkdir -p /etc/openvpn/easy-rsa/keys # cp -rf /usr/share/easy-rsa/2.0/* /etc/openvpn/easy-rsa # cat /etc/openvpn/easy-rsa/vars [...] export KEY_SIZE=4096 <- Generate 4096 bits asymmetric keys export CA_EXPIRE=1826 <- Root CA expires in 5 years export KEY_EXPIRE=365 <- Server and client keys expire in 1 year [..]
The expiration values will have implication on the key mangement and renewal process to be put in place. I’ll cover that in another article. Then, we generate the keys with the built-in scripts:
# cd /etc/openvpn/easy-rsa
# source ./vars
# ./clean-all
# ./build-ca
# ./build-dh
# ./build-server-key server <- alternatively you may use the hostname
instead of "server"
# cd ..
# openvpn --genkey --secret ta.key
Now it’s time to create your OpenVPN server configuration. Like for dnsmasq, here is a documented example, as /etc/openvpn/server.conf:
# General # Listen on UDP:1194 port 1194 proto udp # Create a Layer-3 tunnel interface dev tun # Downgrade to a low-privilege user once the socket has been opened # (Limits privilege escalation hack possibilities) user nobody group nobody # If the server has to restart the socket, keep using the key and # tunnel device to speed up startup persist-key persist-tun # Log files-related settings. Check the documentation to understand # what they mean; these settings give a medium verbosity and drop # repeated lines status openvpn-status.log verb 3 mute 20 # Certificates # Location of the keys we've just generated ca /etc/openvpn/easy-rsa/keys/ca.crt cert /etc/openvpn/easy-rsa/keys/server.crt key /etc/openvpn/easy-rsa/keys/server.key dh /etc/openvpn/easy-rsa/keys/dh4096.pem # IP # Have OpenVPN use a /24 subnet for the VPN addresses topology subnet server 10.8.0.0 255.255.255.0 # Store the assigned IP addresses in that file ifconfig-pool-persist ipp.txt # Push the router's address as a route to access the local network # once connected to the VPN, and receive the DNS to be able to resolve # local hosts. push "route 192.168.11.0 255.255.255.0" push "dhcp-option DNS 192.168.11.254" # Allow communication between VPN clients over the tunnel client-to-client # Send a keepalive to the client every 10 seconds, consider it is gone # and attempt a connection restartif no reply is recevied after 120 seconds keepalive 10 120 # Compress the traffic comp-lzo # Crypto # Enforce secure crypto algorithms for the communication over the tunnel tls-auth ta.key 0 tls-version-min 1.2 cipher AES-256-CBC auth SHA384
If you’re going to run OpenVPN an a port different from UDP:3389 or TCP:3389, it needs to be allowed in SELinux contexts. For instance, for UDP:1195:
# yum install policycoreutils-python # semanage port -a -t openvpn_port_t -p udp 1195
When you’re done, assuming your config file is server.cfg:
# systemctl enable openvpn@server # systemctl start openvpn@server
You should now have a running OpenVPN server:
# ss -tan State Recv-Q Send-Q Local Address:Port Peer Address:Port UNCONN 0 0 *:1194 *:*
It’s now time to generate a client key and matching configuration:
# cd /etc/openvpn/easy-rsa
# source ./vars
# ./build-key foo <- replace "foo" with a person of computer
name, this is basically the client identifier
The client configuration file should look like this:
client dev tun proto udp remote (your ChangeIP.com domain) 1194 resolv-retry infinite nobind persist-key persist-tun ca ca.crt cert foo.crt key foo.key remote-cert-tls server tls-auth ta.key 1 tls-version-min 1.2 cipher AES-256-CBC auth SHA384 comp-lzo
Pack together that config file and the files foo.crt, foo.key, ta.key and ca.crt. Ta-daaa, we’ve got a client VPN config pack ! Of course, all of that can be automated. I’ll cover that along with managing the keys expiration in a another post.
To get this to work, we have two last jobs.
First one is to redirect the incoming traffic from the outside all the way to the router VM. On the ISP box, a firewall rule must be created to re-route incoming traffic on port UDP 1194 to the router’s IP address in the ISP Lan, on port 1194. On a Proximus BBox3, this is done under Access Control > Portmapping.
Second job is to adapt the firewall. In the relevant sections, add the following lines and restart the firewall:
# cat /opt/firewall.sh inputWanUdpPorts="3389" # WAN side - Accept defined range of UDP ports $iptables -A INPUT -i $wanIf -p udp -m state --state NEW -m multiport --dports $inputWanUdpPorts -j ACCEPT # /opt/firewall.sh
VPN connection should now work from the outside.
References
[1] Use ddclient with ChangeIP.com, https://blogdotmegajasondotcom.wordpress.com/2011/03/14/use-ddclient-with-changeip-com/
[2] Applied Crypto Hardening, https://bettercrypto.org/
recoll: Going paperless without getting moneyless
Isolation Access Point with CentOS 7
Setting up the Suricata IDPS
About Author
