UniFi IPv6 and Dual Stack PPPoE

Internode offers Dual Stack IPv4/IPv6 connectivity to their broadband customers and has done so since 2011; sadly, said connectivity doesn’t work very well out of the box with the UniFi Security Gateway router.

The problem? UniFi’s router firmware (version 4.4.22.5086045) and cloud controller (version 5.8.30) configure DHCPv6 Prefix Delegation on both the WAN interface and the PPPoE interface; the outcome of which is that the router obtains an IPv6 address – but nothing else in the network does.

This bug manifests itself with any ISP who authenticates customers via PPPoE and provides dual stack connectivity on the PPPoE interface (and assigns a customer-end IPv6 address range via Prefix Delegation) – not just Internode.

Configure IPv6

In case you haven’t already done so, configure IPv6 in the Controller. Internode’s general instruction for IPv6 is that you get given a dynamic /64 for the PPP interface and a static /56 for your LAN, distributed via Prefix Delegation. So far, so easy.

In the Controller, select the site, then Settings -> Networks -> WAN. In the IPv6 section:

  • Connection Type: Using DHCPv6
  • Prefix Delegation Size: 56

Save, then select the local network you want to enable for IPv6. In the Configure IPv6 Network section:

  • IPv6 Interface Type: Prefix Delegation
  • IPv6 Prefix Delegation Interface: WAN (this is the giveaway for where the problem lies..)
  • IPv6 Prefix ID: 56 (unsure whether this is necessary)
  • IPv6 RA: Enable IPv6 Router Advertisement checked (leave settings at the default)
  • DHCPv6/RDNSS DNS Control: Auto

Save and then wait for everything to provision. In theory, this would be enough for everything to just work, but it’s not. If you SSH into the USG and run show interfaces, you’ll see that the ethx interface that corresponds to the local network you configured above doesn’t have an IPv6 address:

Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface    IP Address                        S/L  Description
---------    ----------                        ---  -----------
eth0         -                                 u/u  WAN
eth1         192.168.1.1/24                    u/u  LAN
eth2         -                                 A/D
lo           127.0.0.1/8                       u/u
             ::1/128
pppoe0       my.public.ipv4.address            u/u
             2001:my:public:ipv6:address:d32f/64

Fix Stuff

Knowing that the problem relates to Prefix Delegation being configured on the WAN interface rather than the PPPoE interface, you need to create a script which disables DHCPv6 Prefix Delegation on the WAN interface and restarts DHCPv6-PD on the PPPoE interface.

SSH into the USG and create /config/scripts/post-config.d/dhcp.sh:

#!/bin/vbash
# Script is started on reboot and via a cronjob created during provisioning

readonly logFile="/var/log/postprovision.log"

source /opt/vyatta/etc/functions/script-template

configure > ${logFile}

# Delete cronjob that called this script
delete system task-scheduler task postprovision >> ${logFile}

# Remove DHCPv6-PD from eth0 (WAN)
delete interfaces ethernet eth0 dhcpv6-pd >> ${logFile}
commit
exit

# Obtain IPv6 addresses for Prefix Delegation
release dhcpv6-pd interface pppoe0 >> ${logFile}
delete dhcpv6-pd duid >> ${logFile}
renew dhcpv6-pd interface pppoe0 >> ${logFile}

Make the script executable: sudo chmod +x /config/scripts/post-config.d/dhcp.sh

SSH into the Controller and either create or update the file [unifi_base]/data/sites/site_ID/config.gateway.json – on Debian, and assuming the site ID is “default”, the config.gateway.json file is located at /usr/lib/unifi/data/sites/default/:

{
  "system": {
    "task-scheduler": {
      "task": {
        "postprovision": {
          "executable": {
            "path": "/config/scripts/post-config.d/dhcp.sh"
          },
          "interval": "2m"
        }
      }
    }
  }
}

Always check that the file contents are valid JSON using JSONLint – this will prevent problems with provisioning the USG.

Finally, force a provision of the USG – log into the Controller, select the site, then Devices -> USG -> Config -> Manage Device -> Provision.

Once provisioned, IPv6 should be working on your network!

Note: If your ISP requires a VLAN tag on the WAN interface, there are slightly different steps required – see the first source below.

Sources:
https://spflug.de/?p=175
https://help.ubnt.com/hc/en-us/articles/215458888-UniFi-USG-Advanced-Configuration

UniFi Controller 5.8 on Debian 9

VPS services in Australia have become cheap enough for me to have a dedicated VM to run a UniFi controller, which changes how I’ve gone about setting it all up as compared with a past effort.

The UniFi controller is pretty memory hungry; I’ve got my VM configured with 1.5 GB of RAM and wouldn’t recommend anything less than that. 2 GB of RAM may be better on busier controllers. Aside from RAM, the system requirements are fairly minimal.

This recipe includes setting up the pre-requisites, installing UniFi, adding a firewall and an automatically-updating SSL certificate from Let’s Encrypt.

Pre-requisites

It’s a brand new VM, so make sure everything is up to date – plus install a few dependencies. I’ve had problems with package dependency conflicts when installing the UniFi .deb file, so I prefer to get a few things installed ahead of time.

Note that the versions of the Linux kernel and MongoDB are important! If you’re using this guide with versions of Linux other than Debian 9 – the latest version of Ubuntu has an incompatible version of MongoDB and earlier versions of Debian need a patched kernel. Caveat emptor.. 😉

sudo apt-get update && sudo apt-get upgrade && sudo apt-get dist-upgrade
sudo apt-get install mongodb openjdk-8-jre-headless jsvc ufw curl

Install the UniFi controller

As of now, version 5.8 of the UniFi controller is limited to beta testers – but joining the beta program is easy/quick/painless.

The latest 5.8 controller is available in the UniFi Beta Blog – download the Debian/Ubuntu .deb file and run the following:

sudo dpkg -i unifi_sysvinit_all.deb

If you’re upgrading from an earlier release, expect things to take a little while to stabilise post-install.

Configure the ufw firewall

If you’ve never configured ufw before, please see this DigitalOcean guide – it’s pretty well written and covers how to open up SSH. For obvious reasons, don’t turn on the firewall if you haven’t confirmed that you’ll still have access afterwards!

Create /etc/ufw/applications.d/unifi:

[UniFi]
title=UniFi Controller
description=UniFi Controller
ports=8080,8443,8880,8843/tcp|3478/udp

Enable the UniFi rules and confirm they’re working:

sudo ufw allow unifi
sudo ufw status verbose

Add Let’s Encrypt SSL

By default, the UniFi controller includes an untrusted certificate – which is almost useless, given how much the browsers complain about such certificates. While you can definitely get a certificate from a commercial CA, getting a certificate from Let’s Encrypt is fairly painless (and free!)

The first time you run the certbot command, you’ll need to agree to the Terms of Service and provide an email address – this is a first-time-only thing. You’ll also need to update the DNS name in the second command:

sudo apt-get install certbot
sudo certbot certonly --standalone --preferred-challenges http --pre-hook "ufw allow http" --post-hook "ufw deny http" -d your.dns.name.here

Once you’ve got the certificate, run a script to replace the certificate that comes with the UniFi controller, then add the script to cron to ensure the certificate is renewed and installed into the UniFi controller automagically.

Create /usr/local/bin/unifi-ssl-import.sh:

#!/usr/bin/env bash

# unifi_ssl_import.sh
# UniFi Controller SSL Certificate Import Script for Unix/Linux Systems
# by Steve Jenkins 
# Part of https://github.com/stevejenkins/ubnt-linux-utils/
# Incorporates ideas from https://source.sosdg.org/brielle/lets-encrypt-scripts
# Version 2.8
# Last Updated Jan 13, 2017

# Minor adjustments by Zac Ariel, called out with {ZA}
# Updated Sep 28, 2017

# REQUIREMENTS
# 1) Assumes you have a UniFi Controller installed and running on your system.
# 2) Assumes you already have a valid 2048-bit private key, signed certificate, and certificate authority
#    chain file. The Controller UI will not work with a 4096-bit certificate. See http://wp.me/p1iGgP-2wU
#    for detailed instructions on how to generate those files and use them with this script.

# KEYSTORE BACKUP
# Even though this script attempts to be clever and careful in how it backs up your existing keystore,
# it's never a bad idea to manually back up your keystore (located at $UNIFI_DIR/data/keystore on RedHat
# systems or /$UNIFI_DIR/keystore on Debian/Ubuntu systems) to a separate directory before running this
# script. If anything goes wrong, you can restore from your backup, restart the UniFi Controller service,
# and be back online immediately.

# CONFIGURATION OPTIONS
#UNIFI_HOSTNAME=hostname.example.com
UNIFI_HOSTNAME=your.dns.name.here
UNIFI_SERVICE=unifi

# Uncomment following three lines for Fedora/RedHat/CentOS
#UNIFI_DIR=/opt/UniFi
#JAVA_DIR=${UNIFI_DIR}
#KEYSTORE=${UNIFI_DIR}/data/keystore

# Uncomment following three lines for Debian/Ubuntu
UNIFI_DIR=/var/lib/unifi
JAVA_DIR=/usr/lib/unifi
KEYSTORE=${UNIFI_DIR}/keystore

# FOR LET'S ENCRYPT SSL CERTIFICATES ONLY
# Generate your Let's Encrtypt key & cert with certbot before running this script
#LE_MODE=no
LE_MODE=yes
LE_LIVE_DIR=/etc/letsencrypt/live

# THE FOLLOWING OPTIONS NOT REQUIRED IF LE_MODE IS ENABLED
#PRIV_KEY=/etc/ssl/private/hostname.example.com.key
#SIGNED_CRT=/etc/ssl/certs/hostname.example.com.crt
#CHAIN_FILE=/etc/ssl/certs/startssl-chain.crt

# CONFIGURATION OPTIONS YOU PROBABLY SHOULDN'T CHANGE
ALIAS=unifi
PASSWORD=aircontrolenterprise

#### SHOULDN'T HAVE TO TOUCH ANYTHING PAST THIS POINT ####

printf "\nStarting UniFi Controller SSL Import...\n"

# Check to see whether Let's Encrypt Mode (LE_MODE) is enabled

if [[ ${LE_MODE} == "YES" || ${LE_MODE} == "yes" || ${LE_MODE} == "Y" || ${LE_MODE} == "y" || ${LE_MODE} == "TRUE" || ${LE_MODE} == "true" || ${LE_MODE} == "ENABLED" || ${LE_MODE} == "enabled" || ${LE_MODE} == 1 ]] ; then
        LE_MODE=true
        printf "\nRunning in Let's Encrypt Mode...\n"
        PRIV_KEY=${LE_LIVE_DIR}/${UNIFI_HOSTNAME}/privkey.pem
        SIGNED_CRT=${LE_LIVE_DIR}/${UNIFI_HOSTNAME}/cert.pem
        CHAIN_FILE=${LE_LIVE_DIR}/${UNIFI_HOSTNAME}/chain.pem
else
        LE_MODE=false
        printf "\nRunning in Standard Mode...\n"
fi

if [ ${LE_MODE} == "true" ]; then
        # Check to see whether LE certificate has changed
        printf "\nInspecting current SSL certificate...\n"
        if md5sum -c ${LE_LIVE_DIR}/${UNIFI_HOSTNAME}/cert.pem.md5 &>/dev/null; then
                # MD5 remains unchanged, exit the script
                printf "\nCertificate is unchanged, no update is necessary.\n"
                exit 0
        else
        # MD5 is different, so it's time to get busy!
        printf "\nUpdated SSL certificate available. Proceeding with import...\n"
        fi
fi

# Verify required files exist
if [ ! -f ${PRIV_KEY} ] || [ ! -f ${SIGNED_CRT} ] || [ ! -f ${CHAIN_FILE} ]; then
        printf "\nMissing one or more required files. Check your settings.\n"
        exit 1
else
        # Everything looks OK to proceed
        printf "\nImporting the following files:\n"
        printf "Private Key: %s\n" "$PRIV_KEY"
        printf "Signed Certificate: %s\n" "$SIGNED_CRT"
        printf "CA File: %s\n" "$CHAIN_FILE"
fi

# Create temp files
P12_TEMP=$(mktemp)
CA_TEMP=$(mktemp)

# Stop the UniFi Controller
printf "\nStopping UniFi Controller...\n"
# {ZA} update to a systemd style command
#service ${UNIFI_SERVICE} stop
systemctl stop ${UNIFI_SERVICE}

if [ ${LE_MODE} == "true" ]; then
        
        # Write a new MD5 checksum based on the updated certificate     
        printf "\nUpdating certificate MD5 checksum...\n"

        md5sum ${LE_LIVE_DIR}/${UNIFI_HOSTNAME}/cert.pem > ${LE_LIVE_DIR}/${UNIFI_HOSTNAME}/cert.pem.md5 
        
        # Create local copy of cross-signed CA File (required for keystore import)
        # Verify original @ https://www.identrust.com/certificates/trustid/root-download-x3.html
  cat > "${CA_TEMP}" <<'_EOF'
-----BEGIN CERTIFICATE-----
MIIDSjCCAjKgAwIBAgIQRK+wgNajJ7qJMDmGLvhAazANBgkqhkiG9w0BAQUFADA/
MSQwIgYDVQQKExtEaWdpdGFsIFNpZ25hdHVyZSBUcnVzdCBDby4xFzAVBgNVBAMT
DkRTVCBSb290IENBIFgzMB4XDTAwMDkzMDIxMTIxOVoXDTIxMDkzMDE0MDExNVow
PzEkMCIGA1UEChMbRGlnaXRhbCBTaWduYXR1cmUgVHJ1c3QgQ28uMRcwFQYDVQQD
Ew5EU1QgUm9vdCBDQSBYMzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB
AN+v6ZdQCINXtMxiZfaQguzH0yxrMMpb7NnDfcdAwRgUi+DoM3ZJKuM/IUmTrE4O
rz5Iy2Xu/NMhD2XSKtkyj4zl93ewEnu1lcCJo6m67XMuegwGMoOifooUMM0RoOEq
OLl5CjH9UL2AZd+3UWODyOKIYepLYYHsUmu5ouJLGiifSKOeDNoJjj4XLh7dIN9b
xiqKqy69cK3FCxolkHRyxXtqqzTWMIn/5WgTe1QLyNau7Fqckh49ZLOMxt+/yUFw
7BZy1SbsOFU5Q9D8/RhcQPGX69Wam40dutolucbY38EVAjqr2m7xPi71XAicPNaD
aeQQmxkqtilX4+U9m5/wAl0CAwEAAaNCMEAwDwYDVR0TAQH/BAUwAwEB/zAOBgNV
HQ8BAf8EBAMCAQYwHQYDVR0OBBYEFMSnsaR7LHH62+FLkHX/xBVghYkQMA0GCSqG
SIb3DQEBBQUAA4IBAQCjGiybFwBcqR7uKGY3Or+Dxz9LwwmglSBd49lZRNI+DT69
ikugdB/OEIKcdBodfpga3csTS7MgROSR6cz8faXbauX+5v3gTt23ADq1cEmv8uXr
AvHRAosZy5Q6XkjEGB5YGV8eAlrwDPGxrancWYaLbumR9YbK+rlmM6pZW87ipxZz
R8srzJmwN0jP41ZL9c8PDHIyh8bwRLtTcm1D9SZImlJnt1ir/md2cXjbDaJWFBM5
JDGFoqgCWjBH4d1QB7wCCZAA62RjYJsWvIjJEubSfZGL+T0yjWW06XyxV3bqxbYo
Ob8VZRzI9neWagqNdwvYkQsEjgfbKbYK7p2CNTUQ
-----END CERTIFICATE-----
_EOF
fi

# Create double-safe keystore backup
if [ -s "${KEYSTORE}.orig" ]; then
        printf "\nBackup of original keystore exists!\n"
        printf "\nCreating non-destructive backup as keystore.bak...\n"
        cp ${KEYSTORE} ${KEYSTORE}.bak
else
        cp ${KEYSTORE} ${KEYSTORE}.orig
        printf "\nNo original keystore backup found.\n"
        printf "\nCreating backup as keystore.orig...\n"
fi
         
# Export your existing SSL key, cert, and CA data to a PKCS12 file
printf "\nExporting SSL certificate and key data into temporary PKCS12 file...\n"

openssl pkcs12 -export \
-in ${SIGNED_CRT} \
-inkey ${PRIV_KEY} \
-CAfile ${CHAIN_FILE} \
-out ${P12_TEMP} -passout pass:${PASSWORD} \
-caname root -name ${ALIAS}
        
# Delete the previous certificate data from keystore to avoid "already exists" message
printf "\nRemoving previous certificate data from UniFi keystore...\n"
keytool -delete -alias ${ALIAS} -keystore ${KEYSTORE} -deststorepass ${PASSWORD}
        
# Import the temp PKCS12 file into the UniFi keystore
printf "\nImporting SSL certificate into UniFi keystore...\n"
keytool -importkeystore \
-srckeystore ${P12_TEMP} -srcstoretype PKCS12 \
-srcstorepass ${PASSWORD} \
-destkeystore ${KEYSTORE} \
-deststorepass ${PASSWORD} \
-destkeypass ${PASSWORD} \
-alias ${ALIAS} -trustcacerts

# Import the certificate authority data into the UniFi keystore
printf "\nImporting certificate authority into UniFi keystore...\n\n"
if [ ${LE_MODE} == "true" ]; then
        # Import with additional cross-signed CA file
        java -jar ${JAVA_DIR}/lib/ace.jar import_cert \
        ${SIGNED_CRT} \
        ${CHAIN_FILE} \
        ${CA_TEMP}
else
        # Import in standard mode
        java -jar ${JAVA_DIR}/lib/ace.jar import_cert \
        ${SIGNED_CRT} \
        ${CHAIN_FILE}
fi      

# Clean up temp files
printf "\nRemoving temporary files...\n"
rm -f ${P12_TEMP}
rm -f ${CA_TEMP}
        
# Restart the UniFi Controller to pick up the updated keystore
printf "\nRestarting UniFi Controller to apply new Let's Encrypt SSL certificate...\n"
# {ZA} update to a systemd style command
#service ${UNIFI_SERVICE} start
systemctl start ${UNIFI_SERVICE}

# That's all, folks!
printf "\nDone!\n"

exit 0

Make the file executable and then run it to ensure it actually replaces the certificate in the UniFi controller - note that this script may take some time to complete:

sudo chmod 755 /usr/local/bin/unifi-ssl-import.sh
sudo /usr/local/bin/unifi-ssl-import.sh

Once the import process is working properly, automate it with a daily cron job.

Create /etc/cron.daily/unifi-ssl-import:

#!/bin/sh
/usr/local/bin/unifi-ssl-import.sh

Finally, make the cron script executable:

sudo chmod 755 /etc/cron.daily/unifi-ssl-import

Configuration notes for a Ubiquiti UniFi setup

Ubiquiti make some shiny WiFi gear that falls in to the “Enterprise Lite” space – not quite as robust as Cisco Meraki or Aerohive gear, but also nowhere near as expensive. Here are some notes I took while setting up my home WiFi with two UniFi UAP-AC-PROs and the UniFi Controller running in the cloud:

UniFi Controller

Get the Controller up and running before anything else. The easiest way to do this would probably have been to get a UniFi Cloud Key to run the Controller locally, then manage it using a Ubiquiti account – but seeing as I was too cheap to buy a Cloud Key, and I already have a server that I’m using for other things – it made sense for me to run the Controller on that server. I mostly followed the instructions for a Debian based UniFi Controller, with the following gotchas:

  • The firewall instructions open more ports than are absolutely necessary, and
  • The firewall instructions assume you’re crazy and want to configure iptables by hand, rather than using something like ufw

Another thing to keep in mind: the ports that the UniFi Controller use will cause problems if you’re also running Tomcat on the same server (or plan to do so in the future).

I decided to change the ports for the UniFi Controller – in the following configuration file, add the following two lines (don’t uncomment the lines towards the top of the file – as I understand it, you can’t have whitespace or comments after configuration parameters):

/var/lib/unifi/system.properties

unifi.http.port=9080
unifi.https.port=9443

Restart the Controller:

sudo systemctl restart unifi

Firewall

As I’m running the UniFi Controller out on them Internets and have no intention of accessing the Controller via the web on port 9443, the collection of ports I need to open are:

  • TCP/9080 – the port that the APs use to communicate with the Controller
  • TCP/8880 – used by the guest portal (could be dropped?)
  • TCP/8843 – also used by the guest portal (could also be dropped?)

I’m not including the following ports, for the following reasons:

  • UDP/3478 – used for STUN by the UniFi VoIP software
  • TCP/8081 – I can’t see a good reason to leave a “shutdown” port open
  • TCP/9443 – I’m running a reverse proxy for the UI (see below), so it’s not needed
  • UDP/100001 – only needed when the Controller is in the same subnet as the APs

Because I don’t hate myself, I didn’t attempt to configure iptables by hand. Here’s my ufw application file:

/etc/ufw/applications.d/unifi

[UniFi]
title=UniFi Controller
description=Ubiquiti UniFi Controller
ports=8843,8880,9080/tcp

Once the above file has been created, enabling the firewall rules couldn’t be simpler (you still need to ensure ufw is otherwise correctly configured first):

sudo ufw allow unifi

Apache reverse proxy

The UniFi Controller comes secured with a self-signed SSL certificate, and while it’s possible to change the Controller UI port to 443 and install a regular SSL certificate – it’s also a complete pain in the ass (and/or impossible if you’re already running a webserver). Instead, I set up a reverse proxy in Apache to handle the redirect and make it much, much easier to manage the SSL stuff. Here’s my Apache config file for the relevant virtualhost:

/etc/apache2/sites-available/unifi.controller.fqdn.conf

<IfModule mod_ssl.c>
<VirtualHost *:443>
ServerName unifi.controller.fqdn
ServerAdmin hostmaster@example.com
 
CustomLog /path/to/access.log combined
ErrorLog /path/to/error.log
 
SSLProxyEngine On
SSLProxyCheckPeerCN off
SSLProxyCheckPeerName off
ProxyPreserveHost On
ProxyPass / https://unifi.controller.fqdn:9443/
 
SSLCertificateFile /path/to/fullchain.pem
SSLCertificateKeyFile /path/to/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>
</IfModule>

Couple notes:

  • No need for a DocumentRoot – I’ve set this site up purely as a redirect, but you may want to do things differently
  • All of the SSLProxy.. lines are required – this is to avoid problems with the self-signed certificate on the Controller
  • The Include /etc/letsencrypt.. line – I’m using a Let’s Encrypt certificate, so this line is standard (your SSL configuration lines will probably be different if you’re using a different CA)

Once the above is up and running, you should be able to access the UniFi Controller at https://unifi.controller.fqdn/. Run through the setup wizard, but I’d recommend setting up another site (separate to “Default”) for the first site if you plan to have several sites managed by the Controller.

Setting up the APs

Try though I might, I couldn’t get the various versions of the UniFi Discovery Tool to work at all. I gave up after an hour – save yourself the grief and configure the inform URL by SSH’ing in to each AP after it picks up a DHCP lease (default user/pass is ubnt/ubnt) with the following command:

set-inform http://unifi.controller.fqdn:9080/inform

Wait a few seconds, then log in to the Controller and complete the “adoption” process. After your new AP is adopted and assigned to a site, go back to the SSH console and re-run the same command:

set-inform http://unifi.controller.fqdn:9080/inform

This saves the inform URL in case the AP reboots, etc.

Other Config Notes

Beyond the standard WiFi configuration, I also have the following items set for each AP:

  • Minimum RSSI set to -75 dBm (-80 dBm is a good starting point for experimentation)
  • Band Steering set to Prefer 5G
  • Airtime Fairness set to On

And the following set in the site configuration:

  • Enable status LED unchecked

If you ever need to ssh back in to the APs, the default username and password will not work. The new username and password are shown in the Controller – go to Settings and select the site the APs are connected to – the username and password are listed at the bottom of the page.