I’m curious about firewalls. I have a Protectli FW4B along with a backup edge router ER-X and decided to do some testing. There are a handful of different ideas I wanted to explore.
The first had to do with the value of the next-generation firewall capabilities like threat protection. Practically, it doesn’t matter that much for me, it’s just my wife and I, but I was curious if it would be a valuable thing to deploy for my mom and how difficult they would be to be useful.
I’m also interested in separating my crappy Internet of things devices from various no-name manufacturers off onto their own network for privacy and security reasons. I already have some of the network hardware, but I need a router that supports VLANs (they all do) and mDNS repeater software so devices on my secure lan (like my homepod and my phone) can find them.
I currently have 100 Mb symmetrical service at home. Due to the nature of my work, I’m often uploading large files to and from our corporate artifactory. Because I often saturate the network connection, I need to use something like fq_codel help keep my video and audio conference quality high even while moving around large files.
I also wanted to choose a system that could potentially scale to gigabit fiber. We are working on a project at work that might make it a lot more valuable for me to have that kind of connection when remote. That said, it’s not even clear my work VPN concentrator can handle that throughput.
Finally, I use CenturyLink which does not have native dual stack IPV6. I’m curious about using 6RD to enable that, but all my experiments with pfSense ended badly (very degraded download performance). Not sure if that’s a me thing or a them thing 😊.
Here’s the big table of all these capabilities:
|Firewall||fq_codel||6rd||Easy Threat Intelligence||mDNS repeater||Base OS|
|OPNSense||yes||yes||sort of (Sensei subscription)||yes||HardenedBSD|
BSD vs Linux
I have an underpowered device for gigabit on BSD. The Protectli FW4B has a Celeron J3160 processor. While this does have AES-NI, it doesn’t support PCI Passthrough when running in virtualization. I learned very quickly that trying to run pfSense under virtualization couldn’t support 100Mb/s uploads.
Due to limitations in the architecture of BSD, PPPoE is single threaded (https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203856). This limits the maximum upload and download speeds to the speed of the processor. A low powered Celeron or ARM core will not do gigabit.
While there are some techniques you can do such as deferring interrupt handling to other threads, those also cause extra work and don’t always help as much as you want. This author has a great write up of both performance analysis and the tuning you can do to improve the BSD based routers: https://teklager.se/en/knowledge-base/apu2-1-gigabit-throughput-pfsense/. I was hoping the new SG-2100 might be possible, but it also doesn’t have enough single core performance as it has the same processor as the SG-3100 but more RAM (https://www.reddit.com/r/PFSENSE/comments/7g8n5a/gigabit_pppoe_on_sg3100/)
This matters to me because I do a lot of uploading of large single files with a single connection over PPPoE.
The Linux based routers do not have this problem. They introduce multithreading at a lower layer, and therefore all cores can participate in PPPoE handling. It’s possible you’ll still saturate the system. If it takes almost 2 cores just to deal with PPPoE that’s might not be a lot of headroom depending what else you are running.
That leaves me with the choice of either upgrading my hardware to something based on the i3-6100U or better (single thread CPU mark of 1361 vs 624 with the Celeron I have) or using Linux if I want to route gigabit. The i3 also has the benefit of support PCI passthrough, though that’s less valuable because of the PPPoE single core restrictions.
(perf on i3: https://forum.netgate.com/topic/138926/realtek-nic-pppoe-gig-cpu-usage-seems-odd)
Support for Internet of Crap (Things) VLAN
I’m slowly adopting more and more cheap Internet devices for controlling things around my house. Some of them I write the software for myself, others are provided by the vendor and probably have both security and privacy issues associated with them. Instead of identifying the devices one by one and putting firewall rules on them, I am going to create a separate wireless network for all of them and lock it down.
The crux is that once you’ve locked them down on their own network, your primary network with your iPhone can’t talk to them in less you use a mDNS (aka Bonjour) repeater between the networks. There is a free one named Avahi used in standalone form or bundled with the router.
Untangle and the Sophos products do not support this in their offering. You’ll be stuck setting one up manually either on the firewall itself (gulp) or on a separate device.
This idea can also be done on a raspberry pi on ethernet: http://chrisreinking.com/need-bonjour-across-vlans-set-up-an-avahi-gateway/
By far the easiest set up and the best dashboards out of the box. There is a free version; $50 annual subscription; and $150 annual subscription. The free version does simple routing. Configuring fq_codel was dead simple. To do IPSEC VPN you need the $50 subscription, Wireguard is $150. OpenVPN is free but I worry about performance.
If they supported mDNS repeating I would be using this.
These tools let you do anything and have a tremendous amount of third-party packages available for them. Configuring them is a pain (especially tools like fq_codel) as all that flexibility comes at a cost. They are also the most popular solutions out there in terms of finding free support and reference documentation.
But using them is a challenge in my use case. Due to the performance issues around PPPoE and my cheap hardware I don’t have a
cheap clean upgrade path if I increase my network connection speed.
IPFire makes efficient use of the underlying hardware. Its initial set up is pretty awkward as it forces you to switch between a text only and keyboard only mode for the basic configuration before allowing you to use a web browser. Certain changes aren’t available in the web UI (i.e. changing you from 2 to 3 networks) and force you to drop into the command line and rerun the text only tool. Graphing is not live.
The primary author has some very strong opinions about what capabilities should and should not be in a firewall. His arguments are well reasoned and forced me to reevaluate the some of my preconceived notions (i.e. using a VPN vs an http proxy). But at the same time his viewpoint hasn’t been adopted by the community at large. That makes implementing privacy enhancing controls at a network level more difficult (for example most commercial proxy providers don’t target privacy enhancement). See this discussion for more details: https://community.ipfire.org/t/protect-users-who-get-spammy-phishing-links-in-emails/4111/8
I’m just going to clump UTM and XG together. UTM is their older product that they are sunsetting but supports fq_codel but NOT 6rd. XG is the newer product line that does not support fq_codel but does support 6rd. I love the fact that they offer both of these free for home users. UTM only supports 50 devices and it wasn’t clear to me that it has a bypass mode like Untangle does.
I really tried to make XG work for me as that’s their investment area (https://ideas.sophos.com/forums/17359-sg-utm/suggestions/7131408-allow-more-ip-s-on-home-free-utm) but no fq_codel was a deal breaker in the end.
I’m giving IPFire a shot. I plan on reusing Blue (Wireless) as my IoT network. The default configuration has it separated with its own DHCP server so its 90% of the way there for my needs.
But I’m not thrilled with any of my choices. I like to tinker. Untangle and IPFire are not great for tinkering but work well on my hardware. Sophos XG is difficult and allows tinkering, but no fq_codel (therefore if I upgrade to gigabit could be a contender). Ideally I would have a pfSense that worked well on my hardware. The VyOS commandline would make me cranky.
Hmm. I wonder if Untangle in Proxmox would work well enough that I could run a separate VM for Avahi…