For years now, I've been running my home network from my trusty Buffalo WHR-HP-G54 router running the Tomato firmware.
Tomato is, sadly, no longer under development, but still beats many modern router GUIs; it has features like live bandwidth graphs, dynamically updating device lists with signal strength meters, and more.
I've also been running a small home server for years, mostly as a file storage and backup-taking device, which ended up being a small 2010-era Acer Aspire Revo - a great little computer, but underpowered and with no extra internal hard drive space, so it sat on top of two external USB drives.
All of these had their own power adapters, and so the corner of my living room that housed all this equipment was a rats' nest of cables, not to mention the fact that I didn't have a wireless N network and that the Revo was underpowered enough that backups took ages to run, and trying to access its NFS or Samba shares incurred a couple of seconds' delay.
A plan forms
Now, I like nothing better than spending a day or two fiddling around with low-level Linux hacking, and so when a spare Shuttle case came into my possession I hatched a plan to replace everything with a single box - covering routing, wireless networking, file serving, backup-taking and other purposes.
Of course, the main requirements for a router are a machine with at least two network ports (one for the internal network, and one for the external network - I have a separate ADSL modem that plugs into). For access point duties, it needs at least one wireless card that supports Master mode - in my case, I have two Atheros wireless cards; one to run the 5GHz network, and one to run the 2.4Ghz network.
The Shuttle only has two PCI Express expansion slots, but it fortunately also has a mini-PCI Express slot on the motherboard; this meant I could add in a single network card to provide a second Ethernet port (the motherboard already has one), a PCI Express 5GHz-capable wireless card with three external aerials, and cheap 2.4GHz mini-PCI Express card with a large aerial that lives just inside the case.
In addition, the Shuttle has space for three 3.5" drives inside the case - I've put two hard drives and an SSD in there (there's actually space for a second SSD, too, as you can mount two in a single 3.5" slot).
The back of the router - you can see the two Ethernet ports and the wireless card here.
I installed Ubuntu 13.10 on the server - I would have used Debian testing, but I tend to find that Ubuntu still has slightly better availability of software in repositories (especially the prevalence of PPAs for newer software). That said, I suspect I may later re-image it as Debian testing and use Docker to run Ubuntu containers for any newer software in future.
To run the wireless networks, I'm using two instances of hostapd, a wonderful daemon that you can just give a single configuration file to and it'll run a network any way you like. It'll add itself to ethernet bridges, it'll let you run multiple SSIDs off of one card (as long as they're the same frequency), it'll let you do whatever authentication you like... and more. If you're at all interested in wireless stuff, it's worth a look.
Topology-wise, I bridged together the two internal network ports (eth1 and eth2) and the wireless networks into a single bridge using Linux bridge utils, and then exposed this to the internet (eth0) using the NAT functionality of iptables.
I'm also using iptables to firewall off the machine's ports from the internet - only SSH is exposed, and even that's on a non-standard port.
Finally, I'm using dnsmasq to provide DHCP and internal DNS services; it's another great, lightweight daemon, and it's easy to configure things like static DHCP leases with just a few configuration files.
User Interface
Of course, none of this comes with a nice user interface, and it's really helpful to know what's going on. In particular, I want to know bandwidth usage and see what devices are on the local network; I don't have a lot of downstream bandwidth, so it can be important to see what's using it.
So, I sat down and spent a few hours knocking together a router dashboard using Django. It just queries the ARP table, the wireless drivers and a small packet inspector daemon I wrote to see what local devices there are on the network and what their bandwidth usage is, and then displays it on a single page that auto-updates - it's currently displayed on an iPad that sits in front of the box.
A screenshot of the dashboard
The interface is entirely read-only at the moment, but I'm considering adding things like management of static DHCP leases and port forwarding to make it a bit nicer to manage those.
The Future
Apart from adding a few extra features to the interface, I'd also like to investigate routing all the traffic out via a VPN to an external server - especially while I'm still in the UK, this will get around the infamous BitTorrent site censorship and the impending introduction of the "porn filter" (which, like any filter, blocks a lot of legitimate sites too). It's not the encryption I'm looking for - I'll actually choose a relatively weak one - but the ability to control my own internet connection fully.
I'd rather this didn't turn into a large project - I have no wish to end up maintaining a router distribution (well, unless it pays well, but my time for open source is already stretched thin!). Still, I've open-sourced the code for the UI in a repository on GitHub, if you want to take a look.