Chapter 2

Routing

The complicated part. Let’s get started and break down how we’re segmenting this.

Subsections of Routing

Overview

Chapter 3

Networks

We’re going to be covering about a dozen different segments here.

Be aware that the VLANs and subnets I display here are purely an example, feel free to use whatever RFC 1918 addresses you want.

Network VLAN Default Subnets
Generic Users 1100 10.10.0.0/24
Guest Users 1200 10.20.0.0/24
Management 1990 10.99.0.0/24
Internal Services 1300 10.30.0.0/25
DNS 1900 10.90.0.0/24
WAN Exposure 1800 10.80.0.0/24
IoT 1700 10.70.0.0/16
Security Systems 1750 10.75.0.0/16
Quarantine 1690 10.69.0.0/24
Printer 1420 10.42.0.0/24
Internal/External 7001 172.21.1.0/29
Router Loopbacks N/A 10.100.0.0/16
Dead End 666 N/A

Subsections of Networks

Users

VLAN Default Subnet(s)
1100 10.10.0.0/24

This segment is for stuff like my desktop, laptop dock, phone, trusted IoT devices, etc.

Guest Users

VLAN Default Subnet(s)
1200 10.20.0.0/24

Ideally any endpoint devices that I don’t own go here. This should be the default home for newly connected WiFi devices. This VLAN will have access to the DNS, the media server front ends, and any local gameservers. They will not have the ability to access any management interfaces, the management segment, or the SSH of any local device.

Management

VLAN Default Subnet(s)
1990 10.99.0.0/24

Here’s where the NAS, the ESXi, routers and switches all exist, at least for management purposes. This is primarily meant to be a single network segment because it allows us to designate a port on the switch specifically for recovery purposes. By having access to virtually everything via the same VLAN, I can just plug a computer straight into the port tagged for this network and repair whatever outage I’ve caused.

Although this network is not originally intended to be a big throughput VLAN, I am using this segment for all direct connections between the baremetal ESXi and NAS. The NAS exists natively here, but do be aware that I am putting all media devices on a different segment. This means that the ESXi host itself can get to the NAS on the same L2 domain, but all services still need to pass through the firewall. Depending on how you want to mount the NAS into VMs/containers/services, this allows you to dictate a firewall layer on demand.

Internal Services

VLAN Default Subnet(s)
1300 10.30.0.0/16

Big data happens here. This segment is the primary motivator for designating 3 LAGs on the network. Many VMs are going to be native on this VLAN, virtually all the media related VMs/containers. This is the default landing zone for new services I spin up (that don’t require quarantining anyways).

DNS

VLAN Default Subnet(s)
1900 10.90.0.0/16

All of our DNS servers exist here, including the PiHole. Consider the following IP assignment:

Server IP/CIDR Gateway
NS1 10.90.10.10/28 10.90.10.14
NS2 10.90.20.20/28 10.90.20.30
PiHole 10.90.10.5/28 10.90.10.14

Personally, I like NS1’s IP ending in 10.10, and NS2’s IP ending in 20.20. “But wait!” you might say, “That’s two gateways on the same VLAN!” Yes, and we’re going to handle that with virtual IPs. Not to get terribly ahead of ourselves, but in OPNsense, Interfaces > Virtual IPs > Settings > Add contains the solution to this. We’ll go into greater detail in the router setup section.

DMZ/WAN Exposure

VLAN Default Subnet(s)
1800 10.80.0.0/24

This segment is really for two types of devices: the reverse proxy that is exposed to the WAN, and any game servers/VMs. It should be fairly obvious why these are segmented and they will be somewhat restricted in what they can communicate with on the internal network. A small reminder: because our Edge router is handling the NAT for us, Destination NAT (DNAT, aka Port Forwarding) for these devices will be handled at the Edge.

IoT

VLAN Default Subnet(s)
1700 10.70.0.0/16

Okay this is the first potentially complicated segment, because depending on your needs, you might actually want it to be three different segments with varying levels of controls and severity:

  • Highest severity - This is going to be all the devices that you don’t understand why they have WiFi/internet access but maybe still want to let them do things. This is your refridgerator, washing machine, coffee maker, etc. This segment would be allowed access to the internet and that’s it. They wouldn’t even get access to the internal DNS. They can’t interact with other endpoints locally, they can only reach out and call home.
  • Medium severity - These are devices that desire moderate communication with other devices on the local network. I don’t have any examples of these, but maybe your thermostat needs this level.
  • Low severity - This is where your devices that are fairly trusted but still IoT go. This could be things you build yourself with esp8266 boards, a Hue bridge, an OctoPi server for your 3D printer. They have no outbound SSH ability, but can access the internet and local clients can reach them.

Security Systems

VLAN Default Subnet(s)
1750 10.75.0.0/16

In my network, I do not want security systems to be able to access the WAN, or be accessed from external. They will need access to various appliances, such as the NAS and frontend servers. The goal will be to keep them isolated from all else. If your cameras require access to the WAN, treat them very similar to the Mid or High severity IoT devices networks.

Quarantine

VLAN Default Subnet(s)
1690 10.69.0.0/24

The step-child of networks. This is primarily for projects where I’m standing up VMs that I do not want to touch anything else, but desire some level of network connectivity for reasons. This firewall profile will be the most mutable in my network, as rules will change based on my immediate project goals. Sometimes, this segment might only be allowed to talk to itself, sometimes it might have full network access, or anywhere in between.

Printer

VLAN Default Subnet(s)
1420 10.42.0.0/24

I hate printers. They get their own network where I can lock them down if I need to. Currently this is unused in my network as the printer I have doesn’t do anything I hate. It gets to sit on the WiFi like a real boy.

Routers

VLAN Default Subnet(s)
7001 172.21.1.0/29
N/A 10.100.0.0/16

Wait two segments? And one doesn’t have a VLAN?

Lets talk about the first subnet first, 172.21.1.0/29. Why is it numbered so differently than the other networks? Because it is the network segment that the two routers are going to form an adjacency in. No device other than these two routers needs to talk to these IPs. For monitoring purposes, NEMS is also allowed to ping these. For example, in our scheme, consider the following:

Router Interface Address
Edge eth1 172.21.1.1/29
Internal igc0/eth0 172.21.1.2/29
Edge Loopback lo0 10.100.1.0/32
Internal Loopback lo1 10.100.2.0/32

The first two entries are the interfaces on each router that will be active in the OSPF process. Don’t sweat this too much, just be aware that they send OSPF hellos to eachother from those interfaces, and listen for Link State Advertisements (LSAs).

The second two entries are the loopback interfaces for each router. This is going to be used as the Router ID for the OSPF process. Note that they are /32 addresses because they are purely routed interfaces, they do not have a Layer 2 component. Don’t sweat this either. You don’t have to understand OSPF to follow the instructions later.

Dead End

VLAN Default Subnet(s)
666 N/A

This isn’t a real network, but simply the name we give to our administratively disconnected VLAN. More on this in the Switching section. There is no gateway for this segment.

Chapter 4

Firewall

Lorem Ipsum.

OSPF