Multi-Site Homelab Mesh Networking: Building a Zero-Trust Network Overlay with Tailscale & Subnet Routing

Executive Summary

Connecting servers, storage nodes, and management interfaces scattered across different physical sites and ISP connections presents a classic networking challenge: handling dynamic IP addresses, Carrier-Grade NAT (CGNAT), firewall traversal, and secure remote access without exposing raw ports to the public internet.

In our multi-site architecture, we resolved this by implementing Tailscale—a zero-trust overlay network built on top of the WireGuard protocol. This post outlines how we established end-to-end encrypted mesh connectivity between local servers, remote storage nodes, out-of-band management interfaces, and mobile devices across multiple physical sites (Mircea Eliade Site and Sopor Site).

1. Network Topology & Structural Overview

Our infrastructure spans multiple physical locations, each with unique subnet ranges and internet gateways:

Server & Device Inventory:

• nas1: blog.voina.org (Dynamic DDNS IP, Mircea Eliade Site, SSH port 2222). Runs WordPress, daily backups, and tape archival.
• nas2: 192.168.2.22 (Internal IP / Tailscale IP 100.104.211.90), accessible via nas1 jump gateway or direct Tailscale node address.
• nas3: 192.168.1.50 (Sopor Site local LAN). Primary docker host running monitoring stacks and local services.
• Admin Laptop: Local management workstation requiring direct access to remote subnets.

2. Key Challenges & Architectural Requirements

1. Overcoming Dynamic IPs & CGNAT: ISP fiber connections on nas1 and nas2 use dynamic public IP addresses. Traditional IPSec or OpenVPN site-to-site tunnels required persistent maintenance of DDNS records and complex firewall port forwarding.
2. Access to Out-of-Band Management (HPE iLO): Out-of-band management interfaces (nas1-ilo, nas2-ilo, nas3-ilo) do not support native VPN software installation.
3. Seamless Site-to-Site Subnet Access: Engineers on the road or at one site needed transparent IP routing into remote subnets (e.g., reaching 192.168.6.0/24 or 192.168.25.0/24 directly).

3. Implementation Step-by-Step

Step 3.1: Tailscale Node Provisioning

Tailscale was installed across all primary Linux hosts (nas1, nas2, nas3, and the administrative laptop) using the official repository packages:

# Install Tailscale on Fedora/RHEL nodes
curl -fsSL https://tailscale.com/install.sh | sh

# Authenticate node to tailnet
sudo tailscale up --accept-routes

Step 3.2: Subnet Router Configuration

To expose non-Tailscale devices (such as HPE iLO management processors and local router gateways) to the mesh, subnet routing was enabled on primary Linux hosts.

Enabling IPv4 Packet Forwarding:

echo 'net.ipv4.ip_forward = 1' | sudo tee -a /etc/sysctl.d/99-tailscale.conf
echo 'net.ipv6.conf.all.forwarding = 1' | sudo tee -a /etc/sysctl.d/99-tailscale.conf
sudo sysctl -p /etc/sysctl.d/99-tailscale.conf

Advertising Subnets:

• On nas1 (Mircea Eliade Site): Exposing out-of-band management subnets 192.168.6.0/24 and 192.168.5.0/24:sudo tailscale up --advertise-routes=192.168.6.0/24,192.168.5.0/24 --accept-routes
• On nas3 (Sopor Site): Exposing local network subnets 192.168.1.0/24 and 192.168.25.0/24:sudo tailscale up --advertise-routes=192.168.1.0/24,192.168.25.0/24 --accept-routes

Enabling Route Acceptance on Admin Laptop:

sudo tailscale up --accept-routes

4. MagicDNS & Split-Horizon Name Resolution

With MagicDNS active on the Tailscale admin console, every device in the infrastructure automatically receives a stable FQDN under the *.ts.net namespace.

• Single-Point Addressing: Devices can be accessed by hostname regardless of whether their physical public IP changes.
• Direct SSH & Service Bindings: Internal services (such as SSH, Docker registries, and monitoring dashboards) bind safely to the 100.x.y.z CGNAT adapter interface without public exposure.

5. Security & Verification

1. Traffic Encryption: All traffic between nodes is end-to-end encrypted using standard WireGuard noise protocol primitives (ChaCha20-Poly1305 and Curve25519).
2. Access Verification:
   • nas1 to nas3 direct tailscale ping latency: ~2-5 ms.
   • Laptop reaching nas1-ilo (https://192.168.6.2) across sites: Successful.
   • Laptop reaching nas3-ilo (https://192.168.25.174) across sites: Successful.

6. Lessons Learned & Best Practices

[!TIP]
Persistent Route Acceptance: When managing Linux hosts that serve as both subnet routers and clients, always pass --accept-routes alongside --advertise-routes to ensure bidirectional routing table insertion.

[!NOTE]
Firewall Rules (iptables / firewalld): Ensure firewalld or iptables allows masquerading and forwarding on tailscale0 interfaces:

sudo firewall-cmd --zone=trusted --add-interface=tailscale0 --permanent
sudo firewall-cmd --reload

Conclusion

By layering Tailscale over our multi-site homelab topology, we transformed disparate network segments into a single unified zero-trust mesh network. Out-of-band management interfaces, storage backups, and remote monitoring stacks now communicate seamlessly without exposing a single open port to the public internet.