Private Site-to-Site Mesh with WireGuard and Self-Hosted Headscale

Managing access across multiple VPS providers, home servers, and dev machines gets messy fast. Opening SSH ports to the public internet invites constant brute-force attacks. Static WireGuard tunnels solve the encryption part, but manually updating public keys, endpoint IPs, and routing tables on every single peer whenever a node changes is painful.

Tailscale fixes the operational burden by coordinating WireGuard keys and NAT traversal over a central control plane. The catch is that Tailscale's official control plane is closed-source and hosted on their infrastructure.

Headscale is an open-source, self-hosted implementation of the Tailscale control server. You keep all coordination data, machine keys, and network topology on your own server. Your traffic remains end-to-end encrypted peer-to-peer over WireGuard.

Here is how to set up Headscale on a public Linux server, connect clients across different environments, and configure subnet routing for site-to-site connectivity.

Architecture Overview

A Tailscale network (tailnet) consists of two layers:

1. Control Plane (Headscale): Distributes node public keys, assigns internal IP addresses (100.64.0.0/10), manages ACLs, and coordinates NAT traversal (DERP relays). It never sees or decrypts your actual network payload.
2. Data Plane (WireGuard): Direct, peer-to-peer WireGuard tunnels between your machines. If two nodes cannot establish a direct UDP connection due to strict NATs, encrypted traffic bounces through a DERP relay.

1. Prerequisites

To run this setup, you need:

• One Linux VPS (Ubuntu 22.04/24.04 or Debian 12) with a public IPv4 address and at least 1 GB of RAM.
• A domain or subdomain pointing to your VPS IP (for example, vpn.example.com).
• Ports open on the VPS:
  • 80/tcp and 443/tcp (HTTP/HTTPS for API and node coordination).
  • 3478/udp (STUN for NAT discovery if running an embedded DERP server).

2. Installing Headscale

Log into your VPS. Download the latest release binary from GitHub.

# Check current latest version on GitHub releases
HEADSCALE_VERSION="0.29.3"

# Download binary for amd64 architecture
curl -Lo /usr/local/bin/headscale \
  "https://github.com/juanfont/headscale/releases/download/v${HEADSCALE_VERSION}/headscale_${HEADSCALE_VERSION}_linux_amd64"

# Grant execution rights
chmod +x /usr/local/bin/headscale

Create a system user, group, and required directories:

# Create system user
useradd --system --shell /usr/sbin/nologin --user-group headscale

# Create configuration and data directories
mkdir -p /etc/headscale
mkdir -p /var/lib/headscale
mkdir -p /var/run/headscale

# Set ownership
chown -R headscale:headscale /var/lib/headscale /var/run/headscale /etc/headscale

3. Configuring Headscale

Create /etc/headscale/config.yaml. This file defines your server URL, database path, IP allocation ranges, and embedded DERP configuration.

# /etc/headscale/config.yaml
server_url: https://vpn.example.com
listen_addr: 127.0.0.1:8080
metrics_listen_addr: 127.0.0.1:9090
grpc_listen_addr: 127.0.0.1:50443

noise:
  private_key_path: /var/lib/headscale/noise_private.key

prefixes:
  v4: 100.64.0.0/10
  v6: fd7a:115c:a1e0::/48
  allocation: sequential

derp:
  server:
    enabled: true
    region_id: 999
    region_code: "custom-vps"
    region_name: "Self-Hosted DERP"
    stun_listen_addr: "0.0.0.0:3478"
    private_key_path: /var/lib/headscale/derp_server_private.key
  urls:
    - https://controlplane.tailscale.com/derpmap/default
  auto_update_enabled: true
  update_frequency: 24h

database:
  type: sqlite
  sqlite:
    path: /var/lib/headscale/db.sqlite

dns:
  magic_dns: true
  base_domain: mesh.internal
  nameservers:
    split: {}
    global:
      - 1.1.1.1
      - 1.0.0.1
  extra_records: []

log:
  level: info
  format: text

tls_letsencrypt_hostname: ""
tls_cert_path: ""
tls_key_path: ""

Set file permissions so only the headscale user can read the config:

chmod 640 /etc/headscale/config.yaml
chown headscale:headscale /etc/headscale/config.yaml

4. Setting Up Systemd Service

Create a systemd unit file at /etc/systemd/system/headscale.service:

[Unit]
Description=headscale control server
After=network.target

[Service]
Type=simple
User=headscale
Group=headscale
ExecStart=/usr/local/bin/headscale serve
Restart=always
RestartSec=5

# Sandboxing
ProtectSystem=strict
ProtectHome=yes
ReadWritePaths=/var/lib/headscale /var/run/headscale /etc/headscale
PrivateTmp=yes
CapabilityBoundingSet=CAP_NET_BIND_SERVICE
AmbientCapabilities=CAP_NET_BIND_SERVICE
NoNewPrivileges=yes

[Install]
WantedBy=multi-user.target

Reload systemd, enable the service, and start it:

systemctl daemon-reload
systemctl enable --now headscale
systemctl status headscale

5. Reverse Proxy with Nginx & Let's Encrypt

Headscale listens locally on 127.0.0.1:8080. Put Nginx in front of it to handle TLS termination and WebSocket upgrades.

Install Nginx and Certbot:

apt update && apt install -y nginx certbot python3-certbot-nginx

Obtain an SSL certificate:

certbot certonly --nginx -d vpn.example.com

Create /etc/nginx/sites-available/headscale:

server {
    listen 80;
    listen [::]:80;
    server_name vpn.example.com;
    return 301 https://$host$request_uri;
}

server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    server_name vpn.example.com;

    ssl_certificate /etc/letsencrypt/live/vpn.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/vpn.example.com/privkey.pem;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;

    location / {
        proxy_pass http://127.0.0.1:8080;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
        proxy_set_header Host $server_name;
        proxy_redirect http:// https://;
        proxy_buffering off;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;
}

Enable the configuration and reload Nginx:

ln -s /etc/nginx/sites-available/headscale /etc/nginx/sites-enabled/
nginx -t && systemctl reload nginx

6. Creating Users and Pre-Authentication Keys

Headscale groups nodes under users (previously called namespaces).

Create your first user:

headscale users create infra

To connect headless servers or automate container setups without opening browser verification links, generate a reusable pre-auth key:

# Generate a pre-auth key valid for 90 days, reusable across nodes
headscale preauthkeys create --user infra --reusable --expiration 90d

Save the output string (for example, abc123456789...).

7. Connecting Client Nodes

Install the standard official Tailscale client on your target machines.

On Linux (Debian/Ubuntu)

curl -fsSL https://tailscale.com/install.sh | sh

Connect the node to your self-hosted instance:

tailscale up \
  --login-server https://vpn.example.com \
  --authkey <YOUR_PREAUTH_KEY> \
  --hostname cloud-vps-01

If you do not use a pre-auth key, run tailscale up --login-server https://vpn.example.com. The command outputs a registration URL with a machine key:

To authenticate, visit:
https://vpn.example.com/register/mkey:0123456789abcdef...

On the Headscale server, register the machine manually:

headscale nodes register --user infra --key mkey:0123456789abcdef...

On macOS / Windows / iOS / Android

• Open Tailscale client settings.
• Under custom login server (or hold Alt/Option on macOS menu bar -> Custom Login Server), enter https://vpn.example.com.
• Complete the authentication handshake in your browser.

8. Site-to-Site Subnet Routing

Suppose you have a home network (192.168.1.0/24) containing NAS drives, Raspberry Pis, and local services that cannot run Tailscale directly. You can designate one Linux machine inside that LAN as a subnet router.

Step 1: Enable IP Forwarding on the Gateway

On the local machine inside your LAN:

# Enable IPv4 and IPv6 forwarding
echo 'net.ipv4.ip_forward = 1' | sudo tee -a /etc/sysctl.d/99-tailscale.conf
echo 'net.ipv6.conf.all.forwarding = 1' | sudo tee -a /etc/sysctl.d/99-tailscale.conf
sudo sysctl -p /etc/sysctl.d/99-tailscale.conf

Step 2: Advertise Subnet Route

Start Tailscale on the gateway machine and declare the local subnet:

tailscale up \
  --login-server https://vpn.example.com \
  --authkey <YOUR_PREAUTH_KEY> \
  --advertise-routes=192.168.1.0/24 \
  --accept-routes

Step 3: Approve Route in Headscale

By default, Headscale does not route traffic through advertised subnets until an administrator approves them.

On your Headscale server, list available routes:

headscale routes list

You will see an entry matching 192.168.1.0/24 with its Route ID. Enable it:

# Replace 1 with your route ID
headscale routes enable -r 1

Step 4: Access LAN Devices from Remote Peers

On any remote client (such as your laptop traveling on public Wi-Fi):

tailscale up --login-server https://vpn.example.com --accept-routes

Ping or SSH directly to any local IP on your home network:

ping 192.168.1.50
ssh [email protected]

Traffic flows over encrypted WireGuard from your laptop to the home gateway, which forwards packets to the destination device on the LAN.

9. Configuring an Exit Node

If you want all internet traffic from your laptop or phone routed through a specific VPS (acting like a traditional commercial VPN when using untrusted networks):

Step 1: Advertise Exit Node Routes on VPS

On the node acting as the exit proxy:

# Ensure IP forwarding is enabled
echo 'net.ipv4.ip_forward = 1' | sudo tee -a /etc/sysctl.d/99-tailscale.conf
sudo sysctl -p /etc/sysctl.d/99-tailscale.conf

# Advertise default routes
tailscale up \
  --login-server https://vpn.example.com \
  --advertise-routes=0.0.0.0/0,::/0

Step 2: Approve Default Routes on Headscale

headscale routes list
# Enable the 0.0.0.0/0 and ::/0 route IDs
headscale routes enable -r 2
headscale routes enable -r 3

Step 3: Use Exit Node on Client

tailscale up --login-server https://vpn.example.com --exit-node=cloud-vps-01

Verify your public IP now matches the VPS:

curl ifconfig.me

10. Managing ACLs (Access Control Lists)

By default, every node in Headscale can talk to every other node. If you want to isolate development environments, restrict database ports, or limit subnet router access to specific users, use ACL policies.

Create /etc/headscale/acl.hujson:

{
  "acls": [
    // Allow all nodes within 'infra' user group full mutual access
    {
      "action": "accept",
      "src": ["infra"],
      "dst": ["infra:*"]
    },
    // Restrict access to home subnet: only dev machines can access port 22 and 443
    {
      "action": "accept",
      "src": ["tag:developer"],
      "dst": ["192.168.1.0/24:22", "192.168.1.0/24:443"]
    }
  ],
  "tagOwners": {
    "tag:developer": ["infra"]
  }
}

Update /etc/headscale/config.yaml to point to the ACL file:

acl:
  policy_path: /etc/headscale/acl.hujson

Restart Headscale to apply:

systemctl restart headscale

11. Verification and Troubleshooting

Check peer list and connection states on the Headscale server:

# List registered nodes
headscale nodes list

# Inspect specific node details
headscale nodes inspect --id 1

On any connected client:

# Check tailnet status and assigned 100.64.x.x IPs
tailscale status

# Test point-to-point latency and direct vs DERP routing
tailscale ping 100.64.0.2

# Check NAT traversal state and active DERP relays
tailscale netcheck

If tailscale ping shows via DERP, the nodes are falling back to relaying traffic through your Headscale DERP server because direct UDP handshakes failed. If it shows via <PUBLIC_IP>:<PORT>, nodes established direct, zero-relay WireGuard tunnels.