Skip to content

README.md

Latest commit

 

History

History
240 lines (171 loc) · 12.2 KB

README.md

File metadata and controls

240 lines (171 loc) · 12.2 KB
Raw

MASQUE VPN server installation

This guide is tested on Ubuntu 22.04. Any modern systemd Linux with a public IP should work. All commands are run as root (or with sudo).

From Windows you can use masque-setup.exe (see windows/README.md) to SSH in, check the OS, pick a UDP port, open ufw if it is installed, and deploy this same layout. The GUI installer only supports Ubuntu 22.04/24.04/26.04 and Debian 12.

The server tunnels IP traffic over QUIC + HTTP/3 CONNECT-IP and authenticates clients with mutual TLS (mTLS). From v1.3 the server also:

  • sends QUIC keepalives (KeepAlivePeriod 15s, MaxIdleTimeout 3 minutes);
  • pins each client certificate CN to a stable tunnel /32 so Android can reconnect without rebuilding the TUN.

v1.4 / v1.4.1 clients talk to the same v1.3+ server protocol (sticky /32 after reconnect). v1.4.1 adds optional Docker packaging (server/docker-compose.yml) and graceful shutdown on SIGTERM/SIGINT. v1.5 optionally assigns ULA IPv6 (fd00:8::/64) inside the tunnel (sticky /128) and NAT66s it to the VPS WAN IPv6. IPv4-only configs without the v6 keys keep working. Old Android APKs sink IPv6 instead of forwarding it. v1.5.1 adds a CN denylist (/opt/masque/blocked_cns) and documents a UDP 443 fallback redirect. New client profiles default TUN MTU 1369.

The server binary on the VPS stays the 1.5.x build. It binds [::]:port (from v1.5.6) so QUIC can arrive over IPv4 or IPv6. A 1.7.1 client still connects to it. masque-setup.exe still picks one UDP port and an IPv4/DNS public host; use gen-config.sh --alt-port / --dial (or MASQUE_DIAL on install.sh) for a second port or an IPv6 literal.

v2.0.0 is a pre-release experiment, not Latest (release). That server listens for CONNECT-IP on UDP and TCP on the same port. Client tunnel MTU is 1340. Older UDP clients still connect. TCP fallback needs this binary. Install with install.sh or Docker (image masque-vpn-server:2.0.0, archive masque-docker-2.0.0.zip). masque-setup.exe is not in that release.

Keep the CA private key, server private key, and client private keys out of Git and distribute client bundles only through a secure channel.

1. Prerequisites

# Go 1.25+ (to build the server binary)
cd /tmp
curl -fsSLO https://go.dev/dl/go1.25.5.linux-amd64.tar.gz
rm -rf /usr/local/go && tar -C /usr/local -xzf go1.25.5.linux-amd64.tar.gz
export PATH=$PATH:/usr/local/go/bin
go version

# Build tools + openssl + iptables
apt-get update
apt-get install -y git build-essential openssl iptables

Open the server's UDP port in your firewall and cloud security group. The default is UDP 4433. If you choose another port, change it consistently in the server configuration, client profiles, firewall rules, and service setup. If clients cannot reach UDP 443, keep the process bound to 443 and add a VPS redirect — see UDP 443 blocked on the client path.

UDP socket buffers (required)

QUIC holds many UDP datagrams in the kernel while the process reads them. Stock Linux net.core.rmem_max / net.core.wmem_max are about 208 KiB. Under VPN load the receive queue fills, the kernel drops datagrams, and clients see loss, retransmits, and a slow tunnel.

Raise both ceilings to 64 MiB (67108864 bytes) — the size used on the reference server:

cp server/sysctl/99-masque-udp.conf /etc/sysctl.d/99-masque-udp.conf
sysctl --system
sysctl net.core.rmem_max net.core.wmem_max
# expect: 67108864

The systemd unit also applies the same sysctl -w values on every start. Keep the /etc/sysctl.d/ file so the limits survive reboot even if the service is not running yet. You do not need to change config.server.toml for this.

2. Get the code and build the server

You can compile on the VPS, or copy a prebuilt vpn-server-linux-amd64 / vpn-server-linux-arm64 from GitHub Actions artifacts (and, when a GitHub Release is published, from the release files). The binary contains no certificates or keys — you still run gen-config.sh on the server (or keep an existing CA).

# Example: copy the matching artifact onto the VPS, then:
install -m 0755 vpn-server-linux-amd64 /opt/masque/vpn-server

To build from source (Go 1.25+):

git clone https://github.com/Next1971/masque-vpn.git
cd masque-vpn/android/go-src/masque-vpn
go build -trimpath -ldflags "-s -w" -o vpn-server ./cmd/poc-server

The binary is built from cmd/poc-server. On the VPS you can name it vpn-server or poc-server; the systemd unit’s ExecStart= must match. Common layouts are /opt/masque/vpn-server or /opt/masque/bin/poc-server. v1.3 must replace this binary; client reconnect is not enough if the server still hands out a new /32 every session.

3. Deploy the binary

mkdir -p /opt/masque
cp vpn-server /opt/masque/vpn-server

4. Generate certificates and configuration

The generator creates the CA, the server certificate with the correct SAN, and ready-to-use client bundles for Windows and Android in one step.

cd /path/to/masque-vpn/server/scripts

# Replace the host with YOUR server's public domain or IP.
# --ip adds an extra IP to the server certificate SAN. Use it when clients may
# connect by IP even though --host is a domain.
./gen-config.sh --host vpn.example.com --ip 203.0.113.10 --clients 1

This produces ./out/:

out/
├── ca/                     internal CA  (ca.crt, ca.key)  << KEEP ca.key SECRET / BACK IT UP
├── server/                 server bundle (config.server.toml, server.crt, server.key, ca.crt)
├── windows/                Windows client bundle (profile.client.toml + certs/)
└── android/                Android client bundle (profile.masque + certs/)

Install the server bundle:

mkdir -p /opt/masque/cert
cp out/server/server.crt out/server/server.key out/server/ca.crt /opt/masque/cert/
cp out/server/config.server.toml /opt/masque/config.server.toml
chmod 600 /opt/masque/cert/server.key

Deliver the windows/ bundle to Windows users and the android/profile.masque file to Android users over a secure channel. To add more clients later without invalidating existing ones, reuse the CA:

./gen-config.sh --host vpn.example.com --reuse-ca ./out/ca --clients 1

5. Install the systemd service

# Edit server/systemd/masque.service if your public interface is not "eth0"
# (iptables and ip6tables -o). Find it with: ip route get 1.1.1.1  → the "dev XXX" name.
cp server/systemd/masque.service /etc/systemd/system/masque.service
systemctl daemon-reload
systemctl enable --now masque.service
systemctl status masque.service --no-pager

The unit enables IP forwarding, raises UDP rmem_max/wmem_max to 64 MiB, and adds an iptables MASQUERADE rule so client traffic is NATed out to the internet.

5b. Docker (optional)

Prefer a Linux VPS with Docker Engine. Host networking + /dev/net/tun match the systemd install. Docker Desktop on Windows/macOS is a poor fit for TUN/NAT.

# From the repo root, after gen-config.sh:
mkdir -p server/data/cert
cp out/server/config.server.toml server/data/
cp out/server/server.crt out/server/server.key out/server/ca.crt server/data/cert/
# Edit bind/server_name in config if needed (e.g. 0.0.0.0:443).

docker compose -f server/docker-compose.yml up -d --build
docker logs -f masque

The image builds vpn-server from android/go-src/masque-vpn. The entrypoint enables forwarding, 64 MiB UDP buffers, and MASQUERADE on the default WAN interface (MASQUE_WAN_IF to override). Mount only config + certs — no secrets are baked into the image.

Graceful shutdown: the process exits on SIGTERM/SIGINT (Compose stop, systemd stop).

6. Verify

# Should show the server listening on UDP 4433
ss -ulnp | grep 4433

# Follow logs
journalctl -u masque -f

You should see lines similar to mTLS ENABLED, listening on 0.0.0.0:4433, and IP pool ... ready.

7. Server configuration reference

config.server.toml:

[server]
bind        = "0.0.0.0:4433"       # UDP listen address:port
server_name = "vpn.example.com"    # must match the server certificate CN/SAN

[tls]
cert      = "/opt/masque/cert/server.crt"
key       = "/opt/masque/cert/server.key"
client_ca = "/opt/masque/cert/ca.crt"   # mTLS: clients verified against this CA
# Optional extra blocked CNs (v1.5.1). One CN per line is also read from /opt/masque/blocked_cns.
# blocked_cns = ["masque-client-7"]

[tun]
name = "masque0"
mtu  = 1369

[network]
tun_addr  = "10.8.0.1/24"    # server address on the tunnel
pool_cidr = "10.8.0.0/24"    # client address pool
route     = "0.0.0.0/0"      # route advertised to clients (0.0.0.0/0 = full tunnel)
# Optional dual-stack (v1.5). Omit all three to stay IPv4-only.
# tun_addr_v6  = "fd00:8::1/64"
# pool_cidr_v6 = "fd00:8::/64"
# route_v6     = "::/0"

IPv6 in the tunnel needs WAN IPv6 on the VPS, net.ipv6.conf.all.forwarding=1, and ip6tables MASQUERADE for fd00:8::/64 on the public interface (see masque.service). Clients still connect over IPv4 QUIC; do not publish an AAAA for the VPN hostname until a host-route bypass exists for that address.

UDP 443 blocked on the client path

Some networks (mobile carriers, hotel Wi-Fi, a few VPS providers on the egress side) drop outbound UDP 443 while other UDP ports still work. The MASQUE listener can stay on UDP 443. On the VPS, open an alternate port and redirect it to 443. 2053 is the usual choice.

# Cloud security group + host firewall must allow the alternate port.
ufw allow 2053/udp
ufw allow 2053/tcp

# Keep the server bound to 443. Redirect inbound 2053 → 443 for both transports.
iptables -t nat -C PREROUTING -p udp --dport 2053 -j REDIRECT --to-ports 443 2>/dev/null \
  || iptables -t nat -A PREROUTING -p udp --dport 2053 -j REDIRECT --to-ports 443
iptables -t nat -C PREROUTING -p tcp --dport 2053 -j REDIRECT --to-ports 443 2>/dev/null \
  || iptables -t nat -A PREROUTING -p tcp --dport 2053 -j REDIRECT --to-ports 443

# Persist on Debian/Ubuntu
apt-get install -y iptables-persistent
netfilter-persistent save

Then set the client profile address (or [server].server) to your.host:2053. TLS server_name stays the hostname from the certificate. Do not change bind in config.server.toml for this workaround. The TCP redirect uses the same alternate port, so a client that falls back from UDP reaches the same process.

v1.5.4 clients race the profile port and optional [server].alt_port (first QUIC handshake wins). Older clients ignore alt_port and dial one port. Generate it with gen-config.sh --alt-port 2053 (or any port you DNAT/listen on).

Revoking a client CN

This is a server-side denylist, not a certificate CRL. Append the mTLS Common Name (for example masque-client-7) to /opt/masque/blocked_cns, one name per line, then restart:

printf '%s\n' 'masque-client-7' >> /opt/masque/blocked_cns
chmod 0644 /opt/masque/blocked_cns
systemctl restart masque.service

masque-setup.exe (v1.5.1) does the same from Windows. The process reads the file at start; a live session for that CN is dropped on restart. The certificate files are not deleted.

Troubleshooting

Symptom Checks
No connection Confirm the UDP port in the profile is open in the VPS firewall and cloud firewall; if that port is 443 and packets never arrive, use UDP 443 blocked on the client path
HTTP 403 / rejected blocked CN CN is in /opt/masque/blocked_cns or tls.blocked_cns; restart after editing the file
Service fails to start Run systemctl status masque.service --no-pager and journalctl -u masque -f
Connected but slow / high loss Confirm sysctl net.core.rmem_max and wmem_max are 67108864 (64 MiB); see UDP socket buffers
Connected but no internet Check forwarding, the interface name in masque.service, iptables NAT, and routes
TLS/mTLS error Confirm server name, certificate chain, client certificate, and matching CA

Before sharing logs, remove domains, IP addresses, certificates, tokens, keys, and client-profile secrets.