Slow / unreliable connectivity with macOS client

[nnathan]

I haven't been using dsvpn much in the last couple of years but but in the past few months that when I do go to use it, it has been pretty unreliable and slow regardless of my location to my vantage point (server).

This is currently from my hotel to my server without dsvpn:

m:~ $ ping -c 10 angry.lastninja.net
PING angry.lastninja.net (152.53.240.229): 56 data bytes
64 bytes from 152.53.240.229: icmp_seq=0 ttl=50 time=196.815 ms
64 bytes from 152.53.240.229: icmp_seq=1 ttl=50 time=198.016 ms
64 bytes from 152.53.240.229: icmp_seq=2 ttl=50 time=212.956 ms
64 bytes from 152.53.240.229: icmp_seq=3 ttl=50 time=194.016 ms
64 bytes from 152.53.240.229: icmp_seq=4 ttl=50 time=191.218 ms
64 bytes from 152.53.240.229: icmp_seq=5 ttl=50 time=211.418 ms
64 bytes from 152.53.240.229: icmp_seq=6 ttl=50 time=203.754 ms
64 bytes from 152.53.240.229: icmp_seq=7 ttl=50 time=196.907 ms
64 bytes from 152.53.240.229: icmp_seq=8 ttl=50 time=199.512 ms
64 bytes from 152.53.240.229: icmp_seq=9 ttl=50 time=200.986 ms

--- angry.lastninja.net ping statistics ---
10 packets transmitted, 10 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 191.218/200.560/212.956/6.686 ms

From angry.lastninja.net to 1.1.1.1:

angry:~ $ ping -c 10 1.1.1.1
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
64 bytes from 1.1.1.1: icmp_seq=1 ttl=58 time=1.93 ms
64 bytes from 1.1.1.1: icmp_seq=2 ttl=58 time=1.84 ms
64 bytes from 1.1.1.1: icmp_seq=3 ttl=58 time=1.93 ms
64 bytes from 1.1.1.1: icmp_seq=4 ttl=58 time=1.94 ms
64 bytes from 1.1.1.1: icmp_seq=5 ttl=58 time=1.92 ms
64 bytes from 1.1.1.1: icmp_seq=6 ttl=58 time=1.82 ms
64 bytes from 1.1.1.1: icmp_seq=7 ttl=58 time=1.87 ms
64 bytes from 1.1.1.1: icmp_seq=8 ttl=58 time=1.87 ms
64 bytes from 1.1.1.1: icmp_seq=9 ttl=58 time=1.87 ms
64 bytes from 1.1.1.1: icmp_seq=10 ttl=58 time=1.90 ms

--- 1.1.1.1 ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 9012ms
rtt min/avg/max/mdev = 1.815/1.887/1.940/0.039 ms

Now when I try to use the dsvpn client from my hotel, this is what I am contending with:

m:~ $ ping -c 10 1.1.1.1
PING 1.1.1.1 (1.1.1.1): 56 data bytes
64 bytes from 1.1.1.1: icmp_seq=0 ttl=57 time=552.386 ms
64 bytes from 1.1.1.1: icmp_seq=1 ttl=57 time=371.872 ms
64 bytes from 1.1.1.1: icmp_seq=2 ttl=57 time=385.418 ms
64 bytes from 1.1.1.1: icmp_seq=3 ttl=57 time=479.062 ms
64 bytes from 1.1.1.1: icmp_seq=4 ttl=57 time=982.022 ms
64 bytes from 1.1.1.1: icmp_seq=5 ttl=57 time=376.685 ms
Request timeout for icmp_seq 6
Request timeout for icmp_seq 7
64 bytes from 1.1.1.1: icmp_seq=6 ttl=57 time=2150.549 ms
64 bytes from 1.1.1.1: icmp_seq=7 ttl=57 time=1390.549 ms
64 bytes from 1.1.1.1: icmp_seq=8 ttl=57 time=742.387 ms
64 bytes from 1.1.1.1: icmp_seq=9 ttl=57 time=296.318 ms

--- 1.1.1.1 ping statistics ---
10 packets transmitted, 10 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 296.318/772.725/2150.549/561.615 ms

m:~ $ ping -c 10 angry.lastninja.net
PING angry.lastninja.net (152.53.240.229): 56 data bytes
64 bytes from 152.53.240.229: icmp_seq=0 ttl=50 time=192.224 ms
64 bytes from 152.53.240.229: icmp_seq=1 ttl=50 time=191.164 ms
64 bytes from 152.53.240.229: icmp_seq=2 ttl=50 time=192.316 ms
64 bytes from 152.53.240.229: icmp_seq=3 ttl=50 time=191.309 ms
64 bytes from 152.53.240.229: icmp_seq=4 ttl=50 time=194.101 ms
64 bytes from 152.53.240.229: icmp_seq=5 ttl=50 time=191.567 ms
64 bytes from 152.53.240.229: icmp_seq=6 ttl=50 time=208.852 ms
64 bytes from 152.53.240.229: icmp_seq=7 ttl=50 time=219.499 ms
64 bytes from 152.53.240.229: icmp_seq=8 ttl=50 time=214.687 ms
64 bytes from 152.53.240.229: icmp_seq=9 ttl=50 time=204.168 ms

--- angry.lastninja.net ping statistics ---
10 packets transmitted, 10 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 191.164/199.989/219.499/10.347 ms

The ping to 1.1.1.1 is quite abnormal as it really should be just the RTT to angry.lastninja.net (~210ms) + roughly 2ms.

This is the extent of my troubleshoot so far, was hoping you might have some insight as to what might be going on.

Using the latest version of dsvpn from github on Arch Linux on angry.

angry:dsvpn $ sudo ./dsvpn server vpn.key auto 5000

Using the latest version of dsvpn from homebrew on hostname m.

m:~ $ sudo dsvpn client vpn.key angry.lastninja.net 5000

m:~ $ sw_vers
ProductName:            macOS
ProductVersion:         26.4.1
BuildVersion:           25E253

[nnathan]

I spun up a VM on my mac (m) to do some testing with Arch Linux.

Before connecting via dsvpn:

[root@v dsvpn]# ping -c 100 -f angry.lastninja.net
PING angry.lastninja.net (152.53.240.229) 56(84) bytes of data.

--- angry.lastninja.net ping statistics ---
100 packets transmitted, 100 received, 0% packet loss, time 1317ms
rtt min/avg/max/mdev = 190.769/200.272/275.641/18.239 ms, pipe 22, ipg/ewma 13.300/191.991 ms

After connecting to dsvpn:

[root@v dsvpn]# ping -c 100 -f angry.lastninja.net
PING angry.lastninja.net (152.53.240.229) 56(84) bytes of data.

--- angry.lastninja.net ping statistics ---
100 packets transmitted, 100 received, 0% packet loss, time 1195ms
rtt min/avg/max/mdev = 204.530/298.315/417.764/51.116 ms, pipe 34, ipg/ewma 12.071/285.494 ms

Odd that there's a delta of about 90ms on average once connected via dsvpn, since it is pinging the vpn termination endpoint.

[root@v dsvpn]# ping -c 100 -f 1.1.1.1
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
.
--- 1.1.1.1 ping statistics ---
100 packets transmitted, 99 received, 1% packet loss, time 1194ms
rtt min/avg/max/mdev = 198.087/314.223/617.809/65.315 ms, pipe 35, ipg/ewma 12.061/368.151 ms

Again that's a huge delta for pinging 1.1.1.1.

The issue is a lot less pronounced on the Linux side of things run on a VM on a mac -- that is there's a significant latency delta (but much less compared to the mac host) (see edit below), but it doesn't feel slow or unreliable that I see when connecting on the host (mac) itself.

Edit: Things look pretty normal when you don't flood ping. So maybe that artificial latency could be dsvpn when handling a large flood of packets.

[root@v dsvpn]# ping -c 10 angry.lastninja.net
PING angry.lastninja.net (152.53.240.229) 56(84) bytes of data.
64 bytes from v2202512327516422119.quicksrv.de (152.53.240.229): icmp_seq=1 ttl=64 time=199 ms
64 bytes from v2202512327516422119.quicksrv.de (152.53.240.229): icmp_seq=2 ttl=64 time=204 ms
64 bytes from v2202512327516422119.quicksrv.de (152.53.240.229): icmp_seq=3 ttl=64 time=198 ms
64 bytes from v2202512327516422119.quicksrv.de (152.53.240.229): icmp_seq=4 ttl=64 time=201 ms
64 bytes from v2202512327516422119.quicksrv.de (152.53.240.229): icmp_seq=5 ttl=64 time=196 ms
64 bytes from v2202512327516422119.quicksrv.de (152.53.240.229): icmp_seq=6 ttl=64 time=196 ms
64 bytes from v2202512327516422119.quicksrv.de (152.53.240.229): icmp_seq=7 ttl=64 time=197 ms
64 bytes from v2202512327516422119.quicksrv.de (152.53.240.229): icmp_seq=8 ttl=64 time=207 ms
64 bytes from v2202512327516422119.quicksrv.de (152.53.240.229): icmp_seq=9 ttl=64 time=205 ms
64 bytes from v2202512327516422119.quicksrv.de (152.53.240.229): icmp_seq=10 ttl=64 time=197 ms

--- angry.lastninja.net ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 9023ms
rtt min/avg/max/mdev = 195.778/200.060/207.493/3.880 ms
[root@v dsvpn]# ping -c 10 1.1.1.1
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
64 bytes from 1.1.1.1: icmp_seq=1 ttl=57 time=197 ms
64 bytes from 1.1.1.1: icmp_seq=2 ttl=57 time=198 ms
64 bytes from 1.1.1.1: icmp_seq=3 ttl=57 time=198 ms
64 bytes from 1.1.1.1: icmp_seq=4 ttl=57 time=207 ms
64 bytes from 1.1.1.1: icmp_seq=5 ttl=57 time=199 ms
64 bytes from 1.1.1.1: icmp_seq=6 ttl=57 time=199 ms
64 bytes from 1.1.1.1: icmp_seq=7 ttl=57 time=203 ms
64 bytes from 1.1.1.1: icmp_seq=8 ttl=57 time=198 ms
64 bytes from 1.1.1.1: icmp_seq=9 ttl=57 time=198 ms
64 bytes from 1.1.1.1: icmp_seq=10 ttl=57 time=198 ms

--- 1.1.1.1 ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 9025ms
rtt min/avg/max/mdev = 196.658/199.371/207.496/3.136 ms

[jedisct1]

The direct ping to the server can stay clean while the tunnel is bad: packets inside the tunnel are queued in the single TCP connection used by DSVPN, so any loss/throttling/bufferbloat on that TCP stream can show up as large latency spikes for ICMP through the tunnel.

A few things worth checking:

• Try port 443 instead of 5000. Some hotel/public networks treat non-standard TCP ports very differently, even when ping to the same host looks fine.
• On the Linux server, check what congestion control the DSVPN socket actually gets: sysctl net.ipv4.tcp_congestion_control net.ipv4.tcp_available_congestion_control net.core.default_qdisc. BBR + fq generally behaves much better here.
• While connected on macOS, verify that the route to angry.lastninja.net still goes outside the tunnel: route -n get angry.lastninja.net and netstat -rn -f inet. DSVPN adds a host route for the server before the default split routes, but this is worth confirming.
• Please try building/running the client from current git on macOS too, not only the Homebrew binary, so we can rule out an outdated formula/build difference.
• If possible, compare with a plain TCP test to the server on the same port/network, e.g. iperf3 or even an SSH/scp transfer, to see whether the TCP connection itself is getting queued/retransmitted.

The Linux VM result being much better than the macOS host is interesting. If the same network/server/port behaves badly only on native macOS, this may be macOS TCP/utun behavior or routing rather than the server side.

[nnathan]

I've switched to using the git version of dspvn on the mac.

• Tried port 443, exhibits the same issue as 5000. Both work fine in Linux VM.

• sysctl on angry:

angry:~ $ sysctl net.ipv4.tcp_congestion_control net.ipv4.tcp_available_congestion_control net.core.default_qdisc
net.ipv4.tcp_congestion_control = cubic
net.ipv4.tcp_available_congestion_control = reno cubic bbr
net.core.default_qdisc = fq_codel

• Verified that route get goes direct to the wireless router and route get has a direct host route when dsvpn is connected. Host connectivity appears normal baseline when connected via dsvpn to angry.

• I used scp instead of iperf3 as it was a bit quicker. Without dsvpn I get 2.6-3.7MB/s which is sufficient. I get the same with dsvpn connected.

Everything seems fine on Linux VM.

Although I am on my travel macbook air, I've seen the same issues at my house on my macbook pro 16", so I don't think it is isolated to the laptop I am using at the moment.

[nnathan]

I just had a friend quickly test on his macbook too, it was exhibiting the same symptoms as well. I didn't catch his macOS version though.

[nnathan]

Tried BBR:

angry:~ $ sysctl net.ipv4.tcp_congestion_control net.ipv4.tcp_available_congestion_control net.core.default_qdisc
net.ipv4.tcp_congestion_control = bbr
net.ipv4.tcp_available_congestion_control = reno cubic bbr
net.core.default_qdisc = fq_codel

(Then started dsvpn server again)

I'm getting the following pings:

m:dsvpn $ ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1): 56 data bytes
64 bytes from 1.1.1.1: icmp_seq=0 ttl=57 time=298.207 ms
64 bytes from 1.1.1.1: icmp_seq=1 ttl=57 time=574.025 ms
64 bytes from 1.1.1.1: icmp_seq=2 ttl=57 time=506.731 ms
64 bytes from 1.1.1.1: icmp_seq=3 ttl=57 time=228.739 ms
64 bytes from 1.1.1.1: icmp_seq=4 ttl=57 time=712.191 ms
64 bytes from 1.1.1.1: icmp_seq=5 ttl=57 time=337.765 ms
64 bytes from 1.1.1.1: icmp_seq=6 ttl=57 time=811.473 ms
64 bytes from 1.1.1.1: icmp_seq=7 ttl=57 time=474.684 ms
^C
--- 1.1.1.1 ping statistics ---
9 packets transmitted, 8 packets received, 11.1% packet loss
round-trip min/avg/max/stddev = 228.739/492.977/811.473/189.880 ms

[jedisct1]

Thanks, this is really useful data.

The combination of:

• current git build on macOS behaving the same as Homebrew
• ports 443 and 5000 behaving the same
• Linux VM on the same Mac/network looking fine with normal pings
• plain scp throughput staying normal while DSVPN is connected
• BBR on the server not changing the symptom
• another MacBook showing the same behavior

pretty much rules out the hotel network, the server TCP congestion control, and an outdated formula as the main cause.

This is starting to look like something specific to the native macOS path, probably either utun behavior or a macOS TCP/utun interaction. One suspicious area is that the macOS tun_write() path uses a synchronous writev() to the utun device. If that ever blocks or backpressures, the single event loop can temporarily stop reading/writing the TCP socket, which would show up exactly as bursty latency inside the tunnel while the direct route to the server stays clean.

A couple more small tests would help narrow that down if you have time:

• If you can get your friend's macOS version, that would be useful. Same for any test on an older macOS release if one is available.

• After DSVPN starts, try lowering the utun MTU and then repeat the ping 1.1.1.1 test:

  ifconfig | grep -A4 '^utun'
  sudo ifconfig utunX mtu 1280
  # or even 1200 if 1280 doesn't change anything

• During a bad ping run, check whether the outer TCP connection is actually retransmitting, or whether this is local queueing. A simple before/after is enough:

  netstat -s -p tcp
  ping -c 20 1.1.1.1
  netstat -s -p tcp

  If retransmits don't move much while latency spikes, that would point even more strongly at the local macOS/utun side.

I'll try to reproduce this on macOS and look at making the utun side non-blocking / better isolated from the TCP side. The Linux VM comparison was especially helpful, thanks!

[nnathan]

• Just waiting for friend to respond and I will update his macos version in a new comment. Also should be having another friend test soon as well he has a vintage intel mac so it'll come with an older version of macOS, possibly OSX which could identify any discrepency between versions.

• with mtu 1280

m:~ $ ifconfig utun6
utun6: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1280
        inet 192.168.192.1 --> 192.168.192.254 netmask 0xffffff00
        inet6 fe80::fcbf:bed4:7710:6459%utun6 prefixlen 64 scopeid 0x15
        inet6 64:ff9b::c0a8:c001 --> 64:ff9b::c0a8:c0fe prefixlen 128
        nd6 options=201<PERFORMNUD,DAD>
m:~ $ ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1): 56 data bytes
64 bytes from 1.1.1.1: icmp_seq=0 ttl=57 time=717.629 ms
64 bytes from 1.1.1.1: icmp_seq=1 ttl=57 time=635.001 ms
64 bytes from 1.1.1.1: icmp_seq=2 ttl=57 time=442.260 ms
64 bytes from 1.1.1.1: icmp_seq=3 ttl=57 time=349.370 ms
64 bytes from 1.1.1.1: icmp_seq=4 ttl=57 time=231.616 ms
^C
--- 1.1.1.1 ping statistics ---
5 packets transmitted, 5 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 231.616/475.175/717.629/179.198 ms

• no tcp retransmissions

m:~ $ netstat -s -p tcp | egrep -v '^\s*0'
tcp:
        39 open TCP sockets
m:~ $ ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1): 56 data bytes
64 bytes from 1.1.1.1: icmp_seq=0 ttl=57 time=244.120 ms
64 bytes from 1.1.1.1: icmp_seq=1 ttl=57 time=270.321 ms
64 bytes from 1.1.1.1: icmp_seq=2 ttl=57 time=282.039 ms
64 bytes from 1.1.1.1: icmp_seq=3 ttl=57 time=277.583 ms
64 bytes from 1.1.1.1: icmp_seq=4 ttl=57 time=321.433 ms
64 bytes from 1.1.1.1: icmp_seq=5 ttl=57 time=270.082 ms
64 bytes from 1.1.1.1: icmp_seq=6 ttl=57 time=425.621 ms
64 bytes from 1.1.1.1: icmp_seq=7 ttl=57 time=795.503 ms
64 bytes from 1.1.1.1: icmp_seq=8 ttl=57 time=303.053 ms
64 bytes from 1.1.1.1: icmp_seq=9 ttl=57 time=505.246 ms
64 bytes from 1.1.1.1: icmp_seq=10 ttl=57 time=213.473 ms
^C
--- 1.1.1.1 ping statistics ---
12 packets transmitted, 11 packets received, 8.3% packet loss
round-trip min/avg/max/stddev = 213.473/355.316/795.503/160.525 ms
m:~ $ netstat -s -p tcp | egrep -v '^\s*0'
tcp:
        41 open TCP sockets

[jedisct1]

Thanks! The MTU test not changing things and the TCP counters staying flat are both pretty strong signals.

So this probably isn't fragmentation, and it doesn't look like the outer TCP connection is losing/retransmitting packets. That narrows it down to local queueing/stalling on one of the native macOS sides of the event loop: reading from utun, writing to utun, or the process getting stuck long enough that the TCP socket isn't serviced promptly.

One useful next split would be to test the tunnel endpoint itself, not just traffic forwarded through the server:

ping -c 20 192.168.192.254

from the macOS client while connected. If that shows the same spikes, the forwarding/NAT path to 1.1.1.1 is out of the picture and the issue is entirely inside the client<->server tunnel path.

If you can also run the inverse from the server:

ping -c 20 192.168.192.1

that may help show whether one direction is worse than the other.

I'm going to look at a small macOS-specific test change next: make the utun fd non-blocking and avoid letting a blocked utun write stall the TCP side of the loop. If this is local backpressure, that should at least change the failure mode, and hopefully improve it.

[nnathan]

The tunnel endpoints from both sides don't seem to allow ICMP.

My friend's macOS version that tested earlier:

n@ns-MacBook-Air ~/snap/dsvpn $  sw_vers
ProductName:            macOS
ProductVersion:         26.4.1
BuildVersion:           25E253

[jedisct1]

No worries about the endpoint ICMP test; that's not a blocker.

The friend's version is useful though: so far the native macOS reports are all on 26.4.1 / 25E253, while the Linux VM on the same hardware/network behaves normally. That makes me less inclined to chase routing/NAT/server-side behavior and more inclined to look at a macOS-version-specific utun/socket interaction.

If the Intel Mac / older macOS test happens, that result would be especially interesting. If it works there, we probably have a recent macOS regression or changed utun behavior. If it fails there too, then it's more likely a long-standing bug in DSVPN's macOS event loop that only became visible with your current testing.

I'll focus next on a small test patch for macOS to make the utun side non-blocking / less able to stall the TCP side. Thanks again for doing all these checks.

[nnathan]

My other friend with vintage intel mac:

% sw_vers 
ProductName:        macOS
ProductVersion:        15.7.5
BuildVersion:        24G624

Baseline:

% ping -c 10 angry.lastninja.net
PING angry.lastninja.net (152.53.240.229): 56 data bytes
64 bytes from 152.53.240.229: icmp_seq=0 ttl=52 time=81.979 ms
64 bytes from 152.53.240.229: icmp_seq=1 ttl=52 time=90.636 ms
64 bytes from 152.53.240.229: icmp_seq=2 ttl=52 time=78.496 ms
64 bytes from 152.53.240.229: icmp_seq=3 ttl=52 time=88.752 ms
64 bytes from 152.53.240.229: icmp_seq=4 ttl=52 time=81.737 ms
64 bytes from 152.53.240.229: icmp_seq=5 ttl=52 time=83.821 ms
64 bytes from 152.53.240.229: icmp_seq=6 ttl=52 time=81.974 ms
64 bytes from 152.53.240.229: icmp_seq=7 ttl=52 time=79.629 ms
64 bytes from 152.53.240.229: icmp_seq=8 ttl=52 time=92.545 ms
64 bytes from 152.53.240.229: icmp_seq=9 ttl=52 time=90.183 ms

--- angry.lastninja.net ping statistics ---
10 packets transmitted, 10 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 78.496/84.975/92.545/4.807 ms

Connected to angry:

% ping -c 10 1.1.1.1
PING 1.1.1.1 (1.1.1.1): 56 data bytes
64 bytes from 1.1.1.1: icmp_seq=0 ttl=57 time=89.511 ms
64 bytes from 1.1.1.1: icmp_seq=1 ttl=57 time=84.513 ms
64 bytes from 1.1.1.1: icmp_seq=2 ttl=57 time=83.503 ms
64 bytes from 1.1.1.1: icmp_seq=3 ttl=57 time=82.781 ms
64 bytes from 1.1.1.1: icmp_seq=4 ttl=57 time=86.662 ms
64 bytes from 1.1.1.1: icmp_seq=5 ttl=57 time=84.557 ms
64 bytes from 1.1.1.1: icmp_seq=6 ttl=57 time=85.594 ms
64 bytes from 1.1.1.1: icmp_seq=7 ttl=57 time=86.308 ms
64 bytes from 1.1.1.1: icmp_seq=8 ttl=57 time=82.419 ms
64 bytes from 1.1.1.1: icmp_seq=9 ttl=57 time=85.534 ms

--- 1.1.1.1 ping statistics ---
10 packets transmitted, 10 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 82.419/85.138/89.511/1.988 ms

Doesn't exhibit issue.

[jedisct1]

That's a very useful data point. So macOS 15.7.5 / 24G624 on the Intel Mac does not show the issue, while the machines on 26.4.1 / 25E253 do. That makes this look much more like a recent macOS behavior change/regression than a generic DSVPN-on-macOS problem.

Could you try this small diagnostic patch on one of the affected 26.4.1 machines? It makes the macOS utun fd non-blocking and logs if writes back to utun would block. The patch applies cleanly on current git for me.

--- a/src/os.c
+++ b/src/os.c
@@ -277,6 +277,10 @@
         },
     };
     ret = writev(fd, iov, 2);
+    if (ret < (ssize_t) 0 && (errno == EAGAIN || errno == EWOULDBLOCK)) {
+        errno = EAGAIN;
+        return 0;
+    }
     if (ret <= (ssize_t) 0) {
         return ret;
     }
--- a/src/vpn.c
+++ b/src/vpn.c
@@ -358,6 +358,7 @@
     ssize_t              len;
     int                  found_fds;
     int                  new_client_fd;
+    static unsigned int  tun_eagain_count;
 
     if (exit_signal_received != 0) {
         return -2;
@@ -387,6 +388,9 @@
     }
     if (fds[POLLFD_TUN].revents & POLLIN) {
         len = tun_read(context->tun_fd, tun_buf.data, sizeof tun_buf.data);
+        if (len < (ssize_t) 0 && (errno == EAGAIN || errno == EWOULDBLOCK)) {
+            return 0;
+        }
         if (len <= 0) {
             perror("tun_read");
             return -1;
@@ -428,6 +432,7 @@
         uint16_t binlen;
         size_t   len_with_header;
         ssize_t  readnb;
+        ssize_t  tun_writenb;
 
         if ((readnb = safe_read_partial(context->client_fd, client_buf->len + client_buf->pos,
                                         2 + TAG_LEN + MAX_PACKET_LEN - client_buf->pos)) <= 0) {
@@ -448,7 +453,13 @@
                 sleep(1);
                 return client_reconnect(context);
             }
-            if (tun_write(context->tun_fd, client_buf->data, (size_t) len) != len) {
+            tun_writenb = tun_write(context->tun_fd, client_buf->data, (size_t) len);
+            if (tun_writenb == 0 && errno == EAGAIN) {
+                tun_eagain_count++;
+                if ((tun_eagain_count & 255U) == 1U) {
+                    fprintf(stderr, "utun write would block (%u)\n", tun_eagain_count);
+                }
+            } else if (tun_writenb != len) {
                 perror("tun_write");
             }
             if (2 + TAG_LEN + MAX_PACKET_LEN != len_with_header) {
@@ -603,6 +614,9 @@
         return 1;
     }
     printf("Interface: [%s]\n", context.if_name);
+#ifdef __APPLE__
+    (void) fcntl(context.tun_fd, F_SETFL, fcntl(context.tun_fd, F_GETFL, 0) | O_NONBLOCK);
+#endif
     if (tun_set_mtu(context.if_name, DEFAULT_MTU) != 0) {
         perror("mtu");
     }

Then rebuild/run the macOS client from that tree:

make clean
make
sudo ./dsvpn client vpn.key angry.lastninja.net 443

The interesting result is whether utun write would block (...) appears while the ping latency spikes. If it does, we have a very direct signal that the new macOS utun path is applying backpressure and the current single-loop design handles that poorly. If the latency still spikes but no messages appear, I'll look at the opposite direction next: utun reads or TCP writes.

[nnathan]

Still exhibiting the issue:

m:dsvpn $ ping -c 10 angry.lastninja.net
PING angry.lastninja.net (152.53.240.229): 56 data bytes
64 bytes from 152.53.240.229: icmp_seq=0 ttl=50 time=192.055 ms
64 bytes from 152.53.240.229: icmp_seq=1 ttl=50 time=201.132 ms
64 bytes from 152.53.240.229: icmp_seq=2 ttl=50 time=191.440 ms
64 bytes from 152.53.240.229: icmp_seq=3 ttl=50 time=191.616 ms
64 bytes from 152.53.240.229: icmp_seq=4 ttl=50 time=191.185 ms
64 bytes from 152.53.240.229: icmp_seq=5 ttl=50 time=191.899 ms
64 bytes from 152.53.240.229: icmp_seq=6 ttl=50 time=207.879 ms
64 bytes from 152.53.240.229: icmp_seq=7 ttl=50 time=191.987 ms
64 bytes from 152.53.240.229: icmp_seq=8 ttl=50 time=191.814 ms
64 bytes from 152.53.240.229: icmp_seq=9 ttl=50 time=191.800 ms

--- angry.lastninja.net ping statistics ---
10 packets transmitted, 10 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 191.185/194.281/207.879/5.336 ms
m:dsvpn $ ping -c 10 1.1.1.1
PING 1.1.1.1 (1.1.1.1): 56 data bytes
64 bytes from 1.1.1.1: icmp_seq=0 ttl=57 time=507.827 ms
64 bytes from 1.1.1.1: icmp_seq=1 ttl=57 time=347.397 ms
64 bytes from 1.1.1.1: icmp_seq=2 ttl=57 time=675.382 ms
64 bytes from 1.1.1.1: icmp_seq=3 ttl=57 time=415.643 ms
64 bytes from 1.1.1.1: icmp_seq=4 ttl=57 time=239.096 ms
64 bytes from 1.1.1.1: icmp_seq=5 ttl=57 time=344.753 ms
64 bytes from 1.1.1.1: icmp_seq=6 ttl=57 time=205.273 ms
64 bytes from 1.1.1.1: icmp_seq=7 ttl=57 time=212.278 ms
64 bytes from 1.1.1.1: icmp_seq=8 ttl=57 time=328.411 ms
64 bytes from 1.1.1.1: icmp_seq=9 ttl=57 time=378.873 ms

--- 1.1.1.1 ping statistics ---
10 packets transmitted, 10 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 205.273/365.493/675.382/136.469 ms

This was the ongoing status:

m:dsvpn $ sudo ./dsvpn client vpn.key angry.lastninja.net 5000
Interface: [utun6]
Trying to reconnect
Connecting to 152.53.240.229:5000...
add host 152.53.240.229: gateway 192.168.1.251
add net 0: gateway 192.168.192.254
add net 128: gateway 192.168.192.254
add net 0000::/1: gateway 64:ff9b::192.168.192.254
add net 8000::/1: gateway 64:ff9b::192.168.192.254
Connected