fix(wifi): add reconnect watchdog for brcmfmac roam/SAE wedge

Power save (fixed in #8) was not the only WiFi drop. On a WPA2/WPA3-
transition, band-steering AP (one SSID, several BSSIDs across 2.4/5 GHz),
NetworkManager's background scan roams to another BSSID and brcmfmac fails
SAE external auth on the transition BSSID (kernel: brcmf_cfg80211_external_
auth: External authentication failed: status=1). The link then dies -- or
stays "connected" while passing no traffic -- until a manual nmcli
reconnect. Verified on hardware; power_save was already off.

Install a WiFi recovery watchdog (customize.sh):
/usr/local/sbin/uconsole-wifi-watchdog driven by uconsole-wifi-watchdog.timer
(every 30s). If wlan0 is disconnected, or connected but the default gateway
is unreachable across two probes, it forces a NetworkManager reconnect. It
only acts when an autoconnect wifi profile exists (never fights a deliberate
disconnect) and uses only NetworkManager + iproute2 + iputils (all in base).

Document both drop causes and the per-device 5 GHz band-pin mitigation in
MAINTAINING (EN/JA). The band pin is network-specific, so it stays a device
tweak, not an image default.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Author: fabiiw05

MAINTAINING.ja.md

@@ -203,29 +203,62 @@ timedatectl timesync-status   # ServerName=ntp.nict.jp / synced になれば OK
 systemd-networkd-wait-online` を実行し、上記 `timesyncd.conf.d` を配置しておけば、
 以降のイメージはこの問題を最初から回避できる。
 
-### WiFi 不安定（brcmfmac パワーセーブ）
+### WiFi 不安定（brcmfmac）
 
 **症状**: WiFi が稀に切断される。`nmcli` で再接続すると復旧する。
 
-**原因**: `brcmfmac` ドライバ（CM4 内蔵 WiFi）のデフォルトで省電力
-モード（`Power save: on`）が有効になっており、省電力状態でチップが
-スリープに入り、復帰に失敗して再接続が必要になる。
+`brcmfmac`（CM4 内蔵 WiFi）は少なくとも2種類の理由でハングする。いずれも
+手動再接続まで復帰しないため、イメージには予防策（パワーセーブ無効）と
+復旧策（ウォッチドッグ）の両方を入れている。
 
-**対処（実機）**: NetworkManager でパワーセーブを恒久無効化する。
+**原因1 — パワーセーブ。** `brcmfmac` はデフォルトで省電力（`Power save:
+on`）。この状態でチップがスリープに入り、復帰に失敗することがある。
 
-```sh
-# 即時（再起動まで有効）
-sudo iw dev wlan0 set power_save off
+対処 — NetworkManager でパワーセーブを恒久無効化:
 
-# 恒久（現在・将来の全接続に適用）
-sudo tee /etc/NetworkManager/conf.d/wifi-powersave-off.conf <<'EOF'
+```sh
+sudo iw dev wlan0 set power_save off            # 即時（再起動まで）
+sudo tee /etc/NetworkManager/conf.d/wifi-powersave-off.conf <<'EOF'  # 恒久
 [connection]
 wifi.powersave=2
 EOF
 ```
 
-**イメージ側で根治済み**: `scripts/customize.sh` がビルド時に上記の
-設定ファイルを書き込むため、新規イメージでは最初から無効化されている。
+**原因2 — ローミング失敗。** WPA2/WPA3 transition かつ band-steering の AP
+（1 SSID・2.4/5GHz に複数 BSSID）では、NetworkManager のバックグラウンド
+スキャン（`bgscan simple:30:-65:300`）が別 BSSID へローミングし、transition
+BSSID で `brcmfmac` が SAE 外部認証に失敗する:
+
+```
+kernel: brcmf_cfg80211_external_auth: External authentication failed: status=1
+```
+
+その後リンクが死ぬ（または "connected" のまま無通信になる）まで手動再接続が
+必要になる。パワーセーブは無効なので、これは別バグ。
+
+対処 — バンド固定でローミング自体を止め（実機・接続ごと）、残るケースは
+下記ウォッチドッグで自動復旧させる:
+
+```sh
+# この接続を 5GHz に固定し、cross-band steering を封じる
+sudo nmcli connection modify <SSID> 802-11-wireless.band a
+sudo nmcli connection up <SSID>
+# （任意・最も強力: AP を1台に固定）
+#   sudo nmcli connection modify <SSID> 802-11-wireless.bssid AA:BB:CC:DD:EE:FF
+```
+
+バンド固定は SSID/バンドに依存するネットワーク固有設定なので、イメージ
+デフォルトではなく実機ごとの調整として残す。
+
+**イメージ側で根治済み**:
+- `scripts/customize.sh` がビルド時に上記パワーセーブ設定を書き込む。
+- 併せて **WiFi 復旧ウォッチドッグ** を導入 —
+  `/usr/local/sbin/uconsole-wifi-watchdog` を `uconsole-wifi-watchdog.timer`
+  （30秒毎）で駆動。`wlan0` が `disconnected`、または `connected` でも
+  デフォルトゲートウェイに2回連続到達不可なら、NetworkManager 再接続を強制
+  する。autoconnect な wifi プロファイルが存在する時だけ動作（意図的な切断
+  とは競合しない）。依存は NetworkManager + iproute2 + iputils のみ（すべて
+  base に同梱）。ログは `journalctl -t uconsole-wifi-watchdog` で確認。
 
 ## 今後の方向性
 

MAINTAINING.md

@@ -216,29 +216,62 @@ NetworkManager enablement, also run `systemctl disable systemd-networkd
 systemd-networkd.socket systemd-networkd-wait-online` and drop the
 `timesyncd.conf.d` file above, so future images avoid this out of the box.
 
-### WiFi instability (brcmfmac power save)
+### WiFi instability (brcmfmac)
 
 **Symptom**: WiFi occasionally drops; reconnecting via `nmcli` restores it.
 
-**Root cause**: the `brcmfmac` driver (CM4 onboard WiFi) has power saving
-enabled by default (`Power save: on`). In this state the driver occasionally
-puts the chip to sleep and fails to recover, requiring a manual reconnect.
+The `brcmfmac` driver (CM4 onboard WiFi) wedges in at least two distinct ways.
+Both leave the link dead until a manual reconnect, so the image now ships both a
+preventive fix (power save off) and a recovery net (a watchdog).
 
-**Fix (on device)**: disable power save permanently via NetworkManager.
+**Cause 1 — power save.** `brcmfmac` ships with power saving on (`Power save:
+on`); in this state it occasionally puts the chip to sleep and fails to wake.
 
-```sh
-# Immediate (until reboot)
-sudo iw dev wlan0 set power_save off
+Fix — disable power save permanently via NetworkManager:
 
-# Permanent (applies to all connections, current and future)
-sudo tee /etc/NetworkManager/conf.d/wifi-powersave-off.conf <<'EOF'
+```sh
+sudo iw dev wlan0 set power_save off            # immediate, until reboot
+sudo tee /etc/NetworkManager/conf.d/wifi-powersave-off.conf <<'EOF'  # permanent
 [connection]
 wifi.powersave=2
 EOF
 ```
 
-**Fixed in the image**: `scripts/customize.sh` writes the config file above
-at build time, so fresh images ship with WiFi power saving disabled.
+**Cause 2 — a failed roam.** On a WPA2/WPA3-transition, band-steering AP (one
+SSID, several BSSIDs across 2.4/5 GHz), NetworkManager's background scan
+(`bgscan simple:30:-65:300`) roams to another BSSID, and `brcmfmac` fails SAE
+external auth on the transition BSSID:
+
+```
+kernel: brcmf_cfg80211_external_auth: External authentication failed: status=1
+```
+
+The link then dies (or stays "connected" while passing no traffic) until a
+manual `nmcli` reconnect. Power save is already off, so this is a separate bug.
+
+Fix — cut the failing roam by pinning the band (per-connection, on device), and
+rely on the watchdog below to auto-recover the residual cases:
+
+```sh
+# Lock this connection to 5 GHz so cross-band steering can't trigger the roam.
+sudo nmcli connection modify <SSID> 802-11-wireless.band a
+sudo nmcli connection up <SSID>
+# (Optional, most aggressive: pin one AP outright)
+#   sudo nmcli connection modify <SSID> 802-11-wireless.bssid AA:BB:CC:DD:EE:FF
+```
+
+The band pin is network-specific (it names your SSID/band), so it stays a
+per-device tweak, not an image default.
+
+**Fixed in the image**:
+- `scripts/customize.sh` writes the power-save config above at build time.
+- It also installs a **WiFi recovery watchdog** — `/usr/local/sbin/uconsole-wifi-watchdog`
+  driven by `uconsole-wifi-watchdog.timer` (every 30 s). If `wlan0` is
+  `disconnected`, or `connected` but the default gateway is unreachable across
+  two probes, it forces a NetworkManager reconnect. It only acts when an
+  autoconnect wifi profile exists (never fights a deliberate disconnect) and
+  uses only NetworkManager + iproute2 + iputils (all already in the base).
+  Inspect with `journalctl -t uconsole-wifi-watchdog`.
 
 ## Future directions
 

scripts/customize.sh

@@ -166,6 +166,86 @@ cat > /etc/NetworkManager/conf.d/wifi-powersave-off.conf <<'EOF'
 wifi.powersave=2
 EOF
 
+# WiFi recovery watchdog: power save is not the only way the brcmfmac link
+# wedges. On WPA2/WPA3-transition, band-steering APs a background-scan roam to
+# another BSSID can fail SAE external auth (kernel: "brcmf_cfg80211_external_auth:
+# External authentication failed"), and the driver sometimes stays "connected"
+# while passing no traffic -- in both cases WiFi is dead until a manual `nmcli`
+# reconnect. A 30s timer detects the wedged state and forces NetworkManager to
+# reconnect wlan0, so the link self-heals. Uses only NetworkManager + iproute2 +
+# iputils (all already present). See MAINTAINING "WiFi instability".
+echo "==> [chroot] installing WiFi recovery watchdog (systemd timer)"
+mkdir -p /usr/local/sbin
+cat > /usr/local/sbin/uconsole-wifi-watchdog <<'WD'
+#!/usr/bin/env bash
+# Recover the CM4 onboard brcmfmac WiFi when it wedges (see customize.sh).
+set -u
+IFACE=wlan0
+
+# Nothing to do if the interface is absent or the radio is switched off.
+[[ -e "/sys/class/net/${IFACE}" ]] || exit 0
+[[ "$(nmcli -t -f WIFI radio 2>/dev/null)" == enabled ]] || exit 0
+
+# Only manage the link if an autoconnectable wifi profile exists, so we never
+# fight a user who deliberately disconnected.
+if ! nmcli -t -f TYPE,AUTOCONNECT connection show 2>/dev/null \
+    | grep -q '^802-11-wireless:yes'; then
+  exit 0
+fi
+
+state=$(nmcli -t -f DEVICE,STATE device 2>/dev/null \
+  | awk -F: -v i="$IFACE" '$1==i{print $2}')
+
+case "${state}" in
+  connected)
+    # L2 is up; confirm L3 by pinging the default gateway (LAN-local, cheap). A
+    # wedged brcmfmac often stays "connected" while passing no traffic. Require
+    # two failed probes a few seconds apart to avoid acting on a single loss.
+    gw=$(ip -4 route show default dev "${IFACE}" 2>/dev/null | awk '{print $3; exit}')
+    [[ -n "${gw}" ]] || exit 0
+    ping -c1 -W2 -I "${IFACE}" "${gw}" >/dev/null 2>&1 && exit 0
+    sleep 3
+    ping -c2 -W2 -I "${IFACE}" "${gw}" >/dev/null 2>&1 && exit 0
+    logger -t uconsole-wifi-watchdog "gateway ${gw} unreachable; reconnecting ${IFACE}"
+    nmcli device reconnect "${IFACE}" >/dev/null 2>&1 || true
+    ;;
+  disconnected)
+    # NM failed to recover on its own; force a reconnect to the best BSSID.
+    logger -t uconsole-wifi-watchdog "device ${IFACE} disconnected; forcing connect"
+    nmcli device connect "${IFACE}" >/dev/null 2>&1 || true
+    ;;
+  *)
+    # connecting / unavailable / unmanaged: an attempt is in progress or the
+    # device is not ours to touch -- leave it alone.
+    exit 0
+    ;;
+esac
+WD
+chmod +x /usr/local/sbin/uconsole-wifi-watchdog
+cat > /etc/systemd/system/uconsole-wifi-watchdog.service <<'UNIT'
+[Unit]
+Description=uConsole WiFi recovery watchdog (recover wedged brcmfmac link)
+After=NetworkManager.service
+Wants=NetworkManager.service
+
+[Service]
+Type=oneshot
+ExecStart=/usr/local/sbin/uconsole-wifi-watchdog
+UNIT
+cat > /etc/systemd/system/uconsole-wifi-watchdog.timer <<'UNIT'
+[Unit]
+Description=Run the uConsole WiFi recovery watchdog periodically
+
+[Timer]
+OnBootSec=60
+OnUnitActiveSec=30
+AccuracySec=5s
+
+[Install]
+WantedBy=timers.target
+UNIT
+systemctl enable uconsole-wifi-watchdog.timer || true
+
 # --- Internal speaker amplifier enable -------------------------------
 # The uConsole's onboard speaker amplifier is gated by an enable GPIO
 # (BCM11 / gpiochip0 line 11). Headphones bypass the amp, so without this the