mesh_gate: split by concern, guard the ARP table, add introspection - #2
aldwinhermanudin wants to merge 5 commits into
Conversation
app_main.c was 1013 lines, about three quarters of which was infrastructure
identical for any gate. Split along the seams so each concern reads on its
own, and so the reusable parts are visibly separable if they are ever lifted
into a component:
app_main.c 182 bring-up order and a status loop
gate_netif.c 252 AP-side esp_netif, local delivery, addressing
gate_bridge.c 342 the datapath -- three receive callbacks
gate_arp.c 375 proxy ARP
gate.h 111 the seam between them
No behavioural change was intended. Verified against the unmodified Rimba
fixture test-mesh-gate-sta on the same three-board rig: DHCP lease, 14/15
replies to a mesh node, ttl=64 (pure L2 bridge). The original scored 13/15;
the difference is within the bench's RX-overload variance and is not claimed
as an improvement.
Three things did change:
- The proxy-ARP table is now guarded. It has three concurrent accessors --
both receive tasks snoop into it and the announce task ages and reads it --
and was previously unlocked, so two receive contexts could interleave on a
slot and publish a mismatched IP/MAC pair. The proactive push transmits, so
it snapshots the table under the lock and sends afterwards rather than
holding it across the loop.
- Introspection. The gate could not previously report what it had learned
without editing the source. It now logs associated clients, mesh peers,
cross-bridge ARP mappings and known gates, backed by small accessors.
- A _Static_assert couples EXAMPLE_AP_MAX_STAS to GATE_MAX_CLIENTS. Only the
mesh->AP direction consults the client table, so a client past the end of it
would associate, take a DHCP lease, send into the mesh, and silently never
receive a reply. The Kconfig range already prevents this; the assert is what
keeps the two in step if someone widens that range.
Also folded four open-coded copies of "alloc mmpkt, append, tag VIF_AP,
transmit" into gate_netif_tx_ap(). The netif hook keeps its blocking
tx_wait_until_ready because it runs on the lwIP task; receive-context callers
must not block, and that distinction is now explicit rather than incidental.
The target hardware address sits at ARP-payload offset 18, which is absolute 32 once the 14-byte Ethernet header is counted. The builder wrote it at 34, so: out[32..33] never written -- uninitialised stack went on air out[34..39] the requester MAC, shifted two bytes late out[38..39] written twice, the target protocol address winning Every proxy-ARP reply the gate sent to an AP client therefore carried a malformed target hardware address and leaked two bytes of stack. It resolved correctly anyway, which is why it survived every bridge test: lwIP's etharp_input decides "for us" from the target PROTOCOL address, which was correct, and populates its cache from the sender hardware/protocol pair, which was also correct. Nothing on the receive path reads the field that was wrong. The mesh-side SNAP builder in the same file always had this right (a + 18), and that disagreement between two builders of the same structure is what exposed it.
…l comment Brings the example in step with the shipped Rimba gate, which raised its client ceiling and made the proxy-ARP push event-driven. Client ceiling 8 -> 16, with the arithmetic recorded where the constant is defined. The proactive push is O(clients x mesh hosts) in BOTH directions at ~4.85 ms per push, so with 5 mesh nodes it costs 5.2% of airtime at 16 clients on a 15 s period, 25.8% on a 3 s period, and 51.7% at 32 clients on 3 s. Two ceilings outside the example bind before the array does, and both are noted: MESH_MPP_MAX (32), which lives on every mesh node and which 16 clients on one gate already half-fills, and the flat /24's ~98 usable addresses. None of them is fixed by a faster PHY. ARP_TABLE_MAX is now derived (AP_MAX_STAS + 32) rather than a flat 24. It holds BOTH sides, so at 24 with 5 mesh nodes it saturated near 19 clients and then thrashed its LRU -- silently ceasing to teach ARP rather than failing. The push period default moves 3 s -> 15 s, which is only safe because the push is now event-driven on top of it: arp_learn() wakes the announce task on a genuinely new mapping, and the task waits with ulTaskNotifyTake(pdTRUE, ...) so notifications collapse. A joining host is taught in milliseconds; the period is upkeep. The task is now started BEFORE the receive callbacks are registered, because the wake is by handle and a host learned while that handle is NULL drops its wake silently and waits out a full period. Also corrects an actively wrong comment in gate_netif.c. CONFIG_LWIP_DHCPS_MAX_STATION_NUM bounds the lease-record list, NOT the address range: the range is start = server + 1 = .2 through start + DHCPS_MAX_LEASE (100), so the pool is .2-.101 whatever that Kconfig says, and its top two addresses do overlap the mesh nodes' .100 upwards. Unreachable below ~100 clients, and pinning it risks the DHCP_NOT_STOPPED boot loop, so it is recorded rather than changed. The old comment made raising the ceiling look dangerous when it is not, and made the real overlap look impossible.
The "Verified on hardware" section listed association, DHCP, reply counts and TTL -- all read from serial logs and ping -- under a heading that reads as a stronger claim than the evidence supports. Per the project rule that the verification tier is recorded per frame, the section now says what the evidence is: log-level, no capture, no byte-level comparison against a Linux transmission. The interoperability paragraph had the same problem in a different form. Those runs, including the monitor capture byte-diffing the announcement and beacon gate bit against a live Linux gate, were performed on the Rimba implementation this example derives from and were not re-run here. Same code, different repository's evidence -- now stated rather than implied. The distinction is not pedantry. Log-level evidence shows the datapath works end to end but cannot show that an emitted frame is well-formed, and a malformed field that no receiver happens to read passes every test on that page. That is precisely the proxy-ARP target-hardware-address defect fixed two commits ago, which survived every bridge test for exactly that reason -- so it is now cited there as the concrete example. Also refreshes the defaults changed in the previous commit: the proxy-ARP push interval (3 s -> 15 s, with a note that it is upkeep and a new host is pushed immediately) and the client ceiling's new maximum.
ReviewVerdict: the split is sound and the locking work is the most valuable part of it. Three commits added on top — one of them a real bug this review found. The splitThe seam is the right one. The The ARP table lock is a fix Rimba did not haveThe description undersells this. Three concurrent accessors and no guard was one problem; the announce loop reading entries live across its transmit calls was a second, and arguably worse — a slot could be evicted and refilled mid-push, teaching a host an IP/MAC pair that never coexisted. Snapshotting under the lock and sending afterwards fixes both. This has now been backported to the Rimba gate, which still had the unguarded version in shipped code. Bug found: the proxy-ARP reply was malformed
Every proxy-ARP reply sent to an AP client carried a malformed target hardware address. It resolved correctly anyway, which is why it survived every bridge test on record: lwIP's What gave it away was this file disagreeing with itself — Same defect was present in both Rimba gate apps and is fixed there too. Also pushed
Not addressed
|
The locking added in this branch was correct but used a portMUX critical
section, which disables interrupts -- and it sat on the PER-FRAME path, scanning
the table with interrupts off on every received frame.
That costs real delivery. The Rimba gate carries the same code, and an A/B there
on one night, all three arms under the same RF conditions:
critical sections 28/30 pings, then 28/30 median RTT 46 ms
no locking at all 30/30 median RTT 43 ms
mutex 30/30 median RTT 43 ms
The two 28/30 runs looked like bench variance -- they ran hours after the
baseline with the median RTT up -- so the no-locking control was built and run
rather than argued about.
A mutex is legal here because every accessor is a task: both receive callbacks
are dispatched from the one MAC event loop and the announce task is its own, so
no interrupt context ever touches the table. It costs a take/give and blocks
nothing at the driver level.
The lock is still never held across a transmit; the push snapshots under it and
sends afterwards, or the datapath would block for the whole sweep.
Follow-up to #1. Makes the mesh-gate example navigable before deciding whether its reusable parts should become a component.
app_main.cwas 1013 lines, ~75% of it infrastructure identical for any gate. Split along that seam:app_main.cgate_netif.cgate_bridge.cgate_arp.cgate.hBehaviour
No change intended. Re-verified against the unmodified Rimba fixture
test-mesh-gate-staon the same three-board rig:The original scored 13/15. That difference is within the bench's RX-overload variance and is not claimed as an improvement.
Three real changes
The proxy-ARP table is now guarded. It has three concurrent accessors — both receive tasks snoop into it, the announce task ages and reads it — and was unlocked, so two receive contexts could interleave on a slot and publish a mismatched IP/MAC pair. The proactive push transmits, so it snapshots under the lock and sends afterwards rather than holding it across the loop.
Introspection. The gate could not report what it had learned without editing source. It now logs associated clients, mesh peers, cross-bridge mappings and known gates:
A
_Static_assertcouplesEXAMPLE_AP_MAX_STAStoGATE_MAX_CLIENTS. Only the mesh→AP direction consults the client table, so a client past the end of it would associate, take a lease, send into the mesh, and silently never receive a reply. The Kconfig range already prevents this; the assert keeps them in step if someone widens that range. Verified it fires by temporarily widening the range.Incidental
Four open-coded copies of "alloc mmpkt, append, tag
VIF_AP, transmit" folded intogate_netif_tx_ap(). The netif hook keeps its blockingtx_wait_until_readybecause it runs on the lwIP task; receive-context callers must not block, and that distinction is now explicit.