V2.2.1

.gitignore

@@ -26,6 +26,7 @@ RecycleBin.ini
 test_bintool
 *.bak
 *.log
+*.pcap
 
 # Other
 .vscode
@@ -36,3 +37,4 @@ test_bintool
 /docs/SUBMODULE_GUIDE.md
 /shm_*.sh
 wlan.h
+/TODO_old.md

.vscode/settings.json

@@ -23,5 +23,9 @@
     "cmake.configureOnOpen": false,
     "rust-analyzer.linkedProjects": [
         "${workspaceFolder}/tools/xcpclient/Cargo.toml"
-    ]
+    ],
+    "[rust]": {
+        "editor.defaultFormatter": "rust-lang.rust-analyzer",
+        "editor.formatOnSave": true
+    }
 }

CHANGELOG.md

@@ -2,14 +2,90 @@
 
 All notable changes to XCPlite are documented in this file.
 
+## [V2.2.1] 
+
+- xcpclient related changes:
+    - The call frame information (`.debug_frame`, `.eh_frame`) of 32 bit targets is parsed with the address size of the ELF file. It was parsed with the 8 byte address size of the host, which produced unusable frame descriptions for ARM and Xtensa targets and a CFA offset of 0 by accident. For ARM Cortex-M targets (`freertos_stm32_demo`) the stack frame relative addresses of local variables change, please verify on the target
+    - Xtensa (ESP32): the CFA offset is 0 by design, the trigger macros pass the CFA itself (`__builtin_dwarf_cfa()`), verified on ESP32-S3 with `freertos_esp32_demo`
+    - Location expressions of variables which can not be evaluated (register locations, entry values, optimized code in other compilation units) and variables without a name are reported at debug level instead of error/warning level
+    - New option `--default-event <id>` (also `default_event` in the TOML config file): event used for the DAQ measurement of variables without a fixed event, such as global variables in an A2L file generated from an ELF file. When an A2L file is created from an ELF file, the event is assigned to global variables and to static variables in functions without an event trigger as their default event
+- Split `platform.c/.h` into `platform.c/.h` (threads, mutex, clock, sleep, memory, atomics) and `sockets.c/.h` (socket abstraction for all platforms). `sockets.h` includes `platform.h`; files that use both include both explicitly (IWYU).
+- New raw Ethernet transport `OPTION_ENABLE_UDP_RAW`: XCP on UDP/IPv4 implemented inside xcplib on top of a thin raw Ethernet HAL, for targets without a TCP/IP stack. See `docs/SOCKET_RAW.md`.
+    - New build configuration `raw` (`src/xcplib_raw_cfg.h`, `build-raw/`) with the new example `udp_raw_demo` and the unit test `socket_raw_test` (Linux only)
+    - `src/socket_raw.c` — UDP/IPv4 layer, answer-only ARP, ICMP Echo responder, receive filter with an absolute-deadline loop
+    - `src/socket_raw_hal.h` — raw Ethernet HAL interface, `src/socket_raw_hal_linux.c` — AF_PACKET backend (requires `CAP_NET_RAW`)
+    - `test/test_socket_raw.sh` — isolated veth/netns test setup with ARP, ping and XCP CONNECT checks
+    - Mutually exclusive with `OPTION_ENABLE_UDP`/`OPTION_ENABLE_TCP` and requires `OPTION_QUEUE_32`; both enforced by `#error` in `xcptl_cfg.h`, together with the SHM, multicast and MTU restrictions
+    - The `rtos` configuration keeps using the lwIP socket API - the raw transport is a separate configuration, not an override
+- Optional zero copy transmit for the raw Ethernet transport (`OPTION_UDP_RAW_ZERO_COPY`, on by default): headroom is reserved in front of every transmit queue3/queue32 segment so the Ethernet/IPv4/UDP header is written in place instead of copying the payload into a frame buffer.
+    - New generic queue concept `QUEUE_SEGMENT_HEADER_SIZE` in `queue.h` - reserved once per *segment*, as opposed to the existing per-*message* `QUEUE_ENTRY_USER_HEADER_SIZE`
+    - `queue32.c`/`queue32m.c` gain one guarded field; with the option off the queue entry layout is byte identical to before
+    - Command responses keep the copying path, they are built on the stack and are not hot
+- New `OPTION_UDP_RAW_HAL_EXTERNAL`: an application can supply its own raw Ethernet HAL from outside the library. xcplib then selects no backend and the `eth_hal_*` symbols stay undefined in `libxcplite` until the application links its own implementation. Intended for backends which do not belong in the library, for example ASAM CMP for testing XCP tools through capture modules, or a vendor specific interface such as XLAPI.
+    - Also lifts the Linux-only restriction of the raw transport: without a built in backend there is nothing platform specific left in it
+    - Depends on `libxcplite` being a **static** library, so the undefined `eth_hal_*` resolve at application link time with no indirection in the transmit path. A shared build would resolve them internally and the override would silently not take
+    - `socket_raw_hal_linux.c` is excluded by the same option, so the built in AF_PACKET backend is not linked in
+- `OPTION_MTU` in the `raw` configuration set to **1420**, below the 1500 of a standard Ethernet link, to leave headroom for a HAL backend which encapsulates the frame before putting it on the wire. `XCPTL_MAX_SEGMENT_SIZE` becomes 1392, the largest Ethernet frame 1434 and its IP packet 1420 bytes, i.e. `OPTION_MTU` exactly. An encapsulating backend then fits inside a 1500 byte path: `cmp_demo` wraps that 1434 byte frame in a 34 byte CMP envelope and 28 bytes of outer IPv4/UDP headers, reaching 1496 bytes. At the full link MTU a segment is 1472 bytes and the frame 1514, filling the path on its own, so an encapsulating backend has nothing left and the raw transport, which does not fragment, can only refuse the frame.
+    - The previous value was 1504, a leftover of the pre-V2.1.11 convention where `OPTION_MTU` was the link MTU rounded up to a multiple of 8. Since "Fix UDP segment size calculation from MTU" (V2.1.11) `XCPTL_MAX_SEGMENT_SIZE` is `(OPTION_MTU - 28) & ~7`, so `OPTION_MTU` is the true link MTU and 1504 gained nothing over 1500
+    - **Behaviour change:** `udp_raw_demo` now sends segments of 1392 instead of 1472 bytes
+- New example `cmp_demo`: an emulated **ASAM CMP** (Capture Module Protocol) capture module carrying XCP, for testing XCP tools which communicate through capture modules. Implemented against ASAM CMP Protocol Layer Specification V1.1.0. Nothing CMP specific is in `libxcplite` — the whole protocol lives in the example, behind the `eth_hal_*` interface.
+    - A standalone CMake project consuming an **installed** xcplite via `find_package(xcplite)`, not built from the root `CMakeLists.txt`
+    - CMP over UDP (6.4.2). CMP over raw Ethernet is a possible later step
+    - `src/cmp.c` — the envelope codec, verified byte exact against the sample PCAPNG files shipped with the specification
+    - Injection uses `TX_DATA_MSG` (message type `0x04`), which **CMP 1.1 introduced** and 1.0 does not have. Without it a capture module could only carry DAQ and XCP could never `CONNECT`. Note that a CMP 1.0 dissector reports it as an unknown message type
+    - `src/cmp_rest.c` — the mandatory REST interface (12.3), read only. Its `Transmitter` object is how a Data Sink detects that transmission is supported (7.2.2); without it a tool may never inject
+    - `src/cmp_discovery.c` — `CMP_CM_DISCOVERY` responder (12.1.1) on `239.255.0.0:5556`, serviced by the REST thread. Section 12 requires only one of three discovery approaches and permits static configuration with none, so this is optional
+    - `test/cmp_codec_test.c`, `test/fake_sink.py` (a minimal Data Sink), `test/discovery_probe.py`, `test/test_local.sh` (loopback) and `test.sh` (on target)
+- New `docs/XCP_DISCOVERY.md`: records what xcplib has for XCP's own multicast discovery today — `GET_SERVER_ID_EXTENDED` implemented, `GET_SERVER_ID` stubbed, both behind `XCPTL_ENABLE_MULTICAST`, which no shipped configuration enables and which the raw transport excludes — and the options for it. Nothing is decided or changed. Also records that `XCPTL_MULTICAST_PORT` is 5557 while ASAM CMP 12.1 states XCP uses 5556, which needs checking against ASAM MCD-1 XCP.
+- IPv4 fragmentation is now prevented on the socket transport: `socketOpen` sets the DF bit on UDP sockets (`IP_MTU_DISCOVER`/`IP_PMTUDISC_DO` on Linux, `IP_DONTFRAG` on macOS/BSD, `IP_DONTFRAGMENT` on Windows, no-op on lwIP). Fragmentation is harmful for DAQ - one lost fragment loses the whole datagram and reassembly adds jitter - and an `OPTION_MTU` larger than the path MTU previously degraded measurement silently. Oversized segments now fail with `EMSGSIZE` and a message naming the segment size and the `OPTION_MTU` to reduce. **Behaviour change:** a setup that relied on fragmentation will now report an error instead of silently fragmenting.
+- The lwIP socket path now reports a segment that does not fit the link MTU. lwIP sets no DF option - it has no `IP_DONTFRAG` - so unlike Linux, macOS/BSD, QNX and Windows it does not refuse an oversized datagram: it fragments or drops it according to its own `IP_FRAG` build setting, silently either way. That made lwIP the one transport where an `OPTION_MTU` larger than the link MTU degraded measurement with no diagnostic at all. `socketSendTo` now compares the segment plus 28 bytes of IPv4/UDP headers against `netif_default->mtu` and warns once, with the same wording as the socket path. It still sends: this is a diagnostic, not a guard, since refusing a datagram lwIP may well deliver would change behaviour. `netif_default` is not necessarily the interface routing to the destination on a multi-homed target, so a false report is possible there.
+- `create_thread()` on Windows now returns 0 on success, matching `pthread_create()`, instead of evaluating to the thread `HANDLE` where non-NULL meant success. The two conventions were inverted, so `if (create_thread(...) != 0)` read as an error on POSIX and as success on Windows. A *portable* check is still not possible - both FreeRTOS variants are `do {} while (0)` statements which assert - and `platform.h` now documents the contract per platform. No caller in the repository tests the result, so nothing changes in behaviour.
+- Version aligned to 2.2.1 in `CMakeLists.txt` (what `find_package(xcplite)` reports) and in `OPTION_VERSION_MAJOR`/`_MINOR`/`_PATCH` in `xcplib_cfg.h` (what `XCP_DRIVER_VERSION` reports to the XCP client). The two had drifted apart at 2.1.2 and 2.1.10.
+- Fixed `PLATFORM_32_BIT` typo in `xcplib_cfg.h`, which prevented the automatic selection of `OPTION_QUEUE_32` on 32 bit platforms
+- Fixed missing `#include <errno.h>` in `shm.c`, which broke the `shm` configuration build on macOS
+- Window build workaround to disable section based event pre-registration
+- Fixed older XCP protocol layer versions 0x0100 to 0x0102
+
+## [V2.1.14]
+
+- Support static allocation and improve FreeRTOS robustness (#124)
+    * Support static allocation for FreeRTOS tasks and mutexes
+    * Use static storage when configSUPPORT_STATIC_ALLOCATION is enabled
+    * Preserve dynamic allocation and recursive mutex support
+    * Update FreeRTOS documentation and remove obsolete queue TODOs 
+    * Fix FreeRTOS delay rounding and overflow
+    * Avoid unaligned IPv4 address access in FreeRTOS/lwIP
+    * Fix synchronization of FreeRTOS queue state
+    * Correct FreeRTOS queue and lwIP documentation
+    * Fix FreeRTOS emulator clock test build
+
+
+## [V2.1.12]
+
+- xcpclient related changes (xcpclient version 3.0.10):
+    - Improved variable registration for global and static variables
+    - Global variables and local static variables in functions without event trigger are registered, but not associated to any specific default DAQ event
+    - Unique typedef names for struct/class types with the same name in different namespaces, classes or functions (`motor_control.Input`, `valve_control.Input`). Previously all variables of such types referenced the first registered typedef and were shown with the wrong type.
+    - Global variables with the same name in different namespaces are registered with namespace qualified names (`motor_control.input`, `valve_control.input`) instead of dropping all but the first one
+    - A struct type which is used for measurement and for calibration variables gets separate typedefs
+    - The GCC declaration and definition entries of a namespace scope variable are merged, they no longer produce duplicate instance errors
+    - Variables with internal linkage in a namespace and without DWARF location (e.g. metadata markers `XCP_COMMENT(motor_control__input, ...)` inside a namespace) are resolved by their mangled symbol name
+    - Metadata annotations placed in the same namespace or function as the variable no longer need a scope prefix (`XCP_COMMENT(input, ...)` in namespace `motor_control` annotates `motor_control.input`), explicit prefixes keep working. Metadata for the typedef fields of namespace qualified instances.
+    - Local variables without a DWARF location (optimized away) are no longer resolved to the address of a global variable with the same name, only symbols with local binding are considered for them
+    - no_a2l_demo, no_a2l_demo_cpp: more demo cases, no_a2l_demo_cpp demonstrates types and variables with the same name in different namespaces
+- Documentation: new `docs/OFFLINE_A2L.md` for the offline A2L generation with xcpclient (workflow, naming rules, supported types, diagnostics), `docs/TECHNICAL.md` keeps the library topics and the instrumentation marker contract, xcpclient and no_a2l_demo READMEs updated
+
+
+
 ## [V2.1.11]
 
-* Allow custom memory attributes for queue32m
-* Avoid mutex allocation when queue32m uses critical sections
-* Support optional recursive FreeRTOS mutexes
-* Fix typos in FreeRTOS documentation and comments
-* Fix UDP segment size calculation from MTU
-* Make FreeRTOS queue segment count configurable
+- FreeRTOS related changes:
+    * Allow custom memory attributes for queue32m
+    * Avoid mutex allocation when queue32m uses critical sections
+    * Support optional recursive FreeRTOS mutexes
+    * Fix typos in FreeRTOS documentation and comments
+    * Fix UDP segment size calculation from MTU
+    * Make FreeRTOS queue segment count configurable
 
 
 ## [V2.1.10]
@@ -139,7 +215,7 @@ All notable changes to XCPlite are documented in this file.
 ```c
 void XcpInit(const char *name, const char *epk, uint8_t mode);
 ```
-- The return value contract of `socketRecv` and `socketRecvFrom` has changed. Only code that uses these functions directly (i.e. code that includes `platform.h` is affected)
+- The return value contract of `socketRecv` and `socketRecvFrom` has changed. Only code that uses these functions directly (i.e. code that includes `socket.h` is affected)
 
 ### Experimental
 

CLAUDE.md

@@ -89,7 +89,10 @@ src/queue*.c                    Lock-free/mutex transmit queue implementations (
 src/cal.c/.h                    Calibration segment RCU implementation (page switching, locks)
 src/a2l.c, src/a2l_writer.c     Runtime A2L file generation
 src/persistence.c/.h            Binary (.bin) parameter/event persistence across restarts
-src/platform.c/.h               OS abstraction (threads, sockets, clock, atomics) — Linux/macOS/QNX/Windows/FreeRTOS
+src/platform.c/.h               OS abstraction (threads, clock, mutex, atomics) — Linux/macOS/QNX/Windows/FreeRTOS
+src/sockets.c/.h                Socket abstraction over the OS socket API (all standard platforms)
+src/socket_raw.c                Raw-Ethernet UDP/IPv4 transport (OPTION_ENABLE_UDP_RAW), see docs/SOCKET_RAW.md
+src/socket_raw_hal.h            Raw Ethernet HAL interface; socket_raw_hal_linux.c is the AF_PACKET backend
 src/util.c/.h                   Shared helpers
 ```
 
@@ -119,7 +122,7 @@ XCPlite encodes *where* a measured/calibrated variable lives (global, stack, hea
 - `xcp_evts` section — `tXcpEventDescriptor` constants emitted by `DaqCreateEvent`/`DaqCreateAndTriggerEvent`
 - `xcp_cals` section — `tXcpCalSegDescriptor` constants emitted by `CalSegDecl`+`CalSegCreate`
 
-and DWARF scope anchors named `trg__<mode-letters>__<event-name>` (e.g. `trg__AAS__foo`, letter position = address-extension value [0..]: `A`=absolute, `C`=cal-segment-relative, `S`=stack-relative, `D`=dynamic/heap) emitted by the trigger macros, to reconstruct addressing without any runtime A2L calls. Full details, including what changes if you modify the trigger macros, are in `docs/TECHNICAL.md`.
+and DWARF scope anchors named `trg__<mode-letters>__<event-name>` (e.g. `trg__AAS__foo`, letter position = address-extension value [0..]: `A`=absolute, `C`=cal-segment-relative, `S`=stack-relative, `D`=dynamic/heap) emitted by the trigger macros, to reconstruct addressing without any runtime A2L calls. The marker contract (sections, marker names, trigger anchor naming) is in `docs/TECHNICAL.md`, the tool side (workflow, naming rules, symbol resolution, supported types) in `docs/OFFLINE_A2L.md`.
 
 ### Shared-memory (SHM) multi-application mode (`docs/SHM.md`)