Conversation
A reimaged edge comes back with a new ssh host key, and the transport passed
no host-key options at all: ssh refused the connection, the supervisor retried
forever, and the 4 KB "REMOTE HOST IDENTIFICATION HAS CHANGED" banner landed in
the log three times per attempt.
Three changes to the transport:
* `ssh_host_key_policy` ("auto" default, "strict"). "auto" adds
`-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null`, so a reimaged
device reconnects with no manual step and nothing is recorded; "strict" adds
`-o StrictHostKeyChecking=yes` and defers to ssh's own known_hosts.
ssh_extra_opts still lands last, so it can override either. BatchMode stays:
the transport is key-only.
* `SshPermanentError` for failures retrying cannot fix (auth denied, host key
rejected, no can-utils, no such interface). The class is the verdict, so the
same exception both fails startup and, later, ends a run. Every message is a
copy-paste remedy resolved for that bus and for `sys.platform` (ssh-copy-id,
or the ssh-keygen/type pipeline on Windows). Banner boilerplate is stripped
from every reported tail and capped at ~200 chars, keeping the fingerprint and
the cause; a tail is handed out once per transport, and the stderr drain no
longer logs one of its own.
* RX and TX collapse into ONE ssh session: candump on the channel's stdout, the
cansend read-loop consuming its stdin and doubling as the watchdog that used
to need a `cat`. Same trap/`kill 0` death semantics, one stderr drain, three
threads instead of four, and half the ssh sessions per bus. The remote shell
is now executed under /bin/sh, dash and bash by tests/test_ssh_remote_shell.py
rather than only pattern-matched.
Claude-Session: https://claude.ai/code/session_01PrQ9zeVKbiHKx2jvFaUayW
The supervisor treated every unhealthy link the same: log, rebuild, back off 5 s -> 60 s, forever. A reimaged edge under strict host-key checking, a revoked key, or an edge that lost can-utils therefore looped until someone read the log, and the one message naming the fix scrolled past hundreds of retries. Now the supervisor classifies the failure first. Transient causes (DNS, unreachable, refused, timeout, a link that just dropped) keep the existing backoff. An `SshPermanentError` — from the health tick or from the rebuild that follows it — propagates out of the supervision loop into the app's existing `can.exceptions.CanError` handler, which logs the actionable message once and exits(1), exactly as a bus that fails at startup does. No failed state is held. `ssh_host_key_policy` is plumbed from the bus config (default "auto") and described in the schema alongside the other ssh settings. Claude-Session: https://claude.ai/code/session_01PrQ9zeVKbiHKx2jvFaUayW
`accept-new` trusts a FIRST key and then refuses a changed one, which is the reimage case it was recommended for. Point at the policy field instead, and say which failures stop the bus versus which reconnect. Claude-Session: https://claude.ai/code/session_01PrQ9zeVKbiHKx2jvFaUayW
ssh honours the FIRST -o occurrence, so ssh_extra_opts spliced last could never override the host-key policy, BatchMode, or ConnectTimeout; they now come right after `ssh -T`. Permanent verdicts are gated on the session never having streamed a frame: the stderr ring holds the whole session, so pre-auth noise (a login-script line, a CHANGED banner ssh connects through anyway) plus a later drop read as unfixable. The `auto` policy also ignores GlobalKnownHostsFile so a stale system-wide entry cannot print that banner. Dropped the bare publickey / password needles. `SIOCGIFINDEX: No such device` is transient — after a reboot sshd can be up before can0 is configured. The remote shell's TX read loop is now the shell's own foreground body with candump signalling it on exit, so neither side can die in isolation and leave RX flowing while every TX frame vanishes. Verified executing under sh/dash/bash, each death path asserting stdout reaches EOF. The startup probe joins the stderr drain instead of spinning until the ring is non-empty (under `auto` ssh writes "Permanently added" before auth). The Windows auth remedy derives the .pub from ssh_key_path. Claude-Session: https://claude.ai/code/session_01PrQ9zeVKbiHKx2jvFaUayW
A bus takes an ordered list of databases instead of one file. Order is
precedence, matching the Rust `zelos_can` merge: an identical redefinition
of a CAN id dedupes silently, a differing one lets the later file win with
one warning, and `dbc_conflict: error` refuses the load. Each file loads
into its own cantools Database so the merge owns precedence and can report
which file every surviving message came from. An empty list is a legal
raw-frames-only bus.
The message name is part of the structural signature the merge compares:
it becomes the trace event name, so two same-shaped definitions under
different names are not interchangeable. `messages_by_id` and
`messages_by_name` are both last-wins so they cannot disagree.
Trace layout: with a shared source the codec nests every event it
registers under its bus (`<prefix>/<bus>/<event>`); without one it owns a
source named after the bus and events are unprefixed. Raw frames ride the
same source as `Frame` — the separate `{bus}_raw` source is gone — and
carry the full `zelos.can.frame.v1` shape, so `is_extended`, `is_fd` and
`is_rx` are now recorded.
`get_tx_state` keeps `bus.dbc` (first file, combined hash) for the tx
webapp and adds `bus.dbcs` plus `bus.dbc_conflicts`; `list_messages` and
`describe_message` keep `dbc_name` and add `dbcs` and a per-message
`database`.
One caveat: the Rust codec names decoded events without a bus segment, so
on zelos-socketcan / ssh-socketcan those land directly under the prefix.
The codec logs that once at startup.
Claude-Session: https://claude.ai/code/session_01PrQ9zeVKbiHKx2jvFaUayW
Settings that were never really per-bus move into one collapsed `advanced` object: `prefix` (default `CAN`), `log_raw_frames` (now default on), `receive_own_messages`, `emit_schemas_on_init`, `timestamp_mode` and `log_level`. A saved config's per-bus value still overrides the global one and a top-level `log_level` is still honoured, so old configs keep their settings; the keys are just no longer offered in the schema. With a prefix set, one shared trace source is created before the codecs and handed to all of them, so every bus reads `<prefix>/<bus>/...`. The default prefix is the action prefix, so that source is the same global one `zelos_sdk.init` would create — `init_global_source` is idempotent. The prefix is validated as a trace source name (no `/`) and startup fails with a clear message otherwise. Clearing it restores one source per bus with unprefixed events. The trace log handler moves out of `main.py`'s import into app startup, so logs follow the prefix (`<prefix>/log`) and fall back to their own `can_log` source only when it is cleared. An unnamed bus is now always named after its channel, single-bus included: `can_codec` appears only if someone types it. candump export derives the channel from the raw event's leading segment when there is one, so several buses sharing a prefix source stay distinct in the exported log. The `trace` CLI takes the same DBC list as `convert`. Claude-Session: https://claude.ai/code/session_01PrQ9zeVKbiHKx2jvFaUayW
`convert_can_trace` takes a database list (zero allowed, raw frames only) and a `--prefix`. A conversion now mirrors the live layout: the source is the prefix with the input file's own sanitized stem as the raw-frame segment (`<prefix>/<stem>/Frame`), or that stem as the source name when the prefix is cleared. Raw frames are on by default so a DBC-less conversion still produces something. `timestamp_mode="absolute"` is unchanged. This supersedes the `can_codec` source-name pin in `_make_decoder`: the name is now the prefix, which is configurable by design. The two convert actions take the list too. The standalone one's `database_file` field still accepts a single path, and with it empty it picks up the whole list configured for the first bus, folding a legacy `database_file` the same way the bus config does. Both results keep `database_file` (the first) and add `database_files`. Claude-Session: https://claude.ai/code/session_01PrQ9zeVKbiHKx2jvFaUayW
The parity test is the gate: the extension's cantools merge and `zelos_can.CanDecoder.message_keys()` must agree on the `(frame_id, is_extended, event_name)` set for the same file order, over a 3-file fixture (disjoint ids, one identical redefinition, one conflicting one) and over test.dbc's own intra-file collision on id 800. Plus: the merge's dedupe/conflict/error policy and the warning text, the `advanced` normalisation and prefix validation, the `get_tx_state` and `list_messages` shapes, a zero-DBC bus, the converter's naming with the prefix set and cleared, and a real python-can virtual bus asserting raw frames land as `<bus>/Frame` with `is_rx` set and decoded ones as `<bus>/<id>_<Msg>`. Existing tests updated for the renamed config key, the merged message list, and the retired `_raw` source. Claude-Session: https://claude.ai/code/session_01PrQ9zeVKbiHKx2jvFaUayW
Per-bus and Advanced settings as two tables, the resulting trace layout, and the one-line note that clearing Prefix keeps the previous per-bus layout. The old prose bullet lists go. Claude-Session: https://claude.ai/code/session_01PrQ9zeVKbiHKx2jvFaUayW
The `_reported_tail` self-muting flag only existed because a failed rebuild left `_transport` pointing at the torn-down transport, so every failed tick re-read the dead ring. Null `_transport` after teardown instead and let `stderr_tail()` be a plain read; the rebuild's own warning names the new cause. `-o LogLevel=ERROR` under the `auto` host-key policy suppresses ssh's per-connect "Permanently added" line at the source, so the needle that filtered it back out is gone. Verified against a real sshd that `Permission denied (publickey)` and `Host key verification failed.` still print, so classification is unaffected. Drop `_eof`: every path that set it also returned from its thread, and `healthy` already checks both threads' liveness. A malformed interface name is a plain `CanInitializationError` like its `_candump.py` siblings — `SshPermanentError` now means only "stderr said so". Default the classifier to transient and set permanent in the three arms that are. Tests: keep the needle table in the pure classifier test only (one row covers the streamed veto), reduce the startup probe to one row, drop the remote-command shape assertions the executed shell harness owns, call `_build_argv` directly with no proc or threads, and share one `wait_until` from `tests/conftest.py`. Claude-Session: https://claude.ai/code/session_01PrQ9zeVKbiHKx2jvFaUayW
The Rust codec (`zelos-socketcan`, `ssh-socketcan`) and the converter now take `event_prefix`, so decoded events land at `<prefix>/<bus>/<id>_<Msg>` exactly like the python-can path. Two Rust-path buses on one shared source no longer share each decoded event, and the interim warning about that goes. The converter passes the input file's stem, so a conversion reads `<prefix>/<stem>/<id>_<Msg>`; both surfaces pass None when the prefix is cleared, keeping the per-bus layout. `cli/trace.py` stays prefix-less: pure CLI mode traces one bus and owns its source. Claude-Session: https://claude.ai/code/session_01PrQ9zeVKbiHKx2jvFaUayW
The DBC merge defers to `zelos_can.CanDecoder`: a bus-less parse names the surviving definition of each message id, and the extension pairs each back to its cantools Message. The hand-mirrored signature comparison is gone; it already disagreed with Rust in both directions, which made `dbc_conflicts` and `dbc_conflict: error` path-dependent. `trace_layout(prefix, bus)` is the one naming rule, used by the codec, the converter, app mode and the `trace` CLI (which gains `--prefix` and a channel-derived bus). `zelos_sdk.sanitize_name` is the one name allow-list, for the prefix, an explicit bus name, and channel-derived names. The trace source, the log handler and a `--file` recording all come up before the codecs, so the merge summary and conflict warnings reach `<prefix>/log`. Configuration mistakes (bad policy, bad name, missing DBC, Rust load error) exit with one line like a `CanError` already did. Per-file DBC digests are computed once instead of on every 1 Hz poll. The per-bus schema block is declared once instead of in seven branches, and `prefix` carries `ui:emptyValue` so it can be cleared from the app. Claude-Session: https://claude.ai/code/session_01PrQ9zeVKbiHKx2jvFaUayW
Integration branch for the CAN end-to-end pass: several DBCs per bus with a configurable trace prefix, plus the ssh host-key policy and its permanent- failure class. Conflicts, all additive: - tests/conftest.py: one file with both helpers (`wait_until`, `trace_event_paths`). - tests/test_ssh_codec_integration.py: the full-config schema case takes the host-key policy and the list-shaped `database_files`. - README.md: the host-key policy settings sit alongside the trace layout table; the shared per-bus/Advanced settings replace the retired per-bus list. - cli/app.py: the ExitStack structure keeps its staged excepts and adopts the mid-run wording, since SshPermanentError is a CanError subclass and the `asyncio.run` except already covers it. Claude-Session: https://claude.ai/code/session_01PrQ9zeVKbiHKx2jvFaUayW
`self.messages` came back in the Rust decoder's `(frame_id, is_extended)` key order, so the last-wins `messages_by_name` pass resolved a name defined at two different ids to the HIGHER id rather than the later file. Send / describe by name then reached into the earlier DBC while the same name by id honoured the list's precedence. Rebuild the list in file order: walk the files in list order, each in definition order, keep the survivors the decoder names, and let a repeated id overwrite. Now `messages_by_id` and `messages_by_name` are both later-file-wins and `list_messages` lists in file order. Claude-Session: https://claude.ai/code/session_01PrQ9zeVKbiHKx2jvFaUayW
`kill 0` in the remote command's trap signals the CALLER's process group, and not every server gives the command one of its own. Under Tailscale SSH (`tailscaled be-child ssh --cmd=...`) every session of every user runs in the tailscaled daemon's group, so one bus stopping reaped the other buses' candumps on the same edge and the operator's unrelated sessions with them. OpenSSH's sshd `setsid`s each command, which is why only the field saw it. The session now moves to a group of its own before the trap is armed, and only when it does not already lead one — which is exactly the case where util-linux and busybox `setsid` exec in place instead of forking, so the server keeps one child holding one stdin/stdout and no parent exits early and closes the channel. Edges without `setsid` fall back to today's behaviour. Every death path is unchanged; `$$` is still the shell that runs the loop. Covered by a case per shell that runs the command in a stand-in caller group: its candump must leave that group, and a sibling already in it must outlive the session. Both fail on the old command under sh, dash, and bash. Claude-Session: https://claude.ai/code/session_01PrQ9zeVKbiHKx2jvFaUayW
Brings in the field fix for the remote session's process group: `kill 0` in the trap no longer reaps the caller's group, so two buses on one edge stop reaping each other's candump on every rebuild. No conflicts. Claude-Session: https://claude.ai/code/session_01PrQ9zeVKbiHKx2jvFaUayW
Two config-form hooks, both standalone so they answer before the extension has
ever started, and both pure sysfs (no privileges, no python-can):
* `CAN/list_interfaces` feeds the `action-choices` picker on a socketcan /
zelos-socketcan bus's Channel: every `/sys/class/net/*` netdev of type 280
(ARPHRD_CAN), labelled with its operstate and driver. A typed name still
works, and macOS/Windows answer with an empty list rather than an error.
* `CAN/auto_config` answers the form's Auto-configure button with one
zelos-socketcan bus per interface, hardware before vcan. It returns only
`buses`, so Advanced survives the replace, and it never guesses an ssh bus.
And one classification change: `no such device` on a bus that has never made
contact is a wrong `remote_channel`, not a booting edge, so it stops with the
fix instead of retrying forever. After a session that connected, the same
failure stays transient — a rebooting edge or a re-enumerating adapter
reconnects. The codec remembers ("ever connected"), so a transport rebuilt
after a working session inherits it.
Claude-Session: https://claude.ai/code/session_01PrQ9zeVKbiHKx2jvFaUayW
`CAN/list_interfaces` returned `{value, label}` with the name baked into
the label (`can0 (up, gs_usb)`). The app's picker renders `{value, detail}`
as the value plus a dim right-aligned detail, so the name was printed twice.
Adds a guard that the schema's `ui:options.autoconfig` and the `channel`
fields' `ui:options.action` name actions in the standalone inventory, so a
rename cannot silently break the form.
Claude-Session: https://claude.ai/code/session_01PrQ9zeVKbiHKx2jvFaUayW
…d zelos.can.message.v1 `zelos_can` now keeps every definition of a frame id whose names differ, so the extension's indexes are per definition: `self.messages` holds one cantools `Message` per survivor (selected by `(frame_id, is_extended, name)` from the last file defining it) and `messages_by_id` maps an id to the list of its definitions in definition order. The python-can decode path walks that list, so a frame lands in `<bus>/0302_A` and `<bus>/0302_B` alike, with its own mux subtables, schema cache slot and error blocklist entry per definition. `messages_by_name` stays last-wins, so TX by name is unchanged. `dbc_conflicts` keeps its shape and now means only same id AND same name laid out two ways (later file wins, `error` still refuses the load); `bus.dbc_overlaps` is new and reports the ids carrying several names. Conflicts still warn, overlaps log once at INFO, and the merge summary counts both. Every decoded event the python-can path registers, base and mux, carries the `zelos.can.message.v1` event type the Rust codec already stamps. Claude-Session: https://claude.ai/code/session_01PrQ9zeVKbiHKx2jvFaUayW
…p test; drop the auto-config note The edge's own adapter clock is the better stamp, so `candump` now runs with `-H` where the edge has it: support is detected in the session shell, since `-H` only arrived in can-utils 2020.11 and an older candump answers an unknown option with its usage and a non-zero exit. An interface with no hardware clock (vcan, some slcan) prints `(0000000000.000000)`, which the parser reads as no timestamp at all so the codec stamps wall clock. New per-bus setting `ssh_hw_timestamps`, default on. The executed-shell tests spawn real shells and lost their 4 s races on a loaded box; every wait there polls a process fact, so the bound is raised to 15 s and the stand-ins outlast it. `CAN/auto_config` no longer returns a success `message`: the app's button shows only an error, and the Channel picker already labels a down interface `down`. Claude-Session: https://claude.ai/code/session_01PrQ9zeVKbiHKx2jvFaUayW
Supervisor and reconnect lines now carry the `[<bus>]` prefix the
start/stop lines already use, so a multi-bus log says which bus stalled.
Encoding a DBC message without all its signals surfaced cantools' bare
KeyError ("'state_request'"); the one encode site now raises
ValueError naming every missing signal, and for a multiplexed message
only the selected variant's plus the base ones.
Claude-Session: https://claude.ai/code/session_01PrQ9zeVKbiHKx2jvFaUayW
…, exact needles, port-qualified remedy, unmatched merge keys The remote session ran `cansend IFACE "$f" >/dev/null 2>&1`, so a missing cansend or a failing write (ENOBUFS on a saturated bus) dropped every transmit while the bus reported healthy. cansend must now exist — checked before the trap is armed, exit 127 with the needle the classifier calls permanent — and a cansend that runs keeps its stderr, which the local ring counts as `tx_errors` (rate-limited warning, folded across transport rebuilds). A non-zero cansend still does NOT end the session: a transient write must never tear down a live bus. `list_messages` / `describe_message` listed every surviving definition while TX by name resolves last-wins, so the webapp's first row for a name could transmit a different frame. Both now describe one entry per NAME — the TX-addressable definition — and carry `shadowed`, the same-name definitions at other ids that a transmit will not reach. RX decoding and `messages_by_id` are unchanged. `_ssh_ever_connected` was set right after every transport construction, but the startup probe passes an idle session after 3 s, so a wrong `remote_channel` on a slow connect turned `no such device` transient forever. It is now set only from a transport that actually streamed a frame, read off the outgoing transport at rebuild. Three narrower fixes: the can-utils verdict matches only candump/cansend "not found" (a login profile's unrelated missing command no longer makes a later transient drop permanent); the host-key remedy is port-qualified (`ssh-keygen -R '[host]:port'`, `ssh -p port`) like the auth one; and a definition the two parsers name differently, which pairs with nothing and vanishes from TX and describe, is now logged from both sides with id, name and file instead of silently skipped. Claude-Session: https://claude.ai/code/session_01PrQ9zeVKbiHKx2jvFaUayW
…tart
`_validate_name` moves to codec.py as `name_error`, returning the reason a
name is unusable instead of exiting, so the CLIs report it their own way:
`trace` and `convert` (click and the argparse `converter.main`) now reject a
bad `--prefix` with the same message the app mode logs, rather than failing
deep inside `TraceSource`. An empty value is legal everywhere — a cleared
prefix is a supported option and `sanitize_name("")` returns "unnamed".
A bus explicitly named `can_log` is rejected: that is the extension's own log
source when the prefix is cleared, and its event segment when set.
`_start_ssh` stops and clears the Rust codec + ExternalBus if the transport
fails to construct; a bus whose `start()` raised is never handed to `stop()`,
so the half-built native state leaked.
`test_rx_frames_decode_through_codec` waits on `messages_decoded`, the later
of the two independent counters, before reading the pair.
Claude-Session: https://claude.ai/code/session_01PrQ9zeVKbiHKx2jvFaUayW
Every surviving DBC definition now carries a `key` — `{id hex}_{Name}`, the
name of its trace event, taken from the decoder rather than re-derived.
`list_messages` returns one entry per key in file order, so a name defined at
two ids gets both rows instead of one row and a `shadowed` list.
The transmit and describe actions resolve a key, or a name only one definition
carries; a name at several ids raises and names the keys instead of silently
picking the last file's. Periodic slots are keyed per definition too.
Claude-Session: https://claude.ai/code/session_01PrQ9zeVKbiHKx2jvFaUayW
…n and keep the later file With coexisting definitions of one frame id, a conflict is only the same id under the same name laid out two ways. That always keeps the later file and logs a warning, and the pair is listed under `dbc_conflicts` on `get_tx_state` — there is no policy left for a user to pick. Claude-Session: https://claude.ai/code/session_01PrQ9zeVKbiHKx2jvFaUayW
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Based on
standalone-actions(#28), which it needs for theCAN/action prefix.ssh-socketcan
ssh_host_key_policy:auto(default,StrictHostKeyChecking=no+/dev/nullknown_hosts,LogLevel=ERROR) so a reimaged edge reconnects with no manual step;strictuses ssh's own checking and fails cleanly.ssh_extra_optsis spliced first because ssh honours the first-o.ssh-copy-id(or Windows) command for that bus. Unrecoverable causes (auth, host key under strict, missing can-utils) log once and exit;no such deviceand network causes retry. Permanent verdicts are gated on the session never having streamed, so pre-auth noise cannot misclassify a later drop.kill 0reaped sibling buses on the same edge (observed on a real edge, fixed withsetsid, sibling survival is an executed test on sh/dash/bash).DBCs and trace layout
database_files(ordered list, file-picker) per bus,dbc_conflict: warn|error; later file wins a differing redefinition with one warning; the merge is decided byzelos_canso RX decode and TX encode agree by construction.get_tx_state/list_messages/describe_messagegaindbcs,dbc_conflicts, and per-messagedatabase;bus.dbcanddbc_namestay for the TX webapp.prefix(defaultCAN, clearable): every bus shares one source, pathsCAN/<bus>/<id>_<Msg>, raw framesCAN/<bus>/Frametypedzelos.can.frame.v1, logsCAN/log. Cleared: one source per bus, today's layout. Bus names are always channel-derived.log_raw_frames(default on),receive_own_messages,emit_schemas_on_init,timestamp_mode,log_levelmove under Advanced.traceCLI take N DBCs and--prefix; the input file stem is the bus segment.Not mergeable until
zelos-sdk(typedschemas.CanFrame,TraceLoggingHandler(source)) andzelos-can(database_filelist,dbc_conflict,raw_event_name,event_prefix,databases()/dbc_conflicts()/message_keys()) releases from zeloscloud/src#1487 exist and the floors inpyproject.tomlare raised. The app side ofui:emptyValue(clearing the prefix from the settings dialog) ships in the same monorepo PR.Verification
329 tests (2 Linux-only skips); executed remote-shell harness 15/15 on sh, dash, bash; real-sshd runs for auth denial, changed key, missing can-utils. End-to-end on two real edges (bash and dash
/bin/sh, Tailscale SSH): one session per bus, exactCAN/<bus>/...paths live and after TRZ2 seal, typedFramewith nullis_rx, merge report on the wire, TX by raw and by name echoed through the session, transient recovery, strict-policy clean exit, prefix-cleared layout, converter in both modes, and the sibling-bus survival on a shared edge.https://claude.ai/code/session_01PrQ9zeVKbiHKx2jvFaUayW