Skip to content

Connection monitor logs every transient ICMP probe timeout at error level → high-volume Sentry noise #6307

Description

@rachyandco

Software (please complete the following information):

  • Operating system: iOS 26.6 (iPhone14,2), cellular (LTE / 5G NSA, LQM moderate–minimal)
  • NymVPN version: nym-vpn-lib 2026.12.1-beta.3 (2af42a5); same code on develop

Describe the bug
The connection monitor logs every single in-tunnel probe timeout at error level, so each one becomes a Sentry error event even though the monitor itself treats isolated timeouts as tolerable and recovers on the next probe.

Sentry nym-vpnd issue 7008963337:

Error: failed to send icmp packet
Caused by: Request timeout for icmp_seq 180

Logger: nym_connection_monitor::connection_monitor (crates/nym-connection-monitor/src/connection_monitor.rs:265).

Timeline from the event breadcrumbs:

  • Session start 15:44:49 on cellular, path flapping between pdp_ip0[lte] and pdp_ip0[endc_sub6], LQM moderate/minimal, estimated upload 65536 Bps.
  • Probe periodicity for two-hop ICMP is 10 s, so icmp_seq 180 ≈ 30 min uptime, matching the 16:16:28 event.
  • Tunnel monitor logged Tunnel connection is failing (retry: 1) at 15:54:57, 15:57:16 and 16:02:23. Each recovered on the next probe (never retry: 2). The reported event is the fourth such transient timeout.
  • The tunnel never declared failure and never reconnected. Behaviour was correct for a weak cellular link.

Cause of the severity: in the Err branch of the probe loop, trace_err_chain!(err) expands to tracing::error! before the monitor checks retry > max_retry_count (3 consecutive failures in the monitoring phase). Intermittent failures therefore emit an error-level line and a Sentry event while being classified as IntermittentFailure. The tunnel monitor already logs that case at info and only warns/acts on Failed.

Side observation: dns64: Resolved [ip]:51822 to self appears every few seconds at info level. It comes from the tunnel's own PathMonitor re-resolving the entry endpoint on every cellular path update and only updating the peer if the result changed. Harmless, but on cellular it drowns out useful log lines.

To Reproduce
Steps to reproduce the behavior:

  1. Connect on iOS over a weak cellular link (or use Network Link Conditioner with packet loss).
  2. Stay connected for several minutes.
  3. Any single ICMP probe timeout logs Error: failed to send icmp packet and produces a Sentry error, even though the tunnel keeps working and the monitor recovers on the next probe.

Expected behavior

  • A probe failure within the retry budget should be logged at warn or debug, with the retry count.
  • Only the terminal case (status becomes Failed) should log at error level.
  • DNS64 re-resolution with an unchanged result should log at debug.

Screenshots
N/A. Log line and breadcrumbs above.

Additional context
Recommended changes:

  1. In connection_monitor.rs, replace trace_err_chain!(err) in the Err branch with a level chosen after the retry check: warn/debug for IntermittentFailure { retry }, error for Failed.
  2. Include the retry count in the message so terminal failures remain easy to correlate with tunnel-monitor logs.
  3. In connected_tunnel.rs (iOS path monitor loop) / dns64.rs, downgrade Resolved {} to self to debug when the resolved endpoint equals the previous one.

This is likely one of the highest-volume issues in the nym-vpnd Sentry project; the fix is a log-level change, no behavioural change to the tunnel.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions