Skip to content

Keep the connector's watch alive across sleep and an expired Claude login - #958

Open
robzolkos wants to merge 2 commits into
mainfrom
fix/connect-watch-survives-sleep
Open

robzolkos wants to merge 2 commits into
mainfrom
fix/connect-watch-survives-sleep

Conversation

@robzolkos

@robzolkos robzolkos commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

What people saw

On a laptop with default sleep settings, an agent run through basecamp connect acknowledges a request with 👀 and then never answers. The same happens after Claude Code's login expires, even once the person logs back in.

Why

The connector is fine. It keeps receiving events and boosting them. The Claude Code session watching its output is what stops.

  • The skill told the session to arm a monitor with persistent: true. The Monitor tool has no such option. Every monitor is killed after 30 minutes at most, and only a turn of the session arms it again.
  • When that turn can't reach Claude (a laptop just woken, or a login that expired), Claude Code retries for about four minutes and gives up. Nothing re-arms the watch, so every later request is boosted by the connector and never reaches the session.
  • A request delivered during that window is dropped the same way.

The fix

All of it is in skills/basecamp-connect/SKILL.md. The connector's code doesn't change.

  • Monitor. The session arms the monitor for the full 30 minutes and re-arms it each time it expires.
  • Handled file. Each request the session takes is recorded in a file beside the connector's output. The monitor replays every request line not yet recorded. A request that arrived while no monitor ran is still delivered, and none is handed off twice.
  • Heartbeat. A recurring CronCreate job, every 10 minutes, picks up missed requests and re-arms a dead monitor. Cron jobs keep firing through failed turns, so the first heartbeat after Claude is reachable again restores the watch. It renews itself before Claude Code's 7-day limit on recurring jobs.
  • awk, not grep -v. The filter is awk because Claude Code's own grep holds back -v output on a live pipe. A monitor built on grep -v never fires.

New tests in internal/connector/skill_watch_test.go run the skill's filter line itself. One checks it skips handled requests by exact id (101 handled, 1011 still delivered). The other checks it streams each line while its input stays open. Removing fflush() fails the second test, and loosening the id match fails the first.

How it was verified

I drove real Claude Code sessions (2.1.309) in tmux. Each sat behind a local proxy to the Anthropic API that I could switch to closed connections or to a 401 "OAuth token has expired" reply. A stub basecamp on PATH played the connector, printing request lines in the real format and recording every call. One session followed main's skill and one followed this branch's, under the same script:

  1. Request 101 with the API up.
  2. A 401 outage lasting past the monitor's expiry and past Claude Code's retries ("Please run /login"), with request 102 sent during it.
  3. The API restored, then request 103.
101 102 (during outage) 103 (after) Watch afterwards
main answered never answered never answered no monitor
this branch answered answered answered re-armed

On this branch each request got exactly one acknowledgement update and one reply. To fit the run into minutes, the sessions used a 2-minute monitor and a 1-minute heartbeat in place of 30 and 10.

The filter was tested with GNU awk, GNU awk --posix, and the BWK awk that macOS ships (built from source).

Not verified:

  • A real basecamp connect against Basecamp. The connector's code and output are unchanged.
  • A real Mac or real sleep.
  • The 7-day heartbeat renewal.

Cost

The heartbeat is about 144 small turns a day while the connector runs. Each turn is one awk command, and most end with nothing to do.


Summary by cubic

Stops the connector's watch from dying across laptop sleep and an expired Claude login, so requests boosted with 👀 are answered instead of going silent.

  • The session records every finished request in a .handled file, replays unhandled ones through an awk filter that streams live output, and re-arms the monitor on expiry.
  • A recurring CronCreate heartbeat starts with basecamp connect status, deletes itself when the connector stopped, and otherwise re-arms a dead monitor and delivers requests missed while Claude was unreachable; it renews itself before the 7-day recurring-job limit and costs about 144 small turns a day.
  • A restart copies the old .handled file so a dropped request isn't handled twice, and the monitor and heartbeat are found and cancelled via task, CronList, and cron prefix.
  • New tests run the skill's filter itself, covering exact-id dedupe and live streaming of each line.

Written for commit 0a2ef66. Summary will update on new commits.

View guided diff Turn on auto-fix

…ogin

A Claude Code monitor is killed after 30 minutes at most, and only a turn of
the session arms it again. The skill asked for `persistent: true`, which the
Monitor tool doesn't have. When that turn can't reach Claude, on a laptop
just woken or with a login that expired, nothing re-arms the watch. The
connector keeps boosting each request, and the agent goes silent after the
👀 for as long as the session runs.

The skill now arms the monitor for its full 30 minutes and re-arms it on
expiry. It records each request it takes in a handled file beside the
connector's output, and the monitor replays everything not yet handled, so
a request that arrived while no monitor ran is still delivered, and none is
handed off twice. A recurring CronCreate heartbeat re-arms a dead watch and
picks up missed requests. Cron jobs keep firing through failed turns, so the
first heartbeat after Claude is reachable again restores the watch.

The filter is awk, not `grep -v`. Claude Code's own grep holds back `-v`
output on a live pipe, so a monitor built on it never fires. The new tests
run the skill's filter line itself: it skips handled requests by exact id,
and it streams each line while its input stays open.
Copilot AI balanced review requested due to automatic review settings October 23, 2026 19:08
@robzolkos
robzolkos requested a review from a team as a code owner October 23, 2026 19:08
@github-actions github-actions Bot added tests Tests (unit and e2e) skills Agent skills labels Oct 23, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Handled-request tracking omits local refusals and resets on restart, allowing repeated replies and duplicate work.

2 open issues
What changed in this PR

This PR updates the connector skill to recover its Claude Code watch after sleep or an expired login, without changing connector runtime code.

Changes:

  • Adds monitor renewal, a recurring recovery heartbeat, and handled-request tracking.
  • Updates restart and shutdown instructions to manage the watch lifecycle.
  • Adds tests for exact-ID filtering and live output streaming.

[!TIP]
If you aren't ready for review, convert to a draft PR.
Click "Convert to draft" or run gh pr ready --undo.
Click "Ready for review" or run gh pr ready to reengage.

File Description
skills/​basecamp-connect/​SKILL.md Defines watch recovery, request tracking, and lifecycle cleanup.
internal/​connector/​skill_watch_test.go Tests the skill's filter for exact-ID matching and streaming.

Review effort: Balanced

Comment thread skills/basecamp-connect/SKILL.md Outdated
Comment thread skills/basecamp-connect/SKILL.md Outdated
@jeremy

jeremy commented Oct 23, 2026

Copy link
Copy Markdown
Member

🤖 Reviewed at 6cf77db. CI is green. The diagnosis holds up: the Monitor tool has no persistent option and caps timeout_ms at 30 minutes, so the old instruction couldn't have kept a watch alive. The awk filter, the exact-key match, and the streaming test are right too. Both tests read the skill's own command lines, so either would fail if the filter went back to grep -v or started matching on a prefix. The 10-minute cron sets the retry pace while Claude is unreachable, and a revoked Basecamp credential still ends the connector and goes through When the connector stops. So no tight loop. Two things to change:

1. The heartbeat can outlive the connector, and once renewed it never stops. The skill says to "keep its job id" (L330), and Stopping and When the connector stops delete the heartbeat by that id (L710, L740). But from day six the heartbeat schedules a new copy of itself and deletes the old one (L335-L342), and nothing says to keep the new id. After a renewal, or after a compaction that loses the id, CronDelete targets a job that's already gone. The live heartbeat keeps going, and its prompt doesn't check whether the connector is still running. Every 30 minutes it arms a new tail -f on the dead output file, and every six days it renews itself, so it never ends on its own. Its prompt also ends "If there is nothing to do, say nothing", so a dead connector shows up as silence.

Suggested changes:

  • Put the connector's task id in the heartbeat prompt. If that task has exited, the heartbeat arms nothing, deletes itself, and follows When the connector stops.
  • Have Stopping and restart find the heartbeat with CronList, by its basecamp-connect heartbeat for '<profile>' prefix, instead of a remembered id. That also covers the renewal.

2. Only (b) and (c) write the handled file (L309-L310), so any other way a request ends gets replayed every 10 minutes and on every re-arm. Copilot's thread covers the owner-only refusal. The same gap applies to an owner's control request this session carries out itself instead of dispatching it (a setup change that waits on the person, say), and to any line the session decides to leave alone. Each replay runs step (a) again and repeats whatever reply went out. Changing the rule to "add the event_id when you're done with a request, however it ended" covers the refusal and these cases in one sentence. No need for a list of every path.

Not raised: two monitors after a compaction, or a heartbeat landing next to a monitor notification, can each deliver the same line. The session's turns run one at a time and it checks the handled file first, so that's a duplicate notification, not a duplicate dispatch.

Review found three gaps in the watch.

Only a dispatch or a holding reply wrote the handled file, so a request the
session finished any other way, such as an owner-only refusal, was replayed
by every heartbeat and re-armed monitor, and its reply posted again. The rule
is now: once done with a request, however it ended, record it.

The heartbeat was deleted by a remembered job id, which is gone after it
renews itself or after a compaction, and it never checked whether the
connector still ran. It now starts with `basecamp connect status`: when
`pings.running` is absent it arms nothing and deletes itself. Stopping and
restarting find it with CronList by its prompt's prefix.

A restart began with an empty handled file, although a connector restarted
right after a crash can print a request again. The new handled file now
starts as a copy of the old one.
Copilot AI balanced review requested due to automatic review settings October 23, 2026 20:20
@robzolkos

robzolkos commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator Author

Thanks, Jeremy. Both points are fixed in 0a2ef66.

1. The heartbeat outliving the connector. The heartbeat's prompt now starts with basecamp connect status -P '<profile>' --json. pings.running is set only while the lock holder's process is alive. When it's absent, the heartbeat arms nothing, deletes itself, and follows When the connector stops. Stopping and restarting find the heartbeat with CronList by its basecamp-connect heartbeat for '<profile>' prefix, so a renewal or a compaction no longer leaves a stale id. I checked both paths in a real Claude Code session with a stub connector:

  • Connector exits normally. The session got the exit notice, stopped its monitor and heartbeat, and no heartbeat fired afterwards.
  • Lock gone but task still alive, so no exit notice ever arrives. This stands in for a session that lost track after a compaction. The first heartbeat ran connect status, stopped the monitor and deleted itself, and nothing fired over the next two minutes.

2. Requests that end without (b) or (c). I took your wording: "once you're done with a request, however it ended, add its event_id". It sits in the per-request part, inside the compaction budget. In the same session, a participant's owner-only request got one refusal, and four minutes of heartbeats posted nothing more.

Copilot's restart point is fixed too: the new handled file starts as a copy of the old one.

CI is green on 0a2ef66, and both Copilot threads are resolved. Copilot last reviewed 6cf77db. I couldn't re-request it on the new head with my token, so it hasn't reviewed these fixes.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Recovery can disable itself during connector startup or disappear when an outage spans the heartbeat renewal window.

2 open issues
2 resolved since last review

Review effort: Balanced

Comment on lines +330 to +337
recurring, cron `'7-59/10 * * * *'`. Its prompt names every path, the profile,
today's date and the filter in full, since it must work after a compaction:

> basecamp-connect heartbeat for '<profile>'. Run `basecamp connect status
> -P '<profile>' --json`. If its `pings.running` is absent, the connector has
> stopped: arm nothing, CronDelete this heartbeat, and follow *When the
> connector stops* in the basecamp-connect skill. Otherwise run `<the filter>
> '<connector-output-file>'` and handle each line it prints as a request.
Comment on lines +341 to +342
> <today's date>: from six days on, schedule this prompt again with the new
> date, and CronDelete the older heartbeat. If there is nothing to do, say
@jeremy

jeremy commented Oct 23, 2026

Copy link
Copy Markdown
Member

🤖 Local review at 0a2ef66. CI is green on it (27 passed, 4 skipped, 1 neutral).

Your two fixes.

  1. Heartbeat outliving the connector: fixed for the case I raised, but the new check stops it too early in another case. Finding the heartbeat with CronList by its prefix (L348-L350, L713, L750) covers the renewal and a lost id. The connect status check (L333-L343) ends a heartbeat whose connector is gone, so it no longer re-arms on a dead file forever. The problem is the other direction, which Copilot's open thread on L337 points at, and it holds. pings.running is read from the instance lock (connect_pings.go L211-L224). A start waiting for its token only looks at the lock (connect_run.go L248-L257) and takes it later (L294), and TestAStartWaitingForItsTokenHoldsNoLock requires that. On a first run there is no ledger yet either, so status errors and prints no pings at all. A start on a computer just woken, or offline, still waiting for its token when the next heartbeat fires (at :07, :17 and so on), gets its monitor and heartbeat removed while the connector is still alive. Once it connects, nothing is watching. Your second session test (lock gone, task alive) is exactly this state, and in real use it means "still starting", not "stopped". I'd treat the connector as stopped only once its background task has exited, and put that task's id in the heartbeat prompt. A status that errors or shows no lock while the task runs should mean "arm the monitor and keep going". A smaller point: with --json the field is data.pings.running. A literal reading of "pings.running is absent" against the envelope takes every heartbeat for a stopped connector.

  2. Requests that end some other way: fixed. "However it ended" is now in both the per-request part (L144-L145) and the handled-file rule (L309). That covers the refusal, a control request handled in the session, and a line left alone. Carrying the handled file across a restart (L713-L721) closes Copilot's restart point too.

Tests. 0a2ef66 changes only SKILL.md, so reverting either fix fails no test. The two tests in skill_watch_test.go cover the filter and the record line, and both are unchanged. Most of this is prose that only a real session can check, and your stub-connector runs are the right evidence for it. One piece is code, though: the heartbeat now depends on data.pings.running in connect status --json, and nothing pins it. TestStatusAndDoctorSayWhoHearsPings (connect_pings_test.go L206-L229) decodes lock_holder and pings.saved but not pings.running, and the styled assertions don't touch the JSON key. If the key were renamed or moved, every heartbeat would delete itself and the watch would be gone. Asserting running is present while a connector runs, and absent after stop, would catch that.

The other open Copilot thread (L342, heartbeat expiry): not addressed, and it holds. Renewal starts "from six days on", so the heartbeat has to get one turn through in its last day. An outage that covers day six to day seven, like a login that expires on day five and comes back on day eight, leaves no heartbeat and no monitor, and nothing reports it. Renewing from one day on instead of six means an outage would have to last six days to do that, at the cost of a renewal a day.

Nothing else turned up. The local review ran go test ./internal/connector, which passed. It didn't run the full suite.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skills Agent skills tests Tests (unit and e2e)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants