Skip to content

runtime: lock the timer queue when resetTimer mutates timer fields - #5623

Open
JacksonBopp wants to merge 1 commit into
tinygo-org:devfrom
JacksonBopp:fix-timer-reset-race
Open

JacksonBopp wants to merge 1 commit into
tinygo-org:devfrom
JacksonBopp:fix-timer-reset-race

Conversation

@JacksonBopp

Copy link
Copy Markdown

Summary

resetTimer's fast path for a periodic timer still firing on another goroutine allocates a fresh timerNode that points at the same shared *timer as the node currently running its callback. Every place that reads or writes that timer's when/period fields while it can be firing concurrently takes the scheduler's timer lock first: reAddTimer does, and the two reads in the threads scheduler's timerRunner do too. resetTimer's own writes to t.timer.when and t.timer.period were the one place that didn't, an unsynchronized store racing against those lock-protected accesses.

Update #5469

That issue is a nil pointer dereference during a fuzz-style repeat of the context tests, and #5470 already fixed one instance of this same "accessed outside the lock" pattern in timerRunner. This closes another surviving instance of it: resetTimer's writes now go through the same lock via two new functions, lockTimerQueue/unlockTimerQueue, implemented per scheduler (the real lock for threads and cores, interrupt disable/restore for the cooperative scheduler, and a no-op for scheduler.none, where timers are unsupported entirely).

Test plan

I wasn't able to reproduce the original crash directly. It's rare enough that the issue itself needed a repeated loop to hit it, and a targeted stress test hammering Ticker.Reset concurrently with active firing (16 tickers, 3 goroutines each, several minutes total across both the original and patched runtime) didn't trigger it either way.

What I can confirm:

  • The data race is real and reproducible by inspection: the write in resetTimer has no lock while every other writer and reader of the same fields has one.
  • gofmt is clean on all changed files.
  • The fix doesn't introduce any deadlock or regression: context package tests pass, and the same targeted stress test ran clean (no hangs, no crashes) against the patched runtime.

Happy to add a regression test if there's a preferred pattern for timing-dependent runtime races in this codebase, since a deterministic one isn't straightforward given how narrow the window is.

resetTimer's fast path for a periodic timer still firing on another
goroutine allocates a fresh timerNode that points at the same shared
*timer as the node currently running its callback. Every place that
reads or writes that timer's when/period fields while it can be
firing concurrently takes the scheduler's timer lock first: reAddTimer
does, and the two reads in the threads scheduler's timerRunner do
too. resetTimer's own writes to t.timer.when and t.timer.period were
the one place that didn't, an unsynchronized store racing against
those lock-protected accesses.

Update tinygo-org#5469

That issue is a nil pointer dereference during a fuzz-style repeat of
the context tests, and tinygo-org#5470 already fixed one instance of this same
"accessed outside the lock" pattern in timerRunner. This closes
another surviving instance of it: resetTimer's writes now go through
the same lock via two new functions, lockTimerQueue/unlockTimerQueue,
implemented per scheduler (the real lock for threads and cores,
interrupt disable/restore for the cooperative scheduler, and a no-op
for scheduler.none, where timers are unsupported entirely).

I wasn't able to reproduce the original crash directly. It's rare
enough that the issue itself needed a repeated loop to hit it, and a
targeted stress test hammering Ticker.Reset concurrently with active
firing (16 tickers, 3 goroutines each, several minutes total across
both the original and patched runtime) didn't trigger it either way.
What I can confirm: the data race is real and reproducible by
inspection (the write in resetTimer has no lock while every other
writer and reader of the same fields has one), and the fix doesn't
introduce any deadlock or regression under that same stress test.
@dgryski

dgryski commented Sep 15, 2026

Copy link
Copy Markdown
Member

Should this call to whenTicks also be inside the lock? https://github.com/tinygo-org/tinygo/blob/dev/src/runtime/scheduler_threads.go#L104

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants