Skip to content

perf: batch runs-service NOTIFY payloads - #7807

Open
Rohithmatham12 wants to merge 1 commit into
flyteorg:mainfrom
Rohithmatham12:codex/flyte-notify-pending-batch
Open

perf: batch runs-service NOTIFY payloads#7807
Rohithmatham12 wants to merge 1 commit into
flyteorg:mainfrom
Rohithmatham12:codex/flyte-notify-pending-batch

Conversation

@Rohithmatham12

Copy link
Copy Markdown
Contributor

Summary

  • batch drained runs-service NOTIFY payloads into a single multi-call pg_notify statement per batch
  • cap each batch at 1024 payloads to stay far below PostgreSQL's bind parameter limit
  • retry failed NOTIFY batches instead of dropping all payloads from a transient connection error
  • add unit coverage for query generation and batch draining/capping

Fixes #7756.

Testing

  • go test ./runs/repository/impl -run 'Test(BuildNotifyBatchQuery|DrainNotifyBatch|NotifyActionUpdate_PayloadWithSpecialChars|NotifyRunUpdate_PayloadWithSpecialChars|RunNotifyLoop_NilConnNoPanic)$'

@github-actions github-actions Bot added the flyte2 label Aug 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf: batch pg_notify round trips in the runs-service NOTIFY pump

1 participant