fix(subscriber): discard goal events older than the user's latest evaluation - #2734
Conversation
…luation When the goal event timestamp is older than the user's latest evaluation timestamp, the event can never be linked because the client SDK sets the timestamps when the events are generated (the app tracked the goal before evaluating the user). The two code paths handled this inconsistently: - Fresh delivery: the event was acked and dropped silently while being counted in the "Linked" metric, so it looked like a successful link. - Retry path: the message was re-queued with backoff and retried until it expired (12-24h), even though retrying can never link it since the query always returns the latest evaluation and timestamps only grow. Both paths now discard the event explicitly with a warning log and the GoalEventIssuedBeforeEvaluation metric. The legitimate out-of-order Pub/Sub case (evaluation row not found yet) is unchanged and still goes through the Redis retry queue. Also stop the linker from overwriting the retry message when called from the retry processor, which previously reset the backoff retry count. Co-authored-by: Cursor <cursoragent@cursor.com>
There was a problem hiding this comment.
Pull request overview
This PR improves goal-event linking behavior in the GoalEventDWH subscriber by explicitly discarding goal events whose timestamps are older than the user’s latest evaluation (a non-retriable condition due to SDK-side timestamping), and by preventing retry backoff from being reset on the retry processor path.
Changes:
- Discard (ack/delete) goal events that are older than the latest evaluation, instead of counting them as linked or retrying indefinitely.
- Introduce
ErrGoalEventOlderThanEvaluationand propagate it through both the fresh Pub/Sub path and the Redis retry path. - Add unit tests covering link, discard, and retry/re-queue behavior, including ensuring the retry processor doesn’t overwrite/reset retry state.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| pkg/subscriber/processor/goal_events_dwh.go | Returns a sentinel error when all candidate evaluations are newer than the goal event and discards those events on the fresh path; avoids counting them as Linked. |
| pkg/subscriber/processor/goal_events_dwh_retry.go | Prevents retry backoff reset by disabling retry-message storage inside the linker on retry processing; deletes retry keys for non-linkable old events. |
| pkg/subscriber/processor/errors.go | Adds the new sentinel error ErrGoalEventOlderThanEvaluation with rationale. |
| pkg/subscriber/processor/goal_events_dwh_test.go | Adds unit tests for linking, discarding old events, and correct retry/re-queue behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Nit: Worth deleting the constant in this PR or a fast follow-up. |
The only place incrementing it was the len(evals) == 0 branch of handleNewRetry, which this PR replaced with the explicit ErrGoalEventOlderThanEvaluation discard path. Co-authored-by: Cursor <cursoragent@cursor.com>
@Ubisoft-potato, I have deleted it afc6f76. |
Summary
The client SDK sets the event timestamps when evaluation and goal events are generated. If a goal event's timestamp is older than the user's latest evaluation timestamp, the app tracked the goal before evaluating the user (incorrect SDK usage), so the event can never be linked. Pub/Sub reordering cannot cause this case: reordering only delays delivery, it doesn't change the timestamps inside the payloads.
The goal event DWH subscriber handled this condition inconsistently:
counted in the
Linkedmetric — it looked like a successful link butnothing was written to the DWH.
expired (12–24h), even though retrying can never succeed because the query
always returns the user's latest evaluation and timestamps only grow.
What
retry path deletes the Redis key, each with a warning log and the
GoalEventIssuedBeforeEvaluationmetric.ErrGoalEventOlderThanEvaluation.processor, which previously reset the backoff retry count.
DWH) is unchanged: it still goes to the Redis retry queue and links on
a later pass.
goal_events_dwh_test.go) covering the link, discard,and retry/re-queue paths.