Skip to content

feat: Record exposures under local evaluation - #318

Merged
gagantrivedi merged 3 commits into
mainfrom
feat/local-evaluation-experiment-exposure
Nov 6, 2026
Merged

gagantrivedi merged 3 commits into
mainfrom
feat/local-evaluation-experiment-exposure

Conversation

@gagantrivedi

Copy link
Copy Markdown
Member

Since Flagsmith/flagsmith#8532, the environment document carries metadata.experiment on feature states. The SDK dropped it when building engine models, so under local evaluation flag.experiment was always undefined and getExperimentFlag never recorded an exposure.

Changes

  • FeatureStateModel keeps metadata.experiment; the mapper adds it to the engine context's feature metadata for environment and segment feature states. Identity overrides carry none, as on the API.
  • Flags.fromEvaluationResult sets experiment, and also variant, which local evaluation dropped until now, so local exposures carry the variant key.
  • Locally evaluated environment flags carry no experiment, matching the API, which only sends it for identity flags.

Tests

getExperimentFlag under local evaluation: an enrolled identity records one exposure with its variant and experiment ID; an identity outside the rollout, one with an identity override, and a feature with no experiment record none. Environment flags carry no experiment.

Also checked against a local Flagsmith with the events pipeline: local and remote evaluation give the same value, variant and experiment, and one exposure reached the warehouse.

Not included

Bumping engine-test-data to v3.13.0, which adds the experiment cases: it also adds two numeric trait cases (Flagsmith/engine-test-data#63) the engine already fails, to be fixed separately.

Carry each feature state's experiment metadata from the environment
document into the engine context, and set it and the variant on flags
built from evaluation results, so getExperimentFlag records exposures
under local evaluation as it does under remote evaluation.

Environment flags carry no experiment, matching the API, which only
sends experiment metadata for identity flags.
@coderabbitai

coderabbitai Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

Feature-state evaluation now includes experiment metadata when it is available. Identity flags receive variant and experiment data from evaluation results. Environment evaluation excludes experiment metadata. The SDK does not track an exposure when a flag has no variant. Tests cover local evaluation for enrolled and non-enrolled identities, identity overrides, features without experiments, and environment flags.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: Minimal · up to 4a08a

Local evaluation now carries experiment data without recording exposures for flags lacking variants. No concrete merge-blocking risk was identified.

- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 7dfd96fa-7a77-4cd6-af3d-e07ea351e3ed
Commits

Reviewing files that changed from the base of the PR and between 006e947 and 4a08a96.

Files selected for processing: 3

  • flagsmith-engine/evaluation/evaluationContext/mappers.ts
  • sdk/index.ts
  • tests/sdk/flagsmith-experiments.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

@codspeed

codspeed Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

Merging this PR will improve performance by 41.65%

1 warning

⚠️ 1 benchmark with a significant change ran on different hardware

Its base and head results were measured on machines with a different CPU, core count or memory, so the change might not come from the code. Run base and head on the same runner type.

Learn more about Macro Runners

See affected benchmarks

⚡ 1 improved benchmark
✅ 14 untouched benchmarks

Performance Changes

Benchmark BASE HEAD Efficiency
⚡ getEnvironmentFlags 1.5 ms 1.1 ms +41.65%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.


Comparing feat/local-evaluation-experiment-exposure (4a08a96) with main (ee89f5a)

Open in CodSpeed

@gagantrivedi
gagantrivedi marked this pull request as ready for review November 5, 2026 09:21
@gagantrivedi
gagantrivedi requested a review from a team as a code owner November 5, 2026 09:21
@gagantrivedi
gagantrivedi requested review from Zaimwa9 and khvn26 and removed request for a team and khvn26 November 5, 2026 09:21
@Zaimwa9

Zaimwa9 commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

@codspeedbot fix this regression

@Zaimwa9

Zaimwa9 commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

@themis-blindfold codespeed is under its benchmark, are you able to pinpoint the root cause of the performance loss ?

@themis-blindfold

Copy link
Copy Markdown

I can't pinpoint the cause from the checkout: the CodSpeed results and profiles are not included, and the benchmark dependencies and fixtures are unavailable locally. The PR adds per-flag experiment conversion in sdk/models.ts:201 and an extra pass over environment flags in sdk/index.ts:639; both add work on benchmarked paths, but their contribution to the reported loss is unverified. Please share the affected benchmark results or a CodSpeed profile so the regression can be isolated. No performance fix is present in this PR.

@Zaimwa9 Zaimwa9 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should skip the exposure without variant and try to see where the performance regression comes from

Comment thread sdk/models.ts Resolved
Comment thread sdk/index.ts Resolved

@Zaimwa9 Zaimwa9 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@gagantrivedi
gagantrivedi merged commit 467dc54 into main Nov 6, 2026
7 checks passed
gagantrivedi pushed a commit that referenced this pull request Nov 6, 2026
🤖 I have created a release *beep* *boop*
---


##
[9.3.0](v9.2.0...v9.3.0)
(2026-11-06)


### Features

* Record exposures under local evaluation
([#318](#318))
([467dc54](467dc54))


### Other

* **deps-dev:** bump undici from 6.28.0 to 6.29.1
([#312](#312))
([0cc92c2](0cc92c2))
* Let Renovate keep updating spec bumps the webhook handler commits to
([#319](#319))
([ee89f5a](ee89f5a))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
gagantrivedi added a commit to Flagsmith/flagsmith-java-client that referenced this pull request Nov 6, 2026
Part of Flagsmith/flagsmith-private#325. Java counterpart of
Flagsmith/flagsmith-python-client#276, Flagsmith/flagsmith-go-client#246
and Flagsmith/flagsmith-nodejs-client#318.

`getExperimentFlag` only recorded exposures under remote evaluation. The
environment document carries `metadata.experiment` on feature states,
but local evaluation dropped it, along with the variant.

## Changes

- Map the multivariate option `key` into the evaluation context, and
carry `variant` from the evaluation result onto `Flag`. Locally
evaluated identity flags now report their variant.
- Map the feature state's `metadata.experiment` into the context's
feature metadata, and carry it onto identity flags.
- Environment flags carry no experiment: it is removed once, when the
environment evaluation context is built, so `getEnvironmentFlags` does
no extra work per call.
- `getExperimentFlag` records no exposure when the flag has no variant,
as the other SDKs do.

## Tests

- Local evaluation: an enrolled identity records one exposure with the
variant and `experiment_id`; outside the rollout, an identity override,
and a feature without an experiment record none.
- Environment flags carry no experiment under local evaluation.
- Remote: an enrolled flag without a variant records no exposure.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants