Skip to content

Commit 2809563

Browse files
authored
feat: expose variation ID, SDK status, and harden runtime safety (#14)
1 parent 0ad40d8 commit 2809563

28 files changed

Lines changed: 1613 additions & 175 deletions

‎.github/workflows/code-quality.yaml‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ jobs:
2828
- name: Install dependencies
2929
run: |
3030
python -m pip install --upgrade pip
31-
pip install flake8 pytest pytest-mock
31+
pip install flake8 pytest pytest-mock build
3232
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
3333
- name: Lint with flake8
3434
run: |
@@ -37,3 +37,12 @@ jobs:
3737
- name: Test with pytest
3838
run: |
3939
pytest
40+
- name: Audit concurrency, memory, and thread cleanup
41+
if: matrix.python-version == '3.12'
42+
run: python scripts/resource_audit.py --evaluations 20000 --workers 8
43+
- name: Build production distributions
44+
if: matrix.python-version == '3.12'
45+
run: python -m build
46+
- name: Verify wheel contains production packages only
47+
if: matrix.python-version == '3.12'
48+
run: python -c "import glob, zipfile; names = zipfile.ZipFile(glob.glob('dist/*.whl')[0]).namelist(); assert not any(name.startswith(('tests/', 'featbit_openfeature/')) for name in names)"

‎MANIFEST.in‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
11
include requirements.txt
22
include README.md
33
include dev-requirements.txt
4-
include release/package.json
4+
include release/package.json

‎README.md‎

Lines changed: 39 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ If you want to use your own data source, see [Offline Mode](#offline-mode).
1818
## Get Started
1919

2020
### Installation
21-
install the sdk in using pip, this version of the SDK is compatible with Python 3.6 through 3.11.
21+
Install the SDK using pip. This version is compatible with Python 3.6 through 3.12.
2222

2323
```shell
2424
pip install fb-python-sdk
@@ -66,6 +66,29 @@ client.stop()
6666

6767
- [Python Demo](https://github.com/featbit/featbit-samples/blob/main/samples/dino-game/demo-python/demo_python.py)
6868

69+
### SDK reliability and live verification
70+
71+
The repository includes a repeatable [reliability review](docs/REVIEW.md)
72+
covering unit tests, thread safety, memory retention, background-thread
73+
shutdown, public runtime exception isolation, and redundant code. Run the
74+
local resource audit with:
75+
76+
```shell
77+
python scripts/resource_audit.py --evaluations 80000 --workers 8
78+
```
79+
80+
To verify WebSocket synchronization, remote evaluation, status changes, event
81+
delivery, and clean shutdown against FeatBit Cloud or a self-hosted evaluation
82+
service, set `FEATBIT_ENV_SECRET` and run:
83+
84+
```shell
85+
python scripts/live_integration_check.py
86+
```
87+
88+
The live script never prints or persists the environment secret. See the
89+
review document for all supported environment variables and verification
90+
criteria.
91+
6992
### FBClient
7093

7194
Applications **SHOULD instantiate a single FBClient instance** for the lifetime of the application. In the case where an application
@@ -112,6 +135,16 @@ if client.update_status_provider.wait_for_OKState():
112135

113136
It's possible to set a timeout in seconds for the `wait_for_OKState` method. If the timeout is reached, the method will return `False` and the client will still be in an uninitialized state. If you do not specify a timeout, the method will wait indefinitely.
114137

138+
You can also observe later connection interruptions and recoveries:
139+
140+
```python
141+
def on_status_change(state):
142+
print(state.state_type, state.error_track)
143+
144+
client.update_status_provider.add_listener(on_status_change)
145+
# Remove it during application shutdown:
146+
client.update_status_provider.remove_listener(on_status_change)
147+
```
115148

116149
> To check if the client is ready is optional. Even if the client is not ready, you can still evaluate feature flags, but the default value will be returned if SDK is not yet initialized.
117150
@@ -136,6 +169,7 @@ if client.initialize:
136169
flag_value = client.variation(flag_key, user, default_value)
137170
# evaluate the flag value and get the detail
138171
detail = client.variation_detail(flag_key, user, default=None)
172+
print(detail.variation, detail.variation_id, detail.reason)
139173
```
140174

141175
If you would like to get variations of all feature flags in a special environment, you can use `fbclient.client.FBClient.get_all_latest_flag_variations`, SDK will return `fbclient.common_types.AllFlagStates`, that explain the details of all feature flags. `fbclient.common_types.AllFlagStates.get()` returns the detail of a given feature flag key.
@@ -155,6 +189,10 @@ if client.initialize:
155189

156190
> **Note**
157191
> If evaluation happened before the client is initialized, or you provide the wrong flag key/user for evaluation, the `variation` calls will return the default value. The `fbclient.common_types.EvalDetail` will explain the details of the latest evaluation including error reason.
192+
> Runtime evaluation does not raise solely because a supplied default has an
193+
> unsupported type. If no flag value can be produced, that default object is
194+
> returned unchanged and a diagnostic message is logged. Required client
195+
> configuration remains fail-fast during construction.
158196
159197
### Flag Tracking
160198

‎docs/REVIEW.md‎

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
# Python Server SDK reliability review
2+
3+
This review covers the production risks requested for the Python Server SDK:
4+
unit behavior, WebSocket synchronization, event processing, evaluation,
5+
logging/error isolation, thread safety, shutdown, memory retention, redundant
6+
code, and live FeatBit service interoperability.
7+
8+
## Review result
9+
10+
The review found and fixed the following concrete issues:
11+
12+
- `RepeatableTask` stored an `Event` in `Thread._stop`, which breaks Python's
13+
own `Thread.join()` cleanup path. It now uses a separate stop event and joins
14+
deterministically.
15+
- WebSocket reconnect backoff used an uninterruptible sleep. Client shutdown
16+
now interrupts the wait, closes the socket defensively, stops the ping task,
17+
and joins the streaming thread.
18+
- The notice broadcaster mutated its listener registry concurrently. Listener
19+
operations now use a lock and dispatch from an immutable snapshot; a queue
20+
sentinel makes shutdown immediate and repeatable.
21+
- The event processor did not retain or join its dispatcher. It now owns the
22+
dispatcher lifecycle, joins the periodic flush task, and always completes a
23+
synchronous message even if message handling fails.
24+
- `FBClient.stop()` could expose an extension exception and skip all later
25+
cleanup. Shutdown is now idempotent, producer-first, and isolates each
26+
component failure.
27+
- Custom event processor errors could escape from `identify`, `track_*`, or
28+
`flush`. Event delivery is now best-effort and cannot fail the application
29+
request.
30+
- Invalid offline bootstrap JSON and unsupported runtime fallback objects could
31+
raise from evaluation-related calls. They now return `False` or the supplied
32+
fallback, with a diagnostic log message.
33+
- `Config` and the HTTP helper used mutable default objects. Each client now
34+
gets independent HTTP/WebSocket options and headers.
35+
- A duplicate `FBUser.from_dict()` call in `track_metric` was removed.
36+
37+
Constructor/configuration validation still raises for invalid required
38+
configuration. This is intentional fail-fast behavior before an SDK client is
39+
usable; runtime evaluation, event, status, and shutdown paths are isolated.
40+
41+
## Automated evidence
42+
43+
Run from the repository root:
44+
45+
```shell
46+
python -m pytest
47+
python -m flake8 .
48+
python scripts/resource_audit.py --evaluations 80000 --workers 8
49+
python -m build
50+
```
51+
52+
Result on Python 3.12.13 (2026-08-07):
53+
54+
- 83 unit/integration tests passed.
55+
- Flake8 passed.
56+
- 80,000 evaluations across 8 workers completed with 0 concurrent errors.
57+
- Throughput was 12,490 evaluations/second on the final audit run.
58+
- No FeatBit SDK worker threads remained after shutdown.
59+
- 24,417 traced bytes remained after garbage collection, below the 2 MiB
60+
regression threshold.
61+
- A clean wheel built from the source distribution contained 27 files, no
62+
`tests/` package, and no stale `featbit_openfeature/` package.
63+
64+
The tests specifically cover concurrent evaluation, status-listener ordering,
65+
listener registration/removal under contention, event-processor failure
66+
isolation, repeated client lifecycle, periodic task joining, and interrupting a
67+
blocked WebSocket lifecycle.
68+
69+
## Live FeatBit service verification
70+
71+
Use a real environment Server Key without storing it in the repository:
72+
73+
```shell
74+
export FEATBIT_ENV_SECRET='<server-key>'
75+
export FEATBIT_FLAG_KEY='python-app-release'
76+
python scripts/live_integration_check.py
77+
```
78+
79+
Optional variables are `FEATBIT_STREAMING_URL`, `FEATBIT_EVENT_URL`, and
80+
`FEATBIT_START_WAIT_SECONDS`. The script verifies:
81+
82+
1. authenticated WebSocket connection and initial data synchronization;
83+
2. SDK status reaches `OK`;
84+
3. remote flag evaluation returns value, reason, and variation ID;
85+
4. feature-flag, identify, and custom metric events enter the delivery path;
86+
5. explicit flush and final synchronous drain complete;
87+
6. status changes to `OFF` and no SDK threads remain after close.
88+
89+
The secret is read only from the process environment and is never printed or
90+
written to disk.
91+
92+
A recorded FeatBit Cloud run on 2026-07-24 reached the WebSocket connected
93+
state, processed the initial data-sync payload, evaluated the remote
94+
`python-app-release` flag, exercised the gray-release application under
95+
concurrent HTTP traffic, and later demonstrated automatic reconnection after a
96+
remote-host disconnect. The checked-in script turns that one-off validation
97+
into a repeatable, secret-safe release check.
98+
99+
An additional run on 2026-08-07 used the official FeatBit Docker Compose stack.
100+
It observed a remotely pushed rollout update in 0.5 seconds, completed 4,000
101+
concurrent evaluations without errors, verified the default event-delivery
102+
path, handled a status listener that re-entered `client.stop()` without a
103+
deadlock, reached `OFF`, and left no SDK worker threads behind.
104+
105+
## Remaining operational considerations
106+
107+
- The SDK is designed as a long-lived singleton, not one client per request.
108+
- Event delivery is intentionally best-effort so analytics failure cannot
109+
affect application availability.
110+
- A custom `DataStorage`, `EventProcessor`, or `UpdateProcessor` remains
111+
responsible for its own internal correctness; the client isolates its
112+
runtime and shutdown exceptions.
113+
- Memory thresholds are regression guards, not universal capacity limits;
114+
production sizing should use the application's real flag and segment data.

‎fbclient/__init__.py‎

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -24,9 +24,7 @@ def get() -> FBClient:
2424
If you need to create multiple client instances with different environments, instead of this
2525
singleton approach you can call directly the :class:`fbclient.client.FBClient` constructor.
2626
"""
27-
global __config
2827
global __client
29-
global __lock
3028

3129
try:
3230
__lock.read_lock()
@@ -58,7 +56,6 @@ def set_config(config: Config):
5856
"""
5957
global __config
6058
global __client
61-
global __lock
6259

6360
try:
6461
__lock.write_lock()

0 commit comments

Comments
 (0)