|
| 1 | +# Python Server SDK reliability review |
| 2 | + |
| 3 | +This review covers the production risks requested for the Python Server SDK: |
| 4 | +unit behavior, WebSocket synchronization, event processing, evaluation, |
| 5 | +logging/error isolation, thread safety, shutdown, memory retention, redundant |
| 6 | +code, and live FeatBit service interoperability. |
| 7 | + |
| 8 | +## Review result |
| 9 | + |
| 10 | +The review found and fixed the following concrete issues: |
| 11 | + |
| 12 | +- `RepeatableTask` stored an `Event` in `Thread._stop`, which breaks Python's |
| 13 | + own `Thread.join()` cleanup path. It now uses a separate stop event and joins |
| 14 | + deterministically. |
| 15 | +- WebSocket reconnect backoff used an uninterruptible sleep. Client shutdown |
| 16 | + now interrupts the wait, closes the socket defensively, stops the ping task, |
| 17 | + and joins the streaming thread. |
| 18 | +- The notice broadcaster mutated its listener registry concurrently. Listener |
| 19 | + operations now use a lock and dispatch from an immutable snapshot; a queue |
| 20 | + sentinel makes shutdown immediate and repeatable. |
| 21 | +- The event processor did not retain or join its dispatcher. It now owns the |
| 22 | + dispatcher lifecycle, joins the periodic flush task, and always completes a |
| 23 | + synchronous message even if message handling fails. |
| 24 | +- `FBClient.stop()` could expose an extension exception and skip all later |
| 25 | + cleanup. Shutdown is now idempotent, producer-first, and isolates each |
| 26 | + component failure. |
| 27 | +- Custom event processor errors could escape from `identify`, `track_*`, or |
| 28 | + `flush`. Event delivery is now best-effort and cannot fail the application |
| 29 | + request. |
| 30 | +- Invalid offline bootstrap JSON and unsupported runtime fallback objects could |
| 31 | + raise from evaluation-related calls. They now return `False` or the supplied |
| 32 | + fallback, with a diagnostic log message. |
| 33 | +- `Config` and the HTTP helper used mutable default objects. Each client now |
| 34 | + gets independent HTTP/WebSocket options and headers. |
| 35 | +- A duplicate `FBUser.from_dict()` call in `track_metric` was removed. |
| 36 | + |
| 37 | +Constructor/configuration validation still raises for invalid required |
| 38 | +configuration. This is intentional fail-fast behavior before an SDK client is |
| 39 | +usable; runtime evaluation, event, status, and shutdown paths are isolated. |
| 40 | + |
| 41 | +## Automated evidence |
| 42 | + |
| 43 | +Run from the repository root: |
| 44 | + |
| 45 | +```shell |
| 46 | +python -m pytest |
| 47 | +python -m flake8 . |
| 48 | +python scripts/resource_audit.py --evaluations 80000 --workers 8 |
| 49 | +python -m build |
| 50 | +``` |
| 51 | + |
| 52 | +Result on Python 3.12.13 (2026-08-07): |
| 53 | + |
| 54 | +- 83 unit/integration tests passed. |
| 55 | +- Flake8 passed. |
| 56 | +- 80,000 evaluations across 8 workers completed with 0 concurrent errors. |
| 57 | +- Throughput was 12,490 evaluations/second on the final audit run. |
| 58 | +- No FeatBit SDK worker threads remained after shutdown. |
| 59 | +- 24,417 traced bytes remained after garbage collection, below the 2 MiB |
| 60 | + regression threshold. |
| 61 | +- A clean wheel built from the source distribution contained 27 files, no |
| 62 | + `tests/` package, and no stale `featbit_openfeature/` package. |
| 63 | + |
| 64 | +The tests specifically cover concurrent evaluation, status-listener ordering, |
| 65 | +listener registration/removal under contention, event-processor failure |
| 66 | +isolation, repeated client lifecycle, periodic task joining, and interrupting a |
| 67 | +blocked WebSocket lifecycle. |
| 68 | + |
| 69 | +## Live FeatBit service verification |
| 70 | + |
| 71 | +Use a real environment Server Key without storing it in the repository: |
| 72 | + |
| 73 | +```shell |
| 74 | +export FEATBIT_ENV_SECRET='<server-key>' |
| 75 | +export FEATBIT_FLAG_KEY='python-app-release' |
| 76 | +python scripts/live_integration_check.py |
| 77 | +``` |
| 78 | + |
| 79 | +Optional variables are `FEATBIT_STREAMING_URL`, `FEATBIT_EVENT_URL`, and |
| 80 | +`FEATBIT_START_WAIT_SECONDS`. The script verifies: |
| 81 | + |
| 82 | +1. authenticated WebSocket connection and initial data synchronization; |
| 83 | +2. SDK status reaches `OK`; |
| 84 | +3. remote flag evaluation returns value, reason, and variation ID; |
| 85 | +4. feature-flag, identify, and custom metric events enter the delivery path; |
| 86 | +5. explicit flush and final synchronous drain complete; |
| 87 | +6. status changes to `OFF` and no SDK threads remain after close. |
| 88 | + |
| 89 | +The secret is read only from the process environment and is never printed or |
| 90 | +written to disk. |
| 91 | + |
| 92 | +A recorded FeatBit Cloud run on 2026-07-24 reached the WebSocket connected |
| 93 | +state, processed the initial data-sync payload, evaluated the remote |
| 94 | +`python-app-release` flag, exercised the gray-release application under |
| 95 | +concurrent HTTP traffic, and later demonstrated automatic reconnection after a |
| 96 | +remote-host disconnect. The checked-in script turns that one-off validation |
| 97 | +into a repeatable, secret-safe release check. |
| 98 | + |
| 99 | +An additional run on 2026-08-07 used the official FeatBit Docker Compose stack. |
| 100 | +It observed a remotely pushed rollout update in 0.5 seconds, completed 4,000 |
| 101 | +concurrent evaluations without errors, verified the default event-delivery |
| 102 | +path, handled a status listener that re-entered `client.stop()` without a |
| 103 | +deadlock, reached `OFF`, and left no SDK worker threads behind. |
| 104 | + |
| 105 | +## Remaining operational considerations |
| 106 | + |
| 107 | +- The SDK is designed as a long-lived singleton, not one client per request. |
| 108 | +- Event delivery is intentionally best-effort so analytics failure cannot |
| 109 | + affect application availability. |
| 110 | +- A custom `DataStorage`, `EventProcessor`, or `UpdateProcessor` remains |
| 111 | + responsible for its own internal correctness; the client isolates its |
| 112 | + runtime and shutdown exceptions. |
| 113 | +- Memory thresholds are regression guards, not universal capacity limits; |
| 114 | + production sizing should use the application's real flag and segment data. |
0 commit comments