1- """The OpenFeature Provider Conformance Suite against this provider.
1+ """The OpenFeature Provider Conformance Suite (Appendix F) against this provider.
22
3- The suite is Appendix F of the OpenFeature specification: the same Gherkin scenarios, against the
4- same canonical flag set, that every other language's provider TCK runs. See
5- https://github.com/open-feature/spec/issues/417.
3+ Opt-in: `openfeature-tck` is not on PyPI yet, so this skips unless it is installed from
4+ `open-feature/python-sdk-contrib` (`tools/openfeature-tck`, with its `compose` extra)::
65
7- **This module skips unless the TCK is installed.** `openfeature-tck` is not on PyPI yet -- it
8- currently lives in `open-feature/python-sdk-contrib` under `tools/openfeature-tck`. So the suite is
9- opt-in rather than part of the default `pytest` run:
10-
11- pip install -e /path/to/python-sdk-contrib/tools/openfeature-tck
126 pytest tests/test_provider_conformance.py
137
14- It skips cleanly otherwise, so it costs the normal test run nothing.
15-
16- The backend is a container described by `tests/tck/docker-compose.yaml`. It, and how the four
17- language adoptions compare against it, are documented once in
18- https://github.com/aepfli/flagsmith-tck-testbed rather than restated in each adoption.
8+ Backend and cross-language results: https://github.com/aepfli/flagsmith-tck-testbed
199
20- Results at the time of writing: **29 passed, 11 failed, 25 skipped**. The deviations below say
21- which failures are defects in this provider and which are the backend's type system.
10+ Current: 29 passed, 11 failed, 25 skipped.
2211"""
2312
2413from __future__ import annotations
@@ -73,34 +62,20 @@ def tck_config(tck_backend: RunningBackend) -> TckConfig:
7362
7463 def new_provider () -> FlagsmithProvider :
7564 client = Flagsmith (environment_key = SERVER_SIDE_KEY , api_url = base_url )
76- # use_boolean_config_value=True is required for the canonical flag set, and the reason is
77- # worth recording.
78- #
79- # This provider defaults to resolving a boolean flag from Flagsmith's `enabled` state, with
80- # feature_state_value as the opt-in. The Go and Java providers default the other way round.
81- # Four providers for one product, split two-two on what a boolean flag *is* -- JavaScript
82- # shares this default, Go and Java do not.
83- #
84- # The canonical set models booleans as values, so the default would resolve
85- # boolean-zero-flag to True, which the falsy-value scenario exists to catch.
65+ # Default is False, which resolves a boolean from `enabled` rather than
66+ # feature_state_value -- JavaScript's default too, and the opposite of Go and Java. The
67+ # canonical set models booleans as values.
8668 return FlagsmithProvider (client , use_boolean_config_value = True )
8769
8870 return TckConfig (
8971 name = "flagsmith-python-remote" ,
9072 control = tck_backend .control ,
9173 new_provider = new_provider ,
92- # Capability.VARIANTS is withheld: Flagsmith has no variant concept for a plain feature,
93- # the evaluation response carries no variant key, and no seeding can produce one. That is
94- # permitted rather than defective -- 2.2.4 makes populating the variant a SHOULD and
95- # types.md marks the field optional -- so it carries no deviation entry.
74+ # Withheld, permitted rather than defective, so no deviation: VARIANTS (Flagsmith names no
75+ # value) and the lifecycle/event tags (no observable initialisation).
9676 #
97- # The lifecycle and event capabilities are withheld because this provider implements no
98- # initialize/shutdown lifecycle the suite can assert against. Declaring them would make
99- # those scenarios pass without the provider having done anything, and a vacuous pass is
100- # worse than a skip.
101- # DISABLED_FLAGS and NUMERIC_COERCION are declared and both fail: this provider attempts
102- # each and gets it wrong -- it raises GENERAL for a disabled flag, and type-checks a float
103- # that Flagsmith stores as a string.
77+ # DISABLED_FLAGS and NUMERIC_COERCION are declared and both fail: the provider attempts
78+ # each and gets it wrong.
10479 capabilities = {
10580 Capability .OBJECT ,
10681 Capability .LARGE_INTEGERS ,
@@ -113,30 +88,21 @@ def new_provider() -> FlagsmithProvider:
11388 capability = Capability .DISABLED_FLAGS ,
11489 issue = "" ,
11590 summary = (
116- "A disabled flag raises FlagsmithProviderError with error code GENERAL rather "
117- "than resolving to the caller default with no error code, so the scenario "
118- "fails with GENERAL where it expects none. Neither configuration satisfies "
119- "it: return_value_for_disabled_flags defaults to False and raises, and "
120- "setting it True returns the configured value of the flag instead of the "
121- "caller default, which the scenario also catches, because the configured "
122- "value of each disabled flag differs from the default the scenario passes in. "
123- "The Go and Java Flagsmith providers return the caller default with reason "
124- "DISABLED and no error code, which is what the tag asserts, and both declare "
125- "the capability. The JavaScript provider has the same defect as this one."
91+ "A disabled flag raises GENERAL rather than resolving to the caller default "
92+ "with no error code. Neither configuration satisfies the scenario: "
93+ "return_value_for_disabled_flags defaults to False and raises, and setting it "
94+ "True returns the configured value rather than the caller default, which the "
95+ "scenario also catches. Go and Java pass this; JavaScript has the same defect."
12696 ),
12797 ),
12898 KnownDeviation (
12999 capability = Capability .NUMERIC_COERCION ,
130100 issue = "" ,
131101 summary = (
132- "Declared and failing rather than withheld, because the provider does have a float "
133- "accessor and gets the answer wrong. Flagsmith has no "
134- "float type -- feature_state_value is natively boolean, integer or string -- "
135- "so every float is stored as a string, and _BASIC_FLAG_TYPE_MAPPINGS[FLOAT] "
136- "is a plain isinstance check against `float`. A Flagsmith float therefore "
137- "never satisfies it and get_float_details returns TYPE_MISMATCH, so reading a "
138- "float back as a float does not work at all. The Go provider handles this by "
139- "parsing the string in its Float accessor; this one does not."
102+ "Flagsmith stores floats as strings, and _BASIC_FLAG_TYPE_MAPPINGS[FLOAT] is a "
103+ "plain isinstance check against `float`, so a Flagsmith float never satisfies "
104+ "it and get_float_details returns TYPE_MISMATCH. Reading a float back as a "
105+ "float does not work at all. Go parses the stored string; this one does not."
140106 ),
141107 ),
142108 ],
0 commit comments