Skip to content

Commit 7b0f4c8

Browse files
committed
fix(tck): the mismatch matrix is eight rows, not eleven
The base branch's pin moves three of the matrix's Examples rows behind `@string-typing`, so the outline it is read off is eight rows wide. Only the row-identity test hardcoded the count, and it is the one place that has to: `rows` comes from the message stream and `expected` from the feature file, so without a literal the assertion would pass on two empty lists. The literal is now 8, with a comment saying where the number comes from so the next asset change has somewhere to look. The three descriptions that said "eleven" follow it, including the README's, where the sibling count moved too. Nothing else in the report machinery counts rows -- the canonical total is read from the assets and every other assertion derives its expectation from the feature file, which is why one literal and three sentences is the whole of it. `poe test` is 289 passed, 42 skipped, 2 xfailed, unchanged. Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
1 parent bcc1fdc commit 7b0f4c8

2 files changed

Lines changed: 9 additions & 6 deletions

File tree

‎tools/openfeature-tck/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -363,7 +363,7 @@ capability skip's reason lands and the after-hook where a teardown failure does.
363363
responsible for a skip follows from the pickle's tags and the envelope's `declaration`, which is why
364364
it is not transported once per scenario. And a pickle's `astNodeIds` are `[scenario id, table row
365365
id]`, resolving in the `GherkinDocument` to exactly the cells the feature file wrote — which is what
366-
tells the eleven rows of the type-mismatch matrix apart, one of which differs in outcome from its ten
366+
tells the eight rows of the type-mismatch matrix apart, one of which differs in outcome from its seven
367367
siblings, exactly rather than by a naming convention every implementation would have to reproduce.
368368

369369
So the payload is not a transcription of pytest's summary. The one scenario the Python SDK cannot

‎tools/openfeature-tck/tests/test_report.py‎

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -83,7 +83,7 @@
8383

8484
UNKNOWN_KEY_SCENARIO = "An unknown flag key returns the code default"
8585

86-
# The type-mismatch matrix: eleven Examples rows under one scenario name, one of
86+
# The type-mismatch matrix: eight Examples rows under one scenario name, one of
8787
# which the Python SDK fails. It is the case row identity exists for.
8888
MISMATCH_SCENARIO = "Requesting the wrong type returns the code default"
8989

@@ -669,7 +669,7 @@ def test_every_collected_scenario_appears_exactly_once(run: Run) -> None:
669669
A test case is identified by its uri, its scenario name **and its Examples
670670
row**, all three recovered from the stream by following a pickle's AST node
671671
ids. Name alone is shared by every row of a Scenario Outline, so keying on it
672-
would let eleven rows of the type-mismatch matrix collapse into one and this
672+
would let the eight rows of the type-mismatch matrix collapse into one and this
673673
test would not notice.
674674
675675
Counted against pytest's own collection rather than against a number written
@@ -731,9 +731,9 @@ def test_a_scenario_skipped_for_another_reason_is_still_reported(run: Run) -> No
731731

732732

733733
def test_an_outline_row_is_identified_by_its_ast_node_id(run: Run) -> None:
734-
"""The eleven rows of the type-mismatch matrix are told apart, and only here.
734+
"""The eight rows of the type-mismatch matrix are told apart, and only here.
735735
736-
All eleven share one scenario name, which is the feature file's name and must
736+
All eight share one scenario name, which is the feature file's name and must
737737
stay that way: it is what a report from Go or JavaScript carries for the same
738738
row. What tells them apart is the pickle's second ``astNodeIds`` entry, the
739739
id of the table row it was compiled from, which resolves in the
@@ -743,7 +743,10 @@ def test_an_outline_row_is_identified_by_its_ast_node_id(run: Run) -> None:
743743
"""
744744
rows = run.stream.named(MISMATCH_SCENARIO)
745745
expected = _examples_from_the_feature_file("errors", MISMATCH_SCENARIO)
746-
assert len(rows) == len(expected) == 11
746+
# The literal is what keeps this from passing on two empty lists, and it
747+
# moves when the assets do: the matrix lost the three string rows to
748+
# @string-typing at spec d47a66eb, eleven down to eight.
749+
assert len(rows) == len(expected) == 8
747750

748751
observed = [dict(case.row) for case in rows]
749752
assert len(observed) == len({case.row for case in rows}), "two rows collapsed"

0 commit comments

Comments
 (0)