Skip to content

Commit e2e48b3

Browse files
committed
docs(tck): the report tallies follow the base's new self-tests
The base gained fourteen tests this pass -- seven for the capability an SDK cannot express, seven for where the conformance assets came from -- and this branch runs all of them on top of its own. 276 passed / 41 skipped becomes 289 passed / 42 skipped; the skip is test_spec_assets declining to claim the submodule pin was checked in an environment that cannot read it. No scenario count moves. The new pin, 89b1519a, changes two $comment blocks in canonical-flags.json and no feature file at all, which was checked rather than assumed: the in-memory suite is 43 passed / 21 skipped / 1 xfailed before and after, and every one of the thirteen new passes is a unit test. Measured while doing it, and worth writing down because a later pass will otherwise re-derive it: results.digest is not a function of the spec pin. Two runs of the in-memory suite against identical assets produced different digests and streams 109 bytes apart, because Cucumber Messages carries per-run durations and ids. Nothing here pins one -- test_report computes the digest it compares, and the README's example is elided -- so there is no latent flake to fix, and the claim the README makes for it is the one it can keep: the digest tells a consumer that what it fetched is what the envelope described, not that two runs asked the same questions. The stream carries the executed feature source, and that is what answers the second question. Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
1 parent 127607e commit e2e48b3

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

‎tools/openfeature-tck/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -950,7 +950,7 @@ the block the schema leaves optional, so an empty one is left out rather than em
950950
| `test_canonical_set` | the canonical-set guard | a run that executed less than the canonical set fails and publishes nothing |
951951

952952
```
953-
276 passed, 41 skipped, 2 xfailed
953+
289 passed, 42 skipped, 2 xfailed
954954
```
955955

956956
No Docker and no network. The conformance suites take under a second; `test_report`,

0 commit comments

Comments
 (0)