You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
tests/targets.ini declares NameResHasBlocklist = false for [ci-es] (#137). That was
the right call at the time — namelookup-es implements no blocklist, so all 313 blocklist
sheet entries and the 10 negative-flagged sheet rows failed by construction, reporting the
same known thing every day and drowning the findings that mattered.
But it is a stopgap with no expiry. While it is set, ci-es has no blocklist coverage at
all, and nothing anywhere fails if that stays true after it has stopped being appropriate.
A flag that suppresses tests is exactly the kind that outlives its reason: the suite stays
green either way, so there is no signal to prompt anyone.
What this issue is for
Deciding the trigger for taking it back out, and doing so.
The real fix is NCATSTranslator/Babel#738,
now decided and on Babel v1.19: filter blocked synonyms in Babel so they are never written
out, rather than having each consumer delete them from its own index. Once that ships, no
consumer needs its own blocklist and the flag becomes meaningless rather than merely stale.
Candidate triggers, in rough order of how well they mean what we want:
Babel v1.19 ships. Cheap and dated, but "released" is not "deployed to ci-es and
reindexed", so it can fire early and produce a wave of failures that say nothing.
namelookup-es implements a blocklist of its own, if #738 slips and NameRes ES picks up
the consumer-side blocklist that the Solr deployment has. Then the flag comes out with no
Babel involvement.
Note that (1) and (3) are different worlds — under #738 there is nothing for a consumer to
implement, and the "blocklist" tests become tests that Babel did not emit the term. Worth
deciding whether tests/nameres/test_blocklist.py still makes sense in that world, or
whether the assertion moves to Babel's own validation.
Done when
A trigger is agreed and written down here.
NameResHasBlocklist is gone from [ci-es] (and, if #738 makes per-consumer blocklists
obsolete, from [DEFAULT] and the two skip sites too).
pytest --target ci-es -n auto tests/nameres/ runs the blocklist assertions again and
they pass — tests/unit/test_target_capabilities.py::test_only_the_elasticsearch_namelookup_declares_no_blocklist
will need updating, and is there to make sure this is a deliberate edit rather than a drift.
Related
#137 added the flag. #132 and the rest of milestone 12 are the other
NameRes ES findings. #91 is why the skip has to sit in the test body rather than in pytest_generate_tests.
tests/targets.inideclaresNameResHasBlocklist = falsefor[ci-es](#137). That wasthe right call at the time — namelookup-es implements no blocklist, so all 313 blocklist
sheet entries and the 10
negative-flagged sheet rows failed by construction, reporting thesame known thing every day and drowning the findings that mattered.
But it is a stopgap with no expiry. While it is set,
ci-eshas no blocklist coverage atall, and nothing anywhere fails if that stays true after it has stopped being appropriate.
A flag that suppresses tests is exactly the kind that outlives its reason: the suite stays
green either way, so there is no signal to prompt anyone.
What this issue is for
Deciding the trigger for taking it back out, and doing so.
The real fix is NCATSTranslator/Babel#738,
now decided and on Babel v1.19: filter blocked synonyms in Babel so they are never written
out, rather than having each consumer delete them from its own index. Once that ships, no
consumer needs its own blocklist and the flag becomes meaningless rather than merely stale.
Candidate triggers, in rough order of how well they mean what we want:
ci-esindex has been rebuilt from a Babel release that includes #738. Closest tothe thing we actually care about, and directly checkable — except that NameRes ES
/statusreportsbabel_version: null(NameRes ES: /status reports numDocs 0 and no babel_version #135), so we cannot currently tell which Babelrelease an ES index was built from. Blocked on NameRes ES: /status reports numDocs 0 and no babel_version #135 unless we track it out of band.
reindexed", so it can fire early and produce a wave of failures that say nothing.
the consumer-side blocklist that the Solr deployment has. Then the flag comes out with no
Babel involvement.
Note that (1) and (3) are different worlds — under #738 there is nothing for a consumer to
implement, and the "blocklist" tests become tests that Babel did not emit the term. Worth
deciding whether
tests/nameres/test_blocklist.pystill makes sense in that world, orwhether the assertion moves to Babel's own validation.
Done when
NameResHasBlocklistis gone from[ci-es](and, if #738 makes per-consumer blocklistsobsolete, from
[DEFAULT]and the two skip sites too).pytest --target ci-es -n auto tests/nameres/runs the blocklist assertions again andthey pass —
tests/unit/test_target_capabilities.py::test_only_the_elasticsearch_namelookup_declares_no_blocklistwill need updating, and is there to make sure this is a deliberate edit rather than a drift.
Related
#137 added the flag. #132 and the rest of
milestone 12 are the other
NameRes ES findings. #91 is why the skip has to sit in the test body rather than in
pytest_generate_tests.