Skip to content

Fix flaky HybridQueryExplainIT explanation test via per-_id assertions - #1900

Merged
heemin32 merged 1 commit into
opensearch-project:mainfrom
martin-gaievski:fix-flaky-hybrid-explain-processor-not-configured
Jul 8, 2026
Merged

Fix flaky HybridQueryExplainIT explanation test via per-_id assertions#1900
heemin32 merged 1 commit into
opensearch-project:mainfrom
martin-gaievski:fix-flaky-hybrid-explain-processor-not-configured

Conversation

@martin-gaievski

Copy link
Copy Markdown
Member

Description

Fixed flaky test for hybrid query explanation processor. Switched to new assert per-document explanation trees keyed by _id (routing is deterministic; only tied-hit order isn't), preserving all structure/score checks and dropping only the Lucene-internal child-count couplings.

Related Issues

#1899

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Martin Gaievski <gaievski@amazon.com>
@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🧪 PR contains tests
🔒 No security concerns identified
✅ No TODO sections
🔀 No multiple PR themes
⚡ No major issues detected

@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown

PR Code Suggestions ✨

Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Impact
General
Validate expected document IDs are present

After building explanationById, assert that it contains exactly the three expected
IDs ("1", "2", "3"). Otherwise, if a document is missing or an unexpected _id
appears in the hits, the subsequent assertNotNull checks will still pass on the
wrong entries or fail with a less informative message.

src/test/java/org/opensearch/neuralsearch/query/HybridQueryExplainIT.java [328-331]

 Map<String, Map<String, Object>> explanationById = new HashMap<>();
 for (Map<String, Object> oneHit : hitsNestedList) {
     explanationById.put((String) oneHit.get("_id"), getValueByKey(oneHit, "_explanation"));
 }
+assertEquals(Set.of("1", "2", "3"), explanationById.keySet());
Suggestion importance[1-10]: 4

__

Why: Adding an assertion on the keyset would produce a clearer failure message if hits are missing or unexpected, but the existing assertNotNull checks on each explanationById.get(...) already catch missing documents. It's a minor test robustness improvement.

Low

@codecov

codecov Bot commented Jul 7, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.43%. Comparing base (a5e4e8f) to head (8746220).

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #1900      +/-   ##
============================================
- Coverage     83.46%   83.43%   -0.03%     
+ Complexity     3893     3892       -1     
============================================
  Files           291      291              
  Lines         13835    13835              
  Branches       2300     2300              
============================================
- Hits          11547    11543       -4     
- Misses         1455     1458       +3     
- Partials        833      834       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@martin-gaievski

Copy link
Copy Markdown
Member Author

"Check Workflow Events" CI action will keep failing unless #1901 is merged, there is a bug in repo infra yaml file

@heemin32
heemin32 merged commit 1778358 into opensearch-project:main Jul 8, 2026
49 of 51 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants