Skip to content

feat: make messages if there's assert_no_errors more verbose - #4423

Merged
patrick91 merged 14 commits into
strawberry-graphql:mainfrom
Akay7:testClientVerboseMessaages
Aug 30, 2026
Merged

feat: make messages if there's assert_no_errors more verbose#4423
patrick91 merged 14 commits into
strawberry-graphql:mainfrom
Akay7:testClientVerboseMessaages

Conversation

@Akay7

@Akay7 Akay7 commented May 22, 2026

Copy link
Copy Markdown
Contributor

Description

At test client if there assert_no_errors(default behavior) then displayed just that assert not passed, but no messages. Those changes fix that behavior and will expose response.errors

Port of strawberry-graphql/strawberry-django#828

Types of Changes

  • Core
  • Bugfix
  • New feature
  • Enhancement/optimization
  • Documentation

Checklist

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • I have tested the changes and verified that they work and don't break anything (as well as I can manage).

@sourcery-ai

sourcery-ai Bot commented May 22, 2026

Copy link
Copy Markdown
Contributor

Reviewer's Guide

Makes GraphQL test clients’ assert_no_errors failures include the underlying response.errors and adds cross-backend tests to verify the verbose assertion messages.

File-Level Changes

Change Details Files
Include response.errors in assertion message when assert_no_errors is enabled on test clients.
  • Update aiohttp GraphQLTestClient.query to pass response.errors as the assertion message when errors are present.
  • Update base/strawberry GraphQL test client query to pass response.errors as the assertion message when errors are present.
strawberry/aiohttp/test/client.py
strawberry/test/client.py
Add tests ensuring assert_no_errors failures expose error details for different backends.
  • Introduce a shared invalid query and error-checking helper to validate error structure and message contents.
  • Add ASGI GraphQLTestClient test verifying AssertionError contains the GraphQL errors payload.
  • Add Django GraphQLTestClient test verifying AssertionError contains the GraphQL errors payload.
  • Add aiohttp GraphQLTestClient async test verifying AssertionError contains the GraphQL errors payload, skipping if aiohttp is not installed.
  • Document test intent via a module-level docstring for the test file.
tests/test/test_client.py

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@github-actions

github-actions Bot commented May 22, 2026

Copy link
Copy Markdown
Contributor

Thanks for adding the RELEASE.md file!

Below is the changelog that will be used for the release.


This release adds richer verbose output to assert_no_errors. When GraphQL
errors are detected, the assertion now includes full error details, making it
easier to debug failing tests.

This release was contributed by @Akay7 in #4423

Additional contributors: @bellini666, @pre-commit-ci[bot], @sourcery-ai[bot], @greptile-apps[bot]

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue, and left some high level feedback:

  • The helper check_non_existent_field_error could use a more precise type than Any (e.g. list[dict[str, Any]] or the concrete error type returned by the client) to make its intent clearer and catch shape changes earlier.
  • The repeated imports of schema and test client setup logic across the three new tests could be slightly DRYed up by moving shared constants/imports to module scope or using a small fixture/helper for the GraphQL clients.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The helper `check_non_existent_field_error` could use a more precise type than `Any` (e.g. `list[dict[str, Any]]` or the concrete error type returned by the client) to make its intent clearer and catch shape changes earlier.
- The repeated imports of `schema` and test client setup logic across the three new tests could be slightly DRYed up by moving shared constants/imports to module scope or using a small fixture/helper for the GraphQL clients.

## Individual Comments

### Comment 1
<location path="tests/test/test_client.py" line_range="54-55" />
<code_context>
+    check_non_existent_field_error(exc_info.value.args[0])
+
+
+@pytest.mark.aiohttp
+async def test_aiohttp_client_assert_no_errors_verbose_message():
+    try:
+        from aiohttp import web
+        from aiohttp.test_utils import TestClient as AiohttpTestClient
+        from aiohttp.test_utils import TestServer
+
+        from strawberry.aiohttp.test import GraphQLTestClient
+        from strawberry.aiohttp.views import GraphQLView
+    except ImportError:
+        pytest.skip("Aiohttp not installed")
+
+    from tests.views.schema import schema
+
+    view = GraphQLView(schema=schema)
+    app = web.Application()
+    app.router.add_route("*", "/graphql/", view)
+
+    async with AiohttpTestClient(TestServer(app)) as client:
+        graphql_client = GraphQLTestClient(client)
+
+        with pytest.raises(AssertionError) as exc_info:
+            await graphql_client.query(query_to_non_existent_field)
+
+        check_non_existent_field_error(exc_info.value.args[0])
</code_context>
<issue_to_address>
**suggestion (testing):** Consider adding a test for the generic GraphQLTestClient (non-framework) to mirror the new behavior

You already validate the verbose assertion message for the ASGI, Django, and aiohttp clients. Since there is also a generic `strawberry.test.client.GraphQLTestClient` used outside these frameworks, please add a corresponding test for that client so the new `assert_no_errors` behavior is covered consistently across all variants.

```suggestion
def test_graphql_test_client_assert_no_errors_verbose_message():
    from strawberry.test.client import GraphQLTestClient
    from tests.views.schema import schema  # noqa: F401

    client = GraphQLTestClient(schema)

    with pytest.raises(AssertionError) as exc_info:
        client.query(query_to_non_existent_field)

    check_non_existent_field_error(exc_info.value.args[0])


@pytest.mark.django
def test_django_client_assert_no_errors_verbose_message():
```
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread tests/test/test_client.py Outdated
@greptile-apps

greptile-apps Bot commented May 22, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR improves the assert_no_errors behavior in Strawberry's test clients by including response.errors as the assertion message, so failing assertions now surface the actual GraphQL errors instead of a bare AssertionError.

  • strawberry/test/client.py and strawberry/aiohttp/test/client.py each get a one-line change: assert response.errors is Noneassert response.errors is None, response.errors.
  • Three new integration tests (ASGI, Django, aiohttp) verify that the AssertionError raised by assert_no_errors carries the full error list in exc_info.value.args[0].

Confidence Score: 4/5

Safe to merge — the change is a two-character addition to two assert statements, with no impact on production code paths.

Both source-file changes are minimal and correct. The new tests cover all three client implementations. The only blemish is a dead-code import in the Django test that is suppressed with a noqa comment rather than removed.

tests/test/test_client.py — the unused schema import in the Django test function.

Important Files Changed

Filename Overview
strawberry/test/client.py Single-line fix: added response.errors as the assertion message to assert response.errors is None. All subclasses that don't override query (ASGI, Django) inherit this improvement automatically.
strawberry/aiohttp/test/client.py Same single-line fix applied to the async query override. The aiohttp client duplicates the query logic rather than inheriting it, so the fix correctly needed to be applied here as well.
tests/test/test_client.py Three new per-client tests validate that the assertion error message contains the actual GraphQL errors. The Django test contains an unnecessary schema import suppressed with # noqa: F401.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[client.query called] --> B[Execute HTTP request]
    B --> C[Parse response into Response object]
    C --> D{assert_no_errors?}
    D -- "False / None" --> E[Return Response]
    D -- "True (default)" --> F{response.errors is None?}
    F -- "Yes" --> E
    F -- "No (before PR)" --> G["AssertionError (no message)"]
    F -- "No (after PR)" --> H["AssertionError(response.errors)\n→ errors list visible in output"]
Loading

Reviews (1): Last reviewed commit: "feat: make messages if there's assert_no..." | Re-trigger Greptile

Comment thread tests/test/test_client.py Outdated
Akay7 and others added 4 commits May 22, 2026 16:09
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
@codspeed-hq

codspeed-hq Bot commented May 25, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 34 untouched benchmarks


Comparing Akay7:testClientVerboseMessaages (1786f3b) with main (c3caabd)

Open in CodSpeed

Comment thread RELEASE.md Outdated

@bellini666 bellini666 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please adjust the failing tests

Comment thread RELEASE.md Outdated
release type: minor
---

Make `assert_no_errors` assertion failures report response errors for verbose output. When a test fails due to GraphQL errors, the assertion now includes the actual error details, making debugging easier.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

failures is a work that triggers alex pre-commit. Maybe use another word?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I slightly changed it, hope it works.

@Akay7
Akay7 requested a review from bellini666 June 12, 2026 02:33
Comment thread tests/test/test_client.py Outdated
Comment on lines +13 to +20
def check_non_existent_field_error(errors: Any):
assert isinstance(errors, list)
assert len(errors) == 1
error = errors[0]
assert isinstance(error, dict)
assert "nonExistentField" in error["message"]
assert "Cannot query field" in error["message"]
assert error["locations"]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather assert inline and not have a function like this 😊

and I'd reduce the number of assert to the bare minimum

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed that function and compare error with constant right away at each place where it needed.

@Akay7
Akay7 requested a review from patrick91 August 18, 2026 16:19

@bellini666 bellini666 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simple and effective. Thank you :)

@patrick91
patrick91 merged commit 06e6502 into strawberry-graphql:main Aug 30, 2026
86 checks passed
@botberry

Copy link
Copy Markdown
Member

This PR was published as 0.325.0. Thank you for contributing!

@Akay7
Akay7 deleted the testClientVerboseMessaages branch August 30, 2026 09:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants