Update codeflash and run it with benchmarks

[patrick91]

Description

Types of Changes

• Core
• Bugfix
• New feature
• Enhancement/optimization
• Documentation

Issues Fixed or Closed by This PR

Checklist

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have read the CONTRIBUTING document.
• I have added tests to cover my changes.
• I have tested the changes and verified that they work and don't break anything (as well as I can manage).

Summary by Sourcery

Update the codeflash GitHub Actions workflow to run with benchmarking and standardize YAML quoting

CI:

• Use double quotes consistently in the workflow file
• Add the --benchmark flag to the codeflash run step in CI

[sourcery-ai]

Reviewer's Guide

This PR refines the Codeflash GitHub workflow by standardizing YAML formatting (consistent quoting and removing stray whitespace) and enhancing the Codeflash step to run with benchmarking enabled.

File-Level Changes

Change	Details	Files
Standardize YAML formatting and cleanup	Changed single quotes to double quotes for path and Python version Removed an extra blank line before the jobs section	.github/workflows/codeflash.yaml
Enable benchmarks in Codeflash step	Added the --benchmark flag to the poetry run codeflash command	.github/workflows/codeflash.yaml

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Hey @patrick91 - I've reviewed your changes - here's some feedback:

• Consider adding a caching step (e.g., actions/cache for pip or poetry) to speed up dependency installation in the workflow.
• Since the workflow only tests on Python 3.11, you might use a matrix to validate codeflash against all supported Python versions.
• The path filter is limited to “strawberry/**” — ensure this covers every file that impacts codeflash (e.g., config or benchmark files), or broaden the filter if needed.

Prompt for AI Agents

Please address the comments from this code review:
## Overall Comments
- Consider adding a caching step (e.g., actions/cache for pip or poetry) to speed up dependency installation in the workflow.
- Since the workflow only tests on Python 3.11, you might use a matrix to validate codeflash against all supported Python versions.
- The path filter is limited to “strawberry/**” — ensure this covers every file that impacts codeflash (e.g., config or benchmark files), or broaden the filter if needed.

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[greptile-apps]

Greptile Summary

This PR enhances the Codeflash GitHub Actions workflow by enabling benchmark mode for AI-powered code optimization. The main functional change adds the --benchmark flag to the codeflash command execution, which instructs the tool to perform more comprehensive performance measurements during the optimization process.

Codeflash is an AI-powered tool that automatically optimizes Python code performance while verifying correctness through testing. The workflow triggers on pull requests that modify files in the strawberry directory and uses concurrency control to ensure only the latest run executes for each PR. The enhanced benchmarking capability should provide more accurate performance measurements and potentially lead to better optimization decisions.

The PR also includes minor formatting improvements: standardizing YAML quotes from single to double quotes for consistency and removing an extra blank line. These cosmetic changes likely came from automatic linting/formatting tools.

PR Description Notes:

• The PR description is incomplete - all sections are empty and checkboxes are unchecked
• No description of the changes or rationale is provided
• The template appears to be left unfilled

Confidence score: 4/5

• This is a low-risk enhancement to an existing CI workflow that adds a performance optimization flag
• The change is minimal and well-contained, affecting only the codeflash command execution
• No files need additional attention as the change is straightforward and the workflow structure remains intact

_{1 file reviewed, no comments}

_{Edit Code Review Bot Settings | Greptile}

[botberry]

Apollo Federation Subgraph Compatibility Results

Federation 1 Support	Federation 2 Support
_service 🟢 @key (single) 🟢 @key (multi) 🟢 @key (composite) 🟢 repeatable @key 🟢 @requires 🟢 @provides 🟢 federated tracing 🔲	@link 🟢 @shareable 🟢 @tag 🟢 @override 🟢 @inaccessible 🟢 @composeDirective 🟢 @interfaceObject 🟢

Learn more:

• Apollo Federation Subgraph Specification
• Compatibility Tests

[codspeed-hq]

CodSpeed Performance Report

Merging #3963 will create unknown performance changes

_{Comparing feature/codeflash-bench (8f332e5) with main (e151370)}

Summary

⁉️ 26 dropped benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

	Benchmark	BASE	HEAD	Change
⁉️	test_convert_argument_large_list[1048576]	9.1 ms	N/A	N/A
⁉️	test_convert_argument_large_list[16384]	197.5 µs	N/A	N/A
⁉️	test_convert_argument_large_list[262144]	2.3 ms	N/A	N/A
⁉️	test_convert_argument_large_list[4194304]	36.3 ms	N/A	N/A
⁉️	test_convert_argument_large_list[65536]	613.6 µs	N/A	N/A
⁉️	test_execute_complex_schema[50]	3.9 s	N/A	N/A
⁉️	test_execute	529.8 ms	N/A	N/A
⁉️	test_interface_performance[16]	58.9 ms	N/A	N/A
⁉️	test_interface_performance[1]	58.6 ms	N/A	N/A
⁉️	test_interface_performance[256]	60.9 ms	N/A	N/A
⁉️	test_interface_performance[4096]	67.6 ms	N/A	N/A
⁉️	test_execute_with_1000_items	66.6 ms	N/A	N/A
⁉️	test_execute_with_100_items	10.8 ms	N/A	N/A
⁉️	test_execute_with_10_items	5.3 ms	N/A	N/A
⁉️	test_execute_with_many_fields	37.5 ms	N/A	N/A
⁉️	test_execute_with_many_fields_and_directives	42.9 ms	N/A	N/A
⁉️	test_execute[with_no_extensions-items_10000]	626.9 ms	N/A	N/A
⁉️	test_execute[with_no_extensions-items_1000]	67.1 ms	N/A	N/A
⁉️	test_execute[with_resolveextension-items_10000]	1.9 s	N/A	N/A
⁉️	test_execute[with_resolveextension-items_1000]	183.7 ms	N/A	N/A
...	...	...	...	...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.40%. Comparing base (e151370) to head (8f332e5).
⚠️ Report is 18 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3963      +/-   ##
==========================================
- Coverage   94.41%   94.40%   -0.01%     
==========================================
  Files         528      528              
  Lines       34371    34371              
  Branches     1803     1803              
==========================================
- Hits        32450    32449       -1     
  Misses       1630     1630              
- Partials      291      292       +1     

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[misrasaurabh1]

@patrick91 Once you merge this in, I will monitor the CI to ensure it is working correctly

[patrick91]

@misrasaurabh1 thanks! I need to check what happened with codspeed though as it doesn't report the benchmarks anymore 🤔

[bellini666]

@patrick91 should we merge this? :)

[misrasaurabh1]

yes please! Also now we discourage locking the codeflash version and instead suggest installing the latest version in the CI. Would be great if you can do that, so that you don't need to update the version in the future, codeflash updates quickly. We have optimizations for async coming out very soon.