Skip to content

Check query trees for LINQ providers and translate struct tuples - #20894

Open
xperiandri wants to merge 6 commits into
dotnet:mainfrom
xperiandri:fix/query-struct-tuples
Open

xperiandri wants to merge 6 commits into
dotnet:mainfrom
xperiandri:fix/query-struct-tuples

Conversation

@xperiandri

@xperiandri xperiandri commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #15522

Selecting a struct tuple in a query no longer fails with "unreachable", and taking one apart in a lambda passed as Expression<Func<..>> no longer fails with "Struct tuples store their elements in fields, not properties".

Details
  • Query tests ran over AsQueryable() and compared rows, so any tree LINQ to Objects can run passed, including trees no SQL provider translates. The scenario tests added here also check the tree the provider receives against what a C# expression lambda can contain: the query stays on its provider, and there is no block, no unbound parameter, no call into FSharp.Core, no join nested in a collection selector, and no member read from a tuple built without naming its members.
  • The struct tuple lines in Linq.fs do what Convert struct tuple patterns in LeafExpressionConverter #20886 does, and go from this PR when that one merges.
  • Each scenario lists the problems its tree has today and fails when one of them disappears or another appears, so each follow-up PR removes exactly the entries it makes untrue.

🤖 Generated with Claude Code

xperiandri and others added 2 commits November 1, 2026 22:03
Every query test so far runs over AsQueryable() and compares rows, so a
tree that only LINQ to Objects can run passes. These scenarios also check
the tree: the query must stay on its provider, and must not contain what a
C# expression lambda never does - a block, a parameter nothing binds, a
call into FSharp.Core, a join nested in a collection selector, a member
read from a tuple built without naming its members.

A scenario names what is wrong with its tree today. The name has to be
removed by the change that makes it untrue, because a scenario fails when
its tree is cleaner than it says.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Selecting a struct tuple failed with "unreachable": the tuple was replaced
by an anonymous object, but only reference tuples had an anonymous object
type to be replaced by, and the way back always built a reference tuple.
Both kinds now share the anonymous object of their arity, and the way back
builds the kind the query asked for.

Taking a struct tuple apart in a lambda failed with a tuple index out of
range, because a struct tuple has fields where a reference tuple has
properties.

Fixes dotnet#15522

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings November 1, 2026 21:10
@xperiandri
xperiandri requested a review from a team as a code owner November 1, 2026 21:10
@github-actions

github-actions Bot commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

✅ Release notes checked


✅ Found changes and release notes in following paths:

Change path Release notes path Description
`src/FSharp.Core` docs/release-notes/.FSharp.Core/11.0.200.md

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@xperiandri

Copy link
Copy Markdown
Contributor Author

Covered cases

Checked against EF Core SQLite (SQL executed, rows compared) and the Cosmos LINQ provider (SQL text), with the scenario files of this PR.

EF Core SQLite translates 91 of 128 scenarios, 87 with main's FSharp.Core. Now translated: a struct tuple of two, of nine, inside a tuple, and a tuple inside a struct tuple.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Validation gaps miss nested or captured query violations and client-side terminal execution, while incorrect Join selector-to-input mapping causes false positives and missed violations.

3 open issues
What changed in this PR

This PR addresses #15522 by supporting struct tuples in FSharp.Core query translation and LINQ expression lambdas. It also adds scenario tests that check expression-tree structure alongside query results.

Changes:

  • Preserve struct tuple types through query conversion and read their fields when translating tuple access.
  • Add query scenarios covering projections, operators, keys, nested queries, joins, and method syntax, with explicit expected violations.
  • Document the struct tuple fixes in the release notes.
File Description
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryStructureScenarios.fs Adds loop, nested-query, and left-join scenarios.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryShapeTests.fs Registers scenario groups as parameterized tests.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryScenarioDomain.fs Defines shared entities and query data.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryScenario.fs Compares results, tree violations, and operator chains.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryProviderShape.fs Captures provider expressions and checks tree structure.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryProjectionScenarios.fs Covers tuple, record, and other projections.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryOperatorScenarios.fs Covers sequence and scalar query operators.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryMethodSyntaxScenarios.fs Covers direct LINQ calls and tuple destructuring.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryKeyScenarios.fs Covers join and grouping key shapes.
tests/​FSharp.Core.UnitTests/​FSharp.Core.UnitTests.fsproj Includes the new test files in compilation order.
src/​FSharp.Core/​QueryExtensions.fs Supports struct tuples in anonymous-object conversion.
src/​FSharp.Core/​Query.fs Restores struct tuples when reconstructing query results.
src/​FSharp.Core/​Linq.fs Translates struct tuple access through fields and nested Rest values.
docs/​release-notes/​.FSharp.Core/​11.0.200.md Documents the struct tuple query and expression-tree fixes.

Review effort: Balanced

The shape tests followed only the first source of each operator. Each lambda
parameter is now tied to the sequence it stands for, so the check also covers
the inner input of a join, the second parameter of a join result selector,
a collection selector of SelectMany, an identity Select, a chain inside the
lambda of another operator and a query captured from earlier.

A value scenario now has to end in a Queryable operator that gives the value:
a count made over the rows of the provider leaves it. The validator itself is
tested on trees built by hand.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

@pcshrosbree pcshrosbree left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At 526db26 the struct tuple cases work, reference tuples and records are unchanged, and the validator now catches the nested, captured and Join cases Copilot raised. Please address the remaining validator and release-note gaps.

  • Violations are compared as sets of kinds, so a tree with one block and a tree with two compare equal. The comparison catches a violation kind disappearing, but not a drop in its count.
  • No scenario exercises UnboundParameter or LambdaInvocation: with either check removed, all scenarios still pass. LeftProvider is now covered by one of the new validator tests, but by no scenario.
  • The release note says struct tuples failed with a tuple index out of range. On main the error is ArgumentException: Struct tuples store their elements in fields, not properties.

The Linq.fs change implements the same logic as my #20886, which fixes #20884. My tests there also cover EvaluateQuotation, 16-element tuples with two levels of Rest, and let struct on a named parameter. I can move those tests here, or keep them in #20886 and remove the overlapping code from that PR, whichever you prefer.

@T-Gro T-Gro left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 🕵️ AI review — verify independently.

Comment thread src/FSharp.Core/Linq.fs
// A struct tuple holds its elements in fields, which PreComputeTuplePropertyInfo cannot return
if ty.IsValueType then
if n < 7 then
Expression.Field(argP, ty.GetField("Item" + string (n + 1))) |> asExpr

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 🕵️ [P1] Queryable returns [22] instead of native/Enumerable [21] — Bump mutates the tuple field rather than a defensive copy.

open System.Linq

[<Struct>]
type Counter =
    val mutable Value: int
    new(v) = { Value = v }
    member x.Bump() =
        x.Value <- x.Value + 1
        x.Value

[<ReflectedDefinition>]
let project (struct (counter: Counter, marker: int)) =
    counter.Bump() + counter.Value + marker

let rows = [struct (Counter(5), 10)]
rows |> List.map project // [21]
query { for row in rows do select (project row) } |> Seq.toList // [21]
query { for row in rows.AsQueryable() do select (project row) } |> Seq.toList // [22]

@pcshrosbree pcshrosbree Nov 2, 2026 •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The missing defensive copy is a bug on main already, without struct tuples, so I'll open a separate issue for it. I checked on main and on my #20886, which has the same struct-tuple arm. On main, Queryable gives [12] for a struct bound by let, where F# gives [11]:

[<ReflectedDefinition>]
let projectLet (n: int) =
    let counter = Counter(n)
    counter.Bump() + counter.Value
F# and Enumerable Queryable on main Queryable on #20886 and this PR
your example [21] ArgumentException [22]
projectLet 5 [11] [12] [12]

F# calls Bump on a copy of the immutable counter, while the converted tree calls it on the variable itself. On main your example threw before reaching that point, so the struct-tuple arm exposes the missing copy rather than causing it.

Update: I've opened #20908 for the missing defensive copy, with a fix in #20909. I tested #20886 with that fix applied, and your example gives [21].

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed with @pcshrosbree: this is not about struct tuples. On main a struct bound by let gives [12] where F# gives [11], because the converted tree calls the method on the variable and not on a copy. The struct tuple arm only reaches that code now, so I leave the copy to the separate issue that @pcshrosbree is opening.

@T-Gro T-Gro added the AI-reviewed PR reviewed by AI review council label Nov 2, 2026
@T-Gro
T-Gro self-requested a review November 2, 2026 12:40
xperiandri and others added 2 commits November 2, 2026 14:24
A value scenario was accepted when the last execution at the provider was an
operator that gives a value. A count made by the client over rows taken from
the provider passed whenever a function it ran asked the provider for a count
of its own, because that count came last. Now every execution has to be such
an operator, and so does the enumeration of a source taken as it is, which is
recorded as an execution too.

The violations of a tree are compared with every occurrence: a tree with one
block was equal to a tree with two. The scenarios that have several of a kind
are pinned with their number.

The validator tests build a tree with an invoked lambda and one with an
unbound parameter, which no scenario has, and the two client counts.

The release note names the error a struct tuple gave on main.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…y what they hold

The shape tests did not see an unnamed element that passes through a result
selector giving one of its parameters back, in SelectMany, Join and Zip, and
they skipped the element parameter of Aggregate together with the accumulated
value. Both are checked now. A function with a seed takes the accumulated value
and an element, and one without a seed takes two elements.

A scenario compared an entity by the text it makes of itself, which names its
key only: a result with the right key and the wrong name or age passed. An
object of a class of the scenarios is now compared by every property that gives
a plain value.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@xperiandri

Copy link
Copy Markdown
Contributor Author

Thanks @pcshrosbree. Changes since 526db26:

  • Violations are compared with every occurrence (521e830): a tree that loses one of two blocks no longer passes. The scenarios that have several of a kind are pinned with their number.
  • UnboundParameter and LambdaInvocation have facts with hand-built trees, and LeftProvider has scenarios for both client counts.
  • b418d09 closes two gaps the review bot found in the same validator: a result selector that gives an unnamed element back, and the element parameter of Aggregate. Scenarios now compare an entity by every plain property and not by its key alone.
  • The release note names the error of main, ArgumentException: Struct tuples store their elements in fields, not properties.

#20886 stays yours. The struct tuple lines in Linq.fs go from this PR when it merges, and I rebase on it then.

@T-Gro T-Gro left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 🕵️ Re-reviewed all recent changes as of b418d09; LGTM

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI-reviewed PR reviewed by AI review council

Projects

Status: New

Development

Successfully merging this pull request may close these issues.

[Query] expression does not support [ValueTuple]

4 participants