Repository navigation
Check query trees for LINQ providers and translate struct tuples - #20894
xperiandri wants to merge 6 commits into
Conversation
Every query test so far runs over AsQueryable() and compares rows, so a tree that only LINQ to Objects can run passes. These scenarios also check the tree: the query must stay on its provider, and must not contain what a C# expression lambda never does - a block, a parameter nothing binds, a call into FSharp.Core, a join nested in a collection selector, a member read from a tuple built without naming its members. A scenario names what is wrong with its tree today. The name has to be removed by the change that makes it untrue, because a scenario fails when its tree is cleaner than it says. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Selecting a struct tuple failed with "unreachable": the tuple was replaced by an anonymous object, but only reference tuples had an anonymous object type to be replaced by, and the way back always built a reference tuple. Both kinds now share the anonymous object of their arity, and the way back builds the kind the query asked for. Taking a struct tuple apart in a lambda failed with a tuple index out of range, because a struct tuple has fields where a reference tuple has properties. Fixes dotnet#15522 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
✅ Release notes checked
|
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Covered casesChecked against EF Core SQLite (SQL executed, rows compared) and the Cosmos LINQ provider (SQL text), with the scenario files of this PR. EF Core SQLite translates 91 of 128 scenarios, 87 with main's FSharp.Core. Now translated: a struct tuple of two, of nine, inside a tuple, and a tuple inside a struct tuple. |
There was a problem hiding this comment.
🟡 Changes recommended
Validation gaps miss nested or captured query violations and client-side terminal execution, while incorrect Join selector-to-input mapping causes false positives and missed violations.
3 open issues
What changed in this PR
This PR addresses #15522 by supporting struct tuples in FSharp.Core query translation and LINQ expression lambdas. It also adds scenario tests that check expression-tree structure alongside query results.
Changes:
- Preserve struct tuple types through query conversion and read their fields when translating tuple access.
- Add query scenarios covering projections, operators, keys, nested queries, joins, and method syntax, with explicit expected violations.
- Document the struct tuple fixes in the release notes.
| File | Description |
|---|---|
| tests/FSharp.Core.UnitTests/FSharp.Core/Microsoft.FSharp.Linq/QueryStructureScenarios.fs | Adds loop, nested-query, and left-join scenarios. |
| tests/FSharp.Core.UnitTests/FSharp.Core/Microsoft.FSharp.Linq/QueryShapeTests.fs | Registers scenario groups as parameterized tests. |
| tests/FSharp.Core.UnitTests/FSharp.Core/Microsoft.FSharp.Linq/QueryScenarioDomain.fs | Defines shared entities and query data. |
| tests/FSharp.Core.UnitTests/FSharp.Core/Microsoft.FSharp.Linq/QueryScenario.fs | Compares results, tree violations, and operator chains. |
| tests/FSharp.Core.UnitTests/FSharp.Core/Microsoft.FSharp.Linq/QueryProviderShape.fs | Captures provider expressions and checks tree structure. |
| tests/FSharp.Core.UnitTests/FSharp.Core/Microsoft.FSharp.Linq/QueryProjectionScenarios.fs | Covers tuple, record, and other projections. |
| tests/FSharp.Core.UnitTests/FSharp.Core/Microsoft.FSharp.Linq/QueryOperatorScenarios.fs | Covers sequence and scalar query operators. |
| tests/FSharp.Core.UnitTests/FSharp.Core/Microsoft.FSharp.Linq/QueryMethodSyntaxScenarios.fs | Covers direct LINQ calls and tuple destructuring. |
| tests/FSharp.Core.UnitTests/FSharp.Core/Microsoft.FSharp.Linq/QueryKeyScenarios.fs | Covers join and grouping key shapes. |
| tests/FSharp.Core.UnitTests/FSharp.Core.UnitTests.fsproj | Includes the new test files in compilation order. |
| src/FSharp.Core/QueryExtensions.fs | Supports struct tuples in anonymous-object conversion. |
| src/FSharp.Core/Query.fs | Restores struct tuples when reconstructing query results. |
| src/FSharp.Core/Linq.fs | Translates struct tuple access through fields and nested Rest values. |
| docs/release-notes/.FSharp.Core/11.0.200.md | Documents the struct tuple query and expression-tree fixes. |
Review effort: Balanced
The shape tests followed only the first source of each operator. Each lambda parameter is now tied to the sequence it stands for, so the check also covers the inner input of a join, the second parameter of a join result selector, a collection selector of SelectMany, an identity Select, a chain inside the lambda of another operator and a query captured from earlier. A value scenario now has to end in a Queryable operator that gives the value: a count made over the rows of the provider leaves it. The validator itself is tested on trees built by hand. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
pcshrosbree
left a comment
There was a problem hiding this comment.
At 526db26 the struct tuple cases work, reference tuples and records are unchanged, and the validator now catches the nested, captured and Join cases Copilot raised. Please address the remaining validator and release-note gaps.
- Violations are compared as sets of kinds, so a tree with one block and a tree with two compare equal. The comparison catches a violation kind disappearing, but not a drop in its count.
- No scenario exercises
UnboundParameterorLambdaInvocation: with either check removed, all scenarios still pass.LeftProvideris now covered by one of the new validator tests, but by no scenario. - The release note says struct tuples failed with a tuple index out of range. On main the error is
ArgumentException: Struct tuples store their elements in fields, not properties.
The Linq.fs change implements the same logic as my #20886, which fixes #20884. My tests there also cover EvaluateQuotation, 16-element tuples with two levels of Rest, and let struct on a named parameter. I can move those tests here, or keep them in #20886 and remove the overlapping code from that PR, whichever you prefer.
T-Gro
left a comment
There was a problem hiding this comment.
🤖 🕵️ AI review — verify independently.
| // A struct tuple holds its elements in fields, which PreComputeTuplePropertyInfo cannot return | ||
| if ty.IsValueType then | ||
| if n < 7 then | ||
| Expression.Field(argP, ty.GetField("Item" + string (n + 1))) |> asExpr |
There was a problem hiding this comment.
🤖 🕵️ [P1] Queryable returns [22] instead of native/Enumerable [21] — Bump mutates the tuple field rather than a defensive copy.
open System.Linq
[<Struct>]
type Counter =
val mutable Value: int
new(v) = { Value = v }
member x.Bump() =
x.Value <- x.Value + 1
x.Value
[<ReflectedDefinition>]
let project (struct (counter: Counter, marker: int)) =
counter.Bump() + counter.Value + marker
let rows = [struct (Counter(5), 10)]
rows |> List.map project // [21]
query { for row in rows do select (project row) } |> Seq.toList // [21]
query { for row in rows.AsQueryable() do select (project row) } |> Seq.toList // [22]There was a problem hiding this comment.
The missing defensive copy is a bug on main already, without struct tuples, so I'll open a separate issue for it. I checked on main and on my #20886, which has the same struct-tuple arm. On main, Queryable gives [12] for a struct bound by let, where F# gives [11]:
[<ReflectedDefinition>]
let projectLet (n: int) =
let counter = Counter(n)
counter.Bump() + counter.Value| F# and Enumerable | Queryable on main | Queryable on #20886 and this PR | |
|---|---|---|---|
| your example | [21] |
ArgumentException |
[22] |
projectLet 5 |
[11] |
[12] |
[12] |
F# calls Bump on a copy of the immutable counter, while the converted tree calls it on the variable itself. On main your example threw before reaching that point, so the struct-tuple arm exposes the missing copy rather than causing it.
Update: I've opened #20908 for the missing defensive copy, with a fix in #20909. I tested #20886 with that fix applied, and your example gives [21].
There was a problem hiding this comment.
Agreed with @pcshrosbree: this is not about struct tuples. On main a struct bound by let gives [12] where F# gives [11], because the converted tree calls the method on the variable and not on a copy. The struct tuple arm only reaches that code now, so I leave the copy to the separate issue that @pcshrosbree is opening.
A value scenario was accepted when the last execution at the provider was an operator that gives a value. A count made by the client over rows taken from the provider passed whenever a function it ran asked the provider for a count of its own, because that count came last. Now every execution has to be such an operator, and so does the enumeration of a source taken as it is, which is recorded as an execution too. The violations of a tree are compared with every occurrence: a tree with one block was equal to a tree with two. The scenarios that have several of a kind are pinned with their number. The validator tests build a tree with an invoked lambda and one with an unbound parameter, which no scenario has, and the two client counts. The release note names the error a struct tuple gave on main. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…y what they hold The shape tests did not see an unnamed element that passes through a result selector giving one of its parameters back, in SelectMany, Join and Zip, and they skipped the element parameter of Aggregate together with the accumulated value. Both are checked now. A function with a seed takes the accumulated value and an element, and one without a seed takes two elements. A scenario compared an entity by the text it makes of itself, which names its key only: a result with the right key and the wrong name or age passed. An object of a class of the scenarios is now compared by every property that gives a plain value. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
Thanks @pcshrosbree. Changes since 526db26:
#20886 stays yours. The struct tuple lines in |

Fixes #15522
Selecting a struct tuple in a query no longer fails with "unreachable", and taking one apart in a lambda passed as
Expression<Func<..>>no longer fails with "Struct tuples store their elements in fields, not properties".Details
AsQueryable()and compared rows, so any tree LINQ to Objects can run passed, including trees no SQL provider translates. The scenario tests added here also check the tree the provider receives against what a C# expression lambda can contain: the query stays on its provider, and there is no block, no unbound parameter, no call into FSharp.Core, no join nested in a collection selector, and no member read from a tuple built without naming its members.Linq.fsdo what Convert struct tuple patterns in LeafExpressionConverter #20886 does, and go from this PR when that one merges.🤖 Generated with Claude Code