Repository navigation
Quote lambdas that read a parameter of the lambda they are in - #20896
xperiandri wants to merge 12 commits into
Conversation
Every query test so far runs over AsQueryable() and compares rows, so a tree that only LINQ to Objects can run passes. These scenarios also check the tree: the query must stay on its provider, and must not contain what a C# expression lambda never does - a block, a parameter nothing binds, a call into FSharp.Core, a join nested in a collection selector, a member read from a tuple built without naming its members. A scenario names what is wrong with its tree today. The name has to be removed by the change that makes it untrue, because a scenario fails when its tree is cleaner than it says. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Selecting a struct tuple failed with "unreachable": the tuple was replaced by an anonymous object, but only reference tuples had an anonymous object type to be replaced by, and the way back always built a reference tuple. Both kinds now share the anonymous object of their arity, and the way back builds the kind the query asked for. Taking a struct tuple apart in a lambda failed with a tuple index out of range, because a struct tuple has fields where a reference tuple has properties. Fixes dotnet#15522 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
✅ Release notes checked
|
Covered casesEF Core SQLite translates 132 of 154 scenarios, 123 on #20895. Now translated: Unchanged: a method that takes an expression and reads nothing of the query still runs inside a query. |
There was a problem hiding this comment.
🟡 Changes recommended
Active findings show that let-binding rewrites can reorder side effects and mutate source structs instead of local copies.
4 open issues
What changed in this PR
This PR updates FSharp.Core’s LINQ conversion so nested lambdas that capture an enclosing lambda parameter reach providers as quoted expressions. It builds on the stacked changes for struct tuples and provider-friendly let bindings.
Changes:
- Quote correlated nested lambdas while preserving runtime conversion for independent or unsupported lambdas.
- Include struct-tuple support and selective
letsubstitution from the preceding changes. - Add expression-shape and evaluation tests, plus compatibility documentation.
| File | Description |
|---|---|
| tests/FSharp.Core.UnitTests/FSharp.Core/Microsoft.FSharp.Linq/QueryStructureScenarios.fs | Covers nested queries, loops, and left joins. |
| tests/FSharp.Core.UnitTests/FSharp.Core/Microsoft.FSharp.Linq/QueryShapeTests.fs | Runs the query scenario suites. |
| tests/FSharp.Core.UnitTests/FSharp.Core/Microsoft.FSharp.Linq/QueryScenarioDomain.fs | Defines shared query data and fixtures. |
| tests/FSharp.Core.UnitTests/FSharp.Core/Microsoft.FSharp.Linq/QueryScenario.fs | Checks query results and expression shapes. |
| tests/FSharp.Core.UnitTests/FSharp.Core/Microsoft.FSharp.Linq/QueryProviderShape.fs | Captures queries and detects problematic expression nodes. |
| tests/FSharp.Core.UnitTests/FSharp.Core/Microsoft.FSharp.Linq/QueryProjectionScenarios.fs | Covers tuple, record, and object projections. |
| tests/FSharp.Core.UnitTests/FSharp.Core/Microsoft.FSharp.Linq/QueryOperatorScenarios.fs | Covers filtering, ordering, joins, and aggregates. |
| tests/FSharp.Core.UnitTests/FSharp.Core/Microsoft.FSharp.Linq/QueryMethodSyntaxScenarios.fs | Covers LINQ method syntax and correlated lambdas. |
| tests/FSharp.Core.UnitTests/FSharp.Core/Microsoft.FSharp.Linq/QueryKeyScenarios.fs | Covers composite join and grouping keys. |
| tests/FSharp.Core.UnitTests/FSharp.Core/Microsoft.FSharp.Linq/ExpressionLambdaConversionTests.fs | Checks quoting, substitution, and evaluation semantics. |
| tests/FSharp.Core.UnitTests/FSharp.Core.UnitTests.fsproj | Registers the new test files. |
| src/FSharp.Core/QueryExtensions.fs | Preserves struct-tuple identity during query adaptation. |
| src/FSharp.Core/Query.fs | Integrates expression conversion and projection rewriting. |
| src/FSharp.Core/Linq.fsi | Declares shared internal conversion helpers. |
| src/FSharp.Core/Linq.fs | Implements nested-lambda quoting and selective substitution. |
| docs/release-notes/PENDING_BREAKING_CHANGES.md | Documents expression-shape and evaluation changes. |
| docs/release-notes/.FSharp.Core/11.0.200.md | Adds release notes for the LINQ fixes. |
Review effort: Balanced
| let reads = VariableReads (vP, value ()) | ||
| let replaced = reads.Visit bodyP | ||
|
|
||
| if reads.Count = 1 && readsFirst vP bodyP then |
There was a problem hiding this comment.
Fixed in 31c93a1 (#20895), covered here in c9c710b. A let of plain arithmetic (+, -, * on constants and reads) is now replaced by its value before the body is converted, so let n = x + 1 in LinqExpr.FromFunc(fun y -> y + n) quotes the nested lambda and has no block. A let that is not a read keeps its block and is evaluated once: let n = counter.Next() + x is a test. The operands are never arithmetic themselves, so a value built on another one does not grow with every let.
The shape tests followed only the first source of each operator. Each lambda parameter is now tied to the sequence it stands for, so the check also covers the inner input of a join, the second parameter of a join result selector, a collection selector of SelectMany, an identity Select, a chain inside the lambda of another operator and a query captured from earlier. A value scenario now has to end in a Queryable operator that gives the value: a count made over the rows of the provider leaves it. The validator itself is tested on trees built by hand. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A record or anonymous record with fields written out of order, a tupled lambda parameter, a struct copy such as x.Nullable.HasValue on a class property, or an explicit let: each of these is a let in the quotation, and the converter turned it into an Expression.Block with a local variable. No LINQ provider translates a block, so EF Core and Cosmos failed on all of these. Inside a lambda that goes to a provider - a query operator over IQueryable, or an F# lambda passed as Expression<Func<..>> - the value now takes the place of the variable where that computes the same: - the value only reads: a constant, a parameter of the lambda or a member of one, or what a member or method of a string, number or date makes of those. Such a read can be made at every use; - or the variable is read once, as the first thing the rest evaluates, so that the value is computed at the same moment as before. Any other let keeps its block and is evaluated once where it is written: a call read twice, read by a nested lambda or read after something else has run, an object that is created, a struct of the user's own or a boxed value, a mutable variable, and every let of a lambda that assigns or runs a statement. So does everything EvaluateQuotation and QuotationToExpression convert outside such a lambda. The lets the compiler adds for record fields written out of order are left out as a case of their own, in lambdas and, before the query translator looks at the record, in a select or join over IQueryable. The fields are then evaluated in declaration order, which is the one thing a lambda with side effects in its fields can notice. A join whose key is such a record on one side and comes ready-made from a function on the other keeps the key as written on both sides: the two no longer end up with different types. Fixes dotnet#11131 Fixes dotnet#15648 Fixes dotnet#11332 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A let is a record field that the compiler moved out of place only when its variable is named like the field whose place it takes. A let of the code with another name is evaluated first, like any other, even if a record follows it. A nullable is a plain value only when the type it holds is one, and the arguments of a method of a plain value must be plain too: a method can run code of the user's on an argument, once for every time it is made. A mutable let named like the copy the compiler makes of a struct gives way only for a plain value, so that a struct of the user's keeps its copy. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A let that adds, subtracts or multiplies constants and the data a lambda is given gives way to its value, like a let that only names a member: let next = p.Age + 1 no longer makes a block, and a lambda nested in the one that has it reads the value through the parameter. The operands are constants and reads, never arithmetic themselves, so a value that is built on another one does not grow with every let. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A lambda passed as Expression<Func<..>> inside another lambda that goes to a LINQ provider - db.Orders.Any(fun o -> o.CustomerId = c.Id) in a query where, a second for over db.Orders.Where(..) - reached the provider as a call that converts the inner lambda's quotation when the tree runs, with the values of the variables it captures: QuotationToLambdaExpression(SubstHelper(<quotation>, [c], [c])). A provider does not run the tree, so c never gets a value: EF Core failed on it and Cosmos rejected it. Such a lambda is now converted with the outer one and quoted, which is what the C# compiler emits for a nested expression lambda, so the provider sees an ordinary subquery whose parameter it knows. A nested lambda that reads no parameter of the outer lambda keeps the call. It depends on nothing the provider has to translate, so EF Core and Cosmos evaluate it before translating, as they do any captured value, and a method of the user's own that takes an expression keeps working inside a query. A nested lambda that cannot be converted keeps the call as well, whatever the conversion fails with: the call fails the same way, and only if the tree gets to run it. The scenario check stops reporting a call that a provider computes before it translates - one that holds no parameter, lambda, query or statement - and checks the lambda such a call gives. It also looks at the operator chains inside lambdas for members read from tuples built without naming them, which were only reported through the call before. Fixes dotnet#2758 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…t read a let A selector that the provider computes before it translates, which the compiler makes of a lambda that reads nothing of the tree, is evaluated and checked like a written one, and one that is not a lambda is passed over: the check no longer fails on `select (db.Orders.Select(fun o -> o.Id))`. Tests: that query as a scenario, a hand-built tree with a computed predicate, and a nested lambda that reads a let of arithmetic, which is quoted, and one that reads a let of arithmetic on a call, which is evaluated once. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
6486324 to
c9c710b
Compare


Fixes #2758
Stacked on #20895
db.Orders.Any(fun o -> o.CustomerId = c.Id)in a query'swhere, or a secondforoverdb.Orders.Where(fun o -> ..), now reaches the provider as an ordinary subquery instead of a call that converts the inner lambda when the tree runs.Details
🤖 Generated with Claude Code