Skip to content

Quote lambdas that read a parameter of the lambda they are in - #20896

Open
xperiandri wants to merge 12 commits into
dotnet:mainfrom
xperiandri:fix/query-nested-lambda
Open

xperiandri wants to merge 12 commits into
dotnet:mainfrom
xperiandri:fix/query-nested-lambda

Conversation

@xperiandri

Copy link
Copy Markdown
Contributor

Fixes #2758

Stacked on #20895

db.Orders.Any(fun o -> o.CustomerId = c.Id) in a query's where, or a second for over db.Orders.Where(fun o -> ..), now reaches the provider as an ordinary subquery instead of a call that converts the inner lambda when the tree runs.

Details
  • A lambda nested in a lambda that goes to a provider is converted together with it and quoted, as the C# compiler does, when it reads a parameter of the outer lambda.
  • A nested lambda that reads none keeps the run-time call. It depends on nothing the provider translates, so EF Core and Cosmos evaluate it up front like any captured value, and a method of your own that takes an expression keeps working inside a query.
  • A nested lambda that cannot be converted keeps the call as well, whatever the conversion fails with, and fails only if the tree gets to run it.

🤖 Generated with Claude Code

xperiandri and others added 3 commits November 1, 2026 22:03
Every query test so far runs over AsQueryable() and compares rows, so a
tree that only LINQ to Objects can run passes. These scenarios also check
the tree: the query must stay on its provider, and must not contain what a
C# expression lambda never does - a block, a parameter nothing binds, a
call into FSharp.Core, a join nested in a collection selector, a member
read from a tuple built without naming its members.

A scenario names what is wrong with its tree today. The name has to be
removed by the change that makes it untrue, because a scenario fails when
its tree is cleaner than it says.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Selecting a struct tuple failed with "unreachable": the tuple was replaced
by an anonymous object, but only reference tuples had an anonymous object
type to be replaced by, and the way back always built a reference tuple.
Both kinds now share the anonymous object of their arity, and the way back
builds the kind the query asked for.

Taking a struct tuple apart in a lambda failed with a tuple index out of
range, because a struct tuple has fields where a reference tuple has
properties.

Fixes dotnet#15522

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings November 1, 2026 21:15
@xperiandri
xperiandri requested a review from a team as a code owner November 1, 2026 21:15
@github-actions

github-actions Bot commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

✅ Release notes checked


✅ Found changes and release notes in following paths:

Change path Release notes path Description
`src/FSharp.Core` docs/release-notes/.FSharp.Core/11.0.200.md

@xperiandri

Copy link
Copy Markdown
Contributor Author

Covered cases

EF Core SQLite translates 132 of 154 scenarios, 123 on #20895. Now translated: where, Select and SelectMany with a lambda over another source, a second for over a source filtered by a lambda, All and Count over another source, a nested lambda that reads a let naming a member, and a lambda nested two levels deep.

Unchanged: a method that takes an expression and reads nothing of the query still runs inside a query.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Active findings show that let-binding rewrites can reorder side effects and mutate source structs instead of local copies.

4 open issues
What changed in this PR

This PR updates FSharp.Core’s LINQ conversion so nested lambdas that capture an enclosing lambda parameter reach providers as quoted expressions. It builds on the stacked changes for struct tuples and provider-friendly let bindings.

Changes:

  • Quote correlated nested lambdas while preserving runtime conversion for independent or unsupported lambdas.
  • Include struct-tuple support and selective let substitution from the preceding changes.
  • Add expression-shape and evaluation tests, plus compatibility documentation.
File Description
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryStructureScenarios.fs Covers nested queries, loops, and left joins.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryShapeTests.fs Runs the query scenario suites.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryScenarioDomain.fs Defines shared query data and fixtures.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryScenario.fs Checks query results and expression shapes.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryProviderShape.fs Captures queries and detects problematic expression nodes.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryProjectionScenarios.fs Covers tuple, record, and object projections.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryOperatorScenarios.fs Covers filtering, ordering, joins, and aggregates.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryMethodSyntaxScenarios.fs Covers LINQ method syntax and correlated lambdas.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryKeyScenarios.fs Covers composite join and grouping keys.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​ExpressionLambdaConversionTests.fs Checks quoting, substitution, and evaluation semantics.
tests/​FSharp.Core.UnitTests/​FSharp.Core.UnitTests.fsproj Registers the new test files.
src/​FSharp.Core/​QueryExtensions.fs Preserves struct-tuple identity during query adaptation.
src/​FSharp.Core/​Query.fs Integrates expression conversion and projection rewriting.
src/​FSharp.Core/​Linq.fsi Declares shared internal conversion helpers.
src/​FSharp.Core/​Linq.fs Implements nested-lambda quoting and selective substitution.
docs/​release-notes/​PENDING_BREAKING_CHANGES.md Documents expression-shape and evaluation changes.
docs/​release-notes/​.FSharp.Core/​11.0.200.md Adds release notes for the LINQ fixes.

Review effort: Balanced

Comment thread src/FSharp.Core/Linq.fs Resolved
Comment thread src/FSharp.Core/Linq.fs Resolved
Comment thread src/FSharp.Core/Linq.fs
Comment on lines +1063 to +1066
let reads = VariableReads (vP, value ())
let replaced = reads.Visit bodyP

if reads.Count = 1 && readsFirst vP bodyP then

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 31c93a1 (#20895), covered here in c9c710b. A let of plain arithmetic (+, -, * on constants and reads) is now replaced by its value before the body is converted, so let n = x + 1 in LinqExpr.FromFunc(fun y -> y + n) quotes the nested lambda and has no block. A let that is not a read keeps its block and is evaluated once: let n = counter.Next() + x is a test. The operands are never arithmetic themselves, so a value built on another one does not grow with every let.

xperiandri and others added 9 commits November 1, 2026 22:55
The shape tests followed only the first source of each operator. Each lambda
parameter is now tied to the sequence it stands for, so the check also covers
the inner input of a join, the second parameter of a join result selector,
a collection selector of SelectMany, an identity Select, a chain inside the
lambda of another operator and a query captured from earlier.

A value scenario now has to end in a Queryable operator that gives the value:
a count made over the rows of the provider leaves it. The validator itself is
tested on trees built by hand.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A record or anonymous record with fields written out of order, a tupled
lambda parameter, a struct copy such as x.Nullable.HasValue on a class
property, or an explicit let: each of these is a let in the quotation,
and the converter turned it into an Expression.Block with a local
variable. No LINQ provider translates a block, so EF Core and Cosmos
failed on all of these.

Inside a lambda that goes to a provider - a query operator over
IQueryable, or an F# lambda passed as Expression<Func<..>> - the value
now takes the place of the variable where that computes the same:

- the value only reads: a constant, a parameter of the lambda or a
  member of one, or what a member or method of a string, number or date
  makes of those. Such a read can be made at every use;
- or the variable is read once, as the first thing the rest evaluates,
  so that the value is computed at the same moment as before.

Any other let keeps its block and is evaluated once where it is
written: a call read twice, read by a nested lambda or read after
something else has run, an object that is created, a struct of the
user's own or a boxed value, a mutable variable, and every let of a
lambda that assigns or runs a statement. So does everything
EvaluateQuotation and QuotationToExpression convert outside such a
lambda.

The lets the compiler adds for record fields written out of order are
left out as a case of their own, in lambdas and, before the query
translator looks at the record, in a select or join over IQueryable.
The fields are then evaluated in declaration order, which is the one
thing a lambda with side effects in its fields can notice.

A join whose key is such a record on one side and comes ready-made from
a function on the other keeps the key as written on both sides: the two
no longer end up with different types.

Fixes dotnet#11131
Fixes dotnet#15648
Fixes dotnet#11332

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A let is a record field that the compiler moved out of place only when its
variable is named like the field whose place it takes. A let of the code with
another name is evaluated first, like any other, even if a record follows it.

A nullable is a plain value only when the type it holds is one, and the
arguments of a method of a plain value must be plain too: a method can run
code of the user's on an argument, once for every time it is made. A mutable
let named like the copy the compiler makes of a struct gives way only for a
plain value, so that a struct of the user's keeps its copy.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A let that adds, subtracts or multiplies constants and the data a lambda is
given gives way to its value, like a let that only names a member:
let next = p.Age + 1 no longer makes a block, and a lambda nested in the
one that has it reads the value through the parameter. The operands are
constants and reads, never arithmetic themselves, so a value that is built on
another one does not grow with every let.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A lambda passed as Expression<Func<..>> inside another lambda that goes
to a LINQ provider - db.Orders.Any(fun o -> o.CustomerId = c.Id) in a
query where, a second for over db.Orders.Where(..) - reached the
provider as a call that converts the inner lambda's quotation when the
tree runs, with the values of the variables it captures:
QuotationToLambdaExpression(SubstHelper(<quotation>, [c], [c])). A
provider does not run the tree, so c never gets a value: EF Core failed
on it and Cosmos rejected it.

Such a lambda is now converted with the outer one and quoted, which is
what the C# compiler emits for a nested expression lambda, so the
provider sees an ordinary subquery whose parameter it knows.

A nested lambda that reads no parameter of the outer lambda keeps the
call. It depends on nothing the provider has to translate, so EF Core
and Cosmos evaluate it before translating, as they do any captured
value, and a method of the user's own that takes an expression keeps
working inside a query.

A nested lambda that cannot be converted keeps the call as well,
whatever the conversion fails with: the call fails the same way, and
only if the tree gets to run it.

The scenario check stops reporting a call that a provider computes
before it translates - one that holds no parameter, lambda, query or
statement - and checks the lambda such a call gives. It also looks at
the operator chains inside lambdas for members read from tuples built
without naming them, which were only reported through the call before.

Fixes dotnet#2758

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…t read a let

A selector that the provider computes before it translates, which the compiler
makes of a lambda that reads nothing of the tree, is evaluated and checked like
a written one, and one that is not a lambda is passed over: the check no longer
fails on `select (db.Orders.Select(fun o -> o.Id))`.

Tests: that query as a scenario, a hand-built tree with a computed predicate,
and a nested lambda that reads a let of arithmetic, which is quoted, and one
that reads a let of arithmetic on a call, which is evaluated once.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@xperiandri
xperiandri force-pushed the fix/query-nested-lambda branch from 6486324 to c9c710b Compare November 1, 2026 22:52

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: New

Development

Successfully merging this pull request may close these issues.

Nested Linq expression trees result in quotation related calls

2 participants