Skip to content

Replace lets by their values in lambdas handed to a LINQ provider - #20895

Open
xperiandri wants to merge 13 commits into
dotnet:mainfrom
xperiandri:fix/query-inline-let
Open

xperiandri wants to merge 13 commits into
dotnet:mainfrom
xperiandri:fix/query-inline-let

Conversation

@xperiandri

@xperiandri xperiandri commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #11131
Fixes #15648
Fixes #11332

Stacked on #20894

A record or anonymous record with fields out of declaration order, a tupled lambda parameter, x.Created.HasValue on a class property and a let that names a member or is read once no longer reach LINQ providers as Expression.Block, which none of them translates.

Details
  • Inside a lambda that goes to a provider, the value of a let takes the place of the variable where that computes the same: the value only reads (a constant, the lambda's parameter or a member of it, or what a member or method of a string, number or date makes of those), or the variable is read once, as the first thing the rest of the lambda evaluates.
  • Every other let keeps its block and is evaluated once where it is written: a call read twice, by a nested lambda or after something else has run, an object that is created, a struct of your own or a boxed value, a mutable variable, and every let of a lambda that assigns or runs a statement.
  • The lets the compiler adds for record fields written out of order are treated like any other: a field whose value only reads is evaluated where it is used, and one that does more keeps its block, so the fields are evaluated in the order they are written.
  • A value that a call makes anew, such as p.Name.ToUpper(), keeps its block when it is used more than once and a use can tell one object from another, as obj.ReferenceEquals(upper, upper) does.
  • A join whose key one side builds as a tuple and the other side has ready-made no longer fails with "Type mismatch when building 'ty'".
  • EvaluateQuotation and QuotationToExpression outside such a lambda still produce a block.

🤖 Generated with Claude Code

xperiandri and others added 3 commits November 1, 2026 22:03
Every query test so far runs over AsQueryable() and compares rows, so a
tree that only LINQ to Objects can run passes. These scenarios also check
the tree: the query must stay on its provider, and must not contain what a
C# expression lambda never does - a block, a parameter nothing binds, a
call into FSharp.Core, a join nested in a collection selector, a member
read from a tuple built without naming its members.

A scenario names what is wrong with its tree today. The name has to be
removed by the change that makes it untrue, because a scenario fails when
its tree is cleaner than it says.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Selecting a struct tuple failed with "unreachable": the tuple was replaced
by an anonymous object, but only reference tuples had an anonymous object
type to be replaced by, and the way back always built a reference tuple.
Both kinds now share the anonymous object of their arity, and the way back
builds the kind the query asked for.

Taking a struct tuple apart in a lambda failed with a tuple index out of
range, because a struct tuple has fields where a reference tuple has
properties.

Fixes dotnet#15522

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings November 1, 2026 21:14
@xperiandri
xperiandri requested a review from a team as a code owner November 1, 2026 21:14
@github-actions

github-actions Bot commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

✅ Release notes checked


✅ Found changes and release notes in following paths:

Change path Release notes path Description
`src/FSharp.Core` docs/release-notes/.FSharp.Core/11.0.200.md

@xperiandri

Copy link
Copy Markdown
Contributor Author

Covered cases

EF Core SQLite translates 122 of 145 scenarios, 92 without this change. Now translated: records and anonymous records with fields out of order in select, in join, groupJoin and leftOuterJoin keys and in groupBy keys, in a query and in Select; struct anonymous records, CLIMutable records, an anonymous record inside an anonymous record, method calls in out-of-order fields; Select on a nullable property; a let in where, select and exists; a string method or a count in a let used twice; nested lets; a Where over a query built earlier with a let.

Cosmos translates 26 scenarios, up from 17.

linq2db returned wrong rows for OrderBy and GroupBy with a tupled lambda parameter, and returns the right ones now.

Not changed: a call that is not a member of a string, number or date keeps its block when a let binds it and it is read twice, as in let orders = c.Orders.Count() in orders + orders.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Critical substitution issues can reorder explicit-let evaluation, mutate struct fields instead of local copies, and duplicate side effects through provider callbacks or custom nullable structs.

4 open issues
What changed in this PR

This PR improves FSharp.Core’s LINQ-provider compatibility by replacing eligible let bindings with their values instead of emitting Expression.Block. It builds on #20894’s struct-tuple support and query-shape tests to address record-field ordering, tupled parameters, and nullable-property access.

Changes:

  • Substitute eligible bindings inside provider expression lambdas while retaining blocks for guarded cases.
  • Add query-shape and evaluation-semantics regression coverage.
  • Document expression-tree changes and their evaluation-order impact.
File Description
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryStructureScenarios.fs Covers loops, subqueries, and left joins.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryShapeTests.fs Runs the query scenario suites.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryScenarioDomain.fs Supplies shared query data and types.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryScenario.fs Checks results, provider retention, and tree shape.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryProviderShape.fs Captures queries and detects unsupported tree patterns.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryProjectionScenarios.fs Covers tuple and record projections.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryOperatorScenarios.fs Covers filtering, ordering, and aggregates.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryMethodSyntaxScenarios.fs Covers direct LINQ method calls.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryKeyScenarios.fs Covers composite join and grouping keys.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​ExpressionLambdaConversionTests.fs Tests substitution, evaluation order, and identity.
tests/​FSharp.Core.UnitTests/​FSharp.Core.UnitTests.fsproj Registers the new test files.
src/​FSharp.Core/​QueryExtensions.fs Adapts struct tuples to query representations.
src/​FSharp.Core/​Query.fs Integrates provider-specific conversion and record-field inlining.
src/​FSharp.Core/​Linq.fsi Declares internal conversion helpers.
src/​FSharp.Core/​Linq.fs Implements binding substitution and struct-tuple access.
docs/​release-notes/​PENDING_BREAKING_CHANGES.md Documents tree-shape and evaluation-order changes.
docs/​release-notes/​.FSharp.Core/​11.0.200.md Adds LINQ compatibility release notes.

Review effort: Balanced

Comment thread src/FSharp.Core/Linq.fs Resolved
Comment thread src/FSharp.Core/Linq.fs Resolved
Comment thread src/FSharp.Core/Linq.fs Resolved
Comment thread src/FSharp.Core/Linq.fs Resolved
The shape tests followed only the first source of each operator. Each lambda
parameter is now tied to the sequence it stands for, so the check also covers
the inner input of a join, the second parameter of a join result selector,
a collection selector of SelectMany, an identity Select, a chain inside the
lambda of another operator and a query captured from earlier.

A value scenario now has to end in a Queryable operator that gives the value:
a count made over the rows of the provider leaves it. The validator itself is
tested on trees built by hand.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@xperiandri
xperiandri force-pushed the fix/query-inline-let branch from ac05f7d to 31c93a1 Compare November 1, 2026 22:52

@pcshrosbree pcshrosbree left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the current head, the #11332 predicate gives the same result as plain F#, and its expression tree contains no blocks. Please expand PENDING_BREAKING_CHANGES and add tests for the remaining changes in when and how often expressions run. I ran the expression trees through AsQueryable() and compared the results with main and plain F#.

In the record-field cases I checked, user-written let bindings to Counter.Next() now keep their block unless the bound variable has the same name as the field that uses it. The example below shows that exception: the call bound to B still runs after the call for A.

let B = Counter.Next() in { A = Counter.Next(); B = B }
// This PR:  { A = 1; B = 2 }
// Plain F#: { A = 2; B = 1 }

The note could mention this remaining reordering.

PENDING_BREAKING_CHANGES:

  • A member read substituted into a nested Func lambda can run once per inner element. With names a List<string>, custs.Select(fun c -> let n = c.Name in names.Count(fun s -> s = n)) reads Name 8 times for 2 customers and 4 names; main reads it twice. This also affects arithmetic expressions bound by let, such as let a = x.Age + 1, when a nested lambda uses the bound value. In that probe, Age was read 4 times, compared with once on main.
  • The note says EvaluateQuotation and QuotationToExpression are unaffected. But QuotationToLambdaExpression changes for every caller, and EvaluateQuotation <@ ppl.Select(fun x -> let n = x.Name in n + n).ToArray() @> reads Name twice where main reads it once.
  • A let binding that would throw can now be skipped when a conditional takes a branch that does not use the bound value. let s = x.Name.Substring(5) in if x.Name.Length > 5 then s else "" returns "" for "ab", where F# throws ArgumentOutOfRangeException.

Tests. Please add tests that count reads of a getter on the lambda's parameter, including a case where a nested lambda uses the getter's bound value. "Member of an object read before a call keeps the value it had" passes because items is a captured constant. If items is replaced with the lambda's parameter, the result is 1 on main and 2 on this PR.

The join-key fallback in Query.fs also fixes join p in ps on ((k.KA, k.KB) = p.Pair), which throws Type mismatch when building 'ty' on main; it could have a release-note line and a test.

@T-Gro T-Gro left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖🕵️ AI review — verify independently.

Comment thread src/FSharp.Core/Linq.fs
let isPlainValueMethod (call: MethodCallExpression) =
match call.Object with
| null -> false
| _ -> isPlainValueType call.Method.DeclaringType && isPlainValueType call.Type

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖🕵️ [P2] Lost string identity after duplicating a call, even without getters or side effects.

open System
open Microsoft.FSharp.Linq.RuntimeHelpers
let f = LeafExpressionConverter.QuotationToLambdaExpression
            <@ Func<string, bool>(fun s ->
                let upper = s.ToUpperInvariant()
                obj.ReferenceEquals(upper, upper)) @>
f.Compile().Invoke "ab" // base: true; this head: false

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 4360ae3. A let whose value a call makes anew is no longer replaced at each of several uses when one of them can tell one object from another: the visitor ObjectIdentityUses looks at what the rest of the lambda does with the variable, so obj.ReferenceEquals(upper, upper) keeps its block and is true. Uses that only look at the string, like upper.StartsWith("A") && upper.EndsWith("B"), still take the value in. The tests are String made by a call is one string where a use can tell one from another and String made by a call is made at each use that only looks at it.

@T-Gro T-Gro Nov 2, 2026 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖🕵️ Still loses string identity at 9e58c229 through an identity-preserving call or a single captured use in a nested lambda.

open System
open System.Linq
open Microsoft.FSharp.Linq.RuntimeHelpers

let f = LeafExpressionConverter.QuotationToLambdaExpression
            <@ Func<string, bool>(fun s ->
                let upper = s.ToUpperInvariant()
                obj.ReferenceEquals(upper.Substring(0), upper.Substring(0))) @>
f.Compile().Invoke "ab" // base: true; 9e58c229: false

let g = LeafExpressionConverter.QuotationToLambdaExpression
            <@ Func<string, Collections.Generic.IEnumerable<string>>(fun s ->
                let upper = s.ToUpperInvariant()
                Enumerable.Select([| 1; 2 |], Func<int, string>(fun _ -> upper))) @>
let items = g.Compile().Invoke("ab").ToArray()
obj.ReferenceEquals(items.[0], items.[1]) // base: true; 9e58c229: false

xperiandri and others added 9 commits November 2, 2026 14:24
A value scenario was accepted when the last execution at the provider was an
operator that gives a value. A count made by the client over rows taken from
the provider passed whenever a function it ran asked the provider for a count
of its own, because that count came last. Now every execution has to be such
an operator, and so does the enumeration of a source taken as it is, which is
recorded as an execution too.

The violations of a tree are compared with every occurrence: a tree with one
block was equal to a tree with two. The scenarios that have several of a kind
are pinned with their number.

The validator tests build a tree with an invoked lambda and one with an
unbound parameter, which no scenario has, and the two client counts.

The release note names the error a struct tuple gave on main.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…y what they hold

The shape tests did not see an unnamed element that passes through a result
selector giving one of its parameters back, in SelectMany, Join and Zip, and
they skipped the element parameter of Aggregate together with the accumulated
value. Both are checked now. A function with a seed takes the accumulated value
and an element, and one without a seed takes two elements.

A scenario compared an entity by the text it makes of itself, which names its
key only: a result with the right key and the wrong name or age passed. An
object of a class of the scenarios is now compared by every property that gives
a plain value.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A record or anonymous record with fields written out of order, a tupled
lambda parameter, a struct copy such as x.Nullable.HasValue on a class
property, or an explicit let: each of these is a let in the quotation,
and the converter turned it into an Expression.Block with a local
variable. No LINQ provider translates a block, so EF Core and Cosmos
failed on all of these.

Inside a lambda that goes to a provider - a query operator over
IQueryable, or an F# lambda passed as Expression<Func<..>> - the value
now takes the place of the variable where that computes the same:

- the value only reads: a constant, a parameter of the lambda or a
  member of one, or what a member or method of a string, number or date
  makes of those. Such a read can be made at every use;
- or the variable is read once, as the first thing the rest evaluates,
  so that the value is computed at the same moment as before.

Any other let keeps its block and is evaluated once where it is
written: a call read twice, read by a nested lambda or read after
something else has run, an object that is created, a struct of the
user's own or a boxed value, a mutable variable, and every let of a
lambda that assigns or runs a statement. So does everything
EvaluateQuotation and QuotationToExpression convert outside such a
lambda.

The lets the compiler adds for record fields written out of order are
left out as a case of their own, in lambdas and, before the query
translator looks at the record, in a select or join over IQueryable.
The fields are then evaluated in declaration order, which is the one
thing a lambda with side effects in its fields can notice.

A join whose key is such a record on one side and comes ready-made from
a function on the other keeps the key as written on both sides: the two
no longer end up with different types.

Fixes dotnet#11131
Fixes dotnet#15648
Fixes dotnet#11332

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A let is a record field that the compiler moved out of place only when its
variable is named like the field whose place it takes. A let of the code with
another name is evaluated first, like any other, even if a record follows it.

A nullable is a plain value only when the type it holds is one, and the
arguments of a method of a plain value must be plain too: a method can run
code of the user's on an argument, once for every time it is made. A mutable
let named like the copy the compiler makes of a struct gives way only for a
plain value, so that a struct of the user's keeps its copy.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A let that adds, subtracts or multiplies constants and the data a lambda is
given gives way to its value, like a let that only names a member:
let next = p.Age + 1 no longer makes a block, and a lambda nested in the
one that has it reads the value through the parameter. The operands are
constants and reads, never arithmetic themselves, so a value that is built on
another one does not grow with every let.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The lets that the compiler adds for record fields written out of order were
left out whatever their value was, so a field with a side effect was evaluated
after the fields that follow it: { Second = counter.Next(); First =
counter.Next() } gave First = 1, and so did a let written by hand and named
like the field it feeds. These lets are ordinary lets now. The construction
of the anonymous object that stands for the record stays behind them, so a
field that only reads still reaches the provider without a block.

A let of a call that makes a string was replaced by the call at every use, so
obj.ReferenceEquals(upper, upper) was false. It is replaced when it is used
once, or when every use only looks at what the string holds.

A division of reads by a constant other than 0 and -1 is a read as well, so the
lets of a key like { KeyAge = c.Age / 10; KeyCity = c.City } stay out of the
query.

The note on breaking changes lists what else changes when and how often a read
runs: in a nested lambda, after a call that changes its source, and on a path
that does not use it. It says what EvaluateQuotation does with a let inside a
lambda that goes to a provider. The release note names the join key that is
built on one side only, and the part of dotnet#3782 that this fixes.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A record selected with its fields written out of order is converted like one
with its fields in order, and distinct compares the converted objects by their
fields. For a record type with an Equals of its own that was already so when the
fields were in order, and the test pins both orders to the same result. The
breaking changes name it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@xperiandri
xperiandri force-pushed the fix/query-inline-let branch from 31c93a1 to 9e58c22 Compare November 2, 2026 18:57
xperiandri added a commit to xperiandri/fsharp that referenced this pull request Nov 2, 2026
The note no longer credits dotnet#3782: the record with fields out of order is
fixed by dotnet#20895. It names the distinct that returned a city once for each id,
because the provider applied DISTINCT to both columns of the earlier
projection. The note on breaking changes says that the SQL now names its
columns, and that a groupBy on an item of a selected tuple needs dotnet#20899.

A scenario takes one item of a tuple apart in the next for and applies
distinct to it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@xperiandri

Copy link
Copy Markdown
Contributor Author

Thanks @pcshrosbree. Changes since 31c93a1:

  • Record fields. The let the compiler adds for a field it moves is no longer told apart from one written by hand: either is replaced only when its value is a read, and otherwise keeps its block. { A = Counter.Next(); B = B } after let B = Counter.Next() gives { A = 2; B = 1 } as F# does, and so does a field that is not named like its variable. Tests: Let written by hand is evaluated before the fields that follow it, Let named like the record field it feeds is evaluated before the fields that follow it, and their AsQueryable() variants.
  • Nested lambdas. PENDING_BREAKING_CHANGES.md says that a read taken into a nested lambda runs for each inner element, arithmetic included, with your names.Count(fun s -> s = n) example. Member of the parameter that a nested lambda uses is read for each element of the nested lambda counts the reads of a getter on the lambda's parameter.
  • EvaluateQuotation. The claim that it is unaffected is gone. A let inside a lambda that goes to a provider is replaced however the quotation is converted, and EvaluateQuotation replaces a let inside a lambda that goes to a provider pins your n + n example at two reads.
  • Skipped branch. The Substring example is in the notes, and Let whose value would throw is not evaluated on a path that does not use it pins it. A division by a parameter keeps its block, so its failure is not skipped.
  • items. Member of the parameter that a let names is read after a call that changes it uses the lambda's parameter and gives 2, with the 1 of main in the notes.
  • Join key. join p in ps on ((k.KA, k.KB) = p.Pair) has a line in the release note and the test Join of a key built on one side with a tuple property on the other. The record case of Query builder with EF core can't return fsharp tuples (and sometimes new objects) using async methods #3782 is credited to this PR.
  • A record with a type of its own for Equals and fields written out of order is converted like one with its fields in order, which distinct shows; see 9e58c22 and the notes.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: New

4 participants