Skip to content

Name the members of tuples and records built in expression trees - #20897

Open
xperiandri wants to merge 8 commits into
dotnet:mainfrom
xperiandri:fix/query-new-members
Open

xperiandri wants to merge 8 commits into
dotnet:mainfrom
xperiandri:fix/query-new-members

Conversation

@xperiandri

Copy link
Copy Markdown
Contributor

Stacked on #20894

A query composed over a tuple or record projection, such as a Where on the result of select (c.Id, c.Name), now translates in EF Core, and Cosmos accepts tuple, record and anonymous record projections.

Details
  • Tuples, struct tuples, records and anonymous records were built with Expression.New and no Members, so a provider could not tell which argument a later read of Item1 or of a field refers to. They now name their members, as C# anonymous types do.
  • A record whose getters are not public is built as before.
  • EF Core now translates the final tuple of a query on the server where it used to build it on the client. With EF Core 10 on SQL Server, a few multi-join queries that it translates with a lateral join then fail inside EF Core with "Nullable object must have a value".

🤖 Generated with Claude Code

xperiandri and others added 3 commits November 1, 2026 22:03
Every query test so far runs over AsQueryable() and compares rows, so a
tree that only LINQ to Objects can run passes. These scenarios also check
the tree: the query must stay on its provider, and must not contain what a
C# expression lambda never does - a block, a parameter nothing binds, a
call into FSharp.Core, a join nested in a collection selector, a member
read from a tuple built without naming its members.

A scenario names what is wrong with its tree today. The name has to be
removed by the change that makes it untrue, because a scenario fails when
its tree is cleaner than it says.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Selecting a struct tuple failed with "unreachable": the tuple was replaced
by an anonymous object, but only reference tuples had an anonymous object
type to be replaced by, and the way back always built a reference tuple.
Both kinds now share the anonymous object of their arity, and the way back
builds the kind the query asked for.

Taking a struct tuple apart in a lambda failed with a tuple index out of
range, because a struct tuple has fields where a reference tuple has
properties.

Fixes dotnet#15522

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings November 1, 2026 21:15
@xperiandri
xperiandri requested a review from a team as a code owner November 1, 2026 21:15
@github-actions

github-actions Bot commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

✅ Release notes checked


✅ Found changes and release notes in following paths:

Change path Release notes path Description
`src/FSharp.Core` docs/release-notes/.FSharp.Core/11.0.200.md

@xperiandri

Copy link
Copy Markdown
Contributor Author

Covered cases

EF Core SQLite translates 104 of 142 scenarios, 91 without this change. Now translated: Where, OrderBy and GroupBy reading a record or an anonymous record, a Where over a query that selects a record, a tuple, struct tuple or tuple of nine taken apart by the next for, a record or anonymous record read by the next for, and exists over two fors.

Cosmos translates 46 scenarios, up from 17: every tuple, struct tuple, record and anonymous record projection. With the other PRs of the stack merged in, these projections were also run against the Cosmos emulator and came back as the tuples and records the query selects.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Active issues leave unnamed-member detection incomplete for nested and identity-select query chains and break constructor conversion for public marker types without ItemN properties.

3 open issues
What changed in this PR

This PR improves FSharp.Core’s LINQ expression trees so providers can connect tuple and record member reads to projected constructor arguments. It builds on #20894’s struct-tuple support and query-shape tests.

Changes:

  • Name constructor members for tuples, struct tuples, records, and anonymous records, preserving the fallback for non-public record getters.
  • Carry struct-tuple identity through query conversion and support field-based element access.
  • Add projection assertions, query-shape scenarios, and release notes describing the expression-tree change.
File Description
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryStructureScenarios.fs Cover loops, subqueries, and left joins.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryShapeTests.fs Run the query scenario groups.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryScenarioDomain.fs Supply shared query data and projection types.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryScenario.fs Compare results and expected tree violations.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryProviderShape.fs Capture queries and detect provider-shape problems.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryProjectionScenarios.fs Cover tuple and record projections.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryOperatorScenarios.fs Cover query operators and aggregates.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryMethodSyntaxScenarios.fs Cover method-syntax projections and composition.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​QueryKeyScenarios.fs Cover composite join and grouping keys.
tests/​FSharp.Core.UnitTests/​FSharp.Core/​Microsoft.FSharp.Linq/​ExpressionNewMembersTests.fs Assert constructor-member metadata and private-record fallback.
tests/​FSharp.Core.UnitTests/​FSharp.Core.UnitTests.fsproj Register the new tests in compilation order.
src/​FSharp.Core/​QueryExtensions.fs Adapt struct tuples while preserving their identity.
src/​FSharp.Core/​Query.fs Reconstruct struct tuples in query results.
src/​FSharp.Core/​Linq.fs Emit named constructor members and struct-tuple field reads.
docs/​release-notes/​PENDING_BREAKING_CHANGES.md Document the changed expression-tree shape.
docs/​release-notes/​.FSharp.Core/​11.0.200.md Record struct-tuple and projection improvements.

Review effort: Balanced

Comment thread src/FSharp.Core/Linq.fs Resolved
xperiandri and others added 5 commits November 1, 2026 22:55
The shape tests followed only the first source of each operator. Each lambda
parameter is now tied to the sequence it stands for, so the check also covers
the inner input of a join, the second parameter of a join result selector,
a collection selector of SelectMany, an identity Select, a chain inside the
lambda of another operator and a query captured from earlier.

A value scenario now has to end in a Queryable operator that gives the value:
a count made over the rows of the provider leaves it. The validator itself is
tested on trees built by hand.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A tuple, struct tuple, record or anonymous record was built with
Expression.New(ctor, args) and no Members. A provider can materialize
such an object at the end of a query, but it cannot tell which
argument a later read of Item1 or of a field refers to, so EF Core
failed on any query composed over a tuple or record projection - a
Where on the result of select (c.Id, c.Name), or a record read by the
next for - and Cosmos rejected every such projection as a constructor
call.

The constructions now name the member each argument becomes, as C#
anonymous types do: the ItemN and Rest properties of System.Tuple, the
fields of System.ValueTuple, and the fields of a record, whose getters
have to be public for that; otherwise the record is built as before.
AnonymousObject members are taken in Item1..ItemN order rather than in
whatever order reflection returns them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
NewAnonymousObjectHelper is public, and a constructor of any type can be marked
with it. The members are named Item1, Item2 and so on only for the
AnonymousObject types of FSharp.Core; the constructor of any other type keeps
the properties its type has, as it did before.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@xperiandri
xperiandri force-pushed the fix/query-new-members branch from 7b27c59 to fc7ac0e Compare November 1, 2026 22:52

@pcshrosbree pcshrosbree left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the current head, nothing that EF Core translated on #20894 stops translating in my probes, and previously failing queries now translate with the right rows. Please clarify the dependencies and the SQL Server failures described below, and document and test the distinct fix. I checked EF Core 10.0.14 with SQLite execution and with SQL Server translation only.

  • The release note links #3782 as fixed, but its record example still fails with this PR alone. For a record declared { BId; BName }, the query select { BName = b.Name; BId = b.Id } throws NotSupportedException. The out-of-order fields produce a block, which this PR doesn't translate. {| Name = b.Name; Id = b.Id |} fails the same way. Both translate with #20895, which fixes them on its own. Please attribute this part of #3782 to #20895.
  • The description says some multi-join queries now fail on SQL Server with "Nullable object must have a value", but PENDING_BREAKING_CHANGES doesn't mention it. I couldn't check that execution failure without a server. The fifteen lateral and left-join shapes I tried compile to the same SQL as on #20894, apart from column aliases. A list of the affected shapes would help.
  • for (city, id) in (select (b.Address.City, b.Id)) do groupBy city into g; select (g.Key, g.Count()) used to fail with EF's "could not be translated"; now it fails inside EF Core with KeyNotFoundException: The given key 'Item1' was not present in the dictionary. With #20899 merged it translates, so this could be stated as a dependency.
  • This PR also fixes a wrong result. for (city, _) in (select (b.Address.City, b.Id)) do select city; distinct returns London; London; Paris on main, from SELECT DISTINCT "b"."Address_City", "b"."Id", and London; Paris here. It deserves a release-note line and a scenario that checks the rows.
  • The generated SQL now names its columns (SELECT "b"."Id" AS "Item1", ..), which SQL snapshot tests will notice.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: New

Development

Successfully merging this pull request may close these issues.

3 participants