Repository navigation
Fix expression compiler compatibility - #558
jogibear9988 wants to merge 188 commits into
Conversation
|
Maaaaaan, a lot of Visitors and seems a lot of allocations too. |
Allocation/performance measurements and independent reviewFollowing @dadhi's concern about visitors and allocations, I compared the current patch against both its parent and current upstream, and reviewed the production code independently of the earlier test-fixing work. The allocation concern is justified. I recommend keeping this draft while the common-path regressions and the newly reproduced correctness issue below are addressed. Passing the imported test suite alone is not sufficient evidence for merging this change. Revisions: upstream Main observations
Independent review findings
The added visitor families are feature-gated. ArgumentSpillScope scans runtime Expression invocation/Dynamic arguments; QuoteVariableScope examines quotes, and QuoteBinder rebuilds captured quotes at execution. They are not all run on every ordinary CompileFast call. There is still duplicated feature-specific traversal, linear bound/free-variable searches, and quote rebuilding allocation. Unconditional scope bookkeeping and broad capture representation have measurable costs even where those visitors do not run. Exception-region objects are also allocated for handler trees without jumps. A blanket visitor rewrite is not justified by this review; remove duplicate work and isolate common-path costs first. Source locations at the reviewed head:
Measurement scope and methodWindows ARM64 virtual machine (Apple Silicon host, six exposed logical processors), SDK 10.0.301; .NET 10.0.9 and 8.0.28, Release, native ARM64. Production source snapshots were compiled unchanged in isolated dependency-free library projects with the appropriate target-framework and LIGHT_EXPRESSION symbols. The main checkout and PR head were not changed. 31 scenario shapes: 28 work on both baselines/head, and 3 newly supported shapes (closed/captured quotes and RuntimeVariables) are reported without a before/after ratio. Each backend's observable outputs are checked against System compilation for inputs -2, 0 and 3 before timing; returned captures/quotes are executed during verification and RuntimeVariables get/set is checked. This is a targeted control, not exhaustive semantic equivalence. Strict FEC is used with no outer fallback. Measured separately: tree construction (there is no text parser here), compilation of a reused tree, compilation plus first outer execution, and warm invocation. For escaping captures, first-use/warm measurements also create and call the returned delegate; for quotes/RuntimeVariables, they create/return the object, while returned quote compilation is only part of correctness verification. System uses its IL compiler, not the interpreter. Light trees are constructed natively, and equivalent System trees are prepared before System compilation timing; Light -> System conversion is excluded from that comparison. The RuntimeVariables Light factory is unavailable publicly, so that one construction case uses System construction + ToLightExpression and is explicitly not a native-Light construction comparison. Two fresh-process rounds per version, forward then reverse version order, seven warmed samples per phase, adaptive batches targeting ~8ms (caps 2,000 compile/construction or 1,000,000 invocation operations). TieredCompilation and ReadyToRun are disabled consistently. GC is forced outside each timed sample. Allocations use This measures managed allocation volume, not retained heap size or native/JIT memory; static initialization and application cold startup are warmed away. First execution means the newly created method's first call in a warmed process. It does not claim all possible expression graphs, Framework/x64, or production workload performance were measured. No whole imported correctness rerun was needed for this read-only review; the independent reproduction above adds a known issue beyond that historical baseline. Complete measured phase resultsIn the tables, B -> A is parent -> head, .NET 10.0.9, standard: all 31 scenarios, all four phases
.NET 10.0.9, light: all 31 scenarios, all four phases
.NET 8.0.28, standard: all 31 scenarios, all four phases
.NET 8.0.28, light: all 31 scenarios, all four phases
ReproductionsFresh Dynamic reduction reproduction (.NET 8/10, Standard FEC)using System;
using System.Dynamic;
using System.Linq.Expressions;
using FastExpressionCompiler;
class FreshDynamic : Expression {
public int Reductions;
readonly Expression argument;
readonly bool cache; Expression cached;
public FreshDynamic(Expression a,bool cache=false){argument=a;this.cache=cache;}
public override Type Type=>typeof(int);
public override ExpressionType NodeType=>ExpressionType.Extension;
public override bool CanReduce=>true;
public override Expression Reduce(){Reductions++;if(cache&&cached!=null)return cached;return cached=Convert(Dynamic(new AddBinder(),typeof(object),argument,Constant(2)),typeof(int));}
}
class AddBinder : BinaryOperationBinder {
public AddBinder():base(ExpressionType.Add){}
public override DynamicMetaObject FallbackBinaryOperation(DynamicMetaObject target,DynamicMetaObject arg,DynamicMetaObject errorSuggestion)=>new DynamicMetaObject(Expression.Convert(Expression.Add(Expression.Convert(target.Expression,typeof(int)),Expression.Convert(arg.Expression,typeof(int))),typeof(object)),BindingRestrictions.GetTypeRestriction(target.Expression,typeof(int)).Merge(BindingRestrictions.GetTypeRestriction(arg.Expression,typeof(int))));
}
class Program {
static void Main(){
foreach(var flags in new[]{CompilerFlags.Default,CompilerFlags.NoInvocationLambdaInlining}) {
var x=Expression.Parameter(typeof(int));var extension=new FreshDynamic(x);var tree=Expression.Lambda<Func<int,int>>(extension,x);
try {var f=tree.CompileFast(true,flags);Console.WriteLine($"FEC {flags}: {(f==null?"null":f(5).ToString())}; reductions={extension.Reductions}");}catch(Exception e){Console.WriteLine($"FEC {flags}: reductions={extension.Reductions}; "+e);}
Console.WriteLine("System: "+tree.Compile()(5));
}
var p=Expression.Parameter(typeof(int));var direct=Expression.Lambda<Func<int,int>>(Expression.Convert(Expression.Dynamic(new AddBinder(),typeof(object),p,Expression.Constant(2)),typeof(int)),p);
var result=direct.CompileFast(true);Console.WriteLine("Direct Dynamic FEC: "+(result==null?"null":result(5).ToString()));
var stable=Expression.Lambda<Func<int,int>>(new FreshDynamic(p,true),p).CompileFast(true);
Console.WriteLine("Cached reduction FEC: "+(stable==null?"null":stable(5).ToString()));
}
}Head: NRE at Dynamic metadata lookup, reductions=2, under both flags. System/direct/cached controls return 7. Parent/upstream: null, not an exception. Inherited #555 integration control (Light)using System;
using System.Linq.Expressions;
using FastExpressionCompiler.LightExpression;
using E=System.Linq.Expressions.Expression;
class Program {
static void Main(){
var p=E.Parameter(typeof(decimal));
var tree=E.Lambda<Func<decimal,decimal>>(E.AddAssign(p,E.Constant(1m),typeof(decimal).GetMethod("op_Addition",new[]{typeof(decimal),typeof(decimal)}),null),p);
Console.WriteLine("System: "+tree.Compile()(2m));
var light=tree.ToLightExpression();Console.WriteLine("FEC direct: "+light.CompileFast(true)(2m));
try {var roundtrip=(System.Linq.Expressions.Expression<Func<decimal,decimal>>)light.ToExpression();Console.WriteLine("Roundtrip: "+roundtrip.Compile()(2m));}catch(Exception e){Console.WriteLine("Roundtrip: "+e);}
}
}Head/parent: System=3, direct FEC=3, round-trip=NRE. Upstream: all three return 3. Reproducing the measurementsThe source harness below is shared across versions. For Standard, compile it against the four production Set Benchmark harness and snapshot build scriptsusing System;
using System.Linq;
using System.Collections.Generic;
using System.Diagnostics;
using System.Runtime.CompilerServices;
using System.Text.Json;
#if LIGHT
using FastExpressionCompiler.LightExpression;
using E = FastExpressionCompiler.LightExpression.Expression;
using X = FastExpressionCompiler.LightExpression.Expression;
using L = FastExpressionCompiler.LightExpression.LambdaExpression;
#else
using FastExpressionCompiler;
using E = System.Linq.Expressions.Expression;
using X = System.Linq.Expressions.Expression;
using L = System.Linq.Expressions.LambdaExpression;
#endif
public class Box { public int Value; public Box(int x) { Value=x; } public int Add(int x)=>Value+x; }
public static class Program {
static object sink; static long checksum;
#if LIGHT
static L systemSource;
static System.Linq.Expressions.LambdaExpression systemPrepared;
#endif
record Scenario(string Name, Func<L> Build);
static X Balanced(X x,int depth) => depth==0?x:E.Add(Balanced(x,depth-1),Balanced(x,depth-1));
static L Build(string name) {
var x=E.Parameter(typeof(int),"x"); X body=x;
switch(name) {
case "constant": body=E.Constant(42);break;
case "add": body=E.Add(x,E.Constant(1));break;
case "arithmetic64": for(int i=0;i<64;i++)body=E.Add(body,E.Constant(i));break;
case "balanced256":body=Balanced(x,8);break;
case "conditional":body=E.Condition(E.LessThan(x,E.Constant(0)),E.Negate(x),x);break;
case "nullable":body=E.Coalesce(E.Convert(x,typeof(int?)),E.Constant(7));break;
case "new_member":body=E.Field(E.New(typeof(Box).GetConstructor(new[]{typeof(int)}),x),"Value");break;
case "constant_object":body=E.Call(E.Constant(new Box(7)),typeof(Box).GetMethod("Add"),x);break;
case "array":body=E.ArrayIndex(E.Constant(new[]{2,3,5,7}),E.And(x,E.Constant(3)));break;
case "new_array":body=E.ArrayLength(E.NewArrayInit(typeof(int),x,x,x));break;
case "interned_string":body=E.Property(E.Constant("fixed literal"),"Length");break;
case "noninterned_string":body=E.Property(E.Constant(new string('z',37)),"Length");break;
case "try_catch":body=E.TryCatch(E.Divide(E.Constant(42),x),E.Catch(typeof(DivideByZeroException),E.Constant(-1)));break;
case "try_finally":body=E.TryFinally(E.Add(x,E.Constant(1)),E.Empty());break;
case "switch":body=E.Switch(x,E.Constant(8),E.SwitchCase(E.Constant(3),E.Constant(0)),E.SwitchCase(E.Constant(4),E.Constant(1)));break;
case "goto":{var t=E.Label(typeof(int));body=E.Block(E.Return(t,x),E.Label(t,E.Constant(0)));break;}
case "loop":{var v=E.Variable(typeof(int));var t=E.Label(typeof(int));body=E.Block(new[]{v},E.Assign(v,E.Constant(0)),E.Loop(E.IfThenElse(E.LessThan(v,E.Constant(8)),E.Block(E.PostIncrementAssign(v),E.Empty()),E.Break(t,v)),t));break;}
case "blocks32":case "blocks128":{int n=name=="blocks32"?32:128;var arr=new List<X>();for(int i=0;i<n;i++){var v=E.Variable(typeof(int),"v"+i);arr.Add(E.Block(new[]{v},E.Assign(v,x),E.Add(v,E.Constant(i))));}body=E.Block(arr);break;}
case "invoke_literal":{var p=E.Parameter(typeof(int));body=E.Invoke(E.Lambda<Func<int,int>>(E.Add(p,E.Constant(2)),p),x);break;}
case "invoke_delegate":body=E.Invoke(E.Constant((Func<int,int>)(v=>v+2)),x);break;
case "invoke_try_argument":{var p=E.Parameter(typeof(int));body=E.Invoke(E.Lambda<Func<int,int>>(E.Add(p,E.Constant(2)),p),E.TryFinally(x,E.Empty()));break;}
case "capture1": return E.Lambda<Func<int,Func<int>>>(E.Lambda<Func<int>>(E.Add(x,E.Constant(1))),x);
case "capture_reference":{var v=E.Variable(typeof(Box),"box");return E.Lambda<Func<int,Func<int>>>(E.Block(new[]{v},E.Assign(v,E.Constant(new Box(7))),E.Lambda<Func<int>>(E.Field(v,"Value"))),x);}
case "nested_blocks8":case "nested_blocks32":case "nested_blocks128":{int count=int.Parse(name.Substring("nested_blocks".Length));for(int i=0;i<count;i++)body=E.Block(body);break;}
case "capture8":{var vs=Enumerable.Range(0,8).Select(i=>E.Variable(typeof(int),"v"+i)).ToArray();X sum=x;foreach(var v in vs)sum=E.Add(sum,v);return E.Lambda<Func<int,Func<int>>>(E.Block(vs,vs.Select(v=>(X)E.Assign(v,x)).Concat(new X[]{E.Lambda<Func<int>>(sum)})),x);}
case "quote_closed":return E.Lambda<Func<int,System.Linq.Expressions.Expression<Func<int>>>>(E.Quote(E.Lambda<Func<int>>(E.Constant(9))),x);
case "quote_capture":return E.Lambda<Func<int,System.Linq.Expressions.Expression<Func<int>>>>(E.Quote(E.Lambda<Func<int>>(x)),x);
case "runtime_variables":
#if LIGHT
var sp=System.Linq.Expressions.Expression.Parameter(typeof(int));
return System.Linq.Expressions.Expression.Lambda<Func<int,IRuntimeVariables>>(System.Linq.Expressions.Expression.RuntimeVariables(sp),sp).ToLightExpression();
#else
return E.Lambda<Func<int,IRuntimeVariables>>(E.RuntimeVariables(x),x);
#endif
default:throw new Exception(name);
}
return E.Lambda<Func<int,int>>(body,x);
}
static Delegate Compile(L tree,bool system) {
if(system) {
#if LIGHT
if(!ReferenceEquals(systemSource,tree)){systemSource=tree;systemPrepared=(System.Linq.Expressions.LambdaExpression)tree.ToExpression();}
return systemPrepared.Compile();
#else
return tree.Compile();
#endif
}
return tree.CompileFast(true)??throw new NotSupportedException("strict FEC returned null");
}
static string Verify(Delegate d) {
var parts=new List<string>();
foreach(int v in new[]{-2,0,3}) {
var result=d.DynamicInvoke(v);
if(result is Func<int> nested) result=nested();
if(result is System.Linq.Expressions.Expression<Func<int>> quote) result=quote.Compile()();
if(result is IRuntimeVariables vars) {var initial=vars[0];vars[0]=17;result=initial+":"+vars[0];}
parts.Add(result?.ToString()??"null");
}
return string.Join("|",parts);
}
static Action<int> Driver(Delegate d) {
if(d is Func<int,int> f)return n=>{long s=0;for(int i=0;i<n;i++)s+=f(3);checksum=s;};
if(d is Func<int,Func<int>> f2)return n=>{long s=0;for(int i=0;i<n;i++){var v=f2(3);sink=v;s+=v();}checksum=s;};
if(d is Func<int,System.Linq.Expressions.Expression<Func<int>>> fq)return n=>{for(int i=0;i<n;i++)sink=fq(3);};
if(d is Func<int,IRuntimeVariables> fr)return n=>{for(int i=0;i<n;i++)sink=fr(3);};
throw new Exception("driver "+d.GetType());
}
record Sample(string Version,string Runtime,string Flavor,string Scenario,string Phase,int Round,int SampleIndex,int Operations,double NsPerOp,double BytesPerOp,int Gen0,int Gen1,int Gen2);
static void Measure(string version,string flavor,string scenario,string phase,int round,Action<int> op) {
op(30);
int n=phase=="invoke"?10000:20;
var sw=Stopwatch.StartNew();op(n);sw.Stop();
n=(int)Math.Clamp(n*8.0/Math.Max(sw.Elapsed.TotalMilliseconds,0.01),phase=="invoke"?1000:10,phase=="invoke"?1000000:2000);
for(int k=0;k<7;k++) {
GC.Collect();GC.WaitForPendingFinalizers();GC.Collect();
long allocated=GC.GetAllocatedBytesForCurrentThread();int g0=GC.CollectionCount(0),g1=GC.CollectionCount(1),g2=GC.CollectionCount(2);
long start=Stopwatch.GetTimestamp();op(n);long end=Stopwatch.GetTimestamp();
long bytes=GC.GetAllocatedBytesForCurrentThread()-allocated;
Console.WriteLine(JsonSerializer.Serialize(new Sample(version,Environment.Version.ToString(),flavor,scenario,phase,round,k,n,(end-start)*1e9/Stopwatch.Frequency/n,bytes/(double)n,GC.CollectionCount(0)-g0,GC.CollectionCount(1)-g1,GC.CollectionCount(2)-g2)));
}
}
public static void Main(string[] args) {
string version=args[0],flavor=args[1];int round=int.Parse(args[2]);bool system=version=="system";
string[] names={"constant","add","arithmetic64","balanced256","conditional","nullable","new_member","constant_object","array","new_array","interned_string","noninterned_string","try_catch","try_finally","switch","goto","loop","blocks32","blocks128","invoke_literal","invoke_delegate","invoke_try_argument","capture1","capture8","quote_closed","quote_capture","runtime_variables"};
names=names.Concat(new[]{"capture_reference","nested_blocks8","nested_blocks32","nested_blocks128"}).ToArray();
if(args.Length>3)names=names.Where(n=>args.Skip(3).Contains(n)).ToArray();
foreach(string name in names) {
try {
var tree=Build(name);var oracle=Compile(tree,true);var d=Compile(tree,system);var expected=Verify(oracle);var actual=Verify(d);
if(expected!=actual)throw new Exception("behavior mismatch "+actual+" expected "+expected);
Console.Error.WriteLine("PASS "+version+" "+flavor+" "+name+" "+actual);
Action<int> build=n=>{for(int i=0;i<n;i++)sink=Build(name);};
Action<int> compile=n=>{for(int i=0;i<n;i++)sink=Compile(tree,system);};
Action<int> first=n=>{for(int i=0;i<n;i++){var c=Compile(tree,system);if(c is Func<int,int> f)checksum=f(3);else if(c is Func<int,Func<int>> ff){var nested=ff(3);sink=nested;checksum=nested();}else if(c is Func<int,IRuntimeVariables> fr)sink=fr(3);else sink=((Func<int,System.Linq.Expressions.Expression<Func<int>>>)c)(3);}};
Measure(version,flavor,name,"construct",round,build);
Measure(version,flavor,name,"compile",round,compile);
Measure(version,flavor,name,"compile_first",round,first);
Measure(version,flavor,name,"invoke",round,Driver(d));
} catch(Exception e) {Console.Error.WriteLine("FAIL "+version+" "+flavor+" "+name+" "+e);}
}
GC.KeepAlive(sink);GC.KeepAlive(checksum);
}
}
from pathlib import Path
import subprocess,json
root=Path(__file__).resolve().parent
repo=Path(r'C:\Data\Git\Github\FastExpressionCompiler')
commits={'upstream':'ff907e1986700a30f05a56e5cee28e952faee363','parent':'e4c232da108d98790f0ca73f23ec90756da0cad7','head':'b8f8b38bc44367f129b49a55b2b2043e81b04ccb'}
for name,commit in commits.items():
for kind in ('standard','light'):
dest=root/name/kind;dest.mkdir(parents=True,exist_ok=True)
paths=['src/FastExpressionCompiler/'+f for f in ['FastExpressionCompiler.cs','ImTools.cs','ILReader.cs','TestTools.cs']]
if kind=='light': paths+=['src/FastExpressionCompiler.LightExpression/'+f for f in ['Expression.cs','ExpressionVisitor.cs','FlatExpression.cs']]
for p in paths:
(dest/Path(p).name).write_bytes(subprocess.check_output(['git','show',commit+':'+p],cwd=repo))
defines='<DefineConstants>LIGHT_EXPRESSION</DefineConstants>' if kind=='light' else ''
(dest/'Library.csproj').write_text('<Project Sdk="Microsoft.NET.Sdk"><PropertyGroup><TargetFrameworks>net8.0;net10.0</TargetFrameworks><LangVersion>latest</LangVersion><AssemblyName>FecSnapshot.'+kind+'</AssemblyName><Optimize>true</Optimize>'+defines+'</PropertyGroup></Project>')
(root/'commits.json').write_text(json.dumps(commits,indent=2))
from pathlib import Path
import subprocess
root=Path(__file__).resolve().parent
for version in ('upstream','parent','head'):
for flavor in ('standard','light'):
dest=root/version/flavor
(dest/'Harness.csproj').write_text('<Project Sdk="Microsoft.NET.Sdk"><PropertyGroup><OutputType>Exe</OutputType><TargetFrameworks>net8.0;net10.0</TargetFrameworks><LangVersion>latest</LangVersion><EnableDefaultCompileItems>false</EnableDefaultCompileItems><Optimize>true</Optimize>'+('<DefineConstants>LIGHT</DefineConstants>' if flavor=='light' else '')+'</PropertyGroup><ItemGroup><Compile Include="../../Program.cs"/><ProjectReference Include="Library.csproj"/></ItemGroup></Project>')
# Exclude harness from library default items (harness source is outside project).
with (dest/'build.log').open('w') as log:
p=subprocess.run(['dotnet','build',str(dest/'Harness.csproj'),'-c','Release','--nologo','-v:q'],stdout=log,stderr=subprocess.STDOUT)
print(version,flavor,p.returncode,flush=True)
if p.returncode: print((dest/'build.log').read_text());raise SystemExit(1)No production fixes, new commits, pushes, or changes to #557/#559 were made as part of this review. The PR remains a draft. Suggested next steps: address the Dynamic reduction failure, remove duplicated Invoke reflection, reduce capture metadata/reflection allocations, and eliminate label-free scope-stack heap growth; then rerun the same paired measurements and correctness controls. Shared-cell semantics should be preserved throughout. |
Follow-up: regression repairs and measured resultsPushed 46 logical source/test repair commits plus the evidence commit, through @dadhi, the allocation concern was valid. The worst common-path allocation regressions are repaired, but before-PR performance parity is still not achieved. What improvedRepresentative Standard/.NET 10 managed B/op from the latest broad run (rounded):
The mechanisms are per-compilation metadata/signature reuse, conservative capture storage selection, lazy quote/runtime-variable metadata, and bounded/compressed scope buffers. There is no cross-compilation user-type cache or outer System compiler fallback. The direct emitter and object[] closure layout are preserved. Quote/runtime-variable exposure and shared mutation retain the required live storage. The final compact scope pool saves 8,104 B in the deep-scope cold causal control; measured warm/reentrant diagnostics show no new scope-buffer misses. Deferring unrelated quote metadata removes the 880 B (.NET 8) / 888 B (.NET 10) common cold overhead in the causal control. Light private array capture is 32 B below pre-PR on both runtimes. The independently reproduced fresh-extension-reduction + Dynamic failure is fixed by preserving collected reductions; regression coverage also includes Quote composition. Framework readonly reference loads now carry the required types in verified anonymous methods. The previous What is still worse
Ratios above one mean slower. Compile time is still about 11.4% / 9.8% worse by this descriptive median. These medians do not promise parity in each scenario; the report lists every timing residual, including runtime phases.
No total native/JIT or retained-memory parity claim is made. Matching finalized managed IL in a capture control also does not prove identical native code or invalidate an observed timing residual. The known deep-tree crash remains deferred as requested. Validation and reproduction
Evidence directory · Decisions and limitations · Exact remaining costs · Validation · Repair commits. The directory contains the interactive report, complete phase CSV/JSON (including System), raw latest/historical archives, reproduction scripts, source/assembly hashes, and causal controls. Historical failures and rejected experiments remain preserved separately. The PR description has been updated with the same results and limitations. This remains a draft with unresolved performance regressions. |
Further allocation/compilation repairs@dadhi, pushed 7 additional logical fixes and their full measurement/validation evidence. Measured source:
The five Framework stages separately reduce constructor reflection (-192 B in the mutable control), reuse typed rented signatures (-64/-120 B), restore owned boxed cells (-320 B), and omit proven redundant delegate/closure casts (-32 B and -120 to -264 B). Runtime allocations do not increase in these causal controls. Compared with pre-PR, mutable nested invocation improves 24 B -> 0 B (factory plus invocation 208 B -> 184 B), while compilation still costs more. The other repairs avoid unused validator JIT and bypass the large collector for eligible binary expressions without a second pass. Full per-commit observations and cold controls are included; no uniformly faster CPU claim is made.
Measured compilation medians remain above pre-PR. These samples do not establish CPU parity; every scenario and phase remains in the full results. Allocation lower/same/higher counts for 41 common Standard shapes: .NET 8 14/27/0; .NET 10 14/27/0. The full table includes every runtime/activation phase, both libraries, Framework and System controls. Small fractional allocation differences in the adaptive runs did not reproduce with equal operation counts: none of 100 targeted current-versus-published phase comparisons allocate more (52 modern, 48 Framework). Empty observer windows allocate 0 bytes; no counter overhead is subtracted. The original results and modern/Framework confirmations are both retained. The allocation table above uses these equal-count controls. Still open: deep mixed128 cold scope allocation cost, small-EH cold initialization costs, protected/typed Framework costs and the timing residuals listed in the report. Shared-VM timings have substantial variation; total native/JIT/retained memory was not measured. Universal pre-PR parity is not established. Rejected experiments are retained, including wrapper cleanup with a consistently higher sampled cold allocation mode. Validation: 35,064 imported passes, 0 failures/crashes among executed cases, existing 69 exclusions unchanged; full native .NET 8/.NET 10 and Framework suites plus all configured production builds pass. Fresh 47-shape matrices cover before/published/current/System on Standard/Light modern runtimes and Framework. Each phase has two opposite-order process rounds and 7 samples; causal microexperiments retain separate raw/reversed/cold observations. Report and complete evidence | Per-commit measurements | Exact remaining costs | Validation | Commits | Decisions. |
| // Framework prohibits using the anonymous module with the module overload. | ||
| return new DynamicMethod(string.Empty, returnType, parameterTypes, restrictedSkipVisibility: true); | ||
| #else | ||
| // Preserve the anonymous module's non-Exception catch policy, without the |
There was a problem hiding this comment.
@jogibear9988 The hell is that hack? :)
Anyway, I am already feeling the boost in my knowledge of .NET internals.
There was a problem hiding this comment.
From where should I know?
I now wanted to let AI finish this so we get nearly the same perf. and know it can work. Then we can use this as a base when you want to do it error by error.
There was a problem hiding this comment.
Yes — this is a runtime-specific workaround, and your objection is fair. Looking more closely at the runtime source, I should remove this version.
The reason for changing the host is observable exception behavior: the host assembly's WrapNonExceptionThrows policy affects matching of raw CLR throws (e.g. a string) and RuntimeWrappedException. FEC's ordinary owner-hosted method and System's anonymous method do not have the same policy. The regression tests exercise raw/wrapped payload identity, catch matching, filters/rethrow, and private-member access; simply returning to the ordinary host for these methods would undo that behavior fix.
The workaround tries to get both properties: reuse the anonymous module for its exception policy, but use the module constructor to avoid the restricted anonymous constructor's eager JIT in CreateDelegate. However, .NET's initialization code explicitly asserts that this anonymous module must not be passed by callers. It works in the tested .NET 8/10 release runtimes, but that is not a supported contract. Framework already rejects this combination, hence the conditional fallback.
I removed the module-reuse workaround in 70f297cb and now use the supported anonymous DynamicMethod(..., restrictedSkipVisibility: true) constructor for exception-aware methods, retaining ordinary owner hosting elsewhere. .NET explicitly compiles that restricted method in CreateDelegate, so the measurements report compile-only, compile + first execution, steady-state calls, and allocations separately. Moving JIT cost into compilation must not be described as an equivalent increase in total first-use cost.
The correction, validation and measurements are pushed, and the description is updated. Complete results and remaining regressions. The PR remains a draft: universal pre-PR performance/allocation parity is not established.
| @@ -645,6 +779,184 @@ public struct CompilerContext | |||
| internal SmallMap4<Expression, NoValue, RefEq<Expression>> ArgsContainingComplexExpression; | |||
|
|
|||
| internal bool HasComplexExpression; | |||
| internal bool HasExceptionBlock; | |||
There was a problem hiding this comment.
combine those and follow up bools into the bit flags
There was a problem hiding this comment.
I tried packing the nine internal booleans into a ushort flag field, keeping the public LastEmitIsAddress field and existing ClosureStatus unchanged. It saves 8 bytes in CompilerContext on every tested target. I also tested a compact single-load setter form: each setter's IL shrinks by 5 bytes, with the same representation and semantics.
The correctness checks pass, but I have held both versions under the requirement not to introduce performance regressions. In the bounded matched tests, the compact version's Framework Light capture8 compilation increased from 6896.625 to 7088.625 B, and compile+first from 7304 to 7496 B: +192 B in each phase. The exact allocation mechanism is unresolved. Some compilation controls also slowed, despite a small improvement in the median across scenarios. The smaller context alone is therefore not enough to claim this change is an improvement. The original sources, both candidates, raw samples, IL evidence and limitations are preserved in the report; the production flags remain unchanged.
| // Frames follow the existing collector traversal; persistent ancestry is allocated | ||
| // only for expressions that actually contain a label or a jump. | ||
| internal SmallList<LabelScopeFrame, Stack4<LabelScopeFrame>, ScopeArrayPool<LabelScopeFrame>> LabelScopes; | ||
| internal SmallList<(LabelTarget Target, int Scope), Stack4<(LabelTarget Target, int Scope)>, NoArrayPool<(LabelTarget Target, int Scope)>> ScopedLabels; |
There was a problem hiding this comment.
Do we need whole int for the Scope?
Maybe for the compactness and usability we can put Target and Scope into struct with strict layout?
(same for the Target Region pair above)
There was a problem hiding this comment.
I checked the actual layout: the existing target+int tuples occupy 16 bytes on all six tested Standard/Light runtime configurations. Named sequential structs keep that reference-alignment footprint, so replacing the tuples alone does not produce the proposed saving.
Scope and Region are ancestry indices with no established ushort bound; the 1024 pool-retention limit does not limit valid input size. I kept full-width IDs and the existing named tuple fields rather than adding an input limit or an unaligned-reference layout. The separate one-word region-frame experiment is reported below, including its costs.
| return; | ||
| } | ||
| } | ||
| PushNewLabelScopeFrame(kind); |
There was a problem hiding this comment.
wut? so here you specifically prevent inlining?
There was a problem hiding this comment.
Yes, deliberately. PushLabelScope is aggressively inlined: repeated adjacent scope kinds update the last frame and return. The separate helper keeps the append/list-growth path outlined, instead of pulling that path into the common wrapper. This is a performance choice, not required for correctness or added just for measuring.
I tested removing only NoInlining against 8f, leaving the automatic JIT policy to decide. All 252 flag/version/shape checks passed, but the bounded matched benchmark did not establish a consistent win: 42 compilation comparisons had a median paired ratio 1.015, with controls ranging 0.921–1.149. For example Light .NET 8 label/goto compilation was 14.9% slower and Light .NET 10 balanced compilation 9.5% slower in that run. These are observations, not proof that this attribute is universally optimal, and unchanged generated-delegate runtime differences should not be attributed to the compiler helper. I retained the outline and added a source comment explaining it in ed431c7. The raw samples and limitations are in the report.
| internal int Region; | ||
| } | ||
|
|
||
| // These scalar scope lists grow their heap part as 4, 9, 19, 39, ... . |
There was a problem hiding this comment.
The logarithm reduces the pool's bookkeeping array, not the size of the rented buffers. These lists grow their heap storage as 4, 9, 19, 39, ... (2*n+1), so each normal growth length has a different floor(log2(length)) bucket. The 1024-element retention limit therefore needs 11 array-reference slots, instead of a table indexed by every possible length (1024 or 1025 slots). This avoids allocating about 8 KB of bookkeeping at the first overflow on 64-bit runtimes.
Different arbitrary lengths can still share a bucket. That can evict a reusable buffer, but cannot give the caller the wrong size: exact rents check equality, minimum rents check the actual length and search larger buckets, and a miss allocates the requested length. Interlocked.Exchange transfers ownership; the 1024 retention bound is unchanged. I clarified this in the source comment in ed431c7; the pool policy is unchanged.
| internal SmallList<LightAndSysExpr> Converted; | ||
| } | ||
| #endif | ||
| private class QuoteVariableScope : System.Linq.Expressions.ExpressionVisitor |
There was a problem hiding this comment.
Those are visitors, each for the specific case, but there is already manual visiting traversal in CollectAndValidate. Can we move this code there?
There was a problem hiding this comment.
Some shared traversal is possible, but these visitors have different stages and semantics. QuoteBinder runs at execution time with the current activation's live capture cells, which CollectAndValidate does not have. Quoted syntax is returned data: executable collection would prune branches, lower Dynamic expressions, collect nested lambdas, or reduce opaque extensions that must be preserved/rejected without callbacks. The existing lexical scope handling is already shared between quote analysis and binding.
ArgumentSpillScope also deliberately sees Try/Goto in constant-pruned branches. That controls whether runtime-expression compilation and Dynamic target reads happen before or after argument effects/jumps; the ordinary collector prunes those branches. Re-running collection would also risk duplicate Dynamic reductions and extension callbacks. I added boundary tests for opaque quoted extensions and pruned Try/Goto argument ordering in 35622f3, comparing the latter with System.
I measured three bounded cleanups rather than merging the traversals blindly: a lazy free-variable list (32/40 B less per captured-quote activation), a separate compile-only quote collector (40/48 B less), and a shared lazy argument scanner (24 B less terminal-argument compilation). They pass the correctness controls, but the quote revisions slowed compile+first-use, and argument timings were mixed. I held these changes to respect the no-regression requirement. The report retains their sources, benefits, costs and limitations; no visitor change is included in this update.
|
@dadhi, the anonymous-module workaround is removed in Four logical commits:
Validation: 35,064 imported passes, zero failures/crashes among executed cases; all production/native Standard/Light builds and suites pass. The existing 69 exclusions, including deferred deep-tree modes, are unchanged. The final matrix compares the last pushed
The decimal-chain first-use cost improves 14–35% versus the last pushed revision across the six configurations. Warm compilation removes 5,832 B on modern .NET / 14,048 B on Framework. However, modern EH compile-only now includes eager JIT: the .NET 10 chain's compile-only time is 57.3x its previously deferred-JIT baseline, while compile + first is 0.703x that baseline. The phase boundary matters. Identical emitted IL alone does not establish identical call performance. The try/catch and real-finally controls have matching canonical instructions/locals/EH but different host flags; .NET 10 Standard try/catch call time changes approximately 2.29 -> 1.87 ns. This does not prove a particular native-entry mechanism. Separate same-compiler A/A and native-code diagnostics are included; they are not used to erase observed costs. Remaining costs, retained in full:
The restore-guard experiment was rejected and empty-finally remains held. The unsupported supplied-ref-operator and preexisting pending-operand/outward-goto families remain documented; no new imported exclusions were added. Earlier pre-PR measurements retain their original source identities rather than being relabeled as final results. |
|
Reviewed all seven comments in review 5577873439 and replied individually with the decisions and measurements. Pushed three commits: The proposed packing/visitor changes remain held. They have real memory benefits, but the measured costs prevent claiming an improvement under the requested performance constraint:
Full validation: 35,064 imported cases pass, zero executed failures or crashes, 69 existing exclusions unchanged. Both production builds and .NET 8/10 native suites pass; Framework passes 1,041 TestX and 1,595 legacy tests. No emitter or production behavior change is included in this follow-up. Report, exact sources, all 13,596 warm observations, 60 cold allocation observations, hashes and validation logs. Compilation, first invocation, activation and nested invocation are separate; these are bounded managed-allocation/time measurements, not a claim of universal CPU parity. Review threads are left for the maintainer to resolve. This PR stays draft; #557 and #559 were not changed. |
187b9b8 to
ae4d9a5
Compare
|
Rebased #558 onto current dadhi/master The conflict resolutions preserve upstream's map-to-set cleanup and the existing compatibility fixes. Both expression membership trackers now use Fresh validation at
Rebase mapping, integration diff, hashes and full validation logs. Earlier performance data retains its original source hashes; no new timing or performance-parity claim is made for this base. Draft status is retained, and #559 was not changed. |
|
Merged current dadhi/master
Validation at Narrow before/after measurements: merged source Timing medians are mixed: converted-shape compilation ranges from 8.9% faster to 2.8% slower. Slower cases are Light net8 addition (+2.8%) and Light net10 selected method (+2.7%). Runtime medians range from about 0.2% faster to 2.5% slower, with the largest absolute increase about 0.036 ns. These small synthetic measurements do not establish universal speedup or parity; cold/first-use, tree construction, Framework/x64 timing and application workloads were not remeasured. Historical reports remain tied to their original revisions. Exact results, all raw samples, hashes, emitted IL, scripts and full validation logs. This remains draft. #559 is unchanged. |
|
Follow-up merge is pushed: Fresh checks pass: 35,064 imported cases, zero executed failures/crashes, 69 unchanged exclusions; both production builds and full .NET 8/10 ARM64 / Framework native suites pass. Exact counters, hashes and full logs. The runtime-value source clarification and review reply are included in |
…sTrue, Quote; opti for Invocation(Lambda(..))
aaa79fe to
dd98968
Compare
jogibear9988
changed the title
Freeze master f49acef, locally merged dadhi#558 at 593c72e2, and optimized dadhi#559 source ef577910. Record 44 workloads, Standard/Light, .NET 8/10, construction/compilation/warm invocation plus a separate compilation-and-first-invocation pass. Retain 34496 main/reference samples, per-commit allocation/timing evidence, actual root and nested IL strings, source/binary hashes, rejected candidates and validation logs. Compile-only medians improve 0.7-2.2% versus dadhi#558 but remain 8.2-9.9% above master. Compilation plus first invocation remains 0.1-1.5% above master on median workload ratios. The apparent nested-exception compile-only outlier shifts JIT work into CreateDelegate for the correctness-required anonymous host; the end-to-end pass accounts for it. Keep slower System comparison cases and unchanged-code allocation controls explicit. Correct the historical IL claim: its serializer compared StringBuilder metadata. The new actual-instruction check matches all 176 root/nested bundles between dadhi#558 and dadhi#559. Production and native suites, Framework 1051 TestX plus 1601 legacy tests, and all 35064 executed imported tests pass. Existing exclusions unchanged; production source unchanged by this evidence commit.
Replace assembly identity trust with exact built-in intrinsic types. Sealed constructor nodes and exact conversion types expose their operands; arbitrary derived user intrinsics remain conservative even when FEC sources are embedded into the user assembly. Reject Extension before structural matching because reduction may reveal additional quotes or runtime-variable storage uses; analysis never executes user reduction. Add a custom reducible Unary Extension hiding a quote alongside the existing custom intrinsic case. Native net10 Std/Light1712 tests and registered regression fixtures pass807ms. Standalone source-embedded net472/net10 probe proves both same-assembly custom node forms retain typed cells. No timing performed.
Add a captured mutable struct instance method call with a try/finally argument. Assert repeated calls mutate the same shared value. The native net10 Std/Light test passes; a source-embedded net472 probe exposes a VerificationException from caching the unbox pointer in a local. Framework emitter adaptation is pending in a separate fix.
Add a third distinct reference capture type to the success/failure/concurrent compilation fixture. Modern integer captures no longer require StrongBox metadata, so int/string/object alone fits the two inline cache entries and stopped exercising overflow storage. Require an overflow array to exist after return, then recursively assert its entries and inline slots retain no metadata references. Fresh native Release net10 Std/Light runner passes1712 legacy tests and registered fixtures, zero warnings/errors.
Keep the original tree and every lexical ancestor while collecting empty-variable block chains in a single pass. Stop at custom Light intrinsics. Native Standard and Light net10 tests pass. Paired compile samples show roughly neutral to four percent improvements on deep block chains, with unchanged allocation and generated runtime behavior; raw trials are retained for the repair report.
Avoid a second virtual NodeType read on recursive collector entry; tail iterations still read their next node and preserve null/error handling. Native Standard and Light net10 tests pass. Collection-only paired samples improve balanced trees by about seven percent on net10 and three percent on net8; integrated compilation and runtime measurements follow in the repair report.
Quote and RuntimeVariables hoists now mark the root as requiring typed cells before any existing-hoist early return, then skip the ordinary mutation scan. Ordinary capture requests still analyze the entire root before choosing boxed storage, so any earlier nested body has already detected quotes, runtime variables or unknown storage uses and retained typed cells. Later ordinary analysis preserves the explicit typed flag. Native net10 Std/Light1712 legacy tests and regression fixtures pass, including independent Quote-only/RV-only/mixed earlier-child cases. Source-embedded net472/net10 probes pass same-assembly custom intrinsic and reducible Extension hidden-quote cases plus receiver/EH behavior. Both builds have zero warnings/errors. No timing or cold-start measurements performed.
Initialize CaptureAnalysisRoot at the public collection boundary before terminal intrinsic callbacks, rather than after the private terminal fast return. An opaque root intrinsic can collect a mutable nested lambda and then a quote through separate public calls; the old path analyzed only the first child, compiled boxed access, then changed the owner's storage to StrongBox and threw InvalidCastException. All validated roots and public custom collection calls funnel through the boundary, while recursive private helpers inherit the initialized root. Remove the repeated nonnull root probe from recursive collection. Add a root Constant intrinsic regression collecting both children separately and checking shared typed storage under default, no-inline and debug flags. Isolated publishedb8 control passes8/10/472; pre-fix09050 reproduces InvalidCastException8/10 and passes472; fixed passes all3 runtimes. Native net10 Std/Light1712 legacy tests and registered regressions pass1135ms, zero warnings/errors. No timing performed.
Use the existing argument-provider helpers when examining NewArray operands, matching other capture-analysis traversals. Light small-array Expressions getters allocate a new array on every access, so the previous Count/index/Count loop allocated three operand arrays per visit for a one-element array. This only changes access to the same operands; the private-reference safety proof and closure layout are unchanged. Prepared source-only while the frozen20d benchmark runs continue. Native correctness and targeted before/after allocation/performance validation remain pending; no measured parity claim is made.
Framework anonymous EH methods verify object[] slot loads as object. Cast readonly nonpassed reference values to their declared type before field/method access or typed returns, both in the normal captured-value emitter and the public reference-slot helper. Include UserProvided contexts because collection-skipping APIs also use anonymous hosting without a collected HasExceptionBlock flag. Owned ordinary methods and modern emitters retain their existing IL. Regular closure constants already cast on Framework; also type the declaring object before a Light public refField constant load on verified contexts. No closure representation, hosting, factory or security policy changes. Cover readonly local/parameter field and method captures across EH, typed/null/interface/array captures, ordinary/precreated constants, the public UserProvided slot helper, and Light field-backed constants under default/no-inline/debug flags. Focused CapturedValueRef fixture passes Std29/Light32 tests on net472/net8/net10, all6 builds clean. Isolated beforee4 returns7, unfixed20d reproduces VerificationException, and fixed returns7 with castclass Box in the same anonymous host. No benchmark timing performed.
Keep the three helper MethodInfo caches in one precisely initialized nested holder. Plain expressions avoid their reflection initialization, while quote and runtime-variable compilation resolves all three together. This preserves the old feature initialization point and does not move merge metadata work to the first quoted-runtime-variable invocation. Existing emitted calls, closure layout, and object.GetType metadata remain unchanged. Allocation-only validation retained all 216 samples: nine fresh processes for each scenario/runtime/version comparison on .NET 8 and .NET 10. Add and balanced compilation medians save 880/888 bytes relative to 0fa003b and match e4c232d. Quote/runtime-variable cold allocation envelopes are unchanged. First-invocation and warm compile/invoke min, median, and max allocations never exceed the current baseline; normal values match. All cold bimodality and occasional baseline noise remain recorded. No timing claim is made.
The nested holder does not inherit the surrounding compiler's RequiresUnreferencedCode context. Apply the same contract to its reflection initializers instead of disabling analysis. Normal native and analyzer-enabled Standard/Light production builds pass.
Replace the 1024-entry exact-length tables used only by lexical and exception scope metadata with a lazy internal 11-bucket pool. Preserve SmallList growth, requested allocation lengths, global sharing, atomic exclusive leases, and the 1024-element retention limit. The used heap sizes 4,9,19,39,79,159,319,639 occupy separate buckets, retaining reentrant size classes without one-largest-buffer eviction. No traversal or validation changes. Allocation-only causal controls versus 0ec7a13 on .NET 8 and 10: cold nested_mixed128 saves 8104 bytes in all three fresh-process repetitions; cold add and balanced256 remain identical. Twelve warmed scenarios retain the same allocation floors. Instrumented scratch-only repeats (5x1000 operations on locals, mixed scopes, capture8 and reentrant nested compilation per runtime) report zero new scope-buffer misses; small 160/168-byte runtime allocation events occur without pool misses. Full Standard and Light native .NET 10 runner: all 1712 tests pass with normal analyzers, including concurrent scope-overflow compilation. No timing claim.
Guard the post-collection validators with their existing goto-list counts. ValidateLexicalJumps only traverses ScopedGotos and ValidateExceptionJumps only traverses ExceptionGotos; when both lists are empty neither has work or side effects. Preserve the existing scope-info cleanup and all validation for trees with jumps. This adds no prescan, cache, visitor or closure layout change. EventPipe JIT controls confirm the two unused validators are no longer compiled on the first ordinary addition. Nine alternating fresh-process controls for five shapes per runtime keep all managed allocation medians identical. Cold compile-plus-first ratios versus 6487c9b are 0.907-0.974 on .NET 8 and 0.964-1.013 on .NET 10; these include host/process variability and are not a universal speedup claim. Warm compilation controls are mostly within a few percent with unchanged allocation medians. Runtime controls and their variability are retained alongside the cold samples, not treated as an emitter change. Full native Standard/Light suites pass: .NET 10 1712, .NET 8 1711, including lexical/exception jump rejection and valid control-flow regressions.
CLR 4 allocates 144 bytes for each closed generic constructor resolved with GetMethodFromHandle. Its ordinary empty constructor lookup allocates nothing, and value constructor lookup needs 96 bytes with the existing rented Type array. Use those APIs for Framework only, and avoid initializing unused open handles. Keep modern constructor handles and all closure representations unchanged. Allocation-only causal probe: per-type constructor lookups 288 -> 96 bytes. Separate-process warmed mutable-capture compilation: 5272 -> 5080 bytes (-192). Activation stays 184 bytes, nested invocation 0, factory/invoke 184. Readonly capture8/reference bytes stay unchanged. Forward/reverse fresh-process timing medians observe mutable compilation ratio 0.927; unchanged-path timing variation is retained and is not attributed to this metadata-only change. Validation: normal Standard and Light production builds pass every configured target with zero warnings/errors. CapturedValueRefRegressionTests passes 29 Standard and 32 Light cases on net472, net8.0 and net10.0. Parsing/compilation, activation, nested runtime and factory/runtime measurements captured separately for Framework baseline/current/candidate and modern current/candidate controls. Scratch reproduction and raw data: work/pr558-repair/frame-capture-round2/.
For Framework nested lambdas with user parameters, declare the hidden argument as ArrayClosureWithNonPassedParams. Existing Curry<C,...> already supports this type, so capture accesses no longer need repeated closure casts. Record the private method's exact signature in a status bit; public manual emitter contexts keep their original cast behavior. Keep zero-user-argument signatures unchanged: a separate cached array caused a measured cold allocation regression and was rejected. No closure fields or runtime allocation layout changes. Add regression coverage for private reference captures under exception handling, zero-argument parent signature reuse, Action<T> currying and shared mutation, debug flags, ordinary parent signature rentals and concurrent compilation. Framework warmed and first-use allocation: Func<T> and EH Func<T> compile -120 B; Action<T> -64 B; constant-folded dead capture unchanged. Runtime activation, nested invocation and factory/invoke allocation unchanged. Modern control bytes unchanged. Parsing/runtime timings are captured independently; mixed compiler ratios and VM variability do not establish a CPU guarantee. No extra first-use signature array is allocated by this conservative version. Validation: normal Standard and Light production builds pass all configured targets without warnings/errors. CapturedValueRefRegressionTests: Standard30, Light33 on net472/net8/net10. The broader typed prototype also passed full Framework1003 TestX and1595 legacy tests before zero-argument paths were narrowed back to the existing implementation. Scratch source/raw measurements and the rejected no-argument cold tradeoff remain under work/pr558-repair/frame-capture-*.
Collect eligible root binary operands directly through the existing terminal collector. This avoids JIT-compiling the large scoped visitor for a simple root, without adding a prepass, changing closure storage, or bypassing jump checks on complex trees. Preserve left-to-right failures, constant identity and public cleanup. Trust ordinary Light nodes as requested; explicit intrinsic roots or operands retain their existing collection callbacks and scopes. Measured Standard add warm compile ratios: 0.961 (.NET 8), 0.927 (.NET 10). Nine fresh-process add compile+first medians: -21% / -34%, identical bytes. Framework add cold compile+first: 156.40 -> 118.90 ms; 27080 -> 27040 B. Trusted Light paired add warm compile ratios: 0.961 / 0.920, 872 B unchanged. All measured warmed allocation medians are unchanged, including invocation. Cold and fallback timing controls varied substantially with host load; retain all raw results rather than claim universal CPU parity or speedups. The rejected Light generic-type-provenance prototype added 160/168 B cold and is not included. Add public collector error-order and bound-constant identity regressions, plus Light intrinsic callback/scope regressions. Full native Standard/Light suites: 1712 passing on .NET 10, 1711 on .NET 8. Separate Framework harness checks agree with System results. Evidence: workspace work/pr558-round2/trivial-binary*, to be published with the integrated round-2 report after final validation.
Enable the existing native boxed value-cell representation for nonnullable Framework captures only in the normal fully collecting root entry. Carry that root eligibility through nested contexts and require completed capture analysis. Any exception region anywhere in the root, quote, runtime variables, or unknown node retains the StrongBox representation throughout the root. Manual and collection-skipping APIs never establish owned-root eligibility, including their privately compiled children. Hosting/security policy and the existing object[] closure layout are unchanged. Add controls for sibling order, outer exception regions, flags/debug mode, shared identity, and manual default/UserProvided collection. Reuse existing struct, byref, enum, nullable, quote, runtime-variable and unknown-node tests. Framework measurements against 7a871c4d, separate one-compiler processes: - Mutable capture compile: 5080 -> 4760 B (-320 B). Pre-PR e4c232d is 4376 B, so the remaining compile allocation residual is explicitly +384 B. - Activation/invoke stay 184 B, nested invocation stays 0 B. - Fresh first compile+invoke: 40272 -> 37888 B (-2384 B). - Capture8/reference/protected capture compile and runtime bytes unchanged. - All 32 modern runtime/scenario/phase allocation controls unchanged. - Descriptive two-order warm medians: mutable compile 43.92 -> 41.92 us, activation 31.80 -> 30.23 ns, nested invocation 6.064 -> 5.993 ns. Three fresh cold medians: compile+first 246.84 -> 221.64 ms. Unchanged controls show substantial host timing variation; these timing observations do not establish universal CPU parity or causal speedups. Validation: normal Standard/Light production builds all configured targets, zero warnings/errors; focused Standard/Light fixture 32/35 on Framework, 31/34 on net8 and net10, all passing; full Framework en-US runner passes 1007 TestX and 1595 legacy tests. Initial German-culture run had only existing English-message assertion failures. Independent source review found no actionable correctness defect. Raw evidence: work/pr558-repair/frame-owned-boxed.
Restore FEC's original nested-delegate slot load only when the normal root has completed capture analysis and proves owned hosting throughout without Try, Quote, runtime variables or unknown nodes. Manual/UserProvided and anonymous exception-hosted methods retain the exact verifier cast. Hosting, closure layout and delegate/Curry construction are unchanged. Against 25955bb5, four-shape one-compiler Framework probes show -32 B compile for mutable, capture8 and reference captures, warm and cold; protected capture_try and all runtime allocations are unchanged. Mutable compile is 4728 B versus pre-PR 4376 B, leaving +352 B explicitly unresolved. Two opposite process-order warm rounds and three fresh cold reps measure mutable compile 40.81 -> 36.21 us, activation 26.02 -> 24.88 ns, nested invocation 5.467 -> 5.483 ns, compile+first 185.55 -> 183.44 ms. Host timing variation in unchanged controls prevents universal CPU claims. Normal Standard/Light builds all targets: zero warnings/errors. Focused fixtures pass on Framework/net8/net10 (32/31/31 Standard, 35/34/34 Light). Full Framework en-US runner: 1007 TestX +1595 legacy passing. Independent source audit confirms the conservative owned-host guard. Evidence: work/pr558-repair/frame-owned-delegate/measurements.json and raw logs.
Omit the hidden closure cast only under the same completed ordinary owned-root proof as the nested delegate cast. Typed signatures keep their existing fast path; all anonymous/manual/UserProvided/EH/Quote/RV/unknown cases keep exact typing. No signature arrays, metadata caches, hosting or layout changes. Against 3320b13e, Framework warm/cold compilation saves 208 B for mutable, 120 B for readonly reference and 264 B for capture8; protected capture_try and runtime allocations are unchanged. Mutable compile is now 4520 B versus pre-PR 4376 (+144 B); reference is 4256 versus 4264 (-8 B). Two opposite process-order warm rounds and three fresh cold repetitions: mutable compile 42.83 -> 38.82 us, nested invocation 5.730 -> 4.963 ns, compile+first 239.07 -> 204.11 ms. Activation 25.67 -> 28.55 ns despite its unchanged emitted code illustrates host noise; no universal CPU claim. Normal Standard/Light builds all targets pass with zero warnings/errors. Focused fixtures pass on Framework/net8/net10 (32/31/31 Standard, 35/34/34 Light), including protected siblings, manual contexts and shared mutation. Combined final full suite is scheduled by the parent task. Evidence: work/pr558-repair/frame-owned-closure/measurements.json and raw logs.
Reuse the existing atomic-operand predicate for non-lifted operands. Keep complex right operands, nullable lifting, custom Light intrinsics and operator evaluation order on their existing paths. No new metadata query, cache, analysis pass or closure layout is introduced. For 64 decimal additions under real finally blocks, isolated compilation allocations fall from 11496 to 5664 B on modern .NET and 21136 to 7088 B on Framework. Warm compile improves 16-30%, compile plus first execution 12-44%, and invocation 7-19% across Standard/Light on .NET 8/10/Framework. Native fixture checks pass in all six configurations. Cold controls and material timing outliers remain in the evidence; universal parity is not established. Full combined validation follows this logical commit.
Emit non-byref try body and catch results with the existing value-parent mask before storing the typed result temporary. Receiver, lvalue and duplication flags must not cause a managed address to be stored as a struct value. Preserve enclosing EH/inlining context and genuine byref Try results; mark a loaded ordinary result as a value. Fixes the introduced Light new-struct receiver regression and existing variable/field/ref-parameter receiver forms, preserving mutation-copy semantics. Register seven native regression methods for Standard/Light on modern .NET and Framework. All six focused configurations pass40cases; modern byref-return aliases remain correct. No hosting, metadata cache, closure layout or empty-finally elimination is changed. Performance controls retain unchanged runtime/activation allocations. A genuine Framework Light compilation-history allocation residual of 192B in two capture controls remains documented; isolated controls match. This correctness repair does not claim universal performance parity. Full combined native/imported validation follows this logical commit.
Retain collision and stable-reference tests against the master implementation.
…ware methods Address dadhi review discussion_r4318926508: passing the runtime anonymous module into the module overload violates an explicit CoreCLR assertion even though tested release runtimes accept it. Remove that module-reuse workaround and its lazy holder. Use the public restricted anonymous constructor on every target; ordinary owner-hosted methods are unchanged. Keep raw/wrapped non-Exception catch matching and private-member access semantics. The supported constructor eagerly JITs at CreateDelegate, so compile-only timings must be reported separately from compile plus first execution. NonExceptionThrowPolicy, ExceptionRegionValidation and TryResultValue fixtures pass Standard/Light on net8.0, net10.0 and net472.
dd98968 to
02a794a
Compare
This adds the remaining expression compiler compatibility fixes and regression coverage from the imported dotnet/runtime expression tests. Compilation stays in FEC's existing emitter and closure layout.
Current base and history
Rebased on fetched
dadhi/masterca552c9f9b8ac95dbc3e6b58534347aa979ad51e(27 October 2026). The branch is linear, with 188 commits and no merge commits above master.Master now implements several changes originally carried here. Their duplicate implementations have been removed, while retaining the additional lifted arithmetic, unary operator, and TypeEqual regression tests. Master's unary emitters, selected-operator fast paths, TypeIs/TypeEqual implementation, and LightExpression lifting metadata are preserved. The now-redundant reference-comparison refactor commit is folded into its original fix. Rebase integration corrections are included in their owning fixes rather than separate merge-fix commits.
The rebase also adapts address/value handling to master's unary helpers, keeps explicit comparison methods on the common arithmetic path, and corrects checked floating-point arithmetic to use floating-point opcodes. Mixed nullable shift metadata is checked against System's metadata, alongside value, null, and evaluation-order assertions.
Remaining changes
The native regressions cover both System expressions and LightExpression. The imported runtime suite exercises strict Standard FEC compilation with fallback disabled.
Validation on this rebase
Release builds on Windows ARM64, SDK 10.0.301:
The Framework runner is invoked with en-US culture because existing Issue261 assertions expect English exception-message text. Production code and those assertions are unchanged for this workaround.
Fresh logs, source hashes, the commit mapping, and cleanup details are in the 27 October rebase report.
Limits
The 59 documented exclusions remain, including deferred deep-tree stress and tests tied to System's exact IL/layout or interpreter-specific behavior. Behavior proofs cover the alternative supported IL paths; no new skips were added for this rebase. Unsupported nested exception handling in filters remains rejected, as by System's IL compiler. NativeAOT and trimming publication have not been validated.
This rebase does not make a new timing or allocation claim. Earlier benchmark records remain historical evidence and are not measurements of this head. Performance changes in stacked PR #559 are outside this PR; #559 was not rewritten by this rebase.